SlideShare a Scribd company logo
Scaling Islandora
❏
❏
❏
❏
❏
Scaling Islandora
Scaling Islandora
Scaling Islandora
Scaling Islandora
● Inactive project
● Performance suffers with
triplestore managing 50
million or more triples
● No ability to cluster or
balance load
● Incomplete implementation
of Sparql
Scaling Islandora
Scaling Islandora
Scaling Islandora
● Isolate derivative generation
to 20 server cluster
● Asynchronous and parallel
derivative creation with
Islandora Jobs (Gearman)
● Using Parallels instead of
Drush scripts
Asynchronous and Parallel Derivative Creation
Scaling Islandora
● Benchmark Testing
● Block Obscure Crawlers
● Manage Caching
● Ensure Sufficient Server
Resources
Scaling Islandora
Scaling Islandora
Scaling Islandora
Scaling Islandora
Room to improve scaling and
performance through exploration:
● Media Streaming
● Load Balancing
● More Aggressive Caching
Configuration
● Clustering Solr
Scaling Islandora
Developing a scalable repository
back end to satisfy growing
performance requirements
Will accommodate:
● More concurrent users
● Many millions of objects
● Faster response times
May be used as a Fedora
replacement
Continued work on Fedora 4
integration
Scaling Islandora
Scaling Islandora
Scaling Islandora

More Related Content

PDF
Update on Crimson - the Seastarized Ceph - Seastar Summit
ScyllaDB
 
PPTX
KDB+ Lite
Sayanosauras
 
PDF
Monitoring with Clickhouse
unicast
 
PDF
KDB database (EPAM tech talks, Sofia, April, 2015)
Martin Toshev
 
PDF
Gluster as Block Store in Containers
Gluster.org
 
PPTX
DMS (Database Migration Service) - Mydbops Team
Mydbops
 
PPTX
CrateDB - Giacomo Ceribelli
ceribbo
 
PDF
Running OpenStack in Production - Barcamp Saigon 2016
Thang Man
 
Update on Crimson - the Seastarized Ceph - Seastar Summit
ScyllaDB
 
KDB+ Lite
Sayanosauras
 
Monitoring with Clickhouse
unicast
 
KDB database (EPAM tech talks, Sofia, April, 2015)
Martin Toshev
 
Gluster as Block Store in Containers
Gluster.org
 
DMS (Database Migration Service) - Mydbops Team
Mydbops
 
CrateDB - Giacomo Ceribelli
ceribbo
 
Running OpenStack in Production - Barcamp Saigon 2016
Thang Man
 

What's hot (19)

PDF
Using Ceph in OStack.de - Ceph Day Frankfurt
Ceph Community
 
PDF
Scaling Up with PHP and AWS
Heath Dutton ☕
 
PDF
Kubernetes
Diego Pacheco
 
PDF
State of the_gluster_-_lceu
Gluster.org
 
PDF
Optimising for Performance
thomas_mb
 
PDF
Web Performance Part 3 "Server-side tips"
Binary Studio
 
ODP
Join the super_colony_-_feb2013
Gluster.org
 
ODP
Sdc challenges-2012
Gluster.org
 
ODP
Tiering barcelona
Gluster.org
 
PDF
Frontend Track NodeJS
Marcelo Serpa
 
PDF
Arbiter volumes in gluster
itisravi
 
PDF
Cncf meetup-rook
Juraj Hantak
 
PPTX
Performance Web
Rodolfo Fadino Junior
 
PDF
Cncf meetup-rook
Juraj Hantak
 
PPTX
Replication and rebuild in cStor
OpenEBS
 
PDF
Ndb cluster 80_use_cases
mikaelronstrom
 
ODP
Dedupe nmamit
Gluster.org
 
PDF
Cassandra
Diego Pacheco
 
PDF
Disaster Recovery and Ceph Block Storage: Introducing Multi-Site Mirroring
Jason Dillaman
 
Using Ceph in OStack.de - Ceph Day Frankfurt
Ceph Community
 
Scaling Up with PHP and AWS
Heath Dutton ☕
 
Kubernetes
Diego Pacheco
 
State of the_gluster_-_lceu
Gluster.org
 
Optimising for Performance
thomas_mb
 
Web Performance Part 3 "Server-side tips"
Binary Studio
 
Join the super_colony_-_feb2013
Gluster.org
 
Sdc challenges-2012
Gluster.org
 
Tiering barcelona
Gluster.org
 
Frontend Track NodeJS
Marcelo Serpa
 
Arbiter volumes in gluster
itisravi
 
Cncf meetup-rook
Juraj Hantak
 
Performance Web
Rodolfo Fadino Junior
 
Cncf meetup-rook
Juraj Hantak
 
Replication and rebuild in cStor
OpenEBS
 
Ndb cluster 80_use_cases
mikaelronstrom
 
Dedupe nmamit
Gluster.org
 
Cassandra
Diego Pacheco
 
Disaster Recovery and Ceph Block Storage: Introducing Multi-Site Mirroring
Jason Dillaman
 
Ad

Viewers also liked (20)

PDF
Introduction to A-Frame
Daosheng Mu
 
PPTX
UCSD / DBMI seminar 2015-02-6
Andrew Su
 
PDF
ePADD and Access -- Society of American Archivists (SAA) Annual Meeting, 2015
Josh Schneider
 
PPT
Social Media and the Archive. Anthony Browne. BBC Scotland - FIAT/IFTA MMC Se...
FIAT/IFTA
 
PPTX
NSW Open Data Challenge: Data Request Service
Cofluence
 
PPTX
The Danish Open Access Indicator
Mikael Elbæk
 
PPTX
Imperial College London - journey to open scholarship
Torsten Reimer
 
PDF
Shifting Scientific Practice - ORCID 2015
Kaitlin Thaney
 
PDF
FIBO & Schema.org
Richard Wallis
 
PPTX
Figshare for institutions presentation swets customer day 2014
Swetsbelgie
 
PDF
Lecture 1: Human-Computer Interaction Course (2015) @VU University Amsterdam
Lora Aroyo
 
PPT
Dsp bbc-jem rayfield-semtech2011
Jem Rayfield
 
PPTX
The DiNAR Project: Meaningful Mixed Reality for Heritage - Gareth Beale
Museums Computer Group
 
PDF
Web Archiving: A Brief Introduction
Sawood Alam
 
PDF
The FP7 Post-Grant Open Access Pilot: An All-Encompassing Gold Open Access Fu...
OpenAIRE
 
PPTX
2016 07-kdl-interr-infra
Paul Spence
 
PDF
[3.8] Archiving and Publishing in Practice Event Logs - Joos Buijs [3TU.Datac...
3TU.Datacentrum
 
PDF
Knowledge Patterns SSSW2016
Aldo Gangemi
 
PDF
Securing the future of OA policies - Rob Johnson
Rob Johnson
 
PPTX
Pedagogy in Public: Open Education Unbound
Robin DeRosa
 
Introduction to A-Frame
Daosheng Mu
 
UCSD / DBMI seminar 2015-02-6
Andrew Su
 
ePADD and Access -- Society of American Archivists (SAA) Annual Meeting, 2015
Josh Schneider
 
Social Media and the Archive. Anthony Browne. BBC Scotland - FIAT/IFTA MMC Se...
FIAT/IFTA
 
NSW Open Data Challenge: Data Request Service
Cofluence
 
The Danish Open Access Indicator
Mikael Elbæk
 
Imperial College London - journey to open scholarship
Torsten Reimer
 
Shifting Scientific Practice - ORCID 2015
Kaitlin Thaney
 
FIBO & Schema.org
Richard Wallis
 
Figshare for institutions presentation swets customer day 2014
Swetsbelgie
 
Lecture 1: Human-Computer Interaction Course (2015) @VU University Amsterdam
Lora Aroyo
 
Dsp bbc-jem rayfield-semtech2011
Jem Rayfield
 
The DiNAR Project: Meaningful Mixed Reality for Heritage - Gareth Beale
Museums Computer Group
 
Web Archiving: A Brief Introduction
Sawood Alam
 
The FP7 Post-Grant Open Access Pilot: An All-Encompassing Gold Open Access Fu...
OpenAIRE
 
2016 07-kdl-interr-infra
Paul Spence
 
[3.8] Archiving and Publishing in Practice Event Logs - Joos Buijs [3TU.Datac...
3TU.Datacentrum
 
Knowledge Patterns SSSW2016
Aldo Gangemi
 
Securing the future of OA policies - Rob Johnson
Rob Johnson
 
Pedagogy in Public: Open Education Unbound
Robin DeRosa
 
Ad

Similar to Scaling Islandora (20)

PPTX
MyHeritage backend group - build to scale
Ran Levy
 
PDF
FFFast Drupal backend No kidding!
i20 Group
 
PDF
Pavel Prischepa. Fffast Drupal backend.
DrupalSib
 
PDF
NoSQL, Apache SOLR and Apache Hadoop
Dmitry Kan
 
PDF
100 Sites. A Mixed Drupal Platform. One Federated Search Solution.
Acquia
 
PPTX
ElasticSearch as (only) datastore
Tomas Sirny
 
PDF
BP-8 Global Federation and Search
Alfresco Software
 
PPT
Integrating Drupal & Fedora
Don Gourley
 
PPTX
Black Friday and Cyber Monday- Best Practices for Your E-Commerce Database
Tim Vaillancourt
 
PDF
Cassandra Summit 2015 - Building a multi-tenant API PaaS with DataStax Enterp...
Restlet
 
PDF
Solr @ Etsy - Apache Lucene Eurocon
Giovanni Fernandez-Kincade
 
PPTX
Вадим Абрамчук — Big Drupal: Issues We Met
LEDC 2016
 
PPTX
Architecture at PBS
Public Broadcasting Service
 
PDF
Drupal for Higher Education and Virtual Learning
Gabriel Dragomir
 
PDF
Sprint 111
ManageIQ
 
PDF
Full stack development
Pavlo Iuriichuk
 
ZIP
Introduction to Google App Engine
mdc11
 
PPTX
Built for Integration: Why Drupal 8 Should Be the Glue to Your Stack
Acquia
 
PDF
CouchConf SF 2012 Lightning Talk - Operational Excellence
Laine Campbell
 
PDF
A Day in the Life of a Silicon Valley Startup
Sachin Rekhi
 
MyHeritage backend group - build to scale
Ran Levy
 
FFFast Drupal backend No kidding!
i20 Group
 
Pavel Prischepa. Fffast Drupal backend.
DrupalSib
 
NoSQL, Apache SOLR and Apache Hadoop
Dmitry Kan
 
100 Sites. A Mixed Drupal Platform. One Federated Search Solution.
Acquia
 
ElasticSearch as (only) datastore
Tomas Sirny
 
BP-8 Global Federation and Search
Alfresco Software
 
Integrating Drupal & Fedora
Don Gourley
 
Black Friday and Cyber Monday- Best Practices for Your E-Commerce Database
Tim Vaillancourt
 
Cassandra Summit 2015 - Building a multi-tenant API PaaS with DataStax Enterp...
Restlet
 
Solr @ Etsy - Apache Lucene Eurocon
Giovanni Fernandez-Kincade
 
Вадим Абрамчук — Big Drupal: Issues We Met
LEDC 2016
 
Architecture at PBS
Public Broadcasting Service
 
Drupal for Higher Education and Virtual Learning
Gabriel Dragomir
 
Sprint 111
ManageIQ
 
Full stack development
Pavlo Iuriichuk
 
Introduction to Google App Engine
mdc11
 
Built for Integration: Why Drupal 8 Should Be the Glue to Your Stack
Acquia
 
CouchConf SF 2012 Lightning Talk - Operational Excellence
Laine Campbell
 
A Day in the Life of a Silicon Valley Startup
Sachin Rekhi
 

Recently uploaded (20)

PPTX
classification of computer and basic part of digital computer
ravisinghrajpurohit3
 
PDF
49784907924775488180_LRN2959_Data_Pump_23ai.pdf
Abilash868456
 
PPTX
Role Of Python In Programing Language.pptx
jaykoshti048
 
PDF
MiniTool Power Data Recovery Crack New Pre Activated Version Latest 2025
imang66g
 
PDF
Protecting the Digital World Cyber Securit
dnthakkar16
 
PDF
Applitools Platform Pulse: What's New and What's Coming - July 2025
Applitools
 
PPT
Activate_Methodology_Summary presentatio
annapureddyn
 
PDF
WatchTraderHub - Watch Dealer software with inventory management and multi-ch...
WatchDealer Pavel
 
PDF
Bandai Playdia The Book - David Glotz
BluePanther6
 
PPTX
slidesgo-unlocking-the-code-the-dynamic-dance-of-variables-and-constants-2024...
kr2589474
 
PDF
advancepresentationskillshdhdhhdhdhdhhfhf
jasmenrojas249
 
PPTX
Maximizing Revenue with Marketo Measure: A Deep Dive into Multi-Touch Attribu...
bbedford2
 
PPTX
Explanation about Structures in C language.pptx
Veeral Rathod
 
PDF
On Software Engineers' Productivity - Beyond Misleading Metrics
Romén Rodríguez-Gil
 
PDF
New Download MiniTool Partition Wizard Crack Latest Version 2025
imang66g
 
PPTX
GALILEO CRS SYSTEM | GALILEO TRAVEL SOFTWARE
philipnathen82
 
PPTX
TRAVEL APIs | WHITE LABEL TRAVEL API | TOP TRAVEL APIs
philipnathen82
 
PPTX
AI-Ready Handoff: Auto-Summaries & Draft Emails from MQL to Slack in One Flow
bbedford2
 
PPTX
Visualising Data with Scatterplots in IBM SPSS Statistics.pptx
Version 1 Analytics
 
PPTX
ASSIGNMENT_1[1][1][1][1][1] (1) variables.pptx
kr2589474
 
classification of computer and basic part of digital computer
ravisinghrajpurohit3
 
49784907924775488180_LRN2959_Data_Pump_23ai.pdf
Abilash868456
 
Role Of Python In Programing Language.pptx
jaykoshti048
 
MiniTool Power Data Recovery Crack New Pre Activated Version Latest 2025
imang66g
 
Protecting the Digital World Cyber Securit
dnthakkar16
 
Applitools Platform Pulse: What's New and What's Coming - July 2025
Applitools
 
Activate_Methodology_Summary presentatio
annapureddyn
 
WatchTraderHub - Watch Dealer software with inventory management and multi-ch...
WatchDealer Pavel
 
Bandai Playdia The Book - David Glotz
BluePanther6
 
slidesgo-unlocking-the-code-the-dynamic-dance-of-variables-and-constants-2024...
kr2589474
 
advancepresentationskillshdhdhhdhdhdhhfhf
jasmenrojas249
 
Maximizing Revenue with Marketo Measure: A Deep Dive into Multi-Touch Attribu...
bbedford2
 
Explanation about Structures in C language.pptx
Veeral Rathod
 
On Software Engineers' Productivity - Beyond Misleading Metrics
Romén Rodríguez-Gil
 
New Download MiniTool Partition Wizard Crack Latest Version 2025
imang66g
 
GALILEO CRS SYSTEM | GALILEO TRAVEL SOFTWARE
philipnathen82
 
TRAVEL APIs | WHITE LABEL TRAVEL API | TOP TRAVEL APIs
philipnathen82
 
AI-Ready Handoff: Auto-Summaries & Draft Emails from MQL to Slack in One Flow
bbedford2
 
Visualising Data with Scatterplots in IBM SPSS Statistics.pptx
Version 1 Analytics
 
ASSIGNMENT_1[1][1][1][1][1] (1) variables.pptx
kr2589474
 

Scaling Islandora

  • 7. ● Inactive project ● Performance suffers with triplestore managing 50 million or more triples ● No ability to cluster or balance load ● Incomplete implementation of Sparql
  • 11. ● Isolate derivative generation to 20 server cluster ● Asynchronous and parallel derivative creation with Islandora Jobs (Gearman) ● Using Parallels instead of Drush scripts
  • 12. Asynchronous and Parallel Derivative Creation
  • 14. ● Benchmark Testing ● Block Obscure Crawlers ● Manage Caching ● Ensure Sufficient Server Resources
  • 19. Room to improve scaling and performance through exploration: ● Media Streaming ● Load Balancing ● More Aggressive Caching Configuration ● Clustering Solr
  • 21. Developing a scalable repository back end to satisfy growing performance requirements Will accommodate: ● More concurrent users ● Many millions of objects ● Faster response times May be used as a Fedora replacement Continued work on Fedora 4 integration