«NoSQL Databases and Polyglot Persistence»
Download as PPTX, PDF4 likes2,334 views
This document discusses polyglot persistence, which is using multiple data storage technologies chosen based on how applications use data. It covers NoSQL databases including key-value stores, column-family stores, document databases and graph databases. Common polyglot persistence patterns are described like using a relational database for transactions, a key-value store for sessions, and a document database for product catalogs. The challenges of a science warehouse project are presented, and using a polyglot approach with NoSQL databases for logging and price history is proposed as a solution.
«NoSQL Databases and Polyglot Persistence»
- 1. © ALTOROS Systems | CONFIDENTIAL NoSQL Databases and Polyglot Persistence Sergey Kovalev, Senior Java Developer, [email protected]
- 2. © ALTOROS Systems | CONFIDENTIAL 2 The Challenge “One Size Does Not Fit All”
- 3. © ALTOROS Systems | CONFIDENTIAL 3 • What is Polyglot Persistence? • RDBMS vs NoSQL • NoSQL DB types • Polyglot Persistence Patterns • Real project experience Agenda:
- 4. © ALTOROS Systems | CONFIDENTIAL Polyglot Persistence - 4 Using multiple data storage technologies, chosen based upon the way data is being used by individual applications or components of single application. Martin Fowler http://martinfowler.com/articles/nosql-intro.pdf What is Polyglot Persistence?
- 5. © ALTOROS Systems | CONFIDENTIAL 5 NoSQL Zooo What is Polyglot Persistence? http://nosql-databases.org 150+ NoSQL databases
- 6. © ALTOROS Systems | CONFIDENTIAL 6 What is Polyglot Persistence? Who use polyglot persistence?
- 7. © ALTOROS Systems | CONFIDENTIAL 7 • What is Polyglot Persistence? • RDBMS vs NoSQL • NoSQL DB types • Polyglot Persistence Patterns • Real project experience Agenda:
- 8. © ALTOROS Systems | CONFIDENTIAL RDMS are great 8 • Widely used and understood • Tested in real environments • Efficient use of storage space if data normalized properly • Great tools support ACID semantics • Incredibly flexible and powerful query language • Great framework support RDBMS vs NoSQL
- 9. © ALTOROS Systems | CONFIDENTIAL RDBMS have limitations 9 • Complex object graphs does not map very well with flat tables. • Strict Schema • Difficult to evolve Schema with time. • Data constraints and JOINs can be expensive at runtime. • Difficult to scale horizontally. RDBMS vs NoSQL
- 10. © ALTOROS Systems | CONFIDENTIAL NoSQL features 10 • Flexible schema • Easy to scale horizontally to add processing power and storage • High Performance RDBMS vs NoSQL
- 11. © ALTOROS Systems | CONFIDENTIAL 11 • What is Polyglot Persistence? • RDBMS vs NoSQL • NoSQL DB types • Polyglot Persistence Patterns • Real project experience Agenda:
- 12. © ALTOROS Systems | CONFIDENTIAL Key Value Stores 12 • A key-value store is a simple hash table • Primarily used when all access to the database is via primary key • Value is a blob with the data store not caring or knowing what is inside NoSQL DB types
- 13. © ALTOROS Systems | CONFIDENTIAL Column-Family Stores 13 • Store data in column families as rows that have many columns associated with a row key • Column families are groups of related data, often accessed together • Aggregate-Oriented NoSQL DB types
- 14. © ALTOROS Systems | CONFIDENTIAL 14 NoSQL DB types Document Databases • Stores and retrieves documents, which can be XML, JSON, BSON, … • Documents are self-describing, hierarchical tree data structures which can consist of maps, collections and scalar values • Aggregate-Oriented
- 15. © ALTOROS Systems | CONFIDENTIAL 15 NoSQL DB types Graph Databases • Allow to store entities and relationships between these entities • Entities are known as nodes, which have properties • Relations are known as edges, which also have properties
- 16. © ALTOROS Systems | CONFIDENTIAL 16 • What is Polyglot Persistence? • RDBMS vs NoSQL • NoSQL DB types • Polyglot Persistence Patterns • Real project experience Agenda:
- 17. © ALTOROS Systems | CONFIDENTIAL RDBMS 17 • Transactional data (payment and account) in a ACID compliant traditional RDBMS stores such as MySQL or Oracle. • Reporting Tools Polyglot Persistence Patterns RDBMS Web application Reporting Tools Transactional data
- 18. © ALTOROS Systems | CONFIDENTIAL Key Value Stores 18 • Session data in Redis or Hazelcast. Session data requires faster read and write but not durability. • Shopping cart data in a high availability and fault-tolerance data store such as Riak. Polyglot Persistence Patterns RDBMS Web application Reporting Tools Transactional data Key-Value store Sessions and Shopping cart
- 19. © ALTOROS Systems | CONFIDENTIAL Column-Family Stores 19 • Log level data (audit and activity) in a very high write throughput data store such as Cassandra. • Also good for analytic and real-time data mining such as product ranking Polyglot Persistence Patterns RDBMS Web application Reporting Tools Transactional data Key-Value store Sessions and Shopping cart Column-Family store Logging Analytic
- 20. © ALTOROS Systems | CONFIDENTIAL Document Databases 20 • Product catalogue in a document oriented data store such as MongoDB. Requirement include high read throughput, frequent data change (stock level information) • Customer profile data including purchase history, shipping & billing address Polyglot Persistence Patterns Web application Reporting Tools Transactional data Key-Value store Sessions and Shopping cart Column-Family store Logging Analytic Document store RDBMS Product catalogue
- 21. © ALTOROS Systems | CONFIDENTIAL Graph Databases 21 • Data for product recommendations, related products and similar products in a graph database such as Neo4j. Polyglot Persistence Patterns Web application Reporting Tools Transactional data Key-Value store Sessions and Shopping cart Column-Family store Logging Analytic Document store RDBMS Product catalogue Graph Databases Social graph
- 22. © ALTOROS Systems | CONFIDENTIAL Summary 22 Polyglot Persistence Patterns
- 23. © ALTOROS Systems | CONFIDENTIAL 23 • What is Polyglot Persistence? • RDBMS vs NoSQL • NoSQL DB types • Polyglot Persistence Patterns • Real project experience Agenda:
- 24. © ALTOROS Systems | CONFIDENTIAL ScienceWarehouse challenges 24 • Hundreds of thousands of records are added to the database daily • Visitors’ activities logging generates huge amount of write queries • Price table contains approximately 100,000,000 records • Price history table contains approximately 500.000.000 • Data rapidly grows Real project experience
- 25. © ALTOROS Systems | CONFIDENTIAL 25 • Upgrade hardware (vertical scaling) • Upgrade to the Oracle Database Enterprise Edition Options for ScienceWarehause • Use NoSQL and Polyglot Persistence or Real project experience
- 26. © ALTOROS Systems | CONFIDENTIAL 26 Real project experience Solutions for ScienceWarehouse • Add a caching layer • Write the log data on visitors’ activities to a NoSQL database (Cassandra or HBase) • Move price history(500,000,000 records table) information to NoSQL data base(Cassandra or HBase)
- 27. © ALTOROS Systems | CONFIDENTIAL 27 Real project experience ScienceWarehouse High Level Architecture Web application Legacy DB Key-Value store Sessions and Shopping cart Column-Family store Logging Analytic RDBMS
- 28. © ALTOROS Systems | CONFIDENTIAL 28 Sergey Kovalev, Senior Java Developer, [email protected] NoSQL Databases and Polyglot Persistence