Michael Hackstein - Polyglot Persistence & Multi-Model NoSQL Databases - NoSQL matters Paris 2015
3 likes3,195 views
The document discusses the concept of polyglot persistence and multi-model databases, highlighting the shift from single-model databases to more flexible systems that can store multiple data formats like key-value pairs, documents, and graphs. It emphasizes the benefits of this approach, such as tailored queries, optimized data storage, and a focus on business logic while managing redundancy and synchronization challenges. Key features of multi-model databases, specifically ArangoDB, include AQL for complex queries, ACID transactions, and the ability to extend functionality with Foxx microservices.
![ACID - Transactions
‣ Invoke a transaction:
db._executeTransaction({
collections: {
write: ["users", "products"],
read: "recommendations"
},
action: function() {
// all operations go here
}
});
14
throw "failure"; // Triggers rollback](https://image.slidesharecdn.com/multi-model-databases-150327075822-conversion-gate01/85/Michael-Hackstein-Polyglot-Persistence-Multi-Model-NoSQL-Databases-NoSQL-matters-Paris-2015-14-320.jpg)
Michael Hackstein - Polyglot Persistence & Multi-Model NoSQL Databases - NoSQL matters Paris 2015
- 1. www.arangodb.com Polyglot Persistence & Multi-Model Databases Michael Hackstein @mchacki NoSQL Matters Paris
- 2. Michael Hackstein ‣ ArangoDB Core Team ‣ Web Frontend ‣ Graph visualisation ‣ Graph features ‣ Host of cologne.js ‣ Master’s Degree (spec. Databases and Information Systems) 2
- 3. The Single Model era is over 3 Relational World
- 4. The Multi Model era begins 4 NoSQL World Documents - JSON K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V K => V Graphs Key Value { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } ‣ Map value data to unique string keys (identifiers) ‣ Treat data as opaque (data has no schema) ‣ Can implement scaling and partitioning easily ‣ Focussed on m-to-n relations between entities ‣ Stores property graphs: entities and edges can have attributes ‣ Easily query paths of variable length ‣ Normally based on key-value stores (each document still has a unique key) ‣ Allow to save documents with logical similarity in “collections” ‣ Treat data records as attribute-structured documents (data is no more opaque) ‣ Often allow querying and indexing document attributes
- 5. An e-commerce system in Relational World 5 Sales-History Shopping-Cart Recommendations Customer Product-Catalog
- 6. Polyglot Persistence 6 Source: Martin Fowler, http://martinfowler.com/articles/nosql-intro.pdf Reporting RDBMS Product Catalog MongoDB Shopping Cart Riak User activity log Cassandra Analytics Cassandra Recommendations Neo4J Financial Data RDBMS User Sessions Redis Shopping Cart KeyValue Reporting RDBMS Product Catalog Document User activity log Column Analytics Column Recommendations Graph Financial Data RDBMS User Sessions KeyValue
- 7. Single Model Databases 7 Recommendations Product-CatalogShopping-Cart Sales-History Customer { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” } { “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, } 423453453 4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => 874365563 5463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } DocumentStore GraphStore DocumentStore DocumentStoreKeyValueStore
- 8. Benefits ‣ Natural mapping of data into DB ‣ DB optimized for the data format ‣ Queries are tailored for your data format ‣ Focus on writing business logic 8 ‣ Data has to be stored redundantly and has to be kept in sync ‣ Several technologies involved ‣ Administration effort is huge Overhead&
- 9. Solution: Multi Model Database ‣ Can natively store several kinds of data models: ‣ Key-value pairs ‣ Documents ‣ Graphs ‣ Delivers query mechanisms for all data models 9
- 10. Polyglot Persistence Revisited 10 Source: Martin Fowler, http://martinfowler.com/articles/nosql-intro.pdf Shopping Cart KeyValue Reporting RDBMS Product Catalog Document User activity log Column Analytics Column Recommendations Graph Financial Data RDBMS User Sessions KeyValue Reporting RDBMS Product Catalog ArangoDB Shopping Cart ArangoDB User activity log Cassandra Analytics Cassandra Recommendations ArangoDB Financial Data ArangoDB User Sessions ArangoDB
- 11. Use Case: Multi-Model-Databases 11 Recommendations Product-CatalogShopping-Cart Sales-History Customer { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” } { “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, } 423453453 4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => 874365563 5463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } DocumentStore GraphStore DocumentStore DocumentStoreKeyValueStore
- 12. My four favorite features of ‣ AQL offering joins & traversals ‣ ACID including Multi Collection Transactions ‣ MULTI-MODEL stores graphs and documents 12 ‣ FOXX extend the API and adapt it to your needs
- 13. AQL 13 FOR p IN products RETURN p FILTER p.color == "yellow" FOR d IN GRAPH_DISTANCE_TO("ecom", p._id, "customer/alice", {direction: "inbound"}) LET r = {distance: x.distance, product: p} FILTER r.distance > 1 SORT r.distance LIMIT 10 RETURN r
- 14. ACID - Transactions ‣ Invoke a transaction: db._executeTransaction({ collections: { write: ["users", "products"], read: "recommendations" }, action: function() { // all operations go here } }); 14 throw "failure"; // Triggers rollback
- 15. ‣ Native mapping of data into DB ‣ DB optimized ‣ Queries are tailored for your data format ‣ Focus on writing business logic 15 ‣ Data has to be stored redundantly and has to be kept in sync ‣ Several technologies ‣ Administration effort is ‣ One technology involved Benefits Overhead&
- 16. Foxx ‣ Add your own customized and versioned REST-API on top of ArangoDB in JavaScript ‣ Include as a microservice in Rails, Node.js etc. ‣ Ship an administration fronted with it ‣ Built-in authentication using OAuth2.0 or HTTP-Basic Auth ‣ Operations are encapsulated in the database ‣ low network traffic, direct data access ‣ increases data privacy ➡ Multi-device setups ➡ Microservice Architectures / (~( ) ) /_/ ( _-----_(@ @) ( / /|/--| V " " " " 16
- 17. Foxx Example 17 FOR p IN products FILTER p.color == @color FOR d IN GRAPH_DISTANCE_TO("ecom", p._id, "customer/alice", {direction: "inbound"}) LET r = {distance: x.distance, product: p} FILTER r.distance > 1 SORT r.distance LIMIT 10 RETURN r', controller.get("/recommend/:color", function(req, res) { res.json(db._query('FOR p IN products }); {color: req.params("color")}).toArray());
- 18. ‣ open source and free (Apache 2 license) ‣ sharding & replication ‣ JavaScript throughout (V8 built into server) ‣ drivers for a wide range of languages ‣ web frontend ‣ good & complete documentation ‣ professional as well as community support 18 An overview of other features
- 19. Join our growing community 19 .. working on the geo index, the full text search and many APIs: Ruby, Python, PHP, Java, D, Javascript, ...
- 20. Thank you ‣ Further questions? ‣ Follow me on twitter/github: @mchacki ‣ Write me a mail: [email protected] ‣ Join or google group: https://groups.google.com/forum/#!forum/arangodb 20