SlideShare a Scribd company logo

Polyglot Persistence & Multi-Model Databases

ArangoDB Database
ArangoDB Database

The document discusses the shift from single-model databases to multi-model databases, emphasizing polyglot persistence in handling diverse data types such as key-value pairs, documents, and graphs. It highlights the importance of logical organization of data in collections and the flexibility of querying attributes. The presentation includes examples of documents and their structures in an e-commerce use case, illustrating the benefits of multi-model databases in modern data management.

Technology
1 of 57
Downloaded 39 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
www.arangodb.com Polyglot Persistence & Multi-Model Databases Michael Hackstein @mchacki Java User Group Hessen
Michael Hackstein ‣ ArangoDB Core Team ‣ Web Frontend ‣ Graph visualisation ‣ Graph features ‣ Host of cologne.js ‣ Master’s Degree
 (spec. Databases and
 Information Systems) 2
The Single Model era is over 3 Relational World
The Multi Model era begins 4 NoSQL World Documents - JSON Graphs Key Value { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" }
The Multi Model era begins 4 NoSQL World Documents - JSON Graphs Key Value { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } ‣ Normally based on key-value stores (each document still has a unique key) ‣ Allow to save documents with logical similarity in “collections” ‣ Treat data records as attribute-structured documents (data is no more opaque) ‣ Often allow querying and indexing document attributes
The Multi Model era begins 4 NoSQL World Documents - JSON Graphs Key Value { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } ‣ Focussed on m-to-n relations between entities ‣ Stores property graphs: entities and edges can have attributes ‣ Easily query paths of variable length ‣ Normally based on key-value stores (each document still has a unique key) ‣ Allow to save documents with logical similarity in “collections” ‣ Treat data records as attribute-structured documents (data is no more opaque) ‣ Often allow querying and indexing document attributes
The Multi Model era begins 4 NoSQL World Documents - JSON Graphs Key Value { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } ‣ Map value data to unique string keys (identifiers) ‣ Treat data as opaque (data has no schema) ‣ Can implement scaling and partitioning easily ‣ Focussed on m-to-n relations between entities ‣ Stores property graphs: entities and edges can have attributes ‣ Easily query paths of variable length ‣ Normally based on key-value stores (each document still has a unique key) ‣ Allow to save documents with logical similarity in “collections” ‣ Treat data records as attribute-structured documents (data is no more opaque) ‣ Often allow querying and indexing document attributes
e-commerce system in Relational World 5 Sales-History Shopping-Cart Recommendations Customer Product-Catalog
Polyglot Persistence 6 Source: Martin Fowler, http://martinfowler.com/articles/nosql-intro.pdf Reporting RDBMS Product Catalog MongoDB Shopping Cart Riak User activity log Cassandra Analytics Cassandra Recommendations Neo4J Financial Data RDBMS User Sessions Redis
Polyglot Persistence 6 Source: Martin Fowler, http://martinfowler.com/articles/nosql-intro.pdf Reporting RDBMS Product Catalog MongoDB Shopping Cart Riak User activity log Cassandra Analytics Cassandra Recommendations Neo4J Financial Data RDBMS User Sessions Redis Shopping Cart KeyValue Reporting RDBMS Product Catalog Document User activity log Column Analytics Column Recommendations Graph Financial Data RDBMS User Sessions KeyValue
Single Model Databases 7 Recommendations Product-CatalogShopping-Cart Sales-History Customer { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” } { “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, } 423453453 4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => 874365563 5463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" }
Single Model Databases 7 Recommendations Product-CatalogShopping-Cart Sales-History Customer { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” } { “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, } 423453453 4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => 874365563 5463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } DocumentStore GraphStore DocumentStore DocumentStoreKeyValueStore
Single Model Databases 7 Recommendations Product-CatalogShopping-Cart Sales-History Customer { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” } { “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, } 423453453 4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => 874365563 5463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } DocumentStore GraphStore DocumentStore DocumentStoreKeyValueStore
Benefits ‣ Natural mapping of data into DB ‣ DB optimized for the data format ‣ Queries are tailored for your data format ‣ Focus on writing business logic 8 ‣ Data has to be stored redundantly and has to be kept in sync ‣ Several technologies involved ‣ Administration effort is huge Overhead&
Solution: Multi Model Database ‣ Can natively store several kinds of data models: ‣ Key-value pairs ‣ Documents ‣ Graphs ‣ Delivers query mechanisms for all data models 9
Polyglot Persistence Revisited 10 Source: Martin Fowler, http://martinfowler.com/articles/nosql-intro.pdf Shopping Cart KeyValue Reporting RDBMS Product Catalog Document User activity log Column Analytics Column Recommendations Graph Financial Data RDBMS User Sessions KeyValue
Polyglot Persistence Revisited 10 Source: Martin Fowler, http://martinfowler.com/articles/nosql-intro.pdf Shopping Cart KeyValue Reporting RDBMS Product Catalog Document User activity log Column Analytics Column Recommendations Graph Financial Data RDBMS User Sessions KeyValue Reporting RDBMS Product Catalog ArangoDB Shopping Cart ArangoDB User activity log Cassandra Analytics Cassandra Recommendations ArangoDB Financial Data ArangoDB User Sessions ArangoDB
Use Case: Multi-Model-Databases 11 Recommendations Product-CatalogShopping-Cart Sales-History Customer { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” } { “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, } 423453453 4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => 874365563 5463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" }
Use Case: Multi-Model-Databases 11 Recommendations Product-CatalogShopping-Cart Sales-History Customer { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” } { “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, } 423453453 4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => 874365563 5463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } DocumentStore GraphStore DocumentStore DocumentStoreKeyValueStore
Use Case: Multi-Model-Databases 11 Recommendations Product-CatalogShopping-Cart Sales-History Customer { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” } { “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, } 423453453 4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => 874365563 5463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } DocumentStore GraphStore DocumentStore DocumentStoreKeyValueStore
My four favorite features of ‣ AQL offering joins & traversals ‣ ACID including Multi Collection Transactions ‣ MULTI-MODEL stores graphs and documents 12
My four favorite features of ‣ AQL offering joins & traversals ‣ ACID including Multi Collection Transactions ‣ MULTI-MODEL stores graphs and documents 12 ‣ FOXX extend the API and adapt it to your needs
AQL 13 FOR user IN users RETURN user
AQL 14 FOR user IN users FILTER user.name == "alice" RETURN user
AQL 15 FOR user IN users FILTER user.name == "alice" FOR invoice IN invoices FILTER user._key == invoice.customer RETURN { user: user, invoice: invoice }
AQL 16 FOR user IN users FILTER user.name == "alice" RETURN { user: user, invoices: ( FOR invoice IN invoices FILTER user._key == invoice.customer RETURN invoice ) }
AQL 17 FOR user IN users FILTER user.name == "alice" RETURN { user: user, hasToPay: SUM( FOR invoice IN invoices FILTER user._key == invoice.customer FILTER !invoice.payed RETURN invoice.price ) }
AQL - Alternative 18 FOR user IN users FILTER user.name == "alice" FOR invoice IN invoices FILTER user._key == invoice.customer FILTER !invoice.payed COLLECT u = user AGGREGATE toPay = SUM(invoice.price) RETURN { user: u, hasToPay: toPay }
AQL 19 FOR user IN users FILTER user.name == "alice" FOR product IN OUTBOUND user has_bought RETURN product
AQL 20 FOR user IN users FILTER user.name == "alice" FOR product IN OUTBOUND user has_bought LIMIT 10 RETURN product
AQL 21 FOR user IN users FILTER user.name == "alice" FOR product, action IN OUTBOUND user has_bought SORT action.timestamp DESC LIMIT 10 RETURN product
AQL 22 FOR user IN users FILTER user.name == "alice" FOR recommendation IN 3 ANY user has_bought OPTIONS { bfs: true, uniqueVertices: "global" } LIMIT 10 RETURN recommendation
AQL 23 FOR user IN users FILTER user.name == "alice" FOR recommendation, action, path IN 3 ANY user has_bought FILTER path.vertices[2].age <= user.age + 5 AND path.vertices[2].age >= user.age - 5 LIMIT 10 RETURN recommendation
AQL 24 FOR user IN users FILTER user.name == "alice" FOR recommendation, action, path IN 3 ANY user has_bought FILTER path.vertices[2].age <= user.age + 5 AND path.vertices[2].age >= user.age - 5 FILTER recommendation.price < 25 LIMIT 10 RETURN recommendation
ACID - Transactions ‣ Invoke a transaction: db._executeTransaction({ collections: { write: ["users", "products"], read: "has_bought" }, action: function() { // all operations go here } }); 25
ACID - Transactions ‣ Invoke a transaction: db._executeTransaction({ collections: { write: ["users", "products"], read: "has_bought" }, action: function() { // all operations go here } }); 25 throw "failure"; // Triggers rollback
‣ Native mapping of data into DB ‣ DB optimized ‣ Queries are tailored for your data format ‣ Focus on writing business logic 26 ‣ Data has to be stored redundantly and has to be kept in sync ‣ Several technologies involved ‣ Administration effort is huge Benefits Overhead&
‣ Native mapping of data into DB ‣ DB optimized ‣ Queries are tailored for your data format ‣ Focus on writing business logic 26 ‣ One technology involved Benefits Overhead&
Benchmark Comparison Source: https://www.arangodb.com/2015/10/benchmark-postgresql-mongodb-arangodb/
Foxx ‣ Add your own customized and versioned REST-API on top of ArangoDB in JavaScript ‣ Include as a microservice in Rails, Node.js etc. ‣ Ship an administration fronted with it ‣ Built-in authentication using OAuth2.0 or HTTP-Basic Auth ‣ Operations are encapsulated in the database ‣ low network traffic, direct data access ‣ increases data privacy ➡ Multi-device setups ➡ Microservice Architectures / (~( ) ) /_/ ( _-----_(@ @) ( / /|/--| V " " " " 28
Foxx Example 29 router.get("/recommend/:price", function(req, res) { res.json(db._query( }); { maxPrice: req.params("price"), name: req.session.username }).toArray()); `FOR user IN users FILTER user.name == @name FOR recommendation, action, path IN 3 ANY user has_bought OPTIONS { bfs: true, uniqueVertices: "global" } FILTER path.vertices[2].age <= user.age + 5 AND path.vertices[2].age >= user.age - 5 FILTER recommendation.price < @maxPrice LIMIT 10 RETURN recommendation`,
Does it Scale? ‣ Sharding (Huge Dataset, Write-Scaling) ‣ Collection: distributed across several servers ‣ Distributed by: List of Attributes (default: _key) ‣ Number of Shards immutable ‣ Has to be defined on creation ‣ Can be different for all collections ‣ Suggested: #Servers 30 2
Does it Scale? ‣ Sharding (Huge Dataset, Write-Scaling) ‣ Collection: distributed across several servers ‣ Distributed by: List of Attributes (default: _key) ‣ Number of Shards immutable ‣ Has to be defined on creation ‣ Can be different for all collections ‣ Suggested: #Servers 30 2 ‣ Replication (Failover, Read-Scaling) ‣ Shard: can have n Followers (Replicas) ‣ Followers: ‣ do not accept writes ‣ synchronous (identical data) ‣ placed on different machine (if possible)
Speaking of Clusters ... 31
32
33 Marathon
‣ Connects the cluster ‣ Ressource management ‣ Knows which servers are online ‣ Knows which resources they have 34
Marathon ‣ Builds on top of Mesos ‣ Application Monitor ‣ Can start new applications ‣ "Monitors" running applications ‣ If a machine/process dies Marathon restarts it ‣ Can stop applications ➡ Can scale up / down running applications within a cluster ‣ Offers available resources to new applications ‣ If Marathon dies it is restarted somewhere else 35
‣ Builds on top of Marathon ‣ "Data-Center-Operating-System" ‣ Offers one-click installers for applications (DC/OS Universe) ‣ Includes Service Discovery ‣ Includes Reverse Proxying ➡ You just run an Application through DC/OS and you can communicate with it without ever knowing any IP address 36
Frameworks ‣ Marathons little helpers ‣ Know details about complete Applications ‣ Communicate with Marathon to order sub-processes ‣ Can describe additional conditions for processes ‣ Like: "Do not run all DB instances on the same Machine" ‣ Can trigger marathon to scale up/down processes ➡ ArangoDB has a Marathon Framework 37
Live Demo ArangoDB on DC/OS 38
Is Mesosphere required? ‣ ArangoDB can run clusters without it ‣ Setup Requires manual effort (can be scripted): ‣ Configure IP addresses ‣ Correct startup ordering ‣ This works: ‣ Automatic Failover (Follower takes over if leader dies) ‣ Rebalancing of shards ‣ Everything inside of ArangoDB ‣ This is based on Mesos: ‣ Complete self healing ‣ Automatic restart of ArangoDBs (on new machines) ➡We suggest you have someone on call 39
Does your data-model scale? ‣ Rule of thumb: "The more complex the query, the less it scales". ‣ Indexed-Attribute lookups: ‣ Scale "infinitely" ‣ Examples ‣ Key/Value ‣ FOR doc IN sharded FILTER doc.name == 'Michael' ‣ Joins: ‣ Each join operation requires one additional round-trip (find left side, than find right side) ‣ Scale "okayish" ‣ Graphs: ‣ Each search depth potentially requires 2 join-like operations ‣ A single-server needs to hold a large intermediate result ‣ Does not scale too good 40
‣ open source and free (Apache 2 license) ‣ sharding & replication ‣ JavaScript throughout (V8 built into server) ‣ drivers for a wide range of languages ‣ web frontend ‣ good & complete documentation ‣ professional as well as community support 41 An overview of other features
What about Java? ‣ Basic Driver developed in-house ‣ Tutorial: https://www.arangodb.com/tutorials/tutorial-java/ ‣ We are working on a Spring-Data integration ‣ We are very happy to get input on this ‣ Feedback ‣ Feature Requests ‣ Personal Opinion on other Drivers/Integration you worked with ‣ Development 42
Join our growing community 43 .. working on the geo index, the full text search and many APIs: Ruby, Python, PHP, Java, D, Javascript, ...
Thank you ‣ Further questions? ‣ Follow us on twitter: @arangodb ‣ Join or community slack: arangodb-community.slack.com ‣ Join or google group: https://groups.google.com/forum/#!forum/arangodb ‣ Follow me on twitter/github: @mchacki ‣ Write me a mail: michael@arangodb.com 44

More Related Content

Viewers also liked (20)

PDF
Software + Babies
ArangoDB Database
 
PDF
Domain driven design @FrOSCon
ArangoDB Database
 
PDF
Extensibility of a database api with js
ArangoDB Database
 
PDF
Creating data centric microservices
ArangoDB Database
 
PDF
Guacamole
ArangoDB Database
 
PDF
Dr. dzaharudin mansor microsoft
Soo Chin Hock
 
PDF
Microservice-based software architecture
ArangoDB Database
 
PDF
Processing large-scale graphs with Google(TM) Pregel
ArangoDB Database
 
PDF
Performance comparison: Multi-Model vs. MongoDB and Neo4j
ArangoDB Database
 
PDF
Handling Billions of Edges in a Graph Database
ArangoDB Database
 
PDF
Shipment address classification in logistics, Ravindra Babu, Flipkart
Mohit Ranjan
 
PPTX
Optimization Modeling and Decision Support for Wireless Infrastructure Deploy...
streamspotter
 
DOC
Supplementary specification
Heba Fathy
 
PDF
Creating Fault Tolerant Services on Mesos
ArangoDB Database
 
PPTX
Requirment anlaysis
csk selva
 
PDF
Polyglot Persistence with MongoDB and Neo4j
Corie Pollock
 
PDF
NoSQL meets Microservices
ArangoDB Database
 
PDF
UMLtoGraphDB: Mapping Conceptual Schemas to Graph Databases
Gwendal Daniel
 
PPTX
Persistence that tells the truth: event sourcing
John Platte
 
PDF
Introduction to column oriented databases
ArangoDB Database
 
Software + Babies
ArangoDB Database
 
Domain driven design @FrOSCon
ArangoDB Database
 
Extensibility of a database api with js
ArangoDB Database
 
Creating data centric microservices
ArangoDB Database
 
Guacamole
ArangoDB Database
 
Dr. dzaharudin mansor microsoft
Soo Chin Hock
 
Microservice-based software architecture
ArangoDB Database
 
Processing large-scale graphs with Google(TM) Pregel
ArangoDB Database
 
Performance comparison: Multi-Model vs. MongoDB and Neo4j
ArangoDB Database
 
Handling Billions of Edges in a Graph Database
ArangoDB Database
 
Shipment address classification in logistics, Ravindra Babu, Flipkart
Mohit Ranjan
 
Optimization Modeling and Decision Support for Wireless Infrastructure Deploy...
streamspotter
 
Supplementary specification
Heba Fathy
 
Creating Fault Tolerant Services on Mesos
ArangoDB Database
 
Requirment anlaysis
csk selva
 
Polyglot Persistence with MongoDB and Neo4j
Corie Pollock
 
NoSQL meets Microservices
ArangoDB Database
 
UMLtoGraphDB: Mapping Conceptual Schemas to Graph Databases
Gwendal Daniel
 
Persistence that tells the truth: event sourcing
John Platte
 
Introduction to column oriented databases
ArangoDB Database
 

Similar to Polyglot Persistence & Multi-Model Databases (20)

PDF
OSDC 2015: Ingo Friepoertner | Polyglot Persistence & Multi-Model NoSQL Datab...
NETWAYS
 
PDF
Multi model-databases 29-10-2014 LJC
ArangoDB Database
 
PDF
Michael Hackstein - Polyglot Persistence & Multi-Model NoSQL Databases - NoSQ...
NoSQLmatters
 
PDF
Multi model-databases
Michael Hackstein
 
PDF
Multi model-databases
ArangoDB Database
 
KEY
Optimize drupal using mongo db
Vladimir Ilic
 
KEY
MongoDB, E-commerce and Transactions
Steven Francia
 
PDF
Oslo baksia2014
Max Neunhöffer
 
PDF
Go fast in a graph world
Andrea Giuliano
 
PPTX
Retail referencearchitecture productcatalog
MongoDB
 
PDF
Steam Learn: Introduction to NoSQL with MongoDB
inovia
 
PPTX
Nosql Now 2012: MongoDB Use Cases
MongoDB
 
PDF
Common MongoDB Use Cases
DATAVERSITY
 
PDF
Is multi-model the future of NoSQL?
Max Neunhöffer
 
PDF
A Graph Database That Scales - ArangoDB 3.7 Release Webinar
ArangoDB Database
 
PPTX
Common MongoDB Use Cases Webinar
MongoDB
 
PDF
Oslo bekk2014
Max Neunhöffer
 
PDF
Knowledge Graphs and AI to Hyper-Personalise the Fashion Retail Experience at...
Connected Data World
 
PPTX
A Practical Look at the NOSQL and Big Data Hullabaloo
Andrew Brust
 
PPTX
Retail Reference Architecture
MongoDB
 
OSDC 2015: Ingo Friepoertner | Polyglot Persistence & Multi-Model NoSQL Datab...
NETWAYS
 
Multi model-databases 29-10-2014 LJC
ArangoDB Database
 
Michael Hackstein - Polyglot Persistence & Multi-Model NoSQL Databases - NoSQ...
NoSQLmatters
 
Multi model-databases
Michael Hackstein
 
Multi model-databases
ArangoDB Database
 
Optimize drupal using mongo db
Vladimir Ilic
 
MongoDB, E-commerce and Transactions
Steven Francia
 
Oslo baksia2014
Max Neunhöffer
 
Go fast in a graph world
Andrea Giuliano
 
Retail referencearchitecture productcatalog
MongoDB
 
Steam Learn: Introduction to NoSQL with MongoDB
inovia
 
Nosql Now 2012: MongoDB Use Cases
MongoDB
 
Common MongoDB Use Cases
DATAVERSITY
 
Is multi-model the future of NoSQL?
Max Neunhöffer
 
A Graph Database That Scales - ArangoDB 3.7 Release Webinar
ArangoDB Database
 
Common MongoDB Use Cases Webinar
MongoDB
 
Oslo bekk2014
Max Neunhöffer
 
Knowledge Graphs and AI to Hyper-Personalise the Fashion Retail Experience at...
Connected Data World
 
A Practical Look at the NOSQL and Big Data Hullabaloo
Andrew Brust
 
Retail Reference Architecture
MongoDB
 
Ad

More from ArangoDB Database (20)

PPTX
ATO 2022 - Machine Learning + Graph Databases for Better Recommendations (3)....
ArangoDB Database
 
PPTX
Machine Learning + Graph Databases for Better Recommendations V2 08/20/2022
ArangoDB Database
 
PPTX
Machine Learning + Graph Databases for Better Recommendations V1 08/06/2022
ArangoDB Database
 
PPTX
ArangoDB 3.9 - Further Powering Graphs at Scale
ArangoDB Database
 
PDF
GraphSage vs Pinsage #InsideArangoDB
ArangoDB Database
 
PDF
Webinar: ArangoDB 3.8 Preview - Analytics at Scale
ArangoDB Database
 
PDF
Graph Analytics with ArangoDB
ArangoDB Database
 
PDF
Getting Started with ArangoDB Oasis
ArangoDB Database
 
PDF
Custom Pregel Algorithms in ArangoDB
ArangoDB Database
 
PPTX
Hacktoberfest 2020 - Intro to Knowledge Graphs
ArangoDB Database
 
PDF
gVisor, Kata Containers, Firecracker, Docker: Who is Who in the Container Space?
ArangoDB Database
 
PDF
ArangoML Pipeline Cloud - Managed Machine Learning Metadata
ArangoDB Database
 
PDF
ArangoDB 3.7 Roadmap: Performance at Scale
ArangoDB Database
 
PDF
Webinar: What to expect from ArangoDB Oasis
ArangoDB Database
 
PDF
ArangoDB 3.5 Feature Overview Webinar - Sept 12, 2019
ArangoDB Database
 
PDF
3.5 webinar
ArangoDB Database
 
PDF
Webinar: How native multi model works in ArangoDB
ArangoDB Database
 
PDF
An introduction to multi-model databases
ArangoDB Database
 
PDF
Running complex data queries in a distributed system
ArangoDB Database
 
PDF
Guacamole Fiesta: What do avocados and databases have in common?
ArangoDB Database
 
ATO 2022 - Machine Learning + Graph Databases for Better Recommendations (3)....
ArangoDB Database
 
Machine Learning + Graph Databases for Better Recommendations V2 08/20/2022
ArangoDB Database
 
Machine Learning + Graph Databases for Better Recommendations V1 08/06/2022
ArangoDB Database
 
ArangoDB 3.9 - Further Powering Graphs at Scale
ArangoDB Database
 
GraphSage vs Pinsage #InsideArangoDB
ArangoDB Database
 
Webinar: ArangoDB 3.8 Preview - Analytics at Scale
ArangoDB Database
 
Graph Analytics with ArangoDB
ArangoDB Database
 
Getting Started with ArangoDB Oasis
ArangoDB Database
 
Custom Pregel Algorithms in ArangoDB
ArangoDB Database
 
Hacktoberfest 2020 - Intro to Knowledge Graphs
ArangoDB Database
 
gVisor, Kata Containers, Firecracker, Docker: Who is Who in the Container Space?
ArangoDB Database
 
ArangoML Pipeline Cloud - Managed Machine Learning Metadata
ArangoDB Database
 
ArangoDB 3.7 Roadmap: Performance at Scale
ArangoDB Database
 
Webinar: What to expect from ArangoDB Oasis
ArangoDB Database
 
ArangoDB 3.5 Feature Overview Webinar - Sept 12, 2019
ArangoDB Database
 
3.5 webinar
ArangoDB Database
 
Webinar: How native multi model works in ArangoDB
ArangoDB Database
 
An introduction to multi-model databases
ArangoDB Database
 
Running complex data queries in a distributed system
ArangoDB Database
 
Guacamole Fiesta: What do avocados and databases have in common?
ArangoDB Database
 
Ad

Recently uploaded (20)

PPTX
AI Penetration Testing Essentials: A Cybersecurity Guide for 2025
defencerabbit Team
 
PPTX
COMPARISON OF RASTER ANALYSIS TOOLS OF QGIS AND ARCGIS
Sharanya Sarkar
 
PPTX
Building Search Using OpenSearch: Limitations and Workarounds
Sease
 
PDF
LOOPS in C Programming Language - Technology
RishabhDwivedi43
 
PPTX
Future Tech Innovations 2025 – A TechLists Insight
TechLists
 
PDF
Building Real-Time Digital Twins with IBM Maximo & ArcGIS Indoors
Safe Software
 
DOCX
Cryptography Quiz: test your knowledge of this important security concept.
Rajni Bhardwaj Grover
 
PDF
Achieving Consistent and Reliable AI Code Generation - Medusa AI
medusaaico
 
PDF
POV_ Why Enterprises Need to Find Value in ZERO.pdf
darshakparmar
 
PDF
Transcript: New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
PDF
CIFDAQ Token Spotlight for 9th July 2025
CIFDAQ
 
PDF
The Rise of AI and IoT in Mobile App Tech.pdf
IMG Global Infotech
 
PDF
"Beyond English: Navigating the Challenges of Building a Ukrainian-language R...
Fwdays
 
PDF
Biography of Daniel Podor.pdf
Daniel Podor
 
PDF
Go Concurrency Real-World Patterns, Pitfalls, and Playground Battles.pdf
Emily Achieng
 
PPTX
Webinar: Introduction to LF Energy EVerest
DanBrown980551
 
PDF
CIFDAQ Market Insights for July 7th 2025
CIFDAQ
 
DOCX
Python coding for beginners !! Start now!#
Rajni Bhardwaj Grover
 
PPTX
The Project Compass - GDG on Campus MSIT
dscmsitkol
 
PPTX
AUTOMATION AND ROBOTICS IN PHARMA INDUSTRY.pptx
sameeraaabegumm
 
AI Penetration Testing Essentials: A Cybersecurity Guide for 2025
defencerabbit Team
 
COMPARISON OF RASTER ANALYSIS TOOLS OF QGIS AND ARCGIS
Sharanya Sarkar
 
Building Search Using OpenSearch: Limitations and Workarounds
Sease
 
LOOPS in C Programming Language - Technology
RishabhDwivedi43
 
Future Tech Innovations 2025 – A TechLists Insight
TechLists
 
Building Real-Time Digital Twins with IBM Maximo & ArcGIS Indoors
Safe Software
 
Cryptography Quiz: test your knowledge of this important security concept.
Rajni Bhardwaj Grover
 
Achieving Consistent and Reliable AI Code Generation - Medusa AI
medusaaico
 
POV_ Why Enterprises Need to Find Value in ZERO.pdf
darshakparmar
 
Transcript: New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
CIFDAQ Token Spotlight for 9th July 2025
CIFDAQ
 
The Rise of AI and IoT in Mobile App Tech.pdf
IMG Global Infotech
 
"Beyond English: Navigating the Challenges of Building a Ukrainian-language R...
Fwdays
 
Biography of Daniel Podor.pdf
Daniel Podor
 
Go Concurrency Real-World Patterns, Pitfalls, and Playground Battles.pdf
Emily Achieng
 
Webinar: Introduction to LF Energy EVerest
DanBrown980551
 
CIFDAQ Market Insights for July 7th 2025
CIFDAQ
 
Python coding for beginners !! Start now!#
Rajni Bhardwaj Grover
 
The Project Compass - GDG on Campus MSIT
dscmsitkol
 
AUTOMATION AND ROBOTICS IN PHARMA INDUSTRY.pptx
sameeraaabegumm
 

Polyglot Persistence & Multi-Model Databases

  • 2. Michael Hackstein ‣ ArangoDB Core Team ‣ Web Frontend ‣ Graph visualisation ‣ Graph features ‣ Host of cologne.js ‣ Master’s Degree
 (spec. Databases and
 Information Systems) 2
  • 3. The Single Model era is over 3 Relational World
  • 4. The Multi Model era begins 4 NoSQL World Documents - JSON Graphs Key Value { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" }
  • 5. The Multi Model era begins 4 NoSQL World Documents - JSON Graphs Key Value { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } ‣ Normally based on key-value stores (each document still has a unique key) ‣ Allow to save documents with logical similarity in “collections” ‣ Treat data records as attribute-structured documents (data is no more opaque) ‣ Often allow querying and indexing document attributes
  • 6. The Multi Model era begins 4 NoSQL World Documents - JSON Graphs Key Value { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } ‣ Focussed on m-to-n relations between entities ‣ Stores property graphs: entities and edges can have attributes ‣ Easily query paths of variable length ‣ Normally based on key-value stores (each document still has a unique key) ‣ Allow to save documents with logical similarity in “collections” ‣ Treat data records as attribute-structured documents (data is no more opaque) ‣ Often allow querying and indexing document attributes
  • 7. The Multi Model era begins 4 NoSQL World Documents - JSON Graphs Key Value { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } ‣ Map value data to unique string keys (identifiers) ‣ Treat data as opaque (data has no schema) ‣ Can implement scaling and partitioning easily ‣ Focussed on m-to-n relations between entities ‣ Stores property graphs: entities and edges can have attributes ‣ Easily query paths of variable length ‣ Normally based on key-value stores (each document still has a unique key) ‣ Allow to save documents with logical similarity in “collections” ‣ Treat data records as attribute-structured documents (data is no more opaque) ‣ Often allow querying and indexing document attributes
  • 8. e-commerce system in Relational World 5 Sales-History Shopping-Cart Recommendations Customer Product-Catalog
  • 9. Polyglot Persistence 6 Source: Martin Fowler, http://martinfowler.com/articles/nosql-intro.pdf Reporting RDBMS Product Catalog MongoDB Shopping Cart Riak User activity log Cassandra Analytics Cassandra Recommendations Neo4J Financial Data RDBMS User Sessions Redis
  • 10. Polyglot Persistence 6 Source: Martin Fowler, http://martinfowler.com/articles/nosql-intro.pdf Reporting RDBMS Product Catalog MongoDB Shopping Cart Riak User activity log Cassandra Analytics Cassandra Recommendations Neo4J Financial Data RDBMS User Sessions Redis Shopping Cart KeyValue Reporting RDBMS Product Catalog Document User activity log Column Analytics Column Recommendations Graph Financial Data RDBMS User Sessions KeyValue
  • 11. Single Model Databases 7 Recommendations Product-CatalogShopping-Cart Sales-History Customer { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” } { “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, } 423453453 4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => 874365563 5463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" }
  • 12. Single Model Databases 7 Recommendations Product-CatalogShopping-Cart Sales-History Customer { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” } { “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, } 423453453 4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => 874365563 5463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } DocumentStore GraphStore DocumentStore DocumentStoreKeyValueStore
  • 13. Single Model Databases 7 Recommendations Product-CatalogShopping-Cart Sales-History Customer { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” } { “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, } 423453453 4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => 874365563 5463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } DocumentStore GraphStore DocumentStore DocumentStoreKeyValueStore
  • 14. Benefits ‣ Natural mapping of data into DB ‣ DB optimized for the data format ‣ Queries are tailored for your data format ‣ Focus on writing business logic 8 ‣ Data has to be stored redundantly and has to be kept in sync ‣ Several technologies involved ‣ Administration effort is huge Overhead&
  • 15. Solution: Multi Model Database ‣ Can natively store several kinds of data models: ‣ Key-value pairs ‣ Documents ‣ Graphs ‣ Delivers query mechanisms for all data models 9
  • 16. Polyglot Persistence Revisited 10 Source: Martin Fowler, http://martinfowler.com/articles/nosql-intro.pdf Shopping Cart KeyValue Reporting RDBMS Product Catalog Document User activity log Column Analytics Column Recommendations Graph Financial Data RDBMS User Sessions KeyValue
  • 17. Polyglot Persistence Revisited 10 Source: Martin Fowler, http://martinfowler.com/articles/nosql-intro.pdf Shopping Cart KeyValue Reporting RDBMS Product Catalog Document User activity log Column Analytics Column Recommendations Graph Financial Data RDBMS User Sessions KeyValue Reporting RDBMS Product Catalog ArangoDB Shopping Cart ArangoDB User activity log Cassandra Analytics Cassandra Recommendations ArangoDB Financial Data ArangoDB User Sessions ArangoDB
  • 18. Use Case: Multi-Model-Databases 11 Recommendations Product-CatalogShopping-Cart Sales-History Customer { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” } { “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, } 423453453 4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => 874365563 5463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" }
  • 19. Use Case: Multi-Model-Databases 11 Recommendations Product-CatalogShopping-Cart Sales-History Customer { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” } { “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, } 423453453 4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => 874365563 5463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } DocumentStore GraphStore DocumentStore DocumentStoreKeyValueStore
  • 20. Use Case: Multi-Model-Databases 11 Recommendations Product-CatalogShopping-Cart Sales-History Customer { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “userID": 239178239, “productID”: 128623883, “number": 5, “price”: 12.20, } { “Name": "Smith", “lastLogin”: “2012-11-01", “Visits": 121, “shipping address”: “abc”, “shipping address”: “def” } { “Name": "Meyer", “lastLogin”: “2012-11-21", “Visits": 20, “shipping address”: “xyz”, } 423453453 4328, “shirt”, “L”, 1, 12.99 6378, “sweater”, “M”, 2, 37.95 3245, “sweater”, “blue”, 1, 99.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => 874365563 5463, “shirt”, “S”, 1, 9.99 6378, “sweater”, “M”, 2, 37.95 3245, “pants”, “32/34”, “black”, 1, 99.95 => { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } DocumentStore GraphStore DocumentStore DocumentStoreKeyValueStore
  • 21. My four favorite features of ‣ AQL offering joins & traversals ‣ ACID including Multi Collection Transactions ‣ MULTI-MODEL stores graphs and documents 12
  • 22. My four favorite features of ‣ AQL offering joins & traversals ‣ ACID including Multi Collection Transactions ‣ MULTI-MODEL stores graphs and documents 12 ‣ FOXX extend the API and adapt it to your needs
  • 23. AQL 13 FOR user IN users RETURN user
  • 24. AQL 14 FOR user IN users FILTER user.name == "alice" RETURN user
  • 25. AQL 15 FOR user IN users FILTER user.name == "alice" FOR invoice IN invoices FILTER user._key == invoice.customer RETURN { user: user, invoice: invoice }
  • 26. AQL 16 FOR user IN users FILTER user.name == "alice" RETURN { user: user, invoices: ( FOR invoice IN invoices FILTER user._key == invoice.customer RETURN invoice ) }
  • 27. AQL 17 FOR user IN users FILTER user.name == "alice" RETURN { user: user, hasToPay: SUM( FOR invoice IN invoices FILTER user._key == invoice.customer FILTER !invoice.payed RETURN invoice.price ) }
  • 28. AQL - Alternative 18 FOR user IN users FILTER user.name == "alice" FOR invoice IN invoices FILTER user._key == invoice.customer FILTER !invoice.payed COLLECT u = user AGGREGATE toPay = SUM(invoice.price) RETURN { user: u, hasToPay: toPay }
  • 29. AQL 19 FOR user IN users FILTER user.name == "alice" FOR product IN OUTBOUND user has_bought RETURN product
  • 30. AQL 20 FOR user IN users FILTER user.name == "alice" FOR product IN OUTBOUND user has_bought LIMIT 10 RETURN product
  • 31. AQL 21 FOR user IN users FILTER user.name == "alice" FOR product, action IN OUTBOUND user has_bought SORT action.timestamp DESC LIMIT 10 RETURN product
  • 32. AQL 22 FOR user IN users FILTER user.name == "alice" FOR recommendation IN 3 ANY user has_bought OPTIONS { bfs: true, uniqueVertices: "global" } LIMIT 10 RETURN recommendation
  • 33. AQL 23 FOR user IN users FILTER user.name == "alice" FOR recommendation, action, path IN 3 ANY user has_bought FILTER path.vertices[2].age <= user.age + 5 AND path.vertices[2].age >= user.age - 5 LIMIT 10 RETURN recommendation
  • 34. AQL 24 FOR user IN users FILTER user.name == "alice" FOR recommendation, action, path IN 3 ANY user has_bought FILTER path.vertices[2].age <= user.age + 5 AND path.vertices[2].age >= user.age - 5 FILTER recommendation.price < 25 LIMIT 10 RETURN recommendation
  • 35. ACID - Transactions ‣ Invoke a transaction: db._executeTransaction({ collections: { write: ["users", "products"], read: "has_bought" }, action: function() { // all operations go here } }); 25
  • 36. ACID - Transactions ‣ Invoke a transaction: db._executeTransaction({ collections: { write: ["users", "products"], read: "has_bought" }, action: function() { // all operations go here } }); 25 throw "failure"; // Triggers rollback
  • 37. ‣ Native mapping of data into DB ‣ DB optimized ‣ Queries are tailored for your data format ‣ Focus on writing business logic 26 ‣ Data has to be stored redundantly and has to be kept in sync ‣ Several technologies involved ‣ Administration effort is huge Benefits Overhead&
  • 38. ‣ Native mapping of data into DB ‣ DB optimized ‣ Queries are tailored for your data format ‣ Focus on writing business logic 26 ‣ One technology involved Benefits Overhead&
  • 40. Foxx ‣ Add your own customized and versioned REST-API on top of ArangoDB in JavaScript ‣ Include as a microservice in Rails, Node.js etc. ‣ Ship an administration fronted with it ‣ Built-in authentication using OAuth2.0 or HTTP-Basic Auth ‣ Operations are encapsulated in the database ‣ low network traffic, direct data access ‣ increases data privacy ➡ Multi-device setups ➡ Microservice Architectures / (~( ) ) /_/ ( _-----_(@ @) ( / /|/--| V " " " " 28
  • 41. Foxx Example 29 router.get("/recommend/:price", function(req, res) { res.json(db._query( }); { maxPrice: req.params("price"), name: req.session.username }).toArray()); `FOR user IN users FILTER user.name == @name FOR recommendation, action, path IN 3 ANY user has_bought OPTIONS { bfs: true, uniqueVertices: "global" } FILTER path.vertices[2].age <= user.age + 5 AND path.vertices[2].age >= user.age - 5 FILTER recommendation.price < @maxPrice LIMIT 10 RETURN recommendation`,
  • 42. Does it Scale? ‣ Sharding (Huge Dataset, Write-Scaling) ‣ Collection: distributed across several servers ‣ Distributed by: List of Attributes (default: _key) ‣ Number of Shards immutable ‣ Has to be defined on creation ‣ Can be different for all collections ‣ Suggested: #Servers 30 2
  • 43. Does it Scale? ‣ Sharding (Huge Dataset, Write-Scaling) ‣ Collection: distributed across several servers ‣ Distributed by: List of Attributes (default: _key) ‣ Number of Shards immutable ‣ Has to be defined on creation ‣ Can be different for all collections ‣ Suggested: #Servers 30 2 ‣ Replication (Failover, Read-Scaling) ‣ Shard: can have n Followers (Replicas) ‣ Followers: ‣ do not accept writes ‣ synchronous (identical data) ‣ placed on different machine (if possible)
  • 45. 32
  • 47. ‣ Connects the cluster ‣ Ressource management ‣ Knows which servers are online ‣ Knows which resources they have 34
  • 48. Marathon ‣ Builds on top of Mesos ‣ Application Monitor ‣ Can start new applications ‣ "Monitors" running applications ‣ If a machine/process dies Marathon restarts it ‣ Can stop applications ➡ Can scale up / down running applications within a cluster ‣ Offers available resources to new applications ‣ If Marathon dies it is restarted somewhere else 35
  • 49. ‣ Builds on top of Marathon ‣ "Data-Center-Operating-System" ‣ Offers one-click installers for applications (DC/OS Universe) ‣ Includes Service Discovery ‣ Includes Reverse Proxying ➡ You just run an Application through DC/OS and you can communicate with it without ever knowing any IP address 36
  • 50. Frameworks ‣ Marathons little helpers ‣ Know details about complete Applications ‣ Communicate with Marathon to order sub-processes ‣ Can describe additional conditions for processes ‣ Like: "Do not run all DB instances on the same Machine" ‣ Can trigger marathon to scale up/down processes ➡ ArangoDB has a Marathon Framework 37
  • 52. Is Mesosphere required? ‣ ArangoDB can run clusters without it ‣ Setup Requires manual effort (can be scripted): ‣ Configure IP addresses ‣ Correct startup ordering ‣ This works: ‣ Automatic Failover (Follower takes over if leader dies) ‣ Rebalancing of shards ‣ Everything inside of ArangoDB ‣ This is based on Mesos: ‣ Complete self healing ‣ Automatic restart of ArangoDBs (on new machines) ➡We suggest you have someone on call 39
  • 53. Does your data-model scale? ‣ Rule of thumb: "The more complex the query, the less it scales". ‣ Indexed-Attribute lookups: ‣ Scale "infinitely" ‣ Examples ‣ Key/Value ‣ FOR doc IN sharded FILTER doc.name == 'Michael' ‣ Joins: ‣ Each join operation requires one additional round-trip (find left side, than find right side) ‣ Scale "okayish" ‣ Graphs: ‣ Each search depth potentially requires 2 join-like operations ‣ A single-server needs to hold a large intermediate result ‣ Does not scale too good 40
  • 54. ‣ open source and free (Apache 2 license) ‣ sharding & replication ‣ JavaScript throughout (V8 built into server) ‣ drivers for a wide range of languages ‣ web frontend ‣ good & complete documentation ‣ professional as well as community support 41 An overview of other features
  • 55. What about Java? ‣ Basic Driver developed in-house ‣ Tutorial: https://www.arangodb.com/tutorials/tutorial-java/ ‣ We are working on a Spring-Data integration ‣ We are very happy to get input on this ‣ Feedback ‣ Feature Requests ‣ Personal Opinion on other Drivers/Integration you worked with ‣ Development 42
  • 56. Join our growing community 43 .. working on the geo index, the full text search and many APIs: Ruby, Python, PHP, Java, D, Javascript, ...
  • 57. Thank you ‣ Further questions? ‣ Follow us on twitter: @arangodb ‣ Join or community slack: arangodb-community.slack.com ‣ Join or google group: https://groups.google.com/forum/#!forum/arangodb ‣ Follow me on twitter/github: @mchacki ‣ Write me a mail: [email protected] 44