Graph Databases - Where Do We Do the Modeling Part?
5 likes3,920 views
Karen Lopez discusses the importance of data modeling in graph databases, emphasizing hybrid technologies that include both SQL and NoSQL. She shares various modeling techniques, the structural concepts of graphs, and how to effectively transition from relational to graph frameworks. Among her insights are practical tips for architects on leveraging graph databases and the need for ongoing adaptation in modeling practices.
![Graph Databases – Neo4j
CREATE (matrix1:Movie { title : 'The Matrix', year : '1999-03-31' })
CREATE (matrix2:Movie { title : 'The Matrix Reloaded', year : '2003-05-07' })
CREATE (matrix3:Movie { title : 'The Matrix Revolutions', year : '2003-10-27' })
CREATE (keanu:Actor { name:'Keanu Reeves' })
CREATE (laurence:Actor { name:'Laurence Fishburne' })
CREATE (carrieanne:Actor { name:'Carrie-Anne Moss' })
CREATE (keanu)-[:ACTS_IN { role : 'Neo' }]->(matrix1)
CREATE (keanu)-[:ACTS_IN { role : 'Neo' }]->(matrix2)
CREATE (keanu)-[:ACTS_IN { role : 'Neo' }]->(matrix3)
CREATE (laurence)-[:ACTS_IN { role : 'Morpheus' }]->(matrix1)
CREATE (laurence)-[:ACTS_IN { role : 'Morpheus' }]->(matrix2)
CREATE (laurence)-[:ACTS_IN { role : 'Morpheus' }]->(matrix3)
CREATE (carrieanne)-[:ACTS_IN { role : 'Trinity' }]->(matrix1)
CREATE (carrieanne)-[:ACTS_IN { role : 'Trinity' }]->(matrix2)
CREATE (carrieanne)-[:ACTS_IN { role : 'Trinity' }]->(matrix3)
http://neo4j.com/docs/stable/cypherdoc-movie-database.html](https://image.slidesharecdn.com/2016maygraphheartdata-160503035456/85/Graph-Databases-Where-Do-We-Do-the-Modeling-Part-33-320.jpg)
![Querying Neo4j with Cypher
MATCH (you {name:"You"})
MATCH (expert)-
[:WORKED_WITH]-
>(db:Database
{name:"Neo4j"}) MATCH
path = shortestPath( (you)-
[:FRIEND*..5]-(expert) )
RETURN db,expert,path](https://image.slidesharecdn.com/2016maygraphheartdata-160503035456/85/Graph-Databases-Where-Do-We-Do-the-Modeling-Part-35-320.jpg)
![Querying IBM Graph
var url = process.env.graphDBURL + '/gremlin';
var query = "def g = graph.traversal(); g.V().has('code','" + req.body.orig +
"').out('route').has('code', '" + req.body.dest + "')";
var opts = { auth: { user: process.env.username, pass: process.env.password
}, json: { gremlin: query } };
request.post(url, opts, function(error, resp, obj) {
var result = (obj.result && obj.result.data && obj.result.data.length >
0) ? obj.result.data[0] : null;
if (result) { // found a route from orig to dest console.log('route
exists from ' + req.body.orig + ' to ' + req.body.dest); } });](https://image.slidesharecdn.com/2016maygraphheartdata-160503035456/85/Graph-Databases-Where-Do-We-Do-the-Modeling-Part-37-320.jpg)
Graph Databases - Where Do We Do the Modeling Part?
- 1. Karen Lopez @datachick #HeartData Heart of Data Modeling Graph Databases: Where does the modeling go?
- 2. Yes, Please do Tweet/Share today’s event @datachick #heartdata
- 3. Karen López Karen has 20+ years of data and information architecture experience on large, multi-project programs. She is a frequent speaker on data modeling, data-driven methodologies and pattern data models. She wants you to love your data… She is loves new tech and gadgets
- 4. How new tech are you? ...so let’s get to know you….
- 5. Use Q&A for formal questions Get them in now! Use chat to discuss with each other We have a great community Yes! Slides Recording …next week…
- 6. Plan for Today Why topic? Graphy Stuff in Relational World Another way? Graph Resources
- 7. Modern Data Architectures will have hybrid Technologies WHERE ‘HYBRID’ = ‘SQL and NoSQL’
- 8. Clarifying terminology Graphs Graph Databases Graph Processing 8
- 9. NoSQL, Not Only SQL 9 Relational Graph Columnar Key Value Document Databases Column Family
- 10. Graph & Hierarchy Concepts Overview
- 13. What is this structure?
- 14. Ragged Hierarchies A hierarchy where there is variability in the number of levels across branches. Node Node Node Node Node Node Node Node Node Node Node Node 14
- 15. Automobile Engine Fuel Line Valve Injector Fan Fan Blade Bearing Bolt Fanbelt Entertainment System Bolt Radio Satellite Radio Media Player Backup camera Automobile Engine Injection System Fuel Line Valve Injector Fan Fan Blade Fan Bearing Fanbelt Entertainment System Bolt Radio Satellite Radio Media Player Backup camera Energy Graph What Happens When…….? Sometimes we take a group of “sibling widgets” and make them a widget just for them. Think “subassembly”. Then we have to think of this new group as a widget. 15
- 16. How we Model Graph in Relational Lots of tricks and tips happening here.
- 17. Recursive Relationship Self Join Recursive Association Dog Ear / Mouse Ear Bill of Materials ???
- 18. Data Model – Hierarchy Recursive
- 19. Data Model – Hierarchy Recursive 19 What happens when: We add a new level? Take one away? Promote someone?
- 20. Relational Performance Tricks Special data types Adjacency Lists Path Enumerations Closure Tables Nested Sets 20
- 22. Now we have M:N 22 Employee Reporting Structure
- 23. Hierarchy vs. Real World 23
- 27. But aren’t relational databases about relationships?
- 28. Labeled Property Graph Nodes have properties (think key-value pairs) Nodes have labels (think meta-data and categories) Relationships are directed Relationships have names Relationships have a start and end node Relationships have properties 28 Node property property Node property NodeNode Node Label LabelLabel Label
- 32. TripleStores Come from semantic technologies movement A triple is a subject:predicate:object data structure Individually triples are semantically poor En masse they provide rich dataset to harvest knowledge and infer connections Use RDF and XML--SPARQL for queries Ginger dances with Fred Fred likes ice cream Karen loves data 32
- 33. Graph Databases – Neo4j CREATE (matrix1:Movie { title : 'The Matrix', year : '1999-03-31' }) CREATE (matrix2:Movie { title : 'The Matrix Reloaded', year : '2003-05-07' }) CREATE (matrix3:Movie { title : 'The Matrix Revolutions', year : '2003-10-27' }) CREATE (keanu:Actor { name:'Keanu Reeves' }) CREATE (laurence:Actor { name:'Laurence Fishburne' }) CREATE (carrieanne:Actor { name:'Carrie-Anne Moss' }) CREATE (keanu)-[:ACTS_IN { role : 'Neo' }]->(matrix1) CREATE (keanu)-[:ACTS_IN { role : 'Neo' }]->(matrix2) CREATE (keanu)-[:ACTS_IN { role : 'Neo' }]->(matrix3) CREATE (laurence)-[:ACTS_IN { role : 'Morpheus' }]->(matrix1) CREATE (laurence)-[:ACTS_IN { role : 'Morpheus' }]->(matrix2) CREATE (laurence)-[:ACTS_IN { role : 'Morpheus' }]->(matrix3) CREATE (carrieanne)-[:ACTS_IN { role : 'Trinity' }]->(matrix1) CREATE (carrieanne)-[:ACTS_IN { role : 'Trinity' }]->(matrix2) CREATE (carrieanne)-[:ACTS_IN { role : 'Trinity' }]->(matrix3) http://neo4j.com/docs/stable/cypherdoc-movie-database.html
- 34. Notice anyting?
- 35. Querying Neo4j with Cypher MATCH (you {name:"You"}) MATCH (expert)- [:WORKED_WITH]- >(db:Database {name:"Neo4j"}) MATCH path = shortestPath( (you)- [:FRIEND*..5]-(expert) ) RETURN db,expert,path
- 36. IBM Graph
- 37. Querying IBM Graph var url = process.env.graphDBURL + '/gremlin'; var query = "def g = graph.traversal(); g.V().has('code','" + req.body.orig + "').out('route').has('code', '" + req.body.dest + "')"; var opts = { auth: { user: process.env.username, pass: process.env.password }, json: { gremlin: query } }; request.post(url, opts, function(error, resp, obj) { var result = (obj.result && obj.result.data && obj.result.data.length > 0) ? obj.result.data[0] : null; if (result) { // found a route from orig to dest console.log('route exists from ' + req.body.orig + ' to ' + req.body.dest); } });
- 38. Another IBM Graph model
- 39. Titan
- 40. Tools and Graph Databases •No native supportERwin •No native supportER/Studio •No native supportPowerDesigner “the data model is the database” “the database is the data model” ODBC / JDBC connectively for querying.
- 41. So what about modeling for graph? Marketing vs. Real Project There is a model The model isn’t the structure The model would be used to design the graph(s) Same modeling issues: Naming Properties Rules Consistency Governance
- 42. Data Modeling & Graph No* Logical + Physical Data model The graph is the data model...and the database Whiteboard data modeling Traditional data models still have a role
- 43. Requirements Data Model Database* More requirements / changes / tuning / whims + Non Model Stuff Data Model Driven Data Model Driven
- 45. 10+ Tips for Architects 1. Understand the use cases for graph technologies 2. Evaluate/profile your data requirements for suitability for graph databases and/or graph processing 3. ACID support varies across products. You’ll want to test your use cases. 4. Your query data stories will guide your decisions 5. Test your current development tools for support
- 46. 10+ Tips for Architects 6. Test your database design/data modeling tools 7. Leverage your existing metadata/models 8. True hierarchies are VERY RARE in the real world. 9. Know the questions you have to ask about all the exceptions 10.Keep asking where the data integrity happens/is relevant 46
- 47. Resources 47
- 49. Fun with Graphs Scotch Whiskeys Belgian Beer Bank Fraud Detection Access Control Management …and 100 more…. GraphGist Project - http://gist.neo4j.org/
- 51. Trees and Hierarchies in SQL for Smarties 51
- 55. “Every design decision should include cost, benefit and risk” - Karen Lopez
- 56. Thank you, you were great. Really, really great. Karen Lopez @datachick www.datamodel.com