Django and Neo4j - Domain modeling that kicks ass
47 likes30,463 views
This document discusses Neo4j, a graph database that models connections between entities, addressing the limitations of traditional relational databases. It highlights its open-source nature, transactional capabilities, and high performance in handling complex relationships through a schema-free design. The document also provides examples of graph modeling and operations using Neo4j's API.
![Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ]
order = neo4j.BREADTH_FIRST
stop = neo4j.STOP_AT_END_OF_GRAPH
returnable = neo4j.RETURN_ALL_BUT_START_NODE
20](https://image.slidesharecdn.com/djangoandneo4j-100721185000-phpapp02/85/Django-and-Neo4j-Domain-modeling-that-kicks-ass-35-320.jpg)
![Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ]
order = neo4j.BREADTH_FIRST
stop = neo4j.STOP_AT_END_OF_GRAPH
returnable = neo4j.RETURN_ALL_BUT_START_NODE
for friend_node in Friends(mr_anderson):
print "%s (@ depth=%s)" % ( friend_node["name"],
friend_node.depth )
20](https://image.slidesharecdn.com/djangoandneo4j-100721185000-phpapp02/85/Django-and-Neo4j-Domain-modeling-that-kicks-ass-36-320.jpg)
![Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ]
order = neo4j.BREADTH_FIRST
stop = neo4j.STOP_AT_END_OF_GRAPH
returnable = neo4j.RETURN_ALL_BUT_START_NODE
for friend_node in Friends(mr_anderson):
print "%s (@ depth=%s)" % ( friend_node["name"],
friend_node.depth )
20](https://image.slidesharecdn.com/djangoandneo4j-100721185000-phpapp02/85/Django-and-Neo4j-Domain-modeling-that-kicks-ass-37-320.jpg)
![Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1)
order = neo4j.BREADTH_FIRST
stop = neo4j.STOP_AT_END_OF_GRAPH
returnable = neo4j.RETURN_ALL_BUT_START_NODE
for friend_node in Friends(mr_anderson):
print "%s (@ depth=%s)" % ( friend_node["name"],
friend_node.depth )
20](https://image.slidesharecdn.com/djangoandneo4j-100721185000-phpapp02/85/Django-and-Neo4j-Domain-modeling-that-kicks-ass-38-320.jpg)
![Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1)
order = neo4j.BREADTH_FIRST Trinity (@ depth=1)
stop = neo4j.STOP_AT_END_OF_GRAPH
returnable = neo4j.RETURN_ALL_BUT_START_NODE
for friend_node in Friends(mr_anderson):
print "%s (@ depth=%s)" % ( friend_node["name"],
friend_node.depth )
20](https://image.slidesharecdn.com/djangoandneo4j-100721185000-phpapp02/85/Django-and-Neo4j-Domain-modeling-that-kicks-ass-39-320.jpg)
![Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1)
order = neo4j.BREADTH_FIRST Trinity (@ depth=1)
stop = neo4j.STOP_AT_END_OF_GRAPH
Cypher (@ depth=2)
returnable = neo4j.RETURN_ALL_BUT_START_NODE
for friend_node in Friends(mr_anderson):
print "%s (@ depth=%s)" % ( friend_node["name"],
friend_node.depth )
20](https://image.slidesharecdn.com/djangoandneo4j-100721185000-phpapp02/85/Django-and-Neo4j-Domain-modeling-that-kicks-ass-40-320.jpg)
![Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1)
order = neo4j.BREADTH_FIRST Trinity (@ depth=1)
stop = neo4j.STOP_AT_END_OF_GRAPH
Cypher (@ depth=2)
returnable = neo4j.RETURN_ALL_BUT_START_NODE
Agent Smith (@ depth=3)
for friend_node in Friends(mr_anderson):
print "%s (@ depth=%s)" % ( friend_node["name"],
friend_node.depth )
20](https://image.slidesharecdn.com/djangoandneo4j-100721185000-phpapp02/85/Django-and-Neo4j-Domain-modeling-that-kicks-ass-41-320.jpg)
![Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1)
order = neo4j.BREADTH_FIRST Trinity (@ depth=1)
stop = neo4j.STOP_AT_END_OF_GRAPH
Cypher (@ depth=2)
returnable = neo4j.RETURN_ALL_BUT_START_NODE
Agent Smith (@ depth=3)
for friend_node in Friends(mr_anderson):
print "%s (@ depth=%s)" % ( friend_node["name"],
friend_node.depth )
20](https://image.slidesharecdn.com/djangoandneo4j-100721185000-phpapp02/85/Django-and-Neo4j-Domain-modeling-that-kicks-ass-42-320.jpg)
![Finding a place to start
๏ Traversals need a Node to start from
• QUESTION: How do I find the start Node?
• ANSWER:You use an Index
๏ Indexes in Neo4j are different from Indexes in Relational Databases
• RDBMSes use them for Joining
• Neo4j use them for simple lookup
index = graphDb.index["name"]
mr_anderson = index["Thomas Anderson"]
performTraversalFrom( mrAnderson )
21](https://image.slidesharecdn.com/djangoandneo4j-100721185000-phpapp02/85/Django-and-Neo4j-Domain-modeling-that-kicks-ass-43-320.jpg)
Django and Neo4j - Domain modeling that kicks ass
- 1. Django and Neo4j Domain modeling that kicks ass! twitter: @thobe / #neo4j Tobias Ivarsson email: [email protected] web: http://www.neo4j.org/ Hacker @ Neo Technology web: http://www.thobe.org/
- 2. It all started with this guy. Emil Eifrem, CEO of Neo Technology. We picked him apart, and inside his brain we found the base for a database that models the connections in bet ween entities. 2
- 3. It all started with this guy. Emil Eifrem, CEO of Neo Technology. We picked him apart, and inside his brain we found the base for a database that models the connections in bet ween entities. Image credits: US Army 2
- 4. It all started with this guy. Emil Eifrem, CEO of Neo Technology. We picked him apart, and inside his brain we found the base for a database that models the connections in bet ween entities. 2
- 5. Neo4j It all started with this guy. Emil Eifrem, CEO of Neo Technology. We picked him apart, and inside his brain we found the base for a database that models the connections in bet ween entities. 2
- 6. NOSQL is a wide area 3
- 7. The problems NOSQL focuses on Relational database Requirement of application Focus area of many NOSQL Databases ๏ Huge amounts of data Performance ๏ (mostly) Disjoint data Salary List ๏ Heavy load most focus on... Majority of Webapps ๏ Many concurrent writers Social network Semantic Trading All NOSQL databases focus on solving problems where RDBMSes fail. While this handles the load, it lacks in “social” } custom Data complexity 4
- 8. The evolution of data Giant Global Graph (GGG) Ontologies RDF Folksonomies Information connectivity Tagging Wikis User-generated content Blogs ... but it turns out that RSS data evolves to become MORE interconnected Hypertext (as well as greater sizes) Text documents web 1.0 web 2.0 “web 3.0” 1990 2000 2010 2020 5
- 9. Neo4j is a Graph Database Graph databases FOCUS on the interconnection bet ween entities. 6
- 10. IS_A Neo4j Graph Database Graph databases FOCUS on the interconnection bet ween entities. 6
- 11. Scaling to size vs. Scaling to complexity Size Key/Value stores Bigtable clones Document databases Graph databases Complexity 7
- 12. Scaling to size vs. Scaling to complexity Size Key/Value stores Bigtable clones Document databases Graph databases Billions of nodes and relationships > 90% of use cases Complexity 7
- 13. What is Neo4j? ๏ Neo4j is a Graph Database • Non-relational (“#nosql”), transactional (ACID), embedded • Data is stored as a Graph / Network ‣Nodes and relationships with properties ‣“Property Graph” or “edge-labeled multidigraph” • Schema free, bottom-up data model design ๏ Neo4j is Open Source / Free (as in speech) Software Prices are available at http://neotechnology.com/ • AGPLv3 Contact us if you have questions and/or special license needs (e.g. if you • Commercial (“dual license”) license available want an evaluation license) ‣First server is free (as in beer), next is inexpensive 8
- 14. More about Neo4j ๏ Neo4j is stable • In 24/7 operation since 2003 ๏ Neo4j is in active development • Neo Technology received VC funding October 2009 ๏ Neo4j delivers high performance graph operations • traverses 1’000’000+ relationships / second on commodity hardware (1000~2500 traversals/ms) 9
- 15. Building business applications with Neo4j ๏ Try it out! It’s all open source! • Build a prototype, find out your needs and how Neo4j matches • AGPL this stage thisusers should have access to your code - at says all your is your employees / co-workers 10
- 16. Building business applications with Neo4j ๏ Try it out! It’s all open source! • Build a prototype, find out your needs and how Neo4j matches • AGPL this stage thisusers should have access to your code - at says all your is your employees / co-workers ๏ Put it in front of users! The license is free for the first server! • Contact Neo Technology sales to get a free single server license • You’ll (probably) not have massive load the first days 10
- 17. Building business applications with Neo4j ๏ Try it out! It’s all open source! • Build a prototype, find out your needs and how Neo4j matches • AGPL this stage thisusers should have access to your code - at says all your is your employees / co-workers ๏ Put it in front of users! The license is free for the first server! • Contact Neo Technology sales to get a free single server license • You’ll (probably) not have massive load the first days ๏ As you grow, Neo4j grows with you! • Aslicenseneeds and revenue increase you can by an advanced your (prices are resonable) 10
- 18. Graphs are all around us A B C D ... 1 17 3.14 3 17.79333333333 2 42 10.11 14 30.33 3 316 6.66 1 2104.56 4 32 9.11 592 0.492432432432 5 Even if this spreadsheet looks like it could be a fit for a RDBMS 2153.175765766 it isn’t: •RDBMSes have problems with ... extending indefinitely on both rows and columns •Formulas and data dependencies would quickly lead to heavy join operations 11
- 19. Graphs are all around us A B C D ... 1 17 3.14 3 = A1 * B1 / C1 2 42 10.11 14 = A2 * B2 / C2 3 316 6.66 1 = A3 * B3 / C3 4 32 9.11 592 = A4 * B4 / C4 5 = SUM(D2:D5) With data dependencies ... the spread sheet turns out to be a graph. 12
- 20. Graphs are all around us A B C D ... 1 17 3.14 3 = A1 * B1 / C1 2 42 10.11 14 = A2 * B2 / C2 3 316 6.66 1 = A3 * B3 / C3 4 32 9.11 592 = A4 * B4 / C4 5 = SUM(D2:D5) With data dependencies ... the spread sheet turns out to be a graph. 12
- 21. Graphs are all around us If we add external data sources the problem becomes even more interesting... 17 3.14 3 = A1 * B1 / C1 42 10.11 14 = A2 * B2 / C2 316 6.66 1 = A3 * B3 / C3 32 9.11 592 = A4 * B4 / C4 = SUM(D2:D5) 13
- 22. Graphs are all around us If we add external data sources the problem becomes even more interesting... 17 3.14 3 = A1 * B1 / C1 42 10.11 14 = A2 * B2 / C2 316 6.66 1 = A3 * B3 / C3 32 9.11 592 = A4 * B4 / C4 = SUM(D2:D5) 13
- 23. The Neo4j Graph data model •Nodes •Relationships bet ween Nodes •Relationships have Labels •Relationships are directed, but traversed at equal speed in both directions •The semantics of the direction is up to the application (LIVES WITH is reflexive, LOVES is not) •Nodes have key-value properties •Relationships have key-value properties 14
- 24. The Neo4j Graph data model •Nodes •Relationships bet ween Nodes •Relationships have Labels •Relationships are directed, but traversed at equal speed in both directions •The semantics of the direction is up to the application (LIVES WITH is reflexive, LOVES is not) •Nodes have key-value properties •Relationships have key-value properties 14
- 25. The Neo4j Graph data model LIVES WITH LOVES OWNS DRIVES •Nodes •Relationships bet ween Nodes •Relationships have Labels •Relationships are directed, but traversed at equal speed in both directions •The semantics of the direction is up to the application (LIVES WITH is reflexive, LOVES is not) •Nodes have key-value properties •Relationships have key-value properties 14
- 26. The Neo4j Graph data model LOVES LIVES WITH LOVES OWNS DRIVES •Nodes •Relationships bet ween Nodes •Relationships have Labels •Relationships are directed, but traversed at equal speed in both directions •The semantics of the direction is up to the application (LIVES WITH is reflexive, LOVES is not) •Nodes have key-value properties •Relationships have key-value properties 14
- 27. The Neo4j Graph data model name: “Mary” LOVES name: “James” age: 35 age: 32 LIVES WITH twitter: “@spam” LOVES OWNS DRIVES •Nodes •Relationships bet ween Nodes •Relationships have Labels brand: “Volvo” •Relationships are directed, but traversed at model: “V70” equal speed in both directions •The semantics of the direction is up to the application (LIVES WITH is reflexive, LOVES is not) •Nodes have key-value properties •Relationships have key-value properties 14
- 28. The Neo4j Graph data model name: “Mary” LOVES name: “James” age: 35 age: 32 LIVES WITH twitter: “@spam” LOVES OWNS item type: “car” DRIVES •Nodes •Relationships bet ween Nodes •Relationships have Labels brand: “Volvo” •Relationships are directed, but traversed at model: “V70” equal speed in both directions •The semantics of the direction is up to the application (LIVES WITH is reflexive, LOVES is not) •Nodes have key-value properties •Relationships have key-value properties 14
- 29. Graphs are Whiteboard Friendly The domain I specify is the domain I implement. No mismatch, no ER-modeling. 15
- 30. Graphs are Whiteboard Friendly The domain I specify is the domain I implement. odin No mismatch, no ER-modeling. thobe dude #17 #6 #14 Wardrobe Strength Joe project #32 Call site caching Hello world OSCON Best panncakes Optimizing Jython 15
- 31. Graphs are Whiteboard Friendly odin dude #17 #6 #14 username: “thobe” Wardrobe Strength Joe project #32 name: “Tobias Ivarsson” twitter: “@thobe” password: “**********” Call site caching Hello world OSCON Best panncakes Optimizing Jython 16
- 32. Graphs are Whiteboard Friendly odin thobe dude #17 #6 #14 address: “http://journal.thobe.org” #32 title: “Wardrobe Strengthproject Joe ” tagline: “Good enough thoughts” Call site caching Hello world OSCON Best panncakes Optimizing Jython 17
- 33. Building a graph - the basic API import neo4j grapDb = neo4j.GraphDatabase( PATH_TO_YOUR_NEO4J_DATASTORE ) with graphDb.transaction: # All writes require transactions # Create Thomas 'Neo' Anderson mrAnderson = graphDb.node(name="Thomas Anderson", age=29) # Create Morpheus morpheus = graphDb.node(name="Morpheus", rank= "Captain", occupation= "Total bad ass") # Create relationship representing they know each other mrAnderson.KNOWS( morpheus ) # ... similarly for Trinity, Cypher, Agent Smith, Architect 18
- 34. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” 19
- 35. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] order = neo4j.BREADTH_FIRST stop = neo4j.STOP_AT_END_OF_GRAPH returnable = neo4j.RETURN_ALL_BUT_START_NODE 20
- 36. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] order = neo4j.BREADTH_FIRST stop = neo4j.STOP_AT_END_OF_GRAPH returnable = neo4j.RETURN_ALL_BUT_START_NODE for friend_node in Friends(mr_anderson): print "%s (@ depth=%s)" % ( friend_node["name"], friend_node.depth ) 20
- 37. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] order = neo4j.BREADTH_FIRST stop = neo4j.STOP_AT_END_OF_GRAPH returnable = neo4j.RETURN_ALL_BUT_START_NODE for friend_node in Friends(mr_anderson): print "%s (@ depth=%s)" % ( friend_node["name"], friend_node.depth ) 20
- 38. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1) order = neo4j.BREADTH_FIRST stop = neo4j.STOP_AT_END_OF_GRAPH returnable = neo4j.RETURN_ALL_BUT_START_NODE for friend_node in Friends(mr_anderson): print "%s (@ depth=%s)" % ( friend_node["name"], friend_node.depth ) 20
- 39. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1) order = neo4j.BREADTH_FIRST Trinity (@ depth=1) stop = neo4j.STOP_AT_END_OF_GRAPH returnable = neo4j.RETURN_ALL_BUT_START_NODE for friend_node in Friends(mr_anderson): print "%s (@ depth=%s)" % ( friend_node["name"], friend_node.depth ) 20
- 40. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1) order = neo4j.BREADTH_FIRST Trinity (@ depth=1) stop = neo4j.STOP_AT_END_OF_GRAPH Cypher (@ depth=2) returnable = neo4j.RETURN_ALL_BUT_START_NODE for friend_node in Friends(mr_anderson): print "%s (@ depth=%s)" % ( friend_node["name"], friend_node.depth ) 20
- 41. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1) order = neo4j.BREADTH_FIRST Trinity (@ depth=1) stop = neo4j.STOP_AT_END_OF_GRAPH Cypher (@ depth=2) returnable = neo4j.RETURN_ALL_BUT_START_NODE Agent Smith (@ depth=3) for friend_node in Friends(mr_anderson): print "%s (@ depth=%s)" % ( friend_node["name"], friend_node.depth ) 20
- 42. Graph traversals name: “The Architect” disclosure: “public” name: “Thomas Anderson” age: 29 name: “Cypher” last name: “Reagan” KNOWS name: “Morpheus” KNOWS KNOWS rank: “Captain” CODED BY LOVES occupation: “Total badass” KNOWS KNOWS name: “Trinity” disclosure: “secret” name: “Agent Smith” version: “1.0b” since: “meeting the oracle” since: “a year before the movie” language: “C++” cooperates on: “The Nebuchadnezzar” import neo4j class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1) order = neo4j.BREADTH_FIRST Trinity (@ depth=1) stop = neo4j.STOP_AT_END_OF_GRAPH Cypher (@ depth=2) returnable = neo4j.RETURN_ALL_BUT_START_NODE Agent Smith (@ depth=3) for friend_node in Friends(mr_anderson): print "%s (@ depth=%s)" % ( friend_node["name"], friend_node.depth ) 20
- 43. Finding a place to start ๏ Traversals need a Node to start from • QUESTION: How do I find the start Node? • ANSWER:You use an Index ๏ Indexes in Neo4j are different from Indexes in Relational Databases • RDBMSes use them for Joining • Neo4j use them for simple lookup index = graphDb.index["name"] mr_anderson = index["Thomas Anderson"] performTraversalFrom( mrAnderson ) 21
- 44. Indexes in Neo4j ๏ The Graph *is* the main index • Use relationship labels for navigation • Build index structures *in the graph* ‣Search trees, tag clouds, geospatial indexes, et.c. ‣Linked/skip lists or other data structures in the graph ‣We have utility libraries for this ๏ External indexes used *for lookup* • Finding a (number of) points to start traversals from • Major difference from RDBMS that use indexes for everything 22
- 45. Django integration does all of this for you! 23
- 46. Implementing the domain user blog comment entry 24
- 47. from neo4j.model import django_model as models user blog comment entry 25
- 48. from neo4j.model import django_model as models class User(models.NodeModel): username = models.Property(indexed=True) name = models.Property() blog comment blogs = models.Relationship(Blog, type=models.Outgoing.member_of, related_name="users") def __unicode__(self): return self.name entry 25
- 49. from neo4j.model import django_model as models user class Blog(models.NodeModel): identifier = models.Property(indexed=True) comment title = models.Property() def __unicode__(self): return self.title entry 26
- 50. from neo4j.model import django_model as models user class Entry(models.NodeModel): title = models.Property() text = models.Property() blog comment date = models.Property() blog = models.Relationship(Blog, type=models.Outgoing.posted_on, single=True, optional=False, related_name="articles") author = models.Relationship(User, type=models.Outgoing.authored_by, single=True, optional=False, related_name="articles") 27
- 51. models.py from neo4j.model import django_model as models class Blog(models.NodeModel): identifier = models.Property(indexed=True) title = models.Property() The rest of the code for working with the domain class User(models.NodeModel): objects is (mostly) the same username = models.Property(indexed=True) as you are used to in Django. name = models.Property() blogs = models.Relationship(Blog, type=models.Outgoing.member_of, related_name="users") class Entry(models.NodeModel): title = models.Property() text = models.Property() date = models.Property() blog = models.Relationship(Blog, type=models.Outgoing.posted_on, single=True, optional=False, related_name="articles") author = models.Relationship(User, type=models.Outgoing.authored_by, single=True, optional=False, 28 related_name="articles")
- 52. Why not use an O/R mapper? ๏ Model evolution in ORMs is a hard problem • virtually unsupported in most ORM systems ๏ SQL is “compatible” across many RDBMSs • data is still locked in ๏ Each ORM maps object models differently • Moving to another ORM == legacy schema support ‣except your legacy schema is a strange auto-generated one ๏ Object/Graph Mapping is always done the same way • allows you to keep your data through application changes • or share data between multiple implementations 29
- 53. What an ORM doesn’t do ๏Deep traversals ๏Graph algorithms ๏Shortest path(s) ๏Routing ๏etc. 30
- 54. Path exists in social network ๏ Each person has on average 50 friends The performance impact in Neo4j depends only on the degree of each node. in Tobias an RDBMS it depends on the number of entries in the tables involved in the join(s). Emil Johan Peter Database # persons query time Relational database 1 000 2 000 ms Neo4j Graph Database 1 000 2 ms Neo4j Graph Database 1 000 000 2 ms Relational database 1 000 000 way too long... 31
- 55. Path exists in social network ๏ Each person has on average 50 friends The performance impact in Neo4j depends only on the degree of each node. in Tobias an RDBMS it depends on the number of entries in the tables involved in the join(s). Emil Johan Peter Database # persons query time Relational database 1 000 2 000 ms Neo4j Graph Database 1 000 2 ms Neo4j Graph Database 1 000 000 2 ms Relational database 1 000 000 way too long... 31
- 56. On-line real time routing with Neo4j ๏ 20 million Nodes - represents places ๏ 62 million Edges - represents direct roads between places • These edges have a length property, for the length of the road ๏ Average optimal route, 100 separate roads, found in 100ms ๏ Worst case route we could find: • Optimal route is 5500 separate roads • Total length ~770km There’s a difference • Found in less than 3 seconds bet ween least number of hops and least cost. ๏ Uses A* “best first” search 32
- 57. Jython vs. CPython ๏ Neo4j with the Python bindings work in both • Requires no code modification in your code ๏ Neo4j at its core is an Embedded (in-process) database • CPython manages concurrency by forking multiple processes • Jython has full concurrency support in the same JVM • Stand-alone Neo4jon is being worked server-process with (C)Python client ๏ Neo4j has a RESTful interface • There are Python clients • The API differs slightly (no transactions) 33
- 58. Finding out more ๏ http://neo4j.org/ - project website - main place for getting started ‣Contains screen casts, download links, et.c. ‣http://api.neo4j.org/ and http://components.neo4j.org/ ‣Specifically http://components.neo4j.org/neo4j.py/ ‣http://wiki.neo4j.org/ - HowTos, Tutorials, Examples, FAQ, et.c. ‣http://planet.neo4j.org/ - aggregation of blogs about Neo4j ‣http://github.com/neo4j-examples - small example applications ๏ https://lists.neo4j.org/ - community mailing list ๏ http://twitter.com/neo4j/team - follow the Neo4j team ๏ http://neotechnology.com/ - commercial licensing 34
- 59. Helping out! ๏ Neo4j and the Python integration is all Open Source ๏ The Python bindings in particular would benefit from more devs... • Integrate more of the Neo4j components ‣Neo4j Spatial ‣The Graph Algorithms package ‣The Graph Matching component • Trimming off the rough edges in the Django integration • Native client for CPython 35
- 60. Buzzword summary http://neo4j.org/ Semi structured SPARQL AGPLv3 ACID transactions Open Source Object mapping Gremlin Shortest path NOSQL Traversal RESTful Software Transactional Memory Query language whiteboard friendly Beer A* routing Embedded Schema free Scaling to complexity Free Software Polyglot persistence 36