The Inside Scoop on Neo4j: Meet the Builders

© 2022 Neo4j, Inc. All rights reserved.
The Inside Scoop on Neo4j:
Meet the Builders
Stu Moore,
Product Manager for Neo4j Database
YOUR LOGO HERE

Inside Scoop on Neo4j
New ways to query the graph
• Node patterns [Gustav]
• Querying Fabric [Tobias]
Finding information faster
• Relationship indexes [Linnea]
Overcoming bottlenecks
• Relationship chain locks
[Valdemar]
2
Meet the Builders:

3
Gustav Hendergran
Core Database Engineer

4
Node pattern predicates
What is it?
- More advanced pattern matching
- A syntactic alternative for Cypher node patterns
Where is it used?
- MATCH and inside pattern comprehension
When to use it?
- As expressions become more complex
When can't you use it?
- MATCH (a:Person)-->(b:Person) WHERE a.age >
b.age
Improved
Node
Pattern
Matching
in Neo4j 4.4

5
Cypher in Neo4j 4.4
MATCH (n:Person WHERE n.age < 2)
RETURN n
MATCH (a:Person)-->(b:Person)
WHERE a.age > b.age
MATCH (n:Person)
WHERE n.age < 2
RETURN n
MATCH (a:Person)-->(b:Person)
WHERE a.age > b.age
Cypher before Neo4j 4.4

neo4j@neo4j> MATCH (a:Person {name: 'Keanu Reeves'})
RETURN [(a)-->(b:Movie)<--(c:Person)
WHERE b.released > 1997
AND c.name CONTAINS "Nicholson" | b.title]
AS movies;
+----------------------------+
| movies |
+----------------------------+
| ["Something's Gotta Give"] |
+----------------------------+
8
Pattern comprehension before Neo4j 4.4

neo4j@neo4j> MATCH (a:Person {name: 'Keanu Reeves'})
RETURN [(a)-->(b:Movie WHERE b.released > 1997)<--
(c:Person WHERE c.name CONTAINS "Nicholson") | b.title]
AS movies;
+----------------------------+
| movies |
+----------------------------+
| ["Something's Gotta Give"] |
+----------------------------+
9
Pattern comprehension in Neo4j 4.4

10
with Fabric
Tobias Johansson

11
Why Fabric? A way to…
Unlock more business value
• Query across multiple graphs -> FEDERATION
Operate at scale
• Virtually unlimited horizontal scale
• Increase performance w/ vertical scale
• Improved operations
◦ Manage smaller data sets
◦ Backup / restore
• SHARD TB data into 100s GB
Hybrid Cloud queries
• From Neo4j query across Aura & Neo4j*
What is it?
• Execute queries
in parallel across
databases
• Chain queries for
sophisticated
real-time analysis
• Query a
database
composed of
other databases

12
cards
shard01 shard02
"Fabric" Database
loans
shard03
Compose a graph from databases across clusters
Cluster01 Cluster02

13
Fabric a social example
TBD: Set the scene for your social fabric example
Suggest a data model showing the people and the forums / posts and any info
you are going to show in the example etc

14
people posts1
social
posts2
Compose a graph from databases across clusters
cluster
Fabric
database

neo4j@social> CALL {
USE social.people
MATCH (p:Person {id: 3})
RETURN p.id AS pid
}
CALL {
USE social.posts1
WITH pid
MATCH (p:Post {pid: pid})
RETURN p.title AS title
}
RETURN title;
15
Querying Fabric
Returns
+----------+
| title |
+----------+
| "Post 2" |
| "Post 3" |
+----------+

neo4j@social> CALL {
USE social.people
MATCH (p:Person {id: 1})-
[:FOLLOWS]->(f)
RETURN f.id AS pid, f.name AS name
}
UNWIND social.graphIds() AS gid
CALL {
USE social.graph(gid)
WITH pid
MATCH (p:Post {pid: pid})
RETURN p.title AS title
}
RETURN gid, name, title;
16
Querying Fabric
Returns
+---------------------------+
| gid | name | title |
+---------------------------+
| 1 | "Daniel" | "Post 4" |
| 2 | "Daniel" | "Post 2" |
| 1 | "Daniel" | "Post 5" |
| 1 | "Daniel" | "Post 6" |
| 1 | "Cindy" | "Post 2" |
| 2 | "Cindy" | "Post 1" |
| 1 | "Cindy" | "Post 3" |
+---------------------------+

neo4j@social> UNWIND social.graphIds() AS gid
CALL {
USE social.graph(gid)
MATCH (p:Post)
RETURN p.pid AS pid,
p.text AS text,
size(p.text) AS length
ORDER BY length DESC
LIMIT 1
}
RETURN *
ORDER BY length DESC
LIMIT 1;
17
Querying Fabric
Returns
+------------------------------
| gid | length | pid | text
+------------------------------
| 2 | 86 | 4 | "A great
+------------------------------

18
Finding information faster
Linnéa Andersson,

What it is
19
For Cypher users and Developers
looking for a significant performance
gain when using MATCH on
relationships by type and property
Similar to indexes for properties
associated to nodes, Neo4j 4.3+
provides native index capabilities for
properties associated to
relationships.
Relationship Type &
Property Indexes

What it is
20
For DBAs and architects who need to
optimize performance with indexes.
Neo4j 4.3 provided a new index for
node label and relationship types.
Created by default, when a database
is created with empty stores.
When stores are imported in a
database at creation, existing indexes
for labels and types are acquired,
administrators can modify them by
using CREATE/DROP INDEX commands.
Node Label &
Relationship Type lookup
indexes

21
Goal: find all the twins in the
database (as quickly as possible)
Relationship type indexes

neo4j@neo4j> MATCH (me:Person)-[:TWIN_OF]->(twin:Person) RETURN *
+-------------------------------------+-------------------------------+------+
+-------------------------------------+-------------------------------+------+
| +ProduceResults@neo4j | me, twin | 3981 |
| | +-------------------------------+------+
| +Filter@neo4j | me:Person AND twin:Person | 3981 |
| | +-------------------------------+------+
| +DirectedRelationshipTypeScan@neo4j | (me)-[anon_0:TWIN_OF]->(twin) | 3981 |
+-------------------------------------+-------------------------------+------+
23
To find all twins in a Neo4j 4.2 database

24
Goal: find all parents on parental
leave in the database (as quickly
as possible)
Relationship property indexes

neo4j@neo4j> MATCH (parent:Person)-[r:PARENT_OF]->(child:Person)
WHERE r.on_parental_leave=true return *;
+------------------------+---------------------------------------------+--------+
+------------------------+---------------------------------------------+--------+
| +ProduceResults@neo4j | parent, child | 84 |
| | +---------------------------------------------+--------+
| +Filter@neo4j | r.on_parental_leave = true AND child:Person | 84 |
| | +---------------------------------------------+--------+
| +Expand(All)@neo4j | (parent)-[r:PARENT_OF]->(child) | 102000 |
| | +---------------------------------------------+--------+
| +NodeByLabelScan@neo4j | parent:Person | 204872 |
+------------------------+---------------------------------------------+--------+
25
Goal find all parents on parental leave in Neo4j 4.2

neo4j@neo4j> CREATE INDEX parental_leave IF NOT EXISTS FOR ()-[r:PARENT_OF]-()
ON ( r.on_parental_leave );
neo4j@neo4j> MATCH (parent:Person)-[r:PARENT_OF]->(child:Person)
WHERE r.on_parental_leave=true return *;
+-----------------------------+---------------------------------------------------------+-----+
| Operator | Details |Rows |
+-----------------------------+---------------------------------------------------------+-----+
| +ProduceResults@neo4j | parent, child | 84 |
| | +---------------------------------------------------------+-----+
| +Filter@neo4j | parent:Person AND child:Person | 84 |
| | +---------------------------------------------------------+-----+
| +DirectedRelationshipIndexS*| BTREE INDEX (parent)-[r:PARENT_OF(on_parental_leave)]-> | 84 |
| | (child) WHERE on_parental_leave = true | |
+-----------------------------+---------------------------------------------------------+-----+
*Abbreviated from DirectedRelationshipIndexScan@neo4j
26
Find all the parents on leave in Neo4j 4.3+

Relationship type/property planning
27
Relationship must
have type specified.
MATCH (me)-[knows:KNOWS]->(friend)-
[married:MARRIED_TO]->(friendSpouse)
:TYPE
MATCH (me)-[r]->(friend)-[m]->(friendSpouse)
The following wont work

WHERE knows.since IS NOT NULL
WHERE knows.since = 2010
WHERE knows.since IN [2010, 2020]
WHERE knows.since > 2010
WHERE 2000 < knows.since < 2010
WHERE married.witnessed_by STARTS WITH “Dr”
WHERE distance(knows.met_first, point({x:1,y:2})) < 2
Relationship type/property planning
28
• Composite
indexes are
supported.
• Can solve all
the same
predicates as
node indexes
WHERE married.witnessed_by ENDS WITH “Who”
WHERE married.witnessed_by CONTAINS “Who”
Following supported by "lucene+btree-3.0" index provider only
Follow with any one of the following WHERE statements

29
Overcoming bottlenecks in the
model
How Neo4j solves for dense nodes, so you don't
have to redesign your model

30
Dense node (un)locking
in Neo4j 4.3
Valdemar Roxling,

...
Bob
Bob
Bob
Bob
Bob
Dense nodes
Logical representation
GraphConnect Valdemar
SPEAKS_AT
Bob Bob
Bob
Bob
Bob
Bob
Alice

...
Bob
Bob
Bob
Bob
Bob
Dense nodes
Logical representation
GraphConnect Valdemar
SPEAKS_AT
Bob Bob
Bob
Bob
Bob
Bob
Alice
Node store Relationship Group store Relationship store
Graph
Connect
SPEAKS_AT
ATTENDS_TO
R1
R2 R3 Rx
... Ry
Physical representation (seen from dense node)

Locking problem
Relationships additions/removals prevents any other concurrent changes to either affected node
GraphConnect Bob
TX1

Locking problem
GraphConnect Bob
TX1
Holds

Locking problem
GraphConnect Bob
Alice
TX1
TX2
Holds

Granular locking
Only lock what is necessary to preserve data integrity
R1 R2 R3 R4 R5 R6 R7

Granular locking
R1 R2 R3 R4 R5 R6 R7
TX1

Granular locking
R1 R2 R3 R4 R5 R6 R7
TX1 TX2

Granular locking
R1 R2 R3 R4
R1 R2 Rtx1 R3 R4
R5 R6 R7
TX1 TX2
R5 Rtx2 R6 R7
TX1
TX2

Concurrent modifications made total throughput
lower. Why?
41

Deadlocks
More granular locks increases the risk of locks taken “out of order”
Transaction 1
Transaction 2

Deadlocks
Transaction 1
1
Transaction 2

Deadlocks
Transaction 1
1
Transaction 2
2

Deadlocks
Transaction 1
1 2
Transaction 2
2
Wait
for

Deadlocks
Transaction 1
1 2
Transaction 2
1
2
Wait
for
Deadlock!

Sorting locks
Taking locks in specified order eliminates deadlocks
Transaction
db.beginTx()

Sorting locks
Transaction
db.beginTx()
denseNode.createRelationshipTo(node)
4
5

Sorting locks
Transaction
db.beginTx()
denseNode.createRelationshipTo(otherNode)
4
5
2

Sorting locks
Transaction
db.beginTx()
4
5
2
1
doWork()
3

Sorting locks
Transaction
db.beginTx()
tx.commit()
Commit
1
2 3
4
5
doWork()

Sorting locks
Transaction
db.beginTx()
tx.commit()
Commit
1
2
3
4
5
doWork()

Summary
• Granular locking on dense nodes
◦ Allows concurrent additions/deletions
• Reworked locking scheme with sorted locks
◦ Greatly reduces risk of unexpected deadlocks
◦ Holds locks for a shorter duration

54
Thank you!
Contact us at
sales@neo4j.com
YOUR LOGO HERE

The Inside Scoop on Neo4j: Meet the Builders

More Related Content

Similar to The Inside Scoop on Neo4j: Meet the Builders (20)

More from Neo4j (20)

Recently uploaded (20)

The Inside Scoop on Neo4j: Meet the Builders

Editor's Notes