Introducing Neo4j 3.1: New Security and Clustering Architecture
11 likes1,710 views
Neo4j 3.1 introduces enhanced security features and a new clustering architecture designed for high availability, massive throughput, and data redundancy. This version utilizes causal clustering with writeable core and read-only replicas, enabling fault-tolerant consensus commits for data safety. Key improvements also include new language drivers and integration with LDAP for user authentication and authorization.
![Routed read queries
driver = GraphDatabase.driver( "bolt+routing://aCoreServer" );
try ( Session session = driver.session( AccessMode.READ ) )
{
try ( Transaction tx = session.beginTransaction() )
{
tx.run( "MATCH (user:User {userId: {userId}})-[*]-(:Product) RETURN *",
parameters( "userId", userId ) );
tx.success();
}
}](https://image.slidesharecdn.com/3-161027021849/85/Introducing-Neo4j-3-1-New-Security-and-Clustering-Architecture-56-320.jpg)
![neo4j-admin restore --from=<backup-directory> --database=<database-name> [--force]
Restore a backed up database.
neo4j-admin dump
neo4j-admin load
neo4j-admin backup [--from=<address>] --to=<backup-path> [--check-consistency] [--additional-config=<config-file-path>] [--timeout=<timeout>]
Perform a backup, over the network, from a running Neo4j server into a local copy of the database store (the backup).
neo4j-admin check-consistency --database=<database> [--additional-config=<file>] [--verbose]
Check the consistency of a database.
neo4j-admin import --mode={database|csv} --database=<database-name>
Import a collection of CSV files with --mode=csv, or a database from a pre-3.0 installation with --mode=database.](https://image.slidesharecdn.com/3-161027021849/85/Introducing-Neo4j-3-1-New-Security-and-Clustering-Architecture-109-320.jpg)
Introducing Neo4j 3.1: New Security and Clustering Architecture
- 1. Introducing Neo4j 3.1 New Security and Clustering Architecture
- 4. ● ● ● ○ ○ ○
- 5. 3.1
- 6. New Binary Protocol ProceduresNew Language Drivers
- 7. 3.1
- 11. 3.1
- 14. Data Redundancy
- 15. Data Redundancy
- 16. Data Redundancy
- 17. Data Redundancy
- 20. High Availability Error! 503: Service Unavailable
- 21. High Availability Error! 503: Service Unavailable
- 22. High Availability Error! 503: Service Unavailable
- 23. High Availability Error! 503: Service Unavailable
- 24. High Availability ✓ Error! 503: Service Unavailable
- 25. Data RedundancyMassive Throughput High Availability
- 26. Data RedundancyMassive Throughput High Availability 3.0
- 27. Data RedundancyMassive Throughput High Availability 3.0 Bigger Clusters Consensus Commit Built-in load balancing 3.1 Causal Clustering
- 28. Replica Core
- 29. • Small group of Neo4j databases • Fault-tolerant Consensus Commit • Responsible for data safety Core
- 30. Writing to the Core Cluster Neo4j Driver Neo4j Cluster
- 31. Writing to the Core Cluster Neo4j Driver CREATE (:User {...}) ✓ Neo4j Cluster
- 32. Writing to the Core Cluster Neo4j Driver CREATE (:User {...}) ✓ Neo4j Cluster
- 33. Writing to the Core Cluster Neo4j Driver CREATE (:User {...}) ✓ ✓ ✓ Neo4j Cluster
- 34. Writing to the Core Cluster Neo4j Driver CREATE (:User {...}) ✓ ✓ ✓ Neo4j Cluster
- 35. Writing to the Core Cluster Neo4j Driver CREATE (:User {...}) ✓ ✓ ✓ Neo4j Cluster
- 36. Writing to the Core Cluster Neo4j Driver ✓ ✓ ✓ Success Neo4j Cluster
- 37. Writing to the Core Cluster Neo4j Driver ✓ ✓ ✓ Success Neo4j Cluster ✓ ✓
- 38. • Small group of Neo4j databases • Fault-tolerant Consensus Commit • Responsible for data safety Core
- 39. • For massive query throughput • Read-only replicas • Not involved in Consensus Commit • Disposable, suitable for auto-scaling Replica
- 40. Propagating updates to the Replica Neo4j Driver Neo4j Cluster
- 41. Propagating updates to the Replica Neo4j Driver Neo4j Cluster Write
- 42. Propagating updates to the Replica Neo4j Driver Neo4j Cluster Write
- 43. Reading from the Replica Neo4j Driver Neo4j Cluster Read
- 44. Replica Core Updating the graph Queries, analysis, reporting
- 46. :sysinfo
- 47. Writing an application for Neo4j Causal Clustering
- 49. Java <dependency> <groupId>org.neo4j.driver</groupId> <artifactId>neo4j-java-driver</artifactId> </dependency> Python pip install neo4j-driver .NET PM> Install-Package Neo4j.Driver JavaScript npm install neo4j-driver
- 53. GraphDatabase.driver( "bolt+routing://aCoreServer" ) Bootstrap: specify any core server to route load across the whole cluster bolt+routing://
- 55. Routed write statements driver = GraphDatabase.driver( "bolt+routing://aCoreServer" ); try ( Session session = driver.session( AccessMode.WRITE ) ) { try ( Transaction tx = session.beginTransaction() ) { tx.run( "MERGE (user:User {userId: {userId}})", parameters( "userId", userId ) ); tx.success(); } }
- 56. Routed read queries driver = GraphDatabase.driver( "bolt+routing://aCoreServer" ); try ( Session session = driver.session( AccessMode.READ ) ) { try ( Transaction tx = session.beginTransaction() ) { tx.run( "MATCH (user:User {userId: {userId}})-[*]-(:Product) RETURN *", parameters( "userId", userId ) ); tx.success(); } }
- 57. Consistency
- 58. Register Login You need to login in to continue your purchase!
- 59. Register Login You need to login in to continue your purchase! Username: Password: Create Account
- 60. Register Login You need to login in to continue your purchase! Username: jim_w Password: ******** Create Account
- 61. Register Login You need to login in to continue your purchase! Username: Password: Login
- 64. Username: jim_w Password: ******** A few moments later... ✓ Login
- 66. Q Why didn’t this work? A Eventual Consistency
- 67. 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 Create Account App Server A Driver
- 68. 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 CREATE (:User) Create Account App Server A Driver
- 69. 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 CREATE (:User) Create Account App Server A Driver
- 70. 0 1 2 3 4 5 6 7 8 9 10 11CREATE (:User) Create Account App Server A Driver 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
- 71. 0 1 2 3 4 5 6 7 8 9 10 11CREATE (:User) Create Account App Server A Driver 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
- 72. 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 CREATE (:User) Create Account App Server A Driver
- 73. 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 CREATE (:User) Create Account App Server A Driver MATCH (:User) Login App Server B Driver
- 74. 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 CREATE (:User) Create Account App Server A Driver MATCH (:User) Login App Server B Driver
- 75. Bookmark • Session token • String (for portability) • Opaque to application • Represents ultimate user’s most recent view of the graph • More capabilities to come
- 76. Let’s try again, with Causal Consistency
- 77. 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 Create Account App Server A Driver
- 78. 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 CREATE (:User) Create Account App Server A Driver
- 79. 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 CREATE (:User) Create Account App Server A Driver
- 80. 0 1 2 3 4 5 6 7 8 9 10 11CREATE (:User) Create Account App Server A Driver 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
- 81. 0 1 2 3 4 5 6 7 8 9 10 11CREATE (:User) Create Account App Server A Driver 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
- 82. 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 CREATE (:User) Create Account App Server A Driver
- 83. 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 CREATE (:User) Create Account App Server A Driver MATCH (:User) Login App Server B Driver
- 84. 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 CREATE (:User) Create Account MATCH (:User) Login App Server A App Server B Driver Driver
- 85. 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 CREATE (:User) Create Account MATCH (:User) Login App Server A App Server B Driver Driver 11
- 86. 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 CREATE (:User) Create Account MATCH (:User) Login App Server A App Server B Driver Driver 11
- 87. Replica Core Updating the graph Queries, analysis, reporting
- 88. Neo4j 3.0 Neo4j 3.1 High Availability Cluster Causal Cluster Master-Slave architecture Paxos consensus used for master election Raft protocol used for leader election, membership changes and commitment of all transactions Two part cluster: writeable Core and read-only read replicas. Transaction committed once written durably on the master Transaction committed once written durably on a majority of the core members Practical deployments: 10s servers Practical deployments: 100s servers
- 89. 3.1
- 91. • • •
- 92. # Choose LDAP connector as both authentication and authorization provider dbms.security.auth_provider=ldap # Configure LDAP connector to point to the AD server dbms.security.ldap.host=ldap://myactivedirectory.example.com # In case where defined users are not allowed to search for themselves, # we can specify credentials for user with read access to all users and groups dbms.security.ldap.authorization.use_system_account=true dbms.security.ldap.system_username=CN=admin,OU=people,DC=example,DC=com dbms.security.ldap.system_password=admin-password # Provide details on user structure within LDAP dbms.security.ldap.user_dn_template=CN={0},OU=people,DC=example,DC=com dbms.security.ldap.authorization.user_search_base=OU=people,dc=example,dc=com dbms.security.ldap.authorization.user_search_filter=(&(objectClass=*)(CN={0})) dbms.security.ldap.authorization.group_membership_attributes=memberOf ./conf/neo4j.conf
- 93. dbms.security.ldap.authorization.group_to_role_mapping= "CN=Neo4j Read Only,OU=groups,DC=example,DC=com" = reader; "CN=Neo4j Read-Write,OU=groups,DC=example,DC=com" = publisher; "CN=Neo4j Schema Manager,OU=groups,DC=example,DC=com" = architect; "CN=Neo4j Administrator,OU=groups,DC=example,DC=com" = admin; "CN=Neo4j Procedures,OU=groups,DC=example,DC=com" = allowed_role ./conf/neo4j.conf
- 100. # Configure mapping between groups in the LDAP and roles in Neo4j dbms.security.ldap.authorization.group_to_role_mapping= “CN=Neo4j Accounting,OU=groups,DC=example,DC=com” = accounting; “CN=Neo4j Operator,OU=groups,DC=example,DC=com” = operator CALL dbms.security.createRole(‘accounting’) CALL dbms.security.addRoleToUser(‘accounting’, ‘bobsmith’)
- 101. • • • • •
- 102. 3.1
- 105. • • •
- 109. neo4j-admin restore --from=<backup-directory> --database=<database-name> [--force] Restore a backed up database. neo4j-admin dump neo4j-admin load neo4j-admin backup [--from=<address>] --to=<backup-path> [--check-consistency] [--additional-config=<config-file-path>] [--timeout=<timeout>] Perform a backup, over the network, from a running Neo4j server into a local copy of the database store (the backup). neo4j-admin check-consistency --database=<database> [--additional-config=<file>] [--verbose] Check the consistency of a database. neo4j-admin import --mode={database|csv} --database=<database-name> Import a collection of CSV files with --mode=csv, or a database from a pre-3.0 installation with --mode=database.
- 110. ● ○ ● ○ ● ○ ●