New Data Technologies, Graph Computing and Relationship Discovery in the Enterprise - Darren
5 likes•867 views
This document discusses the unique attributes and advantages of graph databases, particularly how they optimize data relationships and facilitate complex queries and navigations. It highlights practical applications such as pathfinding in intelligence and financial analytics, emphasizing the need for high performance and distributed processing. The text also compares traditional SQL approaches with graph structures, illustrating the benefits of a graph-oriented data model.
![Thankyou ! Copyright © InfiniteGraph [email_address] Twitter - @infinitegraph](https://image.slidesharecdn.com/darren-101013180747-phpapp01/85/New-Data-Technologies-Graph-Computing-and-Relationship-Discovery-in-the-Enterprise-Darren-20-320.jpg)
New Data Technologies, Graph Computing and Relationship Discovery in the Enterprise - Darren
- 1. Graph Databases – What’s so Different ? Darren Wood Chief Architect, InfiniteGraph
- 2. Graph Databases Key technical attributes How Infinite Graph addresses these Query and navigation Challenges/Requirements of Distribution Practical applications Copyright © InfiniteGraph
- 3. Graph Databases Optimized around data relationships Relationships as first class citizens Super fast traversal between entities Rich/flexible annotation of connections Small focused API (typically not SQL) Natively work with concepts of Vertex/Edge SQL has no concept of “navigation” Most attempts based in SQL are convoluted Copyright © InfiniteGraph
- 4. Some Code Copyright © InfiniteGraph Vertex alice = myGraph.addVertex(new Person(“Alice”)); Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Alice Carlos Charlie Bob Meets Calls Pays Calls
- 5. Physical Storage Comparison Copyright © InfiniteGraph Meetings P1 Place Time P2 Alice Denver 5-27-10 Bob Calls From Time Duration To Bob 13:20 25 Carlos Bob 17:10 15 Charlie Payments From Date Amount To Carlos 5-12-10 100000 Charlie Met 5-27-10 Alice Called 13:20 Bob Payed 100000 Carlos Charlie Called 17:10 Rows/Columns/Tables Relationship/Graph Optimized
- 6. Query and Navigation Queries – but not as you know them More like a rules based search and discovery Asynchronous Results Copyright © InfiniteGraph Alice Carlos Charlie Bob Meets Calls Pays Calls “ Find all paths between Alice and Charlie” “ Find all paths between Alice and Charlie – within 2 degrees” “ Find all paths between Alice and Charlie – events in May 2010”
- 7. Navigation Example Copyright © InfiniteGraph // Create a qualifier that describes the target vertex Qualifier findCharliePredicate = new VertexPredicate(personType, "name == ’Charlie'" ); // Construct a navigator which starts with Alice and uses a result qualifier // to find all paths in the graph to Charlie Navigator charlieFinder = alice.navigate( Guide.SIMPLE_BREADTH_FIRST, // default guide Qualifier.ANY, // no path constraints findCharliePredicate , // find paths ending with Charlie myResultHandler); // fire results to supplied handler // Start the navigator charlieFinder.start();
- 8. Management of Large Data Graphs Graphs grow quickly Billions of phone calls / day in US Emails, social media events, IP Traffic Financial transactions Some analytics require navigation of large sections of the graph Each step (often) depends on the last Must distribute data and go parallel Copyright © InfiniteGraph
- 9. Distributing Navigation Graph algorithms naturally branch Can be automated or guided Copyright © InfiniteGraph Alice Carlos Charlie Bob Meets Calls Pays Dave Eve Chuck Calls Lives With Meets
- 10. Partitioning – Typical Big Data Copyright © InfiniteGraph Distributed API Application(s) Partition 1 Partition 3 Partition 2 Partition ... n Processor Processor Processor Processor
- 11. Graph Partitioning Graph partitioning is not as simple Graph operations are rarely partition bound Graphs are ‘alive’ Repartitioning is expensive Partitions must co-operate Copyright © InfiniteGraph
- 12. Graph Partitioning – Reality ! Copyright © InfiniteGraph Distributed API Application(s) Partition 1 Partition 3 Partition 2 Partition ... n Processor Processor Processor Processor
- 13. Distributed Graph Must Haves High performance distributed persistence Ability to deal with remote data reads (fast) Intelligent local cache of subgraphs Distributed navigation processing Distributed, multi-source concurrent ingest Write modes supporting both strict and eventual consistency Copyright © InfiniteGraph
- 14. Practical Applications Copyright © InfiniteGraph
- 15. Pathfinding Intelligence, Police, Counter Terrorism Financial Transactions and Fraud Discover Paths Between Entities One to One, Many to Many Constrain by Edge and Vertex Type Edge and Vertex Attributes (including temporal) Required Sequence of Events Length (Hops), Total Edge Weight Copyright © InfiniteGraph
- 16. Graph Analysis (Algorithms) Copyright © InfiniteGraph Bob Sam Degree Centrality Closeness and Betweeness Centrality Fred Jo
- 17. Graph Analysis (Algorithms) Social Networks Most connected participants Influencers Important Syndicates or Sub-networks Central figures in crime organisations Business Intelligence Discovering Knowledge Assets Complex analytics Copyright © InfiniteGraph
- 18. Graph Analysis (Patterns) Crime (again) Recognize common patterns of activity Complex chains of interaction Security Recognize attack/threat patterns Auditing / log analytics Targeting Advertising To specific browsing patterns Copyright © InfiniteGraph
- 19. Many Many More ! Spatial data Defence / Situational Awareness Sciences Health Care Genealogy Logistics Tracking Copyright © InfiniteGraph
- 20. Thankyou ! Copyright © InfiniteGraph [email_address] Twitter - @infinitegraph