Neo4j Generative AI workshop at GraphSummit London 14 Nov 2023.pdf
1 like•329 views
The document outlines an agenda for a generative AI workshop covering topics such as building a knowledge graph from a GDB dataset, semantic search using vector indexes and graph patterns, generating node embeddings, and developing a marketing campaign application that integrates all the techniques discussed.
![© 2022 Neo4j, Inc. All rights reserved.
Cypher: powerful and expressive query language
30
MATCH (:Person { name:“Dan”} ) -[:LOVES]-> (:Person { name:“Ann”} )
Person
NODE NODE
LABEL PROPERTY
LABEL PROPERTY
CREATE
RELATIONSHIP
name: ‘Ann’
LOVES
Person
name: ‘Dan’](https://image.slidesharecdn.com/neo4jgenaiworkshopatgraphsummitlondon14nov2023-231117111419-507183c4/85/Neo4j-Generative-AI-workshop-at-GraphSummit-London-14-Nov-2023-pdf-30-320.jpg)
![© 2022 Neo4j, Inc. All rights reserved.
Cypher: powerful and expressive query language
31
MARRIED_TO
Person
name: ‘Dan’
MATCH (p:Person { name:“Dan”} ) -[:MARRIED_TO]-> (spouse)
NODE RELATIONSHIP TYPE
LABEL PROPERTY VARIABLE
spouse
NODE
RETURN p.name as husband, spouse
VARIABLE](https://image.slidesharecdn.com/neo4jgenaiworkshopatgraphsummitlondon14nov2023-231117111419-507183c4/85/Neo4j-Generative-AI-workshop-at-GraphSummit-London-14-Nov-2023-pdf-31-320.jpg)
![© 2022 Neo4j, Inc. All rights reserved.
MATCH (boss)-[:MANAGES*0..3]->(sub),
(sub)-[:MANAGES*1..3]->(report)
WHERE boss.name = “John Doe”
RETURN sub.name AS Subordinate,
count(report) AS Total
Easily express complex queries with Cypher
Find all direct reports and how many
people they manage,
up to three levels down
32
Cypher Query
SQL Query](https://image.slidesharecdn.com/neo4jgenaiworkshopatgraphsummitlondon14nov2023-231117111419-507183c4/85/Neo4j-Generative-AI-workshop-at-GraphSummit-London-14-Nov-2023-pdf-32-320.jpg)
Neo4j Generative AI workshop at GraphSummit London 14 Nov 2023.pdf
- 1. Generative AI Workshop GraphSummit London, 14 Nov 2023 Zach Blumenfeld, Kristof Neys
- 2. What We Will Cover 1. Neo4j Introduction: Basics of product (What is our product, Cypher, how it works, etc.) 2. Knowledge Graph Building a. Load GDB b. Explore the graph in Browser 3. Search a. Text Embedding (With OpenAI) b. Vector Index c. Search with Vector Index d. Search with hops (+ graph traversal to search for users, departments, etc.) 4. Q&A Chatbot a. Using Chat completion API in RAG workflow 5. Enhancing AI Inference (Example with Recommendations) a. Introduce GDS b. Add Additional relationships to graph to scale recommendations (embeddings, KNN, other algos?) c. Demonstrate recommendations in semantic search Neo4j Inc. All rights reserved 2023 2
- 3. What We Will Cover 1. Part 0: Introductory comments & Short Demo 2. Part 1: Creating the Knowledge Graph a. Some Theory b. Set-up c. Dataset intro & Data ingestion 3. Part 2: Vector search, Semantic Search with Contex a. Some Theory b. LLM integrations & Vector indexes Coffee Break c. Vector Search & Graph patterns 4. Part 3: Graph Data Science a. Some Theory b. Creating Node embeddings c. Retrieval Augmented Generation 5. Part 4: Putting it all together: Marketing Campaign App Neo4j Inc. All rights reserved 2023 3
- 4. Neo4j Inc. All rights reserved 2023 4 Part 0: The opportunity of GenAI
- 5. 5 $6.6 Trillion In Economic Value Generative AI is Predicted to Unlock Up to 3.3% productivity improvement annually
- 6. Neo4j Inc. All rights reserved 2023 6 Automate data retrieval tasks 1 Improve customer service experiences Expedite reading, understanding, & summarizing 2 3 Generate content & code 4 Generative AI Provides Amazing Opportunities
- 7. Neo4j Inc. All rights reserved 2023 7 LLMs Core function…To manipulate strings… • GPT training focuses primarily on predicting the probability of text sequences and specifically for predicting which words come next in a sequence “When you play the game of thrones…. Input LLM …you win or you die. There is no middle ground.” Output
- 8. Neo4j Inc. All rights reserved 2023 8 Nothing is perfect… but don’t push it!
- 10. There is no need for this…! Neo4j Inc. All rights reserved 2023 10
- 11. Whereas we all know what The Truth is… Neo4j Inc. All rights reserved 2023 11
- 12. Lack of enterprise domain knowledge Inability to verify or explain answers Hallucination Fine-tuning limitations Sensitive to prompt (input) phrasing Ethical & data bias concerns 12 But There Are Challenges… Neo4j Inc. All rights reserved 2023
- 13. 13 Knowledge Facts Context Language Statistics Creativity KGs LLMs + BETTER TOGETHER Solution: Generative AI, LLMs, and Knowledge Graphs Neo4j Inc. All rights reserved 2023
- 14. © 2023 Neo4j, Inc. All rights reserved. 14 Setting the context - From a user point of view
- 15. © 2023 Neo4j, Inc. All rights reserved. 15 We are not making this up…
- 16. © 2023 Neo4j, Inc. All rights reserved. 16 Three main ways to let LLM’s work with your Graph
- 17. Neo4j Inc. All rights reserved 2023 17 Part 1: Knowledge Graph
- 18. Neo4j Inc. All rights reserved 2023 18 A Knowledge Graph is a structured representation of facts, consisting of entities, relationships and semantic descriptions ● Entities can be real-world objects and abstract concepts ● Relationships represent the connections between entities ● Semantic description of the entities and relationships
- 19. Neo4j Inc. All rights reserved 2023 Nodes represent entities in the graph 19 Person Person Car Knowledge Graph Components
- 20. Neo4j Inc. All rights reserved 2023 Nodes represent entities in the graph Relationships represent associations or interactions between nodes 20 Person Person Car LOVES KNOWS KNOWS LIVES WITH O W N S D R I V E S Knowledge Graph Components
- 21. Neo4j Inc. All rights reserved 2023 Person Person Name: “Andre” Born: May 29, 1970 Twitter: “@dan” Name: “Mica” Born: Dec 5, 1975 Car Brand “Volvo” Model: “V70” Description: Since: Jan 10, 2011 LOVES KNOWS KNOWS LIVES WITH O W N S D R I V E S Nodes represent entities in the graph Relationships represent associations or interactions between nodes Properties represent attributes of nodes or relationships 21 Knowledge Graph Components
- 22. © 2023 Neo4j, Inc. All rights reserved. User :VISITED Website User IPLocation Website IPLocation Website Website Website :VISITED :VISITED :VISITED :USED :USED : U S E D : V I S I T E D : V I S I T E D :VISITED :SAME_AS Graphs allows you to make implicit relationships…. ….explicit And they Grow too…?! 22
- 23. © 2023 Neo4j, Inc. All rights reserved. :SAME_AS User :VISITED Website User IPLocation Website IPLocation Website Website Website :VISITED :VISITED :VISITED :USED :USED : U S E D : V I S I T E D : V I S I T E D :VISITED User :SAM E_AS :USED :VISITED PersonId: 1 PersonId: 1 PersonId: 1 User PersonId: 2 :VISITED …and can then group similar nodes…and create a new graph from the explicit relationships… A graph grows organically - gaining insights and enriching your data Graphs….Grow! 23
- 24. © 2023 Neo4j, Inc. All rights reserved. Come and see us for the book on Knowledge Graphs 24
- 25. Neo4j Inc. All rights reserved 2023 25 What We Will Build Today
- 26. Improve GenAI Using Knowledge Graphs with Retrieval Augmented Generation (RAG) Neo4j Inc. All rights reserved 2023 26
- 27. RAG enables natural language search on factual information retrieved from a database Prompt + Relevant Information LLM API Embedding and/or Natural Language Query LLM Completion API User Database Search Prompt Response Relevant Results / Documents 2 3 1 Knowledge Graph Retrieval Augmented Generation Neo4j Inc. All rights reserved 2023 27
- 28. Today we Build a Targeted, Personalized, Content Generator Neo4j Inc. All rights reserved 2023 28
- 29. Neo4j Inc. All rights reserved 2023 29 Some Cypher Basics
- 30. © 2022 Neo4j, Inc. All rights reserved. Cypher: powerful and expressive query language 30 MATCH (:Person { name:“Dan”} ) -[:LOVES]-> (:Person { name:“Ann”} ) Person NODE NODE LABEL PROPERTY LABEL PROPERTY CREATE RELATIONSHIP name: ‘Ann’ LOVES Person name: ‘Dan’
- 31. © 2022 Neo4j, Inc. All rights reserved. Cypher: powerful and expressive query language 31 MARRIED_TO Person name: ‘Dan’ MATCH (p:Person { name:“Dan”} ) -[:MARRIED_TO]-> (spouse) NODE RELATIONSHIP TYPE LABEL PROPERTY VARIABLE spouse NODE RETURN p.name as husband, spouse VARIABLE
- 32. © 2022 Neo4j, Inc. All rights reserved. MATCH (boss)-[:MANAGES*0..3]->(sub), (sub)-[:MANAGES*1..3]->(report) WHERE boss.name = “John Doe” RETURN sub.name AS Subordinate, count(report) AS Total Easily express complex queries with Cypher Find all direct reports and how many people they manage, up to three levels down 32 Cypher Query SQL Query
- 33. Enough Theory - some practice… part 1 Neo4j Inc. All rights reserved 2023 33
- 34. Neo4j Inc. All rights reserved 2023 34 Back to Theory… Semantic Search
- 35. Neo4j and Vector Search Neo4j Inc. All rights reserved 2023 35 Find relevant documents and content for user queries Find entities associated to content and patterns in connected data. Improve search relevance & insights by enhancing a Knowledge Graph. Use graph algorithms and ML to discover new relationships, entities, and groups. Vector Similarity Search Graph Traversals & Pattern Matching Knowledge Graph Inference & ML Vector Search Graph Database
- 36. What are vector embeddings Neo4j Inc. All rights reserved 2023 36 ● Same concepts, just “an arrow” ● 100s or 1000s dimensions
- 37. Finding Similar vectors Neo4j Inc. All rights reserved 2023 37 ● cosine ● direction / angle based vector point query nearest 4 ● Euclidean ● distance based
- 38. Why & What is a Vector Index? ● Data applied on: encoding vectors of mainly unstructured data such as text, audio, video that is converted using embedding models (“Raw” vectors). ● Main purpose: deploy approximate methods to perform similarity search at lower computational cost. ● Once an embedding vector has been created as a node property a vector index can be created across those properties. ● This indexing is an algorithm that maps the original vector to a data structure that enables faster search. ● By creating a vector index a data structure optimized for queries is created at “store time” (as opposed to GDS similarity search at query time). Neo4j Inc. All rights reserved 2023 38
- 39. Type of Algorithms Some of the algorithms that can be applied are: ● Random projection ● Product Quantization ● Locality-sensitive hashing ● Hierarchical Navigable Small World (HNSW) Neo4j Inc. All rights reserved 2023 39
- 40. How is search performed? Neo4j Inc. All rights reserved 2023 40 ● The Query vector is any piece of unstructured data that is being converted to an encoding vector (the “Raw” vector) and is mapped to an index using the same Algorithm (i.e. Hierarchical Navigable Small World). ● The “Key” vectors are the stored vectors that have been indexed. ● When search is performed between the query vector and the stored vectors a similarity function is applied. ● Several similarity measures can be used, including: ○ Cosine similarity ○ Euclidean similarity ○ Dot product
- 41. • GenAI orchestration Framework • Many out-of-the-box components and integrations • Prebuilt chains and agents LangChain why & how
- 42. Enough Theory - some practice… part 2 Neo4j Inc. All rights reserved 2023 42
- 43. Neo4j Inc. All rights reserved 2023 43 Back to Theory… Graph Data Science to enhance your Knowledge Graph
- 44. Neo4j Graph Data Science Algorithms, Procedures & ML Graph Projections Neo4j Database Graph DS/ML Workspace highly optimized, massively parallel, scalable ● Run graph algorithms to generate insights: 65+ algorithms across centrality, path finding, community detection, similarity, and more ● Engineer graph features for ML: Leverage relationship information with algorithms & node embeddings ● Build graph native ML pipelines: Link prediction, node classification & property regression ● Integrate with external ML frameworks: Python client, blazing fast import & export, formatting for dataframes and tensors Neo4j Inc. All rights reserved 2023 44
- 45. © 2022 Neo4j, Inc. All rights reserved. 45 Graph Data Science Use unsupervised machine learning techniques to identify associations, anomalies, and trends. Use embeddings to learn the features in your graph that you don’t even know are important yet. Train in-graph supervised ML models to predict links, labels, and missing data. Find the patterns you’re looking for in connected data Knowledge Graphs Graph Algorithms Graph Native Machine Learning
- 46. © 2022 Neo4j, Inc. All rights reserved. Similarity • K-Nearest Neighbors (KNN) • Node Similarity • Filtered KNN & Node Similarity • Cosine & Pearson Similarity Functions • Euclidean Distance Similarity Function • Euclidean Similarity Function • Jaccard & Overlap Similarity Functions Graph Embeddings • Fast Random Projection (FastRP) • FastRP with Property Weights • GraphSAGE • Node2Vec • HashGNN Machine Learning • Knowledge Graph Embeddings • Link Prediction Pipelines • Node Classification Pipelines • Node Regression Pipelines The Largest Library Of Graph Algorithms & ML Community Detection • Weakly Connected Components • Conductance Metric • K-1 Coloring • K-Means Clustering • Label Propagation • Leiden Algorithm • Local Clustering Coefficient • Louvain Algorithm • Max K-Cut • Modularity Optimization • Speaker Listener Label Propagation • Strongly Connected Components • Triangle Count Centrality • ArticleRank • Betweenness Centrality & Approx. • Closeness Centrality • Degree Centrality • Eigenvector Centrality • Harmonic Centrality • Hyperlink Induced Topic Search (HITS) • Influence Maximization (CELF) • PageRank • Personalized PageRank Pathfinding & Search • A* Shortest Path • All Pairs Shortest Path • Breadth & Depth First Search • Delta-Stepping Single-Source • Dijkstra Single-Source • Dijkstra Source-Target • Minimum Spanning Tree & K-Spanning Tree • Random Walk • Yen’s K Shortest Path • Minimum Directed Steiner Tree • Topological Sort • Longest Path Link Prediction • Adamic Adar • Common Neighbors • Preferential Attachment • Resource Allocations • Same Community • Total Neighbors … and more! • Collapse Paths • Graph Sampling • Graph Stratified Sampling • One Hot Encoding • Pregel API (write your own algos) • Property Scaling • Split Relationships • Synthetic Graph Generation 46
- 47. © 2022 Neo4j, Inc. All rights reserved. The Power of Graph Embeddings The ‘bridge’ to traditional Machine Learning 47
- 48. © 2022 Neo4j, Inc. All rights reserved. The representation of nodes as low-dimensional vectors that summarize their graph position, the structure of their local graph neighborhood as well as any possible node features NODE EMBEDDING 48
- 49. © 2022 Neo4j, Inc. All rights reserved. NODE EMBEDDING 49
- 50. © 2022 Neo4j, Inc. All rights reserved. Node Embeddings 4 algorithms…and counting ● FastRP (Fast Random Projection) - Calculates embeddings extremely fast using probabilistic sampling and linear algebra. ● GraphSAGE (Graph SAmple and aggreGatE) - Trains a Graph Neural Network (GNN) to generate embeddings on old and new graph data. Uses batch sampling procedures for scalability. ● Node2Vec - Creates embeddings that represent nodes in similar neighborhoods and/or structural “roles” in the graph using adjustable random walks. ● HashGNN - Quickly generates embeddings on heterogeneous graphs. Like a GNN but much faster and simpler with comparable benchmarked performance. Leverages a clever application of hashing functions rather than training a model. New 50
- 51. Enough Theory - some practice… part 3 Neo4j Inc. All rights reserved 2023 51
- 52. Neo4j Inc. All rights reserved 2023 52 Generating Grounded Content with Knowledge Graph and LLM
- 53. Improve GenAI Using Knowledge Graphs with Retrieval Augmented Generation (RAG) Neo4j Inc. All rights reserved 2023 53
- 54. RAG enables natural language search on factual information retrieved from a database Prompt + Relevant Information LLM API Embedding and/or Natural Language Query LLM Completion API User Database Search Prompt Response Relevant Results / Documents 2 3 1 Knowledge Graph Retrieval Augmented Generation Neo4j Inc. All rights reserved 2023 54
- 55. Run the App! Neo4j Inc. All rights reserved 2023 55
- 56. Neo4j Inc. All rights reserved 2023 56 Github Repo: https://github.com/neo4j-product-examples/genai-workshop/tree/main Resources / Education: Graph academy: https://graphacademy.neo4j.com/ Resources: https://neo4j.com/resources/ Books: https://neo4j.com/books/ Video's: https://neo4j.com/videos/ Blog: https://neo4j.com/blog/ Community: https://community.neo4j.com/ Book on Knowledge Graphs: https://neo4j.com/books/knowledge-graphs-practitioners-guide/ Neo4j Sandbox: https://sandbox.neo4j.com/ Resources & Material
- 57. Neo4j Inc. All rights reserved 2023 57 The End…