SlideShare a Scribd company logo

Natural Language Search with Knowledge Graphs (Chicago Meetup)

•
Trey Grainger, profile picture
Trey Grainger

Trey Grainger, Chief Algorithms Officer at Lucidworks, discusses AI-powered search and natural language processing using knowledge graphs. The document outlines the principles of semantic search, relevance tuning, and query disambiguation, emphasizing the importance of structured and unstructured data in search applications. Additionally, it covers various tools and methodologies for enhancing search experiences and the role of knowledge graphs in improving natural language search capabilities.

Technology
1 of 83
Downloaded 41 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
Trey Grainger Chief Algorithms Officer Natural Language Search with Knowledge Graphs September 25, 2019
Trey Grainger Chief Algorithms Officer • Previously: SVP of Engineering @ Lucidworks; Director of Engineering @ CareerBuilder • Georgia Tech – MBA, Management of Technology • Furman University – BA, Computer Science, Business, & Philosophy • Stanford University – Information Retrieval & Web Search Other fun projects: • Co-author of Solr in Action, plus numerous research publications • Advisor to Presearch, the decentralized search engine • Lucene / Solr contributor About Me
http://aiPoweredSearch.com ... is my new book!
Agenda • About Lucidworks • What is AI-powered Search? • What is Natural Language Search? • What is a Knowledge Graph (and related terminology)? • Philosophy of Language (enough to get the approach…) • Semantic Knowledge Graphs • Knowledge Graph Goals for Natural Language Search • Relevant Tools • Solr Text Tagger • Solr’s Semantic Knowledge Graph • Automated Graph Generation • Demos!
The Search & AI Conference COMPANY BEHIND Who are we? 300+ CUSTOMERS ACROSS THE FORTUNE 1000 400+EMPLOYEES OFFICES IN San Francisco, CA (HQ) Raleigh-Durham, NC Cambridge, UK Bangalore, India Hong Kong Employ about 40% of the active committers on the Solr project 40% Contribute over 70% of Solr's open source codebase 70% DEVELOP & SUPPORT Apache
Industry’s most powerful Intelligent Search & Discovery Platform.
What is ?
AI? Machine Learning? Data Science? Neural Search? AI-Powered Search? Deep Learning?
Natural Language Search with Knowledge Graphs (Chicago Meetup)
Natural Language Search with Knowledge Graphs (Chicago Meetup)
Natural Language Search with Knowledge Graphs (Chicago Meetup)
Natural Language Search with Knowledge Graphs (Chicago Meetup)
AI-powered Search
AI-powered Search Question / Answer Systems Virtual Assistants • Signals Boosting Models • Learning to Rank • Semantic Search • Collaborative Filtering • Personalized Search • Content Clustering • NLP / Entity Resolution • Semantic Knowledge Graphs • Document Classification • etc. • Neural Search • Word Embeddings • Vector Search • Image / Voice Search • etc. • Question / Answer Systems • Virtual Assistants • Chatbots • Rules-based Relevancy • etc.
Proudly built with open-source tech at its core: Apache Solr & Apache Spark Personalizes search with applied machine learning Proven on the world’s biggest information systems
Signals Boosting Popularized Relevancy Learning to Rank Generalized Relevancy Collaborative Filtering Personalized Relevancy
Knowledge Graphs Domain modeling Multi-modal Learning Merged Content Modalities Thought Vectors Conceptual scoring
Basic Keyword Search (inverted index, tf-idf, bm25, multilingual text analysis, query formulation, etc.) Query Intent (query classification, semantic query parsing, knowledge graphs, concept expansion, rules, clustering, classification) Relevancy Tuning (signals, AB testing/genetic algorithms, Learning to Rank, Neural Networks) Self-learning Relevance Engineering Sophistication Context for this Talk Taxonomies / Entity Extraction (entity recognition, basic ontologies, synonyms, etc.)
What is Natural Language Search?
Natural Language Search with Knowledge Graphs (Chicago Meetup)
Natural Language Search with Knowledge Graphs (Chicago Meetup)
What is a Knowledge Graph? (vs. Ontology vs. Taxonomy vs. Synonyms, etc.)
Natural Language Search with Knowledge Graphs (Chicago Meetup)
Overly Simplistic Definitions Alternative Labels: Substitute words with identical meanings [ CTO => Chief Technology Officer; specialise => specialize ] Synonyms List: Provides substitute words that can be used to represent the same or very similar things [ human => homo sapien, mankind; food => sustenance, meal ] Taxonomy: Classifies things into Categories [ john is Human; Human is Mammal; Mammal is Animal ] Ontology: Defines relationships between types of things [ animal eats food; human is animal ] Knowledge Graph: Instantiation of an Ontology (contains the things that are related) [ john is human; john eats food ] In practice, there is significant overlap…
Natural Language Search with Knowledge Graphs (Chicago Meetup)
What sort of Knowledge Graph can help us with the kinds of problems we encounter in Search use cases?
Natural Language Search with Knowledge Graphs (Chicago Meetup)
most often used in reference to “free text”
But… unstructured data is really more like “hyper-structured” data. It is a graph that contains much more structure than typical “structured data.”
Structured Data Employees Table id name company start_date lw100 Trey Grainger 1234 2016-02-01 dis2 Mickey Mouse 9123 1928-11-28 tsla1 Elon Musk 5678 2003-07-01 Companies Table id name start_date 1234 Lucidworks 2016-02-01 5678 Tesla 1928-11-28 9123 Disney 2003-07-01 Discrete Values Continuous Values Foreign Key
Unstructured Data Trey Grainger works at Lucidworks. He is speaking at the Activate 2019 conference. #Activate19 (Activate) is being held in Washington, DC September 9-12, 2019. Trey got his masters from Georgia Tech.
Trey Grainger works for Lucidworks. He is speaking at the Activate 2019 conference. #Activate19 (Activate) is being held in Washington, DC September 9-12, 2019. Trey got his masters degree from Georgia Tech. Trey’s Voicemail Unstructured Data
Trey Grainger works for Lucidworks. He is speaking at the Activate 2019 conference. #Activate19 (Activate) is being held in Washington, DC September 9-12, 2019. Trey got his masters degree from Georgia Tech. Trey’s Voicemail Foreign Key?
Trey Grainger works for Lucidworks. He is speaking at the Activate 2019 conference. #Activate19 (Activate) is being held in Washington, DC September 9-12, 2019. Trey got his masters degree from Georgia Tech. Trey’s Voicemail Fuzzy Foreign Key? (Entity Resolution)
Trey Grainger works for Lucidworks. He is speaking at the Activate 2019 conference. #Activate19 (Activate) is being held in Washington, DC September 9-12, 2019. Trey got his masters degree from Georgia Tech. Trey’s Voicemail Fuzzier Foreign Key? (metadata, latent features)
Trey Grainger works for Lucidworks. He is speaking at the Activate 2019 conference. #Activate19 (Activate) is being held in Washington, DC September 9-12, 2019. Trey got his masters degree from Georgia Tech. Trey’s Voicemail Fuzzier Foreign Key? (metadata, latent features) Not so fast!
Natural Language Search with Knowledge Graphs (Chicago Meetup)
Natural Language Search with Knowledge Graphs (Chicago Meetup)
Giant Graph of Relationships... Trey Grainger works for Lucidworks. He is speaking at the Activate 2019 conference. #Activate19 (Activate) is being held in Washington, DC September 9-12, 2019. Trey got his masters degree from Georgia Tech. Trey’s Voicemail
How do we easily harness this “semantic graph” of relationships within unstructured information?
Semantic Knowledge Graph
id: 1 job_title: Software Engineer desc: software engineer at a great company skills: .Net, C#, java id: 2 job_title: Registered Nurse desc: a registered nurse at hospital doing hard work skills: oncology, phlebotemy id: 3 job_title: Java Developer desc: a software engineer or a java engineer doing work skills: java, scala, hibernate field doc term desc 1 a at company engineer great software 2 a at doing hard hospital nurse registered work 3 a doing engineer java or software work job_title 1 Software Engineer … … … Terms-Docs Inverted IndexDocs-Terms Forward IndexDocuments Source: Trey Grainger, Khalifeh AlJadda, Mohammed Korayem, Andries Smith.“The Semantic Knowledge Graph: A compact, auto-generated model for real-time traversal and ranking of any relationship within a domain”. DSAA 2016. Knowledge Graph field term postings list doc pos desc a 1 4 2 1 3 1, 5 at 1 3 2 4 company 1 6 doing 2 6 3 8 engineer 1 2 3 3, 7 great 1 5 hard 2 7 hospital 2 5 java 3 6 nurse 2 3 or 3 4 registered 2 2 software 1 1 3 2 work 2 10 3 9 job_title java developer 3 1 … … … …
Serves as a “data science toolkit” API that allows dynamically navigating and pivoting through multiple levels of relationships between items in a domain. Semantic Knowledge Graph API Core similarity engine, exposed via API Any product can leverage the core relationship scoring engine to score any list of entities against any other list Full domain support Keywords, categories, tags, based upon any field on your documents. Graph is build automatically from the content representing your domain. Intersections, overlaps, & relationship scoring, many levels deep Users can either provide a list of items to score, or else have the system dynamically discover the most related items (or both). Knowledge Graph
Source: Trey Grainger, Khalifeh AlJadda, Mohammed Korayem, Andries Smith.“The Semantic Knowledge Graph: A compact, auto- generated model for real- time traversal and ranking of any relationship within a domain”. DSAA 2016. Knowledge Graph Set-theory View Graph View How the Graph Traversal Works skill: Java skill: Scala skill: Hibernate skill: Oncology has_related_skill has_related_skill has_related_skill doc 1 doc 2 doc 3 doc 4 doc 5 doc 6 skill: Java skill: Java skill: Scala skill: Hibernate skill: Oncology Data Structure View Java Scala Hibernate docs 1, 2, 6 docs 3, 4 Oncology doc 5
DOI: 10.1109/DSAA.2016.51 Conference: 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA) Source: Trey Grainger, Khalifeh AlJadda, Mohammed Korayem, Andries Smith.“The Semantic Knowledge Graph: A compact, auto-generated model for real-time traversal and ranking of any relationship within a domain”. DSAA 2016. Knowledge Graph Graph Traversal Data Structure View Graph View doc 1 doc 2 doc 3 doc 4 doc 5 doc 6 skill: Java skill: Java skill: Scala skill: Hibernate skill: Oncology doc 1 doc 2 doc 3 doc 4 doc 5 doc 6 job_title: Software Engineer job_title: Data Scientist job_title: Java Developer …… Inverted Index Lookup Forward Index Lookup Forward Index Lookup Inverted Index Lookup Java Java Developer Hibernate Scala Software Engineer Data Scientist has_related_skill has_related_skill has_related_skill has_related_job_title has_related_job_title has_related_job_title has_related_job_title has_related_job_title has_related_job_title
Scoring of Node Relationships (Edge Weights) Foreground vs. Background Analysis Every term scored against it’s context. The more commonly the term appears within it’s foreground context versus its background context, the more relevant it is to the specified foreground context. countFG(x) - totalDocsFG * probBG(x) z = -------------------------------------------------------- sqrt(totalDocsFG * probBG(x) * (1 - probBG(x))) { "type":"keywords”, "values":[ { "value":"hive", "relatedness":0.9773, "popularity":369 }, { "value":"java", "relatedness":0.9236, "popularity":15653 }, { "value":".net", "relatedness":0.5294, "popularity":17683 }, { "value":"bee", "relatedness":0.0, "popularity":0 }, { "value":"teacher", "relatedness":-0.2380, "popularity":9923 }, { "value":"registered nurse", "relatedness": -0.3802 "popularity":27089 } ] } We are essentially boosting terms which are more related to some known feature (and ignoring terms which are equally likely to appear in the background corpus) + - Foreground Query: "Hadoop" Knowledge Graph
Related term vector (for query concept expansion) http://localhost:8983/solr/stack-exchange-health/skg
Content-based Recommendations (More Like This on Steroids) http://localhost:8983/solr/job-postings/skg
Who’s in Love with Jean Grey?
Differentiating related terms Misspellings: managr => manager Synonyms: cpa => certified public accountant rn => registered nurse r.n. => registered nurse Ambiguous Terms*: driver => driver (trucking) ~80% likelihood driver => driver (software) ~20% likelihood Related Terms: r.n. => nursing, bsn hadoop => mapreduce, hive, pig *differentiated based upon user and query context
Thought Exercise What do you think of when I say the word “driver”? What about “architect”?
Use Case: Query Disambiguation Source: M. Korayem, C. Ortiz, K. AlJadda, T. Grainger. "Query Sense Disambiguation Leveraging Large Scale User Behavioral Data". IEEE Big Data 2015. Example Related Keywords (representing multiple meanings) driver truck driver, linux, windows, courier, embedded, cdl, delivery architect autocad drafter, designer, enterprise architect, java architect, designer, architectural designer, data architect, oracle, java, architectural drafter, autocad, drafter, cad, engineer … …
Use Case: Query Disambiguation Example Related Keywords (representing multiple meanings) driver truck driver, linux, windows, courier, embedded, cdl, delivery architect autocad drafter, designer, enterprise architect, java architect, designer, architectural designer, data architect, oracle, java, architectural drafter, autocad, drafter, cad, engineer … … Source: M. Korayem, C. Ortiz, K. AlJadda, T. Grainger. "Query Sense Disambiguation Leveraging Large Scale User Behavioral Data". IEEE Big Data 2015.
A few methodologies: 1) Query Log Mining 2) Semantic Knowledge Graph Knowledge Graph
Disambiguation by Category Example Meaning 1: Restaurant => bbq, brisket, ribs, pork, … Meaning 2: Outdoor Equipment => bbq, grill, charcoal, propane, …
Disambiguated meanings (represented as term vectors) Example Related Keywords (Disambiguated Meanings) architect 1: enterprise architect, java architect, data architect, oracle, java, .net 2: architectural designer, architectural drafter, autocad, autocad drafter, designer, drafter, cad, engineer driver 1: linux, windows, embedded 2: truck driver, cdl driver, delivery driver, class b driver, cdl, courier designer 1: design, print, animation, artist, illustrator, creative, graphic artist, graphic, photoshop, video 2: graphic, web designer, design, web design, graphic design, graphic designer 3: design, drafter, cad designer, draftsman, autocad, mechanical designer, proe, structural designer, revit … … Source: M. Korayem, C. Ortiz, K. AlJadda, T. Grainger. "Query Sense Disambiguation Leveraging Large Scale User Behavioral Data". IEEE Big Data 2015.
Every term or phrase is a Context-dependent cluster of meaning with an ambiguous label
What does “love” mean? http://localhost:8983/solr/thesaurus/skg
What does “love” mean in the context of “hug”? http://localhost:8983/solr/thesaurus/skg "embrace"
What does “love” mean in the context of “child”? http://localhost:8983/solr/thesaurus/skg
So what’s the end goal here? User’s Query: machine learning research and development Portland, OR software engineer AND hadoop, java Traditional Query Parsing: (machine AND learning AND research AND development AND portland) OR (software AND engineer AND hadoop AND java) Semantic Query Parsing: "machine learning" AND "research and development" AND "Portland, OR" AND "software engineer" AND hadoop AND java Semantically Expanded Query: "machine learning"^10 OR "data scientist" OR "data mining" OR "artificial intelligence") AND ("research and development"^10 OR "r&d") AND AND ("Portland, OR"^10 OR "Portland, Oregon" OR {!geofilt pt=45.512,-122.676 d=50 sfield=geo}) AND ("software engineer"^10 OR "software developer") AND (hadoop^10 OR "big data" OR hbase OR hive) AND (java^10 OR j2ee)
Semantic Search Components: • Apache Solr • Solr Text Tagger • Semantic Knowledge Graph • Statistical Phrase Identifier • Fusion Semantic Query Pipelines • Fusion AI Synonyms Job • Fusion AI Token & Phrase Spell Correction Job • Fusion AI Head/Tail Analysis Job • Fusion AI Phrase Identification Job • Fusion Query Rules Engine
I talked about integrating these last year: See my Activate 2018 talk on “How to Build a Semantic Search System” For details on extended Lucidworks Fusion capabilities.
For Today’s talk, I’ll only focus on these two • Solr Text Tagger • Semantic Knowledge Graph
Natural Language Search with Knowledge Graphs (Chicago Meetup)
Natural Language Search with Knowledge Graphs (Chicago Meetup)
Graph Query Parser • Query-time, cyclic aware graph traversal is able to rank documents based on relationships • Provides controls for depth, filtering of results and inclusion of root and/or leaves • Limitations: distributed queries only traverse intra-shard docs Examples: • http://localhost:8983/solr/graph/query?fl=id,score& q={!graph from=in_edge to=out_edge}id:A • http://localhost:8983/solr/my_graph/query?fl=id& q={!graph from=in_edge to=out_edge traversalFilter='foo:[* TO 15]'}id:A • http://localhost:8983/solr/my_graph/query?fl=id& q={!graph from=in_edge to=out_edge maxDepth=1}foo:[* TO 10]
Example Query:
Demo Data Places (also includes geonames database) Entities (includes search commands) Text Content [ Web crawl of restaurant and product reviews sites ]
Find Location (Graph Query) http://localhost:8983/solr/POI/select
Graph Traversal converted to Facet http://localhost:8983/solr/POI/select
For Remaining keywords, find doc type + related terms http://localhost:8983/solr/POI/select
Full Knowledge Graph Traversal in Single Request!
Tricks for Automated Graph Generation
Named Entity Recognition (NER) NER translates… Barack Obama was the president of the United States of America. Before that, Obama was a senator. into… <person id="barack_obama">Barack Obama</person> was the <role>president</role> of the <country id="usa">United States of America</country>. Before that, <person id="barack_obama">Obama</person> was a <role>senator</role>. In Solr, this would become: text: Barack Obama was the president of the United States of America. Before that, Obama was a senator. person: Barack Obama country: United States of America role: [ president, senator ]
Open Information Extraction (automatic RDF triple extraction / explicit knowledge graph learning)
Demo!
Demo!
popular barbeque near Activate (popular same as "good", "top", "best") Hotels near Activate hotels near popular BBQ in Washington DC BBQ near airports near Activate hotels near movie theaters in Washington DC … And that’s really just the beginning!
But it’s unfortunately also the end of our presentation for today : (
We operationalize AI for the largest businesses on the planet.
Questions?
Trey Grainger trey@lucidworks.com @treygrainger Other presentations: http://www.treygrainger.com 40% Discount code: mtpchic19 http://aiPoweredSearch.com http://solrinaction.com Books: Thank You!

More Related Content

PDF
Natural Language Search with Knowledge Graphs (Activate 2019)
Trey Grainger
 
PPTX
Searching for Meaning
Trey Grainger
 
PDF
Reflected Intelligence: Real world AI in Digital Transformation
Trey Grainger
 
PPTX
The Relevance of the Apache Solr Semantic Knowledge Graph
Trey Grainger
 
PDF
The Future of Search and AI
Trey Grainger
 
PPTX
How to Build a Semantic Search System
Trey Grainger
 
PDF
Natural Language Search with Knowledge Graphs (Haystack 2019)
Trey Grainger
 
PDF
AI, Search, and the Disruption of Knowledge Management
Trey Grainger
 
Natural Language Search with Knowledge Graphs (Activate 2019)
Trey Grainger
 
Searching for Meaning
Trey Grainger
 
Reflected Intelligence: Real world AI in Digital Transformation
Trey Grainger
 
The Relevance of the Apache Solr Semantic Knowledge Graph
Trey Grainger
 
The Future of Search and AI
Trey Grainger
 
How to Build a Semantic Search System
Trey Grainger
 
Natural Language Search with Knowledge Graphs (Haystack 2019)
Trey Grainger
 
AI, Search, and the Disruption of Knowledge Management
Trey Grainger
 

What's hot (20)

PDF
The Next Generation of AI-powered Search
Trey Grainger
 
PPTX
South Big Data Hub: Text Data Analysis Panel
Trey Grainger
 
PDF
Measuring Relevance in the Negative Space
Trey Grainger
 
PPTX
The Apache Solr Semantic Knowledge Graph
Trey Grainger
 
PPTX
The Intent Algorithms of Search & Recommendation Engines
Trey Grainger
 
PPTX
Reflected Intelligence: Lucene/Solr as a self-learning data system
Trey Grainger
 
PDF
Crowdsourced query augmentation through the semantic discovery of domain spec...
Trey Grainger
 
PDF
Balancing the Dimensions of User Intent
Trey Grainger
 
PPTX
Self-learned Relevancy with Apache Solr
Trey Grainger
 
PDF
Thought Vectors and Knowledge Graphs in AI-powered Search
Trey Grainger
 
PPTX
Searching on Intent: Knowledge Graphs, Personalization, and Contextual Disamb...
Trey Grainger
 
PPTX
The Semantic Knowledge Graph
Trey Grainger
 
PPTX
The Apache Solr Smart Data Ecosystem
Trey Grainger
 
PDF
Reflected intelligence evolving self-learning data systems
Trey Grainger
 
PPTX
Haystack 2019 - Natural Language Search with Knowledge Graphs - Trey Grainger
OpenSource Connections
 
PPTX
Building Search & Recommendation Engines
Trey Grainger
 
PPTX
Leveraging Lucene/Solr as a Knowledge Graph and Intent Engine
Trey Grainger
 
PDF
Enhancing relevancy through personalization & semantic search
Trey Grainger
 
PPTX
Intent Algorithms: The Data Science of Smart Information Retrieval Systems
Trey Grainger
 
PDF
Tutorial@BDA 2017 -- Knowledge Graph Expansion and Enrichment
Paris Sud University
 
The Next Generation of AI-powered Search
Trey Grainger
 
South Big Data Hub: Text Data Analysis Panel
Trey Grainger
 
Measuring Relevance in the Negative Space
Trey Grainger
 
The Apache Solr Semantic Knowledge Graph
Trey Grainger
 
The Intent Algorithms of Search & Recommendation Engines
Trey Grainger
 
Reflected Intelligence: Lucene/Solr as a self-learning data system
Trey Grainger
 
Crowdsourced query augmentation through the semantic discovery of domain spec...
Trey Grainger
 
Balancing the Dimensions of User Intent
Trey Grainger
 
Self-learned Relevancy with Apache Solr
Trey Grainger
 
Thought Vectors and Knowledge Graphs in AI-powered Search
Trey Grainger
 
Searching on Intent: Knowledge Graphs, Personalization, and Contextual Disamb...
Trey Grainger
 
The Semantic Knowledge Graph
Trey Grainger
 
The Apache Solr Smart Data Ecosystem
Trey Grainger
 
Reflected intelligence evolving self-learning data systems
Trey Grainger
 
Haystack 2019 - Natural Language Search with Knowledge Graphs - Trey Grainger
OpenSource Connections
 
Building Search & Recommendation Engines
Trey Grainger
 
Leveraging Lucene/Solr as a Knowledge Graph and Intent Engine
Trey Grainger
 
Enhancing relevancy through personalization & semantic search
Trey Grainger
 
Intent Algorithms: The Data Science of Smart Information Retrieval Systems
Trey Grainger
 
Tutorial@BDA 2017 -- Knowledge Graph Expansion and Enrichment
Paris Sud University
 
Ad

Similar to Natural Language Search with Knowledge Graphs (Chicago Meetup) (20)

PDF
Introduction to Knowledge Graphs for Information Architects.pdf
Heather Hedden
 
PDF
Knowledge graphs + Chatbots with Neo4j
Christophe Willemsen
 
PPTX
KM SHOWCASE 2020 - "Lessons Learned Building a Knowledge Graph" - Chris Marino
KM Institute
 
PDF
The technical case for a semantic web
Tony Dobaj
 
PPTX
AI, Knowledge Representation and Graph Databases -
 Key Trends in Data Science
Optum
 
PDF
Knowledge Graphs, Ontologies, and AI Applications
Earley Information Science
 
PDF
Knowledge Graphs and Chatbots with Neo4j and IBM Watson - Christophe Willemsen
GraphAware
 
PDF
Knowledge Graph Maintenance
Paul Groth
 
PPTX
From Artwork to Cyber Attacks: Lessons Learned in Building Knowledge Graphs u...
Craig Knoblock
 
PPTX
Semantic Web Technologies
KANIMOZHIUMA
 
PDF
From Linked Data to Semantic Applications
Andre Freitas
 
PDF
Knowledge Graphs - The Power of Graph-Based Search
Neo4j
 
PDF
Session 0.0 poster minutes madness
semanticsconference
 
PDF
Introduction to Knowledge Graphs and Semantic AI
Semantic Web Company
 
PPTX
Using Knowledge Graph for Promoting Cognitive Computing
Artificial Intelligence Institute at UofSC
 
PPTX
BrightTALK - Semantic AI
Semantic Web Company
 
PDF
Linking SharePoint Documents with Structured Data
Semantic Web Company
 
PDF
Benefiting from Semantic AI along the data life cycle
Martin Kaltenböck
 
PDF
X api chinese cop monthly meeting feb.2016
Jessie Chuang
 
PDF
Knowledge Graphs - Creating Connected Search Experiences
Enterprise Knowledge
 
Introduction to Knowledge Graphs for Information Architects.pdf
Heather Hedden
 
Knowledge graphs + Chatbots with Neo4j
Christophe Willemsen
 
KM SHOWCASE 2020 - "Lessons Learned Building a Knowledge Graph" - Chris Marino
KM Institute
 
The technical case for a semantic web
Tony Dobaj
 
AI, Knowledge Representation and Graph Databases -
 Key Trends in Data Science
Optum
 
Knowledge Graphs, Ontologies, and AI Applications
Earley Information Science
 
Knowledge Graphs and Chatbots with Neo4j and IBM Watson - Christophe Willemsen
GraphAware
 
Knowledge Graph Maintenance
Paul Groth
 
From Artwork to Cyber Attacks: Lessons Learned in Building Knowledge Graphs u...
Craig Knoblock
 
Semantic Web Technologies
KANIMOZHIUMA
 
From Linked Data to Semantic Applications
Andre Freitas
 
Knowledge Graphs - The Power of Graph-Based Search
Neo4j
 
Session 0.0 poster minutes madness
semanticsconference
 
Introduction to Knowledge Graphs and Semantic AI
Semantic Web Company
 
Using Knowledge Graph for Promoting Cognitive Computing
Artificial Intelligence Institute at UofSC
 
BrightTALK - Semantic AI
Semantic Web Company
 
Linking SharePoint Documents with Structured Data
Semantic Web Company
 
Benefiting from Semantic AI along the data life cycle
Martin Kaltenböck
 
X api chinese cop monthly meeting feb.2016
Jessie Chuang
 
Knowledge Graphs - Creating Connected Search Experiences
Enterprise Knowledge
 
Ad

Recently uploaded (20)

PPTX
Agile Chennai 18-19 July 2025 | Emerging patterns in Agentic AI by Bharani Su...
AgileNetwork
 
PDF
GDG Cloud Munich - Intro - Luiz Carneiro - #BuildWithAI - July - Abdel.pdf
Luiz Carneiro
 
PDF
NewMind AI Weekly Chronicles - July'25 - Week IV
NewMind AI
 
PDF
How Open Source Changed My Career by abdelrahman ismail
a0m0rajab1
 
PDF
Google I/O Extended 2025 Baku - all ppts
HusseinMalikMammadli
 
PDF
Data_Analytics_vs_Data_Science_vs_BI_by_CA_Suvidha_Chaplot.pdf
CA Suvidha Chaplot
 
PDF
How ETL Control Logic Keeps Your Pipelines Safe and Reliable.pdf
Stryv Solutions Pvt. Ltd.
 
PDF
AI Unleashed - Shaping the Future -Starting Today - AIOUG Yatra 2025 - For Co...
Sandesh Rao
 
PPTX
cloud computing vai.pptx for the project
vaibhavdobariyal79
 
PPTX
Introduction to Flutter by Ayush Desai.pptx
ayushdesai204
 
PDF
The Future of Artificial Intelligence (AI)
Mukul
 
PPTX
IT Runs Better with ThousandEyes AI-driven Assurance
ThousandEyes
 
PPTX
Simple and concise overview about Quantum computing..pptx
mughal641
 
PDF
Presentation about Hardware and Software in Computer
snehamodhawadiya
 
PPTX
The Future of AI & Machine Learning.pptx
pritsen4700
 
PPTX
What-is-the-World-Wide-Web -- Introduction
tonifi9488
 
PDF
Trying to figure out MCP by actually building an app from scratch with open s...
Julien SIMON
 
PDF
Oracle AI Vector Search- Getting Started and what's new in 2025- AIOUG Yatra ...
Sandesh Rao
 
PDF
Software Development Methodologies in 2025
KodekX
 
PPTX
AI in Daily Life: How Artificial Intelligence Helps Us Every Day
vanshrpatil7
 
Agile Chennai 18-19 July 2025 | Emerging patterns in Agentic AI by Bharani Su...
AgileNetwork
 
GDG Cloud Munich - Intro - Luiz Carneiro - #BuildWithAI - July - Abdel.pdf
Luiz Carneiro
 
NewMind AI Weekly Chronicles - July'25 - Week IV
NewMind AI
 
How Open Source Changed My Career by abdelrahman ismail
a0m0rajab1
 
Google I/O Extended 2025 Baku - all ppts
HusseinMalikMammadli
 
Data_Analytics_vs_Data_Science_vs_BI_by_CA_Suvidha_Chaplot.pdf
CA Suvidha Chaplot
 
How ETL Control Logic Keeps Your Pipelines Safe and Reliable.pdf
Stryv Solutions Pvt. Ltd.
 
AI Unleashed - Shaping the Future -Starting Today - AIOUG Yatra 2025 - For Co...
Sandesh Rao
 
cloud computing vai.pptx for the project
vaibhavdobariyal79
 
Introduction to Flutter by Ayush Desai.pptx
ayushdesai204
 
The Future of Artificial Intelligence (AI)
Mukul
 
IT Runs Better with ThousandEyes AI-driven Assurance
ThousandEyes
 
Simple and concise overview about Quantum computing..pptx
mughal641
 
Presentation about Hardware and Software in Computer
snehamodhawadiya
 
The Future of AI & Machine Learning.pptx
pritsen4700
 
What-is-the-World-Wide-Web -- Introduction
tonifi9488
 
Trying to figure out MCP by actually building an app from scratch with open s...
Julien SIMON
 
Oracle AI Vector Search- Getting Started and what's new in 2025- AIOUG Yatra ...
Sandesh Rao
 
Software Development Methodologies in 2025
KodekX
 
AI in Daily Life: How Artificial Intelligence Helps Us Every Day
vanshrpatil7
 

Natural Language Search with Knowledge Graphs (Chicago Meetup)

  • 1. Trey Grainger Chief Algorithms Officer Natural Language Search with Knowledge Graphs September 25, 2019
  • 2. Trey Grainger Chief Algorithms Officer • Previously: SVP of Engineering @ Lucidworks; Director of Engineering @ CareerBuilder • Georgia Tech – MBA, Management of Technology • Furman University – BA, Computer Science, Business, & Philosophy • Stanford University – Information Retrieval & Web Search Other fun projects: • Co-author of Solr in Action, plus numerous research publications • Advisor to Presearch, the decentralized search engine • Lucene / Solr contributor About Me
  • 4. Agenda • About Lucidworks • What is AI-powered Search? • What is Natural Language Search? • What is a Knowledge Graph (and related terminology)? • Philosophy of Language (enough to get the approach…) • Semantic Knowledge Graphs • Knowledge Graph Goals for Natural Language Search • Relevant Tools • Solr Text Tagger • Solr’s Semantic Knowledge Graph • Automated Graph Generation • Demos!
  • 5. The Search & AI Conference COMPANY BEHIND Who are we? 300+ CUSTOMERS ACROSS THE FORTUNE 1000 400+EMPLOYEES OFFICES IN San Francisco, CA (HQ) Raleigh-Durham, NC Cambridge, UK Bangalore, India Hong Kong Employ about 40% of the active committers on the Solr project 40% Contribute over 70% of Solr's open source codebase 70% DEVELOP & SUPPORT Apache
  • 6. Industry’s most powerful Intelligent Search & Discovery Platform.
  • 8. AI? Machine Learning? Data Science? Neural Search? AI-Powered Search? Deep Learning?
  • 14. AI-powered Search Question / Answer Systems Virtual Assistants • Signals Boosting Models • Learning to Rank • Semantic Search • Collaborative Filtering • Personalized Search • Content Clustering • NLP / Entity Resolution • Semantic Knowledge Graphs • Document Classification • etc. • Neural Search • Word Embeddings • Vector Search • Image / Voice Search • etc. • Question / Answer Systems • Virtual Assistants • Chatbots • Rules-based Relevancy • etc.
  • 15. Proudly built with open-source tech at its core: Apache Solr & Apache Spark Personalizes search with applied machine learning Proven on the world’s biggest information systems
  • 16. Signals Boosting Popularized Relevancy Learning to Rank Generalized Relevancy Collaborative Filtering Personalized Relevancy
  • 17. Knowledge Graphs Domain modeling Multi-modal Learning Merged Content Modalities Thought Vectors Conceptual scoring
  • 18. Basic Keyword Search (inverted index, tf-idf, bm25, multilingual text analysis, query formulation, etc.) Query Intent (query classification, semantic query parsing, knowledge graphs, concept expansion, rules, clustering, classification) Relevancy Tuning (signals, AB testing/genetic algorithms, Learning to Rank, Neural Networks) Self-learning Relevance Engineering Sophistication Context for this Talk Taxonomies / Entity Extraction (entity recognition, basic ontologies, synonyms, etc.)
  • 22. What is a Knowledge Graph? (vs. Ontology vs. Taxonomy vs. Synonyms, etc.)
  • 24. Overly Simplistic Definitions Alternative Labels: Substitute words with identical meanings [ CTO => Chief Technology Officer; specialise => specialize ] Synonyms List: Provides substitute words that can be used to represent the same or very similar things [ human => homo sapien, mankind; food => sustenance, meal ] Taxonomy: Classifies things into Categories [ john is Human; Human is Mammal; Mammal is Animal ] Ontology: Defines relationships between types of things [ animal eats food; human is animal ] Knowledge Graph: Instantiation of an Ontology (contains the things that are related) [ john is human; john eats food ] In practice, there is significant overlap…
  • 26. What sort of Knowledge Graph can help us with the kinds of problems we encounter in Search use cases?
  • 28. most often used in reference to “free text”
  • 29. But… unstructured data is really more like “hyper-structured” data. It is a graph that contains much more structure than typical “structured data.”
  • 30. Structured Data Employees Table id name company start_date lw100 Trey Grainger 1234 2016-02-01 dis2 Mickey Mouse 9123 1928-11-28 tsla1 Elon Musk 5678 2003-07-01 Companies Table id name start_date 1234 Lucidworks 2016-02-01 5678 Tesla 1928-11-28 9123 Disney 2003-07-01 Discrete Values Continuous Values Foreign Key
  • 31. Unstructured Data Trey Grainger works at Lucidworks. He is speaking at the Activate 2019 conference. #Activate19 (Activate) is being held in Washington, DC September 9-12, 2019. Trey got his masters from Georgia Tech.
  • 32. Trey Grainger works for Lucidworks. He is speaking at the Activate 2019 conference. #Activate19 (Activate) is being held in Washington, DC September 9-12, 2019. Trey got his masters degree from Georgia Tech. Trey’s Voicemail Unstructured Data
  • 33. Trey Grainger works for Lucidworks. He is speaking at the Activate 2019 conference. #Activate19 (Activate) is being held in Washington, DC September 9-12, 2019. Trey got his masters degree from Georgia Tech. Trey’s Voicemail Foreign Key?
  • 34. Trey Grainger works for Lucidworks. He is speaking at the Activate 2019 conference. #Activate19 (Activate) is being held in Washington, DC September 9-12, 2019. Trey got his masters degree from Georgia Tech. Trey’s Voicemail Fuzzy Foreign Key? (Entity Resolution)
  • 35. Trey Grainger works for Lucidworks. He is speaking at the Activate 2019 conference. #Activate19 (Activate) is being held in Washington, DC September 9-12, 2019. Trey got his masters degree from Georgia Tech. Trey’s Voicemail Fuzzier Foreign Key? (metadata, latent features)
  • 36. Trey Grainger works for Lucidworks. He is speaking at the Activate 2019 conference. #Activate19 (Activate) is being held in Washington, DC September 9-12, 2019. Trey got his masters degree from Georgia Tech. Trey’s Voicemail Fuzzier Foreign Key? (metadata, latent features) Not so fast!
  • 39. Giant Graph of Relationships... Trey Grainger works for Lucidworks. He is speaking at the Activate 2019 conference. #Activate19 (Activate) is being held in Washington, DC September 9-12, 2019. Trey got his masters degree from Georgia Tech. Trey’s Voicemail
  • 40. How do we easily harness this “semantic graph” of relationships within unstructured information?
  • 42. id: 1 job_title: Software Engineer desc: software engineer at a great company skills: .Net, C#, java id: 2 job_title: Registered Nurse desc: a registered nurse at hospital doing hard work skills: oncology, phlebotemy id: 3 job_title: Java Developer desc: a software engineer or a java engineer doing work skills: java, scala, hibernate field doc term desc 1 a at company engineer great software 2 a at doing hard hospital nurse registered work 3 a doing engineer java or software work job_title 1 Software Engineer … … … Terms-Docs Inverted IndexDocs-Terms Forward IndexDocuments Source: Trey Grainger, Khalifeh AlJadda, Mohammed Korayem, Andries Smith.“The Semantic Knowledge Graph: A compact, auto-generated model for real-time traversal and ranking of any relationship within a domain”. DSAA 2016. Knowledge Graph field term postings list doc pos desc a 1 4 2 1 3 1, 5 at 1 3 2 4 company 1 6 doing 2 6 3 8 engineer 1 2 3 3, 7 great 1 5 hard 2 7 hospital 2 5 java 3 6 nurse 2 3 or 3 4 registered 2 2 software 1 1 3 2 work 2 10 3 9 job_title java developer 3 1 … … … …
  • 43. Serves as a “data science toolkit” API that allows dynamically navigating and pivoting through multiple levels of relationships between items in a domain. Semantic Knowledge Graph API Core similarity engine, exposed via API Any product can leverage the core relationship scoring engine to score any list of entities against any other list Full domain support Keywords, categories, tags, based upon any field on your documents. Graph is build automatically from the content representing your domain. Intersections, overlaps, & relationship scoring, many levels deep Users can either provide a list of items to score, or else have the system dynamically discover the most related items (or both). Knowledge Graph
  • 44. Source: Trey Grainger, Khalifeh AlJadda, Mohammed Korayem, Andries Smith.“The Semantic Knowledge Graph: A compact, auto- generated model for real- time traversal and ranking of any relationship within a domain”. DSAA 2016. Knowledge Graph Set-theory View Graph View How the Graph Traversal Works skill: Java skill: Scala skill: Hibernate skill: Oncology has_related_skill has_related_skill has_related_skill doc 1 doc 2 doc 3 doc 4 doc 5 doc 6 skill: Java skill: Java skill: Scala skill: Hibernate skill: Oncology Data Structure View Java Scala Hibernate docs 1, 2, 6 docs 3, 4 Oncology doc 5
  • 45. DOI: 10.1109/DSAA.2016.51 Conference: 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA) Source: Trey Grainger, Khalifeh AlJadda, Mohammed Korayem, Andries Smith.“The Semantic Knowledge Graph: A compact, auto-generated model for real-time traversal and ranking of any relationship within a domain”. DSAA 2016. Knowledge Graph Graph Traversal Data Structure View Graph View doc 1 doc 2 doc 3 doc 4 doc 5 doc 6 skill: Java skill: Java skill: Scala skill: Hibernate skill: Oncology doc 1 doc 2 doc 3 doc 4 doc 5 doc 6 job_title: Software Engineer job_title: Data Scientist job_title: Java Developer …… Inverted Index Lookup Forward Index Lookup Forward Index Lookup Inverted Index Lookup Java Java Developer Hibernate Scala Software Engineer Data Scientist has_related_skill has_related_skill has_related_skill has_related_job_title has_related_job_title has_related_job_title has_related_job_title has_related_job_title has_related_job_title
  • 46. Scoring of Node Relationships (Edge Weights) Foreground vs. Background Analysis Every term scored against it’s context. The more commonly the term appears within it’s foreground context versus its background context, the more relevant it is to the specified foreground context. countFG(x) - totalDocsFG * probBG(x) z = -------------------------------------------------------- sqrt(totalDocsFG * probBG(x) * (1 - probBG(x))) { "type":"keywords”, "values":[ { "value":"hive", "relatedness":0.9773, "popularity":369 }, { "value":"java", "relatedness":0.9236, "popularity":15653 }, { "value":".net", "relatedness":0.5294, "popularity":17683 }, { "value":"bee", "relatedness":0.0, "popularity":0 }, { "value":"teacher", "relatedness":-0.2380, "popularity":9923 }, { "value":"registered nurse", "relatedness": -0.3802 "popularity":27089 } ] } We are essentially boosting terms which are more related to some known feature (and ignoring terms which are equally likely to appear in the background corpus) + - Foreground Query: "Hadoop" Knowledge Graph
  • 47. Related term vector (for query concept expansion) http://localhost:8983/solr/stack-exchange-health/skg
  • 48. Content-based Recommendations (More Like This on Steroids) http://localhost:8983/solr/job-postings/skg
  • 49. Who’s in Love with Jean Grey?
  • 50. Differentiating related terms Misspellings: managr => manager Synonyms: cpa => certified public accountant rn => registered nurse r.n. => registered nurse Ambiguous Terms*: driver => driver (trucking) ~80% likelihood driver => driver (software) ~20% likelihood Related Terms: r.n. => nursing, bsn hadoop => mapreduce, hive, pig *differentiated based upon user and query context
  • 51. Thought Exercise What do you think of when I say the word “driver”? What about “architect”?
  • 52. Use Case: Query Disambiguation Source: M. Korayem, C. Ortiz, K. AlJadda, T. Grainger. "Query Sense Disambiguation Leveraging Large Scale User Behavioral Data". IEEE Big Data 2015. Example Related Keywords (representing multiple meanings) driver truck driver, linux, windows, courier, embedded, cdl, delivery architect autocad drafter, designer, enterprise architect, java architect, designer, architectural designer, data architect, oracle, java, architectural drafter, autocad, drafter, cad, engineer … …
  • 53. Use Case: Query Disambiguation Example Related Keywords (representing multiple meanings) driver truck driver, linux, windows, courier, embedded, cdl, delivery architect autocad drafter, designer, enterprise architect, java architect, designer, architectural designer, data architect, oracle, java, architectural drafter, autocad, drafter, cad, engineer … … Source: M. Korayem, C. Ortiz, K. AlJadda, T. Grainger. "Query Sense Disambiguation Leveraging Large Scale User Behavioral Data". IEEE Big Data 2015.
  • 54. A few methodologies: 1) Query Log Mining 2) Semantic Knowledge Graph Knowledge Graph
  • 55. Disambiguation by Category Example Meaning 1: Restaurant => bbq, brisket, ribs, pork, … Meaning 2: Outdoor Equipment => bbq, grill, charcoal, propane, …
  • 56. Disambiguated meanings (represented as term vectors) Example Related Keywords (Disambiguated Meanings) architect 1: enterprise architect, java architect, data architect, oracle, java, .net 2: architectural designer, architectural drafter, autocad, autocad drafter, designer, drafter, cad, engineer driver 1: linux, windows, embedded 2: truck driver, cdl driver, delivery driver, class b driver, cdl, courier designer 1: design, print, animation, artist, illustrator, creative, graphic artist, graphic, photoshop, video 2: graphic, web designer, design, web design, graphic design, graphic designer 3: design, drafter, cad designer, draftsman, autocad, mechanical designer, proe, structural designer, revit … … Source: M. Korayem, C. Ortiz, K. AlJadda, T. Grainger. "Query Sense Disambiguation Leveraging Large Scale User Behavioral Data". IEEE Big Data 2015.
  • 57. Every term or phrase is a Context-dependent cluster of meaning with an ambiguous label
  • 58. What does “love” mean? http://localhost:8983/solr/thesaurus/skg
  • 59. What does “love” mean in the context of “hug”? http://localhost:8983/solr/thesaurus/skg "embrace"
  • 60. What does “love” mean in the context of “child”? http://localhost:8983/solr/thesaurus/skg
  • 61. So what’s the end goal here? User’s Query: machine learning research and development Portland, OR software engineer AND hadoop, java Traditional Query Parsing: (machine AND learning AND research AND development AND portland) OR (software AND engineer AND hadoop AND java) Semantic Query Parsing: "machine learning" AND "research and development" AND "Portland, OR" AND "software engineer" AND hadoop AND java Semantically Expanded Query: "machine learning"^10 OR "data scientist" OR "data mining" OR "artificial intelligence") AND ("research and development"^10 OR "r&d") AND AND ("Portland, OR"^10 OR "Portland, Oregon" OR {!geofilt pt=45.512,-122.676 d=50 sfield=geo}) AND ("software engineer"^10 OR "software developer") AND (hadoop^10 OR "big data" OR hbase OR hive) AND (java^10 OR j2ee)
  • 62. Semantic Search Components: • Apache Solr • Solr Text Tagger • Semantic Knowledge Graph • Statistical Phrase Identifier • Fusion Semantic Query Pipelines • Fusion AI Synonyms Job • Fusion AI Token & Phrase Spell Correction Job • Fusion AI Head/Tail Analysis Job • Fusion AI Phrase Identification Job • Fusion Query Rules Engine
  • 63. I talked about integrating these last year: See my Activate 2018 talk on “How to Build a Semantic Search System” For details on extended Lucidworks Fusion capabilities.
  • 64. For Today’s talk, I’ll only focus on these two • Solr Text Tagger • Semantic Knowledge Graph
  • 67. Graph Query Parser • Query-time, cyclic aware graph traversal is able to rank documents based on relationships • Provides controls for depth, filtering of results and inclusion of root and/or leaves • Limitations: distributed queries only traverse intra-shard docs Examples: • http://localhost:8983/solr/graph/query?fl=id,score& q={!graph from=in_edge to=out_edge}id:A • http://localhost:8983/solr/my_graph/query?fl=id& q={!graph from=in_edge to=out_edge traversalFilter='foo:[* TO 15]'}id:A • http://localhost:8983/solr/my_graph/query?fl=id& q={!graph from=in_edge to=out_edge maxDepth=1}foo:[* TO 10]
  • 69. Demo Data Places (also includes geonames database) Entities (includes search commands) Text Content [ Web crawl of restaurant and product reviews sites ]
  • 70. Find Location (Graph Query) http://localhost:8983/solr/POI/select
  • 71. Graph Traversal converted to Facet http://localhost:8983/solr/POI/select
  • 72. For Remaining keywords, find doc type + related terms http://localhost:8983/solr/POI/select
  • 73. Full Knowledge Graph Traversal in Single Request!
  • 75. Named Entity Recognition (NER) NER translates… Barack Obama was the president of the United States of America. Before that, Obama was a senator. into… <person id="barack_obama">Barack Obama</person> was the <role>president</role> of the <country id="usa">United States of America</country>. Before that, <person id="barack_obama">Obama</person> was a <role>senator</role>. In Solr, this would become: text: Barack Obama was the president of the United States of America. Before that, Obama was a senator. person: Barack Obama country: United States of America role: [ president, senator ]
  • 76. Open Information Extraction (automatic RDF triple extraction / explicit knowledge graph learning)
  • 77. Demo!
  • 78. Demo!
  • 79. popular barbeque near Activate (popular same as "good", "top", "best") Hotels near Activate hotels near popular BBQ in Washington DC BBQ near airports near Activate hotels near movie theaters in Washington DC … And that’s really just the beginning!
  • 80. But it’s unfortunately also the end of our presentation for today : (
  • 81. We operationalize AI for the largest businesses on the planet.
  • 83. Trey Grainger [email protected] @treygrainger Other presentations: http://www.treygrainger.com 40% Discount code: mtpchic19 http://aiPoweredSearch.com http://solrinaction.com Books: Thank You!