SlideShare a Scribd company logo

Tutorial "Linked Data Query Processing" Part 4 "Execution Process" (WWW 2013 Ed.)

Olaf Hartig
Olaf Hartig

The document outlines a tutorial on linked data query processing presented at the WWW 2013 conference, focusing on execution processes in query execution strategies, including separated and integrated execution approaches. It discusses the advantages and challenges of both methods, illustrating concepts such as push-based implementation, link traversal-based query execution, and the use of iterators for data access. Key components like data retrieval, result construction, and various operator interactions are also explored.

TechnologyBusiness
1 of 52
Downloaded 68 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
Linked Data Query Processing Tutorial at the 22nd International World Wide Web Conference (WWW 2013) May 14, 2013 http://db.uwaterloo.ca/LDQTut2013/ 4. Execution Process Olaf Hartig University of Waterloo
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 2 Outline  Separated Execution  Integrated Execution ➢ General Idea and Properties ➢ Push-Based Implementation [LT10, LT11] ➢ Link Traversal Based Query Execution
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 3 … clearly separate data retrieval and result construction into two consecutive phases Separated Execution Approaches Query-local data http://mdb.../Paul http://geo.../Berlin http://mdb.../Ric http://geo.../Rome ?loc?actor GET http://.../movie2449 1 2
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 4 Properties of Separated Execution Query-local data http://mdb.../Paul http://geo.../Berlin http://mdb.../Ric http://geo.../Rome ?loc?actor GET http://.../movie2449 1 2 ● Advantage: straightforward to implement ● Can be combined with any source selection strategy ● A traditional query execution plan might then be used for constructing the result ● Downside: First solutions cannot be reported before data retrieval has been completed
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 5 Query-local data http://mdb.../Paul http://geo.../Berlin http://mdb.../Ric http://geo.../Rome ?loc?actor GET http://.../movie2449 Integrated Execution Approaches … intertwine data retrieval and result construction
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 6 Query-local data http://mdb.../Paul http://geo.../Berlin http://mdb.../Ric http://geo.../Rome ?loc?actor GET http://.../movie2449 Integrated Execution Approaches ● Implementations may report first solutions early ● For monotonic queries ● Implementations may process data in a streaming manner ● May require less query-local memory (b/c query-local dataset need not be materialized) ● Can also be combined with any source selection strategy … intertwine data retrieval and result construction
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 7 Outline  Separated Execution  Integrated Execution ➢ General Idea and Properties ➢ Push-Based Implementation [LT10, LT11] ➢ Link Traversal Based Query Execution √ √
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 8 Push-Based Implementation TP Operator TP Operator TP Operator Join Operator Join Operator ● Each operator runs as an asynchronous thread ● Connected via message queues Data Retrieval Operator
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 9 Push-Based Implementation TP Operator TP Operator TP Operator Join Operator Join Operator ● Each operator runs as an asynchronous thread ● Connected via message queues ● Data retrieval operator ● May apply any possible source selection strategy ● Usually, implemented via multiple lookup threads ● Pushes any incoming matching triple to the corresponding triple pattern operator Data Retrieval Operator
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 10 Push-Based Implementation (Example) TP Operator ( ?p, interested in, ?i ) TP Operator ( ?p, affiliated with, orgax ) TP Operator ( ?i, rdf:type, Book ) Join Operator Join Operator Data Retrieval Operator
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 11 Push-Based Implementation (Example) TP Operator ( ?p, interested in, ?i ) TP Operator ( ?p, affiliated with, orgax ) TP Operator ( ?i, rdf:type, Book ) Join Operator Join Operator Data Retrieval Operator { ?p → alice } ( alice, affiliated with, orgax ) ( alice, affiliated with, orgax )
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 12 Push-Based Implementation (Example) TP Operator ( ?p, interested in, ?i ) TP Operator ( ?p, affiliated with, orgax ) TP Operator ( ?i, rdf:type, Book ) Join Operator Join Operator Data Retrieval Operator { ?p → alice } ( alice, interested in, yoga ) { ?p → alice, ?i → yoga } ( alice, interested in, yoga ) { ?p → alice, ?i → yoga }
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 13 Symmetric Hash Join (SHJ) SHJ Operator # Bucket 1 {?p → alice,?i → yoga} 2 3 {?p → bob,?i → tea}, {?p → bob,?i → chess} # Bucket 1 {?p → alice,?o → orgax} 2 {?p → eve,?o → orgax} 3 ● Maintains a hash table for each input ● Same hash function for both tables ● Each bucket contains a set of input solutions
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 14 How SHJ Processes Incoming Solution SHJ Operator # Bucket 1 {?p → alice,?i → yoga} 2 3 {?p → bob,?i → tea}, {?p → bob,?i → chess} # Bucket 1 {?p → alice,?o → orgax} 2 {?p → eve,?o → orgax} 3 ● Hash the value that μinput binds to the join variable μinput = {?p → bob,?o → acme}
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 15 How SHJ Processes Incoming Solution SHJ Operator # Bucket 1 {?p → alice,?i → yoga} 2 3 {?p → bob,?i → tea}, {?p → bob,?i → chess} # Bucket 1 {?p → alice,?o → orgax} 2 {?p → eve,?o → orgax} 3 ● Hash the value that μinput binds to the join variable ● Insert μinput into the corresponding input hash table hfct(bob) = 3μinput = {?p → bob,?o → acme}
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 16 How SHJ Processes Incoming Solution SHJ Operator # Bucket 1 {?p → alice,?i → yoga} 2 3 {?p → bob,?i → tea}, {?p → bob,?i → chess} # Bucket 1 {?p → alice,?o → orgax} 2 {?p → eve,?o → orgax} 3 {?p → bob,?o → acme} ● Hash the value that μinput binds to the join variable ● Insert μinput into the corresponding input hash table ● Probe into the other hash table to find join candidates ● Merge μinput with each join candidate, push results to output {?p → bob,?o → acme,?i → tea} hfct(bob) = 3 {?p → bob,?o → acme,?i → chess} μinput = {?p → bob,?o → acme}
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 17 Outline  Separated Execution  Integrated Execution ➢ General Idea and Properties ➢ Push-Based Implementation [LT10, LT11] ➢ Link Traversal Based Query Execution √ √ √
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 18 Link Traversal Based Query Execution … is the combination of integrated execution and live exploration (i.e., the push-based approach can be used as an implementation of LTBQE) An even closer combination presents the following pull-based implementation approach ...
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 19 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query Seed: <http://.../orgaX> Pipeline of Link Traversing Iterators I3 I2 I1
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 20 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 Seed: <http://.../orgaX> Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 21 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query Seed: <http://.../orgaX> tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 query-local dataset Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 22 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query Seed: <http://.../orgaX> tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 query-local dataset Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 23 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query Seed: <http://.../orgaX> tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 query-local dataset Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 24 query-local dataset Next? tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 25 query-local dataset Next? Next? tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 26 query-local dataset Next? Next? Next? tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 27 Next? Next? Next? tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 : <http://.../alice> ex:affiliated_with <http://.../orgaX> : query-local dataset Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 28 Next? tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 : <http://.../alice> ex:affiliated_with <http://.../orgaX> : query-local dataset { ?p = <http://.../alice> } Next? tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 29 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 query-local dataset { ?p = <http://.../alice> } Next? Next? tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 30 Next? Next? tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 query-local dataset { ?p = <http://.../alice> } Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 31 Next? Next? tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 query-local dataset { ?p = <http://.../alice> } : <http://.../alice> ex:interested_in <http://.../b1> : Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 32 Next? Next? tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> } : <http://.../alice> ex:interested_in <http://.../b1> : query-local dataset Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 33 Next? tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> } : <http://.../alice> ex:interested_in <http://.../b1> : query-local dataset { ?p = <http://.../alice> , ?b = <http://.../b1> } Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 34 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 Next? { ?p = <http://.../alice> } query-local dataset tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> , ?b = <http://.../b1> } Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 35 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 Next? { ?p = <http://.../alice> } query-local dataset tp3 = ( ?b , rdf:type , <http://.../Book> ) tp3 ' = ( <http://.../b1> , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> , ?b = <http://.../b1> } Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 36 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 Next? { ?p = <http://.../alice> } tp3 = ( ?b , rdf:type , <http://.../Book> ) tp3 ' = ( <http://.../b1> , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> , ?b = <http://.../b1> } query-local dataset : <http://.../Book> rdfs:subClassOf <http://.../CreativeWork> : Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 37 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 Next? { ?p = <http://.../alice> } tp3 = ( ?b , rdf:type , <http://.../Book> ) tp3 ' = ( <http://.../b1> , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> , ?b = <http://.../b1> } query-local dataset : <http://.../b1> rdf:type <http://.../Book> : Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 38 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 Next? { ?p = <http://.../alice> } tp3 = ( ?b , rdf:type , <http://.../Book> ) tp3 ' = ( <http://.../b1> , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> , ?b = <http://.../b1> } : <http://.../b1> rdf:type <http://.../Book> : query-local dataset Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 39 { ?p = <http://.../alice> , ?b = <http://.../b1> } tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 { ?p = <http://.../alice> } tp3 = ( ?b , rdf:type , <http://.../Book> ) tp3 ' = ( <http://.../b1> , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> , ?b = <http://.../b1> } query-local dataset Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 40 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp3 = ( ?b , rdf:type , <http://.../Book> ) tp3 ' = ( <http://.../b1> , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 Pipeline of Link Traversing Iterators { ?p = <http://.../alice> } { ?p = <http://.../alice> , ?b = <http://.../b1> } Next? query-local dataset
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 41 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 Pipeline of Link Traversing Iterators { ?p = <http://.../alice> } Next? Next? query-local dataset
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 42 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 Pipeline of Link Traversing Iterators { ?p = <http://.../alice> } Next? Next? query-local dataset
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 43 ● Deeply intertwines integrated execution + live exploration ● Data retrieval is a side-effect of calling link traversing iterators ● No need for a separate data retrieval operator ● Makes adding LTBQE to existing SPARQL engines easy ● Because those usually use iterators for execution ● Caveat: does not achieve the full flexibility that is possible with LTBQE ● Iterators evaluate the triple patterns in a fixed order ● Iterators discard each input solution after using it Pipeline of Link Traversing Iterators
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 44 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query Seed: <http://.../orgaX> tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX>) I1 Alternative Execution Order
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 45 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query Seed: <http://.../orgaX> tp2 = ( ?p , ex:interested_in , ?b ) I2 Alternative Execution Order tp3 = ( ?p , ex:affiliated_with , <http://.../orgaX>) I3 tp1 = ( ?b , rdf:type , <http://.../Book> ) I1
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 46 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query Seed: <http://.../orgaX> tp2 = ( ?p , ex:interested_in , ?b ) I2 query-local dataset Iterator Based Execution tp1 = ( ?b , rdf:type , <http://.../Book> ) I1 tp3 = ( ?p , ex:affiliated_with , <http://.../orgaX>) I3
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 47 tp3 = ( ?p , ex:affiliated_with , <http://.../orgaX>) I3 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp1 = ( ?b , rdf:type , <http://.../Book> ) I1 : <http://.../alice> ex:affiliated_with <http://.../orgaX> : query-local dataset Alternative Execution Order
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 48 Next?: <http://.../alice> ex:affiliated_with <http://.../orgaX> : query-local dataset Next? tp1 = ( ?b , rdf:type , <http://.../Book> ) I1 Alternative Execution Order Next? tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?p , ex:affiliated_with , <http://.../orgaX>) I3
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 49 Next?: <http://.../alice> ex:affiliated_with <http://.../orgaX> : query-local dataset Next? tp1 = ( ?b , rdf:type , <http://.../Book> ) I1 Alternative Execution Order END! tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?p , ex:affiliated_with , <http://.../orgaX>) I3
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 50 query-local dataset tp1 = ( ?b , rdf:type , <http://.../Book> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?p , ex:affiliated_with , <http://.../orgaX>) I3 END! END! END! Computed query result may depend on the order of triple patterns = logical query execution plan Alternative Execution Order
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 51 Outline  Separated Execution  Integrated Execution ➢ General Idea and Properties ➢ Push-Based Implementation [LT10, LT11] ➢ Link Traversal Based Query Execution √ √ √ √ Next part: 5. Query Planning and Optimization ...
WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 52 These slides have been created by Olaf Hartig for the WWW 2013 tutorial on Link Data Query Processing Tutorial Website: http://db.uwaterloo.ca/LDQTut2013/ This work is licensed under a Creative Commons Attribution-Share Alike 3.0 License (http://creativecommons.org/licenses/by-sa/3.0/)

More Related Content

What's hot (20)

PPTX
IEEE IRI 16 - Clustering Web Pages based on Structure and Style Similarity
Thamme Gowda
 
PDF
Joins in a distributed world - Lucian Precup
distributed matters
 
PPTX
“Open Data Web” – A Linked Open Data Repository Built with CKAN
Chengjen Lee
 
PPTX
The FLuID Meta Model: Incrementally Compute Schema-level Indices for the Web...
Till Blume
 
PDF
Are Linked Datasets fit for Open-domain Question Answering? A Quality Assessment
Harsh Thakkar
 
PPTX
Detection of Related Semantic Datasets Based on Frequent Subgraph Mining
Mikel Emaldi Manrique
 
PPTX
2015 TaPP - Interoperability for Provenance-aware Databases using PROV and JSON
Boris Glavic
 
PPTX
eNanoMapper database, search tools and templates
Nina Jeliazkova
 
PDF
SF Python Meetup: TextRank in Python
Paco Nathan
 
PPTX
RDF-Gen: Generating RDF from streaming and archival data
Giorgos Santipantakis
 
PPT
Achieving time effective federated information from scalable rdf data using s...
తేజ దండిభట్ల
 
PDF
Interactive exploration of complex relational data sets in a web - SemWeb.Pro...
Logilab
 
PDF
Oshs_9_11_2015
Béatrice Bouchou
 
ODP
Graph databases
Karol Grzegorczyk
 
PPSX
Elasticsearch - basics and beyond
Ernesto Reig
 
PPTX
Optimized index structures for querying rdf from the web
Mahdi Atawneh
 
PDF
Instance Matching
Robert Isele
 
PDF
Automatic creation of mappings between classification systems
Magnus Pfeffer
 
PDF
Automatic creation of mappings between classification systems for bibliograph...
Magnus Pfeffer
 
PPTX
Relational to Graph - Import
Neo4j
 
IEEE IRI 16 - Clustering Web Pages based on Structure and Style Similarity
Thamme Gowda
 
Joins in a distributed world - Lucian Precup
distributed matters
 
“Open Data Web” – A Linked Open Data Repository Built with CKAN
Chengjen Lee
 
The FLuID Meta Model: Incrementally Compute Schema-level Indices for the Web...
Till Blume
 
Are Linked Datasets fit for Open-domain Question Answering? A Quality Assessment
Harsh Thakkar
 
Detection of Related Semantic Datasets Based on Frequent Subgraph Mining
Mikel Emaldi Manrique
 
2015 TaPP - Interoperability for Provenance-aware Databases using PROV and JSON
Boris Glavic
 
eNanoMapper database, search tools and templates
Nina Jeliazkova
 
SF Python Meetup: TextRank in Python
Paco Nathan
 
RDF-Gen: Generating RDF from streaming and archival data
Giorgos Santipantakis
 
Achieving time effective federated information from scalable rdf data using s...
తేజ దండిభట్ల
 
Interactive exploration of complex relational data sets in a web - SemWeb.Pro...
Logilab
 
Oshs_9_11_2015
Béatrice Bouchou
 
Graph databases
Karol Grzegorczyk
 
Elasticsearch - basics and beyond
Ernesto Reig
 
Optimized index structures for querying rdf from the web
Mahdi Atawneh
 
Instance Matching
Robert Isele
 
Automatic creation of mappings between classification systems
Magnus Pfeffer
 
Automatic creation of mappings between classification systems for bibliograph...
Magnus Pfeffer
 
Relational to Graph - Import
Neo4j
 

Viewers also liked (20)

PPTX
Capacity planning
Dimitar Bakardzhiev
 
PPTX
Tools for capacity planning, measurement of capacity, capacity planning process
Rohan Monis
 
PDF
Getting Things Done met David Allen - Masterclass met ScaleUp Company 22 apri...
Erno Hannink
 
PPTX
Telework: The Who, What, When, Where & Why
Global Workplace Analytics
 
PDF
Nationale Groeimasterclass 2015 een terugblik #ngmc2015 #ngm15
Erno Hannink
 
PDF
The Focused Creator: Learn the art of follow through and achieve your goals
Jake Jorgovan
 
PPT
Direct Marketing 101 Workshop 5
patmcgraw
 
PPT
Strategic plan on a page
NeighborSpace
 
PPT
Direct Marketing 101 Workshop 1
patmcgraw
 
PDF
Understanding People: Personality Traits and DISC Model
Mohammad Tawfik
 
PDF
12 Habits of Effective Lean Leadership
Em Campbell-Pretty
 
PPTX
Disc Intro for teambuild
Brent Green
 
PDF
Ceo 4 dx of execution
AIESEC
 
PPTX
DiSC in a Nutshell - Quick View
iamcollinlam
 
PPT
Network Marketing Explained
traderbiju
 
PPT
Meeting Management
bucpunar
 
PDF
Health & Well-being: Science, Medical Education And Public Health
The Rockefeller Foundation
 
PPTX
Persuasion
Nurulhuda Ismail
 
PPTX
Leading effective meetings slides
Laura Staley
 
PPSX
7 Steps Project Execution Process
C K Vishwakarma
 
Capacity planning
Dimitar Bakardzhiev
 
Tools for capacity planning, measurement of capacity, capacity planning process
Rohan Monis
 
Getting Things Done met David Allen - Masterclass met ScaleUp Company 22 apri...
Erno Hannink
 
Telework: The Who, What, When, Where & Why
Global Workplace Analytics
 
Nationale Groeimasterclass 2015 een terugblik #ngmc2015 #ngm15
Erno Hannink
 
The Focused Creator: Learn the art of follow through and achieve your goals
Jake Jorgovan
 
Direct Marketing 101 Workshop 5
patmcgraw
 
Strategic plan on a page
NeighborSpace
 
Direct Marketing 101 Workshop 1
patmcgraw
 
Understanding People: Personality Traits and DISC Model
Mohammad Tawfik
 
12 Habits of Effective Lean Leadership
Em Campbell-Pretty
 
Disc Intro for teambuild
Brent Green
 
Ceo 4 dx of execution
AIESEC
 
DiSC in a Nutshell - Quick View
iamcollinlam
 
Network Marketing Explained
traderbiju
 
Meeting Management
bucpunar
 
Health & Well-being: Science, Medical Education And Public Health
The Rockefeller Foundation
 
Persuasion
Nurulhuda Ismail
 
Leading effective meetings slides
Laura Staley
 
7 Steps Project Execution Process
C K Vishwakarma
 
Ad

Similar to Tutorial "Linked Data Query Processing" Part 4 "Execution Process" (WWW 2013 Ed.) (20)

PDF
Go and Uber’s time series database m3
Rob Skillington
 
PDF
Monitoring Kafka w/ Prometheus
kawamuray
 
PDF
Creating Open Data with Open Source (beta2)
Sammy Fung
 
PPTX
PhD Thesis: Mining abstractions in scientific workflows
dgarijo
 
PPT
Collaborative Workflow Development and Experimentation in the Digital Humanities
cneudecker
 
PPTX
How to Reduce Database Load with Sparse Branches
Perforce
 
PDF
Querying Linked Data with SPARQL (2010)
Olaf Hartig
 
PDF
.NET @ apache.org
Ted Husted
 
PDF
Reactive database access with Slick3
takezoe
 
PDF
Virtuoso RDF Triple Store Analysis Benchmark & mapping tools RDF / OO
Paolo Cristofaro
 
PDF
Apache Spark Structured Streaming for Machine Learning - StrataConf 2016
Holden Karau
 
PDF
JCConf 2016 - Google Dataflow 小試
Simon Su
 
PPTX
About Flink streaming
용휘 김
 
PDF
Web Traffic Time Series Forecasting
BillTubbs
 
PDF
Plugin-based software design with Ruby and RubyGems
Sadayuki Furuhashi
 
PDF
Everything You Wanted to Know About Databases (Keith).pdf
All Things Open
 
PPTX
An Inter-Wiki Page Data Processor for a M2M System @Matsue, 1sep., Eskm2013
Takashi Yamanoue
 
PDF
OQGraph @ SCaLE 11x 2013
Antony T Curtis
 
PDF
Data Science in the Cloud @StitchFix
C4Media
 
PDF
Встраиваем python для появления аналитики в проекте на С++. Александр Боргард...
corehard_by
 
Go and Uber’s time series database m3
Rob Skillington
 
Monitoring Kafka w/ Prometheus
kawamuray
 
Creating Open Data with Open Source (beta2)
Sammy Fung
 
PhD Thesis: Mining abstractions in scientific workflows
dgarijo
 
Collaborative Workflow Development and Experimentation in the Digital Humanities
cneudecker
 
How to Reduce Database Load with Sparse Branches
Perforce
 
Querying Linked Data with SPARQL (2010)
Olaf Hartig
 
.NET @ apache.org
Ted Husted
 
Reactive database access with Slick3
takezoe
 
Virtuoso RDF Triple Store Analysis Benchmark & mapping tools RDF / OO
Paolo Cristofaro
 
Apache Spark Structured Streaming for Machine Learning - StrataConf 2016
Holden Karau
 
JCConf 2016 - Google Dataflow 小試
Simon Su
 
About Flink streaming
용휘 김
 
Web Traffic Time Series Forecasting
BillTubbs
 
Plugin-based software design with Ruby and RubyGems
Sadayuki Furuhashi
 
Everything You Wanted to Know About Databases (Keith).pdf
All Things Open
 
An Inter-Wiki Page Data Processor for a M2M System @Matsue, 1sep., Eskm2013
Takashi Yamanoue
 
OQGraph @ SCaLE 11x 2013
Antony T Curtis
 
Data Science in the Cloud @StitchFix
C4Media
 
Встраиваем python для появления аналитики в проекте на С++. Александр Боргард...
corehard_by
 
Ad

More from Olaf Hartig (20)

PDF
A Context-Based Semantics for SPARQL Property Paths over the Web
Olaf Hartig
 
PDF
Tutorial "An Introduction to SPARQL and Queries over Linked Data" Chapter 3 (...
Olaf Hartig
 
PDF
Tutorial "An Introduction to SPARQL and Queries over Linked Data" Chapter 2 (...
Olaf Hartig
 
PDF
Tutorial "An Introduction to SPARQL and Queries over Linked Data" Chapter 1 (...
Olaf Hartig
 
ODP
An Overview on PROV-AQ: Provenance Access and Query
Olaf Hartig
 
PDF
(An Overview on) Linked Data Management and SPARQL Querying (ISSLOD2011)
Olaf Hartig
 
PDF
Zero-Knowledge Query Planning for an Iterator Implementation of Link Traversa...
Olaf Hartig
 
PDF
The Impact of Data Caching of on Query Execution for Linked Data
Olaf Hartig
 
PDF
How Caching Improves Efficiency and Result Completeness for Querying Linked Data
Olaf Hartig
 
PDF
A Main Memory Index Structure to Query Linked Data
Olaf Hartig
 
PDF
Towards a Data-Centric Notion of Trust in the Semantic Web (A Position Statem...
Olaf Hartig
 
PDF
Brief Introduction to the Provenance Vocabulary (for W3C prov-xg)
Olaf Hartig
 
PDF
Answers to usual issues in getting started with consuming Linked Data (2010)
Olaf Hartig
 
PDF
Linked Data on the Web
Olaf Hartig
 
PDF
Executing SPARQL Queries of the Web of Linked Data
Olaf Hartig
 
PDF
Using Web Data Provenance for Quality Assessment
Olaf Hartig
 
PDF
Answers to usual issues in getting started with consuming Linked Data
Olaf Hartig
 
PDF
Querying Linked Data with SPARQL
Olaf Hartig
 
PDF
Querying Trust in RDF Data with tSPARQL
Olaf Hartig
 
PDF
Database Researchers Map
Olaf Hartig
 
A Context-Based Semantics for SPARQL Property Paths over the Web
Olaf Hartig
 
Tutorial "An Introduction to SPARQL and Queries over Linked Data" Chapter 3 (...
Olaf Hartig
 
Tutorial "An Introduction to SPARQL and Queries over Linked Data" Chapter 2 (...
Olaf Hartig
 
Tutorial "An Introduction to SPARQL and Queries over Linked Data" Chapter 1 (...
Olaf Hartig
 
An Overview on PROV-AQ: Provenance Access and Query
Olaf Hartig
 
(An Overview on) Linked Data Management and SPARQL Querying (ISSLOD2011)
Olaf Hartig
 
Zero-Knowledge Query Planning for an Iterator Implementation of Link Traversa...
Olaf Hartig
 
The Impact of Data Caching of on Query Execution for Linked Data
Olaf Hartig
 
How Caching Improves Efficiency and Result Completeness for Querying Linked Data
Olaf Hartig
 
A Main Memory Index Structure to Query Linked Data
Olaf Hartig
 
Towards a Data-Centric Notion of Trust in the Semantic Web (A Position Statem...
Olaf Hartig
 
Brief Introduction to the Provenance Vocabulary (for W3C prov-xg)
Olaf Hartig
 
Answers to usual issues in getting started with consuming Linked Data (2010)
Olaf Hartig
 
Linked Data on the Web
Olaf Hartig
 
Executing SPARQL Queries of the Web of Linked Data
Olaf Hartig
 
Using Web Data Provenance for Quality Assessment
Olaf Hartig
 
Answers to usual issues in getting started with consuming Linked Data
Olaf Hartig
 
Querying Linked Data with SPARQL
Olaf Hartig
 
Querying Trust in RDF Data with tSPARQL
Olaf Hartig
 
Database Researchers Map
Olaf Hartig
 

Recently uploaded (20)

PDF
Agentic AI lifecycle for Enterprise Hyper-Automation
Debmalya Biswas
 
PDF
AI Agents in the Cloud: The Rise of Agentic Cloud Architecture
Lilly Gracia
 
PDF
SIZING YOUR AIR CONDITIONER---A PRACTICAL GUIDE.pdf
Muhammad Rizwan Akram
 
PDF
UPDF - AI PDF Editor & Converter Key Features
DealFuel
 
PDF
Book industry state of the nation 2025 - Tech Forum 2025
BookNet Canada
 
PDF
Mastering Financial Management in Direct Selling
Epixel MLM Software
 
PDF
UiPath DevConnect 2025: Agentic Automation Community User Group Meeting
DianaGray10
 
PDF
What’s my job again? Slides from Mark Simos talk at 2025 Tampa BSides
Mark Simos
 
PDF
How do you fast track Agentic automation use cases discovery?
DianaGray10
 
PDF
Go Concurrency Real-World Patterns, Pitfalls, and Playground Battles.pdf
Emily Achieng
 
PDF
[Newgen] NewgenONE Marvin Brochure 1.pdf
darshakparmar
 
PPTX
Mastering ODC + Okta Configuration - Chennai OSUG
HathiMaryA
 
PPTX
New ThousandEyes Product Innovations: Cisco Live June 2025
ThousandEyes
 
PDF
Peak of Data & AI Encore AI-Enhanced Workflows for the Real World
Safe Software
 
DOCX
Cryptography Quiz: test your knowledge of this important security concept.
Rajni Bhardwaj Grover
 
PPTX
Digital Circuits, important subject in CS
contactparinay1
 
PDF
The Rise of AI and IoT in Mobile App Tech.pdf
IMG Global Infotech
 
PDF
“Voice Interfaces on a Budget: Building Real-time Speech Recognition on Low-c...
Edge AI and Vision Alliance
 
PPTX
Future Tech Innovations 2025 – A TechLists Insight
TechLists
 
PDF
NASA A Researcher’s Guide to International Space Station : Physical Sciences ...
Dr. PANKAJ DHUSSA
 
Agentic AI lifecycle for Enterprise Hyper-Automation
Debmalya Biswas
 
AI Agents in the Cloud: The Rise of Agentic Cloud Architecture
Lilly Gracia
 
SIZING YOUR AIR CONDITIONER---A PRACTICAL GUIDE.pdf
Muhammad Rizwan Akram
 
UPDF - AI PDF Editor & Converter Key Features
DealFuel
 
Book industry state of the nation 2025 - Tech Forum 2025
BookNet Canada
 
Mastering Financial Management in Direct Selling
Epixel MLM Software
 
UiPath DevConnect 2025: Agentic Automation Community User Group Meeting
DianaGray10
 
What’s my job again? Slides from Mark Simos talk at 2025 Tampa BSides
Mark Simos
 
How do you fast track Agentic automation use cases discovery?
DianaGray10
 
Go Concurrency Real-World Patterns, Pitfalls, and Playground Battles.pdf
Emily Achieng
 
[Newgen] NewgenONE Marvin Brochure 1.pdf
darshakparmar
 
Mastering ODC + Okta Configuration - Chennai OSUG
HathiMaryA
 
New ThousandEyes Product Innovations: Cisco Live June 2025
ThousandEyes
 
Peak of Data & AI Encore AI-Enhanced Workflows for the Real World
Safe Software
 
Cryptography Quiz: test your knowledge of this important security concept.
Rajni Bhardwaj Grover
 
Digital Circuits, important subject in CS
contactparinay1
 
The Rise of AI and IoT in Mobile App Tech.pdf
IMG Global Infotech
 
“Voice Interfaces on a Budget: Building Real-time Speech Recognition on Low-c...
Edge AI and Vision Alliance
 
Future Tech Innovations 2025 – A TechLists Insight
TechLists
 
NASA A Researcher’s Guide to International Space Station : Physical Sciences ...
Dr. PANKAJ DHUSSA
 

Tutorial "Linked Data Query Processing" Part 4 "Execution Process" (WWW 2013 Ed.)

  • 1. Linked Data Query Processing Tutorial at the 22nd International World Wide Web Conference (WWW 2013) May 14, 2013 http://db.uwaterloo.ca/LDQTut2013/ 4. Execution Process Olaf Hartig University of Waterloo
  • 2. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 2 Outline  Separated Execution  Integrated Execution ➢ General Idea and Properties ➢ Push-Based Implementation [LT10, LT11] ➢ Link Traversal Based Query Execution
  • 3. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 3 … clearly separate data retrieval and result construction into two consecutive phases Separated Execution Approaches Query-local data http://mdb.../Paul http://geo.../Berlin http://mdb.../Ric http://geo.../Rome ?loc?actor GET http://.../movie2449 1 2
  • 4. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 4 Properties of Separated Execution Query-local data http://mdb.../Paul http://geo.../Berlin http://mdb.../Ric http://geo.../Rome ?loc?actor GET http://.../movie2449 1 2 ● Advantage: straightforward to implement ● Can be combined with any source selection strategy ● A traditional query execution plan might then be used for constructing the result ● Downside: First solutions cannot be reported before data retrieval has been completed
  • 5. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 5 Query-local data http://mdb.../Paul http://geo.../Berlin http://mdb.../Ric http://geo.../Rome ?loc?actor GET http://.../movie2449 Integrated Execution Approaches … intertwine data retrieval and result construction
  • 6. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 6 Query-local data http://mdb.../Paul http://geo.../Berlin http://mdb.../Ric http://geo.../Rome ?loc?actor GET http://.../movie2449 Integrated Execution Approaches ● Implementations may report first solutions early ● For monotonic queries ● Implementations may process data in a streaming manner ● May require less query-local memory (b/c query-local dataset need not be materialized) ● Can also be combined with any source selection strategy … intertwine data retrieval and result construction
  • 7. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 7 Outline  Separated Execution  Integrated Execution ➢ General Idea and Properties ➢ Push-Based Implementation [LT10, LT11] ➢ Link Traversal Based Query Execution √ √
  • 8. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 8 Push-Based Implementation TP Operator TP Operator TP Operator Join Operator Join Operator ● Each operator runs as an asynchronous thread ● Connected via message queues Data Retrieval Operator
  • 9. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 9 Push-Based Implementation TP Operator TP Operator TP Operator Join Operator Join Operator ● Each operator runs as an asynchronous thread ● Connected via message queues ● Data retrieval operator ● May apply any possible source selection strategy ● Usually, implemented via multiple lookup threads ● Pushes any incoming matching triple to the corresponding triple pattern operator Data Retrieval Operator
  • 10. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 10 Push-Based Implementation (Example) TP Operator ( ?p, interested in, ?i ) TP Operator ( ?p, affiliated with, orgax ) TP Operator ( ?i, rdf:type, Book ) Join Operator Join Operator Data Retrieval Operator
  • 11. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 11 Push-Based Implementation (Example) TP Operator ( ?p, interested in, ?i ) TP Operator ( ?p, affiliated with, orgax ) TP Operator ( ?i, rdf:type, Book ) Join Operator Join Operator Data Retrieval Operator { ?p → alice } ( alice, affiliated with, orgax ) ( alice, affiliated with, orgax )
  • 12. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 12 Push-Based Implementation (Example) TP Operator ( ?p, interested in, ?i ) TP Operator ( ?p, affiliated with, orgax ) TP Operator ( ?i, rdf:type, Book ) Join Operator Join Operator Data Retrieval Operator { ?p → alice } ( alice, interested in, yoga ) { ?p → alice, ?i → yoga } ( alice, interested in, yoga ) { ?p → alice, ?i → yoga }
  • 13. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 13 Symmetric Hash Join (SHJ) SHJ Operator # Bucket 1 {?p → alice,?i → yoga} 2 3 {?p → bob,?i → tea}, {?p → bob,?i → chess} # Bucket 1 {?p → alice,?o → orgax} 2 {?p → eve,?o → orgax} 3 ● Maintains a hash table for each input ● Same hash function for both tables ● Each bucket contains a set of input solutions
  • 14. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 14 How SHJ Processes Incoming Solution SHJ Operator # Bucket 1 {?p → alice,?i → yoga} 2 3 {?p → bob,?i → tea}, {?p → bob,?i → chess} # Bucket 1 {?p → alice,?o → orgax} 2 {?p → eve,?o → orgax} 3 ● Hash the value that μinput binds to the join variable μinput = {?p → bob,?o → acme}
  • 15. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 15 How SHJ Processes Incoming Solution SHJ Operator # Bucket 1 {?p → alice,?i → yoga} 2 3 {?p → bob,?i → tea}, {?p → bob,?i → chess} # Bucket 1 {?p → alice,?o → orgax} 2 {?p → eve,?o → orgax} 3 ● Hash the value that μinput binds to the join variable ● Insert μinput into the corresponding input hash table hfct(bob) = 3μinput = {?p → bob,?o → acme}
  • 16. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 16 How SHJ Processes Incoming Solution SHJ Operator # Bucket 1 {?p → alice,?i → yoga} 2 3 {?p → bob,?i → tea}, {?p → bob,?i → chess} # Bucket 1 {?p → alice,?o → orgax} 2 {?p → eve,?o → orgax} 3 {?p → bob,?o → acme} ● Hash the value that μinput binds to the join variable ● Insert μinput into the corresponding input hash table ● Probe into the other hash table to find join candidates ● Merge μinput with each join candidate, push results to output {?p → bob,?o → acme,?i → tea} hfct(bob) = 3 {?p → bob,?o → acme,?i → chess} μinput = {?p → bob,?o → acme}
  • 17. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 17 Outline  Separated Execution  Integrated Execution ➢ General Idea and Properties ➢ Push-Based Implementation [LT10, LT11] ➢ Link Traversal Based Query Execution √ √ √
  • 18. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 18 Link Traversal Based Query Execution … is the combination of integrated execution and live exploration (i.e., the push-based approach can be used as an implementation of LTBQE) An even closer combination presents the following pull-based implementation approach ...
  • 19. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 19 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query Seed: <http://.../orgaX> Pipeline of Link Traversing Iterators I3 I2 I1
  • 20. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 20 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 Seed: <http://.../orgaX> Pipeline of Link Traversing Iterators
  • 21. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 21 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query Seed: <http://.../orgaX> tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 query-local dataset Pipeline of Link Traversing Iterators
  • 22. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 22 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query Seed: <http://.../orgaX> tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 query-local dataset Pipeline of Link Traversing Iterators
  • 23. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 23 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query Seed: <http://.../orgaX> tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 query-local dataset Pipeline of Link Traversing Iterators
  • 24. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 24 query-local dataset Next? tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 Pipeline of Link Traversing Iterators
  • 25. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 25 query-local dataset Next? Next? tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 Pipeline of Link Traversing Iterators
  • 26. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 26 query-local dataset Next? Next? Next? tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 Pipeline of Link Traversing Iterators
  • 27. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 27 Next? Next? Next? tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 : <http://.../alice> ex:affiliated_with <http://.../orgaX> : query-local dataset Pipeline of Link Traversing Iterators
  • 28. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 28 Next? tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 : <http://.../alice> ex:affiliated_with <http://.../orgaX> : query-local dataset { ?p = <http://.../alice> } Next? tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 Pipeline of Link Traversing Iterators
  • 29. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 29 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 query-local dataset { ?p = <http://.../alice> } Next? Next? tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 Pipeline of Link Traversing Iterators
  • 30. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 30 Next? Next? tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 query-local dataset { ?p = <http://.../alice> } Pipeline of Link Traversing Iterators
  • 31. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 31 Next? Next? tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 query-local dataset { ?p = <http://.../alice> } : <http://.../alice> ex:interested_in <http://.../b1> : Pipeline of Link Traversing Iterators
  • 32. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 32 Next? Next? tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> } : <http://.../alice> ex:interested_in <http://.../b1> : query-local dataset Pipeline of Link Traversing Iterators
  • 33. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 33 Next? tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> } : <http://.../alice> ex:interested_in <http://.../b1> : query-local dataset { ?p = <http://.../alice> , ?b = <http://.../b1> } Pipeline of Link Traversing Iterators
  • 34. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 34 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 Next? { ?p = <http://.../alice> } query-local dataset tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> , ?b = <http://.../b1> } Pipeline of Link Traversing Iterators
  • 35. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 35 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 Next? { ?p = <http://.../alice> } query-local dataset tp3 = ( ?b , rdf:type , <http://.../Book> ) tp3 ' = ( <http://.../b1> , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> , ?b = <http://.../b1> } Pipeline of Link Traversing Iterators
  • 36. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 36 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 Next? { ?p = <http://.../alice> } tp3 = ( ?b , rdf:type , <http://.../Book> ) tp3 ' = ( <http://.../b1> , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> , ?b = <http://.../b1> } query-local dataset : <http://.../Book> rdfs:subClassOf <http://.../CreativeWork> : Pipeline of Link Traversing Iterators
  • 37. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 37 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 Next? { ?p = <http://.../alice> } tp3 = ( ?b , rdf:type , <http://.../Book> ) tp3 ' = ( <http://.../b1> , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> , ?b = <http://.../b1> } query-local dataset : <http://.../b1> rdf:type <http://.../Book> : Pipeline of Link Traversing Iterators
  • 38. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 38 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 Next? { ?p = <http://.../alice> } tp3 = ( ?b , rdf:type , <http://.../Book> ) tp3 ' = ( <http://.../b1> , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> , ?b = <http://.../b1> } : <http://.../b1> rdf:type <http://.../Book> : query-local dataset Pipeline of Link Traversing Iterators
  • 39. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 39 { ?p = <http://.../alice> , ?b = <http://.../b1> } tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 { ?p = <http://.../alice> } tp3 = ( ?b , rdf:type , <http://.../Book> ) tp3 ' = ( <http://.../b1> , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 { ?p = <http://.../alice> , ?b = <http://.../b1> } query-local dataset Pipeline of Link Traversing Iterators
  • 40. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 40 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp3 = ( ?b , rdf:type , <http://.../Book> ) tp3 ' = ( <http://.../b1> , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 Pipeline of Link Traversing Iterators { ?p = <http://.../alice> } { ?p = <http://.../alice> , ?b = <http://.../b1> } Next? query-local dataset
  • 41. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 41 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 Pipeline of Link Traversing Iterators { ?p = <http://.../alice> } Next? Next? query-local dataset
  • 42. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 42 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX> ) I1 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp2 = ( ?p , ex:interested_in , ?b ) tp2 ' = ( <http://.../alice> , ex:interested_in , ?b ) I2 Pipeline of Link Traversing Iterators { ?p = <http://.../alice> } Next? Next? query-local dataset
  • 43. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 43 ● Deeply intertwines integrated execution + live exploration ● Data retrieval is a side-effect of calling link traversing iterators ● No need for a separate data retrieval operator ● Makes adding LTBQE to existing SPARQL engines easy ● Because those usually use iterators for execution ● Caveat: does not achieve the full flexibility that is possible with LTBQE ● Iterators evaluate the triple patterns in a fixed order ● Iterators discard each input solution after using it Pipeline of Link Traversing Iterators
  • 44. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 44 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query Seed: <http://.../orgaX> tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?b , rdf:type , <http://.../Book> ) I3 tp1 = ( ?p , ex:affiliated_with , <http://.../orgaX>) I1 Alternative Execution Order
  • 45. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 45 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query Seed: <http://.../orgaX> tp2 = ( ?p , ex:interested_in , ?b ) I2 Alternative Execution Order tp3 = ( ?p , ex:affiliated_with , <http://.../orgaX>) I3 tp1 = ( ?b , rdf:type , <http://.../Book> ) I1
  • 46. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 46 ?p ex:affiliated_with <http://.../orgaX> ?p ex:interested_in ?b ?b rdf:type <http://.../Book> Query Seed: <http://.../orgaX> tp2 = ( ?p , ex:interested_in , ?b ) I2 query-local dataset Iterator Based Execution tp1 = ( ?b , rdf:type , <http://.../Book> ) I1 tp3 = ( ?p , ex:affiliated_with , <http://.../orgaX>) I3
  • 47. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 47 tp3 = ( ?p , ex:affiliated_with , <http://.../orgaX>) I3 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp1 = ( ?b , rdf:type , <http://.../Book> ) I1 : <http://.../alice> ex:affiliated_with <http://.../orgaX> : query-local dataset Alternative Execution Order
  • 48. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 48 Next?: <http://.../alice> ex:affiliated_with <http://.../orgaX> : query-local dataset Next? tp1 = ( ?b , rdf:type , <http://.../Book> ) I1 Alternative Execution Order Next? tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?p , ex:affiliated_with , <http://.../orgaX>) I3
  • 49. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 49 Next?: <http://.../alice> ex:affiliated_with <http://.../orgaX> : query-local dataset Next? tp1 = ( ?b , rdf:type , <http://.../Book> ) I1 Alternative Execution Order END! tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?p , ex:affiliated_with , <http://.../orgaX>) I3
  • 50. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 50 query-local dataset tp1 = ( ?b , rdf:type , <http://.../Book> ) I1 tp2 = ( ?p , ex:interested_in , ?b ) I2 tp3 = ( ?p , ex:affiliated_with , <http://.../orgaX>) I3 END! END! END! Computed query result may depend on the order of triple patterns = logical query execution plan Alternative Execution Order
  • 51. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 51 Outline  Separated Execution  Integrated Execution ➢ General Idea and Properties ➢ Push-Based Implementation [LT10, LT11] ➢ Link Traversal Based Query Execution √ √ √ √ Next part: 5. Query Planning and Optimization ...
  • 52. WWW 2013 Tutorial on Linked Data Query Processing [ Execution Process ] 52 These slides have been created by Olaf Hartig for the WWW 2013 tutorial on Link Data Query Processing Tutorial Website: http://db.uwaterloo.ca/LDQTut2013/ This work is licensed under a Creative Commons Attribution-Share Alike 3.0 License (http://creativecommons.org/licenses/by-sa/3.0/)