SchemEX - Creating the Yellow Pages for the Linked Open Data Cloud
Download as PPTX, PDF1 like1,102 views
The document discusses the ScheMex project, which aims to create a structured index for querying the Linked Open Data (LOD) cloud, particularly focusing on politicians and actors. It proposes a stream-based schema extraction approach, enabling efficient lookup and federated queries while managing memory use through a FIFO caching system. The results indicate that this method is scalable, effective with large datasets, and operates well on standard hardware.
SchemEX - Creating the Yellow Pages for the Linked Open Data Cloud
- 1. SchemEX Creating the Yellow Pages of the LOD Cloud Mathias Konrath, Thomas Gottron, Ansgar Scherp
- 2. Scenario • People who are politicians and actors • Who else? • Where do they live? • Whom do they know? SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 2 of 12
- 3. Problem • Execute those queries on the LOD cloud • No single federated query interface provided “politicians and actors” SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 3 of 12
- 4. Principle Solution • Suitable index structure for looking up sources “politicians and actors” SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 4 of 12
- 5. The Naive Approach 1. Download the entire LOD cloud 2. Put it into a (really) large triple store 3. Process the data and extract schema 4. Provide lookup - Big machinery - Late in processing the data - High effort to scale with LOD cloud SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 5 of 12
- 6. Yes, we can … SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 6 of 12
- 7. The SchemEX Approach • Stream-based schema extraction • While crawling the data FIFO LOD-Crawler Instance- RDF-Dump Cache RDF Triple Store RDBMS NxParser Nquad- Schema- Parser Schema Stream Extractor SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 7 of 12
- 8. Efficient Instance Cache • Observe a quadruple stream from LD spider • Ring queue, backed up by a hash map • Organizes triples with same subject URI • Dismiss oldest, when cache full (FIFO) → Runtime complexity O(1) SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 8 of 12
- 9. Building the Schema and Index RDF c1 c2 c3 … ck classes consistsOf Type TC1 TC2 … TCm clusters hasEQ Class p1 p2 EQC1 EQC2 … EQCn Equivalence classes hasDataSource … Data DS1 DS2 DS3 DS4 DS5 DSx sources SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 9 of 12
- 10. Computing SchemEX: TimBL Data Set • Analysis of a smaller data set • 11 M triples, TimBL’s FOAF profile • LDspider with ~ 2k triples / sec • Different cache sizes: 100, 1k, 10k, 50k, 100k • Compared SchemEX with reference schema • Index queries on all Types, TCs, EQCs • Good precision/recall ratio at 50k+ SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 10 of 12
- 11. Computing SchemEX: Full BTC 2011 Data Cache size: 50 k SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 11 of 12
- 12. Conclusions: SchemEX • Stream-based approach to schema extraction • Scalable to arbitrary amount of Linked Data • Applicable on commodity hardware (4GB RAM, standard single CPU) • Lookup-index to find relevant data sources • Support federated queries on the LOD cloud SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 12 of 12
- 13. BACKUP SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 13 of 12
- 14. SchemEX Computation: Window Sizes Runtime increases hardly with greater window sizes Crawled TimBL dataset Memory consumption scales (11M triples) with window size SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 14 of 12
- 15. SchemEX Quality: Precision SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 15 of 12
- 16. SchemEX Quality: Recall SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 16 of 12
- 17. Example Data Graph SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 17 of 12
- 18. Output Vocabulary: voiD SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 18 of 12
- 19. SchemEX Extraction: Progress Plot Type-cluster Equivalence classes Count ##processed instances processed 12 instances SchemEX – Mathias Konrath, Thomas Gottron, Ansgar Scherp 19 of