SPARQL graph pattern processing with apache spark
Proceedings of the Fifth International Workshop on Graph Data-management …, 2017•dl.acm.org
A common way to achieve scalability for processing SPARQL queries is to choose
MapReduce frameworks like Hadoop or Spark. Processing basic graph pattern (BGP)
expressions generating large join plans over distributed data partitions is a major challenge
in these frameworks. In this article, we study the use of two distributed join algorithms,
partitioned join and broadcast join, for the evaluation of BGP expressions on top of Apache
Spark. We compare five possible implementation and illustrate the importance of cautiously …
MapReduce frameworks like Hadoop or Spark. Processing basic graph pattern (BGP)
expressions generating large join plans over distributed data partitions is a major challenge
in these frameworks. In this article, we study the use of two distributed join algorithms,
partitioned join and broadcast join, for the evaluation of BGP expressions on top of Apache
Spark. We compare five possible implementation and illustrate the importance of cautiously …
A common way to achieve scalability for processing SPARQL queries is to choose MapReduce frameworks like Hadoop or Spark. Processing basic graph pattern (BGP) expressions generating large join plans over distributed data partitions is a major challenge in these frameworks. In this article, we study the use of two distributed join algorithms, partitioned join and broadcast join, for the evaluation of BGP expressions on top of Apache Spark. We compare five possible implementation and illustrate the importance of cautiously choosing the physical data storage layer and of the possibility to use both join algorithms to efficiently take account of existing data partitioning schemes. Our experimentations with different SPARQL benchmarks over real-world and synthetic workloads emphasize that hybrid join plans introduce more flexibility and often achieve better performance than single kind join plans.
Showing the best result for this search. See all results