NUMA optimized Parallel Breadth first Search on Multicore Single node System
2 likes1,030 views
The document discusses NUMA-optimized parallel breadth-first search (BFS) on multicore single node systems. It presents a NUMA-aware hybrid BFS algorithm that partitions graphs column-wise and binds partitions to local memory to reduce unnecessary edge traversals and remote memory access. Numerical results show the NUMA algorithm scales well up to 64 threads and achieves 11.15 GTEPS, a 2.2x speedup over the original hybrid algorithm.
![Contribution
Efficient hybrid algorithm of BFS [Beamer2011,2012]
reduces unnecessary edge traversal
Our proposal
- NUMA-optimized hybrid algorithm
- Improves locality of memory access
. Library for considering NUMA carefully
. Column-wise graph partitioning](https://image.slidesharecdn.com/numaoptimizedparallelbreadthfirst-150618151707-lva1-app6892/85/NUMA-optimized-Parallel-Breadth-first-Search-on-Multicore-Single-node-System-8-320.jpg)
![References
Parallel Breadth-First Search on Distributed Memory Systems
[Aydın Buluç Kamesh Madduri Computational Research Division Lawrence Berkeley National
Laboratory Berkeley, CA {ABuluc, KMadduri}@lbl.gov]
A Scalable Distributed Parallel Breadth-First Search Algorithm on
BlueGene/L
[Andy Yooy Edmond Chowx Keith Hendersony William McLendonz Bruce Hendricksonz ÄUmit C»
atalyÄurek]
Distributed Memory Breadth-First Search Revisited: Enabling Bottom-Up
Search
[Scott Beamer Aydn Buluc Krste Asanovic David A. Patterson]
Distributed Breadth First Search
[CSE 6220, Spring 2013 Georgia Institute of Technology April 18 -- Guest lecture by Anita Zakrzewska]
Evaluation and Optimization of Breadth-First Search on NUMA Cluster
[Zehan Cui1,2, Licheng Chen1,2, Mingyu Chen1, Yungang Bao1, Yongbing Huang1,2, Huiwei Lv1,2 1 State
Key Laboratory of Computer Architecture, Institute of Computing Technology, Chinese Academy of
Sciences 2 Graduate School of Chinese Academy of Sciences {cuizehan, chenlicheng, cmy, baoyg,
huangyongbing, lvhuiwei}@ict.ac.cn]](https://image.slidesharecdn.com/numaoptimizedparallelbreadthfirst-150618151707-lva1-app6892/85/NUMA-optimized-Parallel-Breadth-first-Search-on-Multicore-Single-node-System-36-320.jpg)
![References
Scaling Techniques for Massive Scale-Free Graphs in Distributed (External)
Memory
[Roger Pearcey, Maya Gokhaley, Nancy M. Amato Parasol Laboratory; Dept. of Computer Science and
Engineering Texas A&M University; College Station, TX yCenter for Applied Scientific Computing
Lawrence Livermore National Laboratory; Livermore, CA frpearce, mayag@llnl.gov frpearce,
amatog@cse.tamu.edu]
Reducing Communication in Parallel Breadth-First Search on Distributed
Memory Systems
[Huiwei Luy, Guangming Tan, Mingyu Chen, Ninghui Sun State Key Laboratory of Computer
Architecture, Institute of Computing Technology, Chinese Academy of Sciences yArgonne National
Laboratory Email: huiweilu@mcs.anl.gov, tgm@ict.ac.cn, cmy@ict.ac.cn, snh@ncic.ac.cn]
Level-Synchronous Parallel Breadth-First Search Algorithms For Multicore
and Multiprocessor Systems
[Rudolf Berrendorf and Matthias Makulla Computer Science Department Bonn-Rhein-Sieg University Sankt
Augustin, Germany e-mail: rudolf.berrendorf@h-brs.de, mathias.makulla@h-brs.de]](https://image.slidesharecdn.com/numaoptimizedparallelbreadthfirst-150618151707-lva1-app6892/85/NUMA-optimized-Parallel-Breadth-first-Search-on-Multicore-Single-node-System-37-320.jpg)
NUMA optimized Parallel Breadth first Search on Multicore Single node System
- 1. NUMA optimized Parallel Breadth first Search on Multicore Single node System Mohammad Opada Al-Bosh Mohammad Tahsin Al-Shalabi Ruba Break Mariam Al-kassar Nagham Ballan
- 2. Outline Background Breadth-first Search (BFS) NUMAI optimized parallel BFS Numerical Results Conclusion
- 3. Background Large scale graph in various fields US Road network: 58 million edges Twitter follow-ship : 1.47 billion edges Neuronal network : 100 trillion edges large
- 4. Background Fast and scalable graph processing by using HPC
- 5. Importance of graph processing Application field Transportation Social network Cyber-security Bioinformatics - Step 3: • concurrent search (breadth first search) • optimization (single source shortest path) • edge-oriented (maximal independent set)
- 6. Breadth first search BFS is important and fundamental graph processing – Obtains relationship of distance (hops) as standIalone – Many algorithm (BC, Max.flow, Max.independent set) Problems of Fast and scalable computation BFS - low arithmetic intensity - irregular memory accesses
- 7. Graph500 Benchmark Measures computer performance using TEPS ratio in graph processing such as BFS (Breath-first search) TEPS ratio = # of Traversed edges per second
- 8. Contribution Efficient hybrid algorithm of BFS [Beamer2011,2012] reduces unnecessary edge traversal Our proposal - NUMA-optimized hybrid algorithm - Improves locality of memory access . Library for considering NUMA carefully . Column-wise graph partitioning
- 9. Example 4-way Intel Xeon E5 (64 CPU cores) • Scalable: Scale well up to 64 threads. • Fast: 11.15 GTEPS and 2.2x speedup compared with original Hybrid algorithm
- 10. Outline Background Breadth-first Search (BFS) NUMAI optimized parallel BFS Numerical Results Conclusion
- 11. Breadth first Search (BFS) Obtain level of each vertices from source vertex Level = certain # of hops away from the source Input: Graph G and source Output: Tree with root as source
- 12. Hybrid BFS for low diameter graph Efficient for Low diameter graph – scale free and/or small world property such as social network. At higher ranks in Graph500 benchmark Hybrid algorithm - combines top-down algorithm and bottom-up algorithm – reduces unnecessary edge traversal
- 13. Hybrid algorithm Top down algorithm Bottom up algorithm Efficient for a small-frontier Efficient for a large-frontier
- 14. Top down algorithm Explores outgoing edges of frontier queue QF Appends unvisited vertices into neighbor queue QN Efficient for a small frontier • Has an unnecessary edge traversal for a large frontier
- 15. Bottom up algorithm Explores frontier queue QF from unvisited vertices. Appends adjacent vertices into neighbors QN Efficient for a large frontier • Has unnecessary edge traversal for a small frontier
- 16. Outline Background Breadth-first Search (BFS) NUMAI optimized parallel BFS Numerical Results Conclusion
- 17. How to speedup the hybrid algorithm? NUMA architecture – Non uniform memory access – Each CPU socket has a local RAM – Fast local RAM and slow non-local RAM 4 socket Intel Xeon E5 system
- 18. Frequent non local memory accesses on NUMA architecture
- 19. Difficulty of considering NUMA architecture 1. How does distribute graph and data to each local RAM? 2. How does bind partial graph and data to each NUMA unit?
- 20. ULIBC: Ubiquity Library for Intelligently Binding Cores 1. NUMACTL (command line tool, library for C/C++) 2. Intel compiler Thread Affinity Interface (API) 3. ULIBC (Our library, library for C/C++) – Processor ID : index of logical processor core – Package ID : index of CPU socket – Core ID : index of physical core in each CPU socket
- 21. NUMA-opt. Column wise Graph Partitioning . Divides G=(V,A) into partial Gk=(Vk,.Ak) and binds local RAM k - Ak is a set of adjacency list that holds incoming edges to Vk.
- 22. NUMA-optimized Top down Explores outgoing edges of frontier queue QF. Appends unvisited vertices into neighbor queue QN. Efficient for a small frontier • Has unnecessary edge traversal for a large frontier
- 23. Details of NUMA-optimized Top-down
- 24. NUMA-optimized Bottom-up Explores frontier queue QF from unvisited vertices. Appends adjacent vertices into neighbors QN. Efficient for a large frontier. • Has unnecessary edge traversal for a small frontier
- 25. Details of NUMA-optimized Bottom- up
- 26. Outline Background Breadth-first Search (BFS) NUMAI optimized parallel BFS Numerical Results Conclusion
- 27. Machine specification 4 way Intel Xeon E5. – CentOS 6.4 (Kernel 2.6.32) – GCC 4.4.7 – 64 logical CPU cores – 4 NUMA units x 16 logical cores 4 way AMD Opteron 6174. – Fedora 19 (Kernel 3.11.2) – GCC 4.8.1 – 48 CPU cores – 8 NUMA units x 6-core
- 28. TEPS ratio varied with problem size NUMA 2.2x speedups compared with original hybrid algorithm NUMA achieves11.15 GTEPS for Kronecker graph (SCALE26).
- 29. Strong scaling on Intel/AMD System Scale well up to # of threads as # of cores
- 30. Twitter network
- 31. Graph500 benchmark Fastest of single node on 4th list (Jue 2012) Fastest of CPU-based single-node on 6th list (June 2013)
- 32. 1st Green Graph500 list on June 2013 Measures power-efficient using TEPS/W ratio Results on various system such as Android ,Linux , and Mac. NUMA Small Data category
- 33. Outline Background Breadth-first Search (BFS) NUMAI optimized parallel BFS Numerical Results Conclusion
- 34. Conclusion NUMA-optimized Hybrid BFS algorithm – Reduces unnecessary edge traversals and remote RAM access carefully considering NUMA Numerical results on 4 way Intel Xeon – scales well up to 64 threads (scalable) – achieves 11.15 GTEPS (fast) – 2.2x speedup compared original Hybrid Graph500 and Green Graph500 – Fastest single-node in June 2012 – Most power-efficient in June 2013
- 35. Future work Further optimizing NUMA-optimized BFS$ Distributed-memory parallel computation
- 36. References Parallel Breadth-First Search on Distributed Memory Systems [Aydın Buluç Kamesh Madduri Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA {ABuluc, KMadduri}@lbl.gov] A Scalable Distributed Parallel Breadth-First Search Algorithm on BlueGene/L [Andy Yooy Edmond Chowx Keith Hendersony William McLendonz Bruce Hendricksonz ÄUmit C» atalyÄurek] Distributed Memory Breadth-First Search Revisited: Enabling Bottom-Up Search [Scott Beamer Aydn Buluc Krste Asanovic David A. Patterson] Distributed Breadth First Search [CSE 6220, Spring 2013 Georgia Institute of Technology April 18 -- Guest lecture by Anita Zakrzewska] Evaluation and Optimization of Breadth-First Search on NUMA Cluster [Zehan Cui1,2, Licheng Chen1,2, Mingyu Chen1, Yungang Bao1, Yongbing Huang1,2, Huiwei Lv1,2 1 State Key Laboratory of Computer Architecture, Institute of Computing Technology, Chinese Academy of Sciences 2 Graduate School of Chinese Academy of Sciences {cuizehan, chenlicheng, cmy, baoyg, huangyongbing, lvhuiwei}@ict.ac.cn]
- 37. References Scaling Techniques for Massive Scale-Free Graphs in Distributed (External) Memory [Roger Pearcey, Maya Gokhaley, Nancy M. Amato Parasol Laboratory; Dept. of Computer Science and Engineering Texas A&M University; College Station, TX yCenter for Applied Scientific Computing Lawrence Livermore National Laboratory; Livermore, CA frpearce, [email protected] frpearce, [email protected]] Reducing Communication in Parallel Breadth-First Search on Distributed Memory Systems [Huiwei Luy, Guangming Tan, Mingyu Chen, Ninghui Sun State Key Laboratory of Computer Architecture, Institute of Computing Technology, Chinese Academy of Sciences yArgonne National Laboratory Email: [email protected], [email protected], [email protected], [email protected]] Level-Synchronous Parallel Breadth-First Search Algorithms For Multicore and Multiprocessor Systems [Rudolf Berrendorf and Matthias Makulla Computer Science Department Bonn-Rhein-Sieg University Sankt Augustin, Germany e-mail: [email protected], [email protected]]