Heterogeneous Particle based Simulation (SIGGRAPH ASIA 2011)
2 likes1,057 views
This document discusses using a heterogeneous CPU-GPU approach for particle-based simulations with particles of varying sizes. Large particles are handled by the CPU due to irregular workloads, while small particles are handled by the GPU due to uniform workloads. Optimizations include multi-threading large-small collisions on the CPU, spatially sorting small particles on the GPU to improve cache utilization, and load balancing work between the CPU and GPU. The approach leverages the strengths of the CPU and GPU on AMD's fusion architecture to efficiently simulate mixed particle systems.
Heterogeneous Particle based Simulation (SIGGRAPH ASIA 2011)
- 1. HETEROGENEOUS PARTICLE BASED SIMULATION Takahiro Harada, AMD
- 2. 2 Harada, Heterogeneous Particle-based Simulation ï§ Large number of particles ï§ Particles with identical size â Work granularity is almost the same â Good for the wide SIMD architecture PARTICLE BASED SIMULATION ON THE GPU Harada et al. 2007
- 3. 3 Harada, Heterogeneous Particle-based Simulation PARTICLE BASED SIMULATION ï§ Collision ï§ Integration ï§ Acceleration structure is used for efficient collide â Uniform grid â Suited for the GPU â Less divergence đđđđđđđđ = đđđ đŁ += đ đ âđĄ đ„ += đŁâđĄ đđŁ đđĄ = đ đ đđ„ đđĄ = đŁ
- 4. 4 Harada, Heterogeneous Particle-based Simulation DIVERGENCE ON SIMD 0 1 2 3 4 5 6 7 Void Kernel() { if(A) FuncA(); else if(B) FuncB(); else FuncC(); }
- 5. 5 Harada, Heterogeneous Particle-based Simulation PARTICLE BASED SIMULATION ON THE GPU ï§ Particle collision using a uniform grid 0 1 2 3 4 5 6 7 Void Kernel() { prepare(); collide(Cell0); collide(Cell1); collide(Cell2); collide(Cell3); collide(Cell4); collide(Cell5); collide(Cell6); collide(Cell7); collide(Cell8); } Cell0 Cell1 Cell2 Cell3 Cell4 Cell5 Cell6 Cell7 Cell8
- 6. 6 Harada, Heterogeneous Particle-based Simulation MIXED PARTICLE SIMULATION ï§ Not only small particles ï§ Difficulty for GPUs â Large particles interact with small particles â Large-large collision
- 7. 7 Harada, Heterogeneous Particle-based Simulation CHALLENGE ï§ Non uniform work granularity â Small-small(SS) collision ï§ Uniform, GPU â Large-large(LL) collision ï§ Non Uniform, CPU â Large-small(LS) collision ï§ Non Uniform, CPU
- 8. 8 Harada, Heterogeneous Particle-based Simulation FUSION ARCHITECTURE ï§ CPU and GPU are: â On the same die â Much closer â Efficient data sharing ï§ CPU and GPU are good at different works â CPU: serial computation, conditional branch â GPU: parallel computation ï§ Able to dispatch works to: â Serial work with varying granularity â CPU â Parallel work with the uniform granularity â GPU
- 9. 9 Harada, Heterogeneous Particle-based Simulation MIXED PARTICLE SIMULATION ï§ Benefit from Fusion Architecture â Different works in a simulation â CPU & GPU are working together â Shares data
- 10. 10 Harada, Heterogeneous Particle-based Simulation METHOD
- 11. 11 Harada, Heterogeneous Particle-based Simulation TWO SIMULATIONS ï§ Small particles ï§ Large particles Build Acc. Structure SS Collision S Integration Build Acc. Structure LL Collision L Integration LS Collision Position Velocity Force Grid Position Velocity Force
- 12. 12 Harada, Heterogeneous Particle-based Simulation ï§ Small particles ï§ Large particles Uniform Work Non Uniform Work CLASSIFY BY WORK GRANULARITY Build Acc. Structure SS Collision S Integration L Integration Position Velocity Force Grid Position Velocity Force LL Collision LS Collision Build Acc. Structure
- 13. 13 Harada, Heterogeneous Particle-based Simulation ï§ Small particles ï§ Large particles GPU CPU CLASSIFY BY WORK GRANULARITY, ASSIGN PROCESSOR Build Acc. Structure SS Collision S Integration L Integration Position Velocity Force Grid Position Velocity Force LL Collision LS Collision Build Acc. Structure
- 14. 14 Harada, Heterogeneous Particle-based Simulation ï§ Small particles ï§ Large particles ï§ Grid, small particle data has to be shared with the CPU for LS collision â Allocated as zero copy buffer GPU CPU DATA SHARING Build Acc. Structure SS Collision S Integration L Integration Position Velocity Force Grid Position Velocity Force LL Collision Build Acc. Structure Position Velocity Grid Force LS Collision
- 15. 15 Harada, Heterogeneous Particle-based Simulation ï§ Small particles ï§ Large particles ï§ Grid, small particle data has to be shared with the CPU for LS collision â Allocated as zero copy buffer GPU CPU SYNCHRONIZATION Position Velocity Force Grid Position Velocity Force SS Collision S Integration L Integration LL Collision Position Velocity Grid Force Synchronization LS Collision Build Acc. Structure Build Acc. Structure Synchronization
- 16. 16 Harada, Heterogeneous Particle-based Simulation GPU CPU VISUALIZING WORKLOADS Build Acc. Structure SS Collision S Integration Position Velocity Force Grid Position Velocity Force LL Collision LS Collision Synchronization L Integration ï§ Small particles ï§ Large particles ï§ Grid construction can be moved at the end of the pipeline â Unbalanced workload
- 17. 17 Harada, Heterogeneous Particle-based Simulation ï§ Small particles ï§ Large particles ï§ To get better load balancing â The sync is for passing the force buffer filled by the CPU to the GPU â Move the LL collision after the sync GPU CPU LOAD BALANCING Build Acc. Structure SS Collision S Integration Position Velocity Force Grid Position Velocity Force LL Collision Synchronization L Integration LS Collision
- 18. 18 Harada, Heterogeneous Particle-based Simulation GPUWork CPUWork
- 19. 19 Harada, Heterogeneous Particle-based Simulation MULTI THREADING (4 THREADS)
- 20. 20 Harada, Heterogeneous Particle-based Simulation FURTHER OPTIMIZATION GPU CPU0 CPU1 CPU2 Build Acc. Structure SS Collision S Integ. LL Collision L Integ. LS Collision Synchronization 1. Not optimized for âLlanoâ which is a 4 core CPU â Only 2 CPU core were used â Can use 2 more cores for LS collision 2. LL collision was not optimized â CPU waits when the GPU was constructing a grid â Use CPU to improve SS collision
- 21. 21 Harada, Heterogeneous Particle-based Simulation OPTIMIZATION1: MULTITHREADING LARGE-SMALL COLLISION ï§ Cannot split the work by large particle indices â More than 1 large particle can collide with a small particle â Have to lock the memory on write â Inefficient ï§ Prepare a local buffer for a thread â A buffer storing force on small particles â Lock free ï§ Local buffers are merged to one L0 S0 S1 L1 Thread0 Thread1 Thread2
- 22. 22 Harada, Heterogeneous Particle-based Simulation OPTIMIZATION1: MULTITHREADING LARGE-SMALL COLLISION GPU Build Acc. Structure SS Collision S Integ. CPU0 LL Collision L Integ. CPU1 CPU2 LS Collision Synchronization
- 23. 23 Harada, Heterogeneous Particle-based Simulation OPTIMIZATION1: MULTITHREADING LARGE-SMALL COLLISION GPU Build Acc. Structure SS Collision S Integ. CPU0 LL Collision L Integ. CPU1 CPU2 LS Collision LS Collision LS Collision Synchronization MergeMergeMerge Synchronization
- 24. 24 Harada, Heterogeneous Particle-based Simulation ï§ Spatially coherent memory layout improves cache utilization ï§ As particles move, spatial locality decreases OPTIMIZATION2: IMPROVING SMALL-SMALL COLLISION
- 25. 25 Harada, Heterogeneous Particle-based Simulation ï§ Spatially coherent memory layout improves cache utilization ï§ As particles move, spatial locality decreases OPTIMIZATION2: IMPROVING SMALL-SMALL COLLISION
- 26. 26 Harada, Heterogeneous Particle-based Simulation ï§ Sort particles by spatial location to improve cache utilization â Z curve SPATIAL SORT
- 27. 27 Harada, Heterogeneous Particle-based Simulation ï§ Sort particles by spatial location to improve cache utilization â Z curve SPATIAL SORT
- 28. 28 Harada, Heterogeneous Particle-based Simulation ï§ Requirements â Full sort was over the budget â Full sort is not âa mustâ â Sort is an optional computation for performance improvement â Incremental sort â Use multiple threads ï§ Solution â Used generalized âOdd-even transition sortâ CHOOSE SORT
- 29. 29 Harada, Heterogeneous Particle-based Simulation BLOCK TRANSITION SORT ï§ Generalized âOdd-even transition sortâ ï§ Instead of sorting 2 adjacent elements, sort adjacent 2 blocks ï§ Iterate until convergence ï§ Use a thread to sort 2 adjacent blocks â 6 blocks for 3 threads â Radix sort Odd-even transition sort Block transition sort
- 30. 30 Harada, Heterogeneous Particle-based Simulation OPTIMIZATION2: IMPROVING SMALL-SMALL COLLISION GPU Build Acc. Structure SS Collision S Integ. CPU0 LL Collision L Integ. CPU1 CPU2 LS Collision LS Collision LS Collision Synchronization MergeMergeMerge Synchronization
- 31. 31 Harada, Heterogeneous Particle-based Simulation OPTIMIZATION2: IMPROVING SMALL-SMALL COLLISION GPU Build Acc. Structure SS Collision S Integ. CPU0 CPU1 CPU2 LS Collision LS Collision LS Collision Synchronization MergeMergeMerge LL Coll. L Integ. Synchronization S Sorting S Sorting S Sorting Synchronization
- 32. 32 Harada, Heterogeneous Particle-based Simulation DEMO GPUWork CPUWork
- 33. 33 Harada, Heterogeneous Particle-based Simulation DEMO GPUWork CPUWork
- 34. 34 Harada, Heterogeneous Particle-based Simulation CONCLUSIONS ï§ Realized a simulation that handles variable sized particles by leveraging the best features of both the CPU and GPU on AMDâs Fusion Architecture â The CPU is used for works with non identical compute granularity â The GPU is used for highly parallel works ï§ Memory sharing between the CPU and GPU is the key for the efficiency â Avoid wasteful memory copies
- 35. 35 Harada, Heterogeneous Particle-based Simulation REFERENCE ï§ Takahiro Harada, Seiichi Koshizuka, Yoichiro Kawaguchi, Smoothed Particle Hydrodynamics on GPUs, Proc. of Computer Graphics International, 63-70(2007) ï§ Justin Hensley, Takahiro Harada, Chapter X OpenCL Case Study:Mixed Particle Simulation, Heterogeneous Computing with OpenCL, Morgan Kaufmann(2011)