PyCuda: Come sfruttare la potenza delle schede video nelle applicazioni python
2 likes•895 views
The document discusses using PyCUDA to harness the power of GPUs with Python. It covers why GPUs are useful, how GPUs work, how to program them using CUDA and the CUDA execution model of threads arranged in blocks and grids. It then discusses how to use PyCUDA in Python including allocating memory on the GPU, copying data to and from the GPU, launching kernels, and handling errors.
![Kernel
__global__ void multiply_them( float *dest,
float *a,
float *b )
{
const int i = threadIdx.x;
dest[i] = a[i] * b[i];
}
Thread
PyCon 4 – Florence 2010 – Fabrizio Milo](https://image.slidesharecdn.com/pycuda-come-sfruttare-la-potenza-delle-schede-vide-100707042527-phpapp02/85/PyCuda-Come-sfruttare-la-potenza-delle-schede-video-nelle-applicazioni-python-38-320.jpg)
![Kernel
__global__ void multiply_them( float *dest,
float *a,
float *b )
{
const int i = threadIdx.x;
dest[i] = a[i] * b[i];
}
Thread
PyCon 4 – Florence 2010 – Fabrizio Milo](https://image.slidesharecdn.com/pycuda-come-sfruttare-la-potenza-delle-schede-vide-100707042527-phpapp02/85/PyCuda-Come-sfruttare-la-potenza-delle-schede-video-nelle-applicazioni-python-39-320.jpg)
![Kernel
__global__ void multiply_them( float *dest,
float *a,
float *b )
{
const int i = threadIdx.x;
dest[i] = a[i] * b[i];
}
Block
PyCon 4 – Florence 2010 – Fabrizio Milo](https://image.slidesharecdn.com/pycuda-come-sfruttare-la-potenza-delle-schede-vide-100707042527-phpapp02/85/PyCuda-Come-sfruttare-la-potenza-delle-schede-video-nelle-applicazioni-python-40-320.jpg)
![1 1 1 1 1 1
0 99
import numpy
my_array = numpy.array([1,] * 100)
PyCon 4 – Florence 2010 – Fabrizio Milo](https://image.slidesharecdn.com/pycuda-come-sfruttare-la-potenza-delle-schede-vide-100707042527-phpapp02/85/PyCuda-Come-sfruttare-la-potenza-delle-schede-video-nelle-applicazioni-python-63-320.jpg)
![1 1 1 0 1 1
import numpy
my_array = numpy.array([1,] * 100)
my_array[3] = 0
PyCon 4 – Florence 2010 – Fabrizio Milo](https://image.slidesharecdn.com/pycuda-come-sfruttare-la-potenza-delle-schede-vide-100707042527-phpapp02/85/PyCuda-Come-sfruttare-la-potenza-delle-schede-video-nelle-applicazioni-python-64-320.jpg)
![Kernel
gpu_mem = cuda.mem_alloc( size_bytes )
cuda.memcpy_htod( gpu_mem, cpu_mem )
SourceModule(“””
__global__ void multiply_them( float *dest, float *a,
float *b )
{
const int i = threadIdx.x;
dest[i] = a[i] * b[i];
}”””)
PyCon 4 – Florence 2010 – Fabrizio Milo](https://image.slidesharecdn.com/pycuda-come-sfruttare-la-potenza-delle-schede-vide-100707042527-phpapp02/85/PyCuda-Come-sfruttare-la-potenza-delle-schede-video-nelle-applicazioni-python-70-320.jpg)
![Kernel Launch
mod = SourceModule(“””
__global__ void multiply_them( float *dest, float *a,
float *b )
{
const int i = threadIdx.x;
dest[i] = a[i] * b[i];
}”””)
multiply_them = mod.get_function(“multiply_them”)
multiply_them ( *args, block=(30, 64, 1))
PyCon 4 – Florence 2010 – Fabrizio Milo](https://image.slidesharecdn.com/pycuda-come-sfruttare-la-potenza-delle-schede-vide-100707042527-phpapp02/85/PyCuda-Come-sfruttare-la-potenza-delle-schede-video-nelle-applicazioni-python-71-320.jpg)
![PyCuda: GpuArray: ElementWise
from pycuda.elementwise import ElementwiseKernel
lincomb = ElementwiseKernel(
” float a , float !x , float b , float !y , float !z”,
”z [ i ] = a !x[ i ] + b!y[i ] ”
)
PyCon 4 – Florence 2010 – Fabrizio Milo](https://image.slidesharecdn.com/pycuda-come-sfruttare-la-potenza-delle-schede-vide-100707042527-phpapp02/85/PyCuda-Come-sfruttare-la-potenza-delle-schede-video-nelle-applicazioni-python-81-320.jpg)
![PyCuda: GpuArray: ElementWise
from pycuda.elementwise import ElementwiseKernel
lin comb = ElementwiseKernel(
” float a , float !x , float b , float !y , float !z”,
”z [ i ] = a !x[ i ] + b!y[i ] ”
)
c gpu = gpuarray. empty like (a gpu)
lincomb (5, a gpu, 6, b gpu, c gpu)
assert la . norm((c gpu ! (5!a gpu+6!b gpu)).get()) < 1e!5
PyCon 4 – Florence 2010 – Fabrizio Milo](https://image.slidesharecdn.com/pycuda-come-sfruttare-la-potenza-delle-schede-vide-100707042527-phpapp02/85/PyCuda-Come-sfruttare-la-potenza-delle-schede-video-nelle-applicazioni-python-82-320.jpg)
PyCuda: Come sfruttare la potenza delle schede video nelle applicazioni python
- 1. PyCUDA: Harnessing the power of GPU with Python
- 2. Talk Structure 1. Why a GPU ? 2. How does It works ? 3. How do I Program it ? 4. Can I Use Python ? PyCon 4 – Florence 2010 – Fabrizio Milo
- 3. Talk Structure 1. Why a GPU ? 2. How does It works ? 3. How do I Program it ? 4. Can I Use Python ? PyCon 4 – Florence 2010 – Fabrizio Milo
- 4. WHY A GPU ? PyCon 4 – Florence 2010 – Fabrizio Milo
- 5. APPLICATIONS & DEMOS PyCon 4 – Florence 2010 – Fabrizio Milo
- 6. Why GPU? PyCon 4 – Florence 2010 – Fabrizio Milo
- 7. Talk Structure 1. Why a GPU ? 2. How does it works ? 3. How do I Program it ? 4. Can I Use Python ? PyCon 4 – Florence 2010 – Fabrizio Milo
- 8. How does it works ? PyCon 4 – Florence 2010 – Fabrizio Milo
- 9. ALU ALU Control ALU ALU Cache DRAM CPU PyCon 4 – Florence 2010 – Fabrizio Milo
- 10. DRAM GPU PyCon 4 – Florence 2010 – Fabrizio Milo
- 11. ALU ALU Control ALU ALU Cache DRAM DRAM CPU GPU PyCon 4 – Florence 2010 – Fabrizio Milo
- 12. CUDA PyCon 4 – Florence 2010 – Fabrizio Milo
- 13. Compute Unified Device Architecture PyCon 4 – Florence 2010 – Fabrizio Milo
- 14. CUDA A Parallel Computing Architecture for NVIDIA GPUs Direct X Compute PyCon 4 – Florence 2010 – Fabrizio Milo
- 15. Execution Model CUDA Device Model PyCon 4 – Florence 2010 – Fabrizio Milo
- 16. EXECUTION MODEL PyCon 4 – Florence 2010 – Fabrizio Milo
- 17. Thread Smallest unit of logic PyCon 4 – Florence 2010 – Fabrizio Milo
- 18. A Block A Group of Threads PyCon 4 – Florence 2010 – Fabrizio Milo
- 19. A Grid A Group of Blocks PyCon 4 – Florence 2010 – Fabrizio Milo
- 20. One Block can have many threads PyCon 4 – Florence 2010 – Fabrizio Milo
- 21. One Grid can have many blocks PyCon 4 – Florence 2010 – Fabrizio Milo
- 22. The hardware DEVICE MODEL PyCon 4 – Florence 2010 – Fabrizio Milo
- 23. Scalar Processor PyCon 4 – Florence 2010 – Fabrizio Milo
- 24. Scalar Processor PyCon 4 – Florence 2010 – Fabrizio Milo
- 25. Many Scalar Processors PyCon 4 – Florence 2010 – Fabrizio Milo
- 26. + Register File PyCon 4 – Florence 2010 – Fabrizio Milo
- 27. + Shared Memory PyCon 4 – Florence 2010 – Fabrizio Milo
- 28. Multiprocessor PyCon 4 – Florence 2010 – Fabrizio Milo
- 29. Device PyCon 4 – Florence 2010 – Fabrizio Milo
- 30. Real Example: 10-Series Architecture " 240 Scalar Processor (SP) cores execute kernel threads " 30 Streaming Multiprocessors (SMs) each contain " 8 scalar processors " 1 double precision unit " Shared memory PyCon 4 – Florence 2010 – Fabrizio Milo
- 31. Software Hardware Scalar Processor Thread Thread Block Multiprocessor Grid Device PyCon 4 – Florence 2010 – Fabrizio Milo
- 32. Global Memory PyCon 4 – Florence 2010 – Fabrizio Milo
- 33. Global Memory PyCon 4 – Florence 2010 – Fabrizio Milo
- 34. RAM CPU Global Memory Host - Device PyCon 4 – Florence 2010 – Fabrizio Milo
- 35. RAM CPU Host – Multi Device PyCon 4 – Florence 2010 – Fabrizio Milo
- 36. 1. Why a GPU ? 2. How does It works ? 3. How do I Program it ? 4. Can I Use Python ? PyCon 4 – Florence 2010 – Fabrizio Milo
- 37. Software Hardware Scalar Processor Thread Thread Block Multiprocessor Grid Device PyCon 4 – Florence 2010 – Fabrizio Milo
- 38. Kernel __global__ void multiply_them( float *dest, float *a, float *b ) { const int i = threadIdx.x; dest[i] = a[i] * b[i]; } Thread PyCon 4 – Florence 2010 – Fabrizio Milo
- 39. Kernel __global__ void multiply_them( float *dest, float *a, float *b ) { const int i = threadIdx.x; dest[i] = a[i] * b[i]; } Thread PyCon 4 – Florence 2010 – Fabrizio Milo
- 40. Kernel __global__ void multiply_them( float *dest, float *a, float *b ) { const int i = threadIdx.x; dest[i] = a[i] * b[i]; } Block PyCon 4 – Florence 2010 – Fabrizio Milo
- 41. Kernel __global__ void kernel( … ) { const int idx = blockIdx.x * blockDim.x + threadIdx.x; … } Grid PyCon 4 – Florence 2010 – Fabrizio Milo
- 42. How do I Program it ? Main Logic Kernel GCC NVCC CPU .bin .cubin GPU PyCon 4 – Florence 2010 – Fabrizio Milo
- 43. How do I Program it ? Main Logic Kernel GCC NVCC GPU .bin .cubin .bin .cubin . CPU PyCon 4 – Florence 2010 – Fabrizio Milo
- 44. RAM CPU Global Memory Host - Device PyCon 4 – Florence 2010 – Fabrizio Milo
- 45. RAM CPU Global Memory PyCon 4 – Florence 2010 – Fabrizio Milo
- 46. Allocate Memory cudaMalloc( pointer, size ) PyCon 4 – Florence 2010 – Fabrizio Milo
- 47. Copy to device cudaMalloc( pointer, size ) cudaMemcpy( dest, src, size, direction) PyCon 4 – Florence 2010 – Fabrizio Milo
- 48. Kernel Launch cudaMalloc( pointer, size ) cudaMemcpy( dest, src, size, direction) Kernel<<< # blocks, # threads >> (*params) PyCon 4 – Florence 2010 – Fabrizio Milo
- 49. Get Back the Results cudaMalloc( pointer, size ) cudaMemcpy( dest, src, size, direction) Kernel<<< # blocks, # threads >> (*params) cudaMemcpy( dest, src, size, direction) PyCon 4 – Florence 2010 – Fabrizio Milo
- 50. Error Handling If(cudaMalloc( pointer, size ) != cudaSuccess){ handle_error() } PyCon 4 – Florence 2010 – Fabrizio Milo
- 51. And soon it becomes … If(cudaMalloc( pointer, size ) != cudaSuccess){ handle_error() } if (cudaMemcpy( dest, src, size, direction ) == cudaSuccess) {} If (Kernel<<< # blocks, # threads >> (*params) != cudaSuccess){ handle_error() } If( cudaMemcpy( dest, src, size, direction) != cudaSuccess) { } PyCon 4 – Florence 2010 – Fabrizio Milo
- 52. And soon it becomes … If(cudaMalloc( pointer, size ) != cudaSuccess){ handle_error() If(cudaMalloc( pointer, size ) != cudaSuccess){ } handle_error() } if (cudaMemcpy( dest, src, size, direction ) == cudaSuccess) {} if (cudaMemcpy( dest, src, size, direction ) == cudaSuccess) {} If (Kernel<<< # blocks, # threads >> (*params) != cudaSuccess){ handle_error() If (Kernel<<< # blocks, # threads >> (*params) != cudaSuccess){ } handle_error() } If( cudaMemcpy( dest, src, size, direction) != cudaSuccess) { } If( cudaMemcpy( dest, src, size, direction) != cudaSuccess) { } If(cudaMalloc( pointer, size ) != cudaSuccess){ handle_error() If(cudaMalloc( pointer, size ) != cudaSuccess){ } handle_error() } if (cudaMemcpy( dest, src, size, direction ) == cudaSuccess) {} if (cudaMemcpy( dest, src, size, direction ) == cudaSuccess) {} If (Kernel<<< # blocks, # threads >> (*params) != cudaSuccess){ handle_error() If (Kernel<<< # blocks, # threads >> (*params) != cudaSuccess){ } handle_error() } If( cudaMemcpy( dest, src, size, direction) != cudaSuccess) { } If( cudaMemcpy( dest, src, size, direction) != cudaSuccess) { } If(cudaMalloc( pointer, size ) != cudaSuccess){ handle_error() If(cudaMalloc( pointer, size ) != cudaSuccess){ } handle_error() } if (cudaMemcpy( dest, src, size, direction ) == cudaSuccess) {} if (cudaMemcpy( dest, src, size, direction ) == cudaSuccess) {} If (Kernel<<< # blocks, # threads >> (*params) != cudaSuccess){ handle_error() If (Kernel<<< # blocks, # threads >> (*params) != cudaSuccess){ } handle_error() } If( cudaMemcpy( dest, src, size, direction) != cudaSuccess) { } If( cudaMemcpy( dest, src, size, direction) != cudaSuccess) { } PyCon 4 – Florence 2010 – Fabrizio Milo
- 53. PyCon 4 – Florence 2010 – Fabrizio Milo
- 54. 1. Why a GPU ? 2. How does It works ? 3. How do I Program it ? 4. Can I Use Python ? PyCon 4 – Florence 2010 – Fabrizio Milo
- 55. + & ANDREAS KLOCKNER = PYCUDA PyCon 4 – Florence 2010 – Fabrizio Milo
- 56. PyCuda Philosopy Provide Complete Access PyCon 4 – Florence 2010 – Fabrizio Milo
- 57. PyCuda Philosopy AutoMatically Manage Resources PyCon 4 – Florence 2010 – Fabrizio Milo
- 58. PyCuda Philosopy Check and Report Errors PyCon 4 – Florence 2010 – Fabrizio Milo
- 59. PyCuda Philosopy Cross Platform PyCon 4 – Florence 2010 – Fabrizio Milo
- 60. PyCuda Philosopy Allow Interactive Use PyCon 4 – Florence 2010 – Fabrizio Milo
- 61. PyCuda Philosopy NumPy Integration PyCon 4 – Florence 2010 – Fabrizio Milo
- 62. NUMPY - ARRAY PyCon 4 – Florence 2010 – Fabrizio Milo
- 63. 1 1 1 1 1 1 0 99 import numpy my_array = numpy.array([1,] * 100) PyCon 4 – Florence 2010 – Fabrizio Milo
- 64. 1 1 1 0 1 1 import numpy my_array = numpy.array([1,] * 100) my_array[3] = 0 PyCon 4 – Florence 2010 – Fabrizio Milo
- 65. PyCuda: Workflow PyCon 4 – Florence 2010 – Fabrizio Milo
- 66. PyCuda: Workflow PyCon 4 – Florence 2010 – Fabrizio Milo
- 67. PyCuda: Workflow PyCon 4 – Florence 2010 – Fabrizio Milo
- 68. Memory Allocation cuda.mem_alloc( size_bytes ) PyCon 4 – Florence 2010 – Fabrizio Milo
- 69. Memory Copy gpu_mem = cuda.mem_alloc( size_bytes ) cuda.memcpy_htod( gpu_mem, cpu_mem ) PyCon 4 – Florence 2010 – Fabrizio Milo
- 70. Kernel gpu_mem = cuda.mem_alloc( size_bytes ) cuda.memcpy_htod( gpu_mem, cpu_mem ) SourceModule(“”” __global__ void multiply_them( float *dest, float *a, float *b ) { const int i = threadIdx.x; dest[i] = a[i] * b[i]; }”””) PyCon 4 – Florence 2010 – Fabrizio Milo
- 71. Kernel Launch mod = SourceModule(“”” __global__ void multiply_them( float *dest, float *a, float *b ) { const int i = threadIdx.x; dest[i] = a[i] * b[i]; }”””) multiply_them = mod.get_function(“multiply_them”) multiply_them ( *args, block=(30, 64, 1)) PyCon 4 – Florence 2010 – Fabrizio Milo
- 72. PyCon 4 – Florence 2010 – Fabrizio Milo
- 73. PyCon 4 – Florence 2010 – Fabrizio Milo
- 74. PyCon 4 – Florence 2010 – Fabrizio Milo
- 75. Hello Gpu DEMO PyCon 4 – Florence 2010 – Fabrizio Milo
- 76. GPUARRAY PyCon 4 – Florence 2010 – Fabrizio Milo
- 77. gpuarray PyCon 4 – Florence 2010 – Fabrizio Milo
- 78. PyCuda: GpuArray gpuarray.to_gpu(numpy array) numpy array = gpuarray.get() PyCon 4 – Florence 2010 – Fabrizio Milo
- 79. PyCuda: GpuArray gpuarray.to_gpu(numpy array) numpy array = gpuarray.get() +, -, !, /, fill, sin, exp, rand, basic indexing, norm, inner product … PyCon 4 – Florence 2010 – Fabrizio Milo
- 80. PyCuda: GpuArray: ElementWise from pycuda.elementwise import ElementwiseKernel PyCon 4 – Florence 2010 – Fabrizio Milo
- 81. PyCuda: GpuArray: ElementWise from pycuda.elementwise import ElementwiseKernel lincomb = ElementwiseKernel( ” float a , float !x , float b , float !y , float !z”, ”z [ i ] = a !x[ i ] + b!y[i ] ” ) PyCon 4 – Florence 2010 – Fabrizio Milo
- 82. PyCuda: GpuArray: ElementWise from pycuda.elementwise import ElementwiseKernel lin comb = ElementwiseKernel( ” float a , float !x , float b , float !y , float !z”, ”z [ i ] = a !x[ i ] + b!y[i ] ” ) c gpu = gpuarray. empty like (a gpu) lincomb (5, a gpu, 6, b gpu, c gpu) assert la . norm((c gpu ! (5!a gpu+6!b gpu)).get()) < 1e!5 PyCon 4 – Florence 2010 – Fabrizio Milo
- 83. Meta-Programming __kernel_template__ = “”” __global__ void kernel( args ) { for (int i=0; i={{ iterations }}; i++){ {{operations}} } }””” See for example jinja2 PyCon 4 – Florence 2010 – Fabrizio Milo
- 84. Meta-Programming PyCon 4 – Florence 2010 – Fabrizio Milo
- 85. Meta-Programming Generate Source ! PyCon 4 – Florence 2010 – Fabrizio Milo
- 86. Performances ? PyCon 4 – Florence 2010 – Fabrizio Milo
- 87. mandelbrot DEMO PyCon 4 – Florence 2010 – Fabrizio Milo
- 88. PyCuda: Documentation PyCon 4 – Florence 2010 – Fabrizio Milo
- 89. PyCuda WebSite: http://mathema.tician.de/software/ pycuda License: X Consortium License (no warranty, free for all use) Dependencies: Python 2.4+, numpy, Boost PyCon 4 – Florence 2010 – Fabrizio Milo
- 90. In the Future … OPENCL PyCon 4 – Florence 2010 – Fabrizio Milo
- 91. THANK YOU & HAVE FUN ! PyCon 4 – Florence 2010 – Fabrizio Milo
- 92. ? PyCon 4 – Florence 2010 – Fabrizio Milo