Intro to Machine Learning for GPUs
5 likes2,391 views
1. The document discusses GPUs and their advantages for machine learning tasks like deep learning and parallel computing. GPUs have many parallel processors that can accelerate matrix multiplications and other computations used in machine learning algorithms. 2. It introduces CUDA and how it allows GPUs to be programmed for general purpose processing through a parallel computing model. Examples are given of how matrix multiplications and convolutional neural network operations can be parallelized on GPUs. 3. H2O is presented as a machine learning platform that supports GPU acceleration for algorithms like gradient boosted machines, enabling faster training on large datasets. Instructions are provided on getting started with CUDA, cuDNN and using GPUs for machine learning.
Intro to Machine Learning for GPUs
- 1. Wen Phan Introduction to GPUs for Machine Learning
- 2. Agenda • Context and Why GPUs? – Matrix Multiplication Example • CUDA • GPU and Machine Learning – Deep Learning – Parallel Computing: GBM, GLM • GPU-Enabled H2O • Getting Started
- 3. Need for More Compute • Lots of Data • Complex Architectures • Many Models
- 4. Historic Ways for More Compute • Faster Clock Rates • Multi-Core • Distributed Computing
- 5. CPU Trends Original data collected and plotted by M. Horowitz, F. Labonte, O. Shacham, K. Olukotun, L. Hammond, and C. Batten, dotted line extrapolations by C. Moore
- 7. Why GPUs?
- 9. GPUs for Parallel Tasks Traditional CPUs are not economically feasible 2.3 PFlops 7000 homes 7.0 Megawatts 7.0 Megawatts CPU Optimized for Serial Tasks GPU Accelerator Optimized for Many Parallel Tasks 10x performance/socket > 5x energy efficiency Era of GPU-accelerated computing is here nVIDIA
- 10. GPU Devotes More Transistors to Data Processing CUDA C Programming Guide
- 12. Latency Versus Throughput • Latency: Time to do a task. • Throughput: Number of tasks per unit time. • Fictitious Example: – CPU • Latency: 1 ns per task • Throughput: (1 task per ns) x (6 cores) = 6 task per ns – GPU • Latency: 10 ns per task • Throughput: (0.1 task per ns) x (2000 cores) = 200 task per ns • CPUs are latency optimized; GPUs are throughput optimized
- 17. Matrix Multiplication 2 4 A 3 5 m ⇥ k 2 4 B 3 5 k ⇥ n = 2 4 C 3 5 m ⇥ n
- 18. Matrix Multiplication 2 6 6 6 6 6 4 a1,1 a1,2 a1,3 . . . a1,k a2,1 a2,2 a2,3 . . . a2,k a3,1 a3,2 a3,3 . . . a3,k ... ... ... ... ... am,1 am,2 am,3 . . . am,k 3 7 7 7 7 7 5 A 2 6 6 6 6 6 4 b1,1 b1,2 b1,3 . . . b1,n b2,1 b2,2 b2,3 . . . b2,n b3,1 b3,2 b3,3 . . . b3,n ... ... ... ... ... bk,1 bk,2 bk,3 . . . bk,n 3 7 7 7 7 7 5 B = 2 6 6 6 6 6 4 c1,1 c1,2 c1,3 . . . c1,n c2,1 c2,2 c2,3 . . . c2,n c3,1 c3,2 c3,3 . . . c3,n ... ... ... ... ... cm,1 cm,2 cm,3 . . . cm,n 3 7 7 7 7 7 5 C ci,j = kX h=1 ai,hbh,j
- 19. CUDA
- 20. CUDA • Historically, GPUs were used for, well, graphics processing. But, people realized that the fine- grained parallelism inherently in GPU architecture could be exploited for general purpose computing. • CUDA (Compute Unified Device Architecture) – Parallel computing platform – Programming model and API – Allows enabled GPUs for general purpose processing
- 21. Speed Up Parallelizable Code Application Code GPU Use GPU to Parallelize Compute-Intensive Functions CPU Rest of Sequential CPU Code nVIDIA
- 24. Allocate Memory and Copy to GPU (Device) CUDA C Programming Guide CPU GPU CPU Memory GPU Memory
- 25. Invoke Kernel (GPU Program) CUDA C Programming Guide CPU GPU CPU Memory GPU Memory
- 32. CUDA Ecosystem
- 37. GPUs and Machine Learning • Poster Child: Deep Learning • Parallel Computing – Model Parallelism – Data Parallelism – Training Parallelism
- 38. Deep Learning
- 39. Multi-Layer Perceptron Neural Network Efron and Hastie. Computer Age Statistical Inference.
- 41. Image as a Tensor TensorFlow
- 44. Neuron
- 46. Layer of Neurons
- 47. Layer of Neurons
- 49. Hidden Layers
- 50. Convolutional Neural Networks • Leverages the fact that data has spatial structure – Add idea of locality • Tremendous success with computer vision tasks • “Put deep learning on the map”
- 51. Frobenius Inner Product X = 2 4 2 2 1 2 0 1 2 1 2 3 5 , K = 2 4 1 1 1 1 1 1 1 1 1 3 5 hX, KiF = X i,j xi,jki,j = (2 ⇤ 1) + (2 ⇤ 1) + (1 ⇤ 1) + (2 ⇤ 1) + (0 ⇤ 1) + (1 ⇤ 1) + (2 ⇤ 1) + (1 ⇤ 1) + (2 ⇤ 1) = 2 + 2 + 1 2 + 0 1 2 + 1 + 2 = 3
- 52. Convolution X = 2 6 6 6 6 6 6 6 6 4 0 0 0 0 0 0 0 0 0 2 1 0 2 0 0 1 2 2 1 0 0 0 0 2 0 1 1 0 0 1 2 1 2 1 0 0 2 0 0 2 1 0 0 0 0 0 0 0 0 3 7 7 7 7 7 7 7 7 5 K = 2 4 1 1 1 1 1 1 1 1 1 3 5 A = 2 6 6 6 6 4 1 4 0 4 1 3 4 1 4 1 4 9 3 5 0 3 0 2 5 0 5 2 3 5 2 3 7 7 7 7 5
- 55. Convolutional Layer w d h g 0 @ X d X i,j xi,j,dki,j,d + b 1 A X Kf X d X i,j xi,j,dki,j,d ¯ b g(·)
- 56. ¯ Convolutional Layer w d h X d X i,j xi,j,dki,j,d b g 0 @ X d X i,j xi,j,dki,j,d + b 1 A g(·) f X K
- 57. Convolutional Layer w d h X Input Image Activation Maps Convolution
- 58. Convolutional Layer • f = receptive field (filter size) • p = padding • s = stride • m = number of filters Input Volume Output Volume Convolution wI hI dI wO dO hO wO = wI f + 2p s + 1 hO = wI f + 2p s + 1 dO = m
- 59. Example: LeNet
- 61. ImageNet
- 62. cuDNN
- 64. CUDA and cuDNN Speed Up
- 66. ImageNet Entries Using GPUs https://devblogs.nvidia.com/parallelforall/nvidia-ibm-cloud-support-imagenet-large-scale-visual-recognition-challenge/
- 67. Deep Water: Next-Gen Distributed Deep Learning One Interface - GPU Enabled - Significant Performance Gains Inherits All H2O Properties in Scalability, Ease of Use and Deployment Recurrent Neural Networks enabling natural language processing, sequences, time series, and more Convolutional Neural Networks enabling Image, video, speech recognition Hybrid Neural Network Architectures enabling speech to text translation, image captioning, scene parsing and more H2O integrates with existing GPU backends for significant performance gains H2O Deep Learning Algo
- 70. Parallel Computing • Model Parallelism: Split up a single model
- 71. Random Forest T1(x) T2(x) T3(x) TB(x) ˆy = f(x; T1, . . . , TB)
- 72. Random Forest T1(x) T2(x) T3(x) TB(x)
- 73. Deep Learning Model Parallelism Large Scale Distributed Deep Networks. J. Dean, et. al.
- 74. Parallel Computing • Model Parallelism: Split up a single model • Data Parallelism: Split up data to train a single model
- 75. Deep Learning Data Parallelism Large Scale Distributed Deep Networks. J. Dean, et. al.
- 76. H2O Deep Learning Architecture
- 77. Gradient Boosting Machine (GBM) T1(x) T2(x) T3(x) TM (x) fM (x) = MX i=1 Ti(x)
- 78. Gradient Boosting Machine (GBM) T1(x) T2(x) T3(x) TM (x) fi(x) = fi 1(x) + Ti(x; ˆ⇥i)
- 79. Gradient Boosting Machine (GBM) T1(x) T2(x) T3(x) TM (x)
- 81. GBM Data Parallelism {Xi; ti}?
- 82. GBM Data Parallelism 1 2 K X = {X1, . . . , XK} math (X1) math (X2) math (XK) {Xi; ti} = f(math (X1) , . . . , math (XK))
- 83. GBM Data Parallelism 1 2 K X = {X1, . . . , XK} math (X1) math (X2) math (XK) {Xi; ti} = f(math (X1) , . . . , math (XK)) Full Data Parallelism for Each Level of Tree Growth!
- 84. CPU Cluster
- 85. GPU Cluster • Level GPUs to accelerate processing and fine-grain parallelism on each node CPU GPU math (X1) math (X2) math (XK)math (X3)
- 86. GBM on GPU T1(x) T2(x) T3(x) TM (x) Application Code GPU Use GPU to Compute-Intensive Functions CPU Rest of Sequential CPU Code
- 87. H2O XGBoost
- 88. Parallel Computing • Model Parallelism: Split up a single model • Data Parallelism: Split up data to train a single model • Training Parallelism: Split up different parts of the training process – Ensemble Base Learners – Cross-Validation – Hyperparameters
- 89. Linear Regression Hastie, Tibshirani, Friedman. Elements of Statistical Learning X = 2 6 6 6 4 x1,1 x1,2 . . . x1,p x2,1 x2,2 . . . x2,p ... ... ... ... xn,1 xn,2 . . . xn,p 3 7 7 7 5 minimize nX i=1 0 @yi 0 pX j=1 xi,j j 1 A 2 minimize ||y X ||2 2
- 90. Ridge Regression Hastie, Tibshirani, Wainwright. Statistical Learning with Sparsity minimize 0, ||y X ||2 2 subject to || ||2 2 t
- 91. Ridge Regression Hastie, Tibshirani, Wainwright. Statistical Learning with Sparsity minimize ||y X ||2 2 + || ||2
- 92. Lasso Regression Hastie, Tibshirani, Wainwright. Statistical Learning with Sparsity minimize 0, ||y X ||2 2 subject to || ||1 t
- 93. Lasso Regression Hastie, Tibshirani, Wainwright. Statistical Learning with Sparsity minimize ||y X ||2 2 + || ||1
- 94. Elastic Net Regression minimize ||y X ||2 2 + ✓ ↵|| ||1 + 1 2 (1 ↵)|| ||2 2 ◆
- 97. Elastic Net Regression Training Parallelism J( ; , ↵) = ||y X ||2 2 + ✓ ↵|| ||1 + 1 2 (1 ↵)|| ||2 2 ◆ minimize J( ; i, ↵1) minimize J( ; i, ↵2) minimize J( ; i, ↵G) GPU1 GPU2 GPUG i 2 {0, . . . , 1}
- 98. Elastic Net Regression Training Parallelism
- 99. GPU-Enabled H2O
- 100. DGX-1
- 104. GPU Open Analytics Initiative
- 106. GTC 2017
- 107. GTC 2017
- 108. Getting Started
- 109. CUDA https://developer.nvidia.com/cuda-downloads
- 111. Try It Today
- 112. Questions?