Alluxio Webinar - Maximize GPU Utilization for Model Training
0 likes•145 views
The document discusses maximizing GPU utilization for model training, emphasizing the importance of efficient data management, processing, and caching. It highlights the challenges of low GPU utilization and presents Alluxio as a solution that significantly reduces data loading times and increases GPU utilization rates. The findings indicate that using Alluxio can improve efficiency in deep learning practices, particularly for large datasets.
Alluxio Webinar - Maximize GPU Utilization for Model Training
- 1. From Idle to Optimal: Maximize GPU Utilization for Model Training Beinan Wang Tarik Bennett
- 2. Senior Staff Engineer @ Alluxio Trino Contributor PrestoDB Committer Senior Solutions Engineer @ Alluxio Dr. Beinan Wang Tarik Bennett 2
- 3. Source: https://www.wsj.com/articles/rush-to-use-generative-ai-pushes-companies-to-get-data-in-order “Training large language models requires ready access to vast amounts of data, whose storage, processing, and protection can be costly.”
- 4. High Scalability Training billions files ESSENTIAL High Availability 99.99% ESSENTIAL High Performance Higher GPU utilization ESSENTIAL Always Increasing Expectations… Icons created by kerismaker, HJ Studio - Flaticon
- 5. What Does Managing Data Involve? Data Preprocessing Improving the quality and reliability of the data for model training Model Training Read training data, vision (image) or NLP/LLM (text), for DL using GPUs Model Deployment Consumption of trained models for online or offline inference Feature Engineering Selecting relevant and informative features from raw data PyTorch | Tensorflow | Spark Spark PyTorch | Tensorflow | Spark Model Training Data Result Model Compute Stage Spark | Trino | Presto Result Curated Data Not discussed today - Security - Privacy (PII) - Data Cleaning - Data Pipelines - Data Governance Curated / Processed Data
- 6. 100,000,000,000,000,000,000,000 bytes of data will be stored in the cloud by 2025 6 Source: Cybersecurity Ventures
- 7. Issues Managing Ultra-Large Datasets Non-Functional Storage Requirements High Performance - Many options Cost-Effective - Commodity storage
- 8. 10% of your data is hot data 8 Source: Alluxio
- 9. 9 Data Caching Helps Boost Performance Save Costs Prevent Network Congestion Offload Under Storage
- 10. 10 Data Caching at Uber Scale 3 Clusters, 1500 Nodes Source: https://www.uber.com/blog/speed-up-presto-with-alluxio-local-cache/ 50% Input Read Performance 10% Data Read Traffic to HDFS
- 12. GPUs are scarce GPUs are expensive Challenges as you try to scale Low GPU Utilization
- 13. Addressing Low GPU Utilization with Caching 13
- 14. Architecture Overview Online ML platform Inference cluster Models Models Training Data Models 1 2 3 4 5 Offline training platform Alluxio Training cluster Training Data 2 14
- 15. AI Training Test with Alluxio 15 Local Folder /dataset Alluxio GPU Training Remote Storage Kubernetes Interactive Notebook Alluxio Operator Visualization Dashboard
- 16. Test Setup ● Alluxio via Kubernetes - Provides caching for training data ● GPU server - AWS EC2/Kubernetes ● Deep learning algorithm (CV) - ResNet (one of the most popular CV algorithms) ● Deep learning framework - PyTorch ● Dataset - ImageNet (subset - ~35k images, each is ~100kB - 200kB) ● Dataset storage - S3 (single region) ● Mounting - FUSE ● Visualization - TensorBoard ● Code execution - Jupyter notebook 16
- 17. Training Test Steps 1. Loading the dataset into Alluxio 2. Running the training job 3. Reading the dataset from Alluxio through PyTorch DataLoader in each epoch 4. Visualizing the GPU utilization and other metrics 17
- 18. 18 Training Directly from Storage - > 80% of total time is spent in DataLoader - Result in Low GPU Utilization Rate (<20%) Visualization Dashboard Results (Control)
- 19. 19 Visualization Dashboard Results (Alluxio) Training with Alluxio - Reduced DataLoader Rate from 82% to 1% (82X) - Increase GPU Utilization Rate from 17% to 93% (5X)
- 20. Source: https://developer.nvidia.com/blog/accelerating-analytics-and-ai-with-alluxio-and-nvidia-gpus/ “The benefits from GPU acceleration are limited if data access dominates the execution time. “
- 21. Thank You twitter.com/alluxio slackin.alluxio.io linkedin.com/alluxio www.alluxio.io JOIN THE CONVERSATION ON SLACK ALLUXIO.IO/SLACK