Deep Learning in the Cloud at Scale: A Data Orchestration Story
0 likes132 views
The document discusses the implementation of deep learning on Azure, focusing on data orchestration and the end-to-end lifecycle for machine learning. It highlights the challenges of distributed training, the need for advanced ML platform requirements, and the use of Kubernetes and Alluxio for efficient resource management. Additionally, it outlines specific deep learning scenarios and the importance of scalable performance in training platforms.
Deep Learning in the Cloud at Scale: A Data Orchestration Story
- 1. Deep Learning in the Cloud at Scale: A Data Orchestration Story Chao Wang, Mickey Zhang, Qianjun Xu
- 2. Overview Machine Learning Requirements End-to-End lifecycle and processes Data Scientist Workflow Deep Learning on Azure Machine Learning Deep Learning: Additional Requirements Distributed Training with Azure ML Compute Kubernetes + Alluxio
- 3. Requirements of an advanced ML Platform
- 4. Machine Learning Typical E2E Process Prepare Data Register and Manage Model Train & Test Model Build Image … Build Model (your favorite IDE) Deploy Service Monitor Model Prepare Experiment Deploy Orchestrate
- 5. DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image … Build Model (your favorite IDE) Deploy Service Monitor Model Train & Test Model
- 6. Deep Learning on Azure Machine Learning
- 7. Characteristics of Deep Learning Massive amounts of training data Excels with raw, unstructured data Automatic feature extraction Computationally expensive
- 8. Distributed training mode: Data parallelism Subset 1 CNN model Worker 1 Subset 2 CNN model Job manager Worker 2 Dataset CNN model
- 9. Distributed training mode: Model parallelism Dataset CNN model CNN model CNN modelSubset 1 Subset 2 Worker 1 Worker 2 Job manager
- 10. Challenges of distributed training Dependencies and Containers Provision clusters of VMs Schedule jobs Distribute data Gather results Handling failures Scale resources Secure Access
- 12. Deep Learning Scenarios 1. Standard ImageNet 2. BERT-Large 3. Checkpoint Save/Load
- 13. Typical Data Consumption Model Storage/NFS Deep Learning Training Platform RAM SSD GPU CPU Azure Kubernetes Service RAM SSD GPU CPU RAM SSD GPU CPU
- 14. Why Alluxio? Scalable Performance is scalable based on the cluster size Lower Cost Leverage idle resources in the cluster Performance Improve data access throughput by distributing across nodes Flexibility Manage multiple data sources in a unified namespace
- 15. Side-Car Model With Alluxio Storage Deep Learning Training Platform RAM SSD GPU CPU Azure Kubernetes Service RAM SSD GPU CPU RAM SSD GPU CPU Data Preloaded in Cluster
- 16. ImageNet PyTorch resnet50 1.3M images, 50~200 KB each
- 17. Bert-Large Training Job 100 partitions, 1.8 GB each
- 18. Checkpoint Save/Load Single file, 4.7 GB Load: 24 processes Save: rank0 process
- 19. Looking Forward • Build pilot experience for customers • Continuing performance improvement
- 20. © Copyright Microsoft Corporation. All rights reserved. Thank you!