DevBCN Vertex AI - Pipelines for your MLOps workflows
0 likes212 views
The document covers Vertex AI, a managed machine learning platform designed to streamline the development, training, and deployment of ML solutions while incorporating MLOps practices. It details the platform's features, including data labeling, AutoML models, and pipelines, which facilitate automated workflows and adaptability to data changes. Concluding remarks emphasize Vertex AI's role in enhancing the developer experience and providing comprehensive tools for ML tasks.
DevBCN Vertex AI - Pipelines for your MLOps workflows
- 1. VertexAI PipelinesforyourMLOpsworkflows Márton Kodok Software Architect at REEA.net July 3-5, Barcelona
- 2. 1. What is Vertex AI? 2. Build, train and deploy ML solutions 3. What is MLOps? 4. Using Pipelines throughout your ML workflow 5. Adapting to changes of data 6. Conclusions Agenda Vertex AI: Pipelines for your MLOps workflows @martonkodok
- 3. ● Among the Top 3 romanians on Stackoverflow 204k reputation ● Google Developer Expert on Cloud technologies (2016→) ● Champion of Google Cloud Innovators program (2021→) ● Crafting Cloud Architecture+ML backends at REEA.net Articles: martonkodok.medium.com Twitter: @martonkodok Slideshare: martonkodok StackOverflow: pentium10 GitHub: pentium10 Vertex AI: Pipelines for your MLOps workflows @martonkodok About me
- 5. “ VertexAI is a managed ML platform for developers @martonkodok
- 6. 1. Build generative AI apps quickly and responsibly by easily accessing a variety of foundation models with Model Garden and Generative AI Studio 2. Accelerate ML with tooling for pre-trained and custom models 3. Implement MLOps practices to efficiently scale, manage and govern your ML workloads Vertex AI: Pipelines for your MLOps workflows @martonkodok Vertex AI: One AI platform, every ML tool you need
- 7. What’s included in VertexAI? @martonkodok Data Labeling AutoML models DL Environment (DL VM + DL Container) Prediction Feature Store Training Experiments Data Readiness Feature Engineering Training/ HP-Tuning Model Monitoring Model serving Understanding/ Tuning Edge Model Management Notebooks Pipelines (Orchestration) Explainable AI Hybrid AI Model Monitoring Metadata Vision-Video-NLP-Translate Models Datasets Custom Models Containers Python Endpoints BigQuery ML BigQuery Models Publisher Pre-trained models Model Garden GenAI Studio Foundation -> LLM Palm 2 API
- 8. Where does VertexAI fit in? Application Servers Vertex AI Desktop client Mobile client End-to-end platform for ML model development and deployment Backend Vertex AI: Pipelines for your MLOps workflows @martonkodok Application Logic
- 9. VertexAI is a serverless MLOps platform Vertex AI: Pipelines for your MLOps workflows @martonkodok Operational Model Programming Model No Infra Management Managed Security Pay only for usage Model-as-a-service oriented Streamlined model development Open SDKs, integrates with ML frameworks
- 10. Vertex AI: Pipelines for your MLOps workflows @martonkodok Model Garden Open Source Models Task Specific AutoML and APIs Foundation Models Now in GA Model Garden
- 11. The PaLM 2 is available only in the US. @martonkodok Foundation Models Embeddings API for Text and Image Extract semantic information from unstructured data PaLM for Text Custom language tasks PaLM for Chat Multi-turn conversations with session context Codey for Code Generation Improve coding and debugging Now in GA Now in GA Preview Preview
- 12. The PaLM 2 is available only in the US. @martonkodok Sample code for GenAI on Google Cloud goo.gle/gen-ai-github
- 13. Vertex AI: Pipelines for your MLOps workflows @martonkodok VertexAI:Managed unified ML platform Fine-tuning *
- 14. “ VertexAI autoML use cases Vertex AI: Pipelines for your MLOps workflows @martonkodok
- 15. What’s included in VertexAI? @martonkodok Data Labeling AutoML models DL Environment (DL VM + DL Container) Prediction Feature Store Training Experiments Data Readiness Feature Engineering Training/ HP-Tuning Model Monitoring Model serving Understanding/ Tuning Edge Model Management Notebooks Pipelines (Orchestration) Explainable AI Hybrid AI Model Monitoring Metadata Vision-Video-NLP-Translate Models Datasets Custom Models Containers Python Endpoints BigQuery ML BigQuery Models Publisher Pre-trained models Model Garden GenAI Studio Foundation -> LLM Palm 2 API
- 16. - Regression/classification - Forecasting - Single-label classification - Multi-label classification - Text entity extraction - Text sentiment analysis - Video action recognition - Video classifications for entire video, shots, frames - Video object tracking AutoML: Managed datasets + objectives Vertex AI: Pipelines for your MLOps workflows @martonkodok Image Tabular Text Video - Single-label classification - Multi-label classification - Image object detection - Image segmentation
- 17. Vertex AI: Managed dataset + objectives Vertex AI: Pipelines for your MLOps workflows @martonkodok Image
- 18. Create an image classification prediction model Vertex AI: Pipelines for your MLOps workflows @martonkodok
- 19. Create an image classification prediction model Vertex AI: Pipelines for your MLOps workflows @martonkodok
- 20. Create an image object detection model Vertex AI: Pipelines for your MLOps workflows @martonkodok
- 21. Vertex AI: Managed dataset + objectives Vertex AI: Pipelines for your MLOps workflows @martonkodok Tabular
- 22. Vertex AI: Managed dataset + objectives Vertex AI: Pipelines for your MLOps workflows @martonkodok Text
- 23. Vertex AI: Managed dataset + objectives Vertex AI: Pipelines for your MLOps workflows @martonkodok Video
- 24. Train models Vertex AI: Pipelines for your MLOps workflows @martonkodok Build dataset Train model Scalably deploy model Evaluate, monitor, retrain
- 25. 1. AutoML out-of-the box training integration No-code solution. You must target one of the AutoML’s predefined objectives. 2. Custom Training - run your own training applications in the cloud Train with one of the Google’s pre-builtcontainers or useyourown. Training @martonkodok
- 26. Pre-built containers for custom training https://cloud.google.com/vertex-ai/docs/training/pre-built-containers @martonkodok Tensorflow ML Framework version 1.15-2.9 use with Cuda 11.x GPU scikit-learn ML Framework version 0.23 No GPUs PyTorch ML Framework version 1.4 - 1.11 use with Cuda 11.x GPU XGBoost ML Framework version 1.1 No GPUs
- 27. Deploying models Vertex AI: Pipelines for your MLOps workflows @martonkodok Build dataset Train model Scalably deploy model Evaluate, monitor, retrain
- 28. “ You can deploy models on VertexAI and get a HTTPs Endpointsto serve predictions rapidly and reliably. Vertex AI: Pipelines for your MLOps workflows @martonkodok
- 29. 1. Deploy a model and get aREST endpointto serve predictions realtime or batched 2. You can use models whetherornotthemodelwastrained on Vertex AI. 3. Specify a prediction traffic split in your endpoint. 4. VPC Private Network option for custom-trained models/tabular models Vertex AI: Endpoints Vertex AI: Pipelines for your MLOps workflows @martonkodok Vertex AI Endpoints Backend Prediction deploy REST
- 31. Elements for ML systems Adapted from Hidden Technical Debt in Machine Learning Systems. @martonkodok
- 32. “ DevOpsprinciples to MLsystems Vertex AI: Pipelines for your MLOps workflows @martonkodok What is MLOps?
- 33. “Continuousdelivery and automationpipelines for machinelearning systems. Vertex AI: Pipelines for your MLOps workflows @martonkodok What is MLOps?
- 34. MLOpslevels @martonkodok 🖐 Level 0: Manual process Level 1: ML pipeline automation Level 2: CI/CD pipeline automation
- 35. Levelsofautomation defines maturity of theMLprocess Process Release cycle Development principle Trigger action MLOps level 0: Manual process Build & deploy is entirely manual Infrequent release iterations No CI/ CD None. Disconnection between ML and operations. MLOps level 1: ML pipeline automation MLOps level 2: CI/CD pipeline automation Vertex AI: Pipelines for your MLOps workflows @martonkodok Continuous training by automating the ML pipeline Achieve continuous delivery of model prediction Automate the build/train/deploy pipeline New pipelines mostly based on new data. Iteratively try out new ML algorithms, orchestrated experiment steps Continuous source code packaging. Build source. Build source, run test, output is pipeline. Trigger pipeline. New CI/CD pipeline based on source code/orchestrated experiments.
- 36. MLOps level 2: CI/CDpipelineautomation @martonkodok
- 37. Levelsofautomation defines maturity of theMLprocess
- 38. Levelsofautomation defines maturity of theMLprocess
- 39. Levelsofautomation defines maturity of theMLprocess
- 40. Using Pipelines throughout your ML workflow Part #3
- 41. What’s included in VertexAI? * in Preview @martonkodok Data Labeling AutoML models DL Environment (DL VM + DL Container) Prediction Feature Store Training Experiments Data Readiness Feature Engineering Training/ HP-Tuning Model Monitoring Model serving Understanding/ Tuning Edge Model Management Notebooks Pipelines (Orchestration) Explainable AI Hybrid AI Model Monitoring Metadata Vision-Video-NLP-Translate Models Datasets Custom Models Containers Python Endpoints BigQuery ML BigQuery Models Publisher Pre-trained models Model Garden GenAI Studio Foundation -> LLM Palm 2 API
- 42. “ Why are MLpipelines useful? Vertex AI: Pipelines for your MLOps workflows @martonkodok
- 43. 1. Orchestrate ML workflow steps as a process. We no longer handle all data gathering, model training, tuning, evaluation, deployment as a monolith. 2. Adopt MLOps for production models. We need a repeatable, verifiable, and automatic process for making any change to a production model. 3. Develop steps independently -as you scale out, enables you to share your ML workflow with others on your team, so they can run it, and contribute code. Enablesyoutotracktheinputandoutputfromeach stepinareproducibleway. Why are ML pipelines useful? @martonkodok
- 44. Vertex AI: Pipelines Vertex AI: Pipelines for your MLOps workflows Source: Piero Esposito https://github.com/piEsposito/vertex-ai-tutorials
- 45. Using Pipelines throughout your ML workflow Vertex AI: Pipelines for your MLOps workflows @martonkodok Gather data Train model Deploy model
- 46. Pipeline Components Vertex AI: Pipelines for your MLOps workflows pipeline_components_automl_images.ipynb github.com/GoogleCloudPlatform/vertex-ai-samples
- 47. Pipeline Components Vertex AI: Pipelines for your MLOps workflows
- 48. Using Pipelines throughout your ML workflow Vertex AI: Pipelines for your MLOps workflows @martonkodok Gather data Train model Evaluate model Scalably deploy model
- 49. Pipeline SDK: Condition Vertex AI: Pipelines for your MLOps workflows automl_tabular_classification_beans.ipynb github.com/GoogleCloudPlatform/vertex-ai-samples
- 50. 1. Use of the Google Cloud Pipeline Components, which support easy access to Vertex AI services 2. Custom Components - function that compiles to a task ‘factory’ function that can be used by pipelines 3. No more Kubeflow Pipelines that must be deployed on a Kubernetes Cluster. 4. Sharing component specifications - the YAML format allows the component to be put under version control and shared with others, or be used by other pipelines by calling the load_from_url function. 5. Leveraging Pipeline step caching to develop and debug Developer friendly components @martonkodok
- 51. What’s included in VertexAI? * in Preview @martonkodok Data Labeling AutoML models DL Environment (DL VM + DL Container) Prediction Feature Store Training Experiments Data Readiness Feature Engineering Training/ HP-Tuning Model Monitoring Model serving Understanding/ Tuning Edge Model Management Notebooks Pipelines (Orchestration) Explainable AI Hybrid AI Model Monitoring Metadata Vision-Video-NLP-Translate Models Datasets Custom Models Containers Python Endpoints BigQuery ML BigQuery Models Publisher Pre-trained models Model Garden GenAI Studio Foundation -> LLM Palm 2 API
- 52. Part #4 Adapting to changes of data
- 53. Automatic CI / CD Perspective with GCP Services Vertex AI: Pipelines for your MLOps workflows @martonkodok Eventarc • Detect changes on data • React to events from Cloud services • Handle events on Cloud Workflows, Cloud Functions, Cloud Run • Reuse pipeline spec.json from GCS • Trigger Vertex AI pipeline • Detect changes in codebase • Build pipeline • Pipeline spec.json to Cloud Storage • Image to Cloud Registry • Trigger Vertex AI pipeline Cloud Build Cloud Scheduler • Poll for changes of any data • Launch based on schedule • In tandem with Cloud Workflows • Trigger Vertex AI pipeline
- 54. Conclusion Vertex AI: Pipelines for your MLOps workflows @martonkodok
- 55. 1. Build with the groundbreaking ML tools that power Google 2. Approachable from the non-ML developer perspective (AutoML, managed models, training) 3. Ease the life of a data scientist/ML (has feature store, managed datasets, endpoints, notebooks) 4. End-to-end integration for data and AI with build pipelines that outperform and solve complex ML tasks 5. GitOps-style continuous delivery with Cloud Build 6. Explainable AI and TensorBoard to visualize and track ML experiments Vertex AI: Enhanced developer experience Vertex AI: Pipelines for your MLOps workflows @martonkodok
- 56. slideshare.net/martonkodok Vertex AI: Pipelines for your MLOps workflows @martonkodok
- 57. Twitter: @martonkodok Thank you. Q&A. Reea.net - Integrated web solutions driven by creativity to deliver projects. Follow for articles: martonkodok.medium.com Slides available on: slideshare.net/martonkodok