SlideShare a Scribd company logo
7
Most read
19
Most read
21
Most read
Productionzing ML Model using
MLflow Model Serving
• Nagaraj Sengodan
• Nitin Raj Soundararajan
S Nagaraj has helped build the larger
enterprise-wide distributed systems.
Senior Manager in Architecture and
Engineering team, helping envision
and deliver the future for enterprise
analytics via Databricks and Azure
Synapse.
Nitin Raj Soundararajan is a technical
consultant focusing on advanced data
analytics, data engineering, cloud
scale analytics and data science.
Agenda
§ MLflow
§ MLflow Serving
§ Manage Served Versions
§ Monitor Served Models
§ Customize Serving Cluster
§ Q & A
MLFlow
• Open machine learning platform
• Works with any ML Library & Language
• Runs the same way anywhere (e.g. any cloud)
• Open interface design (use with any code you already
have)
MLFlow
Machine Learning
lifecycle
MLflow Tracking - Record and query experiments: code, data, config, and
results
MLflow Projects - Package data science code in a format to reproduce runs
on any platform
MLflow Models - Deploy machine learning models in diverse serving
environments
MLflow Registry - Store, annotate, discover, and manage models in a
central repository
MLFlow
AutoML
End-to-End ML Lifecycle
ML Runtime and
Environments
Batch and
Streaming
Online Serving
Data Science Workspace
Prep Data Build Model Deploy/Monitor Model
Open,
pluggable
architecture
MLFlow – Model Serving
Prep Data Build Model Deploy/Monitor Model
AutoML
End-to-End ML Lifecycle
Batch and
Streaming
Online Serving
Data Science Workspace
Open,
pluggable
architecture
ML Runtime and
Environments
Serving
MLflow Serving
• Expose Mlflow model predictions as REST endpoint
• Small cluster is automatically provisioned
• HTTP endpoint publicly exposed
• Limited production capability for now
• For now, intended for light loads and testing
Model Serving from Model Registry
Models
Flavor 2
Flavor 1
Custom
Models
Model Serving from Model Registry
Models Tracking
Flavor 2
Flavor 1
Custom
Models
Parameters Metrics Artifacts
Models
Metadata
Model Serving from Model Registry
Staging Production Archived
Data Scientists Deployment Engineers
v1
v2
v3
Models Tracking
Flavor 2
Flavor 1
Model Registry
Custom
Models
Parameters Metrics Artifacts
Models
Metadata
Model serving from Model Registry
Staging Production Archived
Data Scientists Deployment Engineers
v1
v2
v3
Models Tracking
Flavor 2
Flavor 1
Model Registry
Custom
Models
In-Line Code
Containers
Batch & Stream
Scoring
Cloud Inference
Services
OSS Serving
Solutions
Serving
Parameters Metrics Artifacts
Models
Metadata
MLFlow Serving
Demo
Manage Served Versions
• All active (non-archived) model versions are deployed
• Manage model access rights
• Source deployed model versions
• Source via UI
• Source via REST API request
Monitor Served Models
• Displays status indicators for the serving cluster as well as
individual model versions
• Inspect the state of the serving cluster - displays a list of
all serving events for this model
• To inspect the state of a single model version
Customize Serving Cluster
• Modify the memory size and number of cores of a serving
cluster
• Ability to add the tags
• Ability to edit or delete an existing tags
Demo Code & Deck
• https://github.com/KRSNagaraj/Mod
elServing-DataAISummit2021
Q & A
Feedback
Your feedback is important to us.
Don’t forget to rate and review the sessions.

More Related Content

What's hot (20)

PDF
Databricks Overview for MLOps
Databricks
 
PDF
Drifting Away: Testing ML Models in Production
Databricks
 
PPTX
Pythonsevilla2019 - Introduction to MLFlow
Fernando Ortega Gallego
 
PPTX
What Is Apache Spark? | Introduction To Apache Spark | Apache Spark Tutorial ...
Simplilearn
 
PDF
Apply MLOps at Scale
Databricks
 
PDF
Seamless MLOps with Seldon and MLflow
Databricks
 
PDF
Managing the Complete Machine Learning Lifecycle with MLflow
Databricks
 
PPTX
From Data Science to MLOps
Carl W. Handlin
 
PDF
MLFlow: Platform for Complete Machine Learning Lifecycle
Databricks
 
PPTX
MLOps Virtual Event | Building Machine Learning Platforms for the Full Lifecycle
Databricks
 
PPTX
Free Training: How to Build a Lakehouse
Databricks
 
PPTX
Introducing MlFlow: An Open Source Platform for the Machine Learning Lifecycl...
DataWorks Summit
 
PDF
Mlflow with databricks
Liangjun Jiang
 
PDF
Pinterest - Big Data Machine Learning Platform at Pinterest
Alluxio, Inc.
 
PDF
MLOps with Kubeflow
Saurabh Kaushik
 
PDF
MLOps - Build pipelines with Tensor Flow Extended & Kubeflow
Jan Kirenz
 
PPTX
Introduction to ML with Apache Spark MLlib
Taras Matyashovsky
 
PDF
Introduction to Spark with Python
Gokhan Atil
 
PDF
A Thorough Comparison of Delta Lake, Iceberg and Hudi
Databricks
 
PDF
End to end Machine Learning using Kubeflow - Build, Train, Deploy and Manage
Animesh Singh
 
Databricks Overview for MLOps
Databricks
 
Drifting Away: Testing ML Models in Production
Databricks
 
Pythonsevilla2019 - Introduction to MLFlow
Fernando Ortega Gallego
 
What Is Apache Spark? | Introduction To Apache Spark | Apache Spark Tutorial ...
Simplilearn
 
Apply MLOps at Scale
Databricks
 
Seamless MLOps with Seldon and MLflow
Databricks
 
Managing the Complete Machine Learning Lifecycle with MLflow
Databricks
 
From Data Science to MLOps
Carl W. Handlin
 
MLFlow: Platform for Complete Machine Learning Lifecycle
Databricks
 
MLOps Virtual Event | Building Machine Learning Platforms for the Full Lifecycle
Databricks
 
Free Training: How to Build a Lakehouse
Databricks
 
Introducing MlFlow: An Open Source Platform for the Machine Learning Lifecycl...
DataWorks Summit
 
Mlflow with databricks
Liangjun Jiang
 
Pinterest - Big Data Machine Learning Platform at Pinterest
Alluxio, Inc.
 
MLOps with Kubeflow
Saurabh Kaushik
 
MLOps - Build pipelines with Tensor Flow Extended & Kubeflow
Jan Kirenz
 
Introduction to ML with Apache Spark MLlib
Taras Matyashovsky
 
Introduction to Spark with Python
Gokhan Atil
 
A Thorough Comparison of Delta Lake, Iceberg and Hudi
Databricks
 
End to end Machine Learning using Kubeflow - Build, Train, Deploy and Manage
Animesh Singh
 

Similar to Productionzing ML Model Using MLflow Model Serving (20)

PPTX
DAIS Europe Nov. 2020 presentation on MLflow Model Serving
amesar0
 
PPTX
MLflow Model Serving - DAIS 2021
amesar0
 
PDF
ML Model Serving at Twitter
Zhiyong (Joe) Xie
 
PDF
Productionizing Real-time Serving With MLflow
Databricks
 
PDF
MLflow with Databricks
Liangjun Jiang
 
PDF
MLflow: Infrastructure for a Complete Machine Learning Life Cycle
Databricks
 
PDF
"Managing the Complete Machine Learning Lifecycle with MLflow"
Databricks
 
PPTX
Open, Secure & Transparent AI Pipelines
Nick Pentreath
 
PDF
Managing the Machine Learning Lifecycle with MLflow
Databricks
 
PDF
Managing the Machine Learning Lifecycle with MLOps
Fatih Baltacı
 
PDF
MLflow: Infrastructure for a Complete Machine Learning Life Cycle
Databricks
 
PDF
Flink Forward San Francisco 2018: Dave Torok & Sameer Wadkar - "Embedding Fl...
Flink Forward
 
PDF
Utilisation de MLflow pour le cycle de vie des projet Machine learning
Paris Data Engineers !
 
PPTX
Nasscom ml ops webinar
Sameer Mahajan
 
PDF
MLFlow 1.0 Meetup
Databricks
 
PDF
RESTful Machine Learning with Flask and TensorFlow Serving - Carlo Mazzaferro
PyData
 
PDF
A survey on Machine Learning In Production (July 2018)
Arnab Biswas
 
PDF
Operationalizing Machine Learning: Serving ML Models
Lightbend
 
PDF
Hopsworks at Google AI Huddle, Sunnyvale
Jim Dowling
 
PDF
mlflow: Accelerating the End-to-End ML lifecycle
Databricks
 
DAIS Europe Nov. 2020 presentation on MLflow Model Serving
amesar0
 
MLflow Model Serving - DAIS 2021
amesar0
 
ML Model Serving at Twitter
Zhiyong (Joe) Xie
 
Productionizing Real-time Serving With MLflow
Databricks
 
MLflow with Databricks
Liangjun Jiang
 
MLflow: Infrastructure for a Complete Machine Learning Life Cycle
Databricks
 
"Managing the Complete Machine Learning Lifecycle with MLflow"
Databricks
 
Open, Secure & Transparent AI Pipelines
Nick Pentreath
 
Managing the Machine Learning Lifecycle with MLflow
Databricks
 
Managing the Machine Learning Lifecycle with MLOps
Fatih Baltacı
 
MLflow: Infrastructure for a Complete Machine Learning Life Cycle
Databricks
 
Flink Forward San Francisco 2018: Dave Torok & Sameer Wadkar - "Embedding Fl...
Flink Forward
 
Utilisation de MLflow pour le cycle de vie des projet Machine learning
Paris Data Engineers !
 
Nasscom ml ops webinar
Sameer Mahajan
 
MLFlow 1.0 Meetup
Databricks
 
RESTful Machine Learning with Flask and TensorFlow Serving - Carlo Mazzaferro
PyData
 
A survey on Machine Learning In Production (July 2018)
Arnab Biswas
 
Operationalizing Machine Learning: Serving ML Models
Lightbend
 
Hopsworks at Google AI Huddle, Sunnyvale
Jim Dowling
 
mlflow: Accelerating the End-to-End ML lifecycle
Databricks
 
Ad

More from Databricks (20)

PPTX
DW Migration Webinar-March 2022.pptx
Databricks
 
PPTX
Data Lakehouse Symposium | Day 1 | Part 1
Databricks
 
PPT
Data Lakehouse Symposium | Day 1 | Part 2
Databricks
 
PPTX
Data Lakehouse Symposium | Day 2
Databricks
 
PPTX
Data Lakehouse Symposium | Day 4
Databricks
 
PDF
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
Databricks
 
PDF
Democratizing Data Quality Through a Centralized Platform
Databricks
 
PDF
Learn to Use Databricks for Data Science
Databricks
 
PDF
Why APM Is Not the Same As ML Monitoring
Databricks
 
PDF
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
Databricks
 
PDF
Stage Level Scheduling Improving Big Data and AI Integration
Databricks
 
PDF
Simplify Data Conversion from Spark to TensorFlow and PyTorch
Databricks
 
PDF
Scaling your Data Pipelines with Apache Spark on Kubernetes
Databricks
 
PDF
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Databricks
 
PDF
Sawtooth Windows for Feature Aggregations
Databricks
 
PDF
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Databricks
 
PDF
Re-imagine Data Monitoring with whylogs and Spark
Databricks
 
PDF
Raven: End-to-end Optimization of ML Prediction Queries
Databricks
 
PDF
Processing Large Datasets for ADAS Applications using Apache Spark
Databricks
 
PDF
Massive Data Processing in Adobe Using Delta Lake
Databricks
 
DW Migration Webinar-March 2022.pptx
Databricks
 
Data Lakehouse Symposium | Day 1 | Part 1
Databricks
 
Data Lakehouse Symposium | Day 1 | Part 2
Databricks
 
Data Lakehouse Symposium | Day 2
Databricks
 
Data Lakehouse Symposium | Day 4
Databricks
 
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
Databricks
 
Democratizing Data Quality Through a Centralized Platform
Databricks
 
Learn to Use Databricks for Data Science
Databricks
 
Why APM Is Not the Same As ML Monitoring
Databricks
 
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
Databricks
 
Stage Level Scheduling Improving Big Data and AI Integration
Databricks
 
Simplify Data Conversion from Spark to TensorFlow and PyTorch
Databricks
 
Scaling your Data Pipelines with Apache Spark on Kubernetes
Databricks
 
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Databricks
 
Sawtooth Windows for Feature Aggregations
Databricks
 
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Databricks
 
Re-imagine Data Monitoring with whylogs and Spark
Databricks
 
Raven: End-to-end Optimization of ML Prediction Queries
Databricks
 
Processing Large Datasets for ADAS Applications using Apache Spark
Databricks
 
Massive Data Processing in Adobe Using Delta Lake
Databricks
 
Ad

Recently uploaded (20)

PPTX
apidays Singapore 2025 - The Quest for the Greenest LLM , Jean Philippe Ehre...
apidays
 
PDF
How to Connect Your On-Premises Site to AWS Using Site-to-Site VPN.pdf
Tamanna
 
PDF
apidays Helsinki & North 2025 - How (not) to run a Graphql Stewardship Group,...
apidays
 
PDF
Avatar for apidays apidays PRO June 07, 2025 0 5 apidays Helsinki & North 2...
apidays
 
PDF
Merits and Demerits of DBMS over File System & 3-Tier Architecture in DBMS
MD RIZWAN MOLLA
 
PPTX
apidays Singapore 2025 - Designing for Change, Julie Schiller (Google)
apidays
 
PDF
Choosing the Right Database for Indexing.pdf
Tamanna
 
PPTX
apidays Munich 2025 - Building Telco-Aware Apps with Open Gateway APIs, Subhr...
apidays
 
PPTX
apidays Singapore 2025 - From Data to Insights: Building AI-Powered Data APIs...
apidays
 
PPTX
Module-5-Measures-of-Central-Tendency-Grouped-Data-1.pptx
lacsonjhoma0407
 
PDF
Copia de Strategic Roadmap Infographics by Slidesgo.pptx (1).pdf
ssuserd4c6911
 
PDF
OOPs with Java_unit2.pdf. sarthak bookkk
Sarthak964187
 
PPTX
apidays Helsinki & North 2025 - From Chaos to Clarity: Designing (AI-Ready) A...
apidays
 
PPTX
AI Presentation Tool Pitch Deck Presentation.pptx
ShyamPanthavoor1
 
PDF
Web Scraping with Google Gemini 2.0 .pdf
Tamanna
 
PPTX
SlideEgg_501298-Agentic AI.pptx agentic ai
530BYManoj
 
PPTX
Advanced_NLP_with_Transformers_PPT_final 50.pptx
Shiwani Gupta
 
PPTX
apidays Munich 2025 - Building an AWS Serverless Application with Terraform, ...
apidays
 
PPTX
apidays Helsinki & North 2025 - API access control strategies beyond JWT bear...
apidays
 
PPT
AI Future trends and opportunities_oct7v1.ppt
SHIKHAKMEHTA
 
apidays Singapore 2025 - The Quest for the Greenest LLM , Jean Philippe Ehre...
apidays
 
How to Connect Your On-Premises Site to AWS Using Site-to-Site VPN.pdf
Tamanna
 
apidays Helsinki & North 2025 - How (not) to run a Graphql Stewardship Group,...
apidays
 
Avatar for apidays apidays PRO June 07, 2025 0 5 apidays Helsinki & North 2...
apidays
 
Merits and Demerits of DBMS over File System & 3-Tier Architecture in DBMS
MD RIZWAN MOLLA
 
apidays Singapore 2025 - Designing for Change, Julie Schiller (Google)
apidays
 
Choosing the Right Database for Indexing.pdf
Tamanna
 
apidays Munich 2025 - Building Telco-Aware Apps with Open Gateway APIs, Subhr...
apidays
 
apidays Singapore 2025 - From Data to Insights: Building AI-Powered Data APIs...
apidays
 
Module-5-Measures-of-Central-Tendency-Grouped-Data-1.pptx
lacsonjhoma0407
 
Copia de Strategic Roadmap Infographics by Slidesgo.pptx (1).pdf
ssuserd4c6911
 
OOPs with Java_unit2.pdf. sarthak bookkk
Sarthak964187
 
apidays Helsinki & North 2025 - From Chaos to Clarity: Designing (AI-Ready) A...
apidays
 
AI Presentation Tool Pitch Deck Presentation.pptx
ShyamPanthavoor1
 
Web Scraping with Google Gemini 2.0 .pdf
Tamanna
 
SlideEgg_501298-Agentic AI.pptx agentic ai
530BYManoj
 
Advanced_NLP_with_Transformers_PPT_final 50.pptx
Shiwani Gupta
 
apidays Munich 2025 - Building an AWS Serverless Application with Terraform, ...
apidays
 
apidays Helsinki & North 2025 - API access control strategies beyond JWT bear...
apidays
 
AI Future trends and opportunities_oct7v1.ppt
SHIKHAKMEHTA
 

Productionzing ML Model Using MLflow Model Serving

  • 1. Productionzing ML Model using MLflow Model Serving • Nagaraj Sengodan • Nitin Raj Soundararajan
  • 2. S Nagaraj has helped build the larger enterprise-wide distributed systems. Senior Manager in Architecture and Engineering team, helping envision and deliver the future for enterprise analytics via Databricks and Azure Synapse. Nitin Raj Soundararajan is a technical consultant focusing on advanced data analytics, data engineering, cloud scale analytics and data science.
  • 3. Agenda § MLflow § MLflow Serving § Manage Served Versions § Monitor Served Models § Customize Serving Cluster § Q & A
  • 4. MLFlow • Open machine learning platform • Works with any ML Library & Language • Runs the same way anywhere (e.g. any cloud) • Open interface design (use with any code you already have)
  • 5. MLFlow Machine Learning lifecycle MLflow Tracking - Record and query experiments: code, data, config, and results MLflow Projects - Package data science code in a format to reproduce runs on any platform MLflow Models - Deploy machine learning models in diverse serving environments MLflow Registry - Store, annotate, discover, and manage models in a central repository
  • 6. MLFlow AutoML End-to-End ML Lifecycle ML Runtime and Environments Batch and Streaming Online Serving Data Science Workspace Prep Data Build Model Deploy/Monitor Model Open, pluggable architecture
  • 7. MLFlow – Model Serving Prep Data Build Model Deploy/Monitor Model AutoML End-to-End ML Lifecycle Batch and Streaming Online Serving Data Science Workspace Open, pluggable architecture ML Runtime and Environments
  • 9. MLflow Serving • Expose Mlflow model predictions as REST endpoint • Small cluster is automatically provisioned • HTTP endpoint publicly exposed • Limited production capability for now • For now, intended for light loads and testing
  • 10. Model Serving from Model Registry Models Flavor 2 Flavor 1 Custom Models
  • 11. Model Serving from Model Registry Models Tracking Flavor 2 Flavor 1 Custom Models Parameters Metrics Artifacts Models Metadata
  • 12. Model Serving from Model Registry Staging Production Archived Data Scientists Deployment Engineers v1 v2 v3 Models Tracking Flavor 2 Flavor 1 Model Registry Custom Models Parameters Metrics Artifacts Models Metadata
  • 13. Model serving from Model Registry Staging Production Archived Data Scientists Deployment Engineers v1 v2 v3 Models Tracking Flavor 2 Flavor 1 Model Registry Custom Models In-Line Code Containers Batch & Stream Scoring Cloud Inference Services OSS Serving Solutions Serving Parameters Metrics Artifacts Models Metadata
  • 15. Demo
  • 16. Manage Served Versions • All active (non-archived) model versions are deployed • Manage model access rights • Source deployed model versions • Source via UI • Source via REST API request
  • 17. Monitor Served Models • Displays status indicators for the serving cluster as well as individual model versions • Inspect the state of the serving cluster - displays a list of all serving events for this model • To inspect the state of a single model version
  • 18. Customize Serving Cluster • Modify the memory size and number of cores of a serving cluster • Ability to add the tags • Ability to edit or delete an existing tags
  • 19. Demo Code & Deck • https://github.com/KRSNagaraj/Mod elServing-DataAISummit2021
  • 20. Q & A
  • 21. Feedback Your feedback is important to us. Don’t forget to rate and review the sessions.