Productionizing H2O Models with Apache Spark with Jakub Hava and Michal Malohlava
1 like732 views
The document discusses the integration of H2O.ai and Apache Spark for productionizing machine learning models through a tool called Sparkling Water. It highlights the expertise of the authors and outlines the machine learning lifecycle, focusing on model training, feature engineering, and deployment processes. Additionally, it emphasizes the use of H2O Driverless AI for automatic feature engineering, making machine learning accessible to non-experts.
Productionizing H2O Models with Apache Spark with Jakub Hava and Michal Malohlava
- 1. Jakub Háva, Michal Malohlava; H2O.ai Productionizing H2O Models with Apache Spark #ML4SAIS
- 2. #ML4SAIS Who are we? • Michal • Chief Architect of Platforms at H2O.ai • Creator of Sparkling Water • Ph.D at Charles University (CZ), PostDoc at Purdue Uni (US) • Kuba • Senior Software engineer at H2O.ai - Core Sparkling Water • Master’s at Charles University (CZ) • Implemented high-performance cluster monitoring tool for JVM based languages (JNI, JVMTI, instrumentation) 2
- 6. Example Implementations 6#ML4SAIS Data Engineering Feature Engineering Training Algorithm Deployment Pipeline Model Spark H2O Spark H2O MOJO Spark H2O Driverless AI Spark H2O Driverless AI MOJO Model Building Model Deployment
- 7. H2O + Spark = Sparkling Water #ML4SAIS
- 8. #ML4SAIS H2O + Spark • H2O • Machine Learning Library • Distributed Algorithms • For ML experts • Sparkling Water • Integrates H2O & Spark Ecosystems • Transparent for Spark users • Based on Spark pipelines & H2O 8
- 9. Basic ML Lifecycle: Sparkling Water 9 Model Training Algorithm Feature Engineering Spark Transformers H2O MOJO Model Training Predictions Deployment Predictions AutoML Pipeline #ML4SAIS
- 12. #ML4SAIS H2O Driverless AI • What if I’m not expert ? • H2O Driverless AI • H2O Driverless AI • No expert knowledge required • Automatic Feature Engineering & ML 12
- 13. Basic ML Lifecycle: Driverless AI 13 Model Training Algorithm Feature Engineering Driverless AI Feature Transformations Driverless AI Model Training Predictions Deployment Predictions PipelineDriverless AI MOJO as #ML4SAIS
- 14. Demo: Driverless AI as Spark Pipeline #ML4SAIS
- 15. 15#ML4SAIS
- 20. #ML4SAIS Sparkling Water enables deployment of H2O ML models with Spark Pipelines 20 Thank you!