Unifying Twitter around a single ML platform - Twitter AI Platform 2019
2 likes281 views
The document discusses the unification of Twitter's machine learning (ML) platform, addressing various use cases, requirements, and challenges faced in the process. It highlights the difficulties of the old fragmented ML systems and presents a new comprehensive platform aimed at increasing efficiency and consistency across teams. Key lessons learned include the importance of automation, reproducibility, and collaboration among diverse ML practitioners.
Unifying Twitter around a single ML platform - Twitter AI Platform 2019
- 1. Unifying Twitter Around a Single ML Platform Yi Zhuang (@yz), Nicholas Leonard (@strife076) April 17, 2019
- 2. Overview • ML Use Cases at Twitter • ML Platform Requirements & Challenges • Unifying Twitter Around a Single ML Platform • Technology Migrations • Health ML Use Case • Summary of Lessons Learned • Future of Our ML Platform
- 3. Overview • ML Use Cases at Twitter • ML Platform Requirements & Challenges • Unifying Twitter Around a Single ML Platform • Technology Migrations • Health ML Use Case • Lessons Learned • Future of Our ML Platform
- 4. ML Use Cases: Tweet Ranking
- 5. ML Use Cases at Twitter: Ads pCTR = p ( “click” | if we show this Candidate Ad to this User in this Context) User Candidate Ad Context “Click”
- 6. ML Use Cases at Twitter • Other use cases • Recommending Tweets, Users, Hashtags, News, etc. • Detecting Abusive Tweets and Spam • Detecting NSFW Images and Videos • And so on …
- 7. ML Use Cases at Twitter ML is Everywhere
- 8. Overview • ML Use Cases at Twitter • ML Platform Requirements & Challenges • Unifying Twitter Around a Single ML Platform • Technology Migrations • Health ML Use Case • Summary of Lessons Learned • Future of Our ML Platform
- 9. Requirements of ML Platform Data Scale PBs of data per day Some models train on Tens of TBs of data per day
- 10. Requirements of ML Platform Prediction Throughput Tens of millions of predictions per second
- 11. Requirements of ML Platform Prediction Latency Budget tens of milliseconds
- 12. Example Use Case Ads Prediction Training examples everyday 1+B 1+M Features 40ms Serving latency Predictions every second10+M
- 13. Overview • ML Use Cases at Twitter • ML Platform Requirements & Challenges • Unifying Twitter Around a Single ML Platform • Technology Migrations • Health ML Use Case • Summary of Lessons Learned • Future of Our ML Platform
- 14. Challenges of Old ML Platform Fragmentation of ML Practice PyTorch Scikit Learn In-house Frameworks VW Lua Torch TensorFlow
- 15. Challenges of Old ML Platform Difficulty Sharing Models Tooling & Resources Knowledge
- 16. Challenges of Old ML Platform Inefficiencies Work Duplication
- 17. Example Duplicate Work Various Ways to do Model Training & Serving Model Refreshes Data Cleaning and Preprocessing Experiment Tracking Etc.
- 18. Overview • ML Use Cases at Twitter • ML Platform Requirements & Challenges • Unifying Twitter Around a Single ML Platform • Technology Migrations • Health ML Use Case • Lessons Learned • Future of Our ML Platform
- 19. New Unified ML Platform Overview A Single Consistent ML Platform Across Twitter 5 Producion M odelServing Lorem ipsum dolorsitam et,consectetur adipiscing elit,sed do eiusm od tem por. Donec facilisis lacus egetm auris. 4 Experim entation Tracking 3 M odelTraining and Evaluation Preprocessing and Featurization 21 Pipeline Orchestration
- 20. Overview • ML Use Cases at Twitter • ML Platform Requirements & Challenges • Unifying Twitter Around a Single ML Platform • Technology migrations • Health ML Use Case • Summary of Lessons Learned • Future of Our ML Platform
- 21. Technology Migrations ● Data Analysis: Scalding + PySpark/Notebooks ● Featurization: Feature Store ● ML Frameworks: Java ML -> Lua Torch -> TensorFlow ● Training and deployment cycles: Apache Airflow
- 22. Data Analysis: Scalding ● Scala ● Abstraction over hadoop ● Distributed data processing ● Great for large scale data ● Slow-iteration
- 23. Data analysis: Notebook + Spark ● iPython Notebook + PySpark ● Easier for Python engineers ● Data visualization ● Faster iteration
- 24. Lessons learned ML Practitioner Diversity Production ML Engineers Deep Learning Researcher Data Scientists
- 25. Featurization: Ad Hoc ● Teams use common data sources ○ E.g. user data, tweet data, engagement data ● Every team does their own featurization ○ Duplication of effort ● Difficult to validate features at serving time ○ Inconsistent featurization schemes for training vs serving
- 26. Featurization: Feature Store ● Teams can share, discover and access features ● Consistent training-time vs serving-time featurization
- 27. Lessons learned Consistency Consistency across teams => sharing & efficiency Important: feature consistency between training and serving
- 28. ML Frameworks: Java ML ● Logistic regression ○ Relies on feature discretization ● Typically used in an online learning environment: ○ Model learns new data as it becomes available (~15 min delay)
- 29. ML Frameworks: Lua Torch ● Deep learning ● Feature discretization parity ● ML Engineers didn’t want to learn Lua: ○ Lua hidden via YAML ○ Hard to debug and unit test ● Complex production setup ○ JVM -> JNI -> Lua VMs -> C/C++
- 30. ML Frameworks: TensorFlow ● Google support ● Production ready ○ Export graphs as protobuf ○ Serve graphs from Java/Scala: ■ JVM -> TensorFlow ● TensorBoard ● Large ecosystem (E.g. TFX)
- 31. Lessons learned Reproducibility is hard ... across different ML framework: small differences, large impacts Online experiments take time Need simple setup, fast iterations
- 32. Train and Deploy Cycles Different approaches to productionizing training algorithms: ● Manually re-train and re-deploy the model periodically ○ Retraining frequency varies ● Automate training and deployment cycles: ○ Cron, Aurora, Airflow Jobs ○ Helps reduce model staleness
- 33. Train and Deploy Cycle Apache Airflow: DAGs
- 35. Lessons learned Automation is crucial ML models become stale over time ML Hyperparameter tunings are often tedious
- 36. Overview • ML Use Cases at Twitter • ML Platform Requirements & Challenges at Twitter • Unifying Twitter Around a Single ML Platform • Technology Migrations • Health ML Use Case • Summary of Lessons Learned • Future of Our ML Platform
- 37. Health ML Case Study ● Situation: ○ Models still running using Lua Torch ○ Retrained manually every ~6 months. ● Mission: ○ Migrate Health ML models to new ML Platform ○ Reach metric parity with existing models (minimum)
- 38. ML Pipeline Overview Training Data Preprocessing Feature Store Data Exploration Training Offline Evaluation Model Tuning Experiment Loop Online A/B Testing Production Experiment Prediction Servers
- 39. Lessons Learned Teamwork: Platform, Modeling, Product Integration of All Components
- 40. Overview • ML Use Cases at Twitter • ML Platform Requirements & Challenges at Twitter • Unifying Twitter Around a Single ML Platform • Technology Migrations • Summary of Lessons Learned • Future of Our ML Platform
- 41. Summary of Lessons Learned ● Consistency brings efficiency ● DL Reproducibility is hard ● Automation is crucial ● ML practitioner Diversity ○ ML engineers vs DL researchers ○ Production vs exploration ● Collaboration of platform, modeling, product teams
- 42. Overview • ML Use Cases at Twitter • ML Platform Requirements & Challenges at Twitter • Unifying Twitter Around a Single ML Platform • Technology Migrations • Summary of Lessons Learned • Future of Our ML Platform
- 43. Future 2018 Strategy: Consistency & Adoption 2019 Strategy: Ease of Use & Velocity 10x, 50x training speed Auto model evaluation & validation Auto model deploy & auto scaling Auto hyperparameter tuning & architecture search Continuous Deep Learning Model Training and so on ...
- 44. Thank You If you are interested in learning more about Twitter Cortex, please contact: @yz @strife076