Why we fail at ml ai why we fail at ml_ai
Download as PPTX, PDF1 like28 views
The document discusses a monthly meeting focused on data science and machine learning strategies on the Google Cloud Platform, highlighting challenges and opportunities in the field. It emphasizes the importance of better data practices, collaboration, and innovative thinking to prevent failures in AI/ML initiatives. Additionally, it outlines 15 strategic goals to improve success rates in data-driven projects, stressing the significance of explainability, documentation, and the use of cloud computing tools.
![1. Stop using your laptop
[2]
SECRET SAUCE: Cloud computing](https://image.slidesharecdn.com/whywefailatmlaiwhywefailatmlai-200108232833/85/Why-we-fail-at-ml-ai-why-we-fail-at-ml_ai-13-320.jpg)
![3. Show data better
[3]
SECRET SAUCE: Exploratory Data Analysis
SECRET SAUCE: Unsupervised Machine Learning](https://image.slidesharecdn.com/whywefailatmlaiwhywefailatmlai-200108232833/85/Why-we-fail-at-ml-ai-why-we-fail-at-ml_ai-15-320.jpg)
![9. Teach the art of measuring prediction
[5]
SECRET SAUCE: show and tell, storytelling
SECRET SAUCE: regular checkpoints](https://image.slidesharecdn.com/whywefailatmlaiwhywefailatmlai-200108232833/85/Why-we-fail-at-ml-ai-why-we-fail-at-ml_ai-21-320.jpg)
![10. Learn to get along with other Data Scientists
SECRET SAUCE: collaborate
SECRET SAUCE: else compromise
[5]](https://image.slidesharecdn.com/whywefailatmlaiwhywefailatmlai-200108232833/85/Why-we-fail-at-ml-ai-why-we-fail-at-ml_ai-22-320.jpg)
![11. Take the time to show and tell
SECRET SAUCE: story telling
SECRET SAUCE: checkpoints
[5]](https://image.slidesharecdn.com/whywefailatmlaiwhywefailatmlai-200108232833/85/Why-we-fail-at-ml-ai-why-we-fail-at-ml_ai-23-320.jpg)
![13. Build production pipelines
25
Train / Retrain Model
Review AUC ROC
accuracies and signoff
acquired from business
Deploy Production Model
Live Prediction
Model Life Cycle
Model Repository
Version control of model
(github, bitbucket …)
SECRET SAUCE: CI/CD
SECRET SAUCE: O&M
[6]](https://image.slidesharecdn.com/whywefailatmlaiwhywefailatmlai-200108232833/85/Why-we-fail-at-ml-ai-why-we-fail-at-ml_ai-25-320.jpg)
![14. Treat each prediction as a learning lesson
26
Train / Retrain Model
Review AUC ROC
accuracies and signoff
acquired from business
Deploy Production Model
Live Prediction
Reviewed by a SME
(Subject Matter Expert)
Model Life Cycle
Model Repository
Version control of model
(github, bitbucket …)
Ground Truth Corrections
New Feature Types
Corrected Records
New Records
SECRET SAUCE: Human in the loop
[6]](https://image.slidesharecdn.com/whywefailatmlaiwhywefailatmlai-200108232833/85/Why-we-fail-at-ml-ai-why-we-fail-at-ml_ai-26-320.jpg)
![Credits
[1] 1. Paleoanthropology – The History of Our Tribe: Hominini icon
[2] https://blog.dominodatalab.com/cost-data-science-laptops/
[3] https://blog.magrathealabs.com/choosing-one-of-many-python-visualization-tools-7eb36fa5855f
[4] https://www.businessinsider.com/everything-you-need-to-know-about-beer-in-one-chart-2015-8
[5] https://xkcd.com/605/
[6] Copyright Maven Wave Partners
Giffy: “Machine Times GIF - Find & Share on GIPHY_files”, “Confused Curb Your Enthusiasm GIF - Find & Share on GIPHY_files”, “Stewie Griffin Laughing GIF - Find & Share
on GIPHY_files”, “Laptop Goodbye GIF - Find & Share on GIPHY_files”, “Domestic Violence Caveman GIF - Find & Share on GIPHY_files”, “Fail Star Wars GIF - Find & Share on
GIPHY_files”](https://image.slidesharecdn.com/whywefailatmlaiwhywefailatmlai-200108232833/85/Why-we-fail-at-ml-ai-why-we-fail-at-ml_ai-28-320.jpg)
Why we fail at ml ai why we fail at ml_ai
- 2. Data Science on GCP 4th Meeting! Meetings every month. Website: https://www.gcpchi.com/ Open discussions, presentations and competitions around data science, web analytics and everything GCP (Google Cloud Platform). Come learn, present, eat, and drink
- 3. Why we fail at ML/AI? How to innovate and maintain love & science. Brian Ray, Managing Director of Data Science at Maven Wave Partners CHECK out my podcast http://ai-podcast.com CHECK out my blog https://medium.com/@brianray_7981/ Join the team: [email protected]
- 4. 75% Of all enterprises FAILED to have an AI/ML Strategy. And over half (60%) know they are failing. Is it our fault? IDC survey
- 5. Are data scientists bad at innovation?
- 6. Failure to innovate on a corporate level. Netflix’s 10-Year 4,000% Rally Underlines Shift to Streaming Ryan Vlastelica, Bloomberg, December 30, 2019, 7:00 AM CST
- 7. Typical Timeline No Data, Data Security, Quality of Data Technology isn’t available, too expensive, or accessible Internal team too busy or not qualified It’s just not a priority for now No technology Yay, we won a project, let’s get to work No Data? Unclear on results No Production No Updates No Review Not reproducible
- 8. Caveman goes hunting Attacked by dinosaur Drops Food Goes back Tells other caveman Invest crop into growing Grows food Tells others Wife is mad others are mad?
- 10. Secret sauce in prevent failure Factors Paleoanthropology Data Scientists Business Trust Does the wife trust results? Is the process and data supportive? Does business understand prediction? Census Does the tribe? Do data scientists agree on results? Will the business agree to invest? Process Reap what you sow Reproducible results along the way Can investment produce reward? Tools Stone tools Machine Learning Visualization Tools Success Criteria Not Starve Not Starve Not Starve
- 11. Better Timeline Yay, we won a project, let’s get to work Relevant Data Enough Data Enriched Data Explainable Results Path to production With human in the loop Better Exploratory Data Science Sooner Code reviewed Reproducible And retrainable
- 12. Top 15 goals to help you not fail. 1. Stop using your laptop 2. Start showing data sooner 3. Show data better 4. Fear not to change directions 5. Show other data 6. Learn to try more, faster 7. Keep track of what you tried 8. Focus on explainability 9. Teach the art of measuring prediction 10. Learn to get along with other Data Scientists 11. Take the time to show and tell 12. Forget not what was done and were 13. Build production pipelines 14. Treat each prediction as a learning lesson 15. Repeat With Examples from:
- 13. 1. Stop using your laptop [2] SECRET SAUCE: Cloud computing
- 14. 2. Show data sooner Depending on the data consider: ● Looker ● GCP DataLab ● TensorFlow Board / Projector ● Consider notebook friendly visualization like: ○ Bokeh ○ Plotly ○ …. ● Google Sheet: Data Connectors! SECRET SAUCE: Rapid Dashboarding SECRET SAUCE: Data Visualization TensorFlow Ai Platform Notebooks Looker
- 15. 3. Show data better [3] SECRET SAUCE: Exploratory Data Analysis SECRET SAUCE: Unsupervised Machine Learning
- 16. 4. Fear not change directions A Model to classify 100’s of beers? VS 100 of models to classify 100’s of beers? SECRET SAUCE: Goal based process SECRET SAUCE: discovery head-room
- 17. 5. Show Other Data SECRET SAUCE: Data Access (datalake, swamp, ..) SECRET SAUCE: Cloud computing Reference table BigQuery Unstructured Cloud Storage Data matching Cloud Dataflow
- 18. 6. Learn to try more, faster SECRET SAUCE: Data Science Pipelines SECRET SAUCE: Automation Kubeflow GKE/Kubernetes ETL Dataflow
- 19. 7. Keep track of what you tried SECRET SAUCE: Logging and reporting SECRET SAUCE: Document Collaboration
- 20. 8. Focus on explainability SECRET SAUCE: Game Theory (Shapley)
- 21. 9. Teach the art of measuring prediction [5] SECRET SAUCE: show and tell, storytelling SECRET SAUCE: regular checkpoints
- 22. 10. Learn to get along with other Data Scientists SECRET SAUCE: collaborate SECRET SAUCE: else compromise [5]
- 23. 11. Take the time to show and tell SECRET SAUCE: story telling SECRET SAUCE: checkpoints [5]
- 24. 12. Forget not what was done and were https://www.youtube.com/watch?v=xU_xdog XFeE&feature=youtu.be SECRET SAUCE: use cloud platform tools SECRET SAUCE: use version controls Git Repos Compute Engine Ai Platform Notebooks
- 25. 13. Build production pipelines 25 Train / Retrain Model Review AUC ROC accuracies and signoff acquired from business Deploy Production Model Live Prediction Model Life Cycle Model Repository Version control of model (github, bitbucket …) SECRET SAUCE: CI/CD SECRET SAUCE: O&M [6]
- 26. 14. Treat each prediction as a learning lesson 26 Train / Retrain Model Review AUC ROC accuracies and signoff acquired from business Deploy Production Model Live Prediction Reviewed by a SME (Subject Matter Expert) Model Life Cycle Model Repository Version control of model (github, bitbucket …) Ground Truth Corrections New Feature Types Corrected Records New Records SECRET SAUCE: Human in the loop [6]
- 27. 15. Repeat Brian Ray, Managing Director of Data Science at Maven Wave Partners @brianray CHECK out my podcast http://ai- podcast.com CHECK out my blog https://medium.com/@brianray_7981/ Join the team: [email protected] SECRET SAUCE: Success!
- 28. Credits [1] 1. Paleoanthropology – The History of Our Tribe: Hominini icon [2] https://blog.dominodatalab.com/cost-data-science-laptops/ [3] https://blog.magrathealabs.com/choosing-one-of-many-python-visualization-tools-7eb36fa5855f [4] https://www.businessinsider.com/everything-you-need-to-know-about-beer-in-one-chart-2015-8 [5] https://xkcd.com/605/ [6] Copyright Maven Wave Partners Giffy: “Machine Times GIF - Find & Share on GIPHY_files”, “Confused Curb Your Enthusiasm GIF - Find & Share on GIPHY_files”, “Stewie Griffin Laughing GIF - Find & Share on GIPHY_files”, “Laptop Goodbye GIF - Find & Share on GIPHY_files”, “Domestic Violence Caveman GIF - Find & Share on GIPHY_files”, “Fail Star Wars GIF - Find & Share on GIPHY_files”
Editor's Notes
- #2: Daniel’s slide