Introduction to data science with H2O-Chicago
5 likes4,006 views
This document provides an overview of H2O.ai, an open source in-memory machine learning platform. It describes H2O.ai's product as an in-memory prediction engine, its team of 37 distributed systems engineers doing machine learning, and its headquarters in Mountain View, CA. It also provides details on how to use H2O with R and Python for scalable machine learning on large datasets across distributed systems.
Introduction to data science with H2O-Chicago
- 1. Scalable Machine Learning For Smarter Applications
- 2. Who am I? Hank Roark ([email protected], @hankroark) Data Scientist & Hacker @ H2O.ai Lecturer in Systems Thinking, UIUC 13 years at John Deere doing Research, New Product Development, New High Tech Ventures Previously at startups and consulting Physics Georgia Tech Systems Design & Management MIT
- 3. • Founded: 2011 venture-backed, debuted in 2012 • Product: H2O open source in-memory prediction engine • Team: 37 - Distributed Systems Engineers doing ML • HQ: Mountain View, CA H2O.ai Overview H2O.ai Machine Intelligence
- 4. 25,000 commits / 3yrs H2O World Conference 2014 Team Work @ H2O.ai 28 Join H2O World Nov 9-11 2015!
- 5. What is H2O? Math Platform • Open source in- memory prediction engine • Parallelized and distributed algorithms making the most use API • Easy to use and adopt • Written in Java – perfect for Java Programmer s • REST API (JSON) – Big Data • More data? Or better models? BOTH • Use all of your data – model without down sampling • Run a H2O.ai Machine Intelligence
- 6. Accuracy with Speed and Scale
- 7. H2O.ai Machine Intelligence H2O Software Stack JavaScript R Python Excel/Tableau Network Rapids Expression Evaluation Engine Scala Customer Algorithm Customer Algorithm Parse GLM GBM RF Deep Learning K-Means PCA In-H2O Prediction Engine Fluid Vector Frame Distributed K/V Store Non-blocking Hash Map Job MRTask Fork/Join Flow Customer Algorithm Spark Hadoop Standalone H2O
- 8. H2O.ai Machine Intelligence Reading Data from HDFS into H2O with R STEP 1 R user h2o_df = h2o.importFile(“hdfs://path/to/data.csv”)
- 9. H2O.ai Machine Intelligence Reading Data from HDFS into H2O with R H2O H2O H2O data.csv HTTP REST API request to H2O has HDFS path H2O ClusterInitiate distributed ingest HDFS Request data from HDFS STEP 2 2.2 2.3 2.4 R h2o.importFile() 2.1 R function call
- 10. H2O.ai Machine Intelligence Reading Data from HDFS into H2O with R H2O H2O H2O R HDFS STEP 3 Cluster IP Cluster Port Pointer to Data Return pointer to data in REST API JSON Response HDFS provides data 3.3 3.4 3.1h2o_df object created in R data.csv h2o_df H2O Frame 3.2 Distributed H2O Frame in DKV H2O Cluster
- 11. H2O.ai Machine Intelligence R Script Starting H2O GLM HTTP REST/JSON .h2o.startModelJob() POST /3/ModelBuilders/glm h2o.glm() R script Standard R process TCP/IP HTTP REST/JSON /3/ModelBuilders/glm endpoint Job GLM algorithm GLM tasks Fork/Join framework K/V store framework H2O process Network layer REST layer H2O - algos H2O - core User process H2O process Legend
- 12. H2O.ai Machine Intelligence R Script Retrieving H2O GLM Result HTTP REST/JSON h2o.getModel() GET /3/Models/glm_model_id h2o.glm() R script Standard R process TCP/IP HTTP REST/JSON /3/Models endpoint Fork/Join framework K/V store framework H2O process Network layer REST layer H2O - algos H2O - core User process H2O process Legend
- 13. Step 1 • Download and install h2o: h2o.ai, hit download button • SL Guest, password SerendipityOSW • Only requirement is JDK 1.7+ • plus required packages if using R or Python • Pick R, Python (2.7.x), or Standalone for tonight 13
- 14. Step 2 • http://bit.ly/1SNkmFa • Contains training and validation data, starter R and Python scripts • Unzip chicago.zip 14
- 15. Thank You (final holdout test set for Chicago Meetup is at https://s3.amazonaws.com/0xdata-public/hank/mikeditka.csv) Thank you Chicago for a great time!