Cloudera - Mike Olson - Hadoop World 2010
3 likes•1,008 views
The document discusses the growth and impact of Hadoop over the past 4 years. It notes that Hadoop is already widely used, with over 500PB of data and thousands of implementations. It also discusses how Hadoop enables new types of analytics and insights through its ability to handle large and diverse datasets. The document outlines the growing ecosystem around Hadoop and predicts continued growth and innovation in both the technology and its applications in industries and businesses.
Cloudera - Mike Olson - Hadoop World 2010
- 1. Hadoop: What’s Next? Mike Olson
- 2. Reflections On You 12+ months using on average 114.5TB average size 66 average nodes in Use 500+ certified on Hadoop in 1 year 60+PB Total Data from pre-conference survey
- 4. Immutable Law of Data RDBMS Hadoop Volume, Variety, Velocity increase
- 5. Immutable Law of Data RDBMS Hadoop Volume, Variety, Velocity increase Geopbytes Brontobytes Yottabytes Zettabytes Exabytes Terabytes
- 6. Linked Complex Unstructured Pre-relational Raw Detailed Heterogeneous Dirty Graphs Large Schemaless
- 7. Hadoop Was Built for Data.
- 9. Room to Grow
- 11. Hadoop: The Core of a Platform
- 12. A Platform Built by You Hue Hue SDK OozieOozie HBaseFlume, Sqoop Zookeeper / Avro Hive Pig/ Hive
- 14. A Platform Enabling Applications… Query & Reporting Complex ETL Trade Compliance POS Analysis Search Quality Click Stream Analysis Machine Learning Graph Analysis And More… Fraud Detection Archive Scientific Security
- 15. Solving Critical Business Problems • Modeling true risk • Customer churn analysis • Recommendation engine • Ad targeting • PoS transaction analysis • Analyzing network data to predict failure • Threat analysis • Trade surveillance • Search quality • Data “sandbox”
- 16. • Capture critical IT data • Monitoring usage • Driving bottom line value
- 17. • Risk analysis • Customer insight • Drive growth
- 18. • Customer intimacy • Precision targeting • Driving top line growth
- 19. So Much To See Today! • Optimizing search • Advanced analytics in the Army • Using Flume &Hive for log data • Analyzing VOIP data with R
- 20. What’s Next? Market • Adoption • Agility • Flexibility Technology • Accelerated innovation from community • More tools e.g., monitoring • More automation • More stability • More interfaces
- 21. • At the core of the open source platform for data • Four years old and going strong!
- 23. Organizational Impact • More knobs and dials • Fine grain control • Achieve previously impossible / impractical • Save money • Save time • Greater flexibility with data Copyright 2010 Cloudera Inc. All rights reserved
- 24. Hadoop World Keynote (NOTES) • Themes – Hadoop is already a big deal • Keep in mind the why • Solving real problems now – It is about the platform with Hadoop at the core • Why • Helps you profit • More accessible now than ever, real people with enterprise ops and enterprise skills, no longer the exclusive demand of the PhDs – What’s on the Horizon for Hadoop Copyright 2010 Cloudera Inc. All rights reserved
- 25. Hadoop is Having a Transformative Impact (notes) • Continued growth and excitement • Transformative to your career, your enterprise, your market – Star maker – Get ready for Hadoop being a big deal for your companies – Your market – hyper personalization – Use data to interact in a more customized fashion – “It’s hard not to have a TB of data” – Mike – Operability and SLAs for a critical enterprise platform – Education and training – A new stack for analytics (CEP (flume) CDH (Sqoop) dbms/BI) • Future is now – Use cases now and impact it is having and where it will be, look at Facebook, Yahoo, eBay etc. Copyright 2010 Cloudera Inc. All rights reserved
- 26. What is on the Horizon for Hadoop (notes) • Continued growth and excitement • Transformative to your career, your enterprise, your market – Star maker – • good for your career, help make critical changes in the way customers are supported, major new business opportunities etc. • Pull cloudera certification #’s – Get ready for Hadoop being a big deal for your companies • Enterprise will be more agile and able capture and analyze more data to better target ads, find fraud, etc. • Agility – impacts the things that matter to you • What’s happened before the transaction – Your market – hyper personalization • 100s’s of vertical apps to be created (developers are you listening?) • Trend that crosses? Any other trend we can compare to? DBMS growth? Improvements in operations, • How detailed sources have changed • Devices, understanding how people interact with your business – retail, online entertainment, fin serv, government – Use data to interact in a more customized fashion – “It’s hard not to have a TB of data” – Mike – Operability and SLAs for a critical enterprise platform – Education and training – A new stack for analytics (CEP (flume) CDH (sqoop) dbms/BI) • Future is now – Use cases now and impact it is having and where it will be, look at Facebook, Yahoo, eBay etc. Copyright 2010 Cloudera Inc. All rights reserved
- 27. Emerging Importance of Data Scientist • Able to impact business at many levels • New conference focused data and data related roles — O’Reilly Strata Conference Copyright 2010 Cloudera Inc. All rights reserved
- 28. Unprecedented Data Volume, Velocity and Variety Data Growth Out Pacing Processing Power Organizations Swamped and Turning to Hadoop 61% CAGR 42% CAGR Data Transistors Copyright 2010 Cloudera Inc. All rights reserved
- 29. Transforming Analytic Requirements • Insight into this data needs more than simple tabular analysis – More is needed for meaningful answers • You can and will do deeper and more introspective analysis – Machine learning, natural language processing, clustering, sophisticated statistical analysis, modeling and back testing • Looking for patterns – You can see patterns in lots of data that are invisible in less data. You need pattern discovery tools Copyright 2010 Cloudera Inc. All rights reserved
- 30. Hadoop: Already a Big Deal!! Massive Adoption Vibrant & Growing Community 100’s of PB Under Management 1000’s of Implementations
- 31. Benefitting From a Dynamic OS Community • Community around Hadoop is proliferating and expanding • > ½ Hadoop sub-projects promoted to TLPs • Dozens of related projects • 100’s of developers & growing Copyright 2010 Cloudera Inc. All rights reserved
- 32. Interest in Hadoop Has Exploded More are looking for it Leading analysts report significant growth in inquiries Major increase in coverage Copyright 2010 Cloudera Inc. All rights reserved
- 33. A Data Management Platform Applications Copyright 2010 Cloudera Inc. All rights reserved
- 34. Market Impact • Hyper personalization • Extreme targeting • Expand competitive advantages • Better retention of customers • Improved risk analysis