Netflix Big Data Paris 2017
Download as PPTX, PDF•4 likes•515 views
The document discusses Netflix's use of visual analytics with Tableau and big data to enhance content data engineering and analytics. It details their massive data infrastructure, the tools employed (like Hive, Spark, and Presto), and how they analyze user metrics to improve content delivery and customer experience. Key themes include the importance of scalability, speed in analytics, and strong user adoption to iterate quickly on insights.
Netflix Big Data Paris 2017
- 1. How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix - Content Data Engineering and Analytics #NetflixData
- 2. ● About Netflix ● Tableau + Big Data ○ Lessons Learned ○ Where we are today ● Analytics and Iterating Quickly
- 6. ● 93+ million members ● 190 countries ● 1,000+ devices ● 10B hours/qtr We plan on spending ~$6B in 2017 on content for our members Metrics
- 7. ● ~60 PB DW on S3 ● ~1400 Tableau users ● Live & extract connections ● Analytics on billions of rows
- 8. (Hadoop clusters) Storage Compute Data Interface Data Access, Analytics and Visualization AWS S3
- 9. ● About Netflix ● Tableau + Big Data ○ Lessons Learned ○ Where we are today ● Analytics and Iterating Quickly
- 10. Choosing a source ● Hive ● Spark ● Presto ● Redshift ● Published Data Source ● etc...
- 11. ● Powerful and scalable backend ● “Slower” 1,000,000,000/hr ● Hive + Tableau ○ Thrift Servers ○ Custom SQL vs Tables ○ Metadata ○ ODBC Optimization
- 12. ● Scalable ● Faster than Hive in many cases ● Spark + Tableau ○ Thrift Servers ○ Long running job on Cluster ○ Query reliability
- 13. ● Fast query engine ● Great for experimenting and “smaller” data sets ● Connecting to Tableau ○ Web data connector ○ ODBC
- 14. ● About Netflix ● Tableau + Big Data ○ Lessons Learned ○ Where we are today ● Analytics and Iterating Quickly
- 15. Tableau Data Extract Publish to Server Tableau Extract API
- 16. Create Tableau Data ExtractProvision Container ResourceIssues Command Create Extract Publish to Server Distributed Tableau Extract API
- 17. ● Very fast loads from S3 ● Native Tableau connector ● Quick Tableau Iteration ● Live or Extract ● Concurrency Amazon Redshift
- 18. BIG Data ● Too big to extract? ● Optimized live connections ○ SQL ● Custom data viz with Druid ● Tableau + Hyper!?
- 19. ● About Netflix ● Tableau + Big Data ○ Lessons Learned ○ Where we are today ● Analytics and Iterating Quickly
- 20. Business users Analytics Engineer Analytics: ● Binge Analysis ● Viewing Patterns ● Hours Viewed ● Customer Joy ● Content Quality
- 21. Bringing it all together ● Content analytics ● Iterate quickly ● Move between backend sources ● Strong user adoption