Building a Just in Time Data Warehouse by Dan Morris and Jason Pohl
7 likes•2,893 views
This document summarizes Viacom's approach to building a just-in-time data warehouse for product analytics. It discusses how Viacom uses Spark and Databricks on Amazon S3 to enable flexible exploratory analysis and A/B testing without needing to pre-define schemas or move data into a data warehouse. This just-in-time approach allows Viacom's product analysts to quickly get insights, test hypotheses, and monitor key metrics through dashboards for their mobile apps across 23 brands in over 170 countries. The summary highlights how Viacom uses this flexible approach to support their goals of growing global audiences through disciplined product development and data-driven decision making.
Building a Just in Time Data Warehouse by Dan Morris and Jason Pohl
- 1. Building a Just-In-Time Data Warehouse Dan Morris (Viacom) JasonPohl (Databricks) February 18, 2016
- 2. About Viacom • LeadingGlobalEntertainmentContentCompany • 23 Brandsin 170+ Countries 2
- 3. Introductions Dan Morris • SeniorDirectorof ProductAnalytics • 12 Years @Viacomin a varietyof roles • Intersectionof ProductandData 3
- 4. About My Team • ProductAnalytics team formedone year ago • Ourmissionis to grow our globalaudiencewiththe highestquality userspossible 4
- 5. Key Areas of Focus • Mobilizeeffortsusing growthtargets • Uncoverdeepinsightsusing churnand cohort analysis • Treat all ideas as hypothesesand test them rigorously 5
- 6. Where Are We Today: App Platform 6 Make it extremelysimpleto build and deploy engaging apps around the globe FEATURES UI & ANIMATIONS CONFIGURATION SETTINGS APP BINARY
- 7. Disciplined Product Dev Approach is Key 7 • 23 brandsin 170+ countries • Lots of marketdynamics • Many stakeholders ... Data is a must!
- 8. Sound Data Management is Required 8 9 11 13 14 16 18 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Expected Data Volume Growth (TB) 2016
- 9. Our Data Infrastructure 9 S3 Spark + Databricks Redshift Tableau
- 10. Introducing the TV Land iOS App 10
- 11. Applying Product Analytics to TV Land 11 • Growth Targets • Dashboards • DeepDive Analyses • A/B Testing
- 12. Baselines Used to Set Growth Targets 12 Business ModelingETL Data Volume • 30 sites/apps • 11 TB Data Volume • 30 sites/apps • 1 TB S3 Spark + Databricks Redshift Tableau
- 13. Growth Targets are Monitored via Dashboards 13 1/4/16 1/11/16 1/18/16 1/25/16 New Users Returning Users Weekly Retention by Cohort 0 1 2 3 4 1/4/16 100% 53% 41% 33% 30% 1/11/16 100% 58% 51% 42% 1/18/16 100% 49% 38% 1/25/16 100% 49% Audience Growth by Cohort
- 14. Dashboards Spark Deep Dive Analyses 14 0% 25% 50% 75% 100% 0 1 2 3 4 5 6 7 8 9 10 11 12 Users will quickly churn if not activated within small window of time. %ofusersretained
- 15. Deep Dive Analysis Requires Flexibility 15 Not NeededDeep Dive Analysis S3 Spark + Databricks • Define schema on read instead of write • Work through data quality issues just-in-time. • Tease out business question iterative and interactively. • Use programming language of your choice. Redshift Tableau
- 16. Hypotheses Require A/B Testing 16 Statistical Analysis Data Sets • Adobe Logs • Experiment Logs S3 Spark + Databricks Tableau Not Needed Redshift Tableau
- 17. Summary of Our Setup 17 Just in Time Traditional Primary Audience • Product Analysts • Product Team • Business Stakeholders Tasks • Exploratory Analysis • A/B Testing • Ad Hoc Queries • Dashboards Tools • S3 • Spark • Databricks • Redshift • Tableau
- 18. Coming Soon… 18 • Go live with internalA/B testing platform • Continueto evolve our setup • Further scale model to support Product AnalyticsPan- Viacom
- 19. Questions ? 19