Open Source Framework for Deploying Data Science Models and Cloud Based Applications by Noelle Sio of Pivotal
2 likes1,639 views
The document discusses the use of an open-source framework for deploying data science models in media companies, emphasizing the importance of leveraging both structured and unstructured data to gain insights. It outlines various data science opportunities, such as customer segmentation and campaign optimization, and emphasizes the need for actionable insights that drive business value. Additionally, it highlights the benefits of cloud-based applications and the integration of tools like Hadoop and machine learning for effective data analytics in the industry.
Open Source Framework for Deploying Data Science Models and Cloud Based Applications by Noelle Sio of Pivotal
- 1. Open Source Framework for Deploying Data Science Models and Cloud Based Applications Pivotal Data Science Team
- 3. What happened? What should I do about it? This is where Data Science comes in What will happen next?
- 4. What Thought Leaders Have In Common Large amounts of structured and unstructured data Deep personal knowledge of their audience Quantified understanding of their products Data-driven culture User experience optimized by data science
- 5. Viewership Advertisements Merchandise Sales & Finance $ Market Research & Competitive Information Audience Demographics Internal Data Sources Typical External Sources Semi/Unstructured Data Clickstream Social Media Content
- 6. Data Science Impact Business Motivation Increase Demand Build Brand Equity Increase Production Efficiency Optimize Ad Spend Efficiency Increase Customer Engagement • Campaign Optimization • Marketing Mix Models Data Science Opportunities • Customer segmentation • Affinity analysis • Social media analytics • Supply/Demand forecasting Increase Revenue Reduce Cost
- 7. Example Use Case: Ratings Prediction Use Case: Increase ratings across viewer demographics How: • Data: Viewership, transcripts and show data combined in big data platform • Model: Machine learning used to identify the impact of production decisions on viewership Insights
- 8. Models Insights Actions Models are built to answer business questions e.g. what makes viewers tune- in and tune-out? Data Scientists interpret models for answers e.g. On screen arguments make viewers tune out Report Dashboard BI Tool Email Presentation Cloud App End User A good insight drives action that will generate value for stakeholders
- 9. Revisiting Rating Prediction Use Case Model exposed to end users via cloud application allowing what-if scenario building
- 10. Characteristics Of Actionable Insights Real-time ScalableSocial Relevant Accessible Open
- 11. Benefits Of Cloud Based Applications Service failure or data loss at scale Long innovation cycles Poor experience at scale Resilient, scale-out messaging and processing Agile development with cloud based data services Low-latency, in- memory computing
- 12. Open Source Analytics Ecosystem Media companies benefit from algorithmic breadth and scalability for building and socializing data science models MLlib PL/X Algorithms Visualization Best of breed in-memory and in-database tools for an MPP platform
- 13. Example Scalable Open Source Platform Hadoop++: Complementing the Hadoop platform are Data Science modeling tools. SQL on Hadoop (e.g. HAWQ), Python/R interfaces to SQL, Apache Spark etc. http://opendataplatform.org/ Apps Data Analytics Leading Media companies are moving towards a platform with Hadoop at the core.
- 14. Data Science Pipeline On Hadoop++ MLlib PL/X Data Lake Hadoop++ Structured + Unstructured Data
- 15. Open Source Framework For Ratings Prediction Data Lake Insights and Model Results Ratings Predictions Business Levers Hosted on What-if Scenario ApplicationContains structured + unstructured data MLlib PL/X
- 16. Gather video ads impression stats Data Lake Ingest Message Broker Simulate Ad Server Behavior Impression Forecasts Business Levers Hosted on Business Metrics Dashboard Expanding The Framework To Include Impression Forecasting Modeling MLlib PL/X
- 17. Measuring Audience Engagement : Workflow Parallel Parsing of JSON (PL/Python) Twitter Decahose (~55 million tweets/day) Source: http Sink: hdfs HDFS External Tables PXF Nightly Cron Jobs Topic Analysis through MADlib pLDA Unsupervised Sentiment Analysis (PL/Python) Hosted on
- 18. Key Takeaways • Blended data sets lead to richer models and more valuable insights • Turn Data Science models and insights into value generating actions through data driven applications. • Open source = power and flexibility • Platform extensibility is key to supporting Data Science • Turnkey PaaS is available through CloudFoundry, including infrastructure monitoring, server configuration and scalability.
- 19. THANK YOU!