Hadoop as an Analytic Platform: Why Not?
- 1. Grab some coffee and enjoy the pre-show banter before the top of the hour!
- 2. Hadoop as an Analytic Platform: Why Not? The Briefing Room
- 3. Twitter Tag: #briefr The Briefing Room Welcome Host: Eric Kavanagh [email protected] @eric_kavanagh
- 4. ! Reveal the essential characteristics of enterprise software, good and bad ! Provide a forum for detailed analysis of today’s innovative technologies ! Give vendors a chance to explain their product to savvy analysts ! Allow audience members to pose serious questions... and get answers! Twitter Tag: #briefr The Briefing Room Mission
- 5. This Month: ANALYTIC PLATFORMS November: DISCOVERY & VISUALIZATION December: INNOVATORS Twitter Tag: #briefr The Briefing Room Topics 2014 Editorial Calendar at www.insideanalysis.com/webcasts/the-briefing-room
- 6. A NEW ERA of Architecture Twitter Tag: #briefr The Briefing Room Executive Summary Ø Don’t build CARRIAGES for highways Ø Focus on NEW opportunities Ø SLOWLY ween off old systems
- 7. Twitter Tag: #briefr The Briefing Room Analyst: William McKnight William is President of McKnight Consulting Group. His clients have included 17 of the Global 2000. Many clients have gone public with their success story. His team's implementations have won multiple Best Practices awards. William is an Entrepreneur of the Year Finalist, a frequent best practices judge and an expert witness. He has hundreds of articles and dozens of white papers in publication. William has also given numerous keynote presentations worldwide at major conferences and has given hundreds of public seminars and webinars. William’s experience includes taking his company to placement on the Inc. 500 and the Dallas 100 to seller of a multi-million dollar consulting firm. He is a passionate communicator and motivator, and a former IT VP of a Fortune 50 company.
- 8. Twitter Tag: #briefr The Briefing Room Actian ! Actian is a database and software development company ! The Actian Analytics Platform connects to data and Big Data sources to perform actionable and advanced analytics ! Actian recently released Hadoop SQL Edition, a component that enables SQL access on data stored in Hadoop
- 9. Twitter Tag: #briefr The Briefing Room Guest: Jim Hare Jim Hare is Senior Director of Product Marketing for the Actian Analytics Platform, helping organizations transform big data into business value. Prior to Actian, he was Director of Marketing at IBM responsible for go-to-market strategy and messaging for the big data platform. Prior to joining IBM in 2008, Jim was vice president of product marketing and business development at Celequest, a California-based operational business intelligence vendor, which was acquired by Cognos in 2007. He has over 16 years of deep experience in enterprise software, business intelligence, business process management, business activity monitoring, big data, and automated software testing & monitoring. Jim holds a MS in Systems Management from the University of Southern California, and an undergraduate degree from the University of Colorado at Boulder.
- 10. Hadoop as an Analy'c Pla8orm: Why Not? Jim Hare 14 October 2014 Confiden'al © 2014 10 Ac'an Corpora'on
- 11. Ac'an is Delivering Transforma'onal Value 11 “Actian is now very powerfully positioned in the big data and analytics markets.” Robin Bloor Confiden'al © 2014 11 $140M Revenues + Profitable 10,000+ Customers Global Presence: 8 world-wide offices, 7x 24 multinational support model Ac'an Corpora'on “Actian has assembled all of the next generation IPs into a single analytics platform, allowing users a level of flexibility in data interaction that competitors have not been able to match.” siliconANGLE
- 12. Emergence of Hadoop as the Data Reservoir Confiden'al © 2014 12 Low Cost Storage for New Data & Offload Ac'an Corpora'on
- 13. Benefits of Hadoop Ø Scalable -‐ Ø Cost effec.ve – Confiden'al © 2014 13 store large data sets across low cost servers Ac'an Corpora'on 1/10th the cost of tradi'onal data storage Ø Flexible -‐ quickly and easily land any data in raw format Ø Fast Access –'maps' data wherever it is located on a cluster Ø Resilient -‐ data is replicated to other nodes in the cluster
- 14. Hadoop Deployment Survey Results First Solu.on Deployed 10% Pilot First 14% Solu.on Confiden'al © 2014 14 Suppor.ng Mul.ple Analy.cs 10% Ac'an Corpora'on Only 22% of Hadoop projects are in production today! 14 8% 41% 16% 11% Exploring and educa.ng 41% 16% 8% 14% 11% Conduc.ng POC Developing First Solu.on Source: SandHill Group Research, “How do you Hadoop? A Survey of Big Data Practitioners”, May 2014
- 15. Because It isn’t Easy to Analyze Hadoop Data Confiden'al © 2014 15 Ac'an Corpora'on Batch performance is Slow Lengthy Time to Discover Insights Expensive Skills Lack of Data Access & Security Data preparation is time-consuming Analytics Complexity
- 16. Organiza'ons are Replica'ng Hadoop Data to Overcome Analy'c Challenges Confiden'al © 2014 16 Ac'an Corpora'on Predic've Rela'onal Data Store OLTP, ERP, CRM Unstructured docs, emails Server logs Social/Web data Sensor, machine data Geoloca'on Clickstream Discovery Analy'cs BI Hadoop • Duplicate storage & infrastructure costs • More IT resources to manage • Network bandwidth usage • Less accuracy from data Sampling • Slower ?me to analysis results
- 17. Crea'ng Transforma'onal Value from Hadoop Data Data Explosion Confiden'al © 2014 17 Ac'an Corpora'on Transforma.onal Value ? Customer Delight Competitive Advantage World-Class Risk Management Disruptive New Business Models Actian Analytics PlatformTM Time-‐Sensi.ve Analy.cs Discovery Analy.cs Ac6an Transforms Hadoop from a Data Reservoir into a High Performance Analy6cs PlaCorm ! Highest performing, most industrialized Hadoop analy'cs pla8orm ! Only end-‐to-‐end analy.c processing na'vely in Hadoop ! Most consumable, accessible, manageable Hadoop analy'cs Analyze Act Connect Hadoop
- 18. 7 Ingredients Added to Hadoop to Unlock Value 1. High speed integra'on to on-‐board data from any data Connect to Any Data Anywhere 200+ Connectors Confiden'al © 2014 18 Ac'an Corpora'on Elastic and Secure • Schedule • Transform • Validate • Aggregate • Reformat • Join • Orchestrate ENERPRISE Data SOCIAL Data MACHINE Data CLOUD Data LEGACY Data DEVICE Data Embeddable High Throughput Engine Drag and Drop Workflow Designs Capture Data Feeds in Batch or Real-Time Expandable Plugin Framework High Volume Parallel Data Processing source and any type
- 19. 7 Ingredients Add to Hadoop to Unlock Value 1. High speed integra'on to on-‐board data from any data source and any type 2. Visual Framework for connec'ng, blending, & enriching data, data science discovery, building and tes'ng predic've models Connect Blend & Enrich Discover Build & Test Models Confiden'al © 2014 19 Coding Ac'an Corpora'on
- 20. 7 Ingredients Added to Hadoop to Unlock Value 1. High speed integra'on to on-‐board data from any data source and any type 2. Visual Framework: connec'ng, blending, & enriching data, data science discovery, building and tes'ng predic've models 3. 1500 KNIME Operators + R analy'cs running in parallel on HDFS + Hadoop = The Open Source Trifecta Confiden'al © 2014 20 Ac'an Corpora'on Gartner Magic Quadrant for Advanced Analytics Platforms Source: Gartner (February 2014)
- 21. Complete End-‐to-‐End Analy'cs on Hadoop Confiden'al © 2014 21 Ac'an Corpora'on Source: 2013 Rexer Analy'cs Survey
- 22. 7 Ingredients Added to Hadoop to Unlock Value 1. High speed integra'on to on-‐board data from any data source and any type 2. Visual Framework: connec'ng, blending, & enriching data, data science discovery, building and tes'ng predic've models 3. 1500 KNIME Operators + R analy'cs running in parallel on HDFS + Hadoop = The Open Source Trifecta 4. High-‐Performance, YARN-‐based data processing engine running on HDFS Confiden'al © 2014 22 Read Write Prepare Analyze Analyze Read Write Ac'an Corpora'on On-Node Processing Actian DataPrep LEADER Optimizer
- 23. High Performance, Parallelized Processing on HDFS Without Any Programming Confiden'al © 2014 23 Ac'an Corpora'on Ac.an Analy.cs PlaPorm Query Pipelining CPU Pipelining Optimized, On-HDFS Processing Hadoop – Leader Node Reuse and share all components from operators to workflows Optimize Choose from five sets of operators: Connections Transformation Data Quality Analytics Data Science Automatically detect resources, plan optimal utilization, and parallelize all workloads on Hadoop Use dual pipeline parallelism to accelerate performance 30X Take processing to where the data lives, runs natively on Hortonworks Run fully optimized processing directly on the Hadoop node via YARN Visual Framework Manage the entire analytic process in a visual framework with no coding required. ≠ ☼ ≡ ∞ Δ Σ √ ≈ Σ = ? # ~ ‰
- 24. 7 Ingredients Added to Hadoop to Create Value 1. High speed integra'on to on-‐board data from any data source and any type 2. Visual Framework: connec'ng, blending, & enriching data, data science discovery, building and tes'ng predic've models 3. 1500 KNIME Operators + R analy'cs running in parallel on HDFS + Hadoop = The Open Source Trifecta 4. High-‐Performance, YARN-‐based data processing engine running on HDFS 5. High-‐Performance, vector processing engine as the pajern for SQL on Hadoop Confiden'al © 2014 24 Ac'an Corpora'on
- 25. Vector-‐based SQL Processing Na'vely on HDFS Confiden'al © 2014 25 HADOOP YARN Datanode Visual Data & Analytics Workbench High Performance, Namenode Industrialized SQL X100 X100 X100 HDFS HDFS Ac'an Corpora'on Datanode HDFS Datanode HDFS Datanode HDFS Read Load Ac'an Vector Blend & Enrich Data Science & Analy'cs Datanode HDFS X100 Database High Performance, Parallelized Data Flow Engine SQL Industrialized Standards - ANSI SQL 92 plus advanced analytics Optimized - mature, proven planner and optimizer Secure – native DBMS security Reliable - full ACID-compliance Manageable – YARN certified Performance - 30X faster than Impala Scalable – unlimited expansion as Hadoop cluster grows Native – runs natively on top of HDFS via YARN
- 26. 7 Ingredients Added to Hadoop to Unlock Value 1. High speed integra'on to on-‐board data from any data source and any type 2. Visual Framework: connec'ng, blending, & enriching data, data science discovery, building and tes'ng predic've models 3. 1500 KNIME Operators + R analy'cs running in parallel on HDFS + Hadoop = The Open Source Trifecta 4. High-‐Performance, YARN-‐based data processing engine running on HDFS 5. High-‐Performance, vector processing engine as the pajern for SQL on Hadoop 6. Extreme-‐Performance, super-‐low latency, massively parallel analy'cs engine Confiden'al © 2014 26 Ac'an Corpora'on
- 27. Ac'an Analy'cs Pla8orm: Next Genera'on Big Data Analy'cs Amazon Redshift Confiden'al © 2014 27 Ac'an Corpora'on Actian Analytics PlatformTM Libraries of Analytics Visual Framework for Data and Analytic Workflows Massively Parallel Integration Hadoop Sophisticated, Low Latency Analytics in Database Connections for Any Data Enterprise Data Machine Data Social Data Users Machines Business Processes Applications Data Warehouse Analytic Services Real-Time SaaS Data High Performance Data Science Natively on Hadoop
- 28. 7 Ingredients Added to Hadoop to Unlock Value 1. High speed integra'on to on-‐board data from any data source and any type 2. Visual Framework: connec'ng, blending, & enriching data, data science discovery, building and tes'ng predic've models 3. 1500 KNIME Operators + R analy'cs running in parallel on HDFS + Hadoop = The Open Source Trifecta 4. High-‐Performance, YARN-‐based data processing engine running on HDFS 5. High-‐Performance, vector processing engine as the pajern for SQL on Hadoop 6. Extreme-‐Performance, super-‐low latency, massively parallel analy'cs engine 7. Blueprints to accelerate analy'cs applica'on development and value crea'on Confiden'al © 2014 28 Ac'an Corpora'on
- 29. Big Data 2.0 Media Mix Modeling Blueprint All Relevant Sales Histories Detailed ePOS Receipts Marketing Vehicle Details All Relevant Account Info and Demographics Campaign Response Notes Confiden'al © 2014 29 Ac'an Corpora'on MARKETING IMPACT ANALYSIS IMPACT FORECAST ANALYSIS CRMdb CONNECT BUILD CUSTOMER PROFILE EDWdb ANALYZE ACT MAXIMIZE REVENUE FROM MARKETING SENTIMENT AND CONTENT ANALYSIS AGGREGATE SALES DATA Hadoop Logs SKU LEVEL SALES DATA BY GEO JOIN DERIVE AGGREGATE PREPARE EDWdb CAPTURE MARKETING VEHICLES MARKETING MIX SALES CONTRIBUTION YEARLY CHANGE REPORT SALES VOLUME, EFFECTIVENES S, EFFICIENCY AND ROI REPORT NEW MEDIA MIX OPTIMIZATION MINIMIZE MARKETING SPEND TO REVENUE RATIO Hadoop Text Files CUSTOMER MATCH PREPARE FOR TEXT ANALYTICS WITH CAMPAIGNS VEHICLE RESULTS AT GEO, STORE, SKU AND CUSTOMER LEVEL
- 30. Learn more about the Ac'an Analy'cs Pla8orm – Hadoop SQL Edi'on 1) Stay tuned for several exci'ng announcements on 16 October 2014 at the Strata Conference in NYC! 2) Visit Ac'an at Booth Confiden'al © 2014 30 Ac'an Corpora'on 225 for a demo of Ac'an Analy'cs Pla8orm -‐ the Highest Performing Analy'cs & SQL in Hadoop 3) Download and try it out yourself: bigdata.ac'an.com/sql-‐in-‐Hadoop
- 31. Thank You www.ac'an.com facebook.com/ac'ancorp @ac'ancorp Confiden'al © 2014 31 Ac'an Corpora'on
- 32. Twitter Tag: #briefr The Briefing Room Perceptions & Questions Analyst: William McKnight
- 33. ANALYTICS: A BUSINESS IMPERATIVE Formed from SUMMARIES of data Tied to Business Actions Continual Re-evaluation i.e., Customer Segmentation and Profit Adding Big Data!
- 34. ANALYTICS EXAMPLES Number of customers in each customer state (optionally by product or multiple products) Average balance of customers by geo Average start date in each customer lifetime value decile by geo and device New number of customers in each state Propensity to churn by age band and device Cost of acquisition by age and gender Average session duration by cost of acquisition Session duration differences between first and tenth session Network with highest up time last month Number of calls per session Best performing ad network by day part in a geo, age band and device And on and on and on and on….
- 36. SMARTER MARKETING Spend + Media Arbitrage Opportunities + Incremental Direct Marketing Spend Improvement: Map Media Buys to the Best Customer Demographic Do sponsorships align with customer base? Monitored transactions, renewals, customer care calls Leveraged data to pitch right product, right time Decrease in marketing cost Increase in revenue, profit, customer satisfaction
- 37. VEHICLES FOR BIG DATA Data Warehouse Regional and Departmental Views Applications & Engines Operational Analytics & Hot Views Data Marts ADS Dependent Independent Relational Data Conformed Dimensions
- 38. Last Year THE EVER-EXPANDING DATA WAREHOUSE This Year Next Year • Enterprise Data Warehouse users face huge annual upgrade expenses • To avoid this spend, organizations are looking for lower cost alternatives. • Movement of data to tape not desired, because data is offline and not available for analytics • Moving infrequently used data to Hadoop is a cost-effective, online option that preserves ability to query Cost
- 39. DATA WAREHOUSE EXPANSION 2 Offload data to less expensive Hadoop cluster to save on data management costs As data volume increases exponentially, cost of warehousing rises also Add operational data for greater insight and agility in analytics and BI 4 1 Combine Hadoop data with DW data for a more comprehensive view of history 3 HD FS HD FS HD FS
- 40. QUESTIONS FOR ACTIAN Where should analytics be created – in a relational environment or in Hadoop? Where should they be analyzed? Do we have enough tools in a Hadoop environment to do analysis there? How do businesses analyze a combination of structured and unstructured data? Is it as simple as ‘structured data to the data warehouse or analytic one-offs and unstructured data to Hadoop’? Is using Hadoop as a data refinery the best use of Hadoop? Does any data go to both environments? Or do just summaries get shared? Can price/performance of a database vendor’s product be superior to an open source product?
- 41. Twitter Tag: #briefr The Briefing Room
- 42. This Month: ANALYTIC PLATFORMS November: DISCOVERY & VISUALIZATION December: INNOVATORS Twitter Tag: #briefr The Briefing Room Upcoming Topics 2014 Editorial Calendar at www.insideanalysis.com/webcasts/the-briefing-room 2015 Editorial Calendar coming soon! www.insideanalysis.com
- 43. Twitter Tag: #briefr THANK YOU for your ATTENTION! Opening slide image courtesy of Wikimedia Commons The Briefing Room