Hybrid Cloud Strategy for Big Data and Analytics
6 likes•4,447 views
The document discusses a hybrid cloud strategy for big data and analytics, highlighting the integration of on-premise and public cloud environments to optimize data management and compliance. It outlines the advantages of hybrid cloud, including cost efficiency, agility, and regulatory adherence, and emphasizes the importance of data sovereignty and security. Implementation considerations are addressed, recommending a tailored approach based on specific organizational needs and workloads.
Hybrid Cloud Strategy for Big Data and Analytics
- 1. Hybrid Cloud Strategy for Big Data & Analytics Christine Ouyang Marcio Moura 04/05/2017
- 2. Agenda – What is Hybrid Cloud? – Why Hybrid Cloud? – The North Star, Hybrid Cloud Big Data & Analytics – Implementation Considerations for Hybrid Cloud – Hybrid Cloud Big Data & Analytics Use Cases
- 3. Agenda – What is Hybrid Cloud?
- 4. What is Hybrid Cloud? The connection of on-premise environments and/or dedicated cloud with public cloud. LOCAL PUBLIC ON-PREMISES DEDICATED PRIVATE VPN Tunnel
- 5. The Integrated Digital Enterprise is Hybrid Data Engineers Developer Data Scientists Business Analyst Data Owner CDO Devices IoT Communities APIs/Services Social & Internet Advanced Analytics Big Data Digital Transformation APIs/Service s Real-Time Analytics Statistical Modeling Information Governance & Security Self-Service Data & Analytics Public Cloud Dedicated Cloud On-Premises Metadata Catalog Public Cloud Dedicated Cloud On-Premises / Local Cloud Analytics Statistical Modeling Edge Analytics Sensors Sensors VPN Tunnel Private LOCAL
- 6. Agenda – Why Hybrid Cloud?
- 7. Why Hybrid Cloud? • Provides the ability for different personas to tap into data and analytics where it makes the most sense (localization wise) • Allows the analytics to run where the data is stored (data gravity) • Provides better management of legal and regulatory requirements in terms of privacy regulation associated with data sovereignty and regulatory requirements such as HIPAA, PCI, and SOX • Provides an agile platform for developing/deploying new applications • Allows workload portability and cost optimization
- 8. Hybrid Cloud is the path forward Dedicated Cloud & IT Benefits: Fully customizable Robust management Secure by design Best of both worlds. Better outcomes. Lower total cost of ownership Enhance operational efficiency Facilitate innovation Meet customer expectations more readily Create new business models Benefits: Low entry cost Pay-per-use Highly elastic Hybrid Cloud Public Cloud
- 9. A Hybrid Cloud enables multi-speed IT Composable Environments to rapidly build and deploy new cloud-native and mobile solutions Flexibility to move apps to the cloud as-is or build cloud native solutions Leverage Existing Investments by connecting them to cloud services Cloud Native Solutions Cloud Enabled Solutions Multi-speed IT Partners Apps Access Info Process Interaction Core Enterprise Digital Ecosystem Core Enterprise Digital Ecosystem Traditional Projects: The capabilities to capitalize on your institutional knowledge New Projects: The speed and agility to drive innovation and growth Multi-Speed IT
- 10. Hybrid cloud model enables organizations to redefine customer service Customer-facing architecture: Optimized for speed and agility Transactional architecture: Optimized for availability and stability Dedicated Public On-premises Off-premises Local Private
- 11. Four primary drivers for Hybrid Cloud Integration Workload / Resource Optimization Portability Compliance
- 12. X % 2 70 Banking is projected to account for the largest proportion of cloud activity across any industry Market forecast 2019 $620 bn cloud market Banking —16% Telecom —13% CPG & Retail —13% Government —8% Travel & Transportation —8% Insurance —8% Industrial Products —7% Media & Entertainment —5% Financial Markets —5% Healthcare & Life Sciences —5% Energy and Utility —3% Electronics —3% Education —2% Automotive —2% Chemical & Petroleum —2% Aerospace & Defense —1% increase in the workload on cloud by top- tier banks of total IT spend on cloud solutions by 2020
- 13. Agenda – The North Star, Hybrid Cloud Big Data & Analytics
- 14. The Future of Hybrid Cloud for Big Data & Analytics (The North Star) • Extend the private environment to cloud • Exposing the private environment data to the cloud • Migrating/ modernizing some of the private environment applications to the cloud • Integrating some of the workloads & data across the private environment and public cloud • Provide a seamless & consistent experience for all workloads across all environments of hybrid cloud.
- 15. The North Star Data & Analytics Data Movement Data Preparation & Integration Data Sovereignty & Compliance Data Governance & Security Workload & Portability
- 16. Agenda – Implementation Considerations for Hybrid Cloud
- 17. How to implement a Hybrid Cloud Strategy? Cultural Shift Varying Levels of Hybrid Sophistication
- 18. Hybrid Cloud Scenarios On- Premises / Local Dedicated Public SOE SOAu Public Dedicated On-Premises / Local On-Premises / Local Dedicated Public On-Premises / Local Dedicated Public On-Premises / Local Data Dedicated Public On-Premises / Local Data Dedicated Data1 Public Data2 Next Gen Hybrid Workloads Hybrid Infrastructure Scale Out Hybrid Cloud Brokerage & ManagementIndependent Workloads SOE/SOAu – SOI/SOR Integration Portability & Optimization Capacity Access Backup & Archive Disaster & Recovery Choose on-premises / local, dedicated or public cloud based on independent data & workload requirements Systems of record on private environment and systems of engagement & systems of automation on public cloud, and system of insights across all environments Application and data are portable and can go from on-premises / local to dedicated or public cloud for improved optimization The use of public or dedicated cloud as additional resources for on- premises / local large private jobs / applications Leverage off-premises resources for backup & archive of on-premises / local resources Setup and make available a parallel on-premises / local environment using the public or dedicated cloud Planned or policy based management and sourcing across multiple environments of the hybrid cloud SOI SOI Public CloudDedicated Cloud On-Premises / Local Cloud Data App Data App Private Private SOR Private App Private Data App App App 1 App 1 Data1 App 2 On- Premises / Local Dedicated Public App Data Private Policy
- 19. Big Data & Analytics on Hybrid Cloud Today the most important considerations for Data & Analytics on the Hybrid Cloud strategy are: Data Gravity Localization Wise Data Topology
- 20. Big Data & Analytics on Hybrid Cloud The future of the hybrid cloud strategy for the best-in-class traditional organizations will be based on exposing/extending private environment to the cloud. On the other hand, the internet born organizations are using an 80/20 rule by moving 80% of their data to the cloud with only 20% retained on-premises under the following three categories of data: 1. Data that the organization wants to share publicly (public cloud) – ex(s): Software as a Service (SaaS) online office applications churn or fraud detection 2. Data that the organization wants to share across the enterprise (dedicated cloud) – ex(s): metadata catalog, master customer data, activity data hub, asset hub, statistical models hub, and content hub 3. Data that is highly sensitive data (on-premises) – ex(s): high-confidential customer data, revenue data, and intellectual property
- 21. Analytics Lifecycle on Hybrid Cloud
- 22. Analytics In-Motion Analytics Operating System Security Platform Information Management & Governance Actionable Insight Discovery & Exploration Ingestion & Integration Analytical Data Lake Data Access Data Sources Enhanced Applications Big Data & Analytics Components
- 23. Big Data & Analytics Capabilities Security Information Management & Governance Actionable Insight Analytics In-Motion Enhanced Applications Discovery & Exploration Analytics Operating System Data Sources Ingestion & Integration Data Access Machine & Sensor Data Image & Video Enterprise Content Social Data Weather Data Commercial Data Sets New sources Traditional sources Third-Party Data Transactional Data System of Record Data Dataacquisition&applicationaccess Internet Data Sets Application Data Batch Ingestion Change Data Capture Self-Service Data Virtualization Data Federation APIs Visualization & Storyboarding Reporting, Analysis & Content Analytics Decision Management Predictive Analytics & Modeling Cognitive Analytics Insight as a Service Customer Experience New Business Models Financial performance Risk Fraud & Operations IT Economics In-Memory Processing Simple Programming Paradigm Data Lifecycle Management Master & Entity Data Reference Data Data Catalog Data Models Data Quality Data EncryptionData Masking & Redaction Data Protection Security Intelligence Data Science Search Real-Time ingestion Document Interpretation & Classification Consistent Analytics Engine Data Quality Streaming Analytics Complex Event Processing Data Enrichment Analytical Data Lake Data Warehouses & Data Marts Landing Zone Data Archive Historical Data Deep Analytics Repository Exploratory Analytics Repositories Sand Boxes PlatformOn-Premise Cloud Hybrid
- 24. Data Topology on Hybrid Cloud Mixture: Raw and Modeled Legend RAW (Years) Detailed System of Records Data Modeled (Months)Data Integration (ELT/ELT) (Years) (Years) (Years) (Years) Detailed Provisioning Subject Area Data Delta Processing Detailed Integrated Subject Area Data Detailed Data Aggregates Summary Data Aggregates Dimensional Data Landing Area Self Provisioning Data Private Public / Private Private / On-Premises Data Discovery & ExplorationData Sandbox Areas Data Prediction & Statistical Modeling More Refined Data User Guided, Advanced Analytics & Visualization Business Analysts Data Scientists Analytical or Predictive Models Visualization, Data Mining & Exploration User Reports & Dashboards Landing Zone Provisioning Zone Self Service & Collaboration and Advanced Analytics Zone (Months) Shared Operational Information Data Warehouse & Data Marts Zone Sensors Social Media Customer Conversations External Sources Back Office Applications Devices Information Catalog / Master Data Management Public Internal Sources On-Premises Weather Data Sources Trusted,HighlyGoverned, HighlySecured Agile,LessGoverned, LessSecure Streaming Streaming Activity Hub / Asset Hub / Statistical Models Hub / Content Hub Shared Operational ZoneOn-Line Services Hybrid Cloud Data Lake Front Office Applications Internet Wearable's Sensors Streaming
- 25. Hybrid Cloud Reference Architecture – Big Data & Analytics
- 26. Data Movement on Hybrid Cloud One of the biggest challenges for ta hybrid cloud strategy Top 5 considerations to reduce latency and maintain performance Location, Location, Location Dedicated Connections Streaming Traffic Optimization Workload Optimization
- 27. Data Preparation & Integration on Hybrid Cloud Data prep isn’t just for data pros. The 3 phases of overcoming hybrid cloud data integration Exposing on-premises data to SaaS apps Hybrid Cloud Data Lake Real-time analytics with streaming data
- 28. 28 Security on Hybrid Cloud 3/29/17World of Watson 2016 Security has been and continues to be the #1 concern with the adoption of public cloud, and the same challenges are also present in a hybrid cloud scenario. Some key considerations for security on a hybrid cloud strategy includes: Mind The Gap Get ComplaintDeal with Data
- 29. Hybrid Cloud Performance Network Latency On-premises – Cloud Cloud Services to access on-premises VPN Secure Gateway
- 30. Agenda – Hybrid Cloud Big Data & Analytics Use Cases
- 31. SOE / SOAU – SOI / SOR Integration
- 34. 34 IBM Cloud Architecture Center World of Watson 2016 https://developer.ibm.com/architecture/ • Cloud solutions across eight architectures • Based on open standards from customer- proven enterprise grade architectures Learn. Discover. • Forty-one solutions across domains, cross- domain, emerging and industries • Prescriptive guides explain implementation best practices Try. • Experiment with nine sample solution implementations • Using single click deploy or a guided walkthrough Prove architecture works quickly Build what experts recommend Find your solution Built on IBM Cloud Solutions – Data and Analytics – Internet of Things – Cognitive
- 35. 35 3/29/17 Notices and disclaimers Copyright © 2017 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM. U.S. Government Users Restricted Rights — use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM. Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. This document is distributed “as is” without any warranty, either express or implied. In no event shall IBM be liable for any damage arising from the use of this information, including but not limited to, loss of data, business interruption, loss of profit or loss of opportunity. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided. IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty terms apply.” Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice. Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation. It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law.
- 36. 36 3/29/17 Notices and disclaimers continued Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM expressly disclaims all warranties, expressed or implied, including but not limited to, the implied warranties of merchantability and fitness for a particular, purpose. The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right. IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services®, Global Technology Services®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli® Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.
- 37. Thank You Dr. Christine Ouyang Distinguished Engineer and Master Inventor IBM – [email protected] Marcio Moura Executive Architect IBM – [email protected]