How Big Data Can Enable Analytics from the Cloud (Technical Workshop)
0 likes664 views
The document discusses the advantages of using Cloudera Director for deploying Hadoop in the cloud, emphasizing its ability to minimize on-premise infrastructure, reduce costs, and enhance operational agility. It highlights features such as multi-cloud support, simplified cluster management, and accelerated deployment times, making it easier for organizations to adopt cloud-native analytics. Ultimately, Cloudera Director aims to provide a secure, reliable, and user-friendly solution for managing big data analytics workloads in cloud environments.
How Big Data Can Enable Analytics from the Cloud (Technical Workshop)
- 1. 1© Cloudera, Inc. All rights reserved. How can Big Data enable Analytics from the Cloud Joel Roland, System Engineer, Cloudera
- 2. 2© Cloudera, Inc. All rights reserved. What’s Driving Hadoop to the Cloud? Enterprise customers using cloud for big data analytics Hadoop deployments in cloud are accelerating: ● Executive mandate: minimize on-prem datacenter footprint ● Perceived lower overall TCO ● Increased agility: end-user self-service ● Elasticity: optimize infrastructure usage
- 3. 3© Cloudera, Inc. All rights reserved. Workloads in the cloud Only pay for what you need, when you need it ▪ Transient clusters ▪ Elastic workload ▪ Object storage centric ▪ Cloud-native deployment ETL/Modeling (Data Engineering) App Delivery (Operational Database) Reduce Operating Costs New Insights, New Revenue Run Without Risk BI/Analytics (Analytic Database) Explore and analyze all data, wherever it lives ▪ Transient or Persistent clusters ▪ Sized to demand ▪ HDFS or object storage ▪ Lift-and-shift or cloud-native deployment Enterprise-grade to protect your business, no matter what ▪ Fixed clusters ▪ Periodic sync ▪ All HDFS storage ▪ Lift-and-shift deployment
- 4. 4© Cloudera, Inc. All rights reserved. Embrace Transience for Lower Costs Decoupled Storage and Compute for Elastic Scale Patterns of Cloud-Native Applications Flexibility, Self-Service Models, and New Cost Dynamics Compartmentalize for Greater Isolation Object Store COMPUTE 1hr SPIN UP SPIN DOWN Object Store
- 5. 5© Cloudera, Inc. All rights reserved. Sample CDH in Cloud Architecture Data Sources Real-Time Serving Kafka/ Flume Spark Streaming HBase or Impala/Kudu (beta) Kafka Application S3 Hive/Spark/HoS Impala Analytics Batch Data Transformations Streaming Architecture
- 6. 6© Cloudera, Inc. All rights reserved. Sample CDH in Cloud Architecture Data Sources Real-Time Serving Kafka/ Flume Spark Streaming HBase, or Impala/Kudu (beta) Kafka Application S3 Hive/Spark/HoS Impala Analytics Batch Data Transformations Batch Analytics
- 7. 7© Cloudera, Inc. All rights reserved. Cloud Enabled by Cloudera Director OPERATIONS Cloudera Manager Cloudera Director • Cloudera Director is an integrated part of Cloudera Enterprise; designed to enable organisations to deploy Enterprise Grade Hadoop into the Cloud by making it • Fast, Easy, Secure and Reliable • Cloudera Director complements and extends the existing capabilities of tools such as Cloudera Manager • While enabling organisations to consume new usages patterns as well as reduce time-to-value when deploying Cloudera
- 8. 8© Cloudera, Inc. All rights reserved. Cloudera Director Benefits & Capabilities Eliminate Vendor Lock-in • Native support for multiple cloud-providers (AWS, Google Cloud Compute & Microsoft Azure) • Extend / Enable Hybrid cloud deployments using plugins for VMWare • You control what workloads run on which providers avoiding single vendors lock in.
- 9. 9© Cloudera, Inc. All rights reserved. Cloudera Director Benefits & Capabilities Simplify Cluster Lifecycle & Management • Simple and Easy to use UI to manage, spin up, scale, and spin down cluster. • Can fully automate Cloud Deployments at the click of a button • Dynamic scaling for clusters • Ability to clone clusters on-demand • Allows you to define blueprints for repeatable cloud deployments
- 10. 10© Cloudera, Inc. All rights reserved. Cloudera Director Benefits & Capabilities Accelerating Time-to-Value with Enterprise-ready security and administration • Deploy clusters in as little as 15-30 minutes! • Support for complex cluster topologies (eg HA) • Deployed with compliance-ready security and governance (eg Kerebos) • Clusters Easily connect into Cloudera’s BDR (Backup & Disaster Recovery Solution) • Has contains an extensible Restful API for those who want to script and automate cluster builds
- 11. 11© Cloudera, Inc. All rights reserved. In Summary Cloudera Director makes your journey to the cloud: • Fast • Deploy clusters across multiple cloud providers in minutes vs days • Easy • As Cloudera Director is an integrated part of the Platform, it makes creating, modifying and managing clusters in the cloud simple! • Secure • Deploy secure clusters out of the box • Reliable • Reduce risk or errors by using an automated deployment model
- 12. 12© Cloudera, Inc. All rights reserved. Demo • Overview Web Interface • Create New Cluster (via Console) • Modify Existing Cluster (resize) • Query Data on S3 (Customer Example) • Execute ETL Workflow & Save data to S3
- 13. 13© Cloudera, Inc. All rights reserved. Crunching 1,000+ Business Metrics per Customer with Sub-Second Responses •Enables granular targeting of customers •50% reduction in marketing cost execution at one •Stores & processes 1000s of critical events at scale & low cost •Provides flexibility, agility to support customer needs with Cloudera on Amazon Web Services and on premises CUSTOMER 360 Customer 360° in the Cloud
- 14. 14© Cloudera, Inc. All rights reserved. Thanks! Questions?