Storage solution in the cloud
3 likes•1,667 views
The document outlines an agenda for a presentation on Amazon Web Services (AWS) storage solutions, including various services like Amazon S3, Glacier, and EBS, along with their use cases. It discusses backup, archiving, and disaster recovery strategies tailored for different data storage needs. The presentation emphasizes AWS's cost efficiency and scalability in managing large volumes of data across different industries.
Storage solution in the cloud
- 1. Storage Solution in the Cloud Martin Yan ([email protected])
- 2. Agenda v Intro of Amazon Web Services v AWS Storage op7ons v Demo #1 – Commvault backup to S3 + Glacier, Storage Lifecycle management q Break v Demo #2 – Storage Gateway v Demo #3 – Disaster Recover
- 3. 7 Years Young Amazon S3 launched: March 14th 2006
- 4. Broad & Deep Services to Support Virtually Any Cloud Workload Compute Networking Storage & CDN Database App Services Management Amazon EC2 Amazon EMR Amazon ELB Amazon Route 53 Amazon VPC AWS Direct Connect Amazon S3 Amazon Glacier Amazon EBS AWS Import/Exp Amazon RDS Amazon DynamoDB Amazon Elasticache Amazon RedShift Amazon CloudSearch Amazon SWF Amazon SQS Amazon SNS Amazon SES Amazon Elastic Transcoder AWS IAM Amazon CloudWatch AWS Elastic Beanstalk AWS Cloudformation AWS Data Pipeline AWS OpsWorks AWS CloudHSM AWS Trusted Advisor AWS Marketplace AWS Premium Support AWS Training
- 5. AWS Global Infrastructure 9 regions 25 availability zones 40 edge locations
- 6. $5.2B retail business 7,800 employees A whole lot of servers 2003
- 7. 2012 Every day, AWS adds enough server capacity to power this $5B enterprise $5.2B retail business 7,800 employees A whole lot of servers 2003
- 8. Hundreds of Thousands of Customers in 190 Countries
- 9. Free steak campaign Facebook page Mars exploration ops Consumer social app Ticket pricing optimization SAP & Sharepoint Securities Trading Data Archiving Gene sequencing Marketing web site Interactiv e TV apps Financial markets analytics R&D data analysis Consumer social app Big data analytics Web site & media sharing Disaster recovery Media streaming Web and mobile apps Streaming webcasts Facebook app Consumer social app Every Imaginable Use Case
- 10. Customers in Greater China
- 11. • Multi-Factor Authentication • Identity & Access Management • Security groups for EC2 and VPC Network ACL Comprehensive Security Capabilities to Support Virtually Any Workload • VPC • Direct connect • Dedicated instances • S3 Encryption • CloudHSM • RDS Oracle transparent encryption
- 12. Certifications & Accreditations for Workloads that Matter
- 13. 32 Price Reduction s Since 2006 The AWS Price Reduction Philosophy Ecosystem Global Footprint New Features New Services Infrastructur e Innovation More AWS Usage More Infrastructure Economies of Scale Lower Infrastructu re Costs Reduced Prices More Customer s
- 14. ® Object Storage ® Block Storage ® Sync Volumes ® Relational Databases ® NoSQL Databases AWS Storage Options
- 15. Storage Services Amazon EBS EBS Block storage for use with Amazon EC2 Amazon S3 Images Videos Files Binaries Snapshots Internet scale storage via API AWS Storage Gateway S3, Glacier Integrates on-premises IT and AWS storage Amazon Glacier Images Videos Files Binaries Snapshots Storage for archiving and backup
- 16. ® Object Storage ** ® Block Storage ** ® Sync Volumes ** ® Relational Databases ® NoSQL Databases AWS Storage Options
- 17. We are constantly producing more data
- 19. Amazon S3 Simple Storage Service
- 20. 99.999999999% Durability Over 1Trillion Unique Customer Objects 750 000+ Peak Transactions Per Second Unlimited Objects
- 21. Storage Tiers: Buckets + Unlimited Objects
- 22. 2GB Free $9.99/month 100GB
- 23. Oracle Secure Backup Module Oracle RMAN Ú Amazon S3
- 24. Use Case: Oracle Backup to S3 Oracle Secure Backup (OSB) Cloud Module allows customers to backup Oracle Databases directly to Amazon S3 using the Oracle Recovery Manager (RMAN)
- 25. Restore 7mes reduced from 15 to 2½ hours Savings of over $1M per year No more capacity planning for BD Backup
- 26. ® Unlimited storage capacity ® High durability ® Storage for backups ® Single origin store with delivery via CloudFront Use Amazon S3 When You Need ® Content Storage and Distribu7on ® Storage for Data Analysis ® Backup and Disaster Recovery ® Sta7c Website Hos7ng Common use cases
- 27. Actual cost of on-premise storage • Tradi7onal Providers Quote and Sell “Raw”, Unprotected Gigabytes of Capacity • Protec7on Methods (RAID, Mirror and Erasure Codes) are addi7onal • File and Opera7ng System space is addi7onal • U7liza7on Factor (Growth Buffers) is addi7onal • RAW Price + 25% Protec7on + 7% Overhead + 15% U7liza7on • Usable Per Gigabyte Price = RAW + almost 50% • If you have a Disaster Recovery Site – Your Numbers Double
- 28. Inputs to calculate storage total cost – 5 Data Points 1. RAW Storage Selling Price, Capacity, Data Protec7on Method (Up7me SLA) 2. Data Center or Coloca7on Hos7ng Expense 3. Network (Core, Distribu7on, Load Balancers, and Bandwidth) 4. Staff Labor (Architecture, Opera7ng, Sustainment, and Management) 5. Cost of Funds / Cost of Capital
- 29. Amazon Glacier Low-‐Cost Archiving Service
- 30. 1¢ Per GB / Month $120 Per TB / Year 99.999999999% Durability 3−5 Hours Data Retrieval Storage Costs vs. Retrieval Costs
- 31. ® Inexpensive/Long-term archiving ® Unlimited storage capacity ® Eliminate tape museums ® Eliminate tech refresh ® High durability Use Amazon Glacier When You Need ® Offsite Enterprise Information Archiving ® Archiving Media Assets ® Archiving Research and Scientific Data ® Digital Preservation ® Magnetic Tape Replacement ® HSM – Hierarchical Storage Management Common use cases
- 32. S3 Glacier Policy-‐Based Archiving Service
- 33. HSM for Enterprises – Hierarchical Storage Management Amazon Glacier Amazon S3 Corporate Data Center AWS Cloud Corporate Data Center HSM with AWS versus TradiUonal Approach To HSM SAN SAN Disk Backup Offsite Tape Storage Tier 2 Storage
- 34. Policy-‐Based Archiving Service -‐ Lifecycle Rule
- 36. Amazon EBS ElasUc Block Storage
- 41. 1 GB 1TB Ú
- 42. IOPS Provisioned
- 43. Workloads on EBS Provisioned IOPS " Rela7onal Databases " NoSQL Databases, e.g. MongoDB " High Performance File Systems " Produc7vity applica7ons, e.g. Microsoa Exchange " Enterprise Applica7ons – CRM, ERP, etc. Designed for running transacUonal applicaUons that require high and consistent IO
- 44. Take snapshots of EBS volumes and keep them in S3
- 45. ® Filesystem for an instance NTFS, ExtFS, RAID, LVM… ® Long-term persistent storage ® Data changes frequently ® Access to raw, unformatted block-level storage Use Amazon EBS When You Need
- 46. Demo #1 Commvault backup to S3 Glacier, Storage Lifecycle management
- 47. Let’s start by defining what we mean • A backup or the process of backing up is making copies of data which may be used to restore the original aaer a data loss event. The primary purpose is to recover data aaer its loss, be it by data dele7on or corrup7on. The secondary purpose of backups is to recover data from an earlier 7me. • Archiving is the process of moving data that is no longer ac7vely used to a separate data storage device for long-‐term reten7on. Data archives are indexed and have search capabili7es so that files and parts of files can be easily located and retrieved. • Disaster recovery (DR) is the process, policies and procedures related to preparing for recovery or con7nua7on of technology infrastructure cri7cal to an organiza7on aaer a natural or human-‐induced disaster.
- 49. AWS Storage Gateway
- 50. Storage Gateway – Connect On-‐Prem with the AWS Cloud 1. Local, low-‐latency access to the most frequently used files while storing all data in Amazon S3 (Cached-‐Volumes) Or 2. Scheduled off-‐site backups to Amazon S3 for on-‐premises data (Stored-‐Volumes)
- 51. ® Backup your data ® Synchronize data ® Export data for migration ® Thin provision your SAN ® Departmental fileshare Use AWS Storage Gateway When You Need
- 52. VM Import/Export Easily import VM images to EC2 instances and export them back to your environment
- 53. Import your VM
- 54. Recreate your environment in AWS
- 55. Point your app users to your new DR site
- 56. S3 Website: Static Content
- 57. Common Data Storage Challenges and how AWS helps Primary Block Storage Primary File Storage Archival Storage Disk Based Backup Storage Tape Infrastructure & Management Replicated Storage for Disaster Recovery Offsite Locations Geo- Resilience Next Generation: Gateway & Cloud Storage Reduce SAN Footprint Reduce NAS Footprint Eliminate Architecture Hardware & Software Eliminate Backup Hardware & Software Eliminate Tape Recover in Cloud or any Data Center Eliminate Secondary Data Centers Reduce Complexity. Reduce Cost. Next Generation Enterprise Storage - Benefits
- 58. Here are some ways to reduce costs with AWS • Eliminate physical second site • Less administra7on • Pay only for the infrastructure you use • Free Inbound Data Transfer • Inexpensive DR simula7ons • Easy geographic separa7on of data centers
- 59. What our cost and performance advantages look like Infrastructure Cost Time test test Failover 2nd Site Cost AWS Cost Demand Cost savings Ability to scale – no arbitrary time limit to failback
- 60. Let’s compare the cost of provisioning and opera7ng a DR site over three years $0 $500,000 $1,000,000 $1,500,000 $2,000,000 Second site DR in AWS Cloud Failover * 1 mon. OpEx CapEx Hypothe7cal mid size example with hot standby in remote region. Your results will vary.
- 61. Here’s how to drive home the advantage ProducUon: Routers Firewalls IP Network Applica7on Licenses Opera7ng Systems Hypervisor Servers Storage Network Primary Storage Backup SW Backup Tapes Tape Silos Archive SW Archive Storage DR Site (data center): Routers Firewalls IP Network Applica7on Licenses Opera7ng Systems Hypervisor Servers Storage Network Primary Storage Backup SW Backup Tapes Tape Silos Archive SW Archive Storage DR Site (AWS): Routers Firewalls IP Network Applica7on Licenses Opera7ng Systems Hypervisor Servers Storage Network Snapshot Storage Backup SW Backup Tapes Tape Silos Archive SW Archive Storage
- 62. Demo #3 Disaster Recovery on AWS