Efficient and Secure Data Management with Cloud Storage
Download as PPTX, PDF0 likes61 views
The document outlines cloud storage services offered by Amazon, including Amazon EBS, S3, EFS, and S3 Glacier, detailing their functionalities and use cases. It describes the features, pricing, security measures, and various storage classes available for data management and archiving. The document also emphasizes the differences between these storage options, providing insights into scalability, availability, and data retrieval methods.
![Amazon S3 bucket URLs (two styles)
Amazon S3
[bucket name]
Preview2.mp4
Tokyo Region
(ap-northeast-
1)
To upload your data:
1. Create a bucket in an AWS Region.
2. Upload almost any number of objects to the
bucket.
Bucket path-style URL endpoint:
https://s3.ap-northeast-1.amazonaws.com/bucket-name
Region code Bucket name
Bucket virtual hosted-style URL endpoint:
https:// bucket-name.s3-ap-northeast-1.amazonaws.com
Region code
Bucket name
62](https://image.slidesharecdn.com/cloudstorage-240706030452-241852b0/85/Efficient-and-Secure-Data-Management-with-Cloud-Storage-62-320.jpg)
Efficient and Secure Data Management with Cloud Storage
- 1. Cloud Storage CLO745NAA - Cloud Architecture Prepared By: Stalin Rijal
- 2. Overview • Topics • Amazon Elastic Block Store (Amazon EBS) • Amazon Simple Storage Service (Amazon S3) • Amazon Elastic File System (Amazon EFS) • Amazon Simple Storage Service Glacier
- 3. Objectives • Identify the different types of storage • Explain Amazon S3 • Identify the functionality in Amazon S3 • Explain Amazon EBS • Identify the functionality in Amazon EBS • Perform functions in Amazon EBS to build an Amazon EC2 storage solution • Explain Amazon EFS • Identify the functionality in Amazon EFS • Explain Amazon S3 Glacier • Identify the functionality in Amazon S3 Glacier • Differentiate between Amazon EBS, Amazon S3, Amazon EFS, and Amazon S3 Glacier 3
- 4. Outline • Introduction • Use Cases • S3 Objects • S3 security • Scalability • Availability • Versioning • Comparision • Ff • dd
- 5. Amazon Elastic Block Store (Amazon EBS)
- 6. Amazon EBS Amazon EBS enables you to create individual storage volumes and attach them to an Amazon EC2 instance: • Amazon EBS offers block-level storage. • Volumes are automatically replicated within its Availability Zone. • It can be backed up automatically to Amazon S3 through snapshots. • Uses include – • Boot volumes and storage for Amazon Elastic Compute Cloud (Amazon EC2) instances • Data storage with a file system • Database hosts • Enterprise applications 6
- 7. Amazon EBS volume types 7
- 8. Amazon EBS volume type use cases 8
- 9. Amazon EBS features • Snapshots – • Point-in-time snapshots • Recreate a new volume at any time • Encryption – • Encrypted Amazon EBS volumes • No additional cost • Elasticity – • Increase capacity • Change to different types 9
- 10. Amazon EBS: Volumes, IOPS, and pricing 1. Volumes – • Amazon EBS volumes persist independently from the instance. • All volume types are charged by the amount that is provisioned per month. 2. IOPS – • General Purpose SSD: • Charged by the amount that you provision in GB per month until storage is released. • Magnetic: • Charged by the number of requests to the volume. • Provisioned IOPS SSD: • Charged by the amount that you provision in IOPS (multiplied by the percentage of days that you provision for the month). 10
- 11. Amazon EBS: Snapshots and data transfer 3. Snapshots – • Added cost of Amazon EBS snapshots to Amazon S3 is per GB-month of data stored. 4. Data transfer – • Inbound data transfer is free. • Outbound data transfer across Regions incurs charges. 11
- 12. Amazon Simple Storage Service (Amazon S3)
- 13. Amazon S3 overview • Data is stored as objects in buckets • Virtually unlimited storage • Single object is limited to 5 TB • Designed for 11 9s of durability • Granular access to bucket and objects 13
- 14. Introduction • Amazon S3 allows people to store objects (files) in "buckets" (directories) • Buckets must have a globally unique name (across all regions all accounts) • Buckets are defined at the region level • S3 looks like a global service but buckets are created in a region • Naming convention o No uppercase, No underscore o 3-63 characters long o Not an IP o Must start with lowercase letter or number o Must NOT start with the prefix xn-- o Must NOT end with the suffix -s3alias 14
- 15. Amazon S3 - Objects • Objects (files) have a Key • The key is the FULL path: o s3://my-bucket/my_fiIe.txt o s3://my-bucket/my_foIderI/another_foIder/my_fiIe.txt • The key is composed of prefix + object name o s3://my-bucket/my_foIder/another_foIder/my_fiIe.txt • There's no concept of “directories” within buckets (although the UI will trick you to think otherwise) • Just keys with very long names that contain slashes ("/ ")
- 16. Amazon S3 - Objects (cont.) • Object values are the content of the body: oMax. Object Size is 5TB (5OOOGB) oIf uploading more than 5GB, must use “multi-part upload” • Metadata (list of text key / value pairs — system or user metadata) • Tags (Unicode key / value pair — up to 10) — useful for security / lifecycle • Version ID (if versioning is enabled)
- 17. Amazon S3 storage classes Amazon S3 offers a range of object-level storage classes that are designed for different use cases: • Amazon S3 Standard • Amazon S3 Intelligent-Tiering • Amazon S3 Standard-Infrequent Access (Amazon S3 Standard- IA) • Amazon S3 One Zone-Infrequent Access (Amazon S3 One Zone- IA) • Amazon S3 Glacier 17
- 18. Designed for seamless scaling media/welcome.mp4 prod2.mp4 prod3.mp4 prod4.mp4 prod5.mp4 prod6.mp4 prod7.mp4 prod8.mp4 prod9.mp4 prod10.mp4 prod11.mp4 prod12.mp4 my-bucket-name 18
- 19. Access the data anywhere AWS Management Console AWS Command Line Interface SDK 19
- 20. Amazon S3 common scenarios • Backup and storage • Application hosting • Media hosting • Software delivery Amazon S3 buckets Corporate data center Amazon EC2 instances 20
- 21. Amazon S3 Security • User-Based o IAM Policies — which API calls should be allowed for a specific user from IAM • Resource-Based o Bucket Policies — bucket wide rules from the S3 console - allows cross account o Object Access Control List (ACL) — finer grain (can be disabled) o Bucket Access Control List (ACL) — less common (can be disabled) • Note: an IAM principal can access an S3 object if o The user IAM permissions ALLOW it OR the resource policy ALLOWS it o AND there's no explicit DENY • Encryption: encrypt objects in Amazon S3 using encryption keys
- 22. S3 Bucket Policies • JSON based policies o Resources: buckets and objects o Effect: Allow / Deny o Actions: Set of API to Allow or Deny o Principal: The account or user to apply the policy to • Use S3 bucket for policy to: o Grant public access to the bucket o Force objects to be encrypted at upload o Grant access to another account (Cross Account)
- 23. Example: Public Access - Use Bucket Policy
- 24. Example: User Access to S3 IAM permissions
- 25. Example: EC2 instance access - Use IAM Roles
- 27. Bucket settings for Block Public Access • These settings were created to prevent company data leaks • If you know your bucket should never be public, leave these on • Can be set at the account level
- 28. Amazon - Versioning • You can version your files in Amazon S3 • It is enabled at the bucket level • Same key overwrite will change the “version": 1 2, 3. • It is best practice to version your buckets: o Protect against unintended deletes (ability to restore a version) o Easy roll back to previous version • Notes: o Any file that is not versioned prior to enabling versioning will have version “null” o Suspending versioning does not delete the previous versions
- 29. Amazon S3 pricing • Pay only for what you use, including – • GBs per month • Transfer OUT to other Regions • PUT, COPY, POST, LIST, and GET requests • You do not pay for – • Transfers IN to Amazon S3 • Transfers OUT from Amazon S3 to Amazon CloudFront or Amazon EC2 in the same Region 29
- 30. Amazon S3: Storage pricing To estimate Amazon S3 costs, consider the following: 1. Storage class type – • Standard storage is designed for: • 11 9s of durability • Four 9s of availability • S3 Standard-Infrequent Access (S-IA) is designed for: • 11 9s of durability • Three 9s of availability 2. Amount of storage – • The number and size of objects 30
- 31. Amazon S3: Storage pricing 3. Requests – • The number and type of requests (GET, PUT, COPY) • Type of requests: • Different rates for GET requests than other requests. 4. Data transfer – • Pricing is based on the amount of data that is transferred out of the Amazon S3 Region • Data transfer in is free, but you incur charges for data that is transferred out. 31
- 32. Amazon Elastic File System (Amazon EFS)
- 33. Amazon EFS features • File storage in the AWS Cloud • Works well for big data and analytics, media processing workflows, content management, web serving, and home directories • Petabyte-scale, low-latency file system • Shared storage • Elastic capacity • Supports Network File System (NFS) versions 4.0 and 4.1 (NFSv4) • Compatible with all Linux-based AMIs for Amazon EC2 33
- 34. Amazon EFS architecture VPC Network Interface Mount target Network Interface Mount target Network Interface Mount target Networ k Interfac e Networ k Interfac e Networ k Interfac e Private subnet Private subnet Private subnet Private subnet Elastic File System Availability Zone A Availability Zone B Availability Zone C 34
- 35. Amazon EFS implementation Create your Amazon EC2 resources and launch your Amazon EC2 instance. Create your Amazon EFS file system. Create your mount targets in the appropriate subnets. Connect your Amazon EC2 instances to the mount targets. Verify the resources and protection of your AWS account. 1 2 3 4 5 35
- 36. Amazon EFS resources File system • Mount target • Subnet ID • Security groups • One or more per file system • Create in a VPC subnet • One per Availability Zone • Must be in the same VPC • Tags • Key-value pairs 36
- 38. Amazon S3 Glacier review Amazon S3 Glacier is a data archiving service that is designed for security, durability, and an extremely low cost. • Amazon S3 Glacier is designed to provide 11 9s of durability for objects. • It supports the encryption of data in transit and at rest through Secure Sockets Layer (SSL) or Transport Layer Security (TLS). • The Vault Lock feature enforces compliance through a policy. • Extremely low-cost design works well for long-term archiving. • Provides three options for access to archives—expedited, standard, and bulk—retrieval times range from a few minutes to several hours. 38
- 39. Amazon S3 Glacier • Storage service for low-cost data archiving and long-term backup • You can configure lifecycle archiving of Amazon S3 content to Amazon S3 Glacier • Retrieval options – • Standard: 3–5 hours • Bulk: 5–12 hours • Expedited: 1–5 minutes Amazon S3 bucket Archive after 30 days Amazon S3 Glacier Archive Delete after 5 years 39
- 40. Amazon S3 Glacier use cases Media asset archiving Healthcare information archiving Regulatory and compliance archiving Scientific data archiving Digital preservation Magnetic tape replacement 40
- 41. Using Amazon S3 Glacier RESTful web services Java or .NET SDKs Amazon S3 with lifecycle policies 41
- 42. Lifecycle policies Amazon S3 lifecycle policies enable you to delete or move objects based on age. Amazon S3 Standard Preview2.mp4 30 days Amazon S3 Standard - Infrequent Access Preview2.mp4 60 days Amazon S3 Glacier Preview2.mp4 365 days Delete 42
- 43. Storage comparison Amazon S3 Amazon S3 Glacier Data Volume No limit No limit Average Latency ms minutes/hours Item Size 5 TB maximum 40 TB maximum Cost/GB per Month Higher cost Lower cost Billed Requests PUT, COPY, POST, LIST, and GET UPLOAD and retrieval Retrieval Pricing Per request Per request and per GB 43
- 44. Server-side encryption Corporate data center https Amazon S3 Glacier Amazon S3 AWS Cloud Amazon EC2 Your Applications on Amazon EC2 AWS Cloud Data is encrypted by default Your application must enable server-side encryption 44
- 45. Security with Amazon S3 Glacier Amazon S3 Glacier Control access with IAM Amazon S3 Glacier encrypts your data with AES-256 Amazon S3 Glacier manages your keys for you 45
- 49. Web Hosting • S3 can host static websites and have them accessible on the Internet • The website URL will be (depending on the region) • http://bucket-name.s3-website-region.amazonaws.com • http://bucket-name.s3-website.region.amazonaws.com • If you get a 403 Forbidden error, make sure the bucket policy allows public reads! 49
- 50. AWS, Azure and Google cloud features
- 51. AWS, Azure and Google Cloud Price Comparision
- 52. • Any Queries ?
- 53. • Thank You !
- 56. S3 storage class •Amazon S3 Standard – General Purpose •Amazon S3 Standard – Infrequent Access (IA) •Amazon S3 One Zone – Infrequent Access •Amazon S3 Glacier Instant Retrieval •Amazon S3 Glacier Flexible Retrieval • Amazon s3 Glacier Deep Archive •Amazon s3 Intelligent Tiering
- 57. S3 Durability and Availability • Durability: o High durability (99.999999999%) 11 9’s of objects across multiple AZ. o Same for all storage classes. o If we store 10,000,000 objects with Amazon S3, we can expect incur a loss of a single object every 10,000 years. • Availability: o Measure how readily available a service is o varies depending on storage class o example : s3 standard has 99.99% availability – n/a for 53 mins a year
- 58. S3 Standard – General Purpose • 99.99% Availability • Used for frequently accessed data • Low latency and high throughput • Sustain 2 concurrent facility failures
- 59. S3 Storage Class – Infrequent Access o For data that is less frequently accessed, but required rapid access when needed o Lower cost than S3 standard Amazon S3 standard- infrequent access (S3 standard-IA) o 99.9% Availability o Use cases : Disaster recovery, backups Amazon S3 One Zone-Infrequent Access(S3 One Zone-IA) o High durability 11 9’s in a single AZ, data lost when AZ is destroyed o 99.5% availability o Use case: Storing secondary backup copies of on-premises data, or data you can recreate
- 60. Amazon S3 Glacier Storage Class • Low-cost object storage meant for archiving/backup • Pricing: price for storage + object retrieval cost • Amazon S3 glacier instant retrieval • Millisecond retrieval, great for data accessed once a quarter • Minimum storage duration of 90 days • Amazon S3 Glacier Flexible Retrieval (formerly Amazon S3 Glacier): • Expedited ( I to 5 minutes), Standard (3 to 5 hours), BuII‹ (5 to 12 hours) — free • Minimum storage duration of 90 days • Amazon S3 Glacier Deep Archive — for long term storage: • Standard ( 12 hours), Bulk (48 hours) • Minimum storage duration of 180 days
- 61. S3 Intelligent-Tiering • Small monthly monitoring and auto-tiering fee • Moves objects automatically between Access Tiers based on usage • There are no retrieval charges in S3 Intelligent-Tiering • Frequent Access tier (automatic): default tier • Infrequent Access tier (automatic): objects not accessed for 30 days • Archive lnstont Access tier (automatic): objects not accessed for 90 days • Archive Access tier (optional): configurable from 90 days to 700+ days • Deep Archive Access tier (optional): config. from 180 days to 700+ days
- 62. Amazon S3 bucket URLs (two styles) Amazon S3 [bucket name] Preview2.mp4 Tokyo Region (ap-northeast- 1) To upload your data: 1. Create a bucket in an AWS Region. 2. Upload almost any number of objects to the bucket. Bucket path-style URL endpoint: https://s3.ap-northeast-1.amazonaws.com/bucket-name Region code Bucket name Bucket virtual hosted-style URL endpoint: https:// bucket-name.s3-ap-northeast-1.amazonaws.com Region code Bucket name 62
- 63. • Any Queries ? 63 © 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved.