AWS Fundamentals Basecamp
Larry Cusick, Solutions Architect, 2nd Watch
Lee Smith, Cloud Architect, AWS
Matt Curie, Solutions Architect, AlertLogic
Agenda 8:30am – 9:00am: Check-in
9:00am – 12:00pm: AWS Basecamp
Basecamp Introduction
Introduction to Cloud Computing
AWS Services and Regions Overview
AWS Networking
VPC - Route 53
AWS Security Overview
AWS Compute
EC2 – ELB - ASG
AWS Storage
EBS - S3 - Glacier
AWS Databases
RDS – DynamoDB - Redshift
12:00pm – 1:00pm: Break & Lunch
1:00pm – 3:00pm: Hands-On Lab - Building AWS Infrastructure
2
Introduction to 2nd Watch &
AlertLogic
ProjectBased
Consulting
ManagedCloud
Services(MCS)
Professional
Services
Managed
Capacity
(AWSreseller)
Managed
Services–
Opsupport
2nd Watch - What We Do
We help enterprises adopt and manage the public cloud
2nd Watch – An Automated Approach
Cloud Readiness Assessment
Governance & Security
Cloud-Native Architecture Design
Cloud Ready Applications
Migration Factory
Hyper Scale Applications
Expertise
Cloud Management Platform
DevOps
Breadth and depth with AWS
Original and current AWS
Premier Consulting Partner
Managed Service Partner
100% certified engineers
and architects
Competencies:
DevOps
Big Data
Marketing & Commerce
Life Sciences
SharePoint
Migration
Financial Services
Microsoft Workloads
FULLY-MANAGED SECURITY, DELIVERED AS A SERVICE
• Full-stack security
• Integrated analytics & experts
• Built for cloud
• Cost-effective outcomes
We protect cloud workloads & web applications
Data
Center
Hosting
ASSESS
BLOCK COMPLY
DETECT
Basecamp Introductions
Name:
Company:
Title/Work Area:
8
Introduction to
Cloud Computing
New IT Business Model
Cloud Computing
is first and foremost a
Business Model
10
Business Reasons for Adopting Cloud Computing
GoodNot Good
Go global in minutes
Remove undifferentiated
heavy lifting
Stop guessing capacity
Increased agility
Lower variable expense than they
could achieve on their own
Move from capital expense to
variable expense$
11
Defining Cloud Computing
NIST defined a well accepted, industry
standard definition of Cloud Computing
url: http://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-145.pdf
Covers:
 5 Key Characteristics of Cloud Computing
 3 Service Model
 4 Deployment Models
plus
 5 Cloud Actors
 A Cloud Reference Architecture
 Shared Security model
12
What is Cloud Computing?
NIST 5 Key Characteristics
#1 On-demand self service
“as easy as buying candy from a vending machine”
#2 Broad network access
“access it anytime from anywhere”
#3 Resource pooling
“you’re not the only user”
#4 Rapid elasticity
“scale up and scale down in real-time”
#5 Measured service
“pay only for what you consume”
13
AWS Today - 2017
Public cloud market share
Amazon Web Services: 45%
per Synergy Research Group
AWS generating > $12 billion a year.
14
“It is the overwhelming market share leader, with over 10 times more cloud IaaS
compute capacity in use than the aggregate total of the other 14 providers in this Magic
Quadrant” Gartner Report May, 2015.
AWS Services and
Regions Overview
AWS Services
16
Core Services
17
What are we learning today? Load
balanced Website
• AZ = physical data center
• VPC = virtual data center
• EC2 Instance = virtual machine, VMs
• Security Group = firewall
• AMI = gold image
18
19
AWS Regions – Global Infrastructure
20
Regions and Availability Zones
 Global Resources
» IAM Users
» Route 53 Records
 Regional Resources
» S3 Buckets
» VPCs
» ELB
 AZ Resources
» EBS Volumes
» EC2 Instances
» RDS Instances
» Subnets
21
AWS Networking
Overview
VPC - Virtual Private Cloud Overview
“Your Virtual Datacenter in
the Cloud”
Essential Components:
• Subnets
• Route Tables
• Network ACLs
• Security Groups
• Internet Gateways
• NAT Gateways
• Virtual Private Gateways
23
VPC - Subnets
24
VPC – Network and Subnets
• Network Topology
o Private address space
 Any range is valid, but we suggest a non-routable CIDR
 Public CIDR ranges are only reachable via a Virtual Private Gateway
 CIDR ranges can be as large as a /16 to as small as a /28
• Subnets
o Public subnets have a 0.0.0.0/0 route to the Internet Gateway (IGW)
 Instances that require a public IP need to reside in a public subnet
o Private subnets do not have an outbound route through the IGW
 NAT instances are commonly used as an outbound gateway for private instances
o Subnets cannot span AZ’s, but can share routing tables, which provides similar functionality.
25
VPC – Route Tables
• Route Tables
o Can be applied to multiple subnets
o Typical routing entries
 10.0.0.0/16 = Local
 0.0.0.0/0 = Internet Gateway
(Public Subnet)
 -or-
 0.0.0.0/0 = eni-12345678 (Private
Subnet)
26
VPC – Bringing it Together
27
VPC – On-Premises Connection
28
Route53 - Basic Feature Set
• Zone Creation
• Zone Import
o Import your zone file from a previous provider
o Delegate this zone to the AWS name servers
• Record Types
o A
o CNAME
o TXT,MX
o Alias
o S3 buckets and ELBs can be an alias target, allows zone apex magic
29
Route53 - Advanced Feature Set
• Weighted Resource Record Sets
• Health Checks
• Global Load Balancer
– Using weighted record sets, you can create a pool of
endpoints from which to balance traffic
– Enabling a health-check on this pool allows for a DNS based
load balancer which can be applied to any resource (AWS
or non-AWS)
• Latency Resource Record Sets
• Geolocation Resource Record Sets
30
Route53 – Global Failover
• Global Failover Pattern
• Uses R53 Health Checks
Route 53
Virginia Region
myapp.example.com
Ireland Region
31
AWS Security
Overview
AWS Foundation Services
Compute Storage Database Networking
AWS Global
Infrastructure Regions
Availability Zones
Edge
Locations
Client-side Data
Encryption
Server-side Data
Encryption
Network Traffic
Protection
Platform, Applications, Identity & Access Management
Operating System, Network & Firewall Configuration
Customer content
Customers
Customers are
responsible for
their security IN
the Cloud
AWS is responsible
for the security OF
the Cloud
Security and Compliance – Shared Security Model
Shared Responsibility Model is not Static
• Can shift or differ based on technology, business
purpose, architecture
Infrastructure Services Container Services Abstracted Services
Current vs AWS
• Network ACLs = access control lists
• Security group = firewall
• CloudTrail = audit tool
• IAM = Active Directory
Network ACLs
Behavior:
• Allow/Deny traffic at the subnet level
• Stateless
• Default is Allow All
Pro – further defines permissible traffic
Con – adds complexity and can
complicate troubleshooting
36
Security Group Behavior
• Work at instance level
• Stateful
• Deny by default
• IP Whitelisting (CIDR)
• Allow port and protocol
– Can allow TCP, UDP, ICMP or a
combination of all three
• SG Trust Relationships
– Scoped to the VPC
37
Client Ingress
SSH and RDP Ingress
Platform Server Ingress
Platform Server Egress
Database Access
SID App
APP
SID_App_Ingress
SID Database
SID View
VIEW
SID_Admin_Ingress
Protocol Port Range Source
TCP 22 10.99.101.0/24
TCP 3389 10.99.101.0/24
SID_App_Egress
Protocol Port Range Destination
TCP 443 10.87.14.29/32
TCP 3306 10.24.3.102/32
SID_DB_Ingress
Protocol Port Range Source
TCP 3306 SID_APP_SG
SID_View_Ingress
Protocol Port Range Source
TCP 80 10.12.0.0/16
TCP 80 10.45.2.0/24
TCP 80 10.62.31.0/24
View Firewall Policies
SID_View_Ingress
SID_Admin_Ingress
App Firewall Policies
SID_App_Ingress
SID_Admin_Ingress
SID_App_Egress
Database Firewall Policies
SID_DB_Ingress
Protocol Port Range Source
TCP 80 10.241.25.0/24
TCP 443 10.241.25.0/24
TCP 8080 SID_VIEW_SG
Security Group/Firewall Rules
38
CloudTrail
• Record of API requests and response elements
• who did what and when, from where
• Identity
• Time
• Source
• Parameters
• Response
Alert Logic®
Cloud Defender™
40
Alert Logic®
Cloud Defender
™
+ Web Security Manager Premier
41
AWS Compliance
Certificates Programs
Identity & Access
Management
IAM Terms
• User: an entity that you create in AWS to represent the
person or service that uses it to interact with AWS
• Groups: lets you assign permissions to a collection of
users
• Roles: delegate access to users or services that normally
don't have access to your organization's AWS resources
• Policy: Permissions granted to a users, groups and roles
• Instance Profile: a container for an IAM role that you can
use to pass role information to an EC2 instance when the
instance starts
IAM Users
o Identity and Access Management
o Create Users and Groups
o Establish Trust Relationships
o Govern Access via Policy Documents
45
Amazon
S3
Amazon
DynamoDB
Role: Allow Amazon S3
access but nothing else
Amazon EC2 Instance
IAM - Instance Profiles
46
IAM – AWS Master Account
AWS Account
 Master/Root Account Permissions
 Allow by default
 MFA
Always treat the master account
credentials as if they could launch an
ICBM!
47
AWS Directory
Service
Directory Service - Overview
Three types of directory services:
o Microsoft AD (Enterprise Edition)
o A managed Microsoft Active Directory service running on Windows Server 2012 R2
o Highly availability (multi-AZ), patched, and monitored
o Automatic daily backups (encrypted)
o Can support up to 50,000 users
o Fully functional MS AD
o Simple AD
o Powered by Samba 4 Active Directory
o Users and Groups can be created directly in the AWS console
o Windows servers can auto-join this domain as they would in an AD environment
o Can support 5,000 users
o AD Connector
o Connect your on-prem AD to your AWS account
o Associate AD users/groups with IAM users/groups
o Windows servers can auto-join this domain as they would in an AD environment
o Manage the AWS console using your AD credentials
49
AWS Compute
Overview
Current vs AWS
Now AWS
Server, VM Instance
Template, Image, Host Profile AMI, Instance Type
Hypervisor (usually ESXi, or Hyper-V) Hypervisor (Xen)
VM Snapshot Snapshot
EC2 – Elastic Cloud Compute
AMI
• Instances are based on an Amazon Machine
Image
• You can create new AMIs from a running
instance
• AMIs are unique to each region
52
EC2 - Instance Types
Choosing the correct instance type for the required workload
o T2 for light weight general purpose – but with burstable performance
o M4 for general purpose
o R3, X1 for memory and database heavy applications
o C3, C4 for compute heavy applications
o G2, P2 for GPU intensive applications
o I2, D2 for storage heavy applications (random)
o HS1 for storage heavy applications (sequential)
Example of M4 family of instance type
53
Model vCPU
Mem
(GiB)
Dedicated EBS
Bandwidth (Mbps)
m4.large 2 8 450
m4.xlarge 4 16 750
m4.2xlarge 8 32 1,000
m4.4xlarge 16 64 2,000
m4.10xlarge 40 160 4,000
m4.16xlarge 64 256 10,000
EC2 - Running Instances
Running instances
•Instances are launched into an existing VPC subnet
•CloudWatch monitoring is enabled by default
o CPU Utilization, Network I/O are the primary data points of interest
o Memory and Disk require an additional script that will post a to a custom
CloudWatch metric
•Status checks
o OS check
o Network reachability check
54
EC2 – Instance Recovery
• CloudWatch monitors instance
• Automatically recovers if it becomes impaired
– underlying hardware failure
– problem that requires AWS involvement to repair
• During instance recovery, the instance is migrated during
an instance reboot, and any data that is in-memory is lost
55
EC2 - Monitoring
56
EC2 - Bootstrapping
 User Data
• Provides a hook to inject scripting into any standard instance you decide
to launch
o These include the Amazon Linux, Windows and Ubuntu AMIs
o User Data can only be modified while the instance is stopped
• Suggested patterns
o Install security updates
 yum update -y
o Install middleware
 yum install -y httpd
 chkconfig httpd on
o Download and execute a remote script
 Assign an IAM Profile to the EC2 instance
 Aws s3 cp s3://mybucket/myscript.sh /tmp/myscript.sh
 ./tmp/myscript.sh
57
EC2 - Pricing
On Demand Instance
• This is the most common and flexible pricing option
• Pay only for what you use
• Stopped instances will not accrue hourly compute costs
• Pay by the instance hour
Reserved Instance (RI)
• 1 or 3 year commitment
• Pay for EC2 hourly at reduced rates (from On Demand rates)
• Payment Options
• No Upfront payment: no CapEx, lower hourly rate than On Demand
• Partial Upfront payment: some CapEx, lower hourly rate than No Upfront
• All Upfront payment: larger Capex, lowest hourly rate possible
58
EC2 - Pricing
Spot
• Useful for “worker pool”
scenarios
oTranscode, map reduce
task nodes
• Can be lost as soon as
someone is willing to pay
more for that instance
59
AWS Elastic
Load Balancing
On Premise AWS
Elastic Load Balancer
ELB - Elastic Load Balancer
 Public Side
• Consists of an endpoint which is the equivalent to a
traditional VIP
• Does not use a static IPv4, but rather an Alias/CNAME
• The endpoint will not always resolve to the same IP
 Private Side
• Minimum of one virtual ELB node per AZ
 Certificate Termination
• Only one SSL certificate per ELB
• Multi-Domain certificates are valid
62
ELB – Spans Multiple Availability Zones
63
 Auto Scaling Key Features
• Adds or removes servers based on load
• Self-healing pool of resources
• Every instance is based on a “gold” master image
Auto Scaling - Overview
64
 Auto scaling group
• Instance location
o Subnet
o Load Balancer
• Number of instances
o Min
o Max
o Desired
 Launch config
• Instance details
o Size
o PEM key
o IAM Profile
o Security Group(s)
o User data
Auto Scaling - Components
65
Auto Scaling - Multi-AZ
 Multi-AZ Auto Scaling
• Highly Available
• Production Standard
• Spans Datacenters
66
Auto Scaling - CloudWatch
CloudWatch is the final piece of the auto scaling puzzle. You can create alarms
based on instance metrics which trigger auto scaling actions.
Scaling policies
Scale up alarm
• Execute policy when: CPU is greater than 60%
• Take the action: Add 2 instances
• And then wait: 10 minutes
Scale down alarm
• Execute policy when: CPU is less than 20%
• Take the action: Remove 2 instances
• And then wait: 10 minutes
67
AWS Storage
Overview
Traditional Platform - Storage Architecture
In the old days…
• Hardware acquisition and datacenter space required
advanced planning
• Disk space and I/O allocation juggling for the entire
application lifecycle
• Volume and file redundancy not built-in
• Capital commitment and refresh budget
considerations
/root C:
/swap
Pagefile
Temp Dir
/app
/data
Program
Files
Data
Server Head
NAS or Fileserver
/DirShare 01/
File01
File02
/DirShare 02/
File01
Tape Library
ArchiveVol02
ArchiveVol 01
SMB /CIFS
Platform Monitoring Tools
69
AWS Instance Volumes and Data Storage
The new [improved] way of doing things…
• Elastic pay-as-you-go model
• Redundancy and snapshot utilities built-in
• New APIs and tools simplify application development,
administration and data lifecycle management
70
Storage Services
• EBS – Elastic Block Store (not actually a ”service”)
• S3 – Simple Storage Service (object storage)
• Standard
• Standard I/A – Infrequent Access
• Reduced Redundancy Storage (RRS) – 4 9’s of durability (1 facility)
• Glacier – Archival/Long-term
• Expedited – 1-5 minutes
• Standard – 3-5 hours
• Bulk – 5-12 hours
• EFS – Elastic File System
• AWS Storage Gateway
• Gateway-cached volumes – store primary data in AWS and cache most recently used data locally
• Gateway-stored volumes – store entire dataset onsite and asynchronously replicate data back to S3
• Gateway-virtual tape library – store your virtual tapes in either S3 or Glacier
71
Ephemeral Drives (EC2 Instance Store) Overview
Block device attached to the host machine
• Available to server instance
• May be mounted and used for temporary storage
• No additional usage charges for disk space or I/O
Not redundant: no built-in RAID or snapshot function
Data loss will result if any of the following occur:
• Host server or instance crash
• Instance termination
• Disk failure
/root C:
/swap
Pagefile
Temp Dir
/app
/data
Program
Files
Data
Server Virtual Head
72
EBS - Elastic Block Store
Block storage ideal for creating versatile OS volumes
• Define type, size and optionally I/O capacities [within service limits]
• Magnetic, SSD and Provisioned IOPS
• Mount to a single instance, similar to local drive
• Simplified Encryption options
• Volume sizes from 1GiB to 16TiB
Persistent and durable
• Redundant copies stored in single AZ
• Not permanently bound to a server instance and will survive server crash or shutdown
Snapshot capabilities for point-in-time backups
• Resizing and duplicating volumes
• Moving across AZs; Exporting across Regions
Performance metrics available through CloudWatch
73
S3 - Simple Storage Service
Object storage container with virtually unlimited capacity
• Store files (objects) in containers (buckets)
• Redundant copies for high durability and reliability
• Available on the internet via REST requests directly or through SDK
• Multiple strategies to secure contents
• Set permissions, access policies and optionally require MFA
• Encryption: Server (simplified) or Client-side
• Audit logging (optional) will record all access requests via APIs
• Built-in tools for managing versioning, object lifecycle and creating static websites
• Provides 99.999999999% durability (11 ‘9s’)
• Provides 99.99% availability
• Unlimited storage, but 5TB limit on file size
/mybucket01/
File01
File02
/mybucket02/
File01
Http / Https
Amazon S3
74
Amazon Glacier - Overview
Storage service optimized for reliable and low cost storage of archive data
• Data objects are securely archived, however not immediately accessible
• Create vaults (containers) to hold archives (any file based object)
• Upload archives programmatically
• Submit requests to retrieve archives. Available in about 4 hours
• Cost is approximately $.01/GB/Month plus modest API and retrieval charges [if
applicable]
• Single archive limited to 40TB, but no limit in total amount of storage
75
EFS – Elastic File System
Fully managed file server storage
• Uses NFS (v4.1) protocol
• Linux server only, Windows support planned for future release
• Can be mounted by 1,000s of EC2s
• Can be accessed from on-prem Data Center if using Direct Connect
• Highly available, redundant across multiple Azs
• Can grow from empty file system to multiple petabytes automatically
76
EFS – Comparing EFS and EBS
77
AWS Database
Overview
AWS Database Services
79
Workload Driven Data Store Selection
80
RDS - Relational Database Service
Fully managed relational database service with the following key advantages:
• Amazon manages resource redundancy, software patching, backups, failure
detection and recovery
• Ability to configure specific resources to cost-effectively scale your application
• Pay-as-you-go model offering included license or license portability [see fine print
to ensure license compliance]
• Streamlined management options to easily configure highly available topologies,
create database snapshots and deploy test instances
81
RDS - Relational Database Service
 6 Platforms
1. Oracle
2. MS SQL
3. MySQL
4. PostgreSQL
5. MariaDB
6. Amazon Aurora
 Fully managed
 Zero admin
82
AWS Aurora
Fully managed relational database engine that combines the speed and
availability of high-end commercial databases with the simplicity and cost
effectiveness of open source databases.
Key features:
o Architected for 99.99% availability
o Automatic failover < 30s (Possible Oracle RAC solution)
o Enterprise performance (5x) at 1/10 the cost
o Compatible with MySQL and PostgreSQL
o Automatically grows storage as needed, up to 64 TB
o Easy migration from MySQL
o Up to 15 Aurora Replicas in a region
o Cross-region replication
o Encryption in-transit and at rest
o Continuous backup to S3 (11 9’s data durability)
o Fully managed
83
DynamoDB
Fully managed NoSQL database service offering the following key advantages:
• Seamless and virtually unlimited scalability conveniently managed
automatically by Amazon
• Ability to define specific resource allocation limits to ensure predictable
performance while containing costs
• Easy administration and well-supported development model
• Integration with other core Amazon data services (for example Redshift and
EMR)
84
Redshift
Fully managed Enterprise-class data warehouse service offering the following
advantages:
• High performance, massively parallel columnar storage architecture providing
streamlined scalability
• Mainstream SQL query syntax allowing for rapid platform adoption
• Flexible node type and RI options allowing for workload alignment and cost
efficiency
• Massively scalable 160 GB -> 1.6 PB
85
Database Migration Service (DMS)
• AWS Database Migration Service helps you migrate databases to AWS
easily and securely.
• The source database remains fully operational during the migration,
minimizing downtime to applications that rely on the database.
• Homogenous (Oracle to Oracle) & heterogeneous migrations (ie Oracle
to Aurora, or Microsoft SQL Server to MySQL) using Schema Conversion
Tool
• Can also be used for continuous data replication with high-availability
86
87
General Information
1-888-317-7920
info@2ndwatch.com
www.2ndwatch.com
Contact Us
Larry Cusick
Solutions Architect
lcusick@2ndwatch.com
Locations
BOSTON
SEATTLE
NEW YORK
VIRGINIA
ATLANTA
PHILADELPHIA
DALLAS
LIBERTY LAKE
LOS ANGELES
CHICAGO
Marissa Bybee
Regional Territory Manager
mbybee@2ndwatch.com
Matt Curie
Solutions Engineering
mcurie@alertlogic.com
Thank You | Questions?

AWS Architecture Fundamentals - Houston

  • 1.
    AWS Fundamentals Basecamp LarryCusick, Solutions Architect, 2nd Watch Lee Smith, Cloud Architect, AWS Matt Curie, Solutions Architect, AlertLogic
  • 2.
    Agenda 8:30am –9:00am: Check-in 9:00am – 12:00pm: AWS Basecamp Basecamp Introduction Introduction to Cloud Computing AWS Services and Regions Overview AWS Networking VPC - Route 53 AWS Security Overview AWS Compute EC2 – ELB - ASG AWS Storage EBS - S3 - Glacier AWS Databases RDS – DynamoDB - Redshift 12:00pm – 1:00pm: Break & Lunch 1:00pm – 3:00pm: Hands-On Lab - Building AWS Infrastructure 2
  • 3.
    Introduction to 2ndWatch & AlertLogic
  • 4.
  • 5.
    2nd Watch –An Automated Approach Cloud Readiness Assessment Governance & Security Cloud-Native Architecture Design Cloud Ready Applications Migration Factory Hyper Scale Applications Expertise Cloud Management Platform DevOps
  • 6.
    Breadth and depthwith AWS Original and current AWS Premier Consulting Partner Managed Service Partner 100% certified engineers and architects Competencies: DevOps Big Data Marketing & Commerce Life Sciences SharePoint Migration Financial Services Microsoft Workloads
  • 7.
    FULLY-MANAGED SECURITY, DELIVEREDAS A SERVICE • Full-stack security • Integrated analytics & experts • Built for cloud • Cost-effective outcomes We protect cloud workloads & web applications Data Center Hosting ASSESS BLOCK COMPLY DETECT
  • 8.
  • 9.
  • 10.
    New IT BusinessModel Cloud Computing is first and foremost a Business Model 10
  • 11.
    Business Reasons forAdopting Cloud Computing GoodNot Good Go global in minutes Remove undifferentiated heavy lifting Stop guessing capacity Increased agility Lower variable expense than they could achieve on their own Move from capital expense to variable expense$ 11
  • 12.
    Defining Cloud Computing NISTdefined a well accepted, industry standard definition of Cloud Computing url: http://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-145.pdf Covers:  5 Key Characteristics of Cloud Computing  3 Service Model  4 Deployment Models plus  5 Cloud Actors  A Cloud Reference Architecture  Shared Security model 12
  • 13.
    What is CloudComputing? NIST 5 Key Characteristics #1 On-demand self service “as easy as buying candy from a vending machine” #2 Broad network access “access it anytime from anywhere” #3 Resource pooling “you’re not the only user” #4 Rapid elasticity “scale up and scale down in real-time” #5 Measured service “pay only for what you consume” 13
  • 14.
    AWS Today -2017 Public cloud market share Amazon Web Services: 45% per Synergy Research Group AWS generating > $12 billion a year. 14 “It is the overwhelming market share leader, with over 10 times more cloud IaaS compute capacity in use than the aggregate total of the other 14 providers in this Magic Quadrant” Gartner Report May, 2015.
  • 15.
  • 16.
  • 17.
  • 18.
    What are welearning today? Load balanced Website • AZ = physical data center • VPC = virtual data center • EC2 Instance = virtual machine, VMs • Security Group = firewall • AMI = gold image 18
  • 19.
  • 20.
    AWS Regions –Global Infrastructure 20
  • 21.
    Regions and AvailabilityZones  Global Resources » IAM Users » Route 53 Records  Regional Resources » S3 Buckets » VPCs » ELB  AZ Resources » EBS Volumes » EC2 Instances » RDS Instances » Subnets 21
  • 22.
  • 23.
    VPC - VirtualPrivate Cloud Overview “Your Virtual Datacenter in the Cloud” Essential Components: • Subnets • Route Tables • Network ACLs • Security Groups • Internet Gateways • NAT Gateways • Virtual Private Gateways 23
  • 24.
  • 25.
    VPC – Networkand Subnets • Network Topology o Private address space  Any range is valid, but we suggest a non-routable CIDR  Public CIDR ranges are only reachable via a Virtual Private Gateway  CIDR ranges can be as large as a /16 to as small as a /28 • Subnets o Public subnets have a 0.0.0.0/0 route to the Internet Gateway (IGW)  Instances that require a public IP need to reside in a public subnet o Private subnets do not have an outbound route through the IGW  NAT instances are commonly used as an outbound gateway for private instances o Subnets cannot span AZ’s, but can share routing tables, which provides similar functionality. 25
  • 26.
    VPC – RouteTables • Route Tables o Can be applied to multiple subnets o Typical routing entries  10.0.0.0/16 = Local  0.0.0.0/0 = Internet Gateway (Public Subnet)  -or-  0.0.0.0/0 = eni-12345678 (Private Subnet) 26
  • 27.
    VPC – Bringingit Together 27
  • 28.
    VPC – On-PremisesConnection 28
  • 29.
    Route53 - BasicFeature Set • Zone Creation • Zone Import o Import your zone file from a previous provider o Delegate this zone to the AWS name servers • Record Types o A o CNAME o TXT,MX o Alias o S3 buckets and ELBs can be an alias target, allows zone apex magic 29
  • 30.
    Route53 - AdvancedFeature Set • Weighted Resource Record Sets • Health Checks • Global Load Balancer – Using weighted record sets, you can create a pool of endpoints from which to balance traffic – Enabling a health-check on this pool allows for a DNS based load balancer which can be applied to any resource (AWS or non-AWS) • Latency Resource Record Sets • Geolocation Resource Record Sets 30
  • 31.
    Route53 – GlobalFailover • Global Failover Pattern • Uses R53 Health Checks Route 53 Virginia Region myapp.example.com Ireland Region 31
  • 32.
  • 33.
    AWS Foundation Services ComputeStorage Database Networking AWS Global Infrastructure Regions Availability Zones Edge Locations Client-side Data Encryption Server-side Data Encryption Network Traffic Protection Platform, Applications, Identity & Access Management Operating System, Network & Firewall Configuration Customer content Customers Customers are responsible for their security IN the Cloud AWS is responsible for the security OF the Cloud Security and Compliance – Shared Security Model
  • 34.
    Shared Responsibility Modelis not Static • Can shift or differ based on technology, business purpose, architecture Infrastructure Services Container Services Abstracted Services
  • 35.
    Current vs AWS •Network ACLs = access control lists • Security group = firewall • CloudTrail = audit tool • IAM = Active Directory
  • 36.
    Network ACLs Behavior: • Allow/Denytraffic at the subnet level • Stateless • Default is Allow All Pro – further defines permissible traffic Con – adds complexity and can complicate troubleshooting 36
  • 37.
    Security Group Behavior •Work at instance level • Stateful • Deny by default • IP Whitelisting (CIDR) • Allow port and protocol – Can allow TCP, UDP, ICMP or a combination of all three • SG Trust Relationships – Scoped to the VPC 37
  • 38.
    Client Ingress SSH andRDP Ingress Platform Server Ingress Platform Server Egress Database Access SID App APP SID_App_Ingress SID Database SID View VIEW SID_Admin_Ingress Protocol Port Range Source TCP 22 10.99.101.0/24 TCP 3389 10.99.101.0/24 SID_App_Egress Protocol Port Range Destination TCP 443 10.87.14.29/32 TCP 3306 10.24.3.102/32 SID_DB_Ingress Protocol Port Range Source TCP 3306 SID_APP_SG SID_View_Ingress Protocol Port Range Source TCP 80 10.12.0.0/16 TCP 80 10.45.2.0/24 TCP 80 10.62.31.0/24 View Firewall Policies SID_View_Ingress SID_Admin_Ingress App Firewall Policies SID_App_Ingress SID_Admin_Ingress SID_App_Egress Database Firewall Policies SID_DB_Ingress Protocol Port Range Source TCP 80 10.241.25.0/24 TCP 443 10.241.25.0/24 TCP 8080 SID_VIEW_SG Security Group/Firewall Rules 38
  • 39.
    CloudTrail • Record ofAPI requests and response elements • who did what and when, from where • Identity • Time • Source • Parameters • Response
  • 40.
  • 41.
    Alert Logic® Cloud Defender ™ +Web Security Manager Premier 41
  • 42.
  • 43.
  • 44.
    IAM Terms • User:an entity that you create in AWS to represent the person or service that uses it to interact with AWS • Groups: lets you assign permissions to a collection of users • Roles: delegate access to users or services that normally don't have access to your organization's AWS resources • Policy: Permissions granted to a users, groups and roles • Instance Profile: a container for an IAM role that you can use to pass role information to an EC2 instance when the instance starts
  • 45.
    IAM Users o Identityand Access Management o Create Users and Groups o Establish Trust Relationships o Govern Access via Policy Documents 45
  • 46.
    Amazon S3 Amazon DynamoDB Role: Allow AmazonS3 access but nothing else Amazon EC2 Instance IAM - Instance Profiles 46
  • 47.
    IAM – AWSMaster Account AWS Account  Master/Root Account Permissions  Allow by default  MFA Always treat the master account credentials as if they could launch an ICBM! 47
  • 48.
  • 49.
    Directory Service -Overview Three types of directory services: o Microsoft AD (Enterprise Edition) o A managed Microsoft Active Directory service running on Windows Server 2012 R2 o Highly availability (multi-AZ), patched, and monitored o Automatic daily backups (encrypted) o Can support up to 50,000 users o Fully functional MS AD o Simple AD o Powered by Samba 4 Active Directory o Users and Groups can be created directly in the AWS console o Windows servers can auto-join this domain as they would in an AD environment o Can support 5,000 users o AD Connector o Connect your on-prem AD to your AWS account o Associate AD users/groups with IAM users/groups o Windows servers can auto-join this domain as they would in an AD environment o Manage the AWS console using your AD credentials 49
  • 50.
  • 51.
    Current vs AWS NowAWS Server, VM Instance Template, Image, Host Profile AMI, Instance Type Hypervisor (usually ESXi, or Hyper-V) Hypervisor (Xen) VM Snapshot Snapshot
  • 52.
    EC2 – ElasticCloud Compute AMI • Instances are based on an Amazon Machine Image • You can create new AMIs from a running instance • AMIs are unique to each region 52
  • 53.
    EC2 - InstanceTypes Choosing the correct instance type for the required workload o T2 for light weight general purpose – but with burstable performance o M4 for general purpose o R3, X1 for memory and database heavy applications o C3, C4 for compute heavy applications o G2, P2 for GPU intensive applications o I2, D2 for storage heavy applications (random) o HS1 for storage heavy applications (sequential) Example of M4 family of instance type 53 Model vCPU Mem (GiB) Dedicated EBS Bandwidth (Mbps) m4.large 2 8 450 m4.xlarge 4 16 750 m4.2xlarge 8 32 1,000 m4.4xlarge 16 64 2,000 m4.10xlarge 40 160 4,000 m4.16xlarge 64 256 10,000
  • 54.
    EC2 - RunningInstances Running instances •Instances are launched into an existing VPC subnet •CloudWatch monitoring is enabled by default o CPU Utilization, Network I/O are the primary data points of interest o Memory and Disk require an additional script that will post a to a custom CloudWatch metric •Status checks o OS check o Network reachability check 54
  • 55.
    EC2 – InstanceRecovery • CloudWatch monitors instance • Automatically recovers if it becomes impaired – underlying hardware failure – problem that requires AWS involvement to repair • During instance recovery, the instance is migrated during an instance reboot, and any data that is in-memory is lost 55
  • 56.
  • 57.
    EC2 - Bootstrapping User Data • Provides a hook to inject scripting into any standard instance you decide to launch o These include the Amazon Linux, Windows and Ubuntu AMIs o User Data can only be modified while the instance is stopped • Suggested patterns o Install security updates  yum update -y o Install middleware  yum install -y httpd  chkconfig httpd on o Download and execute a remote script  Assign an IAM Profile to the EC2 instance  Aws s3 cp s3://mybucket/myscript.sh /tmp/myscript.sh  ./tmp/myscript.sh 57
  • 58.
    EC2 - Pricing OnDemand Instance • This is the most common and flexible pricing option • Pay only for what you use • Stopped instances will not accrue hourly compute costs • Pay by the instance hour Reserved Instance (RI) • 1 or 3 year commitment • Pay for EC2 hourly at reduced rates (from On Demand rates) • Payment Options • No Upfront payment: no CapEx, lower hourly rate than On Demand • Partial Upfront payment: some CapEx, lower hourly rate than No Upfront • All Upfront payment: larger Capex, lowest hourly rate possible 58
  • 59.
    EC2 - Pricing Spot •Useful for “worker pool” scenarios oTranscode, map reduce task nodes • Can be lost as soon as someone is willing to pay more for that instance 59
  • 60.
  • 61.
  • 62.
    ELB - ElasticLoad Balancer  Public Side • Consists of an endpoint which is the equivalent to a traditional VIP • Does not use a static IPv4, but rather an Alias/CNAME • The endpoint will not always resolve to the same IP  Private Side • Minimum of one virtual ELB node per AZ  Certificate Termination • Only one SSL certificate per ELB • Multi-Domain certificates are valid 62
  • 63.
    ELB – SpansMultiple Availability Zones 63
  • 64.
     Auto ScalingKey Features • Adds or removes servers based on load • Self-healing pool of resources • Every instance is based on a “gold” master image Auto Scaling - Overview 64
  • 65.
     Auto scalinggroup • Instance location o Subnet o Load Balancer • Number of instances o Min o Max o Desired  Launch config • Instance details o Size o PEM key o IAM Profile o Security Group(s) o User data Auto Scaling - Components 65
  • 66.
    Auto Scaling -Multi-AZ  Multi-AZ Auto Scaling • Highly Available • Production Standard • Spans Datacenters 66
  • 67.
    Auto Scaling -CloudWatch CloudWatch is the final piece of the auto scaling puzzle. You can create alarms based on instance metrics which trigger auto scaling actions. Scaling policies Scale up alarm • Execute policy when: CPU is greater than 60% • Take the action: Add 2 instances • And then wait: 10 minutes Scale down alarm • Execute policy when: CPU is less than 20% • Take the action: Remove 2 instances • And then wait: 10 minutes 67
  • 68.
  • 69.
    Traditional Platform -Storage Architecture In the old days… • Hardware acquisition and datacenter space required advanced planning • Disk space and I/O allocation juggling for the entire application lifecycle • Volume and file redundancy not built-in • Capital commitment and refresh budget considerations /root C: /swap Pagefile Temp Dir /app /data Program Files Data Server Head NAS or Fileserver /DirShare 01/ File01 File02 /DirShare 02/ File01 Tape Library ArchiveVol02 ArchiveVol 01 SMB /CIFS Platform Monitoring Tools 69
  • 70.
    AWS Instance Volumesand Data Storage The new [improved] way of doing things… • Elastic pay-as-you-go model • Redundancy and snapshot utilities built-in • New APIs and tools simplify application development, administration and data lifecycle management 70
  • 71.
    Storage Services • EBS– Elastic Block Store (not actually a ”service”) • S3 – Simple Storage Service (object storage) • Standard • Standard I/A – Infrequent Access • Reduced Redundancy Storage (RRS) – 4 9’s of durability (1 facility) • Glacier – Archival/Long-term • Expedited – 1-5 minutes • Standard – 3-5 hours • Bulk – 5-12 hours • EFS – Elastic File System • AWS Storage Gateway • Gateway-cached volumes – store primary data in AWS and cache most recently used data locally • Gateway-stored volumes – store entire dataset onsite and asynchronously replicate data back to S3 • Gateway-virtual tape library – store your virtual tapes in either S3 or Glacier 71
  • 72.
    Ephemeral Drives (EC2Instance Store) Overview Block device attached to the host machine • Available to server instance • May be mounted and used for temporary storage • No additional usage charges for disk space or I/O Not redundant: no built-in RAID or snapshot function Data loss will result if any of the following occur: • Host server or instance crash • Instance termination • Disk failure /root C: /swap Pagefile Temp Dir /app /data Program Files Data Server Virtual Head 72
  • 73.
    EBS - ElasticBlock Store Block storage ideal for creating versatile OS volumes • Define type, size and optionally I/O capacities [within service limits] • Magnetic, SSD and Provisioned IOPS • Mount to a single instance, similar to local drive • Simplified Encryption options • Volume sizes from 1GiB to 16TiB Persistent and durable • Redundant copies stored in single AZ • Not permanently bound to a server instance and will survive server crash or shutdown Snapshot capabilities for point-in-time backups • Resizing and duplicating volumes • Moving across AZs; Exporting across Regions Performance metrics available through CloudWatch 73
  • 74.
    S3 - SimpleStorage Service Object storage container with virtually unlimited capacity • Store files (objects) in containers (buckets) • Redundant copies for high durability and reliability • Available on the internet via REST requests directly or through SDK • Multiple strategies to secure contents • Set permissions, access policies and optionally require MFA • Encryption: Server (simplified) or Client-side • Audit logging (optional) will record all access requests via APIs • Built-in tools for managing versioning, object lifecycle and creating static websites • Provides 99.999999999% durability (11 ‘9s’) • Provides 99.99% availability • Unlimited storage, but 5TB limit on file size /mybucket01/ File01 File02 /mybucket02/ File01 Http / Https Amazon S3 74
  • 75.
    Amazon Glacier -Overview Storage service optimized for reliable and low cost storage of archive data • Data objects are securely archived, however not immediately accessible • Create vaults (containers) to hold archives (any file based object) • Upload archives programmatically • Submit requests to retrieve archives. Available in about 4 hours • Cost is approximately $.01/GB/Month plus modest API and retrieval charges [if applicable] • Single archive limited to 40TB, but no limit in total amount of storage 75
  • 76.
    EFS – ElasticFile System Fully managed file server storage • Uses NFS (v4.1) protocol • Linux server only, Windows support planned for future release • Can be mounted by 1,000s of EC2s • Can be accessed from on-prem Data Center if using Direct Connect • Highly available, redundant across multiple Azs • Can grow from empty file system to multiple petabytes automatically 76
  • 77.
    EFS – ComparingEFS and EBS 77
  • 78.
  • 79.
  • 80.
    Workload Driven DataStore Selection 80
  • 81.
    RDS - RelationalDatabase Service Fully managed relational database service with the following key advantages: • Amazon manages resource redundancy, software patching, backups, failure detection and recovery • Ability to configure specific resources to cost-effectively scale your application • Pay-as-you-go model offering included license or license portability [see fine print to ensure license compliance] • Streamlined management options to easily configure highly available topologies, create database snapshots and deploy test instances 81
  • 82.
    RDS - RelationalDatabase Service  6 Platforms 1. Oracle 2. MS SQL 3. MySQL 4. PostgreSQL 5. MariaDB 6. Amazon Aurora  Fully managed  Zero admin 82
  • 83.
    AWS Aurora Fully managedrelational database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost effectiveness of open source databases. Key features: o Architected for 99.99% availability o Automatic failover < 30s (Possible Oracle RAC solution) o Enterprise performance (5x) at 1/10 the cost o Compatible with MySQL and PostgreSQL o Automatically grows storage as needed, up to 64 TB o Easy migration from MySQL o Up to 15 Aurora Replicas in a region o Cross-region replication o Encryption in-transit and at rest o Continuous backup to S3 (11 9’s data durability) o Fully managed 83
  • 84.
    DynamoDB Fully managed NoSQLdatabase service offering the following key advantages: • Seamless and virtually unlimited scalability conveniently managed automatically by Amazon • Ability to define specific resource allocation limits to ensure predictable performance while containing costs • Easy administration and well-supported development model • Integration with other core Amazon data services (for example Redshift and EMR) 84
  • 85.
    Redshift Fully managed Enterprise-classdata warehouse service offering the following advantages: • High performance, massively parallel columnar storage architecture providing streamlined scalability • Mainstream SQL query syntax allowing for rapid platform adoption • Flexible node type and RI options allowing for workload alignment and cost efficiency • Massively scalable 160 GB -> 1.6 PB 85
  • 86.
    Database Migration Service(DMS) • AWS Database Migration Service helps you migrate databases to AWS easily and securely. • The source database remains fully operational during the migration, minimizing downtime to applications that rely on the database. • Homogenous (Oracle to Oracle) & heterogeneous migrations (ie Oracle to Aurora, or Microsoft SQL Server to MySQL) using Schema Conversion Tool • Can also be used for continuous data replication with high-availability 86
  • 87.
    87 General Information 1-888-317-7920 [email protected] www.2ndwatch.com Contact Us LarryCusick Solutions Architect [email protected] Locations BOSTON SEATTLE NEW YORK VIRGINIA ATLANTA PHILADELPHIA DALLAS LIBERTY LAKE LOS ANGELES CHICAGO Marissa Bybee Regional Territory Manager [email protected] Matt Curie Solutions Engineering [email protected]
  • 88.
    Thank You |Questions?