Making Sense Out of Amazon EC2 Container Service
2 likes683 views
The document provides an overview of Amazon ECS and its benefits for container orchestration in DevOps practices. It discusses the challenges faced with a monolithic application, the transition to microservices architecture using Docker, and the advantages of using ECS, such as ease of integration, scalability, and automated deployment. Additionally, it outlines the key components of ECS, including container instances, clusters, and task definitions, alongside a typical user workflow for implementing ECS in application management.
![WhiteHedgeKEY COMPONENTS: TASK DEFINITIONS
{
"environment": [],
"name": "simple-demo",
"image": "my-
demo", "cpu": 10,
"memory": 500,
"portMappings": [
{
"containerPort": 80,
"hostPort": 80
}
],
"mountPoints": [
{
"sourceVolume": "my-vol",
"containerPath":
"/var/www/my-
vol"
}
],
"entryPoint": [
"/usr/sbin/apache2"
, "-D",
"FOREGROUND"
],
"essential": true
},
{
"name": "busybox",
"image":
"busybox",
"cpu": 10,
"memory":
500,
"volumesFrom"
: [
{
"sourceContainer": "simple-demo"
}
],
"entryPoint"
: [
"sh",
"-c"
],
"command": [
"/bin/sh -c "while true; do
/bin/date > /var/www/my-vol/date; sleep 1;
done""
],
"essential": false
}](https://image.slidesharecdn.com/vduiwsnusocqai5arige-signature-c8578a1bc9e6c7cbf773c63a56577a9b940da07368a0112ea5b6ee15ef0f5c0f-poli-160215093206/85/Making-Sense-Out-of-Amazon-EC2-Container-Service-27-320.jpg)
![WhiteHedge
{
"environment": [],
"name": "simple-demo",
"image": "amazon/amazon-ecs-
sample", "cpu": 10,
"memory": 500,
"portMappings":
[
{
"containerPort": 80,
"hostPort": 80
}
],
"mountPoints": [
{
"sourceVolume": "my-vol",
"containerPath":
"/var/www/my-
vol"
}
],
"entryPoint": [
"/usr/sbin/apache2
", "-D",
"FOREGROUND"
],
"essential": true
},
KEY COMPONENTS: TASK DEFINITIONS
Essential to our
task
10 CPU units (1024 is full
CPU), 500 megabytes of
memory
Expose port 80 in container
to port 80 on host
Create and mount volumes](https://image.slidesharecdn.com/vduiwsnusocqai5arige-signature-c8578a1bc9e6c7cbf773c63a56577a9b940da07368a0112ea5b6ee15ef0f5c0f-poli-160215093206/85/Making-Sense-Out-of-Amazon-EC2-Container-Service-28-320.jpg)
![WhiteHedge
{
"name": "busybox",
"image":
"busybox",
"cpu": 10,
"memory":
500,
"volumesFrom"
: [
{
"sourceContainer": "simple-demo"
}
],
"entryPoint"
: [
"sh",
"-c"
],
"command": [
"/bin/sh -c "while true; do
/bin/date > /var/www/my-vol/date; sleep 1;
done""
],
"essential": false
}
KEY COMPONENTS: TASK DEFINITIONS
[
{
"image": "tutum/wordpress-stackable",
"name": "wordpress",
"cpu": 10,
"memory": 500,
"essential": true,
"links": [
"db"
],
"entryPoint": [
"/bin/sh",
"-c"
],
"environment": [
…
],
"portMappings": [
{
"containerPort": 80,
"hostPort": 80
}
]
},
]
From Docker Hub
Mount volume from other
container
Command to
exec](https://image.slidesharecdn.com/vduiwsnusocqai5arige-signature-c8578a1bc9e6c7cbf773c63a56577a9b940da07368a0112ea5b6ee15ef0f5c0f-poli-160215093206/85/Making-Sense-Out-of-Amazon-EC2-Container-Service-29-320.jpg)
Making Sense Out of Amazon EC2 Container Service
- 1. MAKING SENSE OUT OF AMAZON ECS Swapnil Dahiphale DevOps Engineer 13 Feb 2016, DevOps Meetup Banglore
- 2. WhiteHedge 2 INTRODUCTION - HELLO Hello! Welcome to WhiteHedge WhiteHedge is an Agile Software Product Development company. But what really describes us is what we think about Our Work , Our Clients. We have deep experience in some of the most sought-after technology domains in the industry today: Mobile Development, Big Data Analytics, Cloud Computing, and DevOps. Having deep knowledge in these domains make us stand out as the best and the coolest provider of services. Our DevOps will bring you a single team that works together with continuous development, continuous testing, continuous deployments, logging, monitoring and security.
- 3. WhiteHedge 3 WHITEHEDGE - AN OVERVIEW Global Presence Agile + Flexible Thorough + Quick Learner Competitive + Comprehensive Honest + Transparent Young + Mature Innovative + Creative More about us… What definesus? California New Jersey Rotterdam Pune EnvisionProducts| Convertintobusinesses 100+employees|50+live productsworld-wide Thebest oftheTalentand Infrastructure Started2003| FocusedAgile ProductDevelopment Selffunded| Wellfunded| Profitable
- 4. WhiteHedge 4 TABLE OF CONTENTS - OUR AGENDA Containers 1 Use case 2 Orchestration 3 Introduction to ECS 4 Key Components 6 Architecture 7 Overview of Build Process 8 Why ECS? 5
- 5. CONTAINERS
- 6. WhiteHedgeWHAT ARE CONTAINERS? OS virtualization Process isolation Images Automation Bins/Libs Docker Engine Bins/Libs App2App1 Host OS Server Guest OS Hypervisor Guest OS Bins/LibsBin/Libs Host OS Server App1 App2 Container 1 Container 2 VM 1 VM 2
- 7. WhiteHedge “automates the deployment of any application as a lightweight, portable, self-sufficient container that will run virtually anywhere” CONTAINERS
- 8. USE CASE
- 9. WhiteHedgeUSE CASE A highly available RESTful API server which fetches data from third-party services and serves its clients.
- 11. WhiteHedgeCONTINUOUS DELIVERYWITH JENKINS Build Image Push Image CodeDeploy
- 12. WhiteHedgeTHE PROBLEM • A large monolithic application that was difficult to run, deploy, and scale. • We were deploying a single docker container to run multiple processes. • Docker was monitored and we were redeploying docker instance if the docker instance runs into problems. • We used to evaluate capacity of EC2 instance manually and are running one container per instance
- 13. WhiteHedgeTHE CHALLENGE • Use microservices architecture using docker • Orchestration of dockers • Replicate infrastructure to many regions
- 14. ORCHESTRATION
- 16. WhiteHedgeORCHESTRATION DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server DockerEngine Host OS Server
- 18. WhiteHedgeWHY ECS? • It is a managedservice! • Easy to integrate with other AWS services • It is great for storing and enforcing task state • Designed with custom schedulers in mind • The agent code is available on a public GitHub repo and … it is in GO!
- 19. WhiteHedgeWHAT ELSE DID WE LOOK AT? Home-grown Tech Tried, but proved to be unreliable Difficult to handle coordination and synchronization Powerful but hard to productionize Needs developers with Experience Not a managed service, higher Ops load “Amazon ECS enabled us to focus on releasing new software rather than spending time managing clusters”
- 21. WhiteHedge “a highly scalable, high performance container management service that supports docker containers and allows you to easily run and manage Docker- enabled applications across cluster of EC2 instances” WHAT IS ECS?
- 22. WhiteHedgeWHAT IS ECS? • Building Block Service • Easily Manage Clusters for Any Scale Nothing to run Complete state Control and monitoring Scale • Flexible Container Placement • Designed for use with other AWS services • Secure • Extensible - ComprehensiveAPIs
- 23. WhiteHedgeKEY COMPONENTS Container Instances Clusters Tasks Run a task Create a service
- 24. WhiteHedgeKEY COMPONENTS • Amazon EC2 Instances • Docker daemon • Amazon ECS agent Container Instances Clusters Tasks Run a task Create a service
- 25. WhiteHedgeKEY COMPONENTS • Grouping of Container Instances • Resource pool • Regional • Start empty, dynamically scalable Container Instances Clusters Tasks Run a task Create a service
- 26. WhiteHedgeKEY COMPONENTS • Unit of work • Grouping of related Containers • Run on Container Instances Container Instances Clusters Tasks Run a task Create a service
- 27. WhiteHedgeKEY COMPONENTS: TASK DEFINITIONS { "environment": [], "name": "simple-demo", "image": "my- demo", "cpu": 10, "memory": 500, "portMappings": [ { "containerPort": 80, "hostPort": 80 } ], "mountPoints": [ { "sourceVolume": "my-vol", "containerPath": "/var/www/my- vol" } ], "entryPoint": [ "/usr/sbin/apache2" , "-D", "FOREGROUND" ], "essential": true }, { "name": "busybox", "image": "busybox", "cpu": 10, "memory": 500, "volumesFrom" : [ { "sourceContainer": "simple-demo" } ], "entryPoint" : [ "sh", "-c" ], "command": [ "/bin/sh -c "while true; do /bin/date > /var/www/my-vol/date; sleep 1; done"" ], "essential": false }
- 28. WhiteHedge { "environment": [], "name": "simple-demo", "image": "amazon/amazon-ecs- sample", "cpu": 10, "memory": 500, "portMappings": [ { "containerPort": 80, "hostPort": 80 } ], "mountPoints": [ { "sourceVolume": "my-vol", "containerPath": "/var/www/my- vol" } ], "entryPoint": [ "/usr/sbin/apache2 ", "-D", "FOREGROUND" ], "essential": true }, KEY COMPONENTS: TASK DEFINITIONS Essential to our task 10 CPU units (1024 is full CPU), 500 megabytes of memory Expose port 80 in container to port 80 on host Create and mount volumes
- 29. WhiteHedge { "name": "busybox", "image": "busybox", "cpu": 10, "memory": 500, "volumesFrom" : [ { "sourceContainer": "simple-demo" } ], "entryPoint" : [ "sh", "-c" ], "command": [ "/bin/sh -c "while true; do /bin/date > /var/www/my-vol/date; sleep 1; done"" ], "essential": false } KEY COMPONENTS: TASK DEFINITIONS [ { "image": "tutum/wordpress-stackable", "name": "wordpress", "cpu": 10, "memory": 500, "essential": true, "links": [ "db" ], "entryPoint": [ "/bin/sh", "-c" ], "environment": [ … ], "portMappings": [ { "containerPort": 80, "hostPort": 80 } ] }, ] From Docker Hub Mount volume from other container Command to exec
- 30. WhiteHedgeKEY COMPONENTS Good for short-lived containers, e.g. batch jobs Container Instances Clusters Tasks Run a task Create a service
- 31. WhiteHedgeKEY COMPONENTS Good for long-running applications and services Container Instances Clusters Tasks Run a task Create a service
- 33. WhiteHedgeTYPICAL USER WORKFLOW I want to run aservice
- 34. WhiteHedgeTYPICAL USER WORKFLOW Run Instances Amazon EC2 Use custom AMI with Docker support and ECS Agent. Instances will register with defaultcluster.
- 35. WhiteHedgeTYPICAL USER WORKFLOW Create Task Definition Declare resource requirements for containers SharedDataVolume Node.jsApp Time of day App
- 36. WhiteHedgeTYPICAL USER WORKFLOW Create Service Declare resource requirements for service SharedDataVolume Node.jsApp Time of day App Elastic Load Balancin g X 3
- 37. WhiteHedgeTYPICAL USER WORKFLOW Describe Service
- 40. WhiteHedgeCONTINUOUS DELIVERYWITH JENKINS Build Image Push Image Update ECS service
- 41. WhiteHedge Features Without ECS Using ECS High availability If any of the container dies, we have to redeploy on all servers. Restarts that container itself. (Its a cluster management system so it will always be in desired state) Managing number of containers on a given EC2 instance It will be complicated to run and manage multiple containers effectively. Can manage multiple containers. Deployment method Handle deployments using various methods. (CodeDeploy, chef etc.) ECS manages deployment of new revision. COMPARISON OF ARCHITECTURES
- 42. WhiteHedge 42 FOLLOW US Questions? http://www.whitehedge.com/devops.html Swapnil Dahiphale [email protected] - SOCIAL NETWORKS / PHONE
- 43. THANK YOU! Have a Nice Day! Swapnil Dahiphale @Swapnil2233 [email protected]
Editor's Notes
- #3: Remove slide?
- #7: Similar to hardware virtualization like ec2 however instead of partitioning machine, they isolate processes that are running on single OS. Useful when you want to use os kernel to create multiple isolated userspace processes that have constraints on them like CPU and memory. Docker enables easy use of it, you can create images of containers, that enables automation. So we are able to define our app, build, share and deploy the image
- #11: Before using ECS
- #20: Different tools available
- #22: the same Docker containers that you currently use will run exactly same on ECS. eliminates the need to install operate and scale your own cluster management system infra handles the complexity of running multiple containers like where in the cluster to schedule the container, availability, scalability and monitoring.
- #23: Use api’s to run apps or build a platform to run our apps on it
- #24: Terminologies: Task: Grouping of related containers
- #25: Terminologies: Task: Grouping of related containers
- #26: Terminologies: Task: Grouping of related containers
- #27: Terminologies: Task: Grouping of related containers
- #28: Defines task: Which containers to be run, how much resources it should have Port mapping links
- #31: Terminologies: Task: Grouping of related containers
- #34: I have a docker image with app contained, how do I run