Orchestrating containers on AWS | AWS Summit Tel Aviv 2019
0 likes383 views
The document discusses container orchestration on AWS, focusing on microservices and the use of Docker for efficient application deployment and management. It highlights AWS services like Elastic Container Service (ECS) and Elastic Kubernetes Service (EKS) for managing and scaling containerized applications. Additionally, it emphasizes the benefits of using containers for isolated execution, rapid development cycles, and built-in security features.
![© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
www.datree.io www.datree.io
About me
Shimon Tolts {
age: 30,
title: “CTO & Co-Founder @ datree.io”,
misc: [“AWS Community Hero”,“Gamer”]
}](https://image.slidesharecdn.com/con201-orchestrating-container-7a3cfa2a-cf41-4aab-b33b-45c94f0bdff6-2103105383-190321085800/85/Orchestrating-containers-on-AWS-AWS-Summit-Tel-Aviv-2019-25-320.jpg)
Orchestrating containers on AWS | AWS Summit Tel Aviv 2019
- 1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Orchestrating containers on AWS Liron Dor Solutions Architect Manager Amazon Web Services C O N 2 0 1 Dima Breydo Solutions Architect Amazon Web Services Shimon Tolts CTO & Co-Founder Datree
- 2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda Microservices and containers AWS container services Datree story and live demo
- 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. When the impact of change is small, release velocity can increase Monolith Does everything Microservices Does one thing
- 4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Monolith development lifecycle monitorreleasetestbuild developers delivery pipelines services
- 5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Microservice development lifecycle developers services monitorreleasetestbuild delivery pipelines monitorreleasetestbuild monitorreleasetestbuild monitorreleasetestbuild monitorreleasetestbuild monitorreleasetestbuild
- 6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers and Docker A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.1 1 https://www.docker.com/resources/what-container Server Operating System Docker Engine AppA AppB AppC AppD
- 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Docker Image Used to launch container Instructions documented in Dockerfile Merge layers into single image Read-only template kernel Base Image Image layer Image layer Image layer References parent image layer
- 8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers and Microservices • Do one thing, really well • Any app, any language • Isolated execution environment • Test and deploy same artifact • Faster startup Container Container Container Container
- 9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers have become the standard for how to ship and run your application in the cloud
- 10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Manually downloading and launching containers by hand is inefficient and error prone
- 11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Container orchestration
- 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Container Services landscape MANAGEMENT Deployment, scheduling, scaling & management of containerized applications HOSTING Where the containers run IMAGE REGISTRY Container image repository
- 14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Gives you primitives for building modern applications Helps you run containers at scale Open source container management platform Kubernetes
- 15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon EKS architecture mycluster.eks.amazonaws.com EKS workers Kubectl AZ 1 AZ 2 AZ 3 Your AWS account VPC
- 16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Kubernetes control plane Highly available and single tenant infrastructure All “native AWS” components Fronted by an NLB VPC API Server ASG Etcd ASG NLB AZ-1 AZ-2 AZ-3 ELB Instances Instances
- 17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon EKS features Certified conformant Integration with Elastic Load Balancing Managed updates IAM authentication
- 18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Elastic Container Service Highly Scalable , Highly Performant Container Management System A managed platform ECS Cluster Management Container Orchestration & Placement Deep AWS Integration
- 19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon ECS key components Developmentcluster Container instance Container instance Container instance Productioncluster Container instance Container instance Container instance AmazonElasticContainer Service (AmazonECS) Container Container Volume Taskdefinition AmazonElasticContainer Registry
- 20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon ECS features Integration with Elastic Load Balancing Service Discovery with AWS CloudMap Task level IAM support Blue/Green Deployments with AWS CodeDeploy Windows Containers Compatibility
- 21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. ECS Agent Docker AgentOS EC2 InstanceEC2 hosting Choose your instance type Connect to the instance Persistent EBS storage Custom kernel modules
- 22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. “Just launch 10 copies of my container distributed across three availability zones and connect them to this load balancer” X 10
- 23. www.datree.io
- 24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io Agenda About me & datree.io What is Fargate TCO? How do we deploy (technical) DEMO Why did we choose to use Fargate 1 2 3 4 5
- 25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io About me Shimon Tolts { age: 30, title: “CTO & Co-Founder @ datree.io”, misc: [“AWS Community Hero”,“Gamer”] }
- 26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io 0 EC2 instances @ datree.io
- 27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io GitOps Datree is a Policy Enforcement Platform for confident and compliant code.
- 28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io GitOps is the new normal Build Test Deploy Test Build Deploy Deploy Code, build, test, package, release, configure, deploy and rollback your application with Git almost instantly Developers have access to production DEVELOPMENT PRODUCTION
- 29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io Datree connects with GitHub Datree connects with GitHub to provide automatic policy compliance checks and insights for every code commit and pull request.
- 30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io Datree connects with GitHub Set Code Policies Create custom policies or choose from recommended defaults, and choose where they should be enforced. Separate secret credentials from source code Include .gitignore in every project Link pull request title to a Jira ticket Create custom policies…
- 31. www.datree.io
- 32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io Apples and oranges ! = Amazon EC2
- 33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io Ops benefits AWS Monthly Fees ECS-Fargate AWS Monthly Fees System Administration Security & Compliance Scaling ECS-EC2
- 34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io No more Amazon EC2 management We no longer configure AMIs Monitoring & logging is built in
- 35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io Operating system management All of our code is packaged using Docker containers, so we are ONLY responsible for what runs within our containers No more: Linux Patching Docker service updates ECS Agent updates
- 36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io Scaling We no longer deal with scaling EC2 fleets Taking care of bin packing our instance to run cost-effectively
- 37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io Compliance & security Compliance and security are our top priorities Out-of-the-box security as a service Fargate comes certified with SOC 2 HIPAA PCI-DSS
- 38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io Building an icecream service ● http://icecream.datree.io ● A simple Node.js app ● A web service using Koa.js ● Serving icecream! 38
- 39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io Pipeline overview ● Source code in GitHub ● AWS Application Load Balancer ● AWS ECS Fargate cluster ● GitHub Actions workflow (CI/CD) 39
- 40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io DEMO time
- 41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io ECS on EC2 vs. Fargate • 10 GB disk space limit • No instance type selection (GPU/CPU/MEM optimized) • Amazon EBS attaching is not available • No Spot Instances support • No Reserved Instance pricing
- 42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io Open case study on AWS Fargate page https://dtr.ee/fargate
- 43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. www.datree.io www.datree.io Resources 1. https://datree.io/blog/migrating-to-aws-ecs-fargate-in-production/ 2. https://www.youtube.com/watch?v=rtk3rRdAZ6s&feature=youtu.be &t=1239 3. https://github.com/silinternational/ecs-deploy
- 44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Related breakouts 14:10 Deep Dive on Amazon Elastic Container Service (ECS) Brent Langston 15:00 Mastering Amazon Elastic Container Service for Kubernetes (Amazon EKS) Kobi Biton, Chen Fisher 15:50 From Code to a running container Alexei Ledenev, Gal Marder
- 45. Thank you! © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. http://bit.ly/2SJ6Md2