Powering Radical Agility with Docker
2 likesā¢2,305 views
The document discusses Zalando's use of Docker technology as part of its robust platform for managing over 18 million active customers across Europe. It outlines the architecture involving AWS, immutable stacks, and the development of an open-source OAuth2 provider, highlighting the importance of resilience, scalability, and monitoring metrics. Various tools and methodologies used by Zalando's engineering teams, such as GOCD and ZMON for CI/CD and monitoring, are also emphasized.
![SENZA: DEFINITION YAML
SenzaInfo:
StackName: hello-world
Parameters:
- ImageVersion:
Description: "Docker image version of Hello World."
SenzaComponents:
- Configuration:
Type: Senza::StupsAutoConfiguration # auto-detect network setup
- AppServer: # will create a launch configuration and ASG with scaling triggers
Type: Senza::TaupageAutoScalingGroup
InstanceType: t2.micro
SecurityGroups: [app-hello-world]
ElasticLoadBalancer: AppLoadBalancer
TaupageConfig:
runtime: Docker
source: "stups/hello-world:{{Arguments.ImageVersion}}"
ports:
8080: 8080](https://image.slidesharecdn.com/2016-06-07docker-meetup-dortmund-160613100102/85/Powering-Radical-Agility-with-Docker-17-320.jpg)
Powering Radical Agility with Docker
- 1. Docker - Powering RA at Zalando Docker Meetup - Dortmund 7.6.2016 | [email protected] | @JanMussler
- 2. 15 countries 3 fulfillment centers 18+ million active customers 3.0+ billion ā¬ revenue 135+ million visits per month 1.000+ employees in tech Europe's Leading Fashion Platform Visit us: tech.zalando.com
- 4. Platform 80+ Engineering teams Platform team deploy Server needs Storage requests
- 8. STUPS
- 9. AWS STUPS DOCKER DEPLOY SSH ACCESS AUDIT REPORTS FULL AWS ACCESS STUPS: A PLATFORM ON TOP OF AMAZON WEB SERVICES
- 11. Internet *.abc.example.org *.xyz.example.org Team ABC Team XYZ ISOLATED AWS ACCOUNTS EC2EC2 ELBELB EC2
- 12. DEPLOYMENT
- 13. IMMUTABLE STACKS ELB myapp-1 myapp.example.org EC2 + Docker EC2 + Docker EC2 + Docker
- 14. IMMUTABLE STACKS ELB myapp-1 EC2 + Docker EC2 + Docker EC2 + Docker ELB myapp-2 EC2 + Docker EC2 + Docker myapp.example.org
- 15. ā Immutable AMI ā YAML user data ā Docker runtime ā Application logging: LogEntries, Scalyr, CloudWatch Logs ā Prometheus Node Agent for metrics ā KMS encrypted env vars TAUPAGE AMI
- 16. Taupage AMI
- 17. SENZA: DEFINITION YAML SenzaInfo: StackName: hello-world Parameters: - ImageVersion: Description: "Docker image version of Hello World." SenzaComponents: - Configuration: Type: Senza::StupsAutoConfiguration # auto-detect network setup - AppServer: # will create a launch configuration and ASG with scaling triggers Type: Senza::TaupageAutoScalingGroup InstanceType: t2.micro SecurityGroups: [app-hello-world] ElasticLoadBalancer: AppLoadBalancer TaupageConfig: runtime: Docker source: "stups/hello-world:{{Arguments.ImageVersion}}" ports: 8080: 8080
- 18. SENZA: STACK DEPLOYMENT $ senza create hello-world.yaml 1 0.2 Generating Cloud Formation template.. OK Creating Cloud Formation stack hello-world-1.. OK $ senza events hello-world.yaml 1 Stack NameāVer.āResource Type āResource ID āStatus āStatus Reason āEvent Time hello-world 1 CloudFormation::Stack hello-world-1 CREATE_IN_PROGRESS User Initiated 10m ago ... hello-world 1 CloudFormation::Stack hello-world-1 CREATE_COMPLETE 6m ago
- 20. SSH ACCESS
- 21. SSH ACCESS: TIME-LIMITED ACCESS TO ANY TEAM SERVER
- 22. LOGGING
- 24. Automation GOCD
- 25. Thoughtworkās GOCD in action
- 26. GOCD - Pipeline example - configuration overlay
- 27. Plan - B
- 28. The OAuth 2.0 authorization framework enables a third-party application to obtain limited access to an HTTP service. - oauth.net OAUTH 2.0?
- 29. ā Robustness & resilience ā Cassandra, no SPOF ā Low latency for token validation ā Token Info next to application ā Horizontal scalability ā Cassandra, āstatelessā Token Info PLAN B: GOALS - Build open source Oauth2 Provider
- 30. PLAN B: COMPLETE PICTURE bobalice create token Token Info validate Provider credential storage Revocation poll public keys poll revocation listsS3 call with Bearer token
- 31. Written in Go ~16 MB Docker image Stateless application CPU bound, Go 1.6 ~40x speedup for EC verify EC2 instance start to healthy: 45sec Scaling Token Info example
- 32. ZMON
- 33. Flexible and extendable: Checks & Alerts in Python Integrate: REST APIs, OAUTH2, AWS Auto Discovery Fully configurable via UI / API: no restarts required! Great for teams: team dashboards, alerts inheritance Fast/scaling metrics: Redis, KairosDB + Grafana3 Hackweek 2015 - iOS app and Android app ;-) ZMON - High Lights ;-)
- 37. Continued ...
- 38. Instance Metrics ā Memory usage ā Disk space usage ā CPU usage ā Application logs ā Application metrics Monitoring instances on AWS Scalyr Agent Log shipping Prometheus Node Agent :9100/metrics Taupage AMI (Ubuntu base) Application Container Go / Spring Boot / Cassandra Docker run time :8080 -> app :7979 -> metrics
- 39. Annotated Metric Data in Grafana
- 40. Annotated Metric Data in Grafana
- 41. Running same Docker Image everywhere
- 42. CLAIR - SQS
- 43. CoreOSā Clair with PierOne - Static vulnerability analysis of images
- 46. Learnings?
- 47. ā AWS terminology and behavior ā OAuth2 + Security + Security Groups ā Ops can be hard -> SaaS? ā CF deployment takes time ā DNS load balancing and switching :-( ā Remember timeout config ā¦!! ā ELB soso ... ā Great flexibility and power though A lot of input to cover ...
- 48. Zalando on Github: https://github.com/zalando STUPS online: https://stups.io ZMON Demo: https://demo.zmon.io Zalando Tech: https://tech.zalando.com