Modern DevOps with Spinnaker/Concourse and Micrometer
- 1. © Copyright 2018 Pivotal Software, Inc. All rights Reserved. Version 1.0 April 17, 2019 Modern DevOps Tools with Spinnaker / Concourse / Micrometer
- 2. Cover w/ Image Topics 1. Spinnaker Overview 2. Spinnaker and PCF 3. Concourse Integration 4. Q&A
- 3. Why Spinnaker for Continuous Delivery?
- 4. Spinnaker Is an OSS Multi-Cloud Delivery Platform Spinnaker Community “...the passionate open source community dedicated to making deployment pain go away.” https://www.spinnaker.io/publications/ebook/
- 5. Application Centric Control Plane =
- 7. Interactive Continuous Delivery Pipeline CI Production Arbitrary Jobs
- 8. Spinnaker Ecosystem Cloud Providers App Engine Amazon Web Services Azure Cloud Foundry DC/OS Google Compute Engine Kubernetes Openstack Oracle CI Systems Jenkins Travis CI Wercker Concourse (coming soon) Artifact Support Docker Google Cloud Storage GitHub HTTP S3 Artifactory Monitoring Datadog Prometheus Stackdriver Atlas SignalFx (10+ more on roadmap) Notifications Email HipChat Slack SMS via Twilio
- 9. Spinnaker Best Practices ● Blue/green deployments ● Rollbacks ● Automated canary analysis
- 10. Minimizing Production Deployment Risk https://www.spinnaker.io/concepts/
- 11. Multiple Application Versions for Rollback Allows zero-downtime rollbacks to a set number of previous application versions without rebuilding the code
- 12. Automated Canary Analysis Ensures your applications meet your SLAs https://medium.com/netflix-techblog/automated-canary-analysis-at-netflix-with-kayenta-3260bc7acc69
- 14. Spinnaker Now Available PCF 2.0+ Announcement
- 15. Spinnaker 1.13 and PCF Details ● Zero-downtime blue/green deployments ● Multi-foundation view of applications ● Manifest-based deployment ● Application management actions and pipeline stages such as scale, enable/disable, destroy and rollback, map/unmap routes ● Clone stage for promotion of applications across environments ● Pipeline stages to deploy/destroy services ● Bind applications to services as part of deploy stages ● Artifact framework allowing triggering and assembling deployments out of different parts
- 16. Spinnaker Roadmap ● Will be available from Pivotal later this year ● Looking for customers ● Add-on to PKS offering ● Deployed to PKS, managing deployments to CF and PKS ● Full integration with CF features (autoscaling, c2c, cf tasks) ● Artifact provenance and compliance ● Dynamic config and RBAC ● Onboarding ● Declarative delivery
- 19. Stateless: Rather than sharing state, every task runs in its own container, controlling its own dependencies. Ephemeral: Clean and consistent builds. Minimizes test infrastructure to maintain and gives you flexibility to scale with the cloud. Continuous integration: Track versions of external artifacts used for CI (e.g., Git, S3, docker image). Build components expressed as code with simple modeling of modular components. Flexible workflow automation: Extensible for compliance or other customized tasks in a pipeline. Stateful: Maintains awareness of the topography (multiple clouds, PCF Foundations, etc.) and keeps record of all pipeline activities for compliance and audit. Application inventory: Maintains inventory of deployed applications, supporting activities like rollback, canary analysis and identifying security vulnerabilities. Sophisticated deployment scenarios: Canary deploys, red/black or blue/green deploys, progressive deployments (e.g., by time zone), multi-cloud/PCF Foundation deployments. How Does Spinnaker Relate to Concourse?
- 20. Right Levels of Automation ● Resource consumption: ○ CI runs each deploy job individually ○ Spinnaker efficiently batches ● Spinnaker itself is fifteen microservices ● CI for getting started, team-centric CI/CD ● Spinnaker for org-centric CI/CD for microservices ● Transition from CI-based tools to Spinnaker leveraging existing investments
- 21. Concourse Trigger and Stage in Spinnaker
- 22. Overview ● Modeled after other Spinnaker CI integrations ● Basic auth Spinnaker and Concourse ● Spinnaker trigger and stage ● Concourse resource for stage ● Concourse trigger with build metadata (optional): ○ Concourse resource for build metadata ○ Ability to trigger Spinnaker pipeline manually and deploy specific Concourse build ○ Other trigger types may be more appropriate for metadata storage such as artifactory, docker, github
- 25. 1. Concourse job runs Any Concourse out resource metadata will be available to a running Spinnaker pipeline.
- 26. Manually starting a pipeline with a Concourse trigger
- 27. Using Concourse trigger info in subsequent stages Ex: use the version to identify the application binary in a Deploy stage
- 28. Using Concourse trigger info in subsequent stages
- 29. 2. Spinnaker pipeline is triggered
- 31. Configuring an Artifactory trigger
- 32. Git Trigger
- 33. Configuring a Git trigger
- 35. Stage configuration The Concourse resource uses these to identify whether a running stage is a match against the job it is attached to.
- 36. 1. Stage starts.
- 37. 2. Concourse check resource identifies the stage.
- 38. 3. Concourse tells Spinnaker which job to watch.
- 39. 4. Concourse job completes.
- 40. 5. Spinnaker sees the Concourse job is finished, and completes the stage.
- 41. Deliver Better Software All the Time Continuous Delivery
- 42. Delivery Patterns at Netflix
- 43. Blue/green deployment of message-driven apps requires coordination. Not enough
- 44. Metrics data is tagged with the same structural information that Spinnaker displays.
- 45. Beware reactive (and predictive) autoscaling.
- 46. Automated Canary Analysis Ensures your applications meet your SLAs https://medium.com/netflix-techblog/automated-canary-analysis-at-netflix-with-kayenta-3260bc7acc69
- 47. Widespread use of circuit breakers mean comprehensive end-to-end environments are not necessary.
- 48. Netflix Prana: A Cautionary Tale
- 49. Mesh will not be able to tag telemetry with the same richness; can only monitor data passing through it.
- 50. Prevents the necessary coordination between the CD system and application for blue/green of event-driven and bidirectional RPC apps.
- 51. Cannot understand where hedge-requesting is appropriate as a load balancing strategy.
- 52. Deliver Better Software All the Time Continuous Delivery [email protected]