Continuous Delivery
7 likes•3,353 views
The document discusses the different approaches to software deployment, contrasting traditional methods used by banks and government agencies with modern techniques favored by social media companies, emphasizing continuous delivery and automation. It outlines the goals of improving software delivery quality, reducing errors, and enabling faster deployments while minimizing downtime and stress. Key components of successful deployments include a well-defined software delivery pipeline, automation of processes, and strategies for handling data migration and error management.
Continuous Delivery
- 1. Software-Deployments ohne graue Haare Continuous Delivery 24. Juni 2013 – Corsin Decurtins
- 3. Netcetera | 3 Some numbers… On the left Traditional application from something like a bank, an insurance company, a government agency, a transport authority … you name it. On the right One of those new fancy social network, Web 2.0-ish application. Think Facebook, Google Mail, Flickr, Twitter, … Both are serious applications with serious business value and mission-critical IT systems.
- 4. Netcetera | 4 Some numbers… Both applications need a high up-time. Deployments to the production environment are risky. Complex deployments with a lot of dependencies. Both companies try to reduce the risk of deployments. Deployments are painful and stressful.
- 5. Netcetera | 5 Two completely different approaches "Banks" try to reduce the risk and effort for deployments by reducing the number of deployments. The "Web 2.0" side tries to reduce the risk of deployments by making much, much smaller deployments … but a lot of them.
- 6. Netcetera | 6 If it hurts, do it often.
- 7. Netcetera | 7 Our goals Improve Quality Fewer errors, less (unexpected) down-time. Standardization Different applications should deploy in a similar way. Make it easier for the operations and the development teams. Faster Deployments Less downtime, smaller need for off-hour deployments. More Deployments Faster time-to-market for features and bug fixes. Less Stress
- 8. prod dev dev test-int test-uat Metrics SCM Repo build unit testing check-in deploy deploy deploy integration testing performance testing user acceptance security review qa CSO ops
- 9. Netcetera | 9 Software Delivery Pipeline The Software Delivery Pipeline is the combined set of processes, procedures and tools that you use to bring code from the development environment into production. Of course you would make the pipeline even wider by including requirements engineering etc. but for the context of this talk, we will start with code.
- 10. Netcetera | 10 Software Delivery Pipeline Builds Propagate through the Pipeline Successful completion of a step triggers the next steps. Errors interrupt the Pipeline In case of an error, the pipeline processing is interrupted. Expensive steps are only done if the previous steps worked For example (manual) User Testing is only done on builds that passed the automated tests. Some Pipeline steps might require manual triggering Deployment to uat or prod is probably still triggered manually
- 11. Netcetera | 11 Software Delivery Pipeline A few things that you should notice right here: Everybody has a Software Delivery Pipeline. You might not have thought about it yet, it might be automated or manual, very simple or complex, but you have a pipeline. Whether you want it or not, whether you manage it or not. Your pipeline might look differently. Depends on the project, the customer, the business domain, the degree of maturity, …
- 12. Netcetera | 12 Continuous Delivery Deliver software changes continuously and in a fully-automated way from the developer to the production environment. Very small increments Automation Fully automated QA Continuously Disruption-free deployments
- 13. prod dev dev test-int test-uat Metrics SCM Repo build unit testing check-in deploy deploy deploy integration testing performance testing user acceptance security review qa CSO ops
- 14. Netcetera | 14 Source Code Management (SCM) CVS and many more Filesystem
- 15. Netcetera | 15 Source Code Management (SCM) We are currently moving from Subversion to Git Git has interesting properties related to Continuous Delivery Potentially different software delivery pipelines per branch Feature branches build, run unit tests, deploy to test, run integration tests, … but no deployment to QA and prod Master branch full software delivery pipeline
- 16. prod dev dev test-int test-uat Metrics SCM Repo build unit testing check-in deploy deploy deploy integration testing performance testing user acceptance security review qa CSO ops
- 17. Netcetera | 17 Artifact Repository Filesystem and many more
- 18. Netcetera | 18 Artifact Repository Important for the reproducability of builds and releases We use Nexus
- 19. prod dev dev test-int test-uat Metrics SCM Repo build unit testing check-in deploy deploy deploy integration testing performance testing user acceptance security review qa CSO ops
- 20. Netcetera | 20 Build Server and many more
- 21. Netcetera | 21 Build Server We use Jenkins (with some plugins) Continuous Delivery requires a lot of build server power We currently have more than 400 build jobs on our main Jenkins server 1 main server and 6 slaves Feature builds are yet to come We (normally) build at most once per hour Expensive build jobs (UI tests, performance tests) normally run only once per day
- 22. prod dev dev test-int test-uat Metrics SCM Repo build unit testing check-in deploy deploy deploy integration testing performance testing user acceptance security review qa CSO ops
- 23. Netcetera | 23 Issue Management and many more
- 24. prod dev dev test-int test-uat Metrics SCM Repo build unit testing check-in deploy deploy deploy integration testing performance testing user acceptance security review qa CSO ops
- 25. Netcetera | 25 Metrics / Monitoring and many more Infostore
- 26. Netcetera | 26 Metrics Continuous collection of metrics about the code, deployments, performance numbers,… We use SonarQube for code analysis SonarQube rocks! Summary data from Sonar as well as other metrics are imported into Infostore Infostore is our company database Includes information on dependencies Hopefully soon as continuously measure performance metrics
- 28. prod dev dev test-int test-uat Metrics SCM Repo build unit testing check-in deploy deploy deploy integration testing performance testing user acceptance security review qa CSO ops
- 29. Netcetera | 29 Testing and many more
- 30. Netcetera | 30 Testing Fully automated and continuous testing is crucial Testing on very different levels and in different places in the software delivery pipeline Unit testing, integration testing, smoke tests, performance tests, load tests, UI tests, …
- 31. prod dev dev test-int test-uat Metrics SCM Repo build unit testing check-in deploy deploy deploy integration testing performance testing user acceptance security review qa CSO ops
- 32. Netcetera | 32 Deployment Installation of a software release on a target environment Replacement of existing software releases Data migration System reconfiguration
- 33. Netcetera | 33 Automation of the Deployment Process $ deploy <application> <version> <environment> Deployment was successful $ deploy <application> <version> <environment> Deployment failed, rolled back to old version
- 34. Netcetera | 34 Deployment Scripts glu and many more
- 35. Netcetera | 35 Deployment Tools (1/2) We use Puppet for the deployment of infrastructure components Application servers, web servers, … Not suited for the application deployments that we do Security (push vs. pull) Lack of control over the deployment time For application deployment, we currently use scripts and Rundeck as well as Flyway for database migrations KISS We have looked at other tools.
- 36. Netcetera | 36 Deployment Tools (2/2) The kind of deployment tool that you need, depends a bit on your needs. Scripts/Rundeck: very low level, but simple and effective LiveRebel: great for disruption-free deployments, but expensive if you do not need this feature glu (and similar tools): great if you need to deploy an application on a farm of servers (and monitor them) Puppet/Chef: great if you do not (have to) care about deployment times
- 37. Netcetera | 37 Error Handling and Rollbacks Ability to roll back at any time Back up the preexisting state of the system Try to make the deployment non-destructive Detecting errors Deployment process has to be able to detect errors Automated Rollbacks Very desirable But also very complex Has to be tested (continuously)
- 38. Netcetera | 38 Manual Rollback It's just a workaround, but often a viable option Probability of failing deployments is relatively small We are testing the deployment continuously after all Automated deployment process provides operations people with the necessary tools and artifacts to roll back easily Backups Non-destructive deployments
- 40. Netcetera | 40 Disruption-Free Deployments If you make very frequent deployments, a downtime is not acceptable anymore. Different tools, technologies and architectures for doing disruption-free deployments.
- 41. Netcetera | 41http://martinfowler.com/bliki/BlueGreenDeployment.html Blue/Green Deployments Two production environments; blue and green. Green is running as the current production system. Deploy to blue, test and verify. Switch from green to blue. Very elegant solution. State (database, session) is an issue.
- 42. Netcetera | 42
- 43. Netcetera | 43 LiveRebel Product from ZeroTurnaround Based on the JRebel technology Replace running code without the need for restarting Very cool technology Commercial product, but worth every penny (if you ask me) Far too expensive, if you do not really need disruption-free deployments
- 44. Netcetera | 44 Tomcat Parallel Deployments
- 45. Netcetera | 45http://tomcat.apache.org/tomcat-7.0-doc/config/context.html#Parallel_deployment Tomcat Parallel Deployments Tomcat 7 Parallel Deployments Different version of a web application can be deployed in parallel Tomcat routes requests to the different versions: If no session information is present in the request, use the latest version. If session information is present in the request, check the session manager of each version for a matching session and if one is found, use that version. If session information is present in the request but no matching session can be found, use the latest version.
- 46. Netcetera | 46 Data Migration Data migration is a tricky thing Slow Complex Error-prone Stop-the-World Automation and continuous testing helps More frequent and therefore smaller deployments help
- 47. Netcetera | 47 Data Migration Patterns On-the-fly migration Application can deal with multiple versions of the database Data migration is done on-the-fly (batch job or touch-and-go) Read-Only availability Database is read-only as long as the migration is on-going Storage Layer abstraction Encapsulation of the storage layer NoSQL storage layer Less stringent requirements on schema compliance
- 49. Netcetera | 49 Deployment Testing Deployments are tested continuously Continuous Testing environment Continuously, triggered automatically Pre-Production environment Regularly, triggered manually "Same" configuration as the production environment "Same" state as the production environment
- 50. Netcetera | 50 Health-Check Pages Simple page/resource in the application Reports the current health status of an application Basically: I'm fine, thanks. Got everything that I need. Not doing very well, missing some dependencies. Trying my best. I'm giving up. Some crucial dependencies missing. Please help. <no answer at all> Can be used for monitoring purposes, but also for deployment smoke tests.
- 52. prod dev dev test-int test-uat Metrics SCM Repo build unit testing check-in deploy deploy deploy integration testing performance testing user acceptance security review qa CSO ops
- 53. Netcetera | 53 How far do you want to go? How far can you go? Is the effort worth it? Is the customer ready? Is the operations team ready? Mobile and client applications Technical limitations Build and test infrastructure Production infrastructure
- 54. Netcetera | 54 A way towards Continuous Delivery It's a maturity process It takes time to get there Address what is hurting most Automate further and further Run continuously
- 55. Netcetera | 55 Our Experiences Increase of quality Lowering of costs Developers Operations Customers Reduction of stress You can do a lot without asking for permission
- 56. Netcetera | 56 Continuous Delivery Software Delivery Pipeline Automation Continuation of Continuous Integration Continuous… … Builds, Integration, Testing … Deployment, Migration Deployment Testing Smaller and faster deployments Reduction of Deployment Risks Quality Improvement Faster Deployments Down-Time minimalization Shorter time-to-production Less stressful deployments Potential for more frequent deployments
- 57. Netcetera | 57 > deploy "Continuous Delivery" audience deployment of Continuous Delivery was started by user corsin Continuous Delivery was successfully deployed to environment audience log file has been stored to: http://www.slideshare.net/netceteragroup/ >
- 58. prod dev dev test-int test-uat Metrics SCM Repo build unit testing check-in deploy deploy deploy integration testing performance testing user acceptance security review qa CSO ops
- 59. Netcetera | 59 Further Reading Continuous Delivery Jez Humble, David Farley Literally the book on "Continuous Delivery".
- 60. Netcetera | 60 Further "Reading" Continuous Delivery (Talk by Jez Humble, DevOps Day 2012) http://www.infoq.com/presentations/Continuous-Delivery Interview with Martin Fowler and Jez Humble on Continuous Delivery http://www.infoq.com/interviews/jez-humble-martin-fowler-cd Continuous Delivery (Blog by Jez Humble) http://continuousdelivery.com/
- 61. Netcetera | 61 [email protected] +41 44 247 70 70 Corsin Decurtins Contact @corsin