DevOps Delivery Pipeline
- 1. DevOps Delivery Pipeline Software Delivery Automation for Beginners
- 2. What is DevOps? DevOps is a set of principles, practices, and technical solutions aimed to facilitate the Software Development Life Cycle by improving collaboration between Software Development and IT Operations teams. The ultimate goals of DevOps are to make the software more reliable, reduce time to market, and bring down the support costs. There are a lot of aspects of the DevOps approach, which are essential for understanding the whole DevOps concept and philosophy. This slide deck is focused on the very basic one: the automated Software Delivery Pipeline. The automation is a core DevOps principle, and a proper process of software delivery is a key success factor of the entire DevOps story. Let’s start with the fundamental concepts and definitions.
- 3. Key Software Delivery Concepts and Definitions Source Code Repository or Version Control System (VCS) A system where the Software Source Code is stored, and the code versions/branches are maintained Build and Build Artifact The Source Code processing/compilation/packaging and creation of a deployable software package named the Build Artifact Build Artifact Repository A system to store the Build Artifacts (compiled binaries) with version management and access control Deployment A list of activities to make a Software Application available and ready for use in a test or production environment using some Build Artifact Environment A consistent set of hardware/system components required to deploy and run Software Applications
- 4. What is the DevOps Delivery Pipeline? The DevOps or CI/CD (Continuous Integration/Continuous Delivery) Pipeline is a set of fully automated steps for delivery of a new version of a software product from a source code repository to the production environment. The automation is an essential principle here because it allows the delivery to be robust, fully controlled, and fast. There are tons of tools on the market helping to build a DevOps Delivery Pipeline, and a lot of publications in the Internet advertising those tools and their combinations. The right tools are important, but it could be difficult for a beginner to navigate through an ocean of possible options without a solid understanding of their underlying principles. The next slides are about the very common things to know regardless of a tool choice.
- 5. The very basic DevOps Delivery Pipeline That’s it! Someone can create a much more sophisticated pipeline with a lot of different steps, but usually, it is possible to attribute any step to either the Build or to the Deploy stage. To enjoy your DevOps pipeline and avoid some common mistakes, you have to follow simple rules, which we are going to review in this slide deck. Build Deploy Enjoy!
- 6. What are the common Build Steps? 1. Take source code from VCS 2. Verify the source code 3. Auto-generate extra code 4. Compile the code 5. Run unit tests 6. Add deployment scripts 7. Package the Build Artifact 8. Publish to Artifact Repository Some of the steps could be optional depending on the nature of the application. For example, sometimes there is no automated code generation or your application requires just some configuration and even no code compilation is needed. The must-have steps are highlighted because you always should start from the source code and provide a build artifact at the end, which is self-sufficient for deployment.
- 7. What are the common Deployment Steps? 1. Take the Build Artifact from the Artifact Repository 2. Take the deployment environment details 3. Run the deployment scripts on the deployment environment 4. Validate the deployment The first three steps are essential here, and the last one is optional but nice to have. The main idea of the deployment stage is to combine environment-agnostic Build Artifact with the environment-specific parameters such as hostnames/IPs, ports, folder paths etc. The deployment validation is nothing else but a set of smoke tests to ensure the application is up and running as expected.
- 8. How to enjoy? 1. Follow the best practices to automate your pipeline 2. Try to avoid common mistakes 3. Choose proper tools which allow doing the above 4. Document the details of your solution 5. Train the team to use it The Internet is full of DevOps materials: papers, blogs, tool reviews etc. Some fundamental things are described above, and on the next slides, the rest could be found once you have started building the DevOps pipeline for your specific application. It’s worth to remember that DevOps pipeline doesn’t exist in isolation: it has its users such as developers, testers, support team etc. Get their feedback and improve your solution. Enjoy!
- 9. DevOps pipeline automation Git with GitHub, GitLab, BitBucket etc. as VCS Git is a de-facto standard today. It can be accomplished by different web-based GUI/collaboration servers (some of them are listed here), cloud-based or on-premises Jenkins, Bamboo, TeamCity etc. as CI/CD server CI/CD server is a key automation component. It triggers Build/Deployment jobs on different events like VCS commit or another build job completion Nexus, Artifactory etc. as the Build Artifact Repository A typical CI/CD server normally has a basic built-in Artifact Repository, but you can add a dedicated Repository Server to have more features and control Ansible, Chef, Puppet etc. as a deployment tool A CI/CD server just triggers some deployment jobs, but the actual deployment is usually done by so-called configuration-management tools Some commonly used tools and concepts in the DevOps pipeline automation
- 10. Most common CI/CD Server features 1. Arranges build and deployment activities into plans/jobs/tasks etc 2. Monitors VCS for commits and triggers relevant builds 3. Provides a simple Build Artifact repo 4. Organizes deployment environments 5. Triggers deployments 6. Does jobs logging and monitoring A CI/CD server is a mandatory component of any DevOps pipeline. A CI/CD server provides a web-based GUI to configure and manage build and deployment jobs of a DevOps pipeline. Once configured, the build/deployment jobs are triggered by the server on different events like code commits or a build completion. In the following slides, we will consider the most common aspects of a CI/CD server usage, applicable to any of them.
- 11. CI/CD Server Usage Tip Why do we need to follow? Setup role-based access control to jobs Sometimes we want users to trigger jobs manually, and the only relevant people should do that. Use of the user roles simplifies access control management a lot Setup relevant notification groups A key point of DevOps success is an early notification about a problem. Create groups of developers and testers to be notified about a build or deployment failure Store job logic in version-controlled scripts Job logic is a code. VCS usage allows us to store job code in a central place with general benefits of code versioning like change tracking etc. Use automated job triggers with caution A build might take a while, and deployment might be unexpected. Job executions should not be unexpected and should be able to complete on time Never store environment details in VCS CI/CD server usually has built-in environment management, which allows storing sensitive environment-specific information (like passwords) securely The very basic tips and tricks of a CI/CD server usage
- 12. Build/Deploy jobs access control Object Build Job Deployment Job Source Code Repository (VCS) ✓ ✗ Build Artifact Repository Read-Write Read Only Environment Details ✗ ✓ You need to restrict access of your Build and Deployment jobs to DevOps infrastructure components in order to have immutable builds and reproducible deployments But what happens if we don’t follow the above principles? See on the next slide
- 13. What if we have... … then what will happen No jobs access control and notifications groups Someone may accidentally change or execute some critical job. Proper people will not be notified about a failed build Jobs configuration via CI/CD server GUI It is sometimes convenient for beginners, but you will not have a track of job changes and end up with CI/CD server vendor lock A lot of job triggers with complicated dependencies A complex trigger logic often became unmanageable, build runs might be queued, and deployment might be unexpected Deployment job access to VCS The deployments will not be reproducible, because the source code in VCS might be changed, and the next deployment will pick up a different version of the code Build job access to environment details Builds will be environment dependent and could be deployed only to a specific environment. A test and the prod deployments should use the same build Common mistakes to avoid
- 14. Thanks for watching! Feel free to contact me via LinkedIn