Continuous Integration/Deployment with Docker and Jenkins
11 likes4,588 views
The document presents a presentation by Francesco Bruni on continuous integration (CI) using Docker and Jenkins, emphasizing the importance of automating the integration, testing, and deployment processes to streamline software development. It outlines CI practices, benefits, and requirements, explaining how CI evolves into continuous delivery and offering practical steps for implementation. Additionally, it includes details on Docker usage and Jenkins functionalities, alongside configuration and testing considerations.
![Dockerfiles
# pull base image.
FROM java:8
# maintainer details
MAINTAINER Francesco Bruni "francesco.bruni@stasbranger.com"
# update packages and install maven
RUN
export DEBIAN_FRONTEND=noninteractive &&
sed -i 's/# (.*multiverse$)/1/g' /etc/apt/sources.list &&
apt-get update &&
apt-get -y upgrade &&
apt-get install -y vim wget curl git maven ssh nano python-pip
# attach volumes
VOLUME /volume/git
# create working directory and set some stuff up
RUN mkdir -p /local/git
WORKDIR /local/git
RUN mkdir -p /root/.ssh
ADD id_rsa /root/.ssh/id_rsa
RUN chmod 700 /root/.ssh/id_rsa
RUN echo "Host <ip>ntStrictHostKeyChecking non" >> /root/.ssh/config
COPY .m2 /root/.m2
COPY run.sh run.sh
RUN chmod a+x run.sh
# run this when container starts
CMD ["./run.sh"]](https://image.slidesharecdn.com/cdcd-160520064454/85/Continuous-Integration-Deployment-with-Docker-and-Jenkins-17-320.jpg)
![Merge/Test
configurations
import os
main_path = os.path.join("src",
"main","resources","application.properties")
test_path = os.path.join("src",
"test","resources","application.properties")
for element in (main_path, test_path,):
with open(element) as old_main:
data = map(lambda line: line.strip(), old_main.readlines())
data[6] = data[6].replace("192.168.1.75", "mysql")
data[6] = data[6].replace("<myIp>", "mysql")
with open(element, "wb") as new_main:
new_main.write(“n".join(data))
Connection().quick_connect("user", "password", “db_test", "<myIp>")
Connection().quick_connect("user", "password", "db", "mysql")](https://image.slidesharecdn.com/cdcd-160520064454/85/Continuous-Integration-Deployment-with-Docker-and-Jenkins-24-320.jpg)
Continuous Integration/Deployment with Docker and Jenkins
- 1. CI with Docker and Jenkins Francesco Bruni Club degli sviluppatori - Bari, May ‘19 @brunifrancesco
- 2. Who am I •MSc Telecommunication Engineer @ Poliba •Despite a Java background, I prefer Python whenever possible •I'm not a computer scientist
- 3. Continuous Integration •Keep integrating multiple times a day the central code repository; •Run as more tests as you can to ensure nothing broke down; •Be ready to replicate environment.
- 4. Why CI matters •Continuous testing/building for all teammates; •Let someone do trivial jobs.
- 5. CI advantage “Continuous Integration doesn’t get rid of bugs, but it does make them dramatically easier to find and remove” M. Fowler
- 6. CI requirements •Single source repo; •Automated building; •Self testing build; •Build fast; •Run tests in a production env clone; •Automate deployment.
- 7. CI cycle Develop Feature Integrate in the main code base Test/build/label Checkout code Correct detected iussues
- 9. CI cycle •Integrating code should be automated; •Multiple levels testing; •Devs must not checkout bugged code; •Devs must not deploy untested code;
- 10. CI drawbacks •Setting files; •Setting all the stuff up requires time and patience.
- 11. CI natural evolution: Continuous Delivery •If a commit builds (all tests succeed), deliver it; •Keep delivery time at minimum; •Multiple deliveries for multiple versions.
- 12. Know your tools •Test everything; •Keep your sources/setting files versioned; •Containerize everything; •Automate as much as you can.
- 13. Test everything •Test your sources, checking coverage; •Test for writing new features; •Test your application settings; •If you want to deliver software, check external requirements.
- 14. Smarter use of VCs •Keep code versioned: application settings/sources/dep indexes; •Keep env settings versioned: scripts and configuration files; •Enjoy version control well defined ‘flows’ :)
- 15. Be containerized •Develop solutions in well separated envs; •Ship software as main part of env; •Don’t play with global deps. •‘Restore’ basic envs to speed up env creation process.
- 16. What’s Docker •Wrap many services in many containers; •Create, play with and destroy containers; •Save and easily replicate isolated containers.
- 17. Dockerfiles # pull base image. FROM java:8 # maintainer details MAINTAINER Francesco Bruni "[email protected]" # update packages and install maven RUN export DEBIAN_FRONTEND=noninteractive && sed -i 's/# (.*multiverse$)/1/g' /etc/apt/sources.list && apt-get update && apt-get -y upgrade && apt-get install -y vim wget curl git maven ssh nano python-pip # attach volumes VOLUME /volume/git # create working directory and set some stuff up RUN mkdir -p /local/git WORKDIR /local/git RUN mkdir -p /root/.ssh ADD id_rsa /root/.ssh/id_rsa RUN chmod 700 /root/.ssh/id_rsa RUN echo "Host <ip>ntStrictHostKeyChecking non" >> /root/.ssh/config COPY .m2 /root/.m2 COPY run.sh run.sh RUN chmod a+x run.sh # run this when container starts CMD ["./run.sh"]
- 18. Automate procedures •Repetitive tasks should be performed by automated agents; •Once you teach the agent what to do, it’s up to him to fulfill the job.
- 19. What’s Jenkins •Automated procedures worker; •Run tasks, report final status, log activities; •Multiple plugin for multiple features; •Use it just for playing too :)
- 20. Deploy w/ Docker w/o Jenkins •Test code (or at least run a set of tests); •Push code to repo; •SSH in the server, enter in container; •Pull changes and restart required services. •Check if it works! :)
- 21. Practical session •Integrate new code into the main branch; •Test all the stuff; •Deploy it :)
- 22. What we’ll need •A GIT web hook; •Set private “credentials”; •A tool to test configuration settings; •A container to run the test suite; •A container to run the new application; •A pipeline.
- 23. GIT web hook • Bind an action (tipically run a script) to GIT events; • Notify Jenkins new code has been pushed over the ‘develop’ branch. #! /bin/bash read oldrev newrev _branch tail=$(git log -1 --pretty=format:'%h %cn: %s%b' $newrev) curl http://<ip>:8080/job/QW/build? token=vH3YKmrRR5KOT4aeAOAV&commitMessage= $tail&cause=JenkinsDeploy
- 24. Merge/Test configurations import os main_path = os.path.join("src", "main","resources","application.properties") test_path = os.path.join("src", "test","resources","application.properties") for element in (main_path, test_path,): with open(element) as old_main: data = map(lambda line: line.strip(), old_main.readlines()) data[6] = data[6].replace("192.168.1.75", "mysql") data[6] = data[6].replace("<myIp>", "mysql") with open(element, "wb") as new_main: new_main.write(“n".join(data)) Connection().quick_connect("user", "password", “db_test", "<myIp>") Connection().quick_connect("user", "password", "db", "mysql")
- 25. Create the container to run test suite Run tests and see if they fails :)
- 26. Deploy the new version • Create the new container and run the new version; • The new container should replace an existing one.
- 27. Going on production No way, do some manual testing and start a new CD build
- 28. All togheter node { stage 'Stage 1' sh 'docker run --rm --link mysqlC:mysql --name sms-checklink-container sms-checklink' stage 'Stage 2' sh 'docker rm -f sms-test-container || echo 'no container to delete'' sh 'docker run --rm --name sms-test-container --link mysqlC:mysql sms- test' stage 'Stage 3' sh 'docker run --rm --name sms-merge-container sms-merge' stage 'Stage 4' sh 'docker rm -f sms-staging-container || echo 'no container to delete'' sh 'docker run -d -p 8433:8080 --name sms-staging-container --link mysqlC:mysql sms-staging' }
- 29. Conclusions • Improve configuration settings handling; • Reduce testing time; • Bind Jenkins job to commit messages; • Improve git-flow integration; • Docker Compose :( • Deploy project on production via manual tests.
- 30. References Continuous Delivery Reliable Software Releases through Build, Test, and Deployment Automation by Jez Humble and David Farley Continuous Integration by Martin Fowler http://martinfowler.com/articles/continuousIntegration.html
- 31. References Docker for beginners https://scotch.io/tutorials/getting-started-with-docker Meet Jenkins https://wiki.jenkins-ci.org/display/JENKINS/Meet+Jenkins Docker Explained: Using Dockerfiles to Automate Building of Images https://www.digitalocean.com/community/tutorials/ docker-explained-using-dockerfiles-to-automate- building-of-images
- 32. Let’s see it @ work