The DevOps paradigm - the evolution of IT professionals and opensource toolkit
2 likes198 views
This document discusses the DevOps paradigm and tools. It begins by defining DevOps as focusing on communication and cooperation between development and operations teams. It then discusses concepts like continuous integration, delivery and deployment. It provides examples of tools used in DevOps like Docker, Kubernetes, Ansible, and monitoring tools. It discusses how infrastructure has evolved to be defined through code. Finally, it discusses challenges of security in DevOps and how DevOps works aligns with open source principles like meritocracy, metrics, and continuous improvement.
The DevOps paradigm - the evolution of IT professionals and opensource toolkit
- 1. edited by Marco Ferrigno The DevOps paradigm the evolution of IT professionals and opensource toolkit Linux Day Napoli 2018 || #ldna18 IT System Engineer | Linux Specialist | Cyber Security Expert | DevOps Consultant
- 2. Step 0Step 0 DevOps as method slide keywords: - communication - cooperation
- 3. What else?What else? slide keyword: - continuous | | integration delivery
- 4. Perché sono cosi forti:Something else?Something else? Dev - Application Performance APM (Application Performance Management) tools to decrease latency and to have complete visibility into code - End User Analytics Understand end users feedback - Quality Code need to ensure their deployment and new release don't implode or degrade the overall performance - Code Level Errors when we have a large distributed application it is vital to lower MTTR(Mean Time To Repair) by finding the root cause of errors and exceptions Ops - Application Availability the application need to be up and running and it is Ops responsability to ensure uptime and SLA (service level agreement) are in order - Application Performance classic Ops generally rely on infrastructure metrics while modern Ops correlate all of those metrics with application metrics to solve problems 10x faster - End User Complaints the goal is to know and fix problems before end users complain, reduce the number of support tickets and eliminate false alerts - Performance Analytics automatically generated baselines of the metrics help Ops understand what has changed and where to focus their troubleshooting efforts
- 5. Perché sono cosi forti:The DevOps roadThe DevOps road Compiling apps from source (gcc, make ...) Bash scripts Hard user editor (vi ...) Commands/Tools Text Manipulation Process Monitoring System Performance Network Unix GNU/Linux OSI model. TCP/IP/UDP/Common ports Knowlege about different file system Setting up a reverse proxy (Nginx ...) Setting up a caching server (Squid, Nginx, Varnish.) Setting up a load balancer (HAProxy, Nginx ...) Setting up a firewall TLS, STARTTLS, SSL, HTTPS, SCP, SSH, SFTP Postmortem analysis Apache Nginx Operating System Love for terminal Web Server Caddy basic knowledge of infrastructures and protocols
- 6. Perché sono cosi forti:The DevOps roadThe DevOps road Docker rkt AWS Azure Kubernetes Mesosphere Mesos Docker Swarm Nomad Cloud Containers Rackspace Google Cloud Cloud Foundry Heroku Digital Ocean LXC Cluster Managers / Orchestrator
- 7. Perché sono cosi forti:The DevOps roadThe DevOps road Nagios / Icinga PagerDuty Jenkins Travis AWS Cloud Formation Terraform Ansible Puppet Chef CI/CD Monitoring & Alerting Drone Bamboo TeamCity Graphite Automation Prometheus Zabbix Salt Stack CF Engine
- 8. Perché sono cosi forti:The three Musketeers: CDThe three Musketeers: CD - Continuous Deployment A moment, wait! But continuous integration and continuous delivery? Continuous integration (CI) usually refers to integrating, building, and testing code within the development environment. The continuous integration pipeline should run on every commit or push. Unlike continuous delivery, continuous integration does not have a clearly defined goal of that pipeline. These steps: ✔ Pushing to the code repository ✔ Static analysis ✔ Pre-deployment testing ✔ Packaging and deployment to the test environment ✔ Post-deployment testing And continuous delivery? A CI in which we trust, without any check!
- 9. Perché sono cosi forti:The three Musketeers: MicroservicesThe three Musketeers: Microservices - Microservices Destroy the monolithic thought: "dividi et impera" The important thing to note is that due to the goals today’s competition sets in front of us (flexibility, speed, and so on), microservices are probably the best type of architecture we can apply
- 10. Perché sono cosi forti:The three Musketeers: ContainersThe three Musketeers: Containers - Containers The definition of the word container is “an object for holding or transporting something”. Most people associate containers with shipping containers. The idea behind “software” containers is similar. They are isolated and immutable images that provide designed functionality in most cases accessible only through their APIs. The way for containers to accomplish this feat is through self-sufficiency and immutability.
- 11. Perché sono cosi forti:The toolkits: an overviewThe toolkits: an overview Docker rkt Containers LXC Sys-admins’ vision - sandbox application processes on a shared Linux OS kernel - simpler, lighter and denser than virtual machines - portable across different environments Developers’ vision - package my application and all of its dependencies - deploy to any environment in second and enable CI/CD - easily access and share containerized components
- 12. Perché sono cosi forti:The toolkits: an overviewThe toolkits: an overview Kubernetes Mesosphere Mesos Docker Swarm Nomad Cluster Managers / Orchestrator “it’s “simply” an operating system for distributed environments” Focus on (here’s how both tools describe themselves): - Kubernetes Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. - Docker Swarm Docker Swarm is native clustering for Docker. It turns a pool of Docker hosts into a single, virtual host.
- 13. Perché sono cosi forti:The toolkits: an overviewThe toolkits: an overview Nagios / Icinga PagerDuty Jenkins Travis CI/CD Monitoring & Alerting Drone Bamboo TeamCity Graphite Prometheus Zabbix
- 14. Perché sono cosi forti:The toolkits: an overviewThe toolkits: an overview AWS Cloud Formation Terraform Ansible Puppet ChefAutomation Salt Stack CF Engine
- 15. Perché sono cosi forti:The evolution: Infrastructure as CodeThe evolution: Infrastructure as Code Step 0 - setup-webserver.sh ---------------------------------------- # Update the apt-get cache sudo apt-get update # Install PHP sudo apt-get install -y php # Install Apache sudo apt-get install -y apache2 # Copy the code from repository sudo git clone https://github.com/mycode/php-app.git /var/www/html/app # Start Apache sudo service apache2 start
- 16. Perché sono cosi forti:The evolution: Infrastructure as CodeThe evolution: Infrastructure as CodePerché sono cosi forti:The evolution: Infrastructure as CodeThe evolution: Infrastructure as Code Step 1: Ansible role - setup-webserver.yml ---------------------------------------- - name: Update the apt-get cache apt: update_cache: yes - name: Install PHP apt: name: php - name: Install Apache apt: name: apache2 - name: Copy the code from repository git: repo=https://github.com/mycode/php-app.git dest=/var/www/html/app - name: Start Apache service: name=apache2 state=started enabled=yes
- 17. Perché sono cosi forti:The evolution: Infrastructure as CodeThe evolution: Infrastructure as Code Server Templating Tools template A server templating tool like Packer can be used to create a self-contained image of a server. You can then use other tools, such as Ansible, to install that image across all of your servers.
- 18. Perché sono cosi forti:The evolution: Infrastructure as CodeThe evolution: Infrastructure as CodePerché sono cosi forti:The evolution: Infrastructure as CodeThe evolution: Infrastructure as Code Server Provisioning Tools template Server provisioning tools can be used with your cloud provider to create servers, databases, load balancers, and all other parts of your infrastructure. resource "aws_instance" "app" { instance_type = "t2.micro" availability_zone = "us-east-1a" ami = "ami-40d28157" user_data = <<-EOF #!/bin/bash sudo service apache2 start EOF }
- 19. Perché sono cosi forti:DevOps Kitchen: use case #1DevOps Kitchen: use case #1 GitHub Jenkins Docker Docker container is builded and start with shell script on server GitHub webhooks trigger Jenkins build Jenkins copies GitHub repo Jenkins build the Docker based on Image pushed on repo
- 20. Perché sono cosi forti:DevOps Kitchen: use case #2DevOps Kitchen: use case #2 Kubernetes Elasticsearch Hawkular Splunk
- 21. Perché sono cosi forti:Security DevOpsSecurity DevOps A great cultural challenge. ✔ Culture challenge #1: An entrenched view of security as something that happens “later” ✔ Culture challenge #2: An “us-versus-them” mindset ✔ Culture challenge #3: The belief that security hinders innovation
- 22. Perché sono cosi forti:DevOps’ work: opensource principlesDevOps’ work: opensource principles 1. Meritocracy Open source communities are level playing fields for contributors: Anyone can make changes to an open source project. If the idea is a good one, it can be accepted and included. Melding Dev and Ops resembles the de facto open organization that has existed in the open source world for decades. 2. Metrics Measurements provide a critique of a project, including data about how well something is working, in order to help developers who are considering contributing to a project. 3. Open governance At its heart, the open source model prescribes open governance, which implies transparent decision making. Otherwise, people who contribute have no real stake in the game, and the level playing field is eliminated. 4. Risk tolerance "Many eyes make all bugs shallow" is a well known open source mantra. Typically, open source communities are less risk prone due to the participation of many contributors. This provides optimal problem resolution and rapid integration of changes. 5. Continuous improvement A project doesn't have to be successful right out of the gate, but it must experience regular contributions with frequent inspections and adaptations.
- 23. Perché sono cosi forti:DevOps’ work: the leadershipDevOps’ work: the leadership 1. Developing transformational leadership skills Ready to change and take risks to help create the culture and environment necessary for DevOps to thrive. 2. Improving your ability to learn from mistakes When issues arise, his advice: Be willing to dig deep in figuring out what went wrong and what good things prevented slip-ups from getting worse. 3. Tackling culture - and middle managers Culture change, tearing down siloed organization structures, and managing technical debt top the list. Interestingly, with all the attention on transformational leaders at the top of organizations, DevOps teams are struggling with middle managers. 4. Growing your own talent When it comes to people, DevOps leaders should focus on growing talent from within rather than just looking elsewhere for DevOps skills. 5. Emphasizing the need for urgency 2018 was the year of enterprise DevOps, according to Forrester, with 50 percent of organizations implementing it now and another 27 percent planning to do so in the next 12 months. DevOps is not just the driver of digital transformation, it’s the fuel.
- 24. Perché sono cosi forti:Contact cardContact card Marco Ferrigno - @marco_ferrigno | [email protected] ✔ Independent researcher and consultant in computer security and systems engineering from birth; ✔ Since 1999, GNU/Linux' addicted; ✔ Since 2008, GNU/Linux evangelist as NaLUG member - Naples Linux Users Group; ✔ Since 2009, Official Debian GNU/Linux Contributor (after 10 years of use and experience); ✔ March 2011, proponent (along with a team of supporters) of the Osservatorio Partenopeo sull’OpenSource; ✔ Since August 2012, member of the ICTTF – International Cyber Threat Task Force; ✔ Since August 2014, IT Infrastructure Engineer and IT Security Manager of Programma il Futuro (Project of the Ministry of the Italian Republic for Education, University and Research and the National Inter-University Consortium for Computer Science); ✔ Since April 2015, Sailfish OS early adopter; ✔ Since September 2015, Digital Supporter in Digital Champions Association; ✔ Since March 2016 lent to the cloud computing as AWS SysOps Administrator; ✔ Since January 2017, IaC Engineer as vocation; ... and it's not over!