Puppet Development Workflow
2 likes1,405 views
The document outlines a puppet development workflow presented by Jeff Smith from Grubhub, emphasizing local workstation setups, problem-solving strategies, and a structured approach to development. It details the usage of tools like Vagrant for local testing environments, best practices for manifest writing, and a comprehensive branching strategy for version control. The document also highlights the importance of automated testing and a gradual approach to code deployment in a production environment.
![config.vm.define "master" do |master|
master.vm.box = "centos65-base-small"
if not ENV['PUPPETREPO'].nil?
puppet_path = ENV['PUPPETREPO']
else
puppet_path = ENV['HOME']+'/Development/puppet'
end
config.vm.synced_folder puppet_path, "/manifests", type: "nfs"
master.vm.network "private_network", ip: "172.16.1.10"
master.vm.network :forwarded_port, host: 10443, guest: 443
config.vm.hostname = "pe.local.vm"
master.vm.provision :hosts
config.pe_build.version = '3.3.2'
config.pe_build.download_root = 'http://s3.amazonaws.com/specialbuildbucket/'
master.vm.provision :hosts do |provisioner|
provisioner.add_host '172.16.1.11', ['agent.local.vm', 'agent']
provisioner.add_host '172.16.1.12', ['agent-2.local.vm', 'agent-2']
end
master.vm.provision :pe_bootstrap do |provisioner|
provisioner.role = :master
#provisioner.relocate_manifests = true
end
master.vm.provision :shell, inline: "service iptables stop;chkconfig --level 3 iptables off"
master.vm.provision :shell, path: 'bootstrap.sh'
end](https://image.slidesharecdn.com/puppetworkflow-150521132741-lva1-app6891/85/Puppet-Development-Workflow-11-320.jpg)
Puppet Development Workflow
- 2. Bio - Jeff Smith • Manager, Site Reliability Engineering at Grubhub • Puppet User for about 2 years • Yes, we are also hiring. • Yes, there is free food. Yes, it's totally awesome to work here. Email: [email protected] Twitter: @DarkAndNerdy Blog: http://www.allthingsdork.com
- 3. Agenda • High level Environment Overview • Local workstation setup • How we solve problems • Branching strategy • Committing, Automated Testing and Publishing
- 5. Local Workstation Setup Editor Setup • VIM • tmux • Nerd Tree • Powerline
- 7. Local Testing Tools Listing tools can help you can syntax errors before you commit code! Local Linting Tools • jsonlint • puppet parser validate • erb syntax checker
- 8. VIM Linting Function Example function LintFile() let l:currentfile = expand('%:p') if &ft == 'puppet' let l:command = "puppet-lint " . l:currentfile elseif &ft == 'eruby.html' let l:command = "erb -P -x -T '-' " . l:currentfile . "| ruby -c" elseif &ft == 'json' let l:command = 'jsonlint -q ' . l:currentfile end silent !clear execute "!" . l:command . " " . bufname("%") endfunction map :call LintFile()
- 9. Guard Guard is a command line tool to easily handle events on file system modifications. guard-puppet - A configuration of Guard geared towards Puppet manifests and syntax. Find more at https://github.com/johnbintz/guard-puppet
- 10. Vagrant Environment It's easier to make changes locally then to a committed repository. Use Vagrant for test VMs • Puppet Master • 2 Agents (only 1 started by default) • Shares the puppet repo on the Host machine and uses that as the module path in the VM
- 11. config.vm.define "master" do |master| master.vm.box = "centos65-base-small" if not ENV['PUPPETREPO'].nil? puppet_path = ENV['PUPPETREPO'] else puppet_path = ENV['HOME']+'/Development/puppet' end config.vm.synced_folder puppet_path, "/manifests", type: "nfs" master.vm.network "private_network", ip: "172.16.1.10" master.vm.network :forwarded_port, host: 10443, guest: 443 config.vm.hostname = "pe.local.vm" master.vm.provision :hosts config.pe_build.version = '3.3.2' config.pe_build.download_root = 'http://s3.amazonaws.com/specialbuildbucket/' master.vm.provision :hosts do |provisioner| provisioner.add_host '172.16.1.11', ['agent.local.vm', 'agent'] provisioner.add_host '172.16.1.12', ['agent-2.local.vm', 'agent-2'] end master.vm.provision :pe_bootstrap do |provisioner| provisioner.role = :master #provisioner.relocate_manifests = true end master.vm.provision :shell, inline: "service iptables stop;chkconfig --level 3 iptables off" master.vm.provision :shell, path: 'bootstrap.sh' end
- 12. How We Approach Development Guiding Principles • YAGNI - You Ain't Gonna Need It • DRY - Don't Repeat Yourself • Modules, Profiles, Roles • Have respect for what you're doing • Write your manifests in Puppet
- 13. You Ain't Gonna Need It It's great to build your module to support CentOS, Ubuntu, Solaris, AIX. But if your shop only has CentOS, has only ever used CentOS and has plans to only use CentOS...........code for CentOS. • Adds too much time to development • Complicates a potentially simple solution • Not being tested in those environments so probably won't work anyways
- 14. Don't Repeat Yourself When you see the same piece of code showing up over and over again, there's an opportunity. • Limit the scope of where changes need to happen • Compose your modules in a way that makes them reusable • Hard-Coding is the devil
- 15. Modules, Profiles and Roles Compose your Puppet manifests into 3 separate categories • Modules - Contains the implementation details of a module. Responsible for the plumbing. • Profiles - Leverages modules to implement business logic. Takes your "Apache" module and turns it into "Payroll_Web" • Roles - Takes all of the profiles necessary to make a complete system. Only assign roles to a node and only 1 role per node
- 16. Have Respect for What You're Doing YOU ARE PROGRAMMING
- 17. Write Your Manifests in Puppet file { '/opt/app/config.ini': ensure => file, content => template('config.erb') } OR payroll_config { 'database_connection': property => 'dbserver', value => 'dbserver1.example.com:3306' }
- 18. Time to Write Some Code!
- 19. Puppet Forge is Your Friend • Someone has the same problem as you • More eyeballs on the same problem • More flexible • Less Work
- 20. Branching Strategy Largely will depend on your environment. We have a single Puppet repository for everything. (Don't do this) 3 Phases of Development Local => Preprod => Production
- 21. Local Development • Use Vagrant for test VMs • Make all changes prior to commit • Use linting • Test using your local Puppet Master VM and Agent • Set FACTS or variables via Custom Facts
- 22. Writing Test Cases Be highly selective about what you test in Puppet. Puppet Code file { '/opt/app/config.ini': ensure => file, content => template('app/config.erb') } RSpec Puppet Test it { should contain_file('/opt/app/config.ini) }
- 23. Puppet
- 24. Things To Test 1. File Syntax 2. Conditional Branches 3. Interpreted Values (e.g. RegEx evaluations, Facts) 4. Catalog Compilation
- 25. Getting Code to the Puppet Master • Work off a branch. Branch name should match your ticket • Push your branch to the remote origin server • Push your branch to the Pre-prod Puppet Master Remote Repo (optional) • Create a Pull Request from your branch, into the Develop Branch
- 27. Automated Test Execution CI Server watches master/develop branch. Executes tests on change 1. Script executes to determine which files have changed 2. Linting is performed on changed files (if applicable) 3. Catalog compilation of changed manifests 4. Execution of manifests specific tests (if applicable) 5. Deployment of code to the Puppet Masters
- 28. Auto Deployment
- 29. Building Confidence With enough confidence in the process you should be able to • Deploy your changes to production after a commit and successful automated testing • Regular Puppet runs on production systems • Iterate on changes faster Move slowly. Go piece by piece. Every small step you take adds value immediately. Bite off small bits.
- 30. Questions?