Setting up Puppet at Colruyt
0 likes1,097 views
This document provides an overview of Puppet implementation at Colruyt, a large Belgian retailer. It discusses why Colruyt chose Puppet for configuration management, how it set up Puppet including custom modules and wrappers, lessons learned, and plans for the future such as potentially switching to Hiera. The setup aims to make Puppet easier for teams to use while maintaining consistency and flexibility.
![Setup: params_systlinux_colruyt
• Sometimes server specific configs are needed
• Module params_systlinux_colruyt
– Defines several ‘global’ parameters
– Can be overridden in the node definition
– Using sensible defaults $topology = 'int',
class params_systlinux_colruyt (
)
– Syntax check
if ! ( $topology in [ 'int', 'dmz', 'shop' ] ) {
fail("topology is not 'int', 'dmz' or 'shop' (${topology})")
}
19](https://image.slidesharecdn.com/puppetcamptalk20130130-130130180510-phpapp02/85/Setting-up-Puppet-at-Colruyt-19-320.jpg)
Setting up Puppet at Colruyt
- 1. Puppet at Colruyt Puppet Camp 2013 [email protected]
- 2. Overview • Colruyt ? • Why Puppet ? • Setup • Lessons learned • Future 2
- 3. Colruyt • Largest retailer in Belgium – Several shop enseignes – Also food services 3
- 4. Colruyt • Also active in France and Luxembourg – Food service over full France territory – Shops 4
- 5. Colruyt Long history of automation 60’s today 5
- 6. Colruyt • Linux usage – 430+ Colruyt shops Colruyt, Okay, Dreamland, Dreambaby, Bioplanet, … – 50+ SPAR shops Migration to Linux busy – 150+ production servers Websites, Databases, Accounting, … 6
- 7. Why Puppet ? • Background in Linux Servers team – System engineers – Do shell programming but not programmers – Long history of ‘Not Build Here’ • Background in IT Colruyt – Specialized groups (DBA, Network, SAN, …) – Most don’t want to package and configure Some want to have some control – Co-managed servers 7
- 8. Why Puppet ? • Old system configuration management – Home build system – Worked well … but … • Takes lots of effort to main • There were important limitations • Not ready for the future • We’re on our own • We’re no experts 8
- 9. Why Puppet ? • Advantages Puppet – Noop (aka dry-run) option – Enterprise support – 3th party support – Large and active community – Flexible –… 9
- 10. Setup • Using Puppet Enterprise • One puppet master for each environment Controlled PE upgrades • Using Subversion as version control system • Other teams: Only package scripts/apps Linux Servers team: Also manages configurations 10
- 11. Setup • Light-weight wrapper CLI-menu – Easier for the teams – Lowers learning curve – Reduces risk for errors Editing package *asslib_infrascript_colruyt* within group *infrascript* 1) ** back 10) ** Status with subversion 2) ** Create file or dir 11) ** View diff with subversion 3) ** Edit file 12) ** Resolve subversion conflict 4) ** Revert file or dir 13) ** Commit to subversion 5) ** Rename file or dir 14) ** Package 6) ** Delete file or dir 15) ** Promote to rese 7) ** Customize spec file 16) ** Promote right away 8) ** View logs from subversion 17) ** Ask to adjust configs 9) ** Update from subversion Enter a number: 11
- 12. Setup • Managing resource definitions – Putting resource definitions of one type of resource in one module • Easier to fine out what is done on our server park • Using virtual resources • Realized with tags when needed 12
- 13. Setup • Managing resource definitions – Creating a wrapper around Puppet code • Easier for the team • Consistency in definitions • Lowers learning curve • Flexibility for the future 13
- 14. Example: cron • Main module: zz_cron_systlinux_colruyt • init.pp class zz_cron_systlinux_colruyt { include setup … @zz_cron_systlinux_colruyt::conf {'netbackup': user => 'root', command => '/opt/openv/netbackup/check_bp_conf.sh >/var/tmp/check_bp.conf.log', minute => '0', hour => '7', tag => 'zz_tag_netbackup_systlinux_colruyt', } } 14
- 15. Example: cron • setup.pp – Installs the needed rpms – Manages the service – Changes OS defaults to our likings 15
- 16. Example: cron • conf.pp – Defines the wrapper – Using sensible defaults define zz_cron_systlinux_colruyt::conf ( { $user, cron { "$user $command": $command, ensure => $ensure, $ensure = present, user => $user, $hour = undef, command => $command, $minute = undef, minute => $minute, $weekday = undef, hour => $hour, $month = undef, weekday => $weekday, $monthday = undef month => $month, ) monthday => $monthday, } } 16
- 17. Example: cron • Realization happens in a separate module – Groups all zz_* modules together – Makes life easier for the team define base_systlinux_colruyt::configure { … include zz_cron_systlinux_colruyt Zz_cron_systlinux_colruyt::Conf <| tag == "zz_tag_${name}" |> … } 17
- 18. Example: cron • In netbackup module – one line to add all zz_* modules class netbackup_systlinux_colruyt { base_systlinux_colruyt::configure {'netbackup_systlinux_colruyt':} … } 18
- 19. Setup: params_systlinux_colruyt • Sometimes server specific configs are needed • Module params_systlinux_colruyt – Defines several ‘global’ parameters – Can be overridden in the node definition – Using sensible defaults $topology = 'int', class params_systlinux_colruyt ( ) – Syntax check if ! ( $topology in [ 'int', 'dmz', 'shop' ] ) { fail("topology is not 'int', 'dmz' or 'shop' (${topology})") } 19
- 20. Setup: params_systlinux_colruyt • Used in node definitions: node 'svlirc99' { class {'params_systlinux_colruyt': topology => 'dmz', } … } • Used in puppet configurations – Templates – .pp files 20
- 21. Setup: base_systlinux_colruyt When requiring … do this: Minimal Red Hat setup and no puppet agent Create iso file with ‘none’ puppet config Reservation server with as little of us as possible Create node and only include class base_systlinux_colruyt::minimal Reservation server with our own packages and Create node and only include class configuration changes base_systlinux_colruyt::basic Server with Create node and include • our basic tools (reports, netbackup agent, ITO • base_systlinux_colruyt agent, sophos, …) Includes basic tools • other software (oracle, was, …) • Other required modules (oracle, was, …) 21
- 22. Lessons learned • Setup naming conventions <bla>_<group>_colruyt – Prevents naming clashes with others – Easier to parse • Humans • Scripts 22
- 23. Lessons learned • When packaging rpms, always start from version control system – Forces people to check in – Ensures you can always recreate rpms from scratch 23
- 24. Lessons learned • Make sure you can link a version on the server back to the version in the VCS – Include VCS revision number in rpm version number • 1.<revision>-1 • 1.<revision>-<ISO date> – Keep track of VCS revision number on Puppet master 24
- 25. Lessons learned • Keep loggings of all executions – Created wrapper puppet run script – Defaults to noop 25
- 26. Puppet annoyances • There are hidden ‘features’ with tags – When using ‘::’ in tags, things get complicated • abc::xyz – Creates tag abc::xyz – Creates tag abc – Creates tag xyz 26
- 27. Puppet annoyances – Every resource in a module gets the module name as tag – Also classes get class name as tag • Added ‘zz_tag_’ to tags 27
- 28. Puppet annoyances • Mount resource adds mount to fstab – Even if the resource fails – Working together with PuppetLabs 28
- 29. Puppet annoyances • Rpm updated outside of puppet ? – No trigger at the next puppet run – Needed • Upgrades done by consultants • Override permissions for Red Hat rpms 29
- 30. Future • Switch to Hiera ? – Removes our ‘house-build’ abstraction system ? – Speed up puppet runs ? • Use Gepetto for more IDE-type editing • Setup Puppet unit testing • Look into MCollective • … 30