20100425 Configuration Management With Puppet Lfnw
2 likes•1,959 views
- Puppet is an open source configuration management tool that allows systems to be declared and configured in code. - It provides a declarative language to describe system configuration and resources to manage packages, files, services, and other common configuration elements. - Puppet helps ensure all systems are consistently configured, allows scaling to manage many systems, and provides change management for system modifications.
![What?
• Puppet is open source and released under GPL
• Backed by Puppet Labs - great training and audits available
• Used by organizations such as Google, Marchex, Whitepages.com,
Stanford University, Speakeasy, Harvard, Fedora, SANS Institute, etc..
• “Puppet is a declarative language for expressing system configuration, a
client and server for distributing it, and a library for realizing the
configuration.” [1]](https://image.slidesharecdn.com/20100425configurationmanagementwithpuppetlfnw-12722616000824-phpapp01/85/20100425-Configuration-Management-With-Puppet-Lfnw-2-320.jpg)
![Certs
• Puppet communication is SSL encrypted XML-RPC
• Supports auto signing - use with caution
[client]$ sudo puppetd --test
[puppetmaster]$ sudo puppetca --sign client.foo.com](https://image.slidesharecdn.com/20100425configurationmanagementwithpuppetlfnw-12722616000824-phpapp01/85/20100425-Configuration-Management-With-Puppet-Lfnw-17-320.jpg)
![Written by Robert Long IV gencert.php
<?php
# don't assume we have a path
$gencert = "/usr/sbin/puppetca -g";
$tar = "/bin/tar";
$sudo = "/usr/bin/sudo";
# set some paths up.
$cabase = "/var/lib/puppet/ssl/";
$certdir = "$cabase/certs";
$private = "$cabase/private_keys";
# yeah, its reverse DNS, but don't assume its safe.
$host = escapeshellarg(gethostbyaddr($_SERVER['REMOTE_ADDR']));
# create the certs
exec("$sudo $gencert $host", $out, $ret);
$ret && error_log("Error creating cert for $host: $outn");
# tar up the three files we need to make the client work.
exec("$sudo $tar -c $certdir/$host.pem $private/$host.pem $certdir/ca.pem 2>/dev/
null", $certs, $ret);
$ret && error_log("Error tar'ing cert for $host: $certsn");
# most of this is useless, but it will give curl an idea of what to expect in terms
of content
header("Content-Description: File Transfer");
header('Content-disposition: attachment; filename='.$host.'tar');
header("Content-Type: application/octet-stream");
header("Content-Transfer-Encoding: binary");
# and the content goes here.
print join("n", $certs);
?>](https://image.slidesharecdn.com/20100425configurationmanagementwithpuppetlfnw-12722616000824-phpapp01/85/20100425-Configuration-Management-With-Puppet-Lfnw-24-320.jpg)
![Ordering - require
class vim {
package { "vim-enhanced": ensure => installed, }
file { "/etc/vimrc":
source => "puppet:///modules/vim/vimrc-$operatingsystem",
mode => "644",
require => Package["vim-enhanced"],
}
}](https://image.slidesharecdn.com/20100425configurationmanagementwithpuppetlfnw-12722616000824-phpapp01/85/20100425-Configuration-Management-With-Puppet-Lfnw-37-320.jpg)
![Ordering - notify
class bind {
package { "bind”: ensure => installed, }
file { "/etc/named.conf":
content => template("bind/named.conf.erb"),
mode => "644",
require => Package[“bind”],
notify => Service["named"],
}
service { “named”:
ensure => running,
enable => true,
require => [ Package[“bind”], File[“/etc/
named.conf”] ],
}
}](https://image.slidesharecdn.com/20100425configurationmanagementwithpuppetlfnw-12722616000824-phpapp01/85/20100425-Configuration-Management-With-Puppet-Lfnw-38-320.jpg)
![Inheritance - postfix
class postfix {
package { “postfix”: ensure => present }
file { “/etc/postfix/aliases”:
require => Package[“postfix”],
content => template(“postfix/aliases.erb”),
notify => Exec[“postalias”],
}
service { “postfix”:
ensure => running,
enable => true,
require => Package[“postfix”],
}
exec { “postalias”:
command => “/usr/sbin/postalias /etc/postfix/aliases”,
require => File[“/etc/postfix/aliases”],
refreshonly => true,
}
}](https://image.slidesharecdn.com/20100425configurationmanagementwithpuppetlfnw-12722616000824-phpapp01/85/20100425-Configuration-Management-With-Puppet-Lfnw-39-320.jpg)
![Inheritance - postfix
class postfix::perim inherits postfix {
File { “/etc/postfix/aliases”:
content => template(“postfix/perim-aliases.erb”),
}
file { “/etc/postfix/sender_regexp”:
require => Package[“postfix”],
notify => Service[“postfix”],
content => template(“postfix/sender_regexp.erb”),
}
}
class postfix::voicemail inherits postfix {
File { “/etc/postfix/aliases”:
content => template(“postfix/voicemail-aliases.erb”),
}
file { “/etc/postfix/network_table”:
require => Package[“postfix”],
notify => Service[“postfix”],
source => “puppet:///postfix/voicemail-network_table”,
}
}](https://image.slidesharecdn.com/20100425configurationmanagementwithpuppetlfnw-12722616000824-phpapp01/85/20100425-Configuration-Management-With-Puppet-Lfnw-40-320.jpg)
![defines
class postfix {
... all that stuff from before
define post_files() {
File {
require => Package[“postfix”],
notify => Service[“postfix”],
}
file {
“/etc/postfix/master.cf”:
source => “puppet:///postfix/$name/master.cf”;
“/etc/postfix/main.cf”:
source => “puppet:///postfix/$name/main.cf”;
}
}
}
class postfix::voicemail inherits postfix {
post_files {“voicemail”: }
}](https://image.slidesharecdn.com/20100425configurationmanagementwithpuppetlfnw-12722616000824-phpapp01/85/20100425-Configuration-Management-With-Puppet-Lfnw-41-320.jpg)
20100425 Configuration Management With Puppet Lfnw
- 1. Configuration Management with Puppet LinuxFest Northwest Garrett Honeycutt April 25th, 2010
- 2. What? • Puppet is open source and released under GPL • Backed by Puppet Labs - great training and audits available • Used by organizations such as Google, Marchex, Whitepages.com, Stanford University, Speakeasy, Harvard, Fedora, SANS Institute, etc.. • “Puppet is a declarative language for expressing system configuration, a client and server for distributing it, and a library for realizing the configuration.” [1]
- 3. OS’s • Linux (RHEL, CentOS, Debian, Ubuntu, Gentoo, SuSE, ...) • BSD (Free, Open) • Mac OS X • Solaris (2.6 - 10) • HP-UX • AIX • Windows scheduled for sometime in 2010
- 4. Packages • Source • Ruby Gems • Solaris - Blastwave • RPM - Fedora & EPEL • Debian / Ubuntu • SuSE • Gentoo • OpenBSD • OS X - MacPorts • ArchLinux • Mandriva
- 5. Why? • Configuration Management != SSH + for loops
- 6. Why? • Configuration Management != SSH + for loops • Abstraction layer lets you talk about WHAT not HOW package {“vim-enhanced”: ensure => installed, }
- 7. Why? • Configuration Management != SSH + for loops • Abstraction layer lets you talk about WHAT not HOW package {“vim-enhanced”: ensure => installed, } • Reduce entropy - why is mail27 so fragile?
- 8. Why? • Configuration Management != SSH + for loops • Abstraction layer lets you talk about WHAT not HOW package {“vim-enhanced”: ensure => installed, } • Reduce entropy - why is mail27 so fragile? • Ability to quickly scale number of machines
- 9. Why? • Configuration Management != SSH + for loops • Abstraction layer lets you talk about WHAT not HOW package {“vim-enhanced”: ensure => installed, } • Reduce entropy - why is mail27 so fragile? • Ability to quickly scale number of machines • Change management - How and when are system being modified?
- 10. Why? • No such thing as a One-Off
- 11. Why? • No such thing as a One-Off • it’s only temporary - HA!
- 12. Why? • No such thing as a One-Off • it’s only temporary - HA! • create replicas in different environments (Dev, QA, Staging)
- 13. Why? • No such thing as a One-Off • it’s only temporary - HA! • create replicas in different environments (Dev, QA, Staging) • disaster recovery
- 14. Client / Server • Clients pulls a catalog from the puppetmaster • puppet.domain.tld • default is every 30 minutes
- 15. Server • webrick by default • best practice is Passenger
- 16. Diagram
- 17. Certs • Puppet communication is SSL encrypted XML-RPC • Supports auto signing - use with caution [client]$ sudo puppetd --test [puppetmaster]$ sudo puppetca --sign client.foo.com
- 18. What? • Only need to describe parts of the system that you care about • So you can start in existing environments • not re-describe what a package already gets right
- 19. Idempotency • Everything is built to be idempotent, even exec’s • no effect if the state is already achieved • safe to run multiple times
- 20. Repeatable State • Start from a known base! • Cobbler
- 21. Repeatable State • Start from a known base! • Cobbler • Have a shared storage plan • Keep persistent data on the network
- 22. Repeatable State • Start from a known base! • Cobbler • Have a shared storage plan • Keep persistent data on the network • Software package repository • Satellite / Spacewalk • Define processes for changing packages
- 23. services --enabled puppet Kickstart %packages facter puppet ruby-rdoc %post # delete unneeded puppet cert find /var/lib/puppet/ssl/ -type f |grep localhost | xargs rm -f # setup puppet cert curl -k https://puppetca | tar xC / # run puppet /usr/sbin/puppetd -t
- 24. Written by Robert Long IV gencert.php <?php # don't assume we have a path $gencert = "/usr/sbin/puppetca -g"; $tar = "/bin/tar"; $sudo = "/usr/bin/sudo"; # set some paths up. $cabase = "/var/lib/puppet/ssl/"; $certdir = "$cabase/certs"; $private = "$cabase/private_keys"; # yeah, its reverse DNS, but don't assume its safe. $host = escapeshellarg(gethostbyaddr($_SERVER['REMOTE_ADDR'])); # create the certs exec("$sudo $gencert $host", $out, $ret); $ret && error_log("Error creating cert for $host: $outn"); # tar up the three files we need to make the client work. exec("$sudo $tar -c $certdir/$host.pem $private/$host.pem $certdir/ca.pem 2>/dev/ null", $certs, $ret); $ret && error_log("Error tar'ing cert for $host: $certsn"); # most of this is useless, but it will give curl an idea of what to expect in terms of content header("Content-Description: File Transfer"); header('Content-disposition: attachment; filename='.$host.'tar'); header("Content-Type: application/octet-stream"); header("Content-Transfer-Encoding: binary"); # and the content goes here. print join("n", $certs); ?>
- 25. Facter • key => value system for retrieving information from your OS architecture => x86_64 domain => garretthoneycutt.com facterversion => 1.5.4 fqdn => blink.garretthoneycutt.com hostname => blink id => gh ipaddress_eth0 => 172.17.2.38 kernel => Linux kernelversion => 2.6.27.12 lsbdistcodename => Cambridge lsbdistdescription => Fedora release 10 (Cambridge) lsbdistrelease => 10 macaddress => 00:00:00:c0:ff:ee memoryfree => 5.87 GB memorysize => 7.80 GB netmask => 255.255.255.128 network_eth0 => 172.17.2.0 operatingsystem => Fedora operatingsystemrelease => 10 physicalprocessorcount => 1 processorcount => 2 rubyversion => 1.8.6
- 26. Facter • Written in Ruby # some_fact.rb Facter.add("some_fact") do setcode do %x{/usr/local/bin/stuff}.chomp end end
- 27. You are a developer • Speakeasy had 6000+ lines of code in the first year • You need a VCS • and plan on how to use it • go talk to your Dev’s - they’ve probably figured this all out
- 28. Different environments • Puppet allows for the notion of different environments • It’s a hack • Run a different puppet server in each environment • Easy to run different puppetmasters off of different branches/tags of your code
- 29. File system layout • You want this in a VCS! puppet |_manifests |_ site.pp |_ sea0 |_ wdc4 |_modules |_ apache |_ ... |_ zenoss |_ files |_ manifests |_ init.pp |_ templates
- 30. Site manifest # Default file parameters File { ignore => ".svn", owner => "root", group => "root", mode => "644", } node default { include base } node ‘cobbler.foo.com’ inherits default { include cobbler }
- 31. Types - brief list • cron • exec • file • group • host • mailalias • mount • nagios • package • service • sshkey • user • yumrepo
- 32. My first module - motd class motd { file { “/etc/motd”: owner => “root”, group => “root”, mode => 644, source => “puppet:///modules/motd/generic_motd”, } }
- 33. Templates - motd • Uses ERB templating class motd { file { “/etc/motd”: owner => “root”, group => “root”, mode => 644, content => template(“motd/motd.erb”), } }
- 34. motd.erb Welcome to <%= fqdn %> My uptime is <%= uptime %>
- 35. More advanced - resolv.conf.erb search <%= dnssearchpath %> options ndots:2 timeout:3 <% nameservers.each do |nameserver| -%> nameserver <%= nameserver %> <% end -%>
- 36. Ordering • Puppet’s catalog is built using a DAG • Has built in devices for ordering • before / require • subscribe / notify
- 37. Ordering - require class vim { package { "vim-enhanced": ensure => installed, } file { "/etc/vimrc": source => "puppet:///modules/vim/vimrc-$operatingsystem", mode => "644", require => Package["vim-enhanced"], } }
- 38. Ordering - notify class bind { package { "bind”: ensure => installed, } file { "/etc/named.conf": content => template("bind/named.conf.erb"), mode => "644", require => Package[“bind”], notify => Service["named"], } service { “named”: ensure => running, enable => true, require => [ Package[“bind”], File[“/etc/ named.conf”] ], } }
- 39. Inheritance - postfix class postfix { package { “postfix”: ensure => present } file { “/etc/postfix/aliases”: require => Package[“postfix”], content => template(“postfix/aliases.erb”), notify => Exec[“postalias”], } service { “postfix”: ensure => running, enable => true, require => Package[“postfix”], } exec { “postalias”: command => “/usr/sbin/postalias /etc/postfix/aliases”, require => File[“/etc/postfix/aliases”], refreshonly => true, } }
- 40. Inheritance - postfix class postfix::perim inherits postfix { File { “/etc/postfix/aliases”: content => template(“postfix/perim-aliases.erb”), } file { “/etc/postfix/sender_regexp”: require => Package[“postfix”], notify => Service[“postfix”], content => template(“postfix/sender_regexp.erb”), } } class postfix::voicemail inherits postfix { File { “/etc/postfix/aliases”: content => template(“postfix/voicemail-aliases.erb”), } file { “/etc/postfix/network_table”: require => Package[“postfix”], notify => Service[“postfix”], source => “puppet:///postfix/voicemail-network_table”, } }
- 41. defines class postfix { ... all that stuff from before define post_files() { File { require => Package[“postfix”], notify => Service[“postfix”], } file { “/etc/postfix/master.cf”: source => “puppet:///postfix/$name/master.cf”; “/etc/postfix/main.cf”: source => “puppet:///postfix/$name/main.cf”; } } } class postfix::voicemail inherits postfix { post_files {“voicemail”: } }
- 42. Scalability • use a load balancer for the puppetmasters • splay the time that your clients connect • dont run puppetd, instead call puppet from cron (puppetd -onetime) • recursive file serving is SLOW
- 43. Q &A Garrett Honeycutt LinuxFest Northwest April 25th, 2010
- 44. References • http://projects.puppetlabs.com/projects/puppet/wiki/Big_Picture • Architecture diagram from slide 12 - http://reductivelabs.com/trac/puppet/ wiki/ParsingArchitecture