Intro to Chef

Introduzione a Chef

Giacomo Bagnoli

Develer Workshops - 12 Settembre 2012

@gbagnoli Introduzione a Chef 12/09/2012 1 / 50

# whoami

• System Administrator
• Operations Engineer
• Python guy (having an aﬀair with Ruby)


Outline

1. Introduction
2. Chef Overview
3. Example


Introduction


Infrastructure as code

Building and managing infrastructure programmatically




Enable the reconstruction of the business from:




• computing resources




• a source code repository




• a source code repository
• data backups


Infrastructure as code (2)

source: Chef wiki

Provisioning Get new computing resources
Conﬁguration Management Keeps track of all steps required to take bare
metal resources to doing their job
System Integration Takes all conﬁgured systems and make them work
together.

What is Chef


What is Chef

• A library/framework for conﬁguration management


What is Chef

• A conﬁguration management system


What is Chef

• A system integration platform


What is Chef

• An API for the infrastructure


What is Chef

• An API for the infrastructure
• Open Source! (Apache License, version 2.0)


Opscode Chef


Chef Principles


Chef Principles

Idempotent Describes states


Chef Principles

Extensible Uses ruby as the DSL


Chef Principles

Order It matters


Chef Principles

Order It matters
Client-Server Thick Clients, thin Server


Chef Principles

Order It matters

Various Flavours:


Chef Principles

Order It matters

Various Flavours:
• Hosted on Opscode platform (chef-hosted)


Chef Principles

Order It matters

Various Flavours:
• Open source Chef Server


Chef Principles

Order It matters

Various Flavours:
• Serverless - chef-solo


Chef Principles

Order It matters

Various Flavours:
• Serverless - chef-solo
• Private Chef (opscode-supported behind-the-ﬁrewall installation)


Chef API

• A RESTful service with JSON responses
• RSA key authentication with Signed Headers
• Search Service
• Derivative (easy to integrate with other tools)
• . . . i.e pychef :-)


Chef API

• A RESTful service with JSON responses
• RSA key authentication with Signed Headers
• Search Service
• Derivative (easy to integrate with other tools)
• . . . i.e pychef :-)

chef-client connects to the server consuming the API.
CLI management tool knife and the webUI use the API too.


Chef Architecture

Solr
CouchDB
Indexer

RabbitMQ

Chef API Server
Chef WebUI

Client Client Client

Knife Knife


API Client

In chef, an API client provides the identity used to authenticate
requests to the API server.


API Client


The public half of the public/private of a key pair is stored in the db on
the server, while the private part is local to the client.


API Client


The public half of the public/private of a key pair is stored in the db on
the server, while the private part is local to the client.

Each request to the API contains a request signature in the HTTP
headers.
The request signature is computed by the hash of the request content and
encrypted with the client private key, so it’s possible to verify the identity
of the user/machine making the request.


Nodes

A Node is a host that runs the chef-client.
• Has attributes
• Has a run list
• Has 0+ roles
• Belongs to an environment

In the common case, 1 host ⇔ 1 node ⇔ 1 client


run list

"run_list": {
"role[python_hosting]",
"recipe[postgresql::client]",
"recipe[chishop]"
}


Roles

• Have attributes
• Have a run list
• Declared in JSON or . . .
• Declared with the ruby DSL (automatically compiled to JSON)


Roles

• Have attributes
• Have a run list
• Declared in JSON or . . .
• Declared with the ruby DSL (automatically compiled to JSON)

If 1+ roles are in the node run list, the node run list is expanded


Roles (2)

An example role (in ruby):
name "python_hosting"
description "Python App hosting"
default_attributes(
"nginx" => {
"default_site_enabled" => false
}
)
run_list(
"recipe[python::virtualenv]",
"recipe[uwsgi]",
"recipe[nginx]"
)


Attributes
Store node data (i.e. ip address, hostname, fqdn, database host address,
etc.)
There are four types of attributes (in order of precedence, lowest to
highest):
• default
• normal
• override
• automatic


Attributes
Store node data (i.e. ip address, hostname, fqdn, database host address,
etc.)
There are four types of attributes (in order of precedence, lowest to
highest):
• default
• normal
• override
• automatic
Attributes can be set in:
• cookbooks
• environments
• roles
• nodes


Attributes (2)
So, in the end, as attributes are deep-merged, the following precedence
applies:


Attributes (2)
So, in the end, as attributes are deep-merged, the following precedence
applies:
• default attributes applied in an cookbook
• default attributes applied in an environment
• default attributes applied in a role
• default attributes applied on a node directly in a recipe
• normal attributes applied in a cookbook
• normal attributes applied on a node directly in a recipe
• override attributes applied in an cookbook
• override attributes applied in an environment
• override attributes applied in a role
• override attributes applied on a node directly in a recipe
• automatic attributes generated by Ohai


Attributes (3)

Automatic, ovverride and default are reset at the beginning of every run.
Normal attributes persist between runs.


Attributes (3)

Automatic, ovverride and default are reset at the beginning of every run.
Normal attributes persist between runs.

Attributes are searchable:

search(:node, ’platform:ubuntu’)

or

knife search node "platform:ubuntu"


Attributes (4)

Summary:


Attributes (4)

Summary:
• (sane) defaults in cookbooks


Attributes (4)

Summary:
• . . . overridden in roles


Attributes (4)

Summary:
• . . . and node-speciﬁc data as normal attributes on the node.


Attributes (4)

Summary:
• . . . and node-speciﬁc data as normal attributes on the node.
• override and node.set can be used to force values


Resources

Chef manages resources on a node.


Resources


Resources are speciﬁed in recipes, recipes stored in cookbooks.


Resources


Resources are speciﬁed in recipes, recipes stored in cookbooks.

The expanded run list speciﬁes all the recipes (and thus the resources) to
manage on a given node.


Resources (2)

A resource
• has a type package "tar" do
• has a name version "1.16.1-1"
action :install
• has parameters
end
• takes actions


Resources (2)

A resource
• has a type package "tar" do
• has a name version "1.16.1-1"
action :install
• has parameters
end
• takes actions

Actions are taken using providers, providers are chosen based on the node
platform.
(i.e. the package resource installs packages using apt on debian/ubuntu
and using yum on centos/RHEL)!


Recipes

Recipes evaluate resources in the order they appear
package "pdns-recursor" do
action :install
end

template "#{node[:pdns][:confd]}/recursor.conf" do
source "recursor.cfg.erb"
owner "root"
group "root"
mode 0644
notifies :restart, "service[pdns-recursor]"
end

service "pdns-recursor" do
action [:enable, :start]
end


Recipes (2)

Recipes can include other resources, and are just ruby code
include_recipe "apache2"

...

%w{config logs files}.each do |dir|
directory "#{node[:myrecipe][:base_dir]}/#{dir}" do
recursive true
owner "myuser"
group "mygroup"
mode 02775
end
end


Cookbooks (2)

Currently 133 cookbooks available on opscode-cookbooks GitHub org.
too many to list them all!

https://github.com/opscode-cookbooks

More cookbooks on the community site at
http://community.opscode.com/cookbooks


Metadata

maintainer "Opscode, Inc."
maintainer_email "cookbooks@opscode.com"
license "Apache 2.0"
description "Installs and configures mysql for client or server"
long_description IO.read(File.join(File.dirname(__FILE__), ’README.md’))
version "1.2.5"
recipe "mysql", "Includes the client recipe to configure a client"
recipe "mysql::client", "Installs packages required for mysql clients using run_action magic"
recipe "mysql::server", "Installs packages required for mysql servers w/o manual intervention"
recipe "mysql::server_ec2", "Performs EC2-specific mountpoint manipulation"

%w{ debian ubuntu centos suse fedora redhat scientific amazon }.each do |os|
supports os
end


Environments
Environments can be used to manage different environments (production,
test, etc) in a single Chef setup.
Roles can have different run list on different environments


Environments
name "production"
description "The production environment"
cookbook_versions(
"mysql" => "= 1.2.5", # use version 1.2.5 only
"apache2" => "~> 1.1" # anything 1.1.0 < x < 1.2.0
)
# default attributes for this environment
attributes(
"apache2" => {
"listen_ports" => ["80", "443"]
}
)


Environments
name "production"
description "The production environment"
cookbook_versions(
"mysql" => "= 1.2.5", # use version 1.2.5 only
"apache2" => "~> 1.1" # anything 1.1.0 < x < 1.2.0
)
# default attributes for this environment
attributes(
"apache2" => {
"listen_ports" => ["80", "443"]
}
)

As with role, the ruby DSL gets compiled to JSON when uploading to server


Environments (2)

$ knife environment list
production
$ knife environment show production -F json

{
"name": "production",
"description": "The production environment",
"cookbook_versions": {
"mysql": "= 1.2.5",
"apache2": "~> 1.1"
},
"json_class": "Chef::Environment",
"chef_type": "environment",
"default_attributes": {
"apache2": {
"listen_ports": [
"80",
"443"
]
}
},
"override_attributes": {
}
}


Environments (3)

Cookbooks can be frozen, so that following uploads with the same version
will fail.
$ # -E automatically sets a requirement for the specified environment
$ knife cookbook upload redis -- freeze -E production
Uploading redis ...
upload complete

$ knife cookbook show redis 1.0.2 | grep " frozen "
frozen ?: true

$ knife cookbook upload redis
Uploading redis ...
ERROR : Version 1.0.2 of cookbook redis is frozen . Use -- force to override .
ERROR : Failed to upload 1 cookbook .

$ knife environment show production | grep redis
users : = 1.0.2


Data Bags

Data bags provide an arbitrary store of globally available JSON data.


Data Bags

Data bags provide an arbitrary store of globally available JSON data.
Data bags can be encrypted (but then cannot be searched, except for id)


Data Bags (Encrypted)

(warn: fake data ahead)

$ knife data bag show accounts gbagnoli

comment : tenoh > dieliSh ’ i7eexeijeiSh ^ u9phaeGhuu4chaa *=
email : A h r 8 i s 3 a h C h o h m 6 a e n e i c ( aef " a h1e eree Voh hie 6Up =
group : Waix8Pa # iniy # oh6eem$eij =
groups : xa i7on g7a ihi u1n eH & ah3ier3Goh } rae7nik$einaeb =
id : gbagnoli
shadow : yi e@ja h0v e$g 2Ae Gh } ido6koobuew | aebeenaequeRo ( xaiYei8eizi + f
7 ohqu < i@enequ & oh7ef - ahdae8dia [ c h a h 7 e e 4 y i e $ N 4 E e B i c h e e 5 e i r o
h2JaGhae ^ k 6a e ph o hj ah s h6 Ae j a ^ cheew } o ) i 0 w o 5 i e s i s h 3 d i g h i e w o h
tohoh0eegho7eik =
shell : aeshi2ohy , ai6ai h2Ahquu =
ssh_keys : [.. cut ..]
uid : thahvo2IGhoh3osho8Ees /a=
username : poh5WiuZ2Er : it ! ee1ahf { u =


Data Bags (Decrypted)

$ knife data bag show accounts gbagnoli -- secret - file ~/. chef / enc_db_secret

comment : Giacomo Bagnoli
email : g . bagnoli@asidev . com
group : wheel
groups : [ asidev , users ]
id : gbagnoli
organization : asidev
shadow : $6$ [... cut ...]
shell : / bin / bash
ssh_keys : [" ssh - rsa [... cut ...] Giacomo Bagnoli "]
uid : 3000
username : g . bagnoli


Anatomy of a Chef Run


• chef-client starts


• Builds node (runs ohai, perform deep-merge of attrs)


• chef-client registers with the server


• Cookbook sync


• Cookbook sync
• Compiles resource collection, loading:


• Cookbook sync
• libraries


• Cookbook sync
• libraries
• resources / deﬁnitions


• Cookbook sync
• libraries
• attributes


• Cookbook sync
• libraries
• attributes
• recipes


• Cookbook sync
• libraries
• attributes
• recipes
• Executes - Conﬁgure Node


• Cookbook sync
• libraries
• attributes
• recipes
• Converge: each resource is mapped to a provider and which takes
action on it


• Cookbook sync
• libraries
• attributes
• recipes
action on it
• Saves Node


• Cookbook sync
• libraries
• attributes
• recipes
action on it
• Saves Node
• Runs notiﬁcation Handlers

On errors, exception handlers are run.

Chef development workﬂow



• Write cookbooks/recipe



• Upload the modiﬁed cookbook to the chef server



• Add the cookbook to a run list (in a node or in a role)



• Wait for chef-client to run on nodes



• Wait for chef-client to run on nodes
• Commit changes in git


Search
Full-text query engine based on Apache Solr.
Searches can be performed from knife and in recipes.
Almost any object is indexed by the chef server, like roles, nodes, api
clients and environments.

$ knife search node " recipes : apache2 "
7 items found
...

$ knife search node " recipes : apache2 AND chef_environment : production "
5 items found
...

$ knife search node " roles : lxc_guest "
9 items found
...

$ knife search client " admin : true "
4 items found
...

$ knife search role " name : lxc *"
2 items found
...


Bootstrapping

Bootstrapping is installing chef on new nodes . . . using chef.
First, create the node:
$ knife node create mynewnode . example . com
# .. fires up $EDITOR
# .. set run_list / attributes / etc / environment


Bootstrapping


Assuming that the new node is a bare ubuntu install, bootstrap the node
$ knife boostrap -N mynewnode . example . com -d ubuntu $NODE_IP -- sudo -V -x ubuntu


Bootstrapping


Assuming that the new node is a bare ubuntu install, bootstrap the node
$ knife boostrap -N mynewnode . example . com -d ubuntu $NODE_IP -- sudo -V -x ubuntu

Or, combine with provisioning (i.e. Amazon AWS)
knife ec2 server create -I ami - db595faf -- flavor t1 . micro -- region eu - west -1
-G default -x ubuntu -N newnode . example . com -d ubuntu -Z eu - west -1 a

Chef Omnibus bootstrap template


Monitoring: CheckMK and Chef


Monitoring
We use CheckMK to conﬁgure Icinga for monitoring our infrastructure.


Monitoring
CheckMK is a general purpose nagios plugin to retrive data from hosts.


Monitoring

1. One active check per host per check interval (calling check mk as a
plugin).


Monitoring

plugin).
2. The connection is done via TCP to the check mk agent on the target
host


Monitoring

plugin).
host (All host data is sent back at once as ASCII text.)


Monitoring

plugin).
3. check mk extracts performance data.


Monitoring

plugin).
3. check mk extracts performance data.
4. check mk checks warn/crit thresholds and submits results to Icinga as
passive checks.

Monitoring - Nodes

On nodes, the check mk::agent recipe, included in all nodes via the base
role, installs the check mk agent.


Monitoring - Nodes


It also sets up xinetd and the ﬁrewall so that connections to the agent
are allowed only from the monitoring host(s).


Monitoring - Nodes


It also sets up xinetd and the ﬁrewall so that connections to the agent
are allowed only from the monitoring host(s).

Monitoring hosts are speciﬁed as attributes in the base role.


Monitoring - Nodes (2)

(almost) Every cookbook pushes a MRPE or check mk plugin check to
the node, so the check mk agent returns data for all conﬁgured services.

i.e. , in the mysql:: server recipe
mrpe_check "mysql" do
script "check_mysql"
variables(
:passwd => node["mysql"]["server_root_password"],
:checks => checks,
:tunables => node["mysql"]["tunable"]
)
end


Monitoring - Nodes (2)

(almost) Every cookbook pushes a MRPE or check mk plugin check to
the node, so the check mk agent returns data for all conﬁgured services.

i.e. , in the mysql:: server recipe
mrpe_check "mysql" do
script "check_mysql"
variables(
:passwd => node["mysql"]["server_root_password"],
:checks => checks,
:tunables => node["mysql"]["tunable"]
)
end

> telnet mysql-server.example.com 6556
[...]
<<<mrpe>>>
(check_mysql) mysql_idx 0 OK - index usage 53.60% | index_usage=53.60%;0:;0:
(check_mysql) mysql_running 0 OK - 0 long running processes | long_running_procs=0;10;20
(check_mysql) mysql_threads 0 OK - 18 client connection threads | threads_connected=18;80;95
[...]


Monitoring - Server

On the server, the check mk:: server recipe installs and conﬁgures icinga,
nsca, nagvis, pnp4nagios smokeping and check mk.


Monitoring - Server


The check mk configuration file (which is used by check mk to configure
active and passive checks in icinga) is managed as a template by the
recipe. The recipe use the search API to discover nodes using a
configurable query.


Monitoring - Server


The check mk configuration file (which is used by check mk to configure
active and passive checks in icinga) is managed as a template by the
recipe. The recipe use the search API to discover nodes using a
configurable query.

nodes = search(:node, node["check_mk"]["search_query"])

default query:
default [”check mk”][”search query”] = ”chef environment:production”


Monitoring - Server (2)

So, when a node is promoted to production, as soon as chef runs on the
monitoring server,



monitoring server,
• It adds the node to the check mk conf ﬁle (the template is the same,
but data has changed)



monitoring server,
• . . . the modiﬁed templates notiﬁes the check mk write conf
resource, which is queued



monitoring server,
• . . . the modiﬁed templates notiﬁes the check mk write conf
resource, which is queued
template "#{node[’check_mk’][’conf_dir’]}/main.mk" do
source "check_mk_main.mk.erb"
variables(
:nodes => nodes,
:hostgroups => hostgroups,
:params => node[’check_mk’][’params’]
)
mode 0644
owner "root"
group "root"
notifies :run, "execute[check_mk_write_conf]"
end



The check mk write conf resource regenerates icinga conf



execute "check_mk_write_conf" do
command "#{node[’check_mk’][’prefix’]}/bin/check_mk -O"
action :nothing
end



execute "check_mk_write_conf" do
command "#{node[’check_mk’][’prefix’]}/bin/check_mk -O"
action :nothing
end

Then the recipe scan nodes for services (using check mk inventory)
This is done only the ﬁrst time (no reinventory)
nodes.each do |n|
if not n[’tags’] or not n[’tags’].include? "noagent"
check_mk_inventory n[’fqdn’]
end
end

check mk inventory is an LWRP deﬁned in the check mk cookbook



That way new nodes are automatically added to the monitoring when they
show up as result of the search query.

Since cookbooks push and conﬁgure check mk plugins, everything gets
monitored and
all the monitoring logic is in the cookbook itself.


Monitoring Chef Clients

Once you start relying on chef, you want to know if chef-client is
correctly running on nodes and if/when it fails.




Instead on relying on the active model of check mk, we use a completely
passive approach. We use NSCA for this.





When chef-client runs on the node, it pushes a report/exception
handler called NSCAHandler.
This handler runs at the end of a chef-client run, and submits the
result to the NSCA server on the monitoring host, so that:





When chef-client runs on the node, it pushes a report/exception
handler called NSCAHandler.
This handler runs at the end of a chef-client run, and submits the
result to the NSCA server on the monitoring host, so that:
• check is in critical state if chef run failed.
• check is in warning/critical state if time elapsed is above thresholds
• check is in warning/critical state if the number of modiﬁed resources
is above thresholds


Monitoring Chef Clients (2)

Passive checks are created on the server for every node which runs the
chef-client

These checks have a freshness threshold of 1 day, so that if no data
arrives from the node the check will enter the UNKOWN state, meaning the
chef-client is not running on that node.


Monitoring - Final Words
The check mk:: server recipe also configures:
• host parents
• host groups
• service groups
• notification periods for services
• contact groups and administrators (data is in the accounts data bag)
• uses smokeping to perfom hosts checks (instead of using check ping)
• . . . and integrates smokeping web ui with check mk multisite.
• host icon for the status map :-)

Behavior can be changed by setting nodes tag with knife. i.e, this changes
the notification period for a host.
knife tag create myhost.example.com workhours


Questions?
Bagnoli Giacomo

g.bagnoli@asidev.com
twitter.com/@gbagnoli
github.com/gbagnoli
bitbucket.org/gbagnoli
gplus.to/gbagnoli


Thank you!


Intro to Chef

More Related Content

What's hot (16)

Viewers also liked (12)

Similar to Intro to Chef (20)

Recently uploaded (20)

Intro to Chef