SlideShare a Scribd company logo

Using Chef for Automated Infrastructure in the Cloud

Download as KEY, PDF
Jesse Robbins
Jesse Robbins

The document discusses automating cloud infrastructure with Chef. It begins by explaining Chef's principles of being idempotent, having reasonable defaults, and supporting various programming styles. It then provides examples of using Chef to automate sudo permissions by writing a recipe to install packages, configure the sudoers file template, and create a role to manage sudo configuration. The overall document introduces Chef and demonstrates how to use it to automate systems configuration.

TechnologyBusinessTravel
1 of 170
Downloaded 1,168 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
Automating the Cloud with Chef Adam Jacob Co-Founder & CTO @ Opscode
• Who am I, who are you, and why are we here? • The Method • EC2 • Chef • Nanite • The Real World • Q &A http://www.flickr.com/photos/niecieden/367343737/sizes/o/
• 13 years as a Systems Administrator • Lots of Mergers and Acquisitions • Consultant • Wrote much of Chef • CTO at Opscode http://www.flickr.com/photos/anotherphotograph/2100904507/sizes/o/
http://www.flickr.com/photos/timyates/2854357446/sizes/l/
• Developers? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
• Developers? • Systems Administrators? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
Why are we here? http://www.flickr.com/photos/murplejane/1033445070/sizes/o/
Total Bootstrapping Time in Weeks 8 6 Corp Approvals Agile Approvals 4 Cloud 2 0 Best Time 0 Worst Time Why are we here? http://www.flickr.com/photos/murplejane/1033445070/sizes/o/
Total Bootstrapping Time in Weeks of ret g. 8 sec utin the mp tue s is Co r vir . hi ud 6 T o he here ot m Cl Corp Approvals ery s fro Ev mApprovals Agile 4 ste Cloud 2 0 Best Time 0 Worst Time Why are we here? http://www.flickr.com/photos/murplejane/1033445070/sizes/o/
For Developers...
For Developers... • Do it yourself.
For Developers... • Do it yourself. • The infrastructure is the application (and vice versa).
For Developers... • Do it yourself. • The infrastructure is the application (and vice versa). • You are not a Systems Administrator.
For Developers... • Do it yourself. • The infrastructure is the application (and vice versa). • You are not a Systems Administrator. • You need tools.
Sysadmins... http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
Sysadmins... • Say “Yes”. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
Sysadmins... • Say “Yes”. • You never liked rack and stack that much anyway. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
Sysadmins... • Say “Yes”. • You never liked rack and stack that much anyway. • You have never been more critical. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
Sysadmins... • Say “Yes”. • You never liked rack and stack that much anyway. • You have never been more critical. • Lean into it. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
The Method http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
The Method Bootstrapping http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
The Method Bootstrapping http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
The Method Bootstrapping Configuration http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
The Method Bootstrapping Configuration http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
The Method Bootstrapping Configuration Command & Control http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
The Method Bootstrapping Configuration Command & Control Nanite! http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
Lightning Strikes! DOOM Webservers Database Servers Webservers
Lightning Strikes! DOOM X Webservers XX Database Servers Webservers
Lightning Strikes! DOOM X 1 1 2 1 Signals Moar! Monitoring System Webservers 5 Updates 2 1 Command & Control XX 3 1 Bootstrapping Provisions 1 3 1 Database Servers 4 1 4 1 Configuration Webservers
EC2 Best Practices http://www.flickr.com/photos/46183897@N00/3442880227/sizes/l/
AMIs http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs • Amazon Machine Images http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs • Amazon Machine Images • Have one AMI with JEOS for each instance size http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs • Amazon Machine Images • Have one AMI with JEOS for each instance size • Use pre-existing images to bootstrap http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs • Amazon Machine Images • Have one AMI with JEOS for each instance size • Use pre-existing images to bootstrap • Include the Configuration and C&C Tools http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs • Re-bundle for specific roles http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs • Re-bundle for specific roles • Use SSH Keys for access http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
AMIs • Re-bundle for specific roles • Use SSH Keys for access • Shut off fsck! http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances • Actual virtual machines http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances • Actual virtual machines • c1.medium is the best bang for your $$ http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances • Actual virtual machines • c1.medium is the best bang for your $$ • Use User Data to inform Configuration http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances • Actual virtual machines • c1.medium is the best bang for your $$ • Use User Data to inform Configuration • Support indexed User Data - launch more than one at a time. (launch-index) http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances • Update the default security group to allow SSH http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances • Update the default security group to allow SSH • Use SSH Keys for authentication http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances • Update the default security group to allow SSH • Use SSH Keys for authentication • Ephemeral Storage is your buddy http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances • Internal and External hostnames are not unique http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Instances • Internal and External hostnames are not unique • Only trust the instance-id http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
EBS http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
EBS • Elastic Block Store http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
EBS • Elastic Block Store • Truly persistent storage http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
EBS • Elastic Block Store • Truly persistent storage • Trivial to snapshot http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
EBS • Elastic Block Store • Truly persistent storage • Trivial to snapshot • Snapshots can bootstrap more EBS devices http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Elastic IP Addresses http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Elastic IP Addresses • Static, publicly routable addresses http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
Elastic IP Addresses • Static, publicly routable addresses • Fast and easy to re-assign http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
http://www.brooklynstreetart.com/theBlog/wp-content/uploads/2008/12/swedish_chef_bork-sleeper-cell.jpg
At a High Level... http://www.flickr.com/photos/asten/2159525309/sizes/l/
At a High Level... • A library for configuration management http://www.flickr.com/photos/asten/2159525309/sizes/l/
At a High Level... • A library for configuration management • A configuration management system http://www.flickr.com/photos/asten/2159525309/sizes/l/
At a High Level... • A library for configuration management • A configuration management system • A systems integration platform http://www.flickr.com/photos/asten/2159525309/sizes/l/
At a High Level... • A library for configuration management • A configuration management system • A systems integration platform • An API for your entire Infrastructure http://www.flickr.com/photos/asten/2159525309/sizes/l/
http://www.flickr.com/photos/gi/518613153/sizes/o/
Principles • Idempotent http://www.flickr.com/photos/gi/518613153/sizes/o/
Principles • Idempotent • Reasonability http://www.flickr.com/photos/gi/518613153/sizes/o/
Principles • Idempotent • Reasonability • Sane defaults, easily changed http://www.flickr.com/photos/gi/518613153/sizes/o/
Principles • Idempotent • Reasonability • Sane defaults, easily changed • Hackability http://www.flickr.com/photos/gi/518613153/sizes/o/
Principles • Idempotent • Reasonability • Sane defaults, easily changed • Hackability • TMTOWTDI http://www.flickr.com/photos/gi/518613153/sizes/o/
Infrastructure as Code Manage configuration as idempotent Resources. Put them together in Recipes. Track it like source code. Configure your servers. You can learn more about Chef at http://wiki.opscode.com/display/chef/Home
Automating Sudo Permissions Write the recipe. http://xkcd.com/149/
Automating Sudo Permissions Write the recipe. • Install the package. http://xkcd.com/149/
Automating Sudo Permissions Write the recipe. • Install the package. • Write out the sudoers file. http://xkcd.com/149/
Automating Sudo Permissions Write the recipe. • Install the package. • Write out the sudoers file. • Use custom attributes for users and groups. http://xkcd.com/149/
Automating Sudo Permissions Write the sudoers template http://xkcd.com/149/
Automating Sudo Permissions Write the sudoers template • Add a warning banner. http://xkcd.com/149/
Automating Sudo Permissions Write the sudoers template • Add a warning banner. • Make sure root always has access. http://xkcd.com/149/
Automating Sudo Permissions Write the sudoers template • Add a warning banner. • Make sure root always has access. • Add the node-specific users. http://xkcd.com/149/
Automating Sudo Permissions Write the sudoers template • Add a warning banner. • Make sure root always has access. • Add the node-specific users. http://xkcd.com/149/ • Add the node-specific groups.
Automating Sudo Permissions Create a role that installs sudo, and sets defaults. http://xkcd.com/149/
Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. http://xkcd.com/149/
Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. http://xkcd.com/149/
Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. • Add the sudo recipe. http://xkcd.com/149/
Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. • Add the sudo recipe. • Add default users. http://xkcd.com/149/
Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. • Add the sudo recipe. • Add default users. http://xkcd.com/149/ • Add default groups.
Automating Sudo Permissions Put it all together!
Automating Sudo Permissions Put it all together! • Assign your role to nodes.
Automating Sudo Permissions Put it all together! • Assign your role to nodes. • When Chef runs, the sudoers file will be populated.
Nanite http://www.flickr.com/photos/etherhill/182345209/sizes/l/
Nanite • “Self Assembling Cluster of Ruby Daemons” http://www.flickr.com/photos/etherhill/182345209/sizes/l/
Nanite • “Self Assembling Cluster of Ruby Daemons” • AMQP - RabbitMQ http://www.flickr.com/photos/etherhill/182345209/sizes/l/
Nanite • “Self Assembling Cluster of Ruby Daemons” • AMQP - RabbitMQ • Actors register Services http://www.flickr.com/photos/etherhill/182345209/sizes/l/
Nanite • “Self Assembling Cluster of Ruby Daemons” • AMQP - RabbitMQ • Actors register Services • Distributed Map/Reduce for your Infrastructure http://www.flickr.com/photos/etherhill/182345209/sizes/l/
Nanite Architecture Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard - these slides pinched from http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
A Nanite Actor... Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
A Nanite Actor... • Advertises Services Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
A Nanite Actor... • Advertises Services • Advertises Tags Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
A Nanite Actor... • Advertises Services • Advertises Tags • Requests can route to Services and Tags Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
Lightning Strikes, Revisited! 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
Lightning Strikes, Monitoring /node/down Service Revisited! Signals Nanite 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
Lightning Strikes, Nanite boots new EC2 Nanite removes Revisited! Instances, with Chef Role + Attribute Data nodes in Chef 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
Lightning Strikes, Revisited! Provisions Instances, EBS, Elastic IPs 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
Lightning Strikes, Revisited! 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration Chef configures nodes according to assigned Roles
Lightning Strikes, Revisited! 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 Chef updates the 1 3 1 5 1 3 1 monitoring system 4 1 4 1 Configuration
In the Real World http://www.flickr.com/photos/kenlund/3376784956/sizes/l/
A Simple Architecture Load Balancing Web Servers Databases http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
A Simple Architecture AWS LB HAProxy S->M Load Balancing Web Servers Databases http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
AWS LB A Simple Architecture HAProxy S->M Load Balancing Your App Stack M Web Servers EBS? Databases http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
AWS LB A Simple Architecture HAProxy S->M Load Balancing Your App Stack M Web Servers EBS? Master/ Slave M->L->XL Databases EBS http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
Load Balancing http://www.flickr.com/photos/jannem/497840412/
Load Balancing • Amazon offers Load Balancing http://www.flickr.com/photos/jannem/497840412/
Load Balancing • Amazon offers Load Balancing • Alternative is HAProxy + Elastic IP http://www.flickr.com/photos/jannem/497840412/
Load Balancing (De)Provision Load Balancers • Amazon offers Load Balancing • Alternative is HAProxy + Elastic IP http://www.flickr.com/photos/jannem/497840412/
Load Balancing (De)Provision Load Balancers • Amazon offers Load Balancing Configure Pools, Associate Backends • Alternative is HAProxy + Elastic IP http://www.flickr.com/photos/jannem/497840412/
Load Balancing (De)Provision Load Balancers • Amazon offers Load Balancing Configure Pools, Associate Backends • Alternative is HAProxy + Elastic IP Reconfigure based on Load, STONITH http://www.flickr.com/photos/jannem/497840412/
Web Frameworks In General.. http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Web Frameworks In General.. • Try and keep things ephemeral http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Web Frameworks In General.. • Try and keep things ephemeral • File uploads belong in S3 or Cloudfront http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Web Frameworks In General.. • Try and keep things ephemeral • File uploads belong in S3 or Cloudfront • Use a shared session storage - preferably cookie or memcached based. http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Web Frameworks In General.. • Try and keep things ephemeral • File uploads belong in S3 or Cloudfront • Use a shared session storage - preferably cookie or memcached based. • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Web Frameworks In General.. • Try and keep things ephemeral (De)Provision Systems • File uploads belong in S3 or Cloudfront • Use a shared session storage - preferably cookie or memcached based. • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Web Frameworks In General.. • Try and keep things ephemeral (De)Provision Systems • File uploads belong in S3 or Cloudfront Configuration, Deployment, • Use a shared session Discovery storage - preferably cookie or memcached based. • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Web Frameworks In General.. • Try and keep things ephemeral (De)Provision Systems • File uploads belong in S3 or Cloudfront Configuration, Deployment, • Use a shared session Discovery storage - preferably cookie or memcached based. Trigger Deployment, Maintenance, Migrations • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Ruby on Rails http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
Ruby on Rails • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
Ruby on Rails • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
Ruby on Rails Deploys Radiant CMS • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
Ruby on Rails • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin Configures Apache http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
Catalyst
• Use Chef for CPAN Dependencies Catalyst
• Use Chef for CPAN Dependencies • Follow the Chef Deploy layout Catalyst
• Use Chef for CPAN Dependencies • Follow the Chef Deploy layout • Use Catalyst::Plugin::Session::Store::Memcached Catalyst
http://www.flickr.com/photos/hoerner_brett/2901426375/
• Use Chef for Egg Dependencies http://www.flickr.com/photos/hoerner_brett/2901426375/
• Use Chef for Egg Dependencies • Use Chef Deploy layout http://www.flickr.com/photos/hoerner_brett/2901426375/
• Use Chef for Egg Dependencies • Use Chef Deploy layout • Memcached Cache Backend http://www.flickr.com/photos/hoerner_brett/2901426375/
• Use Chef for Egg Dependencies • Use Chef Deploy layout • Memcached Cache Backend • Use the Cache Session Backend http://www.flickr.com/photos/hoerner_brett/2901426375/
Databases In General.. http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Databases In General.. • Keep everything on EBS http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Databases In General.. • Keep everything on EBS • Snapshot frequently http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Databases In General.. • Keep everything on EBS • Snapshot frequently • Keep a rolling backlog of snapshots http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Databases In General.. • Keep everything on EBS • Snapshot frequently • Keep a rolling backlog of snapshots • Register databases with Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Databases In General.. (De)Provision Systems, Build and Assign EBS • Keep everything on EBS • Snapshot frequently • Keep a rolling backlog of snapshots • Register databases with Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Databases In General.. (De)Provision Systems, Build and Assign EBS • Keep everything on EBS Configuration, Tuning, • Snapshot frequently User Credentials, • Keep a rolling backlog of Discovery snapshots • Register databases with Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
Databases In General.. (De)Provision Systems, Build and Assign EBS • Keep everything on EBS Configuration, Tuning, • Snapshot frequently User Credentials, • Keep a rolling backlog of Discovery snapshots Maintenance, Migrations, • Register databases with Analytics Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
MySQL http://www.flickr.com/photos/bike/2380021517/sizes/l/
MySQL • Always have a Slave http://www.flickr.com/photos/bike/2380021517/sizes/l/
MySQL • Always have a Slave • EBS is neat, but a crash is a crash - and recovery is time consuming http://www.flickr.com/photos/bike/2380021517/sizes/l/
MySQL • Always have a Slave • EBS is neat, but a crash is a crash - and recovery is time consuming • Building slaves is trivial http://www.flickr.com/photos/bike/2380021517/sizes/l/
PostgreSQL http://www.flickr.com/photos/jimgris/531515485/sizes/l/
PostgreSQL • Replication choices make things harder - PGPool-II appears to be most common on EC2 http://www.flickr.com/photos/jimgris/531515485/sizes/l/
PostgreSQL • Replication choices make things harder - PGPool-II appears to be most common on EC2 • Hot standby with WAL Shipping http://www.flickr.com/photos/jimgris/531515485/sizes/l/
PostgreSQL Are you great at this on EC2? • Replication choices make things harder - PGPool-II appears to be most common on EC2 • Hot standby with WAL Shipping http://www.flickr.com/photos/jimgris/531515485/sizes/l/
Monitoring & Trending In General.. http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
Monitoring & Trending In General.. • Must be integrated with Command & Control http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
Monitoring & Trending In General.. • Must be integrated with Command & Control • Use search features in Chef to dynamically configure http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
Monitoring & Trending In General.. • Must be integrated with Command & Control • Use search features in Chef to dynamically configure • Make sure you monitor internally and externally http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
Nagios http://www.flickr.com/photos/bike/2380021517/sizes/l/
Nagios • Do all that crazy config you have been avoiding http://www.flickr.com/photos/bike/2380021517/sizes/l/
Nagios • Do all that crazy config you have been avoiding • Remarkably easy to automate http://www.flickr.com/photos/bike/2380021517/sizes/l/
A word about Load...
Typical Peak Load 1. Bring on capacity as traffic ramps up 2. Take down capacity as it ramps down 3. 10-15 Minutes on either side, fully unattended Graphs in this portion of the presentation taken from Theo Schlossnagle http://omniti.com/seeds/dissecting-todays-internet-traffic-spikes
Atypical Load No way However, around you are Capacity still better Planning off! 1. Hope you know it is coming. 2. Increase capacity in advance. 3. Take down capacity as it ramps down. Graphs in this portion of the presentation taken from Theo Schlossnagle http://omniti.com/seeds/dissecting-todays-internet-traffic-spikes
Round Up! http://www.flickr.com/photos/benimoto/860423498/sizes/l/
Round Up! Bring new resources online http://www.flickr.com/photos/benimoto/860423498/sizes/l/
Round Up! Bring new resources online Configure and manage them http://www.flickr.com/photos/benimoto/860423498/sizes/l/
Round Up! Bring new resources online Configure and manage them Make the whole thing sing and dance http://www.flickr.com/photos/benimoto/860423498/sizes/l/
http://www.flickr.com/photos/jackol/1766679527/sizes/l/ Q &A Please rate this talk! There is lots more to learn Email: adam@opscode.com about Chef at Twitter: adamhjk http://wiki.opscode.com IRC: irc.freenode.net #chef

More Related Content

Viewers also liked (20)

PDF
Basics of reflection in java
kim.mens
 
PPT
IBM SmartCloud Orchestration
IBM Danmark
 
PPTX
Reflection in Java
Nikhil Bhardwaj
 
PPT
Reflection in java
upen.rockin
 
PPTX
Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014
Miguel Zuniga
 
PPTX
Financial Management with ServiceNow at Franke
Aspediens
 
PDF
System Center + Cireson vs. ServiceNow
Cireson
 
PDF
Docker Introduction
Robert Reiz
 
PDF
HGConcept-ITOM-Service Centric Operations
HGConcept Inc.
 
PDF
Openstack - Enterprise cloud management platform
Nagaraj Shenoy
 
PDF
Ansible - Introduction
Stephane Manciot
 
PDF
The Service-Oriented Enterprise with ServiceNow
Aspediens
 
PPTX
Zenoss & ServiceNow Integration - Incident Management & CMDB
Zenoss
 
PDF
Presentation cloud management platform
xKinAnx
 
PDF
Chef - Configuration Management for the Cloud
James Casey
 
PDF
RHTE2015_CloudForms_OpenStack
Jerome Marc
 
PDF
Cloudforms Workshop
Scalar Decisions
 
PPTX
Jenkins and Chef: Infrastructure CI and Automated Deployment
Dan Stine
 
PPTX
Chef for DevOps - an Introduction
Sanjeev Sharma
 
PPT
Ansible presentation
John Lynch
 
Basics of reflection in java
kim.mens
 
IBM SmartCloud Orchestration
IBM Danmark
 
Reflection in Java
Nikhil Bhardwaj
 
Reflection in java
upen.rockin
 
Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014
Miguel Zuniga
 
Financial Management with ServiceNow at Franke
Aspediens
 
System Center + Cireson vs. ServiceNow
Cireson
 
Docker Introduction
Robert Reiz
 
HGConcept-ITOM-Service Centric Operations
HGConcept Inc.
 
Openstack - Enterprise cloud management platform
Nagaraj Shenoy
 
Ansible - Introduction
Stephane Manciot
 
The Service-Oriented Enterprise with ServiceNow
Aspediens
 
Zenoss & ServiceNow Integration - Incident Management & CMDB
Zenoss
 
Presentation cloud management platform
xKinAnx
 
Chef - Configuration Management for the Cloud
James Casey
 
RHTE2015_CloudForms_OpenStack
Jerome Marc
 
Cloudforms Workshop
Scalar Decisions
 
Jenkins and Chef: Infrastructure CI and Automated Deployment
Dan Stine
 
Chef for DevOps - an Introduction
Sanjeev Sharma
 
Ansible presentation
John Lynch
 

Similar to Using Chef for Automated Infrastructure in the Cloud (20)

KEY
Infrastructure Automation with Chef
Adam Jacob
 
KEY
Continuous Integration, the minimum viable product
Julian Simpson
 
KEY
Userstories a practical intro
Marcus Hammarberg
 
PDF
10+ Deploys Per Day: Dev and Ops Cooperation at Flickr
John Allspaw
 
PDF
The Seven Wastes of Software Development
Matt Stine
 
KEY
TDD Boot Camp Sapporo 1.5
Shuji Watanabe
 
KEY
前端测试之淘宝实践
chencheng 云谦
 
PDF
【前端测试】淘宝前端测试实践
taobao.com
 
PDF
Design for Scale / Surge 2010
Christopher Brown
 
PDF
CloudStack usage service
ShapeBlue
 
PPTX
Sitecore on Azure
ClearPeople
 
PDF
[131] packetbeat과 elasticsearch
NAVER D2
 
PDF
Boris Stoyanov - some new features in Apache cloudStack
ShapeBlue
 
PDF
Deploying distributed software services to the cloud without breaking a sweat
Susan Potter
 
PDF
Distributed software services to the cloud without breaking a sweat
José Ferreiro
 
PPTX
Building Rackspace Cloud Monitoring
gdusbabek
 
KEY
Bitrzr - Ignite Portugal Tecnológico
bitrzr
 
PDF
Jabber Bot
Aizat Faiz
 
PDF
Permettere al cliente di apprezzare l'approccio agile
Steve Maraspin
 
KEY
Startupfest 2012 - Coefficients of friction
Startupfest
 
Infrastructure Automation with Chef
Adam Jacob
 
Continuous Integration, the minimum viable product
Julian Simpson
 
Userstories a practical intro
Marcus Hammarberg
 
10+ Deploys Per Day: Dev and Ops Cooperation at Flickr
John Allspaw
 
The Seven Wastes of Software Development
Matt Stine
 
TDD Boot Camp Sapporo 1.5
Shuji Watanabe
 
前端测试之淘宝实践
chencheng 云谦
 
【前端测试】淘宝前端测试实践
taobao.com
 
Design for Scale / Surge 2010
Christopher Brown
 
CloudStack usage service
ShapeBlue
 
Sitecore on Azure
ClearPeople
 
[131] packetbeat과 elasticsearch
NAVER D2
 
Boris Stoyanov - some new features in Apache cloudStack
ShapeBlue
 
Deploying distributed software services to the cloud without breaking a sweat
Susan Potter
 
Distributed software services to the cloud without breaking a sweat
José Ferreiro
 
Building Rackspace Cloud Monitoring
gdusbabek
 
Bitrzr - Ignite Portugal Tecnológico
bitrzr
 
Jabber Bot
Aizat Faiz
 
Permettere al cliente di apprezzare l'approccio agile
Steve Maraspin
 
Startupfest 2012 - Coefficients of friction
Startupfest
 
Ad

More from Jesse Robbins (15)

PDF
Jesse Robbins @ MWC 2015 - Building Orion Onyx - Real-time wearable push to t...
Jesse Robbins
 
PDF
Orion Labs - From Bits to Atoms
Jesse Robbins
 
PDF
Jesse Robbins Keynote - Hacking Culture @ Cloud Expo Europe 2013
Jesse Robbins
 
PDF
Continuous Deployment & Delivery + Culture Hacks @ QCON 2012
Jesse Robbins
 
PDF
Hacking Culture at VelocityConf
Jesse Robbins
 
PDF
Rebooting a Cloud
Jesse Robbins
 
PDF
GameDay: Creating Resiliency Through Destruction - LISA11
Jesse Robbins
 
PDF
DevOps @ InterOP Las Vegas - Jesse Robbins - Opscode
Jesse Robbins
 
PDF
Gov 2.0: Scaling, Automation, & Management in the Cloud
Jesse Robbins
 
PDF
Cloud Operations Bootcamp: Culture - Jesse Robbins
Jesse Robbins
 
PDF
Failure Happens Interop Nyc
Jesse Robbins
 
PDF
Serving Those That Serve Others Web2 Summit Jesse Robbins Final
Jesse Robbins
 
KEY
Failure Happens: CloudCamp Interop
Jesse Robbins
 
PDF
DisasterTech Presentation @ NEMA
Jesse Robbins
 
PDF
ETech2008 DisasterTech Robbins Maron 20080305a
Jesse Robbins
 
Jesse Robbins @ MWC 2015 - Building Orion Onyx - Real-time wearable push to t...
Jesse Robbins
 
Orion Labs - From Bits to Atoms
Jesse Robbins
 
Jesse Robbins Keynote - Hacking Culture @ Cloud Expo Europe 2013
Jesse Robbins
 
Continuous Deployment & Delivery + Culture Hacks @ QCON 2012
Jesse Robbins
 
Hacking Culture at VelocityConf
Jesse Robbins
 
Rebooting a Cloud
Jesse Robbins
 
GameDay: Creating Resiliency Through Destruction - LISA11
Jesse Robbins
 
DevOps @ InterOP Las Vegas - Jesse Robbins - Opscode
Jesse Robbins
 
Gov 2.0: Scaling, Automation, & Management in the Cloud
Jesse Robbins
 
Cloud Operations Bootcamp: Culture - Jesse Robbins
Jesse Robbins
 
Failure Happens Interop Nyc
Jesse Robbins
 
Serving Those That Serve Others Web2 Summit Jesse Robbins Final
Jesse Robbins
 
Failure Happens: CloudCamp Interop
Jesse Robbins
 
DisasterTech Presentation @ NEMA
Jesse Robbins
 
ETech2008 DisasterTech Robbins Maron 20080305a
Jesse Robbins
 
Ad

Recently uploaded (20)

PDF
AI Agents in the Cloud: The Rise of Agentic Cloud Architecture
Lilly Gracia
 
PDF
Agentic AI lifecycle for Enterprise Hyper-Automation
Debmalya Biswas
 
PPTX
Seamless Tech Experiences Showcasing Cross-Platform App Design.pptx
presentifyai
 
PDF
The 2025 InfraRed Report - Redpoint Ventures
Razin Mustafiz
 
PDF
Kit-Works Team Study_20250627_한달만에만든사내서비스키링(양다윗).pdf
Wonjun Hwang
 
PDF
CIFDAQ Market Wrap for the week of 4th July 2025
CIFDAQ
 
PDF
Future-Proof or Fall Behind? 10 Tech Trends You Can’t Afford to Ignore in 2025
DIGITALCONFEX
 
PPTX
New ThousandEyes Product Innovations: Cisco Live June 2025
ThousandEyes
 
PPTX
Q2 FY26 Tableau User Group Leader Quarterly Call
lward7
 
PDF
The Rise of AI and IoT in Mobile App Tech.pdf
IMG Global Infotech
 
PDF
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
PDF
What’s my job again? Slides from Mark Simos talk at 2025 Tampa BSides
Mark Simos
 
PDF
UPDF - AI PDF Editor & Converter Key Features
DealFuel
 
PDF
“Computer Vision at Sea: Automated Fish Tracking for Sustainable Fishing,” a ...
Edge AI and Vision Alliance
 
PDF
POV_ Why Enterprises Need to Find Value in ZERO.pdf
darshakparmar
 
PDF
Book industry state of the nation 2025 - Tech Forum 2025
BookNet Canada
 
PDF
[Newgen] NewgenONE Marvin Brochure 1.pdf
darshakparmar
 
PPTX
MuleSoft MCP Support (Model Context Protocol) and Use Case Demo
shyamraj55
 
DOCX
Cryptography Quiz: test your knowledge of this important security concept.
Rajni Bhardwaj Grover
 
PPTX
The Project Compass - GDG on Campus MSIT
dscmsitkol
 
AI Agents in the Cloud: The Rise of Agentic Cloud Architecture
Lilly Gracia
 
Agentic AI lifecycle for Enterprise Hyper-Automation
Debmalya Biswas
 
Seamless Tech Experiences Showcasing Cross-Platform App Design.pptx
presentifyai
 
The 2025 InfraRed Report - Redpoint Ventures
Razin Mustafiz
 
Kit-Works Team Study_20250627_한달만에만든사내서비스키링(양다윗).pdf
Wonjun Hwang
 
CIFDAQ Market Wrap for the week of 4th July 2025
CIFDAQ
 
Future-Proof or Fall Behind? 10 Tech Trends You Can’t Afford to Ignore in 2025
DIGITALCONFEX
 
New ThousandEyes Product Innovations: Cisco Live June 2025
ThousandEyes
 
Q2 FY26 Tableau User Group Leader Quarterly Call
lward7
 
The Rise of AI and IoT in Mobile App Tech.pdf
IMG Global Infotech
 
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
What’s my job again? Slides from Mark Simos talk at 2025 Tampa BSides
Mark Simos
 
UPDF - AI PDF Editor & Converter Key Features
DealFuel
 
“Computer Vision at Sea: Automated Fish Tracking for Sustainable Fishing,” a ...
Edge AI and Vision Alliance
 
POV_ Why Enterprises Need to Find Value in ZERO.pdf
darshakparmar
 
Book industry state of the nation 2025 - Tech Forum 2025
BookNet Canada
 
[Newgen] NewgenONE Marvin Brochure 1.pdf
darshakparmar
 
MuleSoft MCP Support (Model Context Protocol) and Use Case Demo
shyamraj55
 
Cryptography Quiz: test your knowledge of this important security concept.
Rajni Bhardwaj Grover
 
The Project Compass - GDG on Campus MSIT
dscmsitkol
 

Using Chef for Automated Infrastructure in the Cloud

  • 1. Automating the Cloud with Chef Adam Jacob Co-Founder & CTO @ Opscode
  • 2. • Who am I, who are you, and why are we here? • The Method • EC2 • Chef • Nanite • The Real World • Q &A http://www.flickr.com/photos/niecieden/367343737/sizes/o/
  • 3. • 13 years as a Systems Administrator • Lots of Mergers and Acquisitions • Consultant • Wrote much of Chef • CTO at Opscode http://www.flickr.com/photos/anotherphotograph/2100904507/sizes/o/
  • 5. • Developers? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • 6. • Developers? • Systems Administrators? http://www.flickr.com/photos/timyates/2854357446/sizes/l/
  • 7. Why are we here? http://www.flickr.com/photos/murplejane/1033445070/sizes/o/
  • 8. Total Bootstrapping Time in Weeks 8 6 Corp Approvals Agile Approvals 4 Cloud 2 0 Best Time 0 Worst Time Why are we here? http://www.flickr.com/photos/murplejane/1033445070/sizes/o/
  • 9. Total Bootstrapping Time in Weeks of ret g. 8 sec utin the mp tue s is Co r vir . hi ud 6 T o he here ot m Cl Corp Approvals ery s fro Ev mApprovals Agile 4 ste Cloud 2 0 Best Time 0 Worst Time Why are we here? http://www.flickr.com/photos/murplejane/1033445070/sizes/o/
  • 11. For Developers... • Do it yourself.
  • 12. For Developers... • Do it yourself. • The infrastructure is the application (and vice versa).
  • 13. For Developers... • Do it yourself. • The infrastructure is the application (and vice versa). • You are not a Systems Administrator.
  • 14. For Developers... • Do it yourself. • The infrastructure is the application (and vice versa). • You are not a Systems Administrator. • You need tools.
  • 15. Sysadmins... http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
  • 16. Sysadmins... • Say “Yes”. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
  • 17. Sysadmins... • Say “Yes”. • You never liked rack and stack that much anyway. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
  • 18. Sysadmins... • Say “Yes”. • You never liked rack and stack that much anyway. • You have never been more critical. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
  • 19. Sysadmins... • Say “Yes”. • You never liked rack and stack that much anyway. • You have never been more critical. • Lean into it. http://covers.oreilly.com/images/9780596007836/lrg.jpg Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft
  • 20. The Method http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 21. The Method Bootstrapping http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 22. The Method Bootstrapping http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 23. The Method Bootstrapping Configuration http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 24. The Method Bootstrapping Configuration http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 25. The Method Bootstrapping Configuration Command & Control http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 26. The Method Bootstrapping Configuration Command & Control Nanite! http://www.flickr.com/photos/wonderlane/2090966628/sizes/l/
  • 27. Lightning Strikes! DOOM Webservers Database Servers Webservers
  • 28. Lightning Strikes! DOOM X Webservers XX Database Servers Webservers
  • 29. Lightning Strikes! DOOM X 1 1 2 1 Signals Moar! Monitoring System Webservers 5 Updates 2 1 Command & Control XX 3 1 Bootstrapping Provisions 1 3 1 Database Servers 4 1 4 1 Configuration Webservers
  • 30. EC2 Best Practices http://www.flickr.com/photos/46183897@N00/3442880227/sizes/l/
  • 32. AMIs • Amazon Machine Images http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 33. AMIs • Amazon Machine Images • Have one AMI with JEOS for each instance size http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 34. AMIs • Amazon Machine Images • Have one AMI with JEOS for each instance size • Use pre-existing images to bootstrap http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 35. AMIs • Amazon Machine Images • Have one AMI with JEOS for each instance size • Use pre-existing images to bootstrap • Include the Configuration and C&C Tools http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 37. AMIs • Re-bundle for specific roles http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 38. AMIs • Re-bundle for specific roles • Use SSH Keys for access http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 39. AMIs • Re-bundle for specific roles • Use SSH Keys for access • Shut off fsck! http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 41. Instances • Actual virtual machines http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 42. Instances • Actual virtual machines • c1.medium is the best bang for your $$ http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 43. Instances • Actual virtual machines • c1.medium is the best bang for your $$ • Use User Data to inform Configuration http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 44. Instances • Actual virtual machines • c1.medium is the best bang for your $$ • Use User Data to inform Configuration • Support indexed User Data - launch more than one at a time. (launch-index) http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 46. Instances • Update the default security group to allow SSH http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 47. Instances • Update the default security group to allow SSH • Use SSH Keys for authentication http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 48. Instances • Update the default security group to allow SSH • Use SSH Keys for authentication • Ephemeral Storage is your buddy http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 50. Instances • Internal and External hostnames are not unique http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 51. Instances • Internal and External hostnames are not unique • Only trust the instance-id http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 53. EBS • Elastic Block Store http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 54. EBS • Elastic Block Store • Truly persistent storage http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 55. EBS • Elastic Block Store • Truly persistent storage • Trivial to snapshot http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 56. EBS • Elastic Block Store • Truly persistent storage • Trivial to snapshot • Snapshots can bootstrap more EBS devices http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 57. Elastic IP Addresses http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 58. Elastic IP Addresses • Static, publicly routable addresses http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 59. Elastic IP Addresses • Static, publicly routable addresses • Fast and easy to re-assign http://www.flickr.com/photos/wwworks/3271208324/sizes/l/
  • 61. At a High Level... http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 62. At a High Level... • A library for configuration management http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 63. At a High Level... • A library for configuration management • A configuration management system http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 64. At a High Level... • A library for configuration management • A configuration management system • A systems integration platform http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 65. At a High Level... • A library for configuration management • A configuration management system • A systems integration platform • An API for your entire Infrastructure http://www.flickr.com/photos/asten/2159525309/sizes/l/
  • 67. Principles • Idempotent http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 68. Principles • Idempotent • Reasonability http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 69. Principles • Idempotent • Reasonability • Sane defaults, easily changed http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 70. Principles • Idempotent • Reasonability • Sane defaults, easily changed • Hackability http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 71. Principles • Idempotent • Reasonability • Sane defaults, easily changed • Hackability • TMTOWTDI http://www.flickr.com/photos/gi/518613153/sizes/o/
  • 72. Infrastructure as Code Manage configuration as idempotent Resources. Put them together in Recipes. Track it like source code. Configure your servers. You can learn more about Chef at http://wiki.opscode.com/display/chef/Home
  • 73. Automating Sudo Permissions Write the recipe. http://xkcd.com/149/
  • 74. Automating Sudo Permissions Write the recipe. • Install the package. http://xkcd.com/149/
  • 75. Automating Sudo Permissions Write the recipe. • Install the package. • Write out the sudoers file. http://xkcd.com/149/
  • 76. Automating Sudo Permissions Write the recipe. • Install the package. • Write out the sudoers file. • Use custom attributes for users and groups. http://xkcd.com/149/
  • 77. Automating Sudo Permissions Write the sudoers template http://xkcd.com/149/
  • 78. Automating Sudo Permissions Write the sudoers template • Add a warning banner. http://xkcd.com/149/
  • 79. Automating Sudo Permissions Write the sudoers template • Add a warning banner. • Make sure root always has access. http://xkcd.com/149/
  • 80. Automating Sudo Permissions Write the sudoers template • Add a warning banner. • Make sure root always has access. • Add the node-specific users. http://xkcd.com/149/
  • 81. Automating Sudo Permissions Write the sudoers template • Add a warning banner. • Make sure root always has access. • Add the node-specific users. http://xkcd.com/149/ • Add the node-specific groups.
  • 82. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. http://xkcd.com/149/
  • 83. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. http://xkcd.com/149/
  • 84. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. http://xkcd.com/149/
  • 85. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. • Add the sudo recipe. http://xkcd.com/149/
  • 86. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. • Add the sudo recipe. • Add default users. http://xkcd.com/149/
  • 87. Automating Sudo Permissions Create a role that installs sudo, and sets defaults. • Give it a name. • A short description. • Add the sudo recipe. • Add default users. http://xkcd.com/149/ • Add default groups.
  • 88. Automating Sudo Permissions Put it all together!
  • 89. Automating Sudo Permissions Put it all together! • Assign your role to nodes.
  • 90. Automating Sudo Permissions Put it all together! • Assign your role to nodes. • When Chef runs, the sudoers file will be populated.
  • 92. Nanite • “Self Assembling Cluster of Ruby Daemons” http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 93. Nanite • “Self Assembling Cluster of Ruby Daemons” • AMQP - RabbitMQ http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 94. Nanite • “Self Assembling Cluster of Ruby Daemons” • AMQP - RabbitMQ • Actors register Services http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 95. Nanite • “Self Assembling Cluster of Ruby Daemons” • AMQP - RabbitMQ • Actors register Services • Distributed Map/Reduce for your Infrastructure http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 96. Nanite Architecture Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard - these slides pinched from http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 97. A Nanite Actor... Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 98. A Nanite Actor... • Advertises Services Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 99. A Nanite Actor... • Advertises Services • Advertises Tags Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 100. A Nanite Actor... • Advertises Services • Advertises Tags • Requests can route to Services and Tags Nanite brought to you by Ezra Zygmuntowicz @ Engine Yard http://www.slideshare.net/ezmobius/erlangfactory http://www.flickr.com/photos/etherhill/182345209/sizes/l/
  • 101. Lightning Strikes, Revisited! 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
  • 102. Lightning Strikes, Monitoring /node/down Service Revisited! Signals Nanite 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
  • 103. Lightning Strikes, Nanite boots new EC2 Nanite removes Revisited! Instances, with Chef Role + Attribute Data nodes in Chef 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
  • 104. Lightning Strikes, Revisited! Provisions Instances, EBS, Elastic IPs 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration
  • 105. Lightning Strikes, Revisited! 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 1 3 1 5 1 3 1 4 1 4 1 Configuration Chef configures nodes according to assigned Roles
  • 106. Lightning Strikes, Revisited! 1 1 2 1 Signals Moar! Monitoring Command & Bootstrapping System Updates Control Provisions 2 Chef updates the 1 3 1 5 1 3 1 monitoring system 4 1 4 1 Configuration
  • 107. In the Real World http://www.flickr.com/photos/kenlund/3376784956/sizes/l/
  • 108. A Simple Architecture Load Balancing Web Servers Databases http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
  • 109. A Simple Architecture AWS LB HAProxy S->M Load Balancing Web Servers Databases http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
  • 110. AWS LB A Simple Architecture HAProxy S->M Load Balancing Your App Stack M Web Servers EBS? Databases http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
  • 111. AWS LB A Simple Architecture HAProxy S->M Load Balancing Your App Stack M Web Servers EBS? Master/ Slave M->L->XL Databases EBS http://www.flickr.com/photos/susanneanette/2710667213/sizes/o/
  • 112. Load Balancing http://www.flickr.com/photos/jannem/497840412/
  • 113. Load Balancing • Amazon offers Load Balancing http://www.flickr.com/photos/jannem/497840412/
  • 114. Load Balancing • Amazon offers Load Balancing • Alternative is HAProxy + Elastic IP http://www.flickr.com/photos/jannem/497840412/
  • 115. Load Balancing (De)Provision Load Balancers • Amazon offers Load Balancing • Alternative is HAProxy + Elastic IP http://www.flickr.com/photos/jannem/497840412/
  • 116. Load Balancing (De)Provision Load Balancers • Amazon offers Load Balancing Configure Pools, Associate Backends • Alternative is HAProxy + Elastic IP http://www.flickr.com/photos/jannem/497840412/
  • 117. Load Balancing (De)Provision Load Balancers • Amazon offers Load Balancing Configure Pools, Associate Backends • Alternative is HAProxy + Elastic IP Reconfigure based on Load, STONITH http://www.flickr.com/photos/jannem/497840412/
  • 118. Web Frameworks In General.. http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 119. Web Frameworks In General.. • Try and keep things ephemeral http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 120. Web Frameworks In General.. • Try and keep things ephemeral • File uploads belong in S3 or Cloudfront http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 121. Web Frameworks In General.. • Try and keep things ephemeral • File uploads belong in S3 or Cloudfront • Use a shared session storage - preferably cookie or memcached based. http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 122. Web Frameworks In General.. • Try and keep things ephemeral • File uploads belong in S3 or Cloudfront • Use a shared session storage - preferably cookie or memcached based. • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 123. Web Frameworks In General.. • Try and keep things ephemeral (De)Provision Systems • File uploads belong in S3 or Cloudfront • Use a shared session storage - preferably cookie or memcached based. • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 124. Web Frameworks In General.. • Try and keep things ephemeral (De)Provision Systems • File uploads belong in S3 or Cloudfront Configuration, Deployment, • Use a shared session Discovery storage - preferably cookie or memcached based. • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 125. Web Frameworks In General.. • Try and keep things ephemeral (De)Provision Systems • File uploads belong in S3 or Cloudfront Configuration, Deployment, • Use a shared session Discovery storage - preferably cookie or memcached based. Trigger Deployment, Maintenance, Migrations • Chef Deploy http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 126. Ruby on Rails http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
  • 127. Ruby on Rails • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
  • 128. Ruby on Rails • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
  • 129. Ruby on Rails Deploys Radiant CMS • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
  • 130. Ruby on Rails • Install Gem Dependencies with Chef • Passenger, Mongrel, or Thin Configures Apache http://www.flickr.com/photos/davestfu/2157396025/sizes/l/
  • 132. • Use Chef for CPAN Dependencies Catalyst
  • 133. • Use Chef for CPAN Dependencies • Follow the Chef Deploy layout Catalyst
  • 134. • Use Chef for CPAN Dependencies • Follow the Chef Deploy layout • Use Catalyst::Plugin::Session::Store::Memcached Catalyst
  • 136. • Use Chef for Egg Dependencies http://www.flickr.com/photos/hoerner_brett/2901426375/
  • 137. • Use Chef for Egg Dependencies • Use Chef Deploy layout http://www.flickr.com/photos/hoerner_brett/2901426375/
  • 138. • Use Chef for Egg Dependencies • Use Chef Deploy layout • Memcached Cache Backend http://www.flickr.com/photos/hoerner_brett/2901426375/
  • 139. • Use Chef for Egg Dependencies • Use Chef Deploy layout • Memcached Cache Backend • Use the Cache Session Backend http://www.flickr.com/photos/hoerner_brett/2901426375/
  • 140. Databases In General.. http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 141. Databases In General.. • Keep everything on EBS http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 142. Databases In General.. • Keep everything on EBS • Snapshot frequently http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 143. Databases In General.. • Keep everything on EBS • Snapshot frequently • Keep a rolling backlog of snapshots http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 144. Databases In General.. • Keep everything on EBS • Snapshot frequently • Keep a rolling backlog of snapshots • Register databases with Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 145. Databases In General.. (De)Provision Systems, Build and Assign EBS • Keep everything on EBS • Snapshot frequently • Keep a rolling backlog of snapshots • Register databases with Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 146. Databases In General.. (De)Provision Systems, Build and Assign EBS • Keep everything on EBS Configuration, Tuning, • Snapshot frequently User Credentials, • Keep a rolling backlog of Discovery snapshots • Register databases with Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 147. Databases In General.. (De)Provision Systems, Build and Assign EBS • Keep everything on EBS Configuration, Tuning, • Snapshot frequently User Credentials, • Keep a rolling backlog of Discovery snapshots Maintenance, Migrations, • Register databases with Analytics Chef in ways that are easily discoverable http://www.flickr.com/photos/ecstaticist/289169665/sizes/o/
  • 148. MySQL http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 149. MySQL • Always have a Slave http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 150. MySQL • Always have a Slave • EBS is neat, but a crash is a crash - and recovery is time consuming http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 151. MySQL • Always have a Slave • EBS is neat, but a crash is a crash - and recovery is time consuming • Building slaves is trivial http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 152. PostgreSQL http://www.flickr.com/photos/jimgris/531515485/sizes/l/
  • 153. PostgreSQL • Replication choices make things harder - PGPool-II appears to be most common on EC2 http://www.flickr.com/photos/jimgris/531515485/sizes/l/
  • 154. PostgreSQL • Replication choices make things harder - PGPool-II appears to be most common on EC2 • Hot standby with WAL Shipping http://www.flickr.com/photos/jimgris/531515485/sizes/l/
  • 155. PostgreSQL Are you great at this on EC2? • Replication choices make things harder - PGPool-II appears to be most common on EC2 • Hot standby with WAL Shipping http://www.flickr.com/photos/jimgris/531515485/sizes/l/
  • 156. Monitoring & Trending In General.. http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
  • 157. Monitoring & Trending In General.. • Must be integrated with Command & Control http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
  • 158. Monitoring & Trending In General.. • Must be integrated with Command & Control • Use search features in Chef to dynamically configure http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
  • 159. Monitoring & Trending In General.. • Must be integrated with Command & Control • Use search features in Chef to dynamically configure • Make sure you monitor internally and externally http://www.flickr.com/photos/jiathwee/2870629436/sizes/l/
  • 160. Nagios http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 161. Nagios • Do all that crazy config you have been avoiding http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 162. Nagios • Do all that crazy config you have been avoiding • Remarkably easy to automate http://www.flickr.com/photos/bike/2380021517/sizes/l/
  • 163. A word about Load...
  • 164. Typical Peak Load 1. Bring on capacity as traffic ramps up 2. Take down capacity as it ramps down 3. 10-15 Minutes on either side, fully unattended Graphs in this portion of the presentation taken from Theo Schlossnagle http://omniti.com/seeds/dissecting-todays-internet-traffic-spikes
  • 165. Atypical Load No way However, around you are Capacity still better Planning off! 1. Hope you know it is coming. 2. Increase capacity in advance. 3. Take down capacity as it ramps down. Graphs in this portion of the presentation taken from Theo Schlossnagle http://omniti.com/seeds/dissecting-todays-internet-traffic-spikes
  • 166. Round Up! http://www.flickr.com/photos/benimoto/860423498/sizes/l/
  • 167. Round Up! Bring new resources online http://www.flickr.com/photos/benimoto/860423498/sizes/l/
  • 168. Round Up! Bring new resources online Configure and manage them http://www.flickr.com/photos/benimoto/860423498/sizes/l/
  • 169. Round Up! Bring new resources online Configure and manage them Make the whole thing sing and dance http://www.flickr.com/photos/benimoto/860423498/sizes/l/
  • 170. http://www.flickr.com/photos/jackol/1766679527/sizes/l/ Q &A Please rate this talk! There is lots more to learn Email: [email protected] about Chef at Twitter: adamhjk http://wiki.opscode.com IRC: irc.freenode.net #chef