Download as PDF, PPTX
![[2009-06-22 16:03:57] [harmes] site deployed (changes...)
Who? When? What?](https://image.slidesharecdn.com/allspawhammondvelocity2009-090623161942-phpapp01/75/10-Deploys-Per-Day-Dev-and-Ops-Cooperation-at-Flickr-32-2048.jpg)
![Feature flags
#php
if ($cfg['enable_feature_video']){
…
}
{* smarty *}
{if $cfg.enable_feature_beehive}
…
{/if}](https://image.slidesharecdn.com/allspawhammondvelocity2009-090623161942-phpapp01/75/10-Deploys-Per-Day-Dev-and-Ops-Cooperation-at-Flickr-40-2048.jpg)
Uploaded byJohn Allspaw
10+ Deploys Per Day: Dev and Ops Cooperation at Flickr
The document discusses the collaboration between development and operations (DevOps) at Flickr, emphasizing the importance of communication and shared responsibilities. It highlights practices such as automated infrastructure, shared version control, and healthy attitudes towards failure to foster a cooperative culture. The authors advocate for mutual respect, trust, and avoiding blame in order to enhance stability and facilitate successful deployments.
More Related Content
Similar to 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr
![Governance, Deployment & Methodologies for Agentic Automation [2/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day2-251112135038-a386476d-thumbnail.jpg?width=640&height=640&fit=bounds)
10+ Deploys Per Day: Dev and Ops Cooperation at Flickr
- 1. 10 deploys perday Dev & ops cooperation at Flickr John Allspaw & Paul Hammond Velocity 2009
- 2. 3 billion photos 40,000 photos per second http://flickr.com/photos/jimmyroq/415506736/
- 3.
- 4. “It’s not mymachines, it’s your code!”
- 5. “It’s not mycode, it’s your machines!”
- 6. Spock Scotty Little bit weird Pulls levers & turns knobs Sits closer to the boss Easily excited Thinks too hard Yells a lot in emergencies
- 8. Says “No” allthe time Afraid that new fangled things will break the site Fingerpointy
- 9. Ops stereotype Because the site breaks unexpectedly Because no one tells them anything Because They say “NO” all the time
- 10. http://www.flickr.com/photos/stewart/461099066/ Traditional thinking Dev’s job is to add new features Ops’ job is to keep the site stable and fast
- 11. Ops’ job isNOT to keep the site stable and fast
- 12. Ops’ job isto enable the business (this is dev’s job too)
- 13. The business requireschange
- 14. But change isthe root cause of most outages!
- 15. Discourage change inthe interests of stability or Allow change to happen as often as it needs to
- 16. Lowering risk ofchange through tools and culture
- 17.
- 18. Ops who thinklike devs Devs who think like ops
- 19.
- 20. You can alwaysthink more like them
- 21.
- 22. 1. Automated infrastructure If there is only one thing you do…
- 23. CFengine Chef BCfg2 FAI 1. Automated infrastructure If there is only one thing you do… System Imager Puppet Cobbler
- 24.
- 25. 2. Shared versioncontrol
- 26. Everyone knows whereto look http://www.flickr.com/photos/thunderchild5/1330744559/
- 27. 3. One stepbuild
- 29. 3. One stepbuild and deploy
- 32. [2009-06-22 16:03:57] [harmes]site deployed (changes...) Who? When? What?
- 34. Small frequent changes http://www.flickr.com/photos/mauren/2429240906/
- 35. 4. Feature flags (akabranching in code)
- 36. 1.0.1 1.0.2 1.0 1.1 1.2 1.1.1 Desktop software
- 37. r2301 r2302 r2306 Web software
- 38.
- 39. Everyone knows exactlywhere to look http://www.flickr.com/photos/thunderchild5/1330744559/
- 40. Feature flags #php if ($cfg['enable_feature_video']){ … } {* smarty *} {if $cfg.enable_feature_beehive} … {/if}
- 41.
- 42.
- 43.
- 44. Free contingency switches http://www.flickr.com/photos/flattop341/260207875/
- 45.
- 47.
- 48.
- 49. Adaptive feedback loops RU ok? App System Metrics maybe?
- 51. 6. IRC andIM robots
- 52. Dev, Ops, andRobots Having a conversation build deploy logs logs alerts monitors IRC search engine
- 53.
- 54. 1. Respect If thereis only one thing you do…
- 55. Don’t stereotype (notall developers are lazy) http://www.flickr.com/photos/aaronjacobs/64368770/
- 56. http://www.flickr.com/photos/chrisdag/2286198568/ Respect other people’s expertise, opinions and responsibilities
- 57.
- 58.
- 59. Developers: Talk toops about the impact of your code: • what metrics will change, and how? • what are the risks? • what are the signs that something is going wrong? • what are the contingencies? This means you need to work this out before talking to ops
- 60.
- 61. Ops needs totrust dev to involve them on feature discussions Dev needs to trust ops to discuss infrastructure changes Everyone needs to trust that everyone else is doing their best for the business http://www.flickr.com/photos/85128884@N00/2650981813/
- 62. http://www.flickr.com/photos/flattop341/224176602/ Shared runbooks & escalation plans
- 63.
- 64. http://www.flickr.com/photos/williamhook/3468484351/ Ops:Be transparent, give devs access to systems
- 65. 3. Healthy attitude about failure
- 66.
- 67. If you thinkyou can prevent failure then you aren’t developing your ability to respond http://www.flickr.com/photos/toms/2323779363/
- 68.
- 69.
- 70.
- 71.
- 72. Fingerpointyness problem!!! argggh! fixed. freaking out, blaming, whining, figuring it fixing things not talking, covering hiding. out finding fault ass hurt egos time
- 73. Being productive problem!!! argggh! fixed. figuring it fixing things feeling move out guilty on with life time
- 74. Developers: Remember thatsomeone else will probably get woken up when your code breaks http://www.flickr.com/photos/alex-s/353218851/
- 75. http://www.flickr.com/photos/allspaw/2819774755/ Ops:provide constructive feedback on current aches and pains
- 76. 1. Automated infrastructure 2.Shared version control 3. One step build and deploy 4. Feature flags 5. Shared metrics 6. IRC and IM robots 1. Respect 2. Trust 3. Healthy attitude about failure 4. Avoiding Blame
- 77. This is noteasy You could just carry on shouting at each other…
- 78.