Effective Devops - Collaboration and Tools - Velocity Santa Clara 2015
4 likes1,600 views
The document outlines key concepts of effective DevOps, highlighting the importance of collaboration, tools, and cultural understanding in high-performing teams. It covers principles such as infrastructure automation, version control, and the significance of continuous testing and feedback in achieving agility and reliability. Additionally, it includes practical examples, team dynamics, and various tools used in the DevOps process to enhance efficiency and communication.
![Example Resource Type - service
A service that should be started:
service "httpd" do
supports :restart => :true
action [:enable, :start]
end
96](https://image.slidesharecdn.com/effectivedevopsvelocity-150530222411-lva1-app6891/85/Effective-Devops-Collaboration-and-Tools-Velocity-Santa-Clara-2015-96-320.jpg)
![.kitchen.yml suites
• target configurations
Example:
name:default
run_list:
- recipe[apache::default]
attributes:
113](https://image.slidesharecdn.com/effectivedevopsvelocity-150530222411-lva1-app6891/85/Effective-Devops-Collaboration-and-Tools-Velocity-Santa-Clara-2015-113-320.jpg)
Effective Devops - Collaboration and Tools - Velocity Santa Clara 2015
- 1. Effective Devops: Collaboration and Tools Jennifer Davis & Katherine Daniels 1
- 2. Jennifer Davis • Automation Engineer, Chef • Co-author of "Effective Devops" • DevOpsDays SV Organizer • Sparkly Devops Princess • CoffeeOps Organizer 2
- 3. Katherine Daniels • Web Operations Engineer, Etsy • Co-author of "Effective Devops" • DevopsDays NYC Organizer • Ladies Who Linux NYC Organizer • Ship Show Podcast Co-host 3
- 4. Communication • Jennifer Davis Twitter: @sigje Email: [email protected] • Katherine Daniels Twitter: @beerops Email: [email protected] #EffectiveDevops 4
- 5. Feedback • Constructive feedback • What did you find helpful? • What would you like to see more/less of? • Was there anything you found unclear? 5
- 6. Schedule • Introduction to teams, devops principles • 10:30-11am Morning Break • Visualization of work, Git, Infrastructure automation • 12:30-1:30pm Lunch 6
- 7. Schedule • Testing infrastucture automation and other changes • 3-3:30pm Afternoon Break • Measuring, monitoring, and wrap-up 7
- 8. Network Connectivity Network Name: Conference_Private Access Code: velocity2015 8
- 9. Expectations • Safe space to share experiences, learn from each other • Code of Conduct • Learn effective workflows for using and testing source control and configuration management 9
- 10. Team Introductions • Meet your team! • Identify your team's... • Speaker • Gatekeeper • Notetaker Time: 10 minutes 10
- 11. 11
- 12. What is Devops • Technical cultural weave that shapes how we work, and why 12
- 13. Folk Models • general popularly understood meaning particular to a socio-cultural grouping but which has not been formally defined or standardized. 13
- 14. Why Devops? 14
- 15. High Performing Devops Teams are more agile 30X more frequent deployments 8000X faster lead times than peers 2014 PuppetLabs State of DevOps Survey 15
- 16. High Performing Devops Teams are more reliable 2X change success rate 12X faster mean time to recovery (MTTR) 2014 PuppetLabs State of DevOps Survey 16
- 17. Five Pillars of Effective Devops • Collaboration • Hiring • Affinity • Tools • Scaling 17
- 18. Collaboration • Individuals Working Together 18
- 20. Affinity • From Individuals to Teams 20
- 21. Tools • Choosing and Using them 21
- 22. Scaling • Growing and Decreasing 22
- 25. The Devops Compact • shared mutual understanding • established boundaries 25
- 26. Stormtrooper Syndrome • Agency • Adaptability: Role adherence 26
- 28. A life becomes meaningful when one sees himself or herself as an actor within the context of a story. -- George Howard 28
- 29. Swift Trust • Skillful self disclosure • What you share, how you share it • Collective tasks vs personal agendas 29
- 30. Team • Common purpose • Defined beliefs • Empowered 30
- 31. Small vs Large teams • Large teams - roles may be highly segregated • Small teams - one person may be responsible for many roles 31
- 32. Cultivating Empathy • Collect stories • Listen • Circle back 32
- 33. Smarter Teams Build Better Value1 • Lots of Communication • Contribute equally to team's discussions • Theory of Mind • Increased diversity 1 Engel, David et al. 'Reading The Mind In The Eyes Or Reading Between The Lines? Theory Of Mind Predicts Collective Intelligence Equally Well Online And Face-To-Face'. PLoS ONE 9.12 (2014): e115212. Web. 26 May 2015. 33
- 34. Critical Habits for Teams • Code Review • Pairing 34
- 35. Code Review • Max 90 minutes in one setting 35
- 36. Pairing • Agile software development • 2 people work together on 1 workstation • Driver - writes code • Observer - reviews each line • Roles switch frequently 36
- 37. Types of Pairing • Expert-expert • Expert-novice • Novice-novice 37
- 38. Discussion: Team Intro • What are motivations? • What are current beliefs? • What are current skills? Gaps in skills? Time: 15 minutes 38
- 39. Share: Team Intro • Team Name: • Common Motivations? • Skills? Gaps? 39
- 40. Fluxx • Game about change. • Rules change as you play. • Goal changes frequently. 40
- 41. Fluxx • Starts out with 1 Rule - Draw 1, Play 1 • Find this card (white backing) • Place card in the middle of table 41
- 42. Fluxx • Shuffle deck • Deal 3 cards to each player 42
- 43. Fluxx • First player is the person to left of Dealer. • Start out following the Draw 1, Play 1 rule until overridden by new rules. 43
- 44. Fluxx 4 types of cards • Goal - pink • Keeper - green played in front of you • Action - blue used once and discarded • Rule - yellow 44
- 45. Fluxx - On your turn • Draw the number of cards currently required. • Play the number of cards currently required. • Discard down to hand limit (if any hand limit). • Discard down to keeper limit (if any keeper limit). 45
- 46. Fluxx • Winner is the first to meet the current Goal condition. Time: 15 minutes 46
- 48. Importance of Games • Build resilience • Build collaboration (or competition) • Roles change 48
- 49. Specific to Fluxx • Frustration of constant changing goals • Completed "work", lost value • Visualization of goals, everyone in alignment 49
- 50. Visualization of Work • Bug/issue queue • Kanban 50
- 51. Kanban • Start with what you do now • Agree to incremental, evolutionary change • Respect • Everyone is a leader 51
- 52. Kanban Practices • Visualize • Limit WIP • Manage flow • Make policies explicit • Implement feedback loops 52
- 53. Visualize • Intent • Alignment • Coherence 53
- 54. Limit WIP • Pull (don't push) 54
- 55. Manage Flow • Monitor/measure/report • Incremental change 55
- 56. Make Policies Explicit • Document processes • Group signoff 56
- 57. Implement Feedback Loops • Collaboration • Retrospectives 57
- 58. Intro JoeNGo 58
- 59. Application Deployment Planning Deming Cycle • Plan: Identify and Analyze problem • Do: Develop and Test potential solution • Check: Measuring effectiveness • Act: Implement solution 59
- 60. Application Deployment Planning • (L)inux • (A)pache • (M)ySQL • (P)HP, Perl, or Python 60
- 61. Group Discussion Time: 15 minutes 61
- 62. Devops Tools • Establish local development environment • Version control • Manual -> Automation -> Continuous • Artifacts • Infrastructure • Sandbox 62
- 63. Local Development Environment (LDE) • Consistent set of tools across the team • Ability to quickly onboard new engineers 63
- 64. Provisioned Node - LDE • AWS instance node • Chef DK • Test Kitchen • Ruby • ChefSpec, ServerSpec • Git 64
- 66. Configuration Management • Process of identifying, managing, monitoring, and auditing a product through its entire life including the processes, documentation, people, tools, software, and systems. 66
- 67. Version Control • Records changes to files or sets of files stored within the system • Enable revisions • Integrity checking • Collaboration 67
- 68. Artifact Repository • Secure • Trusted • Stable • Accessible • Versioned 68
- 69. Introduction to Git and Workflows 69
- 70. Isolated Development Environments • Not automatically updated • Manually pull upstream commits 70
- 71. Workspace Environment $ git init $ git add . adds all files and directories to version control 71
- 72. Commit $ git commit -m "message about commit" -a 72
- 73. All these changes are local! How do you collaborate? 73
- 74. Show all remotes $ git remote -v 74
- 75. Add remote server $ git remote add NAME URL 75
- 76. Update changes to remote directory $ git push REPONAME BRANCH 76
- 77. View the branches $ git branch -a 77
- 78. Creating the branches $ git checkout -b BRANCHNAME 78
- 79. Merging branches $ git merge BRANCH 79
- 80. git fetch • import commits, from remote to local git checkout master git log origin/master git merge origin/master 80
- 81. Different workflows • feature branch • gitflow • forking • centralized 81
- 82. Pull Requests • collaboration prior to integration • discussion • follow up commits • selfie gifs3 3 https://github.com/thieman/github-selfies 82
- 83. git pull git pull REMOTE 83
- 84. git pull --rebase • ensure linear history by preventing unnecessary merge commits 84
- 85. git push remote branch • transfer commits from a local repo to a remote repo. • counterpart to git fetch 85
- 86. 86
- 87. Assignment 1 Effective Devops http://bit.ly/1JVuLh4 Time: 15 minutes 87
- 88. 88
- 89. Infrastructure • Aggregate of applications, configurations, access control, data, compute nodes, network, storage, processes, and people. 89
- 90. Infrastructure Automation • Systems that reduce the burden on people to manage services and increase the quality, accuracy and precision of a service to the consumers of a service 90
- 91. Infrastructure Automation Tools • Chef • Puppet • Ansible • Salt • CFEngine 91
- 93. Resources • Ingredients of infrastructure • Basic building blocks 93
- 94. Resource Declaration RESOURCETYPE "RESOURCE_NAME" do PARAMETER PARAMETER_VALUE end 94
- 95. Example Resource Type - package A package to be installed: package "httpd" do action :install end 95
- 96. Example Resource Type - service A service that should be started: service "httpd" do supports :restart => :true action [:enable, :start] end 96
- 97. Resources A resource is a statement of policy that: • Describes the desired state for an element • Specifies a resource type---such as package, template, or service • Lists additional details (also known as parameters), as necessary • Are grouped into recipes 97
- 98. Recipes • Collection of ordered resources • Combination of ruby and Chef DSL 98
- 99. Cookbooks • Thematic • Collection of recipes and other supporting files 99
- 100. Roles • Abstraction describing function of system • Name • Description • Run list (ordered list of recipes and roles) 100
- 101. Run List • Ordered list of recipes and roles • Specific to a node 101
- 102. Nodes • Machine (virtual, physical, cloud server, or other device) that is managed by Chef 102
- 103. Environments • Abstraction models workflow • Name • Description • Cookbook version pinning 103
- 104. Supermarket • Community site with a number of cookbooks • Read before using in your environment 104
- 105. Chef DK • Chef development kit • Includes a number of utilities and software to facilitate cookbook creation • Free download off of the website 105
- 106. Berkshelf • Dependency management • Included with Chef DK 106
- 107. Test Kitchen • Included with Chef DK • Sandbox automation • Test harness 107
- 108. Test Kitchen • Execute code on one or more platforms • Driver plugins supporting various cloud and virtualization providers 108
- 109. .kitchen.yml • driver • provisioner • platforms • suites 109
- 110. .kitchen.yml driver • virtualization or cloud provider Example: vagrant, docker 110
- 111. .kitchen.yml provisioner • application to configure the node Example: chef_zero 111
- 112. .kitchen.yml platforms • target operating systems Example: centos-6.5 112
- 113. .kitchen.yml suites • target configurations Example: name:default run_list: - recipe[apache::default] attributes: 113
- 114. Kitchen commands (1/2) • kitchen init • kitchen list • kitchen create • kitchen converge 114
- 115. Kitchen commands (2/2) • kitchen verify • kitchen destroy • kitchen test 115
- 116. Docker • Images • Registries • Containers 116
- 117. 117
- 118. Assignment 2 Time: 20 minutes 118
- 119. 119
- 120. Managing Risk • Test • Small frequent releases 120
- 121. Linting • Ensure code adheres to styles and conventions • Weave expectations into development • Encourages collaboration 121
- 122. Testing • Documenting objectives and intent • Measuring "done" 122
- 123. Code Correctness • foodcritic • rubocop 123
- 125. Rubocop • Ruby linter • Ruby style guide • Included with ChefDK 125
- 126. Rubocop Example $ rubocop cookbooks/COOKBOOK1 cookbooks/COOKBOOK2 cookbooks/COOKBOOK4 126
- 127. Reading Rubocop Output Inspecting 8 files CWCWCCCC • . means that the file contains no issues • C means a issue with convention • W means a warning • E means an error • F means an fatal error 127
- 128. Disabling Rubocop cops Any configuration in .rubocop.yml is disabled. To disable string literals: StringLiterals: Enabled: false 128
- 129. Foodcritic • Chef linter • Chef style guide • Included with ChefDK 129
- 130. Foodcritic Example $ foodcritic cookbooks/setup 130
- 131. Reading Foodcritic Output FC008: Generated cookbook metadata needs updating: ./metadata.rb:2 131
- 132. ServerSpec • Tests to verify servers functionality • Resource types • Package, service, user, and many others • Integrates with Test Kitchen • http://serverspec.org 132
- 133. ServerSpec Generic Form describe "<subject>" do it "<description>" do expect(thing).to eq result end end 133
- 134. ServerSpec Potential Tests • Is the service running? • Is the port accessible? • Is the expected content being served? 134
- 135. ServerSpec Example describe 'apache' do it "is installed" do expect(package 'httpd').to be_installed end it "is running" do expect(service 'httpd').to be_running end end 135
- 136. Reading ServerSpec Output app::default httpd service is running Finished in 0.26429 seconds (files took 0.7166 seconds to load) 1 example, 0 failures 136
- 137. 137
- 138. Assignment 3 Time: 10 minutes 138
- 139. 139
- 140. Retrospective on Assignment 3 Time: 15 minutes 140
- 141. 141
- 142. Assignment 4 Time: 20 minutes 142
- 143. 143
- 145. Test, Monitor, or Diagnostic2 1. Where is it going to run? 2. When is it going to run? 3. How often will it run? 4. Who is going to consume the result? 5. What is the entity going to do with it? 2 Lam, Yvonne. 'Sysadvent: Day 5 - How To Talk About Monitors, Tests, And Diagnostics'. Sysadvent.blogspot.com. N.p., 2014. Web. 26 May 2015. 145
- 146. Measuring Impact and Value of Change 146
- 147. Impact of Change 147
- 148. Impact on Availability • Overall site/app availability • Individual service availability 148
- 149. Availability Monitoring • Uptime: • Pingdom, Monitis, Uptrends, etc • Vertical Line Technology: • Availability after deploys/changes 149
- 150. 150
- 151. Eventinator 151
- 152. 152
- 153. Service Availability • Nagios: Service-level monitoring and alerting • Nagios-herald: Alert context • OpsWeekly: Historical alert data 153
- 154. Nagios 154
- 155. define command { command_name check_mongodb_query command_line $USER1$/nagios-plugin-mongodb/check_mongodb.py -H $HOSTADDRESS$ -A $ARG1$ -P $ARG2$ -W $ARG3$ -C $ARG4$ -q $ARG5$ } define service { use generic-service hostgroup_name Mongo Servers service_description Mongo Connect Check check_command check_mongodb!connect!27017!2!4 } 155
- 156. define servicedependency{ host_name WWW1 service_description Apache Web Server dependent_host_name WWW1 dependent_service_description Main Web Site execution_failure_criteria n notification_failure_criteria w,u,c } 156
- 157. Nagios-herald 157
- 158. 158
- 159. OpsWeekly 159
- 160. 160
- 161. 161
- 162. 162
- 163. 163
- 164. Impact on Quality • Service quality (SLAs) • Visibility of quality 164
- 165. Statsd 165
- 166. >>> import statsd >>> >>> timer = statsd.Timer('MyApplication') >>> >>> timer.start() >>> # do something here >>> timer.stop('SomeTimer') 166
- 167. >>> import statsd >>> >>> counter = statsd.Counter('MyApplication') >>> # do something here >>> counter += 1 167
- 168. >>> import statsd >>> >>> average = statsd.Average('MyApplication', connection) >>> # do something here >>> average.send('SomeName', 'somekey:%d'.format(value)) 168
- 169. Graphite 169
- 170. Value of Change 170
- 171. Value of Availability • Better for customers • Better for employees (internal services) • Fewer pages 171
- 172. Value of Quality • Deploys take less time • Also better for customers • More visibility into issues 172
- 173. 173
- 174. Assignment 5 Time: 20 minutes 174
- 175. 175
- 176. Retrospective ! 176
- 177. Review • Recognizing your Devops Narrative • Application Deployment Planning • Infrastructure as code • Introducing repeatable, testable change • Measuring impact and value of change 177
- 178. Next Steps • Manual, Automation to Continuous "X" • Be the storylistener and storyteller in your org • Effective Devops available in Early Release 178
- 179. Slides 179
- 181. 181