Monitoring and automation
0 likes49 views
This document discusses monitoring and automation from an operations and development perspective. It suggests measuring hardware usage, locating bottlenecks, anomalous behavior, and operational characteristics to optimize resources and understand costs. Standards like Prometheus are recommended for visualization. Automation is important for repeatability, consistency and reducing mistakes. Infrastructure as code with versioning enables testing deployments and lifecycles. Ultimately, the document argues that bringing operations and development together through cooperation, shared insights, and reconciling priorities around a shared mission is key to success.
Monitoring and automation
- 1. Monitoring and Automation From developer to devops
- 3. Quick questions ● Who here is doing operations? ● Who develops applications? ● Who develops infrastructure software? ● Who is doing all above? ● Who manages more than 10 servers? ● Who manages more than 100? ● 1000?
- 4. Where do we look?
- 5. Why do we measure? ● Optimize hardware usage ● Locate performance bottlenecks ● Identify anomalous behaviour ● Understand our own operational characteristics ○ How long do we normally take to add a new type of server? ○ What should we automate? ● Find out the costs of a given operation ○ If your operation takes 3 minutes of a server time, how much did we pay for it? ● Metering and billing ● Better understand the user ● Plan for future versions
- 6. Where do we look? CPU counters Revenue Profit Strategies Host OSGuest OSPlatformApplicationInternal processes
- 7. Where do we look? CPU counters Revenue Profit Strategies Host OSGuest OSPlatformApplicationInternal processes OperationslandDeveloperland
- 8. Where do we look? CPU counters Revenue Profit Strategies Host OSGuest OSPlatformApplicationInternal processes OperationslandDeveloperland DevOpsland (according to a very lax definition)
- 9. Where do we look? CPU counters Revenue Profit Strategies Host OSGuest OSPlatformApplicationInternal processes OperationslandDeveloperland DevOpsland Billing, HW usage Performance What to automate?
- 10. How do we look? ● Time resolution (and retention) ● GDPR (don't be evil, or keep what you don't need) ● Push vs. Pull ○ Register clients or let them register themselves ● Is UDP your friend? ○ Fast, cheap and unreliable ○ If you need security, you need to build it ○ You should probably use it in combination with TCP (a subsampled guaranteed channel) ● Standards ○ Prometheus is your friend ● Visualization ○ You'll need to figure it out
- 11. On Automation
- 12. How about measuring ourselves? ● Learn what to automate ○ And what not to ● For stateless clusters, deployment is more important ○ Don't fix stuff you can replace ● For stateful ones, you'll want to keep lifecycle ○ You'll hate when your database server that has a disk space problem gets a brand new bigger volume that also doesn't have your database ● Terraform/Chef/Puppet/Ansible/Salt/Whatever-works-for-you ○ All software sucks in one way or another ● Infrastructure as code ○ Versioned ○ Testable, tested routinely (deployment and life-cycle scenarios)
- 13. Why? ● Repeatability ○ High-fidelity replicas of the production environment ● Consistency ○ Unless you change something, the result should always be the same ■ Either that, or you have a more serious bug ● Less work == fewer mistakes ● Isn't it handy that the script you use to automate is under version control? ● One side can build upon the tools of the other - and vice-versa ○ And collaborate ■ And, ultimately, becoming a single team, even if you have different roles
- 14. So?
- 15. It all boils down to attitude It's about cooperation, about tearing down walls Offering and accepting insights from different people with different priorities Reconciling different priorities around a single mission A drive to understand your tooling, your systems (Random) failure is not an option
- 16. Questions?
- 17. Thank you @rbanffy - twitter, github,