Data-Driven DevOps: Improve Velocity and Quality of Software Delivery with Metrics and Feedback
6 likes6,675 views
The document discusses the importance of metrics and continuous feedback in DevOps to enhance software delivery velocity and quality. It highlights how data-driven approaches can provide transparent insights into development processes and enables stakeholders to identify and improve various performance indicators, including deployment speed and code quality. By leveraging real-time analytics, organizations can align their development efforts with business objectives and improve collaboration between development and operations teams.
Data-Driven DevOps: Improve Velocity and Quality of Software Delivery with Metrics and Feedback
- 1. Copyright © 2016 Splunk Inc. Data-Driven DevOps: Improve Velocity and Quality of Software Delivery with Metrics and Feedback Andi Mann @AndiMann
- 2. Abstract (Hidden) Much of the value of DevOps comes from a (renewed) focus on measurement, sharing, and continuous feedback loops. In increasingly complex DevOps workflows and environments, and especially in larger, regulated, or more crystallized organizations, these core concepts become even more critical. This session will show how, by focusing on 'metrics that matter,' you can provide objective, transparent, and meaningful feedback on DevOps processes to all stakeholders. Learn from real-life examples how to use the data generated throughout application delivery to continuously identify, measure, and improve deployment speed, code quality, process efficiency, outsourcing value, security coverage, audit success, customer satisfaction, and business alignment.
- 3. Defining DevOps INTEGRATION COLLABORATION COMMUNICATION BETWEEN DEV AND OPS METHODS FOR IMPROVING
- 4. CAMS – as close to prescriptive as DevOps gets
- 5. Shared Feedback Enables ‘The Three Ways’ Gene Kim, “DevOps Cookbook” and “The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win.”
- 6. DevOps Workflow is Becoming Complex and Opaque 6 Build (Jenkins, Bamboo) Code (Git, MS-TFS) Plan (Jira, Rally) Test/QA (Cucumber, SonarQube) Stage (Pivotal, AWS) Release (Jenkins, Octopus) Data Center Device Data Engagement Data Config (Puppet, Ansible) Monitor (NewRelic, Dynatrace) Cloud Services Network Services www/HTTP Data Social Sentiment Wire Data Application Data Continuous Integration (CI) / Continuous Delivery (CD) Site Reliability Engineering Business Impact Monitoring API ServicesSecurity/Compliance
- 7. DevOps complexity raises risk of failure ● Slower Speed ● Longer MTTR ● Lower Quality ● Reduced Agility ● Poor Visibility ● Hard to Scale ● Increased Waste ● Impaired Collaboration 7 DevOps From Hype Cycle for Application Services 2015, Gartner Group, July 2015, Betsy Burton, Philip Allega, http://www.gartner.com/document/3096018
- 9. I’m working super hard!! That’s my stapler.
- 10. 10 Yeah, but … … what are you achieving? I’m gonna need you to come in Sunday.
- 11. Gartner’s DevOps ‘Metrics that Matter’ Gartner Inc., Data-Driven DevOps: Use Metrics to Help Guide Your Journey, 29 May 2014 G00264319, Analyst(s): Cameron Haight | Tapati Bandopadhyay
- 12. IDC’s DevOps ‘Metrics that Matter’
- 13. More DevOps Metrics that Might Matter Culture e.g. • Retention • Satisfaction • Callouts Process e.g. • Idea-to-cash • MTTR • Deliver time Quality e.g. • Tests passed • Tests failed • Best/worst Systems e.g. • Throughput • Uptime • Build times Activity e.g. • Commits • Tests run • Releases Impact e.g. • Signups • Checkouts • Revenue
- 14. From every tool, every process, every component, on-prem or off One Constant: Machine Data
- 15. Common Data Fabric 15 API SDKs UI Other Tools Escalation/ Collaboration Visibility Across the Whole Dev Lifecycle Plan Code Build Test/QA Stage Release Config Monitor
- 16. Common Data Fabric 16 API SDKs UI Server, Storage. N/W Server Virtualization Operating Systems Infrastructure Applications Mobile Applications Cloud Services Other Tools Ticketing/Help Desk Custom Applications Visibility Across the Whole Ops Environment API Services
- 17. Data From Dev and Ops Tools 17
- 18. Data From Provisioning and Config 18
- 19. Data from Release Servers 19
- 20. Data from Infrastructure Systems 20
- 21. Data from Database Servers 21
- 22. Data from Customer Systems
- 23. Finding Your Metrics That Matter Work from business backwards Mine realtime machine data Close the feedback loops 23
- 24. Outcomes
- 25. Data drives objective feedback loops Velocity Deliver on time & on budget IT is delivering on time, on budget IT and Business Leaders Impact Deliver code for business needs IT is achieving business goals IT and Business Leaders, Customers, Staff Show you when you deliver. And when you don’t. Quality Deliver the quality you promised We deliver a quality experience for users Dev and Ops Organizations
- 26. Objective data enables continuous improvement Defect Information Capacity Planning Quality Standards Enhancement Requests Integration Requirements Acceptance Metrics Service Levels and KPIs Application Development Test and Acceptance Production BuildCodePlan Test/QA Stage Release Config Monitor Infrastructure Dependencies
- 27. Data clearly identifies ‘waste’ Plan Develop (UI) Develop (Db) Develop (M’ware) Develop (Backend) Security Test Monitor Build (Prod) Architect Secure/ Comply DeployAccept Unit Test Document Cap Plan Train Feedback Integration Test Configure System Test Launch CAB Develop (APIs) Budget Build (Dev) Mgmt/ Tooling W W W W W W W W W 16 40 52 35 96 40 48 24 --8 2 5 6 8 2 12
- 28. Data ensures transparency between all stakeholders • Release when ready, not a date! • Best / worst developers • Best / worst providers • Impact of new code on ops • Impact of new code on biz
- 29. Data-driven feedback accelerates velocity Pivot & improve with Continuous Insights Product Managers identify new opportunities Continuously delivered to market … and Auditors are “happy”
- 30. Data-driven feedback improves quality Code quality scans Static security scans White BoxDeveloper checks in code Automated Acceptance Tests Dynamic Security Scans Black Box “Chaos Monkey” tests Test Fail: Return Test Fail: Return X X Production QA Prod Pattern QA Pattern Library Test Pass: Promote Test Pass: Promote to Production Pattern library used for test and QA
- 31. Data-driven feedback shows business Impact
- 32. Fast-feedback loop for actionable commercial insights Data-driven DevOps allows rapid innovation BUSINESS DEV/OPS CUSTOMERS HOW IS OUR: • Security? • Quality? • Stability? • Performance? • Compliance? HOW IS OUR: • Market Launch? • Feature Usage? • Marketing Changes? • Prioritization? • Customer Sat?
- 33. Summary
- 34. Data-driven DevOps decisions help you … Improve Application Velocity Visibility across silos, tools, and processes exposes bugs and bottlenecks so you can remediate, iterate, and innovate faster. Improve Application Quality Track quality across multiple teams, tools, systems, and service providers, so you can find and fix more issues before production Improve Application Impact Real-time analytics correlates application delivery with business goals, so you can drive better experience and iterate faster
- 35. Further Reading ● Metrics that Matter, http://intellitect.com/devops-metrics/ - Mark Michaelis, IntelliTect ● Gartner Inc., Data-Driven DevOps: Use Metrics to Help Guide Your Journey, 29 May 2014 G00264319, Analyst(s): Cameron Haight | Tapati Bandopadhyay ● DevOps and the Cost of Downtime: Fortune 1000, IDC ● Blogs.splunk.com ● Splunk.com/DevOps ● Pleasediscuss.com/andimann