Agile Gurugram 2022 - Dinker Charak | Line of Sight from Engineering Excellence to Business Outcomes
- 1. © 2022 Thoughtworks Line of sight from engineering excellence to business outcomes http://agilegurugram.com/
- 2. © 2022 Thoughtworks On Fitness Metrics at the scale of an enterprise
- 3. © 2022 Thoughtworks “If you can not measure it, you can not improve it.” “You are what you measure.” 3
- 6. © 2022 Thoughtworks HIGH MEDIUM LOW ELITE METRIC Between once per hour and once per day Between A WEEK TO A MONTH >1 MONTH On-demand (multiple deploys per day) LEAD TIME Between one day and one week Between one week and one month Between one month and six months Less than one hour DEPLOYMENT FREQUENCY Less than one day Less than one day Between one week and one month Less than one hour MEAN TIME TO RESTORE 0-15% 0-15% 46-60% 0-15% CHANGE FAIL PERCENTAGE Source: State of DevOps report 2019 6 What does good engineering outcome looks like?
- 7. © 2022 Thoughtworks LEAD TIME DEPLOYMENT FREQUENCY MEAN TIME TO RESTORE (MTTR) CHANGE FAIL PERCENTAGE ● Code quality ● Managing tech debt ● Optimal Branching strategy ● Design flexibility ● Automated test coverage ● Shift left in security and performance ● Continuous Integration ● UAT ● Continuous delivery ● Configuration Management/ Provisioning ● Story prioritization ● Early feedback and adaptation ● AB testing ● Branching strategy ● Left shift in Test automation ● Left shift in Security ● Left shift in performance test and its automation ● Configuration Management and Provisioning ● Continuous delivery ● Monitoring and Alerts ● Quick rollback strategy ● Feature toggles ● Resilient infrastructure ● Ease of Troubleshooting and RCA by using distributed tracing and logging ● Monitoring and alerting ● Data backup/restore and rollback strategy ● Continuous Delivery ● Configuration Management/ Provisioning ● Canary release capability ● Pre-prod environments - replica of production environment ● Beta Releases Code Quality Architecture/Design Testability Security & Performance Delivery Infra & DevOps Engineering Excellence Success is an outcome of right practices 7 and it takes an organizational effort across people, process, technology and culture
- 8. © 2022 Thoughtworks Build failure rate Security warnings Tech debt (Less is good) (Less is good) (Less is good) 3 Key metrics of software excellence Deployment frequency Change fail percentage (Less is good) (More is good) Lead time (Less is good) Mean time to restore (Less is good) 4 Key metrics of Devops 8 The software development excellence goes beyond developer productivity A Fitness metrics is usually abstract, derived and carries inertia to resist gaming by teams. Any change in these metrics requires programs to make technology, process, people, capability and tooling changes.
- 9. © 2022 Thoughtworks “Speed is irrelevant if you are going in the wrong direction.” 9
- 10. © 2022 Thoughtworks Principle behind Agile: “Business people and developers must work together daily throughout the project” 10 Business Business Business Supporting role Collaboration Tech-led differentiation Tech@Core Tech Tech Tech Business and Tech 3rd industrial revolution information as a resource 4th industrial revolution technology at the heart of value
- 11. © 2022 Thoughtworks Improvement in efficiency & effectiveness Improvement in experience 4 Key business outcomes areas Future sensing Increase in influence Create value by linking measures to business outcomes 11 A Fitness metrics is usually abstract, derived and carries inertia to resist gaming by teams. Any change in these metrics requires programs to make technology, process, people, capability and tooling changes. We arrived at these 11 Key Fitness Metrics that cover from development to outcomes: Build failure rate Security warnings Tech debt (Less is good) (Less is good) (Less is good) 3 Key metrics of software excellence Deployment frequency Change fail percentage (Less is good) (More is good) Lead time (Less is good) Mean time to restore (Less is good) 4 Key metrics of Devops
- 12. © 2022 Thoughtworks 4 key learnings from our experience with Thoughtworks Polaris 12
- 13. © 2022 Thoughtworks 1. Establishing a line of sight is not a top-down 3 - 6 months onwards Few sessions 7-30 days 2-4 sprints 30 days - ongoing Implementing on your product Gathering & observing Discovery of key fitness metrics Remediation & value realization Identify data sources and data elements Data gathering Outcome goals Underlying metrics investigation How to measure for these goals periodicity, directionality, baselining period, domain/industry defaults Identify data elements, data sources and the formula Ownership between business & delivery teams Establish data pipelines Transform data to time-series per Polaris Specs Feed to Polaris UI Monitoring Establishing sensible defaults Make fix in development process, infra, teams, Ops ,etc Re-gather & observe 1 2 3 4
- 14. © 2022 Thoughtworks 2. Don’t cause information overload 14
- 15. © 2022 Thoughtworks 3. Remediation of non-performing metrics 15
- 16. © 2022 Thoughtworks 4. Build it like an asset: should be useful for most teams across your whole organization 16 Delivering Value to Business Process + Learnings Driven Focus on Expertise Focus on Assets Process Driven Focus on Skills Focus on Methodology Organic, hence incremental Systemic, hence exponential Delivering Knowledge to Employees startups enterprises
- 18. © 2022 Thoughtworks To ensure your team achieves business outcomes, focus on data driven discussion of correlation between your engineering outcomes and business outcomes 18 Establish a line of sight between engineering outcomes and business outcomes Identify right business outcome measures before you deep dive into building software Avoid technical metrics that help you and focus on metric that measure the impact of software being developed Apply your domain context to metrics to set meaningful targets Metrics dashboards are place to hold conversations and not prescriptions
- 19. © 2022 Thoughtworks Thank you Dinker Charak [email protected] | www.ddiinnxx.com Head - Thoughtworks Consulting Assets, Thoughtworks 19
- 21. © 2022 Thoughtworks Create value by linking measures to business outcomes Very frequent build failure may imply absence of unit tests, right build tools or non-adherence to coding standards & other best practices by engineering team Build failure rate How many security warnings are currently being reported by the underlying tools. In short term, high is good as it indicated such issues are being caught early Security warnings How much additional rework needs to be done to clean up the hastily written code to enable speedy delivery at the cost of quality. Tech debt Metrics on how the products help improve the efficiency & effectiveness of the workflow the products is the part of Improvement in efficiency & effectiveness Metrics on how the products improve experience for those interacting with it and extracting value from the product. Improvement in experience (Less is good) (Less is good) (Less is good) The specific metrics under each of the business areas vary based on the industry and context. 3 Key metrics of software excellence 4 Key business outcomes areas A Fitness metrics is usually abstract, derived and carries inertia to resist gaming by teams. Any change in these metrics requires programs to make technology, process, people, capability and tooling changes. We arrived at these 11 Key Fitness Metrics that cover from development to outcomes: How often can you push a new revenue generating product / feature or a critical fix to production. Deployment frequency What changes to production had to be rolled back or hotfix / patched. Change fail percentage (Less is good) (More is good) How fast can you take a new revenue generating product / feature to market. Lead time (Less is good) How fast can you recover from non-availability of your products & services caused by an external attack or a botched up release. Mean time to restore (Less is good) 4 Key metrics of Devops* Metrics on how the products help executive leadership sense the early trends in the market to react to them in time. Future sensing Metrics on how the products increase the influence of the business (in the market, within their domain, vs the competitive forces or with regulators) Increase in influence