An Agile approach to Business Metrics
4 likes•929 views
This document discusses an agile approach to measuring business metrics. It proposes using "Analytics Sprints" where each sprint a key performance indicator (KPI) is selected and hypotheses for improving it are tested. At the end of each sprint, results are assessed and the team either pivots experiments or perseveres on the next KPI. Having a dedicated "Metrics Master" role to facilitate this process with stakeholders is also suggested. Case studies from Spotify using iterative metric refinement are provided. Challenges with early metrics and combining incremental values are also outlined. The approach aims to discover and measure true business value in an adaptive, learning-focused way.
An Agile approach to Business Metrics
- 1. An Agile approach to Business Metrics by @Pablo_Valcarcel [email protected]
- 3. Who is this guy? @Pablo_Valcarcel [email protected]
- 4. ‘Conventional’ Agile Metrics Lead Time* Defect count Work in Progress Code coverage Unplanned Changes Velocity (Stpts per sprint) Return on investment* Innovations per sprint Artifacts generated Slack time Failure Load (firefighting time) Iteration Burn-Down Unfinished Stories Customer Satisfaction* LOC (lines of code) Un-deployed Stories # Blocks Budget/Schedule Compliance Flow Efficiency (lead time / touch time) Release Burn-Up *Italics: Connected to business value.
- 5. Your average Sprint Review... KPI KPI KPI KPI Stakeholder Scrum MasterProduct Owner
- 6. Is this ‘Agile’ enough? KPI KPI KPI KPI Stakeholder Scrum MasterProduct Owner ● Does it help increase customer satisfaction? ● We deliver business value on every iteration, but how do we measure it? ● Shouldn’t the definition of metrics be an iterative+feedback-based process on itself? ● Priorities and markets change all the time.
- 7. Enterprise Metrics: The Ocean of Data
- 8. The Startup Way: Go Pirate ‘AARRR!’ -Dave McLure 500 Startups
- 9. The Lean Metrics Process (Croll & Yoskovitz)
- 10. The Lean Sprint (Ash Maurya)
- 11. The Thesis: The Analytics Sprint Agile discipline and ‘Lean Analytics’ thinking can provide a better method to measure value.
- 12. The Analytics Sprint Planning: At the beginning of each sprint discuss: ● With stakeholders: What are the OKRs/key metrics? What’s the KPI we should work on the next iteration? What’s the baseline? ● With team: Hypothesis are defined on why/how to tackle that KPI. All of this: KPI, baseline, hypothesis and experiments are documented in writing.
- 13. The Analytics Sprint Reviewing: At the end of each sprint: ● There’s an assessment. Did we hit the goal? If not pivot into another experiment. If we did we persevere onto the next KPI. ● Data is turned into validated learning. Even failed experiments provide learning (think Google with OKR).
- 14. The Analytics Sprint: The Metrics Master Metrics Master: Role responsible for the definition, revision and accounting of the key metrics. ● Facilitates discussion with stakeholders on business value and key metrics. ● Discusses hypothesis and experiments with team. ● Tracks and does innovation accounting. ● Reports back on the results and either pivots or perseveres on the next metric. ● It can be the Product Owner (business value) or a dedicated team-member (e.g. data-scientist)..
- 15. But wait, I already do that! Even if you’re already measuring a business value metric every sprint: ● When was its value defined? ● Who defined it? ● How’s the reporting being done to stakeholders and customers? (Theatre of success or. customer discovery?).
- 16. Case Study: Spotify Metrics-Driven Development Every retrospective they measure squad performance, and they refine the metric. They also set goal targets at the start of every integration project we do (which may span multiple sprints). For example: “we will know we’re successful when this integration brings us x-amount of users.” Source: Lynn Root
- 17. Case Study: Spotify Metrics-Driven Development ● They consciously avoid waste by measuring everything. ● They also set baselines with historical data. ● Goals are a shorter decision- making cycle as well as make more informed decisions about strategy and partnerships.
- 18. Some pro-tips ● Write it down. Prevents biases. ● Always define a baseline or benchmark. ● Avoid ‘bad metrics’. ● Results will be murky. ● Conversation with stakeholders, customers and team is a value on itself.
- 19. Further Reading: Spotify: Metrics Driven Development by Lynn Root. Lean Analytics by Alistair Croll & Ben Yoskovitz. Appropriate Agile Measurement by Deborah Hartmann & Robin Dymond. Five Tips For Testing Your Idea by Anita Newton, Cindy Alvarez & Alistair Croll. Scaling Lean by Ash Maurya. Managing Metrics in an Iterative Incremental Development Environment by John D. McGregor. Agile Metrics, post by André Dhont.
- 20. A call to action: We like thinking of Agility not as a cargo cult, but as a set of values and principles that we adapt to the specifics of every single case. So, we need you to learn.Please, hit me with your ideas, case studies and questions at [email protected]
- 21. Thanks for your attention!!! @Pablo_Valcarcel [email protected]
- 22. Reasons for an Agile approach ● We don’t know what we should be measuring. ● We discover a better way to measure it. ● Business priorities and political priorities change. ● The product’s incremental development requires an incremental an adaptative approach to its metrics. ● Project (or transition) is too big for our resources and we have to pick our battles. We want to measure true business value as early as possible!
- 23. Challenges with this approach From a business point of view: ● First, what information is available early in the process and how reliable is that information? ● Second, how can we use the early estimates to predict the actual values? ● The incremental aspect of the process requires that there be a plan for combining the values of certain metrics in order to obtain project-wide values (e.g. Addition, Averaging, Min-Max values).
- 24. Startup Vs. Intrapreneurial metrics (by A. Croll)