Pin the tail on the metric v01 2016 oct
Download as PPTX, PDF•0 likes•151 views
The workshop, led by Steve Martin, focuses on understanding the importance of goals in driving metrics, identifying 'good' and 'bad' metrics, and conducting an interactive exercise called 'pin the tail on the metric'. Participants are encouraged to engage with each other to define relevant metrics aligned with their goals while considering the characteristics that make metrics actionable and relevant. The session highlights the need for a culture of trust and iterative improvement in metric evaluation within organizations.
![Table Discussions
Teams Portfolios/Programs Transformations
• Velocity
• # defects resolved
within sprint
• # stories groomed /
ready for planning
• % code coverage
for automated unit
testing
• WIP (# stories)
• % Complete
• # defects [that
made it into] in QA
environment [from
Dev environment at
end of sprint]
• # automated
regression tests
• System
performance
• WIP (# stories)
• Change in
Velocities per team
• Customer
Sentiment
• # of teams trained
• # of
sprints/iterations
completed per team
• # new [paying]
subscribers
» At your table, select the appropriate group of real metrics used at
clients.
» As a table, discuss and decide – which ones tend to be “good”
metrics and which ones tend to be “bad” metrics.
» Time box: 4 minutes](https://image.slidesharecdn.com/pinthetailonthemetricv01psbaw60min2016oct-161212154633/85/Pin-the-tail-on-the-metric-v01-2016-oct-29-320.jpg)
Pin the tail on the metric v01 2016 oct
- 1. Steve Martin, PMI-ACP, PMP Senior Program Manager - Methodology & Governance Pin the Tail on the Metric
- 2. Workshop Objectives 1. Learn the importance of goals and how they drive metrics 2. Be able to spot “good” metrics and “bad” metrics 3. Learn how to run the Pin the Tail on the Metric exercise to identify metrics to use
- 3. Couple housekeeping tips Highly interactive workshop You get out what you put in. Our time is limited - please refrain from using technology during session Please feel free to ask questions But, want to respect our time box together May use a Parking Lot for questions To get electronic copy of the deck, connect via LinkedIn or leave your card
- 7. Metrics
- 8. Metrics
- 10. Small group exercise Introduce yourselves (if not done already) So why do we need metrics anyway? Time box: 3 Mins
- 14. So what do goals have to do with metrics? Your metrics should provide information to help you guide towards achieving your goals.
- 15. There can be goals at multiple levels for multiple purposes Teams Portfolios / Programs Organization-Wide / Company Transformations
- 16. Let’s talk about Goals At your tables, self organize into small groups (about 5ish) around a topic area that you want to discuss more about over the course of this workshop. » Teams » Portfolios / Programs » Organization-Wide / Company (Agile) Transformations You may end up with more than one small group per table. In your small group, discuss what are the main goals/outcomes from {teams, portfolio/programs or org- wide/transformation}. Write goals on cards/post-its on your table. Timebox: 3 mins (to get in small groups and discuss main
- 17. In summary…know your goals Start with goals first. Metrics should enable conversations about future action with respect to your goals, and not to be used as a status report.
- 19. Video “Using Metrics” https://www.youtube.com/watch?v=F4dCLrQpVsM Featuring Martin Klubeck, author of "Metrics: How to Improve Key Business Results"; and Tim Chester, Chief Information Officer at the University of Georgia. Posted February 2012.
- 20. Shout it out What makes a good metric?
- 21. Some Characteristics of “Good Metrics” Aligned with goals (Teams > Portfolios > Organization) Actionable Leading (can they help make decisions in the future) vs. Lagging (they “tell the news” and may make feel good/bad) Easy to understand Easy to collect data Recommended to have small number of good metrics
- 22. Shout it out What makes a bad metric?
- 23. Be aware of “vanity metrics” Those that go “up and to the right” Non-variable Non-actionable: » # of page visits » # unique visitors » Total cumulative sales
- 24. Let’s see some examples.
- 25. Example Radar Chart (1 Team) 0.0 1.0 2.0 3.0 4.0 5.0 Ownership Scrumability Accountability Quality focused Collaboration Resilience Team Average 0.0 1.0 2.0 3.0 4.0 5.0 Ownership Scrumability Accountability Quality focused Collaboration Resilience Team Average Coach Rating After 5 Sprints (1 Team) After 10 Sprints (1 Team)
- 26. What do you think? # Hours spent in sprint planning Give me a thumbs up, thumbs side, thumbs down on these “team” metrics:
- 27. What do you think? # Stories Completed Give me a thumbs up, thumbs side, thumbs down on these “program/portfolio” metrics:
- 28. What do you think? Average Lead Time (i.e. time to complete new requests) Give me a thumbs up, thumbs side, thumbs down on these “organization” metrics:
- 29. Table Discussions Teams Portfolios/Programs Transformations • Velocity • # defects resolved within sprint • # stories groomed / ready for planning • % code coverage for automated unit testing • WIP (# stories) • % Complete • # defects [that made it into] in QA environment [from Dev environment at end of sprint] • # automated regression tests • System performance • WIP (# stories) • Change in Velocities per team • Customer Sentiment • # of teams trained • # of sprints/iterations completed per team • # new [paying] subscribers » At your table, select the appropriate group of real metrics used at clients. » As a table, discuss and decide – which ones tend to be “good” metrics and which ones tend to be “bad” metrics. » Time box: 4 minutes
- 30. What do you think of this “Post Release Scorecard?” Business Expectations: • % User Story Definition Before Coding Begins (%) • Did Release meet release expectations (1-5, user) Quality: • % Code Coverage for Unit Testing (%) • % Code Coverage for Functional Testing (%) • # Runs/Day for Unit Tests (#) • # Runs/Day for Functional Tests (#) Communication: • % Items in Communication Plan Completed (%) • Effectiveness of Communications (1-5, user) Planning: • # Story Points/Day (#) • Velocity Variance (%)
- 31. Thoughts on this CIO Dashboard? Image source: https://www.apptio.com/blog/rethinking-cio-dashboard-tying-technology-spending-business-results
- 32. Other Characteristics of “Good Metrics” Use a small number of metrics » Keeps focus on the goals Revise/update metrics you use over time » Are your metrics still relevant? » Have goals shifted? » Are you “good enough?”
- 33. PIN THE TAIL ON THE METRIC
- 34. Pin the tail on the metric An activity intended to be used with teams, managers, and executives to help identify the highest value-add, actionable metrics. Can be used at any level (e.g. teams, portfolios/programs, organization, transformations, etc.) Supplies needed: » Index cards or sticky notes » Felt-tip pens (such as sharpies) » Blue tape Total timebox for exercise: typically 60-90 mins.
- 35. Pin the tail on the metric Step 1: Create list of your current metrics. Using your sharpies and sticky notes (or cards), write down each metric that you are currently reporting on and to whom. One metric per sticky note. Burn Down Chart Managers % Unit Test Coverage Managers Velocity Managers
- 36. Pin the tail on the metric Step 2: Create your workspace. Use your blue tape. On a wall (or on a table), create the scale as indicated. Painfully Useless Very Useful
- 37. Pin the tail on the metric Step 3: Silently place metrics cards on your workspace. The key really is to be silent. Discussion will happen shortly… Painfully Useless Very Useful Burn Down Chart Managers Velocity Managers # Defects Resolved Managers % Unit Test Coverage Managers # Stories Complete d Team % Complete Managers
- 38. Pin the tail on the metric Step 4: Review your goals together. Post your goals next to the workspace. Read goals aloud. Painfully Useless Very Useful Burn Down Chart Managers Velocity Managers # Defects Resolved Managers % Unit Test Coverage Managers # Stories Complete d Team % Complete Managers Goals: 1. Blah blah blah blah 2. Blah blah blah 3. Blah blah
- 39. Pin the tail on the metric Step 5: Discuss (Majority of time spent here). Together, discuss metrics with respect to goals. Add/remove/modify metrics as needed. Painfully Useless Very Useful Burn Down Chart Teams Velocity Managers # Defects Carried Over Teams % Unit Test Success Rate Teams # Stories Complete d Team % Complete Managers Goals: 1. Blah blah blah blah 2. Blah blah blah 3. Blah blah
- 40. Pin the tail on the metric Step 6: Determine reporting plan and frequency. Transparency is key. You’ll need to determine the (small) group of metrics that will be reported, how and when. Painfully Useless Very Useful Burn Down Chart Teams Velocity Managers # Defects Carried Over Teams % Unit Test Success Rate Teams # Stories Complete d Team % Complete Managers Goals: 1. Blah blah blah blah 2. Blah blah blah 3. Blah blah
- 42. Further Conversations Metrics are only part of the story. You must also look at your Governance process and how you make decisions. You must have a culture of safety and trust with metrics. It’s OK to experiment with metrics. Did this give us the information we thought we needed? Is this type of decision (by which were gathering data for) still needed? Can we retire this metric?
- 43. Workshop Objectives 1. Learn the importance of goals and how they drive metrics 2. Be able to spot “good” metrics and “bad” metrics 3. Learn how to run the Pin the Tail on the Metric exercise to identify metrics to use
- 44. Q & A Steve Martin www.linkedin.com/in/stevemartinpmp
- 45. APPENDIX
- 46. Additional Helpful References https://www.linkedin.com/pulse/forget-roi-use- innovation-metrics-instead-steve-glaveski http://scaledagileframework.com/metrics/ https://insights.sei.cmu.edu/sei_blog/2014/09/ agile-metrics-seven-categories.html
- 47. Pirate Metrics
- 48. Innovation Metrics Most commonly used innovation metrics: 1. Revenue generated by new products 2. Number of projects in the innovation pipeline 3. Stage-gate specific metrics, i.e. projects moving from one stage to the next 4. P&L impact or other financial impact 5. Number of ideas generated Activity metrics show you’re “busy stoking the boilers of innovation”. Examples: # employees trained in LSU, # new product ideas in research Impact metrics show your ship actually going somewhere. Examples: market share, cost reduction, revenue from new products/services in 1st year to market. Source: Kirsner, Scott, “What Big Companies Get Wrong About Innovation Metrics”, Harvard Business Review, May 06, 2015.
- 49. Innovation Metrics Five ways most measurement efforts go wrong: 1. Alignment can take a while 2. Patience is a rarity 3. Failure isn’t fun to measure 4. Having a vision 5. Measuring too much Source: Kirsner, Scott, “What Big Companies Get Wrong About Innovation Metrics”, Harvard Business Review, May 06, 2015.
Editor's Notes
- #6: So when I think about metrics, the first thought that comes to mind is something like what you see on the screen here. There have been many times across my experiences with not only clients, but when I was a full time employee within a company, where I thought “what the heck are we collecting all this information for?”
- #7: The lost hours and stress for gathering this information that gets sent up above to the heavens, where we have really no idea what the folks “upstairs” or “at corporate” are going to do with the information.
- #8: This leads to metrics tending to be seen as the evil incarnate. The information could potentially be used for what is seen as enforcement, or even worse, retribution. This can lead to feelings of being micromanaged, or worse, lack of trust. So, I assert, in general, people don’t like to be measured. I know I tend to not like it personally.
- #9: But, I’m here today to say that metrics don’t have to be like this. Yes, it is a matter of perspective, but only if metrics are “done” in the “right” way.
- #10: Metrics can actually be helpful. Can be insightful. Can help make decisions. And that’s what we’re here to talk about today.
- #19: Total Elapsed Time: 40 mins
- #20: https://www.youtube.com/watch?v=F4dCLrQpVsM (4 Mins) Ask: What were your take-aways from this video? What are characteristics of good metrics? What are characteristics of bad metrics?
- #26: Use of surveys to turn qualitative metrics into something (hopefully) actionable quantitative metrics.