Growing an Experiment-driven Quality Culture (Capgemini TestJam 2021)
0 likes305 views
This document describes experiments conducted to improve testing and quality culture at a company. The author worked with 4 product teams, providing transparency into their current practices, raising awareness of quality issues, and helping teams run experiments. While some success criteria were met, not all teams adopted new practices. To scale the approach, the author proposes providing lightweight feedback to additional teams based on core quality values. The author then reflects on shifting their approach to better understand different stakeholders' needs and inspire curiosity in experimentation, rather than demanding changes.
Growing an Experiment-driven Quality Culture (Capgemini TestJam 2021)
- 1. GROWING AN EXPERIMENT-DRIVEN QUALITY CULTURE LISI HOCKE @lisihocke
- 2. THIS IS A STORY OF EXPERIMENTATION @lisihocke
- 3. THIS IS A STORY OF EXPERIMENTATION @lisihocke
- 4. THIS IS A STORY OF EXPERIMENTATION @lisihocke
- 5. THIS IS A STORY OF EXPERIMENTATION @lisihocke
- 6. THIS IS A STORY OF EXPERIMENTATION @lisihocke
- 7. THIS IS A STORY OF EXPERIMENTATION @lisihocke
- 8. WHAT’S IN THE BOX? OUR STARTING POINT @lisihocke
- 9. @lisihocke THIS IS US AT FLIXMOBILITY
- 10. @lisihocke IT‘S TIME TO FOCUS ON QUALITY
- 12. THE CHALLENGE @lisihocke Quality is value
- 13. THE CHALLENGE @lisihocke Quality is value Lack of transparency
- 14. THE CHALLENGE @lisihocke Quality is value Lack of transparency Knowledge is not common
- 15. THE CHALLENGE @lisihocke Quality is value Lack of transparency Knowledge is not common The need to improve
- 16. THE CHALLENGE @lisihocke Quality is value Lack of transparency Knowledge is not common The need to improve Let’s tackle it before it tackles us
- 17. OUR MISSION @lisihocke Improve the testing & quality culture of our product teams by leveling up knowledge, skills and practices
- 18. LET‘S EXPERIMENT! AN EXPERIMENT OF EXPERIMENTS @lisihocke
- 20. OUR HYPOTHESIS With an experimental group of 4 product teams, we believe that • creating transparency where they currently are regarding testing and quality as well as identifying problems, • raising awareness and knowledge of the options at hand to probe the problems, and • running first experiments to learn what works in their context to enable them to rinse and repeat, will result in increased maturity of each team with regards to testing and quality. … @lisihocke
- 21. OUR HYPOTHESIS … We'll know we have succeeded when each of the 4 teams • has a clear, shared test strategy in place reflecting the current situation as well as identified problems, • has gained further knowledge and skills regarding testing and quality through a workshop, and • has adopted 1 new practice in their testing and quality culture that proved valuable in their context. @lisihocke
- 23. EXIT CRITERIA The experiment will stop and be re-evaluated in the following cases, whichever comes first: • No problem there! We find out that the chosen product teams already have transparency, awareness and experiments in place to improve their testing and quality. • Yay, success :) The hypothesis success criteria are fulfilled as defined above. • Didn't work out - let's rethink it. It is November 1, 2019 (even though success criteria are not fulfilled). @lisihocke
- 24. @lisihocke THE HIGH LEVEL PLAN
- 25. @lisihocke THE HIGH LEVEL PLAN Find contributors
- 26. @lisihocke THE HIGH LEVEL PLAN Find contributors Align among contributors
- 27. @lisihocke THE HIGH LEVEL PLAN Find contributors Align among contributors Select product teams as pilot teams
- 28. @lisihocke THE HIGH LEVEL PLAN Find contributors Align among contributors Select product teams as pilot teams Work with each team
- 29. @lisihocke THE HIGH LEVEL PLAN Find contributors Align among contributors Select product teams as pilot teams Work with each team Evaluate the hypothesis success criteria
- 31. @lisihocke COMMUNICATION GUIDELINES TO APPROACH THE PRODUCT TEAMS
- 32. @lisihocke COMMUNICATION GUIDELINES TO APPROACH THE PRODUCT TEAMS Appreciate what's working.
- 33. @lisihocke COMMUNICATION GUIDELINES TO APPROACH THE PRODUCT TEAMS Appreciate what's working. It's an opportunity to learn.
- 34. @lisihocke COMMUNICATION GUIDELINES TO APPROACH THE PRODUCT TEAMS Appreciate what's working. It's an opportunity to learn. Let's improve together.
- 35. @lisihocke COMMUNICATION GUIDELINES TO APPROACH THE PRODUCT TEAMS Appreciate what's working. It's an opportunity to learn. Let's improve together. It's about you, the team!
- 36. @lisihocke COMMUNICATION GUIDELINES TO APPROACH THE PRODUCT TEAMS Appreciate what's working. It's an opportunity to learn. Let's improve together. It's about you, the team! We support you, custom-tailored to your needs.
- 37. @lisihocke COMMUNICATION GUIDELINES TO APPROACH THE PRODUCT TEAMS Appreciate what's working. It's an opportunity to learn. Let's improve together. It's about you, the team! We support you, custom-tailored to your needs. Senior tech leadership has our back!
- 38. @lisihocke SELECT PILOT TEAMS – LET’S TALK!
- 39. @lisihocke SELECT PILOT TEAMS – LET’S TALK! Building quality in
- 40. @lisihocke SELECT PILOT TEAMS – LET’S TALK! Building quality in What's working
- 41. @lisihocke SELECT PILOT TEAMS – LET’S TALK! Building quality in What's working What's worrying
- 42. @lisihocke SELECT PILOT TEAMS – LET’S TALK! Building quality in What's working What's worrying Eagerness
- 43. @lisihocke SELECT PILOT TEAMS – LET’S TALK! Building quality in What's working What's worrying Eagerness Readiness
- 44. @lisihocke READY, SET GO! LET’S WORK WITH THE TEAMS
- 46. @lisihocke TRANSPARENCY Desired outcome: The team has a clear, shared test strategy in place reflecting the current situation as well as identified problems.
- 47. @lisihocke TRANSPARENCY: PAIN POINTS Shout-out to Ashley Hunsberger & Lisa Crispin Visualize current workflow & identify problems
- 48. @lisihocke TRANSPARENCY: RISK Shout-out to Beren van Daele & Marcel Gehlen RiskStorming to identify context-specific product risks
- 49. @lisihocke TRANSPARENCY: STATUS QUO Document current test strategy
- 51. @lisihocke AWARENESS Desired outcome: The team has gained further knowledge and skills regarding testing and quality through a workshop.
- 53. @lisihocke TEAM EXPERIMENTS Desired outcome: The team has adopted 1 new practice in their testing and quality culture that proved valuable in their context.
- 54. @lisihocke TEAM EXPERIMENTS Review the currently known challenges
- 55. @lisihocke TEAM EXPERIMENTS Review the currently known challenges Pick one challenge to tackle next
- 56. @lisihocke TEAM EXPERIMENTS Review the currently known challenges Pick one challenge to tackle next Brainstorm potential solutions
- 57. @lisihocke TEAM EXPERIMENTS Review the currently known challenges Pick one challenge to tackle next Brainstorm potential solutions Pick one option to try out
- 58. @lisihocke TEAM EXPERIMENTS Review the currently known challenges Pick one challenge to tackle next Brainstorm potential solutions Pick one option to try out Design the experiment
- 59. @lisihocke TEAM EXPERIMENTS Review the currently known challenges Pick one challenge to tackle next Brainstorm potential solutions Pick one option to try out Design the experiment Run the experiment
- 60. @lisihocke TEAM EXPERIMENTS Review the currently known challenges Pick one challenge to tackle next Brainstorm potential solutions Pick one option to try out Design the experiment Run the experiment Evaluate the experiment
- 61. @lisihocke TEAM EXPERIMENTS Challenge: <short challenge description> Hypothesis: We believe that <doing X> will result in <desired outcome>. We'll know we have succeeded when <measurable criterion is achieved>. Experiment: <concrete details like planned actions, experiment run time, exit criteria, etc.>
- 63. TIME TO EVALUATE LESSONS LEARNED @lisihocke
- 65. All 4 teams have a clear, shared test strategy in place reflecting the current situation as well as identified problems HYPOTHESIS EVALUATION @lisihocke
- 66. All 4 teams have a clear, shared test strategy in place reflecting the current situation as well as identified problems All 4 teams have a clear, shared test strategy in place reflecting the current situation as well as identified problems → YES ☺ HYPOTHESIS EVALUATION @lisihocke
- 67. All 4 teams have a clear, shared test strategy in place reflecting the current situation as well as identified problems All 4 teams have a clear, shared test strategy in place reflecting the current situation as well as identified problems → YES ☺ All 4 teams have gained further knowledge and skills regarding testing and quality through a workshop HYPOTHESIS EVALUATION @lisihocke
- 68. All 4 teams have a clear, shared test strategy in place reflecting the current situation as well as identified problems All 4 teams have a clear, shared test strategy in place reflecting the current situation as well as identified problems → YES ☺ All 4 teams have gained further knowledge and skills regarding testing and quality through a workshop All 4 teams have gained further knowledge and skills regarding testing and quality through a workshop → YES ☺ HYPOTHESIS EVALUATION @lisihocke
- 69. All 4 teams have a clear, shared test strategy in place reflecting the current situation as well as identified problems All 4 teams have a clear, shared test strategy in place reflecting the current situation as well as identified problems → YES ☺ All 4 teams have adopted 1 new practice in their testing and quality culture that proved valuable in their context All 4 teams have gained further knowledge and skills regarding testing and quality through a workshop All 4 teams have gained further knowledge and skills regarding testing and quality through a workshop → YES ☺ HYPOTHESIS EVALUATION @lisihocke
- 70. All 4 teams have a clear, shared test strategy in place reflecting the current situation as well as identified problems All 4 teams have a clear, shared test strategy in place reflecting the current situation as well as identified problems → YES ☺ All 4 teams have adopted 1 new practice in their testing and quality culture that proved valuable in their context All 4 teams have adopted 1 new practice in their testing and quality culture that proved valuable in their context → Only 2 All 4 teams have gained further knowledge and skills regarding testing and quality through a workshop All 4 teams have gained further knowledge and skills regarding testing and quality through a workshop → YES ☺ HYPOTHESIS EVALUATION @lisihocke
- 71. @lisihocke IMPACT WE HAD – AND DIDN’T HAVE
- 72. @lisihocke IMPACT WE HAD – AND DIDN’T HAVE ☺ Team conversations about testing and quality
- 73. @lisihocke IMPACT WE HAD – AND DIDN’T HAVE ☺ Team conversations about testing and quality ☺ Raised awareness and increased knowledge and skills
- 74. @lisihocke IMPACT WE HAD – AND DIDN’T HAVE ☺ Team conversations about testing and quality ☺ Raised awareness and increased knowledge and skills ☺ Teams inspired to improve their practices
- 75. @lisihocke IMPACT WE HAD – AND DIDN’T HAVE ☺ Team conversations about testing and quality ☺ Raised awareness and increased knowledge and skills ☺ Teams inspired to improve their practices Not the whole team was on board, silos remained
- 76. @lisihocke IMPACT WE HAD – AND DIDN’T HAVE ☺ Team conversations about testing and quality ☺ Raised awareness and increased knowledge and skills ☺ Teams inspired to improve their practices Not the whole team was on board, silos remained Not all people opened up for new concepts, misconceptions prevailed
- 77. @lisihocke IMPACT WE HAD – AND DIDN’T HAVE ☺ Team conversations about testing and quality ☺ Raised awareness and increased knowledge and skills ☺ Teams inspired to improve their practices Not the whole team was on board, silos remained Not all people opened up for new concepts, misconceptions prevailed Teams struggled with experiments, fell back into everyday business
- 78. @lisihocke TOO MUCH EFFORT TO SCALE THIS APPROACH
- 79. HOW TO SCALE? A LIGHTWEIGHT APPROACH @lisihocke
- 81. @lisihocke A SECOND EXPERIMENT We believe that if we learn about the testing strategy of product teams and provide them peer feedback regarding our perspective on their challenges, then this will • help the teams build increased self-awareness regarding testing and quality, and • help us understand our current testing and quality culture. We'll know we have proven the hypothesis when the teams as well as we are able to explain the difference between their approach and our suggestions and why it might differ from expectations.
- 83. A SECOND PLAN Align within the initiative group on core values and select product teams @lisihocke
- 84. A SECOND PLAN Align within the initiative group on core values and select product teams Interview, present and provide feedback on the teams’ testing strategy @lisihocke
- 85. A SECOND PLAN Align within the initiative group on core values and select product teams Interview, present and provide feedback on the teams’ testing strategy Evaluate what we learned in this experiment and propose the next steps @lisihocke
- 86. @lisihocke CORE VALUES TO BASE OUR FEEDBACK ON
- 87. @lisihocke CORE VALUES TO BASE OUR FEEDBACK ON Quality is Team Responsibility
- 88. @lisihocke CORE VALUES TO BASE OUR FEEDBACK ON Quality is Team Responsibility Tackle the Unknown, Automate the Known
- 89. @lisihocke CORE VALUES TO BASE OUR FEEDBACK ON Quality is Team Responsibility Tackle the Unknown, Automate the Known Invest in Quality and Quality Will Work for You
- 90. @lisihocke INTERVIEW TEAM MEMBERS ON THEIR TEAM’S TESTING STRATEGY
- 91. @lisihocke FEEDBACK SESSION & EXPERIMENT EVALUATION
- 92. @lisihocke FEEDBACK SESSION & EXPERIMENT EVALUATION
- 94. @lisihocke … YET THEN LIFE HAPPENED
- 95. GROWING A CULTURE BELIEFS & NEEDS @lisihocke
- 96. @lisihocke PRODUCT TEAMS: WE NEED YOU TO SOLVE OUR PROBLEMS!
- 97. @lisihocke PRODUCT TEAMS: WE NEED YOU TO SOLVE OUR PROBLEMS! Or rather: we need support so we become aware of our options and enablement so we can solve our own problems
- 98. @lisihocke → INSPIRE & BUILD ON PEOPLE’S CURIOSITY
- 99. @lisihocke → INSPIRE & BUILD ON PEOPLE’S CURIOSITY Experimenting with tech talks and personal conversations inside a domain, creating a “pull”
- 100. @lisihocke TECH LEADS: WE NEED DATA & GLOBAL METRICS!
- 101. Or rather: we need to know which impact we want to have in our specific contexts and what information we need to have it @lisihocke TECH LEADS: WE NEED DATA & GLOBAL METRICS!
- 102. @lisihocke → UNDERSTAND NEEDS & FIND HELPFUL INDICATORS
- 103. @lisihocke → UNDERSTAND NEEDS & FIND HELPFUL INDICATORS Experimenting together with tech leads on perceived quality needs and indicators helping them make better decisions
- 104. @lisihocke → UNDERSTAND NEEDS & FIND HELPFUL INDICATORS Experimenting together with tech leads on perceived quality needs and indicators helping them make better decisions Shout-out to Michael Kutz, João Proença, Toyer Mamoojee, Emanuil Slavov and Jenny Bramble
- 105. @lisihocke CONTRIBUTORS: WE NEED QUALITY TO START WITHIN THE TEAMS!
- 106. @lisihocke CONTRIBUTORS: WE NEED QUALITY TO START WITHIN THE TEAMS! Or rather: we need to change the system so behaviors contributing to quality get rewarded and valued
- 107. @lisihocke → SHAPE THE SYSTEM & REWARD BEHAVIORS CONTRIBUTING TO QUALITY
- 108. @lisihocke → SHAPE THE SYSTEM & REWARD BEHAVIORS CONTRIBUTING TO QUALITY Experimenting on shaping the system through leadership workshops, building a foundation for quality to emerge
- 109. @lisihocke → SHAPE THE SYSTEM & REWARD BEHAVIORS CONTRIBUTING TO QUALITY Experimenting on shaping the system through leadership workshops, building a foundation for quality to emerge Shout-out to Shiva Krishnan
- 110. @lisihocke QUALITY IS ON ALL OF US
- 111. @lisihocke QUALITY IS ON ALL OF US Create transparency on the status quo, pain points and needs → The teams, the leadership, the system
- 112. @lisihocke QUALITY IS ON ALL OF US Create transparency on the status quo, pain points and needs → The teams, the leadership, the system Raise awareness regarding the options at hand to improve
- 113. @lisihocke QUALITY IS ON ALL OF US Create transparency on the status quo, pain points and needs → The teams, the leadership, the system Raise awareness regarding the options at hand to improve Experiment in your context to grow a quality culture - together