Architecting A Testing Strategy
1 like201 views
The document outlines a comprehensive test strategy emphasizing the importance of clarity, team alignment, and risk management in testing processes. It discusses automation, the test pyramid, and the distinct roles of team members in writing and executing tests, ultimately promoting collective ownership and effective communication. Additionally, it highlights the significance of agile strategies and regular evaluations to adapt testing practices to evolving project needs.
Architecting A Testing Strategy
- 2. TEST STRATEGY Guiding principles that provide focus, context, and drive actions of the team to manage risk @theBConnolly
- 3. Testing isn’t always well understood That leads to feelings of fear When things go wrong scared people often look to blame. AM BIGUITY UN CERTAIN TY BLAME WHY STRATEGY MATTERS @theBConnolly
- 4. People love knowing there is a plan Testing means different things to different people Move together towards quality CLARITY ALIGN M EN T QUALITY WHY STRATEGY MATTERS @theBConnolly
- 6. How much time should we spend writing, maintaining, debugging automation? How many tests should we automate? How long should it take to run? What tests should we automate? What isn’t a good fit for automation? LOOKING FOR ANSWERS TIME QUANTITY CHOICE @theBConnolly
- 7. TEST MONUMENTS What can all these Test Automation Pyramids tell us? @theBConnolly
- 8. TEST PYRAMID MATHEMATICS Treating the Test Pyramid as a triangle Use geometry & trigonometry to find the total area of each level @theBConnolly
- 9. For the Original 3 Layer Test Pyramid Assuming Each Layer Is Equally Tall Using Different Dimensions the Results Are Uniform @theBConnolly TEST PYRAMID MATHEMATICS RESULTS 3 levels UI Service Unit Great Pyramid 11.1% 33.3% 55.6% Equilateral 11.1% 33.4% 55.5% 45-45-90 11.17% 33.21% 55.62%
- 10. Same Assumptions... More Levels @theBConnolly TEST PYRAMID MATHEMATICS RESULTS 4 levels UI Component Integration Unit Great Pyramid 6.3% 18.7% 31.3% 43.7% Equilateral 6.3% 18.8% 31.3% 43.7% 45-45-90 6.18% 18.79% 31.15% 43.88% 5 levels UI API Component Integration Unit Great Pyramid 4% 12% 19.8% 28.2% 36%
- 11. We can agree about the Top We can agree about the Bottom We undervalue the Delicious Goodness in the Middle @theBConnolly THE GOOEY CENTER Unit Gooey Center UI 36-55% 33-60% 4-11%
- 12. A QUESTION OF QUANTITY Per the Test Pyramid for every 100 tests added you would have: - 35-55 unit tests, - 33-60 service tests - 4-11 UI / E2E tests. Does that fit with the practices on your team? @theBConnolly
- 13. @theBConnolly A QUESTION OF TIME Per the Test Pyramid for a 40 hour work week: - 18 hours writing & maintaining unit level tests - 18 hours writing & maintaining service level tests - 4 hours writing & maintaining UI level tests.
- 14. A QUESTION OF EFFORT Using Effort for estimation brings automation in line with the feature development. Point Stories Inclusively but use automation effort a metric for comparison. @theBConnolly
- 15. THE HANDFUL HEURISTIC For a given area of your product you only need a handful of end to end tests There is great power in a handful of meaningful & reliable tests @theBConnolly
- 16. CONFIRM & CHARACTERIZE Write the Tests that: - Confirm that the Application is Releasable - Characterize Key Behaviors @theBConnolly
- 18. ROLE OF THE TEAM PERSONAS Your Team is Unique To uncover the best automation strategy Understand your team first @theBConnolly
- 19. The person or group responsible for writing test automation The person or system responsible for running the automated test suite The person or system that derives value from the results of test execution AUTOMATION PERSONAS CREATOR EXECUTOR CONSUMER @theBConnolly
- 20. Writes tests while developing new functionality Runs tests as changes are made Test results inform and guide decisions through development process Developer Developer Developer AUTOMATION PERSONAS UNIT TESTS CASE #1 EXECUTOR CONSUMERCREATOR @theBConnolly
- 21. Writes tests while developing new functionality When code is checked in tests are run and results collected Team knows their individual changes are not unintentionally affecting others Developer CI/CD Pipeline Dev Team AUTOMATION PERSONAS UNIT TESTS CASE #2 EXECUTORCREATOR CONSUMER @theBConnolly
- 22. Writes tests while developing new functionality When code is checked in tests are run and results collected Tooling consumes or analyzes to derive actionable information Developer CI/CD Pipeline Metrics AUTOMATION PERSONAS UNIT TESTS CASE #3 EXECUTORCREATOR CONSUMER @theBConnolly
- 23. Contextual Benefit - Direct & Meaningful Impact on Day to Day Work Collective Ownership - Team comes together to support quality outcomes Multiple Consumers - Value increased and costs reduced as tests provide multiple streams of value. @theBConnolly UNIT TEST VALUE CHARACTERISTICS
- 24. QA / Tester writes automated tests Tests are run on some cadence. Before release, nightly build etc. Who is this for? How Do they benefit? QA Schedule ??? AUTOMATION PERSONAS UI / E2E TESTS EXECUTORCREATOR CONSUMER @theBConnolly
- 25. QA / Tester writes automated tests Tests are run on some cadence. Before release, nightly build etc. QA / Tester reviews the test results focusing on tests they own QA Schedule QA AUTOMATION PERSONAS UI / E2E TESTS CASE #1 EXECUTORCREATOR CONSUMER @theBConnolly
- 26. A dedicated team of automators writes tests QA / Tester runs the tests as needed based on release schedule Managers look at test result reports tracking trends and metrics. Automation Team QA Management AUTOMATION PERSONAS UI / E2E TESTS CASE #2 EXECUTORCREATOR CONSUMER @theBConnolly
- 27. Contextual Benefit - Ensure Tests Provide Direct & Meaningful Value Collective Ownership - It Takes a Village to Support Automation Multiple Consumers - Identify Questions That Automation Can Answer for Consumers @theBConnolly UI / E2E TEST SUCCESS CRITERIA
- 29. A MODEL OF ISOLATION THERE IS ALWAYS A BROADER CONTEXT WE TEST TO UNDERSTAND THE REALITY OF THAT CONTEXT @theBConnolly
- 31. Just like the Test Pyramid a Quality Cathedral is built upon a solid foundation of unit tests QUALITY CATHEDRAL FOUNDATION @theBConnolly
- 32. Focused On Building A Quality Application that Inspires & Engages Our Users QUALITY CATHEDRAL APPLICATION CENTRIC @theBConnolly
- 33. Higher Level Tests Stand Alongside the Application Supporting the It Exactly Where It Needs it QUALITY CATHEDRAL TARGETED SUPPORTS @theBConnolly
- 34. Free Standing Supports that reinforce lower level tests QUALITY CATHEDRAL BUTTRESSING QUALITY @theBConnolly
- 36. A Network of Tests that work together to withstand the demand & stress your product faces QUALITY CATHEDRAL SCALE & WITHSTAND @theBConnolly
- 37. You know your product You know your team Target where your application needs support with your resources to efficiently manage risk QUALITY CATHEDRAL YOUR TEAM YOUR SUPPORTS @theBConnolly
- 39. WATCHING OUT FOR WATERFALLS Heavy Documentation Explicit Processes Bureaucracy These do not guarantee better outcomes Strategy needs to be Agile too @theBConnolly
- 41. TEST STRATEGY CHARTERS Use (Testing / Automation Technique) Led By (Owner / Creator) To Support (Desired Outcomes) @theBConnolly
- 42. TEST STRATEGY CHARTER EXAMPLES Use Unit Testing Led By Developers To Support a healthy codebase and easier refactoring Use End 2 End UI Tests Led By QA with Dev support To Support Exploratory testing on stable builds @theBConnolly
- 43. STRATEGY EVALUATION EVENTS When: - Projects Kickoff / End - Team Members Change - Incidents / Outages - Team Retro’s Discuss, Evolve, Reflect @theBConnolly
- 44. TEST STRATEGY RETROSPECTIVE Even without a documented test strategy, people will have their own ideas about what types of testing activities should or should not be occurring. It is worth periodically checking that everyone is on the same page. - Katrina Clokie https://leanpub.com/testingindevops @theBConnolly
- 45. BUILDING YOUR STRATEGY Your App, Your Team, Your Needs Empower Quality @theBConnolly
- 46. BE WELL, CREATE, SHARE Connect With Me @theBConnolly http://brendanconnolly.net