Agile Testing: The Role Of The Agile Tester
Download as PPTX, PDF102 likes47,545 views
The presentation by Declan Whelan on agile testing outlines the principles of agility, success factors for agile testing, and the roles of testers within agile teams. It emphasizes the importance of collaboration, continuous integration, and adapting testing practices to promote quality throughout the software development process. Additionally, the document discusses session-based testing as a method for auditable exploratory testing and the role of automated tests in supporting rapid iterations.
Agile Testing: The Role Of The Agile Tester
- 1. Agile Testing and the Role of the Agile Tester Declan Whelan [email protected] This presentation is licensed under a Creative Commons Attribution 2.5 License, which means you can share and adapt it, including commercial and derivative works, as long as you include attribution to Declan Whelan.
- 2. Agenda • What is Agility? • Agile Testing Success Factors • Agile/Scrum Overview • What is Quality? • Context-Driven School of Testing • Session-Based Testing & Low-Tech Dashboards • Agile Quality – Agile Testing Matrix – Agile Tester Activities – Test Automation
- 3. Notes on this Presentation This presentation was originally delivered at the QUEST Toronto 2008 conference. Prior to the conference it was discovered that information on session based testing was not properly attributed to James Bach (http://www.satisfice.com). Unfortunately an updated presentation was not available in time to be included on the USB sticks provided. This updated presentation correctly attributes this material to James and also includes some minor formatting changes. Declan wishes to apologize to James for the omission and to thank him both for his understanding and his contributions to exploratory testing.
- 4. Agile Testing Success Factors Testers are part • Be cathedral builders not stone cutters of the team • Collective ownership Agile testing • Drop the “Quality Police” mindset mindset • Focus on team goals & customer value • Automate tests wherever practical Automate tests • Need rapid feedback Look at the big • Balance against developer focus on technical implementation picture • Use agile test matrix as guide Source: http://www.agiletester.ca/
- 5. Agile Testing Success Factors • Session-based testing, Foundation of agile test environments critical practices • Informative workspace • Collaborate with customers Collaborate • Collaborate within team • Team retrospectives Continually • Personal training: reading, blogs, improve QAI, local QA groups Source: http://www.agiletester.ca/
- 6. Traditional Approach - Waterfall Requirements Analysis & Code Test Deploy Cost • Business Design • C#, C, C++ etc. • Validation Tests Of Requirements • System • Big-Bang • Verification • Technical Specifications Integration Tests Change Requirements • Component Specifications Time
- 7. Agile Approach Iteration 1 Iteration 2 Iteration 3 Iteration 4 Requirements Requirements Requirements Requirements Analysis & Design Analysis & Design Analysis & Design Analysis & Design Cost of Code Code Code Code Change Test Test Test Test Deploy Time
- 8. What is Agility? “Values, principles and practices that foster team communication and feedback to regularly deliver customer value through working software.”
- 9. Agile Values Individuals & Interactions Working Software Comprehensive Processes & Tools Documentation Customer Collaboration Responding to Change Contract Negotiation Following a Plan http://agilemanifesto.org/
- 11. Quarry
- 12. Lean
- 13. Engineering
- 14. XP – Customer Bill of Rights • You have the right to an overall plan, to know what can be accomplished, when, and at what cost. • You have the right to see progress in a running system, proven to work by passing repeatable tests that you specify. • You have the right to change your mind, to substitute functionality, and to change priorities. • You have the right to be informed of schedule changes, in time to choose how to reduce scope to restore the original date. You can even cancel at any time and be left with a useful working system reflecting investment to date.
- 15. XP – Tester Bill of Rights • You have the right to bring up issues related to quality and process at any time. • You have the right to ask questions of customers and programmers and receive timely answers. • You have the right to ask for and receive help from anyone on the project team, including programmers, managers and customers. • You have the right to make and update your own estimates for your own tasks and have these included in estimates for stories. • You have the right to the tools you need to do your job in a timely manner. • You have the right to expect your project team, not just yourself, to be responsible for quality. Source: http://home.att.net/~lisa.crispin/XPTesterBOR.htm
- 17. Story Card Confirmation Conversation
- 18. What is a Story? • Functionality that Independent provides value Negotiable • Written in business (domain) language Valuable • Ideally < 2 days work Estimable • “As a <role>, I want to <feature> so that Small <benefit>” Testable
- 19. Story Card • The “headline” for some new functionality. • It is an “invitation to a conversation”. • Written in business (domain) language. • It is not a specification … specifications are documented using story tests. • Stories can be used to drive additional work such as building test scripts/frameworks
- 20. Story Conversation • Exchange of thoughts, opinions, and feelings. • Takes place over time, particularly when the story is estimated. • Conversations also occur at the iteration planning meeting when the story is scheduled for implementation. • Supplement with documents as needed.
- 21. Story Confirmation • Story tests are acceptance tests for the product owner • Product owner specifies the story tests but will collaborate with team to create them • Team can add additional tests • Most tests can and should be automated (e.g. FIT, xBehave, xSpec, stress tests, load tests)
- 22. Task • Some atomic unit of work that can be “done”. • Development estimate includes design, development, unit test, refactoring, check in • Usually maintained on a whiteboard or bulletin board for wide visibility • Add testing tasks for non-functional “ility” testing • Typically 1-4 hours effort
- 24. Scrum Team Product ScrumMaster Team Owner • Feature • Represents • Self organizing definition management • Cross- • Release dates • Removes functional • Single decision obstacles • Estimates point • Ensures Scrum • Tracks • Accepts or process • Gets ‘er done rejects work • Servant leader • ROI
- 25. Product Backlog • Master list of all “features” • High priority features are split into “stories” achievable within an iteration. • Each “story” is prioritized and scoped.
- 26. Sprint Planning Meeting • Highest priority stories are reviewed. • Team selects stories • Team breaks stories down into tasks & re-estimates. • Team commits to next iteration’s deliverables.
- 27. Sprint Backlog
- 28. Daily Scrum • Each team member describes: – What they did – What they plan to do – Obstacles • ScrumMaster tracks and resolves obstacles • 10 – 15 minutes
- 29. Sprint Demo • Team demonstrates working software to product owner • Product owner accepts or rejects completed work • Result should be potentially shippable
- 30. Sprint Retrospective • Team meets to review: – What is working? – What is not working? • Team adds tasks for immediate actions for working better
- 31. What is Quality? Jerry Weinberg: “Value to some person(s).”
- 32. Testing Schools of Thought • How much has • Controlled process • Focus on QA vs. been tested? • Formal rules testing • Objectivity • Standards & • Process • Branch of certification watchdog CS/Math • Conformance focus • Risk driven • QA signoff Analytic Factory Quality • Skilled mental activity • People set context Agile Helps • Testing provides information Set Context • Value driven Context Driven Source: http://www.qaiasia.com/bpw/innerpages/documents/Process_Improvement/Model_and_Framework/PI_007-shrinivas_kulkarni.pdf
- 33. Context-Driven Principles 1. The value of any practice depends on its context. 2. There are good practices in context, but there are no best practices. 3. People, working together, are the most important part of any project's context. 4. Projects unfold over time in ways that are often not predictable. 5. The product is a solution. If the problem isn't solved, the product doesn't work. 6. Good software testing is a challenging intellectual process. 7. Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products. Source: http://www.context-driven-testing.com
- 34. Session Based Testing A method specifically designed to make exploratory testing auditable and measurable on a wider scale. Source: http://en.wikipedia.org/wiki/Exploratory_testing
- 35. Session Based Structure Charter Parsing Session Results Debrief Report
- 36. Testing Dashboard Area Effort Assessment Coverage Comments Source: http://www.satisfice.com/rst.pdf
- 37. Agile Quality – A Team Deliverable Agile Practice Benefits Whole Team • Quality is not just a tester responsibility • Quality is more than just testing • Testing role shifts to quality infusion throughout project life cycle Continuous • Developers cannot check in code Integration with failing tests Continuous • Avoids long delays with “big-bang” Testing testing after the “final build” • Bugs found closer to when they are introduced making them easier to fix
- 38. Agile Testing Challenges • Team may not value testers • Testers may not value team • Unclear role of testers on team • Testing is often squeezed as deadlines approach • Developers and testers are often in different operational silos • Team may not have the skills or domain expertise to develop/test effectively
- 39. Agile Testing Approach • Testers are first class citizens on agile teams and part of the “whole team” supporting customers, business stakeholders, developers and other team members • Testers support quality infusion through entire team and product cycle • Test tasks and stories are planned and executed like development tasks and stories • Automate where possible and use session-based testing for exploratory testing • Communicate through information radiators
- 40. Brian Marick’s Agile Testing Matrix Automate Customer Facing Manual Functional Tests User Acceptance Tests Customer Tests Exploratory Tests Supports Development Critiques Product Story Tests/Examples Usability Tests Q2 Q3 Q1 Q4 Unit Tests Performance Tests Integration Tests Load Tests Automate Technology Facing Tools
- 41. Tester Activities Product Owner Customer Customer Facing Collaboration Collaboration Product Specifications UAT Design Supports Development Test Ideas Exploratory Testing Critiques Product Testing Q2 Q3 Usability Testing Test Ideas Q1 Q4 Test Scripts Test Development Testing Testing Test Analysis Developer IT Technology Facing Collaboration Collaboration
- 42. Agile Testing Iterations Q3, Q4: Q1: Q2: Product Testing & Planning & Testing Collaboration Test Ideas Working Working Working Stories Stories Stories Product Product Product Previous Current Next Iteration Iteration Iteration
- 43. Why Automate Tests? • Provides safety net • Supports rapid iterations • Provides footholds to keep notching upward • Provides rapid feedback • Focuses effort on what is valuable • Frees people to do their best work Need to balance automation costs against delivered value
- 44. Types of Automated Tests TestComplete UI Tests Through UI SilkTest TestDirector c Functional In business Fit/FitNesse o language Tests s Unit Tests In same language as system xUnit t
- 45. Agile Testing Success Factors Testers are part • Be cathedral builders not stone cutters of the team • Collective ownership Agile testing • Drop the “Quality Police” mindset mindset • Focus on team goals & customer value • Automate tests wherever practical Automate tests • Need rapid feedback Look at the big • Balance against developer focus on technical implementation picture • Use agile test matrix as guide Source: http://www.agiletester.ca/
- 46. Agile Testing Success Factors • Session-based testing, Foundation of agile test environments critical practices • Informative workspace • Collaborate with customers Collaborate • Collaborate within team • Team retrospectives Continually • Personal training: improve reading, blogs, QAI, local QA groups Source: http://www.agiletester.ca/
- 47. Agile Testing Reading List • Agile Project Management Jim Highsmith; 2004 • Lean Software Development Mary & Tom Poppendieck; 2003 • Extreme Programming Explained 2nd Edition Kent Beck, Cynthia Andres; 2004 • Testing Extreme Programming Lisa Crispin; 2002 • Agile Testing: The Tester Role on an Agile Project Lisa Crispin, Janet Gregory; Can pre-order on Amazon
- 48. Web Sites • http://www.exampler.com (Brian Marick) • http://www.developsense.com (Michael Bolton) • http://www.satisfice.com (James Bach) • http://www.context-driven-testing.com • http://www.kohl.ca/blog (Jonathon Kohl) • http://tech.groups.yahoo.com/group/agile-testing • http://www.agiletester.ca/ (Lisa Crispin, Janet Gregory) • www.agilemanifesto.com • www.agilealliance.org
- 49. Session Based Testing Extras The following additional information on session based testing was primarily obtained or derived from a set of slides called “Rapid Software Testing” by James Bach. Source: http://www.satisfice.com/rst.pdf
- 50. Session Based Structure Charter Parsing Session Results Debrief Report
- 51. Goals Find bugs fast Fix bugs fast – through rapid testing feedback Visibility Transparency Auditability Maximize testing value
- 52. Charter A goal or agenda for a test session in 1 to 3 sentences. Created by the test team prior to the start of testing, but may be added or changed at any time. Often created from a specification, test plan, or by examining results from previous test sessions. Source: http://en.wikipedia.org/wiki/Session-based_testing#Charter
- 53. Session An uninterrupted period of time spent testing, ideally lasting one to two hours. Each session is focused on a charter, but testers can also explore new opportunities or issues during this time. The tester creates and executes test cases on-the-fly and records their progress. Source: http://en.wikipedia.org/wiki/Session-based_testing#Session
- 54. Report • Charter • Any files used • Areas Tested • % of time spent on • Detailed notes on how charter vs. other: testing was conducted • Session start time and • A list of any bugs found duration • A list of issues: • Tester name – open questions – product concerns – process concerns Source: http://en.wikipedia.org/wiki/Session-based_testing#Session_Report
- 55. Debrief A debrief is a short discussion between the manager and tester (or testers) about the session and the session report. Source: http://en.wikipedia.org/wiki/Session-based_testing#Session_Report
- 56. Parsing Results With a standardized Session Report, software tools can be used to parse and store the results as aggregate data for reporting and metrics. This allows reporting on the number of sessions per area or a breakdown of time spent on testing, bug investigation, and setup / other activities. Source: http://en.wikipedia.org/wiki/Session-based_testing#Parsing_Results
- 57. Low Tech Testing Dashboard Source: http://www.satisfice.com/rst.pdf
- 58. The Test Status Problem “What’s the status of testing?” “What are you doing today?” “When will you be finished?” “Why is it taking so long?” “Have you tested _______, yet?” Source: http://www.satisfice.com/rst.pdf
- 59. Testing Reporting Challenges • Management has little patience for detailed test status reports. • Management doesn’t understand testing: • Testing is confused with improving. • Testing is considered a linear, independent task. • Testing is assumed to be exhaustive. • Testing is assumed to be continuous. • Test results are assumed to stay valid over time. • Impact of regression testing is not appreciated. • Test metrics are hard to interpret. Source: http://www.satisfice.com/rst.pdf
- 60. A Low Tech Dashboard Solution Report test cycle progress in a simple, structured way... ...that shows progress toward a goal... ... manages expectations... ...and inspires support... ...for an effective test process. Source: http://www.satisfice.com/rst.pdf
- 61. Testing Dashboard Area Effort Assessment Coverage Comments Source: http://www.satisfice.com/rst.pdf
- 62. The Dashboard Concept Source: http://www.satisfice.com/rst.pdf
- 66. Quality Assessment Source: http://www.satisfice.com/rst.pdf