YOU Don't Need No Stinking Test Cases? - XBOSoft Webinar
3 likes900 views
The document details a webinar hosted by Xbosoft, focusing on the importance of test cases in software quality assurance. It outlines misconceptions around test cases and presents strategies for effective testing, emphasizing low-overhead formats and prioritizing crucial tests. The speakers, including experts in the field, aim to enhance understanding of test essentials and improve software testing practices.
![A Test Case Is Testing’s Basic Work Unit
What Is a Test Case?
• Essential: Inputs/conditions and expected results
(outputs and changes to stored data, environment, state)
• Test case identification
– ID, version number, name, description, resp. person
– Cross-refs to features, requirements; category
• Pre-conditions (system/configuration, repeatable initial
state, environment)
• Test procedure [advisable to keep separate]
– Set-up, environment, tools and facilities, execution steps,
results capture, environment restoration
http://itknowledgeexchange.techtarget.com/software-quality/top-ten-software-quality-tips-of-2010/](https://image.slidesharecdn.com/aug4webinar-testcasesrobingoldsmithpptr2-150805192158-lva1-app6892/85/YOU-Don-t-Need-No-Stinking-Test-Cases-XBOSoft-Webinar-11-320.jpg)
YOU Don't Need No Stinking Test Cases? - XBOSoft Webinar
- 1. XBOSoft, Inc. All Rights Reserved. 1 YOU Don’t Need No Stinking Test Cases? with Robin Goldsmith #TestCaseTips
- 2. XBOSoft Dedicated to software quality improvement Founded in 2006 We speed products to market with our expert: • Software QA consulting • Software testing Global team with offices in San Francisco,Beijing and Amsterdam “Thorough, accurate and fast” XBOSoft, Inc. All Rights Reserved. 2
- 3. House Rules Participants other than the speakers are muted Questions via the GoToWebinar control on the right side of your screen or through Twitter @XBOSoft Questions may be asked throughout the webinar - we’ll try to answer them at the end You will receive info on recording after the webinar Webinar Hashtag: #TestCaseTips XBOSoft, Inc. All Rights Reserved. 3
- 4. Meet Our Speakers Philip Lew CEO and Founder , XBOSoft • Relevant specialties and passions o Software quality process, evaluation, measurement and improvement o Software quality in use / UX design o Mobile User Experience and usability o Cycling and travel XBOSoft, Inc. All Rights Reserved. 4 Karla Smith Marketing Manager, XBOSoft • Emails you about our industry hot topic webinars • Invites you to download our latest trends in software testing whitepapers XBOSoft Team
- 5. Robin Goldsmith XBOSoft, Inc. All Rights Reserved. 5 • Author of: • ArtechHouse book Discovering REAL Business Requirements for Software Project Success • The forthcoming book Cut Creep—Put Business Back in Business Analysis • Numerous articles and featured presentations • The Proactive Testing™, Proactive SQA™, REAL ROI™, and Beyond the Textbook Software Acquisition™ methodologies • Subject expert on requirements and testing for TechTarget’s SearchSoftwareQuality.com, and subject expert/reviewer for the IIBA Business Analysis Body of Knowledge (BABOK v2) • Works directly with and trains business/systems professionals in quality and testing, requirements analysis, software acquisition, project management and leadership, metrics, ROI, and process improvement www.gopromanagement.com [email protected]
- 7. http://www.quotecounterquote.com/2010/10/we-dont-need-no-stinking-badges-or.html In the classic film The Treasure of the Sierra Madre (1948) “Badges? We ain’t got no badges! We don’t need no badges! I don’t have to show you any stinking badges!” Alfonso Bedoya, as the Mexican bandit “Gold Hat” “We don’t need no stinking badges”
- 8. Why We’re Here XBOSoft, Inc. All Rights Reserved. 8 Test cases are the fundamental unit of test work Many misunderstandings about test cases Often not recognized, which further Magnifies impacts Some think test cases must be written in a particular format, typically high-overhead script Others think they don’t need test cases at all
- 9. Objectives: You Should Be Able To • Understand test case essentials and four keys to making them effective • Gain the benefits of low-overhead test case formats • Recognize you no longer have to overlook so many important test cases Is getting enough, effective test cases an issue for you? Awareness is a start. I’ll show you how to go further.
- 10. Agenda XBOSoft, Inc. All Rights Reserved. 10 • What is a test case • High-overhead test case formats • Low-overhead test case formats • Identifying and focusing on (often-overlooked) most important tests
- 11. A Test Case Is Testing’s Basic Work Unit What Is a Test Case? • Essential: Inputs/conditions and expected results (outputs and changes to stored data, environment, state) • Test case identification – ID, version number, name, description, resp. person – Cross-refs to features, requirements; category • Pre-conditions (system/configuration, repeatable initial state, environment) • Test procedure [advisable to keep separate] – Set-up, environment, tools and facilities, execution steps, results capture, environment restoration http://itknowledgeexchange.techtarget.com/software-quality/top-ten-software-quality-tips-of-2010/
- 12. Estimation Rules of Thumb: Get Consensus--What is a Test Case? 1 of 2 Which level is the test case: A, B, C, D, or E? e.g., B=7 test cases 1. Enter an order for a customer. 1.a Existing customer. 1.a.1 Valid existing customer ID, customer is found. 1.a.2 Invalid customer ID, customer is not found. 1.b New customer. 1.b.1 Valid name and address, added. 1.b.1.1 Valid state abbreviation. 1.b.1.1.1 First state (AK). 1.b.1.1.2 Last state (WY). 1.b.1.1.3 Short state name (IA). 1.b.1.1.4 Long state name (NC). 1.b.1.1.5 Delete and re-enter (MI,MN). 1.b.1.2 Invalid state abbreviation (MM). 1.b.2 Valid name and address, not added. 1.b.2.1 Customer already exists. 1.b.2.2 No disk space. 1.b.2.3 Violates business rule, e.g., won't sell to PO Box. A B C D E A B B C C C C D D D D D D D E E E E E E E E E E E A- 1 B- 7 C-14 D-20 E-24
- 13. Estimation Rules of Thumb: Get Consensus--What is a Test Case? 2 of 2 1.b.3 Invalid state abbreviation, not added. 1.b.3.1 Two alpha characters, but not a real state abbreviation. 1.b.3.2 Lower case not accepted. 1.b.3.3 One alpha character. 1.b.3.4 Blank. 1.c Cancel the transaction, nothing ordered. 1.d Order an item (valid item number and quantity). 1.e Fail to order an item. 1.e.1 Invalid item number. 1.e.2 Invalid quantity. 1.e.3 Valid item number and quantity, none on hand. 1.e.4 Cancel the transaction. 1.f Submit the completed order (valid customer and item/quantity), ordered. 1.g Fail to complete the order. 1.g.1 Submit without valid item/quantity. 1.g.2 System crashes. B B B B B C C C C C C C C C C D D D D D D D D D D D D E E E E E E E E E E E E E
- 14. A Test Case = Inputs and/or Conditions and Expected Results • Whether or not written • Does not need to be in any specific format • Writing has benefits: – Don’t forget – Can share – Can repeat, re-use – Can review – Can refine – Guide action – Track what happened
- 15. Exploratory Testing, Error Guessing (Only Method When No Structure) • Experienced testers find two- three times as many errors with same script (Cem Kaner) • Test Manager’s challenges – Focuses at tail-end on what was written (mainly code), not main error source--what should have been (design) – Write enough to be helpful— no more, but no less – Use as a supplementary, not primary, test technique So don’t waste time writing, just go test (in context) Often believe unwritten means they don’t have test cases
- 16. How Much to Write: Keystroke-Level Procedure Embedded Within Test Case • Pro – Enables execution by low-priced people with negligible knowledge – Increases chances of precise repetition • Con – Lots of high-priced time to create and maintain – Time spent writing reduces number of tests and time for executing tests – Impedes automation – Forces execution unlike a user’s use – Virtually assures finding least amount of errors An automated test execution tool can do both: faster, cheaper, and more reliably
- 17. Keys to Effective Testing Jones, John P. “Added” $14.99 $ .75 tax Define Correctness You Must Know What the Independently of Actual Results “Right Answer” Is Follow Independent Guidelines Systematically Compare to Avoid Overlooking Things Actual to Expected Results Test Input Actual Results Expected Results Cust. #123 John P. Jones New Cust’s Redisplays screen name, address with fields cleared 10 Widgets $14.99
- 18. Defining Test Cases Test Case Specification Input and/or Condition Operator enters customer number at location X. Expected Result System looks up customer in database and displays customer name at location Y. What else do you need to perform this test?
- 19. Defining Test Cases This Way ... Test Case Specification Input and/or Condition Operator enters customer number at location X. Expected Result System looks up customer in database and displays customer name at location Y. Interruptions, delay to find data Possibility of errors Finding input data values Checking validity of results Limitations on who can find data
- 20. Specify Exact Input, Expected Result Test Case Specification Input and/or Condition Operator enters customer number at location X. Expected Result System looks up customer in database and displays customer name at location Y. Test Case Values Customer Number Customer Name C123 Jones, John P. C124 not found Low-overhead test cases. No procedure specified.
- 21. Test Script—Good to Test Navigation Input Expected Result Actual Menu=Find Customer Customer entry screen Cust. No. = C123 Cust. Name Jones, John P. Cancel button Menu Menu=Find Customer Customer entry screen Cust. No. = C124 Cust. Name Not Found Cancel button Menu Can be viewed as several simple test cases Or as a single complex test case—series of inputs/ expected results Low-overhead test cases. No procedure specified.
- 22. Test Matrix Test Input Expected Results Actual No. Cust. No. Type Active Cust. Name 1 C123 10 A Jones, John P. 2 C124 10 A not found Low-overhead test cases. No procedure specified.
- 23. Any Issues with Typical Risk Approach? Say you create 100 test cases and have time to run 10 of them. • What’s the value of the time spent on the other 90 that you don’t run? • Where did you prioritize the other test cases you didn’t think of? • Were these 100 test cases even testing the most important things? Create test cases Analyze and prioritize risks they address Run the higher risk ones
- 24. Testware--Test (Plan) Documentation per ANSI/IEEE Std. 829-2008 • Controversial standard • Frequently interpreted as mandating lots of documentation apparently for its own sake – Instead, view it as a way to organize thinking – Write just enough to be helpful, but no less – See how it can enhance agile without excess effort • Prior version hard to read, no diagrams – My diagram, phrase not in standard but fit it
- 25. Testware--Test (Plan) Documentation per ANSI/IEEE Std. 829-2008 Stds,Policies Sys.Design Project Plan Master Test Plan Bus. Reqs. Acceptance Criteria Test Designs Test Cases Test Logs Incident Rpts Test Summary Rpt Unit Test Plans Special,Sys. Test Plans Independent (QA)Test PlanIntegration Test Plans Acceptance Test Plan Independent Test Cases Acceptance Test Cases Acceptance Test Design Independent Test Design What must we demonstrate to be confident it works? Test Procedures Detailed Test Plans
- 26. Testing Structure’s Advantages 1 of 3 Systematically decompose large risks into smaller, more manageable pieces Pick view for size of need Organize and manage large set of test cases Facilitate thorough test data recreation Reactive and Proactive
- 27. Testing Structure’s Advantages 2 of 3 Show the choices for meaningful prioritization Use powerful Proactive Testing™ techniques to spot ordinarily-overlooked risks Test the biggest risks more thoroughly And earlier Focus first on larger issues, drill down later to detail Proactive
- 28. Testing Structure’s Advantages 3 of 3 Facilitate reuse Where to find Where to put How to make reusable Test cases, typically for regression tests Test design specifications Higher leverage Reactive Proactive
- 29. Test Test Test Design Case Procedure • Input/condition and expected result • What is executed • Specification (in natural language) and data values (which actually are input and expected) • Can be reusable, especially specification • Step-by-step instructions for executing test cases • Includes set-up, establishing pre- conditions • Can get to keystroke level • Often embeds input and expected result data values, which increases maintenance difficulty • Identifies a set (list) of test cases (specifications) that taken together demonstrate the feature, function, or capability works • Can be reusable or application- specific One Many One
- 30. Objectives: You Should Be Able To • Understand test case essentials and four keys to making them effective • Gain the benefits of low-overhead test case formats • Recognize you no longer have to overlook so many important test cases Is getting enough, effective test cases an issue for you? Awareness is a start. I’ll show you how to go further.
- 31. Go Pro Management, Inc. Seminars/Consulting--Relation to Life Cycle Proactive Systems/Software Quality Assurance (SQA)™ Software, Test Process Measurement & Improvement Feasibility Analysis Systems Analysis System Design Develop- ment Implement- ation Operations Maintenance Proactive Testing™: Risk-Based Test Planning, Design, and Management Testing Early in the Life Cycle Re-Engineering: Opportunities for IS Credibly Managing Projects and Processes with Metrics 21 Ways to Review Requirements Making You a Leader Managing Software Acquisition and Outsourcing: > Purchasing Software and Services > Controlling an Existing Vendor’s Performance Proactive User Acceptance Testing Reusable Test Designs Test Estimation Risk Analysis Defining and Managing Business Requirements Writing Testable SW Requirements
- 32. Post your questions on Twitter and we'll answer them @XBOSoft Join us to keep updated on all our webinars, reports and whitepapers: facebook.com/xbosoft +xbosoft linkedin.com/company/xbosoft We post regularly on our blog – check us out! http://xbosoft.com/software-quality-blog/ Why not download our free Whitepapers, available here: http://xbosoft.com/knowledge-center/ You will receive an email with information on today’s webinar slides and recording. Any further queries regarding our services or ideas for future webinars please email us! [email protected] Q+A www.xbosoft.com