Software Testing basics
Download as PPTX, PDF2 likes1,553 views
This document provides an overview of software product testing, including roles and responsibilities, development cycles, testing cycles, definitions, artifacts, tools, levels of testing, methods, techniques and types of testing. It discusses the role of testers and developers. It also includes descriptions of various testing documents, tools, levels of testing including unit, integration, system and acceptance. Different testing methods like black box, white box and gray box testing are defined. Various testing types such as load testing, security testing, compatibility testing are also outlined.
Software Testing basics
- 1. TESTING OF A SOFTWARE PRODUCT. Adapted and schematized Copyright © 2014. All rights reserved. No part of these pages, either text or image may be used for any purpose other than personal use.
- 2. I. Software Product testing: roles and responsibilities, cycles. I.I. Understanding the role of Tester -------------------------------------------------------- Slide 3 I.II. Software Product development cycles ------------------------------------------------- Slide 4 I.III. Software Product testing cycle --------------------------------------------------------- Slide 5 I.IV. Software Product testing definition ---------------------------------------------------- Slide 6 II. Software Product testing: documents (artifacts),tools, templates. II.I. Software Product testing artifacts ------------------------------------------------------- Slide 7 II.II. Software Product testing tools ---------------------------------------------------------- Slide 8 III. Software Product testing: levels, methods, technics, types III.I. Software Product development/testing is like House building --------------------- Slide 9 III.II. Manual – Automation testing ---------------------------------------------------------- Slide 10 III.III. Black Box, White Box, Gray Box testing ------------------------------------------- Slide 11 III.IV. Unit, Integration, System, Acceptance testing -------------------------------------- Slide 12 III.V. Testing levels, methods, approaches, technics, types scheme --------------------- Slide 13 III.VI. Testing short notions in alphabetical order ------------------------------------------ Slide 14-19 CONTENTS
- 3. I need a Software Product. It must be qualitative, correct, complete and secure! This is a Customer These are testers (Testing team) We will test this Software Product! These are developers We will develop this Software Product! Product Software CORRECT COMPLETE QUALITATAIVE SECURE ROLES AND RESPONSIBILITIES I will manage. Everything This is a Project Manager
- 4. SOFTWARE PRODUCT how to develop? Software Development methodology Traditional Software Development Life Cycle methodology (SDLC) Agile methodology Waterfall Spiral Incremental Prototyping Rapid Application Development (RAD) Scrum Extreme Programming (XP) Adaptive Software Development (ASD) Dynamic System Development method (DSDM) It’s our part of work! Developers
- 5. SOFTWARE PRODUCT how to test? We will develop the Software Product using Waterfall Software Development Life Cycle methodology (SDLC) Requirement Analysis Test Planning Test Development Test Execution Test Reporting Test Result Analysis Defect (bug) Retesting Regression Testing Test Closure Software Testing Life Cycle (for Waterfall Development Model) Ok! Then we test the Software Product like this Developers Testers Tester! Don’t create any testing document, don’t test anything, don’t do anything BEFORE READING THE REQUIREMENT!!! REQUIREMENTS – the requirements of a software product is a description of what the customer wants the software product be able to perform. Requirements Gathering and Analysis System Design Implementation Integration and Testing Deployment of System Maintenance Waterfall SDLC
- 6. Software testing is a process of a software product exploration. Purpose of this process is finding ERRORS in a software product (errors called BUGS) Testers check if a Developing/Developed Software product (Actual Results) match the Requirements (Expected Results) Testers identify QUALITY, CORRECTNESS, COMPLETENESS, SECURE of a developed/developing software product Testers VERIFY, VALIDATE a developed/developing software product SOFTWARE TESTING SOFTWARE PRODUCT TESTING – What’s that?
- 7. What Testing Documents do we need to test the Software Product? TESTING DOCUMENTS, aka ARTIFACTS Test Plan Test Cases Testing Effort Estimation Test Case Execution Report Defect (bug) Report Test Closure report Test Metrics Testers 1. ID, or test case number 2. Steps to reproduce 3. Expected results 4. Status (passed, failed, blocked, not executable) I. BUG Characteristics: 1. Title 2. Reporter 3. Product (component) 4. Platform 5. OS 6. Priority 7. Severity 8. Status 9. Assigned to 10. URL II. BUG description: 1. Summary (short) 2. Detailed description III. BUG pics 1. Screenshot/s SOFTWARE PRODUCT TESTING ARTIFACTS
- 8. MOST POPUAR DEFECT (BUG, ISSUE) TRACKING TOOLS BUGZILLA, web-based general purpose bug tracker. It has all the features of essence, convenience and assurance, simple to use. Free and open source. Website: http://www.bugzilla.org/. HP Quality Center. HP ALM is an end-to-end test management solution with an integrated bug tracking mechanism within it. HP Quality Center can be used on Windows platforms with an Internet Explorer browser. In combination with the HP ALM Explorer Add-in, it can be executed as a normal desktop application. It is commercial and has a free trial. Website: http://www8.hp.com/us/en/software-solutions/quality- center-quality-management/index.html?jumpid=va_R11374_us/en/large/eb/go_qc IBM Rational ClearQuest - client-server based web application that supports defect management process. It has an end-to-end, customizable defect tracking systems. It is commercial and has a free trial. Website: www.ibm.com/software/awdtools/clearquest/ JIRA - used for bug tracking, issue tracking and project management. Supports agile projects. JIRA is a commercial tool but Atlassian provides JIRA for free to open source projects that meet certain criteria, and organizations that are non-profit, non-government, non-academic, non-commercial, non-political, and secular. Website: http://www.atlassian.com/software/jira Mantis Bug Tracker, web-based bug tracking system. MantisBT supports the sending of e-mail notifications upon changes being made to issues in the system. It also has integration with Twitter via a plugin, to allow a notification to be tweeted when an issue is resolved. Free and open source. Website: http://www.mantisbt.org/. Some useful links with Testing Tools: http://www.ministryoftesting.com/resources/software-testing-tools/ big list of testing tools; http://www.softwareqatest.com/qatweb1.html list of more then 580 tools; http://en.wikipedia.org/wiki/Comparison_of_issue-tracking_systems comparison of issue tracking tools.
- 9. Let’s call a Software Product – a HOUSE Developers need to develop a Software Product = Developers need to build a House Testers need to test a Software Product = Testers need to check if the House meets the Customer/User of the House expectations or House Software Product, SOFTWARE PRODUCT CONSEPT
- 10. SOFTWARE TESTING: MANUAL, AUTOMATION MANUAL testing- tester plays the role of end user no automated tools, no scripts used when manual testing is preferred over automation? little time to automate (ex. a tight deadline to release the software UI (user interface), as it’s changed very frequently. AUTOMATION testing tester writes scripts and uses automated tools to test the software. what should be automated? automation is good for checking, verifying, validating, any tests that need to be repeated often: ex. regression tests, agile testing and eXtreme programming. Testers Software Testing
- 11. Tester pretends to be a Customer/User of the House. Tester knows NOTHING about the house building. Black- Box Testing Tester doesn’t know the internal structure of code - this method of software testing is called Black-Box Testing. TESTING METHODS BASED ON KNOWLEDGE OF CODE INTERNAL STRUCTURE Tester pretends to be a Builder of the House. Tester knows EVERYTHING about the house building. Tester knows the internal structure of code - this method of software testing is called White-Box Testing. White- Box Testing Tester pretends to be a Student of the Builder College. Tester knows SOMETHING about the house building. Tester has SOME knowledge of internal structure of code but NOT DETAILED. - this method of software testing is called Gray-Box Testing. This is a combination of Black Box and White Box testing methods. Gray- Box Testing
- 12. The BRICK of the House, Unit, need to be tested by a Builder. The smallest part/function of the Software Product is tested by: - developers, White Box testing method used + Regression UNIT TESTING LEVEL INTEGRATION TESTING LEVEL The House, System, need to be tested by a User of this House. Here, bricks, integrated together, windows, doors, weather conditions, rooms overloaded with guests ... are tested as a whole System. Testers validate the software as a whole (software, hardware and network). Testers use Black-Box testing method. Testers perform: - Functional testing, Non-Functional testing; - Regression testing, Smoke testing The smallest integrated together parts of the Software Product, are tested as a group by: 1. developers, White Box testing method used ( to ensure description of major control paths). 2. testers, Black-Box, Gray Box testing methods used. Testers perform: - Non-Functional testing; - Regression testing, Smoke testing The BRICKS, integrated into a WALL, need to be tested as a group by a User of the House. The House, System, need to be tested from the Customer eyes. Team of Builders “Will the Customer buy the House? Did we build what the Customer wanted” Customer: “Will I buy this House. Is it what I wanted?” Testers validate whether or not a system satisfies the acceptance criteria. Testers use Black-Box testing method. Testers perform: - Functional, Non-Functional testing; - Regression testing, Smoke testing SYSTEM TESTING LEVEL To buy, or not to buy? ACCEPTANCE TESTING LEVEL TESTING LEVELS: UNIT, INTEGRATION, SYSTEM, ACCEPTANCE
- 13. TESTING OF A SOFTWARE PRODUCT Static testing Dynamic testing UNIT INTEGRATION SYSTEM ACCEPTANCE Documentation Basic Path testing Branch testing Decision Coverage testing Loop testing Statement Coverage testing Data Flow testing Control Flow/Condition Coverage testing Bing Bang (Usage Model testing) Top Down testing Bottom Up testing Sandwich testing UAT (User Acceptance Testing) Alpha Testing Beta testing Operational Acceptance testing Contract Acceptance testing Regulation Acceptance testing Functional testing Non-Functional testing Scalability testing Stability testing Usability testing Vulnerabiity testing Recovery testing Compliance testing Security testing Penetration testing Globalization testing Localization testing Internalization testing (I18n ) Compatability testing Binary Portability testing Browser Compatability testing (BCT) Certification testing Performance testing Load testing Stress testing Volume testing Spike testing Configuration testing Soak testing (aka Endurance, aka Longevity, aka Aging) Component testing * Smoke testing * Regression testing * Exploratory testing Sanity testing Ad-hoc testing * GUI testing These testing works can be performed at every testing level Buddy testing Pair testing Monkey testing Black Box testing Gray Box testing White Box testing White Box testing Black Box testing Black Box testing
- 14. TESTING SHORT NOTIONS IN ALPHABETICAL ORDER • Acceptance testing – testing performed by a tester or end customers to check if the software confirms to business needs and to the requirements provided earlier. • Ad-hoc testing – testing with purpose to break the software and find defects with the experience and knowledge they have about the domain, requirements and functionality of the software. Ad-hoc testing is intended to find defects that were not found by existing test cases. Ad-hoc testing is quite appropriate to use in any stage of the software development cycle. Ad-hoc testing includes: Buddy testing, Pair testing, Monkey testing. • Alpha testing – testing performed by a tester or end customers at development site. The main purpose of alpha testing is to refine the software product by finding (and fixing) the bugs that were not discovered through previous tests. • Basic path testing – testing performed to check code for every execution path of code at least once; performed by developers. • Beta testing – testing performed by a tester or carried out by end customers before releasing or handing over software to end users. • Binary Portability testing – testing with the purpose to check compatibility of binary or executable files from one hardware and/or operating system to another. • Bing Bang testing - testing of all components or modules integrated simultaneously, after that everything is tested as a whole. • Black Box testing – testing without knowledge of internal code structure. • Bottom Up testing - testing where the lowest level components are tested first, then used to facilitate the testing of higher level components. The process is repeated until the component at the top of the hierarchy is tested. • Branch testing – testing during Unit level testing. The purpose to check code for every branching condition.
- 15. • Browser Compatibility testing – testing performed for web applications with combination of different browsers and operating systems. • Buddy testing - testing in “a buddy pair”, such pair consists of a tester and a developer. A tester and a developer work on identifying defects in the same module. Buddy testing helps the testers develop better test cases and development team can also make design changes early. This testing usually happens after unit testing completion. • Certification testing – testing if a software product is applicable for hardware, different operating systems, browsers. • Compatibility testing – testing with the purpose to check if the software can be run on different hardware, operating system, bandwidth, databases, web servers, application servers, hardware peripherals, emulators, different configuration, processor, different browsers and different versions of the browsers etc. • Compliance testing – testing with the purpose to check if a software product meets the required standards, government laws, company policies. • Component testing – testing of a group of units as code together. • Configuration testing – testing with the purpose to find optimal configuration settings that can make software perform at its best for hardware or for a given operating system. • Contract Acceptance testing – testing of a system against acceptance criteria as documented in a contract, before the system is accepted. • Data flow testing – testing aims to explore sequences of events related to the status of variables or data objects. Dataflow Testing focuses on the points at which variables receive values and the points at which these values are used. • Decision Coverage testing (aka Branch Coverage testing, aka All-Edges Coverage testing) – testing aims to validate that all the branches in the code are reached, that no branches lead to any abnormality of the program’s operation. • Dynamic testing - testing that involves the execution of a software product by giving set of inputs, examined it's output and compared what is expected.
- 16. • Exploratory testing - testing conducted to learn the software at the same time to find f errors that seem non-obvious. Usually done by testers but can be done by other stake holders as well as business analysts, developers, end users. Ad-hoc testing is one of the Exploratory testing variants. • Functional testing - testing software against design document, use cases and requirements document. • Fuzz testing – testing with the purpose to check a software product for failures or error messages that are presented due to the input errors. Unexpected or random inputs are used. • Globalization testing - testing validates whether the application is capable for using all over the world and to check whether the input accepts all the language texts. The purpose of testing is to check whether software can perform properly in any locale or culture & functioning properly with all types of international inputs and steps to effectively make your product truly global. • Gray Box testing – testing with some knowledge of internal code but the knowledge is not detailed. • Graphical User Interface testing (GUI) - testing that involves checking the screens with the controls like menus, buttons, icons, and all types of bars - tool bar, menu bar, dialog boxes and windows. So GUI testing ensures GUI elements of the software are as per approved GUI mockups, detailed design documents and functional requirements. GUI testing can be performed at any level of testing: Unit, Integration, System, Acceptance. • Installation testing – testing that involves executing software installer in different modes like “express” or “custom” modes and on different types of environments like virtual operating systems and on different operating systems. • Integration testing – testing with the aim to check the connectivity among integrated together units/component. Individual software modules are combined and tested as a group. • Internationalization testing - testing with the aim to check the extent to which software can support usage of different languages, different character sets and so on.
- 17. • Load testing - testing intends to find bottlenecks or issues that prevent software from performing as intended at its peak workloads. This testing activity is performed to check the behavior of the software under normal and over peak load conditions. Ideal for automation. • Localization testing - testing with the purpose to validate whether application is capable enough for using in a particular location or country. • Loop testing – testing that concentrates on the validity of loop constructs. • Monkey testing – testing the product or application without test cases with a goal to break the system, testing is random. • Negative testing - testing that is intended to break the software by entering incorrect data like incorrect date, time or string or upload binary file when text files supposed to be upload or enter huge text string for input fields etc., • Non-Functional testing – testing of a software product upon the requirements which are non functional in nature related but important a well such as performance, security, user interface etc. • Operational Acceptance testing – (aka Operational Readiness testing) testing with the purpose to check a system to ensure that processes and procedures are in place to allow the system to be used and maintained. This type of testing includes checks done to back-up facilities, procedures for disaster recovery, training for end users, maintenance procedures, and security procedures. • Pair testing – testing of a software product when two testers work on the same machines to find defects. One does the testing and the other analyzes or reviews the testing. This can be done between a tester and developer or business analyst or between two testers. Pair testing is done when testers have different levels of a software product knowledge. • Penetration testing - is done to check how secure software and its environments (hardware, operating system and network) are when subject to attack by an external or internal intruder.
- 18. • Performance testing - testing checks how well software works in anticipated and peak workloads. There are very many variants of Performance testing like Load testing, Stress testing, Volume testing, Soak testing, Configuration testing. • Positive testing - testing with the purpose to check the application’s functions with valid input and also to verify that the outputs are correct. • Recovery testing - testing performed by software testers. Recovery testing performs with the purpose to check how soon and how efficiently software can recover from software crashes, operating system crashers, and hardware failures. • Regression testing – testing with the purpose to find defects that got introduced to defect fix(es) or introduction of new feature(s). Regression testing is ideal for automation. Sanity testing is a variant of Regression. • Regulation Acceptance testing – testing of a system to ensure it meets governmental, legal and safety standards. • Sandwich testing – testing combination of Top Down testing with Bottom Up testing. • Sanity testing – testing with the purpose to evaluate if the software, environment, network, external systems, software environment as a whole is stable enough to proceed with extensive testing. • Scalability testing - of a software application for measuring its capability to scale up in terms of any of its non-functional capability like load supported, the number of transactions, the data volume and so on. • Security testing – testing checks, how good is software’s authorization mechanism, how strong is authentication, how software maintains confidentiality of the data, how does the software maintain integrity of the data, what is the availability of the software in an event of an attack on the software by hackers and malicious programs. • Smoke testing - testing with the aim to find “show stopper” defects that can prevent testers from testing the application in detail. Testers check if the new build provided by development team is stable enough i.e., major functionality is working as expected in order to carry out further or detailed testing. • Soak testing (aka Endurance testing, aka Longevity testing, aka Aging testing) - testing that is conducted to find errors that result in degeneration of software performance with continued usage, soak testing may go on for few days or even for few weeks. Soak testing is extensively done for electronic devices, which are expected to run continuously for days, or months, or years without restarting or rebooting.
- 19. • Spike testing - testing with the purpose to check how software responds to workloads that are sent in very short span of time and which are not constant over a period of time. • Stability testing – testing with the purpose to check how stable software is when it is subject to loads at acceptable levels, peak loads, loads generated in spikes, with more volumes of data to be processed. • Statement Coverage testing – (aka Line Coverage testing, aka Segment Coverage testing). Testing with the purpose to check the statements executed and where the code is not executed because of blockage. This testing covers only the true conditions. • Static testing - testing when software code is not executed instead it is reviewed for syntax, commenting, naming convention, size of the functions. Static testing can be applied for requirements, designs, test cases by using approaches like reviews or walkthroughs. • Stress testing - is a type of performance testing, in which software is subjected to peak loads and even to a break point to observe how the software would behave at breakpoint. Stress testing also tests the behavior of the software with insufficient resources like CPU, Memory, Network bandwidth, Disk space etc. Stress testing enables to check some of the quality attributes like robustness and reliability. • System testing – testing with the purpose to find errors related to integration among different applications, services, third party vendor applications. • Top Down testing – testing where the top integrated modules are tested and the branch of the module is tested step by step until the end of the related module. • Unit testing – testing performed by developers. The developer tests units of source code like statements, branches, functions, methods, interface. Unit testing is ideal for automation. • Usability testing – testing performed to understand how user friendly the software is. • User Acceptance testing (UAT) – testing with the purpose to check the software product with their actual business or real-world scenarios and to check if the software meets their business requirements. • Volume testing – testing with the purpose to check the response of the software with different sizes of the data being received or to be processed by the software. • Vulnerability testing – testing that involves identifying, exposing the software, hardware or network. • White Box testing – testing with code structure knowledge. HAPPY TESTING!