Software verification & validation

Software verification & validation

• 1.
  Software Verification &Validation By Muhammad Anees Iqbal
• 2.
  Introduction In software projectmanagement, software testing, and software engineering, verification and validation (V&V) is the process of checking that a software system meets specifications and that it fulfills its intended purpose. It may also be referred to as software quality control. It is normally the responsibility of software testers as part of the software development lifecycle.
• 3.
  Software Verification &Validation
• 4.
  Software Verification Verification makessure that the product is designed to deliver all functionality to the customer. The process of evaluating software to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase.
• 5.
  Software Verification Verification isdone at the starting of the development process. It includes reviews and meetings, walkthroughs, inspection, etc. to evaluate documents, plans, code, requirements and specifications.
• 6.
  Software Verification Suppose youare building a table. Here the verification is about checking all the parts of the table, whether all the four legs are of correct size or not. If one leg of table is not of the right size it will imbalance the end product. Similar behavior is also noticed in case of the software product or application. If any feature of software product or application is not up to the mark or if any defect is found then it will result into the failure of the end product. Hence, verification is very important. It takes place at the starting of the development process.
• 7.
  Software Verification It answersthe questions like: Am I building the product right? Am I accessing the data right (in the right place; in the right way). It is a Low level activity Performed during development on key artifacts, like walkthroughs, reviews and inspections, mentor feedback, training, checklists and standards. Demonstration of consistency, completeness, and correctness of the software at each stage and between each stage of the development life cycle.
• 8.
  Software Verification According tothe Capability Maturity Model (CMM) we can also define verification as the process of evaluating software to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. [IEEE-STD-610].
• 9.
  Software Verification Verification isa static practice of verifying documents, design, code and program. It includes all the activities associated with producing high quality software: inspection, design analysis and specification analysis. It is a relatively objective process. Verification will help to determine whether the software is of high quality, but it will not ensure that the system is useful. Verification is concerned with whether the system is well- engineered and error-free.
• 10.
  Methods of Verification StaticTesting Under Static Testing, code is not executed. Rather it manually checks the code, requirement documents, and design documents to find errors. Hence, the name "static".
• 11.
  Methods of Verification Staticverification is an analysis of computer code to ensure that standard coding practices have been adhered to without having to run the program. An analysis is performed on some versions of the source code and provides a way for programmers to debug new code and detect potential errors in compiled code.
• 12.
  Methods of Verification Staticverification is widely used in software for safety-critical computer systems. The key technologies used in static verification are static timing analysis and equivalence checking. Timing and functional verification in a static verification flow are separated and run in parallel. Static timing analysis provides timing checks, while equivalence checking verifies the functional equivalence of two versions of the same circuit as design scales through different transformations such as scan chain reordering, routing and placement.
• 13.
  Methods of Verification Themain objective of this testing is to improve the quality of software products by finding errors in the early stages of the development cycle. This testing is also called a Non-execution technique or verification testing.
• 14.
  Methods of Verification Statictesting involves manual or automated reviews of the documents. This review is done during an initial phase of testing to catch defect early in STLC. It examines work documents and provides review comments
• 15.
  Methods of Verification Examplesof Work documents- • Requirement specifications • Design document • Source Code • Test Plans • Test Cases • Test Scripts • Help or User document • Web Page content
• 16.
  Static Verification Techniques InformalReviews This is one of the type of review which doesn't follow any process to find errors in the document. Under this technique, you just review the document and give informal comments on it.
• 17.
  Static Verification Techniques TechnicalReviews A team consisting of your peers, review the technical specification of the software product and checks whether it is suitable for the project. They try to find any discrepancies in the specifications and standards followed. This review concentrates mainly on the technical documentation related to the software such as Test Strategy, Test Plan and requirement specification documents.
• 18.
  Static Verification Techniques Walkthrough Theauthor of the work product explains the product to his team. Participants can ask questions if any. A meeting is led by the author. Scribe makes note of review comments.
• 19.
  Static Verification Techniques Inspection Themain purpose is to find defects and meeting is led by a trained moderator. This review is a formal type of review where it follows a strict process to find the defects. Reviewers have a checklist to review the work products. They record the defect and inform the participants to rectify those errors.
• 20.
  Static Verification Techniques Staticcode Review This is a systematic review of the software source code without executing the code. It checks the syntax of the code, coding standards, code optimization, etc. This is also termed as white box testing. This review can be done at any point during development.
• 21.
  Advantages of SoftwareVerification • Verification helps in lowering down the count of the defect in the later stages of development. • Verifying the product at the starting phase of the development will help in understanding the product in a better way. • It reduces the chances of failures in the software application or product. • It helps in building the product as per the customer specifications and needs.
• 22.
  Software Validation Validation isdetermining if the system complies with the requirements and performs functions for which it is intended and meets the organization’s goals and user needs. The process of evaluating software during or at the end of the development process to determine whether it satisfies specified requirements.
• 23.
  Software Validation Validation isthe process of evaluating the final product to check whether the software meets the customer expectations and requirements. It is a dynamic mechanism of validating and testing the actual product.
• 24.
  Software Validation Validation isdone at the end of the development process and takes place after Verifications are completed. It answers the question like: Am I building the right product? Am I accessing the right data (in terms of the data required to satisfy the requirement). It is a High level activity. Performed after a work product is produced against established criteria ensuring that the product integrates correctly into the environment. Determination of correctness of the final software product by a development project with respect to the user needs and requirements.
• 25.
  Software Validation According tothe Capability Maturity Model (CMM) we can also define validation as The process of evaluating software during or at the end of the development process to determine whether it satisfies specified requirements. [IEEE-STD-610].
• 26.
  Software Validation A productcan pass while verification, as it is done on the paper and no running or functional application is required. But, when same points which were verified on the paper is actually developed then the running application or product can fail while validation. This may happen because when a product or application is build as per the specification but these specifications are not up to the mark hence they fail to address the user requirements.
• 27.
  Software Validation Validation isbasically done by the testers during the testing. While validating the product if some deviation is found in the actual result from the expected result then a bug is reported or an incident is raised. Not all incidents are bugs. But all bugs are incidents. Incidents can also be of type ‘Question’ where the functionality is not clear to the tester.
• 28.
  Software Validation Hence, validationhelps in unfolding the exact functionality of the features and helps the testers to understand the product in much better way. It helps in making the product more user friendly.
• 29.
  Methods of Validation DynamicTesting Dynamic Testing is defined as a software testing type, which checks the dynamic behavior of the code is analyzed. Dynamic testing is a process of validating software applications as an end user under different environments to build the right software.
• 30.
  Methods of Validation Dynamictesting is when you are working with the actual system by providing an input and comparing the actual behavior of the application to the expected behavior. In other words, working with the system with the intent of finding errors.
• 31.
  Methods of Validation •The main aim of the Dynamic tests is to ensure that software works properly during and after the installation of the software ensuring a stable application without any major flaws( this statement is made because no software is error free, testing only can show presence of defects and not absence) • The main purpose of the dynamic test is to ensure consistency to the software.
• 32.
  Methods of Validation •In a Banking Application, we find different screens like My Accounts Section, Funds Transfer, Bill Pay, etc.. All these screens contain amount field which accepts some characters. • Let's say My Accounts field displays amount as 25,000 and Funds Transfer as $25,000 and Bill pay screen as $25000 though the amount is the same, the way amount is displayed is not the same hence making the software non-consistent.
• 33.
  Methods of Validation Consistencyis not only limited to the functionality it also refers to different standards like performance, usability, compatibility etc. hence it becomes very important to perform Dynamic Testing.
• 34.
  Dynamic Testing Techniques STLCis a process which consists of different tasks like Requirements Analysis, Test Planning, Test Case Design, Environment setup, Test Execution, and Test Closure. In STLC we can say that Dynamic Testing Process starts from Test Case Design
• 35.
  Dynamic Testing Techniques
• 36.
  Dynamic Testing Techniques TestStrategy should mainly focus on the resources available and the timeframe. Based on these factors, the objective of the testing, the scope of testing, phases or cycles of testing, type of environment, assumptions or challenges that might be faced, risks, etc. has to be documented. Once the strategy is defined and is accepted by the management then the actual process test case design starts.
• 37.
  Methods of Validation Testdesign and Implementation • Features to be tested • Derive the Test Conditions • Derive the coverage Items • Derive the Test Cases
• 38.
  Types of DynamicTesting Dynamic Testing is classified into two categories • White Box Testing • Black Box Testing Another Category includes in some cases known as • Grey Box Testing
• 39.
  Types of DynamicTesting
• 40.
  Black Box Testing Blackbox testing is defined as a testing technique in which functionality of the Application Under Test (AUT) is tested without looking at the internal code structure, implementation details and knowledge of internal paths of the software. This type of testing is based entirely on software requirements and specifications.
• 41.
  Black Box Testing InBlackbox Testing we just focus on inputs and output of the software system without bothering about internal knowledge of the software program.
• 42.
  Black Box Testing Hereare the generic steps followed to carry out any type of Black Box Testing. • Initially, the requirements and specifications of the system are examined. • Tester chooses valid inputs (positive test scenario) to check whether SUT processes them correctly. Also, some invalid inputs (negative test scenario) are chosen to verify that the SUT is able to detect them.
• 43.
  Black Box Testing •Tester determines expected outputs for all those inputs. • Software tester constructs test cases with the selected inputs. • The test cases are executed. • Software tester compares the actual outputs with the expected outputs. • Defects if any are fixed and re-tested.
• 44.
  Black Box Testing Definitionby ISTQB (International Software Testing Qualifications Board) Black box testing: Testing, either functional or non-functional, without reference to the internal structure of the component or system. Black box test design technique: Procedure to derive and/or select test cases based on an analysis of the specification, either functional or non-functional, of a component or system without reference to its internal structure.
• 45.
  Types of BlackBox Testing • Functional testing - This black box testing type is related to the functional requirements of a system; it is done by software testers. • Non-functional testing - This type of black box testing is not related to testing of specific functionality, but non-functional requirements such as performance, scalability, usability. • Regression testing - Regression testing is done after code fixes, upgrades or any other system maintenance to check the new code has not affected the existing code.
• 46.
  Black Box TestingTechniques • Equivalence Class Testing: It is used to minimize the number of possible test cases to an optimum level while maintains reasonable test coverage. • Boundary Value Testing: Boundary value testing is focused on the values at boundaries. This technique determines whether a certain range of values are acceptable by the system or not. It is very useful in reducing the number of test cases. It is most suitable for the systems where an input is within certain ranges. • Decision Table Testing: A decision table puts causes and their effects in a matrix. There is a unique combination in each column.
• 47.
  Black Box TestingModel Black box testing has its own life cycle called Software Testing Life Cycle (STLC) and it is relative to every stage of Software Development Life Cycle of Software Engineering. • Requirement - This is the initial stage of SDLC and in this stage, a requirement is gathered. Software testers also take part in this stage.
• 48.
  Black Box TestingModel • Test Planning & Analysis - Testing Types applicable to the project are determined. A Test Plan is created which determines possible project risks and their mitigation. • Design - In this stage Test cases/scripts are created on the basis of software requirement documents • Test Execution- In this stage Test Cases prepared are executed. Bugs if any are fixed and re-tested.
• 49.
  Advantages • Tests aredone from a user’s point of view and will help in exposing discrepancies in the specifications. • Tester need not know programming languages or how the software has been implemented. • Tests can be conducted by a body independent from the developers, allowing for an objective perspective and the avoidance of developer-bias. • Test cases can be designed as soon as the specifications are complete.
• 50.
  Disadvantages • Only asmall number of possible inputs can be tested and many program paths will be left untested. • Without clear specifications, which is the situation in many projects, test cases will be difficult to design. • Tests can be redundant if the software designer/developer has already run a test case. • Ever wondered why a soothsayer closes the eyes when foretelling events? So is almost the case in Black Box Testing.
• 51.
  White Box Testing WHITEBOX TESTING (also known as Clear Box Testing, Open Box Testing, Glass Box Testing, Transparent Box Testing, Code-Based Testing or Structural Testing) is a software testing method in which the internal structure/design/implementation of the item being tested is known to the tester. The tester chooses inputs to exercise paths through the code and determines the appropriate outputs. Programming know-how and the implementation knowledge is essential. White box testing is testing beyond the user interface and into the nitty-gritty of a system.
• 52.
  White Box Testing Thismethod is named so because the software program, in the eyes of the tester, is like a white/transparent box; inside which one clearly sees. Definition by ISTQB White-box testing: Testing based on an analysis of the internal structure of the component or system. White-box test design technique: Procedure to derive and/or select test cases based on an analysis of the internal structure of a component or system.
• 53.
  White Box Testing •White Box Testing method is applicable to the following levels of software testing: • Unit Testing: For testing paths within a unit. • Integration Testing: For testing paths between units. • System Testing: For testing paths between subsystems.
• 54.
  Advantages • Testing canbe commenced at an earlier stage. One need not wait for the GUI to be available. • Testing is more thorough, with the possibility of covering most paths.
• 55.
  Disadvantages • Since testscan be very complex, highly skilled resources are required, with a thorough knowledge of programming and implementation. • Test script maintenance can be a burden if the implementation changes too frequently. • Since this method of testing is closely tied to the application being tested, tools to cater to every kind of implementation/platform may not be readily available.
• 56.
  Grey Box Testing GrayBox Testing is a technique to test the software product or application with partial knowledge of the internal workings of an application. In this process, context-specific errors that are related to web systems are commonly identified. It will increase the testing coverage by concentrating on all of the layers of any complex system.
• 57.
  Grey Box Testing •Gray Box Testing is a software testing method, which is a combination of both White Box Testing and Black Box Testing method. • In White Box testing internal structure (code) is known • In Black Box testing internal structure (code) is unknown • In Grey Box Testing internal structure (code) is partially known
• 58.
  Grey Box Testing InSoftware Engineering, Gray Box Testing gives the ability to test both sides of an application, presentation layer as well as the code part. It is primarily useful in Integration Testing and Penetration Testing.
• 59.
  Grey Box Testing GrayBox Testing is performed for the following reason, • It provides combined benefits of both black box testing and white box testing both. • It combines the input of developers as well as testers and improves overall product quality. • It reduces the overhead of long process of testing functional and non-functional types. • It gives enough free time for a developer to fix defects. • Testing is done from the user point of view rather than a designer point of view.
• 60.
  Grey Box Testing Toperform Gray box testing, it is not necessary that the tester has the access to the source code. A test is designed based on the knowledge of algorithm, architectures, internal states, or other high -level descriptions of the program behavior. To perform Gray box Testing- • It applies a straightforward technique of black box testing • It is based on requirement test case generation, as such, it presets all the conditions before the program is tested by assertion method.
• 61.
  Grey Box Testing Techniquesused for Grey box Testing are- • Matrix Testing: This testing technique involves defining all the variables that exist in their programs. • Regression Testing: To check whether the change in the previous version has regressed other aspects of the program in the new version. It will be done by testing strategies like retest all, retest risky use cases, retest within a firewall. • Orthogonal Array Testing or OAT: It provides maximum code coverage with minimum test cases. • Pattern Testing: This testing is performed on the historical data of the previous system defects. Unlike black box testing, gray box testing digs within the code and determines why the failure happened
• 62.
  Grey Box Testing Stepsto perform Grey box Testing are: • Step 1: Identify inputs • Step 2: Identify the outputs • Step 3: Identify the major paths • Step 4: Identify Subfunctions • Step 5: Develop inputs for Subfunctions • Step 6: Develop outputs for Subfunctions • Step 7: Execute test case for Subfunctions • Step 8: Verify the correct result for Subfunctions • Step 9: Repeat steps 4 & 8 for other Subfunctions • Step 10: Repeat steps 7 & 8 for other Subfunctions
• 63.
  Grey Box Testing GrayBox Testing Challenges • When a component under test encounter a failure of some kind may lead to abortion of the ongoing operation • When test executes in full but the content of the result is incorrect.
• 64.
  Advantages of DynamicTesting • Dynamic Testing can reveal the uncovered defects that are considered to be too difficult or complicated and which cannot be covered through static Analysis • In Dynamic Testing, we execute the software, end to end, ensuring error free software which in turn increases the quality of a product and project. • Dynamic Testing becomes an essential Tool for detecting any security Threats
• 65.
  Disadvantages of DynamicTesting • Dynamic Testing is Time Consuming because it executes the application/software or code which requires huge amount of Resources • Dynamic Testing increases the cost of project/product because it does not start early in the software lifecycle and hence any issues fixed in later stages can result in an increase of cost.
• 66.
  Conclusion In Software Engineering,Verification and Validation are two measures used to check that the software product meets the requirements specifications. Static testing involves verification whereas dynamic testing involves validation. Together they help to deliver a cost effective Quality Software.
• 67.
  Advantages of SoftwareValidation • During verification if some defects are missed then during validation process it can be caught as failures. • If during verification some specification is misunderstood and development had happened then during validation process while executing that functionality the difference between the actual result and expected result can be understood.
• 68.
  Advantages of SoftwareValidation • Validation is done during testing like feature testing, integration testing, system testing, load testing, compatibility testing, stress testing, etc. • Validation helps in building the right product as per the customer’s requirement and helps in satisfying their needs.
• 69.
  Difference b/w Verification& Validation Verification Validation The verifying process includes checking documents, design, code, and program It is a dynamic mechanism of testing and validating the actual product It does not involve executing the code It always involves executing the code Verification uses methods like reviews, walkthroughs, inspections, and desk- checking etc. It uses methods like Black Box Testing, White Box Testing, and non- functional testing
• 70.
  Difference b/w Verification& Validation Verification Validation Whether the software conforms to specification is checked It checks whether the software meets the requirements and expectations of a customer It finds bugs early in the development cycle It can find bugs that the verification process can not catch Target is application and software architecture, specification, complete design, high level, and database design etc. Target is an actual product
• 71.
  Difference b/w Verification& Validation Verification Validation QA team does verification and make sure that the software is as per the requirement in the SRS document. With the involvement of testing team validation is executed on software code. It comes before validation It comes after verification
• 72.
  Software Verification &Validation