1.tool support for testing
- 1. TOOL SUPPORT FOR TESTING Disusun Oleh : Bobi Henfajri Setiawan Nim :11453104759 Program Studi (S1) :Sistem Informasi Testing Dan Implementasi Sistem Fakultas Sains dan Teknologi Universitas Islam Negeri Sultan Syarif Kasim Riau http://www.uin-suska.ac.id/http://sif.uin-suska.ac.id/ http://fst.uin-suska.ac.id/
- 2. You may be wishing that you had a magic tool that would automate all of the testing for you. If so, you will be d isappointed. However, there are a number of very useful tools that can bring significant benefits. In this chapter we will see that there is tool support for many different aspects of software testing. We will see that success wit h tools is not guaranteed, even if an appropriate tool is acquired - there are also risks in using tools. There are so me special considerations mentioned in the Syllabus for certain types of tool: test execution tools, performance t esting tools, static analysis tools and test management tools.
- 3. Test tool classification The tools are grouped by the testing activities or areas that are supported by a set of tools, for exa mple, tools that support management activities, tools to support static testing, etc. . There is not necessarily a one-to-one relationship between a type of tool described here and a tool offered by a commercial tool vendor or an open-source tool. Some tools perform a very specific a nd limited function (sometimes called a 'point solution'), but many of the commercial tools provid e support for a number of different functions (tool suites or families of tools). There are some things that people can do much better or easier than a com- puter can do. For exa mple, when you see a friend in an unexpected place, say in an airport, you can immediately recog nize their face. This is an example of pattern recognition that people are very good at, but it is not easy to write soft- ware that can recognize a face.
- 4. Tool support for management of testing and tests Test management tools The features provided by test management tools include those listed below. Some tools will prov ide all of these features; others may provide one or more of the features, however such tools woul d still be classified as test management tools. Requirements management tools Are requirements management tools really testing tools? Some people may say they are not, but they do provide some features that are very helpful to testing. Because tests are based on requirem ents, the better the quality of the require- ments, the easier it will be to write tests from them. Incident management tools This type of tool is also known as a defect-tracking tool, a defect-management tool, a bug-trackin g tool or a bug-management tool. However, 'incident management tool' is probably a better nam e for it because not all of the things tracked are actually defects or bugs; incidents may also be per ceived problems, anomalies (that aren't necessarily defects) or enhancement requests.
- 5. Tool support for static testing Review process support tools Features or characteristics of review process support tools include support for: – a common reference for the review process or processes to use in different situatio ns; – storing and sorting review comments; – communicating comments to relevant people; – coordinating online reviews; – keeping track of comments, including defects found, and providing statisti cal info rmation about them; – providing traceability between comments, documents reviewed and related docum ents; – a repository for rules, procedures and checklists to be used in reviews, as well as e ntry and exit criteria; – monitoring the review status (passed, passed with corrections, requires re- review); – collecting metrics and reporting on key factors.
- 6. Continue…. Static analysis tools (D) Features or characteristics of static analysis tools include support to: -calculate metrics such as cyclomatic complexity or nesting levels (which can help to -identif y where more testing may be needed due to increased risk); -enforce coding standards; -analyze structures and dependencies; -aid in code understanding; -identify anomalies or defects in the code (as described in Chapter 3). Modeling tools (D) Modeling tools help to validate models of the system or software. For example a tool can ch eck consistency of data objects in a database and can find inconsis- tencies and defects. Thes e may be difficult to pick up in testing - you may have tested with one data item and not reali ze that in another part of the database there is conflicting information related to that item. Mo deling tools can also check state models or object models.
- 7. Tool support for test specification Test design tools Test design tools help to construct test cases, or at least test inputs (which is part of a test case). If an automated oracle is available, then the tool can also con- struct the expected r esult, so it can actually generate test cases (rather than just test inputs). Features or characteristics of test design tools include support for: -generating test input values from: -requirements; -design models (state, data or object); -code; -graphical user interfaces; -test conditions; The benefit of this type of tool is that it can easily and quickly identify the tests (or test in puts) that will exercise all of elements, e.g. input fields, buttons, branches.
- 8. Continue…… Test data preparation tools Setting up test data can be a significant effort, especially if an extensive range or volume of data is n eeded for testing. Test data preparation tools help in this area. They may be used by developers, b ut they may also be used during system or acceptance testing. Features or characteristics of test data preparation tools include support to: -extract selected data records from files or databases; -'massage' data records to make them anonymous or not able to be identified with real people (for d ata protection); -enable records to be sorted or arranged in a different order; -generate new records populated with pseudo-random data, or data set up according to some guideli nes, e.g. an operational profile; -construct a large number of similar records from a template, to give a large set of records for volum e tests, for example.
- 9. Tool support for test execution and logging Test execution tools When people think of a 'testing tool', it is usually a test execution tool that they have in mind, a tool that can run tests. This type of tool is also referred to as a 'test running tool'. Most tools o f this type offer a way to get started by capturing or recording manual tests; hence they are also known as 'capture/playback' tools, 'capture/replay' tools or 'record/playback' tools. The analo gy is with recording a television program, and playing it back. However, the tests are not some thing which is played back just for someone to watch the tests interact with the system, which may react slightly differently when the tests are repeated. The main reason for this is that a captured script is very difficult to maintain because: -It is closely tied to the flow and interface presented by the GUI. -It may rely on the circumstances, state and context of the system at the time the script was rec orded. For example, a script will capture a new order number assigned by the system when a te st is recorded. When that test is played back, the system will assign a different order number an d reject sub sequent requests that contain the previously captured order number. -The test input information is 'hard-coded', i.e. it is embedded in the individ ual script for each test.
- 10. Continue…. Test harness/unit test framework tools (D) Features or characteristics of test harnesses and unit test framework tools include support for: -supplying inputs to the software being tested; -receiving outputs generated by the software being tested; -executing a set of tests within the framework or using the test harness; -recording the pass/fail results of each test (framework tools); -storing tests (framework tools); -support for debugging (framework tools); -coverage measurement at code level (framework tools). Test comparators Is it really a test if you put some inputs into some software, but never look to see whether the softwa re produces the correct result? The essence of testing is to check whether the software produces the correct result, and to do that, we must compare what the software produces to what it should produc e. A test comparator helps to automate aspects of that comparison.
- 11. Tool support for performance and monitoring Dynamic analysis tools are 'dynamic' because they require the code to be running. They are 'analys is' rather than 'testing' tools because they analyze what is happening 'behind the scenes' while the so ftware is running (whether being executed with test cases or being used in operation). Features or characteristics of dynamic analysis tools include support for: -detecting memory leaks; -identifying pointer arithmetic errors such as null pointers; -identifying time dependencies. These tools would typically be used by developers in component testing and component integration testing, e.g. when testing middleware, when testing security or when looking for robustness defects. Another form of dynamic analysis for websites is to check whether each link does actually link to s omething else (this type of tool may be called a 'web spider').
- 12. Continue…. Features or characteristics of performance-testing tools include support for: -generating a load on the system to be tested; -measuring the timing of specific transactions as the load on the system varies; -measuring average response times; -producing graphs or charts of responses over time. Monitoring tools Monitoring tools are used to continuously keep track of the status of the system in use, in orde r to have the earliest warning of problems and to improve service. Features or characteristics of monitoring tools include support for: -identifying problems and sending an alert message to the administrator (e.g. network administr ator); -logging real-time and historical information; -finding optimal settings; -monitoring the number of users on a network; -monitoring network traffic (either in real time or covering a given length of time of operation with the analysis performed afterwards).
- 13. Reference Graham, d., et al. 2006. Foundation of Software Tes ting: ISTQB certification London, UK: Internation al Thomson Business Press