Sqa
- 1. Software Quality Assurance For Software Engineering && Architecture and Design
- 2. Software Quality Assurance • What is “quality”?
- 3. Software Quality Assurance • What is “quality”? • IEEE Glossary: Degree to which a system, component, or process meets (1) specified requirements, and (2) customer or user needs or expectations
- 4. Software Quality Assurance • What is “quality”? • IEEE Glossary: Degree to which a system, component, or process meets (1) specified requirements, and (2) customer or user needs or expectations • ISO: the totality of features and characteristics of a product or service that bear on its ability to satisfy specified or implied needs
- 5. Software Quality Assurance • An alternate view of Quality: – is not absolute – is multidimensional, can be difficult to quantify – has aspects that are not easy to measure – assessment is subject to constraints (e.g., cost) – is about acceptable compromises – criteria are not independent, can conflict
- 6. Software Quality Assurance • Quality Criteria include: – correctness – efficiency – flexibility – integrity – interoperability – maintainability – portability – reliability – reusability – testability – usability
- 7. What is Software Quality Assurance (SQA)? • “Set of systematic activities providing evidence of the ability of the software process to produce a software product that is fit to use” – G. Schulmeyer and J. McManus, Software Quality Handbook, Prentice Hall, 1998.
- 8. What is SQA? • Monitoring processes and products throughout the software development lifecycle to ensure the quality of the delivered product(s) • Monitoring the processes – Provides management with objective feedback regarding process compliance to approved plans, procedures, standards, and analyses • Monitoring the products – Focus on the quality of product within each phase of the SDLC • e.g., requirements, test plan, architecture, etc. – Objective: identify and remove defects throughout the lifecycle, as early as possible
- 9. Quality of Software developed in-house & COTS components • SQA processes apply when integrating purchased or customer-supplied software products into the developed product • Question. How do you determine the “quality” of COTS components? – Current research problem
- 10. Process Assessment • Use of standards and process models has a positive impact on the quality of the software product – Disciplined, controlled development process • Examples include: – ISO 9001 – CMM • CMU SEI, 5 levels – SPICE • Developing a standard for software process assessment • ISO joint committee, Europe, Australia – IEEE 1074, IEEE 12207, …
- 11. Product Assessment • Reviews, inspections, walkthroughs – Specialized techniques available: • How to review/assess requirements, architecture, detailed designs, code • … • Testing • Simulation • Protoyping • Formal verification – Model checking, theorem proving
- 12. Product Assessment • Reviews, inspections, walkthroughs of Plans, reports, models, standards – Project management, quality assurance, training, test plan(s) – Requirements, analysis, architecture, detailed design model, test cases – Issue or problem reports – Metric reports – Traceability reports – Documentation, coding standards – …
- 13. Software Reviews • They may include managerial reviews, acquirer-supplier reviews, technical reviews, inspections, walkthroughs, and audits. • Inspection: – A formal evaluation technique in which an artifact (e.g., software requirements, design, or code) is examined in detail by a person or group other than the originator – detect faults, violations of development standards, and other problems. – review members are peers (equals) of the designer or programmer. – data is collected during inspections for later analysis and to assist in future inspections. Note – Introduced by Fagan, 1976. – Fagan, M., “Design and Code Inspections to Reduce Errors in Program Development”, IBM Systems Journal, 15, 3 (1976), pp. 182-211 – Fagan, M., “Advances in Software Inspections”, IEEE Transactions on Software Engineering, 12, 7(July 1986), pp. 744-751
- 14. Picture from “Inspections” presentation http://www.math.uaa.alaska.edu/~afkjm/cs470/handouts/inspections.pdf
- 15. Defect Checklists • Useful to support reviews, inspections, walkthroughs • Expertise is captured in a list format – Less experienced people can use • Straightforward to use (each check should be clear, simple to assess/apply) – Improve consistency of assessments • Example architecture checklist used in undergrad./grad. courses for OO – spreadsheet in in the course materials subdirectory • One or more architectural styles are selected. • Capabilities and interfaces are defined for subsystems. • Capabilities of and interfaces among subsystems support all of the use cases. • Concurrency defined. • Distribution defined. • Error handling defined. • Start up and shut down defined. • Data persistency defined. • Rationale for the model is provided. • Other
- 16. Verifying Formal Specifications • Formal specifications may be verified in a number of different ways: –Syntax, typechecking • If the notation is typed –Simulated –Model checked (e.g., SPIN) –Proven correct (e.g., HOL, PVS) • More straightforward? Less assurance of correctness; fully automated • Less straightforward? Higher assurance of correctness; not fully automated More straightforward Less straightforward
- 17. Problem Reporting, Tracking, and Resolving • Describe the practices and procedures to be followed for reporting, tracking, and resolving problems – Who can report a problem? – How is it reported? – How is is tracked? – Who determines if it is a problem that going to be resolved? – How is it assigned for resolution? – How does the person indicate it has been corrected? – Who reviews it to determine if it can be closed? • Problems can be product or process related – e.g. incorrect requirement, incomplete class definition, code defect, ambiguous description in user documentation, process to review detailed design is not clearly defined, etc.
- 18. Metrics • Metrics for each artifact • e.g., Requirements – Number of requirements – Number of changes per requirement • Called “churn” rate – Characterization of defects • Not testable, ambiguous, inconsistent, incorrect, incomplete redundant, infeasible, … • Major or minor defect • Phase defect detected • Cost to fix
- 19. Tools, techniques, training • What tools? – e.g., CVS for CM, excel spreadsheet for problem reporting/tracking, ... • What techniques? – e.g., formal peer review for deliverables, checklists for defect detection, ... • What training is needed on tools, techniques?
- 20. Media Control • Identify the media for each intermediate and deliverable artifact • Documentation required to store the media, including the backup and restore process • Protect computer program physical media from: – unauthorized access – inadvertent damage – degradation
- 21. Architecture Analysis Methods • Why evaluate an architecture? http://www.slideshare.net/kevinjew/evaluating-software- architectures-presentation • Specialized techniques available: http://www.slideshare.net/timmenzies/architecture- tradeoff-analysis-method-presentation SEI presentation and technical report on ATAM are in the course subdirectory