Skil storm testing at the speed of business 2

1. SoftwareRequirementsSoftwareDesignSoftwareMaintenanceSoftwareDevelopmentSoftwareTestingGreg SpindlerPrincipalGlen NoesenVice PresidentTesting at the Speed of Business

2. About Today’s PresentationGreg SpindlerPrincipalGlen NoesenVice PresidentEnterprise Project Solutions, a division of SkillStorm, specializes in providing program management services process improvement and project management expertise that can drive initiatives across the full development lifecycle.

3. What the Experts Say SoftwareMaintenanceSoftwareTesting“Software testing is often treated as accidental in industry, where it is sometimes barely done at all, often done poorly, and seldom done well. Testing is often cut when schedules and budgets overrun.“The need for software quality has been increasing dramatically, and new ideas like agile processes put a heavy emphasis on testing”Jeff Offutt - Professor of Software Engineering, George Mason University."The national annual cost of [poor] software testing is estimated to range from $22.2 to $59.5 billion." - US National Institute for Standards and Technology

4. What the Experts Say SoftwareMaintenanceSoftwareTesting“We have as many testers as we have developers. And testers spend all their time testing, and developers spend half their time testing. We're more of a testing,a quality software organization than we're a software organization.”Bill Gates on trustworthy computing. Information Week, May 2002"The national annual cost of [poor] software testing is estimated to range from $22.2 to $59.5 billion." - US National Institute for Standards and Technology

5. What the Experts Say SoftwareMaintenanceSoftwareTesting“Irreproducible bugs become highly reproducible right after delivery to the customer”Michael Stahl’s derivative of Murphy’s Law Jeff Offutt - Professor of Software Engineering, George Mason University."The national annual cost of [poor] software testing is estimated to range from $22.2 to $59.5 billion." - US National Institute for Standards and Technology

6. Our Quality Foundation SoftwareMaintenanceSoftwareTestingThe JURAN TRILOGY“To attain quality you must begin by establishing the vision, policies and goals of the organization. Converting these goals into results is done through three managerial processes” Source: Juran Quality Control Handbook, Fifth Edition, 1999ProcessProcessProcess Prove the need for improvement.

7. Identify specific projects for improvement.

8. Organize to guide the projects.

9. Organize for diagnosis for discovery of causes.

10. Diagnose to find the causes. Provide remedies.

11. Prove remedies are effective under operating conditions. Provide for control to hold the gains.

12. Identify the customers both external & internal.

13. Determine customer needs.

14. Develop features that meet customer needs.

15. Establish quality goals to meet the needs. and do so at a minimum combined cost. Develop a process to produce the needed features.

16. Prove process capability

17. Prove that the process can meet the QA goals under operating conditions. Choose control subjects (what to control). Choose units of measurement.

18. Establish measurement.

19. Establish standards of performance.

20. Measure actual performance.

21. Interpret (actual versus standard).

22. Take action on the difference.Our Quality Foundation SoftwareMaintenanceSoftwareTestingThe JURAN TRILOGYQuality Planning“To attain quality you must begin by establishing the vision, policies and goals of the organization. Converting these goals into results is done through three managerial processes” The structured process for designing products and services that meet new breakthrough goals and ensure that customer needs are met.Source: Juran Quality Control Handbook, Fifth Edition, 1999ProcessProcessProcess Prove the need for improvement.

38. Take action on the difference.Our Quality Foundation SoftwareMaintenanceSoftwareTestingThe JURAN TRILOGYQuality ControlManagerial process for conducting operations so as to provide stability--to prevent adverse change and to “maintain the status quo”Source: Juran Quality Control Handbook, Fifth Edition, 1999ProcessProcessProcess Prove the need for improvement.

54. Take action on the difference.Our Quality Foundation SoftwareMaintenanceSoftwareTestingThe JURAN TRILOGYQuality Improvement“To atainquality you must begin by establishing the vision, policies and goals of the organization. Converting these goals into results is done through three managerial processes” The process for creating breakthrough levels of performance by eliminating wastes and defects to reduce the cost of poor quality Source: Juran Quality Control Handbook, Fifth Edition, 1999ProcessProcessProcess Prove the need for improvement.

70. Take action on the difference.Testing Foundation SoftwareMaintenanceSoftwareTestingTMMi Foundation, Test Maturity Model CornerstonesCopyright TMMi Foundation – No WarrantyLifecycleOrganizationTechniquesInfrastructure

71. Testing Foundation SoftwareMaintenanceSoftwareTestingTMMi Foundation, Test Maturity Model Defect PreventionTest Process OptimizedQuality ControlTest MeasurementSoftware Quality Evaluation.Advanced Peer ReviewsTest OrganizationTest Training ProgramLife Cycle and IntegrationNon-Functional TestingPeer ReviewsTest Policy and StrategyTest PlanningTest Monitoring and ControlTest Design and ExecutionTest EnvironmentNo documented QA processes345 Managed/ MeasuredInitial (Ad-hoc)ManagedDefinedOptimization12Organized and Embedded Test PracticesMeasured andAligned Test PracticesContinuouslyImproving Test Practices Start-upBasic testPractices

72. Economics of Software Quality SoftwareMaintenanceSoftwareTestingCosts Due To Lack Of QualityQA Budget as % of DevelopmentDECLINING QUALITY COSTSTesting ObjectiveMaturity Cost and budget estimates are those of the presenter and are based on personal experience and researchMany IT professionals think of testing as a stage that comes near the end of IT product development.Testing should be done during almost every phase of the IT product development life cycle.

73. SoftwareRequirementsSoftwareDesignSoftwareDevelopmentSoftwareTestingSoftwareMaintenanceAttain a Competitive Advantage Increase ProfitabilityWhere Defects are IntroducedWhere Errors Are FoundThe National Institute of Standards and Technology (NIST) estimates about 70 percent of software defects are introduced in the requirements phase. The later defects are found, the more expensive they are to fix.Identifying the critical bugs earlier in the lifecycle reduces costsSource: NIST 2002 RTI Project 7007.011Source: NIST 2002 RTI Project 7007.011Source: NIST 2002 RTI Project 7007.011

74. Attain a Competitive Advantage Increase ProfitabilityWhere Defects are IntroducedIdentifying the critical bugs earlier in the lifecycle reduces costsSource: NIST 2002 RTI Project 7007.011Source: NIST 2002 RTI Project 7007.011

75. Attain a Competitive Advantage Increase ProfitabilityWhere Defects are IntroducedWhere Errors Are FoundIdentifying the critical bugs earlier in the lifecycle reduces costsSource: NIST 2002 RTI Project 7007.011Source: NIST 2002 RTI Project 7007.011Source: NIST 2002 RTI Project 7007.011

76. SoftwareMaintenanceSoftwareTestingRelative Cost to Repair Defects When Found at Different Stages of SoftwareX is a normalized unit of cost and can be expressed terms of person-hours, dollars, etc.Source: NIST 2002 RTI Project 7007.011RAISE (Reliability, Availability, Install Serviceability, and Ease of Use).

77. It’s Complex so we Focus Business ModelsSkillStormSoftwareMaintenanceSoftwareTestingSoftwareRequirementsLicense ModelKnowledgeSkillsTalentSoftwareDesignSoftwareDevelopmentTechniquesToolsFixed PriceProjects Q.A. ProcessSoftwareTestingSoftwareMaintenanceBuild-OperateTransfer ModelTesting StrategyImproved test coverageHewlett Packard QTPCompatibilityMethodologySolutionFocus AreasIBM RationalHybrid ModelTesting DesignReduced execution timeagilePlanningTime & Material Microsoft VS2010WaterfallTestExecutionDecrease test EscapesDedicated Test Center ModelMicro FocusControlRADTestMaintenanceImproved test repeatabilityWatiN, Watir, SeleniumPrototyping ImprovementIncrementalOptimizedMeasuredInitialManagedDefined

78. Industry ChallengeSoftwareMaintenanceSoftwareTesting Core Beliefs"The surest foundation of a business concern is quality.”Source: Andrew CarnegieA Service Delivery Platform allowing you to focus not simply on formulating a strategy or a vision of the futurebut on building the distinctive organizational and operational capabilities to deliver uncommon results.

79. Business ChallengeSoftwareMaintenanceSoftwareTesting Executives are under tremendous pressure to quickly improve performance with fewer resources than ever before. Senior IT Management respond by off-shoring to reduce cost.

80. This years cost reductions become next years budget challenge

81. The cycle is constant

82. Testing productivity tools have been lagging behind the general market.QA organizations have an opportunity to implement positive change through innovation.

83. TIMEBUDGETSKILLSAutomation ChallengesSoftwareMaintenanceSoftwareTesting“The economics of the current automation tools don’t stack up. To make them work you need too many expensive technical staff. Without them you end up with poorly designed test suites that are not maintainable. Even with the correct level of technical input, the tools don’t yield scripts that are easily linked to the business function they are meant to test” Expensive technical staff Poorly Designed Not Maintainable No Linkage to Business functions

84. Testing Innovation SoftwareMaintenanceSoftwareTestingAxe is INNOVATIONin software test automation that will … Revolutionize the way you currently test applications.

85. Remove the testing barriers that impede timely releases of high quality software. Transform your testing cycle to become a strategic part of your SDLC.

86. Overcome budget restrictions and change your testing practices from risk management to risk mitigation! Time & Speed, Change Software Quality from a Cost Center to a Profit CenterSPEED:Gains of up to 4X the productivity of your current testing resources.

87. Align your process for successSoftwareMaintenanceSoftwareTestingReduce Development & Maintenance CostsAttain a Competitive Advantage Improve Customer Satisfaction Reduce Cycle Time Increase ProfitabilitySimplify and Optimize your Software Testing and MaintenanceDeliverSolutionSoftwareTestingSoftwareMaintenanceNewProjectSoftwareDevelopmentSoftwareRequirementsSoftwareDesignImproved Team CollaborationReduced Cost of ImplementationScalable Proven ROI Easy to use Automate throughout the LifecycleFuture-Proof StrategyProcessTools Go from Serial to Parallel lifecycle and delivery

88. Keyword vs. Code GenerationSoftwareMaintenanceSoftwareTestingCode Generation ApproachTraditional Framework ApproachRequirementsThe time involved in training, process and test-script development is a major part of the automated testing investment. Keyword or Data Driven Frameworks on test tools like QTP, Rational, VS2010 and others, is Labor Intensive, Time Consuming and expensive.Axe completely eliminates the cost and technical sophistication required in developing and integrating with automation tools. An unrivalled platform for harnessing the power of automation, where testers can develop and automate highly sophisticatedtest cases without the requirement to acquire proprietary technical skills.

89. Need for Speed SoftwareMaintenanceSoftwareTestingAutomation teams typically spend 20% of their time on development and 80% on maintenance of test cases.Axe reverses this trend with 80% to 90% of time spent in developing test cases. The result with Axe is that automation scales to the needs of the business.020 40 60 80 100 120SpeedThere is no code to maintain with code generation!The most telling differentiator in business today has become how close a company can come to operating in "real time” a state that is characterized by the "shortest possible lapse between idea and action, between initiation and result. Regis McKenna chairman of the McKenna Group

90. Improved Team CollaborationSoftwareMaintenanceSoftwareTestingAutomation Engineer Testing Expert Design test in Axe Excel SpreadsheetArchitecture Design Generate Test Script with Documentation Object MappingRun Test to ensure correctness & Business processCustomization & Support Upload and commit to Test MGT Tool New Functionality A Clear Delineation Of The Responsibilities And Skill Sets Creating High Performance TeamsAxe is a test automation and Acceleration platform solution, leveraging your current or planned technology investment in functional and automated testing.Addresses the most common problems with test automation, Testersusually do not have automation expertise. Axe clearly delineates the responsibilities and skill sets between Testers and Test Automation Engineers.

91. Tester-centric EnvironmentSoftwareMaintenanceSoftwareTestingTester-centric Environment Axe ExcelTest ModelingCode andDocumentation GenerationTest ExecutionAxe Test ResultsTest Management“Truly Next Generation”A holistic approach to automation which can integrate multiple vendors tools together to meet different business needs and leverage best in class tools.

92. Tester-centric EnvironmentSoftwareMaintenanceSoftwareTestingTester-centric Environment Axe Uses Excel to define test scenarios, the capabilities of Excel combined with the tester’s knowledge to quickly create test scenarios, that can be easily replicated and changed to increase the coverage of test cases against the AUT. Axe leverages the functionality of Excels built-in features (such as date handling and manipulation) to create the code and generate documentation. “Truly Next Generation”A holistic approach to automation which can integrate multiple vendors tools together to meet different business needs and leverage best in class tools.

93. Tester-centric EnvironmentSoftwareMaintenanceSoftwareTestingTester-centric Environment Axe employs a small action set and simple spread sheet layout which can be easily mastered with one day of practical exercise based training. Only five core actions: Set, Get, Val, Load & Save. Axe, can be easily extended to create new actions to enhance the function of the generated scripts or overcome automation difficulties. “Truly Next Generation”A holistic approach to automation which can integrate multiple vendors tools together to meet different business needs and leverage best in class tools.

94. Tester-centric EnvironmentSoftwareMaintenanceSoftwareTestingTester-centric Environment Once test scenarios have been defined, Axe can generate documentation, object use statistics (coverage) and the coded automation scripts required. Supports leading test management and version control tools, Axe can create test items through the tool APIs and upload generated code. “Truly Next Generation”A holistic approach to automation which can integrate multiple vendors tools together to meet different business needs and leverage best in class tools.

95. Tester-centric EnvironmentSoftwareMaintenanceSoftwareTestingTester-centric Environment Axe generated scripts are executed against the AUT and controlled directly from Axe, the test management tool or a scheduler. Alongside execution tool logs, Axe produces a complementary “English” log showing actions on a step by step basis. Following the completed execution of the required test scenarios, Axe can upload results to the test management tool to complete the whole automation process. “Truly Next Generation”A holistic approach to automation which can integrate multiple vendors tools together to meet different business needs and leverage best in class tools.

96. Code SampleSoftwareMaintenanceSoftwareTesting(QTP/VBScript)

97. Documentation SoftwareMaintenanceSoftwareTesting

98. Test Results SoftwareMaintenanceSoftwareTesting

99. Test Model & HierarchySoftwareMaintenanceSoftwareTestingSub-testLogin as user 1RunTest001….Test002…Test003… ad infinitum!Test CaseLogin – enter user ID and PasswordLogin – with no user IDLogin – with no Password

100. Investment LeverageSoftwareMaintenanceSoftwareTestingIntegration and Compatibility

102. Ensures Repeatable Business ProcessSoftwareMaintenanceSoftwareTestingAccurate Defect Replication: Speed:Regression:Platforms:Axe, clients and customers are able to decrease their time to market and deliver complete and fully tested business solutions to their customers and clients, with lower, predictable costs and unmatched speed.

103. Evaluation CriteriaSoftwareMaintenanceSoftwareTestingReduce Development & Maintenance CostsAttain a Competitive Advantage Improve Customer Satisfaction Reduce Cycle Time Increase ProfitabilityEase of Use Reporting Capability RequirementsSelectionProof of Concept Customization CapabilityPlatform Support Testing Requirements & Business processFunctionality Vendor Qualifications Management goals 1. Skills of staff 2. Automation Goals 3. Test Requirements 4. Required Features

104. Client ExampleSoftwareMaintenanceSoftwareTestingClient example 1200 test cases 4 release cycles/yr 4 testers 6 days to test TestingDelivery DelaysAutomation was the only answerDelivery TimeDevelopmentOngoing Releases As Release Cycles Mounted, Testing Became The Bottleneck

105. Clients Case StudyAttain a Competitive Advantage Increase ProfitabilityDeveloped 22,000 automated test cases in just under two yearsAxe Users since August of 2008QA Team is made up of three Axe Testers (non technical) and one Axe Technician (expert on QTP as well as Axe) and one Developer (application support).No experience with automation were creating scripts and running automated tests within hours of their training.Software make up is 5 Axe licenses with 3 QTP licenses.Return on Investment was in-year as opposed to multiyear with QTP alone.

106. Axe Customers and ClientsSoftwareMaintenanceSoftwareTesting

107. Clients TestimonialSoftwareMaintenanceSoftwareTestingIndustry Leaders “Axe has clearly demonstrated its return on investment and proved extremely worthwhile for Serena Dimensions”“Within three months of adoption the Axe tool set has become a core part of our Agile development processallowing us to standardise our testing processes and reap the substantial cost and business benefits that Axe has proven to return” Mir Derakhshan – Vice President SCM R&D Serena

108. Clients TestimonialSoftwareMaintenanceSoftwareTestingIndustry Leaders “We have used Axe on a number of high profile engagements and in conjunction with a range of tools has helped us achieve ahigh-level of compliance and consistency in the delivery of our testing by re-use of tested components. Its design and architecture allows us to deliver Quality, flexibly and at speed with the minimum of maintenance and the requirement to write additional driver scripts” Andy Nichol, Microsoft Services (NHS and E-Gateway)

109. Clients TestimonialSoftwareMaintenanceSoftwareTestingIndustry Leaders I have used Axe for the past 3 years. This tool helped me introduce automation to a QA department that had tried going the automation route a few different times in the past but it never really took off. Axe gave me the ability to teach the QA team quickly. After only a couple hours of train the QA team was creating and executing USEFUL test cases in the current project. The one thing that we need to be kept in mind is my QA team was able to learn and apply Axe, while keeping up with their day to day tasks. The project calendar was not changed because we wanted to introduce automation. As far as I am concerned Axe has set the bar for all other automation tools. I have and I will continue to recommend Axe to all my peers.Erich Scheman – Fiserv

110. Clients TestimonialSoftwareMaintenanceSoftwareTestingIndustry Leaders “With the scope and complexity of recipient clients we need to deliver to, Quality is one of our most important criteria. Axe allows us to operate a continuous change-based environment. This ensures testing is no longer a constraint on our requirement to deliver the highest quality systems quickly to a large number of clients simultaneously”Alan Long, Microsoft Programme Manager

111. Benefits to QA OrganizationSoftwareMaintenanceSoftwareTesting Speed Enduring Quality Return on Investment Scalable

112. ProvenROI

113. Easy to use

114. Automate the Application Lifecycle

115. Future-Proof

116. Team Collaboration

117. Cost of implementation Code generation is the “Next Generation” for automated testing “Remember, testing is not what is ultimately important… performance is.”

118. Benefits to QA OrganizationSoftwareMaintenanceSoftwareTesting Speed Enduring Quality Return on InvestmentScalabilityIn-year Payback Scalable

119. ProvenROI

120. Easy to use

121. Automate the Application Lifecycle

122. Future-Proof

123. Team Collaboration

124. Cost of implementation SynchronizationMaintenanceAgilityLasting ValueCode generation is the “Next Generation” for automated testing “Remember, testing is not what is ultimately important… performance is.”

125. SkillStorm CapabilitiesAttain a Competitive Advantage Increase ProfitabilityConsulting ServicesConduct assessments of Clients ’ current testing methods-processes-culture Recommendations for establishing or augmenting current process for SW testing Development of testing processes at the project, organization and business Level Prepare testing strategies and test plans for projects and client organizations. Provide testing resources for functional and automated testing. Provide business staff, testers, test teams and over-site for SW quality organizations. Test management for efficient testing, defect management, reporting metrics and tool selection. Complete outsourcing of test cases and test script development Complete test outsourcing

126. Q&AGlen Noesen: [email protected] Spindler: [email protected]

127. Free Offer For Webinar AttendeesAxe Solution DemoOn-line POC

Skil storm testing at the speed of business 2

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Skil storm testing at the speed of business 2 (20)

Recently uploaded (20)

Skil storm testing at the speed of business 2

Editor's Notes