Enabling agility with continuous integration testing
6 likes1,539 views
Continuous integration testing can help address challenges with integrating and testing complex systems. Traditional integration testing takes significant time and resources to set up test environments and is difficult to keep pace with agile development. With continuous integration, testing is automated and run on every code change. This finds issues early before they compound, speeds delivery of working software, and reduces rework costs that occur from integration problems. By enabling testing throughout the development process, continuous integration supports agile practices and balancing quality with speed.
Enabling agility with continuous integration testing
- 1. WELCOME Enabling Agility with Continuous Integration Testing Monica Luke September 18, 2012
- 2. IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
- 3. Agenda • Integration Testing Challenges • Continuous Integration Testing • Agile Integration Testing
- 4. Taking a new look at integration testing Do you know what’s underneath the .94 surface? Given high .98 individual x3 systems’ reliability .97 (>.92) .92 Unreliability increases exponentially: R = .79 “In practice, integration has proven to be the most enlightening activity in exposing the risks and uncertainties in software development.”** **Walker Royce, Chief Software Economist at IBM 4
- 5. An estimated 60 - 80 percent of the cost of software development is in rework* Increasing Increasing Balancing Cost of Quality Development Complexity Quality and Speed Public Cloud Private Cloud Traditional Routing Web/ Service Collaboration Internet Content Business Testing Providers EJB Partners Third-party Shared Archives Testing costs Services Portals Services Messaging File Services systems Directory Identity Other costs Data Warehouse Mainframe Enterprise Service Bus Heterogeneous Environments Outsourcing labor is no longer Product and application Productivity is inhibited as test a sustainable model as global wages complexity and size teams can no longer keep up with are increasing are increasing agile development * Source:http://www.sei.cmu.edu/about/message/
- 6. Heterogeneous systems are complex Directory Service Third-party Outsourced IP and Globally Distributed Mobile Device Service-oriented Access Architecture (SOA) In-house Development Partner Commercial Code Off-the-shelf End-user Open Main Cloud / SaaS Web Application Source Frame
- 7. And… hard to configure for integration tests Content Business Partners Providers Web/ Internet Shared Services Portals Messaging Services Routing Service Third-party Directory Identity Services Application Collaboration File systems EJB Enterprise Service Bus Data Warehouse Mainframe Archives
- 8. Does your organization look like this? I just spent 2 weeks setting this up & the build is no good?$%^?? Developer Setup integration test Developer environment Developer Developer Developer Developer Tester N days to install & configure N nightly builds are piling up
- 9. What if we could change it? Takes up to 50% of test cycles QA Stage Develop Independent Test Production Agile Iteration 1… n Days Days Days Days Nightly build Integration Test Package ready Deliver App Package Delay Delay to release Delay Delay into Production Manual setup of Manual setup of Manual setup Manual setup Check ins Test Env Test Env of Test Env of Test Env Run Test Fix Defects Test & Verify Test & Verify Find Defects Test Setup is minutes Time spent on meaningful work expands
- 10. Agenda • Integration Testing Challenges • Continuous Integration Testing • Agile Integration Testing
- 11. What is Service Virtualization? Public Private Cloud App Under Test Cloud Routing App Under Test Service Content EJB Business Collaboration & Model Providers Partners Third- Shared Archives Services party Portals Capture Services Messaging File Services Databases Mainframe systems Third-party Director applications Services y Identity Packaged apps, messaging services, etc. Enterprise Virtual Services Data Mainframe Service Bus Warehouse Heterogeneous Environments •Virtual Services simulate the behavior of an entire application or system during testing •Virtual Services can run on commodity hardware, private cloud, public cloud •Each developer, tester can easily have their own test environment •Developer and testers continue to use their testing tools (Manual, Web performance, UI test automation) 11
- 12. Testing integrations as an evolution Services, applications, systems are Actual Service/App introduced into the continuous integration Virtual Service/App cycle in a prioritized, controlled fashion Incremental Integration Testing 12
- 13. Continuous integration Testing Example C1 C2 C3 ERP WSDL 3rd party UDDI Pass/Fail Test my own piece Real V V V • Test C1 with three virtualized services. • Can use simple or complex integration scenarios. • Quick to setup and low-cost. 13
- 14. Continuous integration Testing Example C1 C2 C3 ERP WSDL 3rd party UDDI Pass/Fail Time Test my own piece Real V V V Integrate w/another Real R V V V This won’t slow me down! Real V V V V First Complete Real V V V V V V System Test All components Real R R V V V V Integrated! • Integrate C2 and C3… with virtualized backend systems • Use enterprise integration scenarios for Testing, inc BVT! • Quick to setup and low-cost. 14
- 15. Continuous integration Testing Example Time C1 C2 C3 ERP WSDL 3rd party UDDI Pass/Fail Test my own piece Real V V • Gradually integrate with expensive backend V systems. Integrate w/another Real R V V • Minimize use of backend systems for testing V This won’t slow me down! Real purposes V V V V • Process offers easy and cost effective defect First Complete System Test Real isolation V V V V V V All components – Integrated! Real R R V V V V The expensive backend Real R R R R V V All Systems UP! Real R R R R R R 15
- 16. Continuous Integration testing for builds Integration Install & Publish Compile Unit Test Test Run Tests Configure Results Environment Rational Quality Manager Green” builds have passed meaningful integration tests
- 17. Agenda • Integration Testing Challenges • Continuous Integration Testing • Agile Integration Testing
- 18. Complex systems Staged agility Iteration 1 Design, Code, Test Iteration 2 Design, Code, Test Weeks 2 - 6 Weeks 8 -12 Integration test Integration test iteration #1 iteration #2 Iteration planning Week 1
- 19. “Stable, working code”? • Are integration tests pushed later and later in the release cycle? • Collaboration challenge: Expectation is now that test is aligned with development • Are developers doing integration validations? • Are you compromising on the definition of stable, working code?
- 20. End-to-end Lifecycle Management Agile Agile with Service Virtualization Accelerators Agility@Scale Continuous Delivery RM DM SCM Build QM Stage Monitor Capabilities CM Planning Lifecycle Design Develop Stage Production
- 21. Whole team alignment • Whole team collaboration • Dev & Test are working on the same code base at the same time • Change the perspective on team metrics • Time to fix defects vs number of defects • Integration testing is happening on every milestone and build • Whole system testing and verification • In Agile terms, you are really “done, done, done” on every milestone • Alignment builds trust delivering on agile vision of self-directed teams
- 22. Introducing IBM Rational’s Service Virtualization and Test Automation Solutions Rational Test Workbench Enable functional, performance and integration testing QA staff throughout your project lifecycle Functional Testing With a scriptless, wizard-driven test authoring environment and support for more than 70 technologies Test Integration Testing Workbench Performance Testing and protocols Regression Testing Rational Test Virtualization Server Model real system behavior to eliminate test Test Load Agents dependencies and simulate production, Servers Virtualization Agents Accelerate testing and reduce the setup and infrastructure costs of traditional testing environments. Rational Performance Test Server Reuse your functional integration tests for performance testing Combine your tests to model real-world scenarios and Test Lab assess the impact of load on your integrated Infrastructure infrastructure. 22
- 23. IBM Rational Quality Management Offerings IBM Collaborative Application Lifecycle Management Rational Quality Manager Quality Dashboard Requirements Test Management Management Defect Management Create Build Manage Execute Report Plan Tests Test Lab Tests Results Best Practice Processes Administration: Users, Collaboration projects, process Presentation: Storage Discovery Search & Query Mashups Unit Testing Open Lifecycle Service Integrations Test Data Integration Management Testing Functional Performance Test Testing Testing Virtualization Rational Integration Optim Tester TDM Rational Rational Rational Test 23 Functional Tester Performance Tester Virtualization
- 24. www.ibm.com
- 25. www.ibm.com © Copyright IBM Corporation 2012. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.