Real Testing Scenario Strategy Practical TestOps Presentation
- 1. Real Testing Scenario Strategy: A Real-life TestOps Environment By Adam Sandman Inflectra Corporation June 21st, 2017
- 2. About Me • Adam Sandman was a programmer from the age of 10 and has been working in the IT industry for the past 20 years. • Currently Adam is a Director of Technology at Inflectra Corporation, where he is interested in technology, business and innovation. • Adam lives in Washington, DC with his family and is excited to be visiting Israel after an absence of 35 years.
- 3. Objectives • Learn about the different tiers of testing • Understand how tools can address the challenges of testing each tier • See a sample architecture for a typical web- based multi-tier application • Ask me lots of challenging questions, so that I get over my jet lag!
- 4. Agenda • Requirements Discovery and management, linked to: – Manual testing – Unit testing (e.g. NUnit, MS-Test, Junit) – UI testing with Rapise – Load testing with NeoLoad or JMeter – API Testing with Rapise • Connecting to CI server such as Jenkins. • Dashboards and reporting
- 5. What Are We Going to Test? A simple n-tier ASP.NET web application:
- 6. What Are We Going to Test? Also We Need to Test Its RESTful APIs:
- 7. So What Do We Need to Do First? • Define Our Requirements • Develop Our Test Strategy
- 8. What Are The Requirements? • We need a set of “requirements” to write our tests, regardless of whether we’re using an Agile or Waterfall methodology.
- 9. What Our Our Requirements? Traditional Requirements Matrix Agile User Stories
- 10. Now What’s Our Test & Dev. Strategy Source Code Mgt. Requirements, User Stories, etc. Subversion Git Continuous Integration (Jenkins) Automated Tests (Checks) Test Environments Acceptance Tests Installation Process Session Testing
- 11. What Tests Should We Automate?
- 12. There Is Never Enough Time for Testing You should not seek to automate 100% of all your tests, because manual testing and automated testing are more efficient at different things
- 13. What Tests Should We Automate? • Automate the repetitive, stable, unchanging, boring things that are ideal for automated testing • Let manual testing find the lurking items that you don’t even know exist (yet!) and cannot automate.
- 14. Continuous Integration (CI) • Ensure that you have a robust set of “smoke” tests that can run after every CI build – Unit, API and Automated UI tests
- 15. Monitoring Test Activities • In the next sections we will discuss the different types of testing. • It is important to be able to monitor all testing activities and track key metrics – Requirements test coverage – Code test coverage – Regression coverage of at-risk areas
- 16. Monitoring Test Activities • Modern test management tools include powerful dashboards and reports.
- 17. Unit Testing • Unit test frameworks are chosen for each technology you use: • Java Development – jUnit or TestNG • .NET Development – NUnit or MS-Test • Similar frameworks for Ruby, Perl, Python, JS, C • Tie them into your test management platform if possible
- 18. API / Service Testing • APIs are now more important than the applications • Business Models such as UBER or AWS Rely on APIs • You need to regression every version of the API you support
- 19. API / Service Testing • Tools such as SOAP-UI, Rapise, POSTman can test different APIs • Some are limited to SOAP, REST, etc. others cover multiple types • Make sure you tie them back into your test management system
- 20. Automated User Interface (UI) Testing • A lot of business rules, functionality still in the UI layer • Automate the parts that are stable and time consuming to manually test. Beware the negative ROI of testing 100% of a highly changing UI. Humans deal with change better* (*until AI solves it!)
- 21. Automated User Interface (UI) Testing • There are various UI testing tools available: – Free tools such as Selenium, Appium, etc. require you to be a programmer – Commercial tools such as Rapise, Ranorex, UFT, TestComplete make it easier for the non-programmer – Choose the tool(s) that work with your application and your testing team’s skill level
- 22. Non-Functional Testing • Don’t leave Performance Testing Until the End (*Big Mistake*) • There are many tools to choose from (JMeter, NeoLoad, LoadRunner, etc.) • Make sure you tie it into your test management platform
- 23. Manual Testing / User Testing • There are many good test management tools to choose from. Don’t default to using Excel, Google Sheets • Understand UAT manual testing vs. Exploratory Testing – They are different and should be used appropriately
- 24. A Practical Example in Action
- 25. Questions?