Automated Testing in DevOps
5 likes1,059 views
Integrating automated testing tools and concepts into pipelines. 1. Coded UI is used for UI testing and Postman + Newman for API testing of a license manager application. Tests are run in parallel on different browsers in the pipeline after deployment. 2. An existing testing framework uses Selenium for UI testing a web application. Tests are structured using page objects and run on Jenkins. 3. Tools and strategies discussed include Page Object Model, test frameworks, test environments, and communication during collaboration on testing. The goal is faster, more reliable releases through prioritized, efficient automated testing.
Automated Testing in DevOps
- 1. Automated Testing in Pipelines Integrating Tools and Concepts Layla Franke, Vlad Ilie, Alina Ionescu dev.haufe.com github.com/Haufe-Lexware @HaufeDev -Lexware
- 2. Scope of this Talk • Coded UI for UI, Postman + Newman for API (LicenseManager – Layla) • Testing Framework for Web UI in Java with Selenium (Umantis – Vlad) • Framework in Java for API testing, Soap UI for API testing (Foundational Services – Alina)
- 3. 1 LicenseManager - Layla Coded UI for UI, Postman + Newman for API
- 4. License Manager • C# Web Application • Inhouse Tool for Management of Serial Numbers • Existing Continuous Delivery Pipeline in Go.CD
- 5. GUI Tests Coded UI Tests in Visual Studio: • Recording via Internet Explorer and built-in recorder • Execution of tests from Commandline with MSTest • Playback of Tests in various browsers
- 6. Build Pipeline Compile Deploy Application Coded UI Test Internet Explorer View Test Reports Do other stuff Pulish App QualityDev Coded UI Tests – Pipeline integration • Compile your code normally • Deploy you application on test machine • Start Coded UI Tests You can use MSTest.exe on Go Agents • Use parameterized URL and Browser Define environment variables in GoCD • Test Replay runs on Go Agent • Coded UI Tests Finished View HTML Test Reports and Coverage Report in Pipeline Coded UI Test Chrome Coded UI Test Firefox Go Agent Go Agent Go Agent
- 7. Coded UI Test Reports - Overview
- 8. Coded UI Test Report - Failed
- 9. Coded UI Test Report - Passed
- 10. Create REST API Tests using Postman Use Postman to create REST API Tests: • Export Tests as a „Postman Collection“ • Have a variable URL using a „ Postman Environment”
- 11. Newman – Pipeline integration • Deploy application on App Server • Execute Newman Tests Newman is a command line tool used for automatic execution of tests created with Postman • View Test Results in Pipeline Build Pipeline Go Agent Newman Test QualityDev Pulish App App Server View Result Compile Deploy App to Server
- 12. CD pipeline integration How to integrate in CD Pipeline? Visual Studio Coded UI Tests • Test DLL is created in Compile Job • Tests run after deployment on test environment in stage QualityDev • Tests for different browsers can be executed in parallel Newman REST API Tests • Can be executed after deployment on test environment
- 13. 2 Umantis - Vlad Testing Framework for Web UI in Java with Selenium
- 14. About Umantis • Web project (perl) • 3 applications (with stable and beta customers) • Bugfix release every 2 weeks • Feature release every 3 months • Prio1 releases
- 15. Test Project Structure • Java project • Built using maven • Selenium web driver • Page Objects • testNG • Git/gerrit • Jenkins
- 16. 1 LicenseManager - Layla Coded UI for UI, Postman + Newman for API
- 17. 1 LicenseManager - Layla Coded UI for UI, Postman + Newman for API
- 18. Page Objects • Keep UI elements private (don’t expose them to tests) • Expose user actions • Use data structures instead of multiple parameters public class JobApplicationExternalFormPage extends PageBase { @FindBy(id = "value_153712") WebElement dataReleaseRadioButtonGroup; .............................. @FindBy(id = "Button_1161113") private WebElement submitFinalApplicationButton; @FindBy(id = "Button_112465") private WebElement continueButton; @FindBy(id = "errorcollectionlabel_3606") private WebElement errorMessage; ............................. private void fillJobApplicationForm(User applicant) { UIActions.waitForElement(titleRadioButtonGroup); UIActions.setValue(new RadioButtonGroup(titleRadioButtonGroup), applicant.getTitleAsValue()); UIActions.setValue(firstNameEditBox, applicant.getFirstName()); ...................................................................... UIActions.setValue(letterOfApplication, applicant.getLetterPath()); UIActions.setValue(resume, applicant.getCvPath()); UIActions.setValue(document1, applicant.getDoc1Path()); try { UIActions.setValue(new RadioButtonGroup(dataReleaseRadioButtonGroup), RadioButtonValue.DATA_NOT_AGREE.getValue()); } catch (NoSuchElementException ignored) { } UIActions.click(iAgreeCheckBox); } .............................
- 19. Page Objects - examples • PageBase.java • CreateNewEmployeePage.java • AddPartialEventPage
- 20. UIActions • Static Class • Entry point for all ui actions • setValue() and checkValue() for different types of controls
- 21. Test Execution • Jenkins – a job for each product and branch • Local – using intelliJ • On each commit (continuous integration) using gerrit and jenkins • Maven • Command line (can be used with any CI or CD tool): java –jar umantis_test.jar ...
- 22. Test Environment • Selenium Grid using docker containers • Build test project container (based on maven image) • Test execution container (based on jdk image) • Selenium grid (based on hub and node selenium images)
- 23. Test Results - overview
- 24. Test Results – Detail • Test failure with history • Exception with stack trace • URL, screenshot and html where the failure occured
- 25. License Manager • C# Web Application • Inhouse Tool for Management of Serial Numbers • Existing Continuous Delivery Pipeline in Go.CD
- 26. What we started from: •Atlantic – the monolith 15 + clients…
- 27. Splitting the testing: • 1. DEV Tests – Junit tests • 2. System Tests - Junit tests • 3. SOAP UI tests • 4. Security Tests • 5. Load Tests • 6. Rollout (Docker) tests • 7. UI Automated tests • 8. Exploratory (manual)
- 28. The Testing Framework Junit Test Framework Atlantic Code Clients Helpers Tests SoapUI Tests DEV Unit Tests Security Tests Load Tests Rollout Docker Tests Atlantic Client Atlantic Client Atlantic Client HP ALM UI Tests Manual Tests
- 30. Tests examples: @Test public void get_userWithUserId() { UserDto user1 = UserClient.createNewPendingActiveUser(); Map<String, String> queryParams = new HashMap(); queryParams.put("userid", user1.getUserID().toString()); com.haufe.atlantic.old.usermanagement.rest.dtos.UserDto result = RestUtilForOldResources.get(com.haufe.atlantic.old.usermanagement.rest.dtos.UserDto.class, headers, queryParams, PathsHelper.umPaths().users()); assertUserValues(user1, result, true);
- 31. Where are they triggered from: • After deployment, from GoCD there are triggered 2 Docker compose files: • 1 with Maven for the Junit tests • 1 with SoapUI for the SoapUI tests They run in parallel.
- 33. Things we learned • Achieve “good enough” coverage • Prioritize • Take/create shortcuts if you can • Bypass Logins if possible • Write small “straight to target” tests (no end-to-end anymore) • Use more code and API less UI • Communication is key to good collaboration
- 34. Goal? Faster, more reliable releases.
- 35. GUI Tests Coded UI Tests in Visual Studio: • Recording via Internet Explorer and built-in recorder • Execution of tests from Commandline with MSTest • Playback of Tests in various browsers