Cerberus : Framework for Manual and Automated Testing (Web Application)
Download as PPTX, PDF3 likes1,216 views
Cerberus is an open-source web application developed by La Redoute since 2011, aimed at automating testing across multiple technologies and environments. It emphasizes the need for a common test repository and automation to improve the quality and speed of software development, integrating with various IT services and tools. The application has implemented automated testing for over 3500 tests, enhancing project management and execution monitoring for improved quality assurance.
Cerberus : Framework for Manual and Automated Testing (Web Application)
- 2. What is it ? Web Application . Developed by La Redoute since 2011, then published in Open source. Source code and documentation available on github and sourceforge : https://github.com/vertigo17/Cerberus Almost 2600 commits by 10 contributeurs Centralized test repository. Automate multi-technologies. (Web, Application Mobile, Heavy Client, SOAP, Rest, SQL…) Multi-environment (DEV, QA, UAT, PROD….) Multi-langages (FR, UK, RU, IT, PL, …)
- 3. Business Development Team Test Team Where does it come from ? • From the observation of the limits of a traditional organization. • Silo Organisation. • Test definition during or after development phase. The possible interpretation of a functional specification can lead to a number of significant go back . The test case, correctly described in earlier in the development phase , is the most effective way to prevent the poor quality . Need for a common test repository to the various players , fed earlier in the development phase.
- 4. Where does it come from ? • From the experimentation of new organizations : Agile / Continuous Integration / Devops time risk timeShort iteration Long iteration Need for test automation No automated Tests = No quick feedback = No short iteration = Increasing risk to production deploy. Regression risk evolution vs delay between each deploy DevOps is an effort to create business value by involving all stakeholders in the software development ( from design to production , including the developers ) through short iterations and recognize the value closely , consists of experimentation and rewind.
- 5. Where does it come from ? • From the experience in test automation. To guarantee test maintenability. a.Avoid code duplication Use step library. b.Guaranty functional test coverage. Functional test coverage should be defined by business/MOA team. c. Deploy one unique repository. Link functional information with technical implementation to allow to easily maintain one unique repository. Need for a test framework, including functional and technical information.
- 6. Test Automation Web (Selenium) IOS and Android Apps (Appium) Heavy Client / GUI (sikuli) Web-services (xml unit) Databases (sql connectors - jdbc) Why do it ? • A common test repository to the various players , fed earlier in the development phase. • Test automation whatever the technology. • Test framework, including functional and technical information. Test Repository Functional description ACTION / CONTROLES / DATA Test centralisation Step library Test description standardisation. Testing Management Testcase execution Dashboard. Performance metrics. Log (screenshot). Standardize test reporting. Link to ticketing system.
- 7. Cerberus Usage: In the context of the project.
- 8. Business Dev TeamQA Team Business Dev TeamQA Team Cerberus usage : in the context of the project Team exchanges around a centralized tool. Become
- 9. Cerberus usage : in the context of the project 1 –Test specification allowing to guaranty the quality of the project • Who ? Business, AMOA. • Where ? In the description fields. • How ? Following the frame (Action/Control). Using step library and data library. • When? As soon as possible in the project. Ideally during functional specification.
- 10. Cerberus usage : in the context of the project 2 –Test Automation • Who ? Developers, QATeam. • Where ? In the script fields. • How ? Following the frame. • When? During development phase. When user interface are designed.
- 11. Cerberus usage : in the context of the project 3 –Test Execution • Who ? All project actors. • Where ? Manual & Automated. In the test execution page. • How ? Manual: Allow to define status for each action. Automated : Launched manually or for continuous integration chain. • When ? During the test phase. Manual execution
- 12. Cerberus usage : in the context of the project 4 –Test project management • Who? Every project actors. • Where ? Reporting page • How ? Follow execution status. Interface with several ticketing tools. • When? During the test phase.
- 13. Cerberus usage : in the context of the project Synthesis: Test specification. • Suivant le cadre défini (Data/Actions/Contrôles) • En utilisant les données et étapes de libraires Test automation (20/80) • Pour le périmètre qui n’est pas en librairie / données manquantes • Permettant de valider les cas passant et quelques cas bloquant Project test execution (manual) Business Dev TeamQA Team GO/NO GO on common reporting Project testing become regression testing for next projects. Project test execution (automated) Regression test execution (automated)
- 14. RegressionTesting : key figures La Redoute : For the perimeter www.laredoute.xx (10 countries) and m.laredoute.xx (10 countries) : 3500 tests launched twice a day (UAT and PreProd). 4 production deploy for each application. Speed of test creation increasing, due to library usage. Evolution du périmètre des TNRs Aout 2014 180 Tests Aout 2015 2000 Tests Janvier 2016 2900 Tests Avril 2016 3500 Tests
- 16. Functional monitoring Run functional to guaranty . Functional test execution (Example : 10 scenarii every 5 minutes) Technical data records • Response time per action • Network Trafic (HAR file) Data exploitation • Services allowing to raise alert on non OK status. • Publish data into Elastic Search /Kibana • Business Activity Monitoring
- 17. Functional monitoring : key figures Follow the availability of the key scenarios: Homepage / Authenticate / Account creation / Navigation / Product List / Product Page / Comparison / Basket / Delivery / Payment At a pertinent frequency: Every 5 minutes, 21h a day >> 24.000.000 step every year for1 application Decorelate the cost of the monitoring from the execution frequancy allow to avoid to opposite quality with cost of the necessary solution.
- 18. Integration into IT service
- 19. Cerberus in an InformationTechnology service Cerberus can be interfaced with a lot of popular tools already in place in most of the IT service Oracle SQL MySQL PostGreSQL DB2 Microsoft SQLServer SSAS Can be launched by any task scheduler. • Via 1 api REST • $U, cron, Jenkins…. • Execute 1 test or even a campaign Push execution status. • OK/KO for a go/no go status Push into ES/Kibana Automatic ticket opening. • In Mantis / Redmine / …. • Forward testcase information Can be intefaced with Centreon Allow to Read/Write any Database.
- 20. Architecture Application server Actif / Actif (Glassfish / Tomcat / Jboss) Database Actif/Passif (MySQL / MariaDB) Robots Selenium / Sikuli (GUI Testing)
- 21. Architecture : For a POC Technical information: 1 VM (Linux) 4 vCPU 8 GB RAM 50 Go Disk Space Application : MySQL (v5.6.xx) Glassfish 4.1.1 Technical information: 3 VM (Linux) 2 vCPU 4 GB RAM 10 Go Disk Space Application : Selenium server Firefox
- 22. RoadMap 2016
- 23. RoadMap 2016 https://github.com/vertigo17/Cerberus/issues • GUI Refactoring (70% Done) • Bootstrap • Interface Internationalisation • Interfacing with Jmeter • Load test scenario centralisation • JMX generation (in study) • Trigger Execution • TestCaseVersionning • Plugin management • Add action and control as plugin