Olli-Pekka Puolitaival - Model-Based Testing for Integration Testing in Real Production - EuroSTAR 2012
0 likes•398 views
Olli-Pekka Puolitaival discusses model-based testing (MBT) and introduces the Osmo Tester, which is an open-source, flexible, and easy-to-extend tool for automating software testing. He emphasizes the importance of high-level, modular modeling and automating test runs for verification and efficiency in testing core functionalities. The document also outlines experiences, challenges, and future steps for enhancing testing across multiple platforms.
![Osmo tester idea
public class ExampleModel{
int i = 0;
@Guard(“increase")
public guardfunction() {
return true; //Return true if this transition is allowed to execute
}
@Transition(“example")
public void transitionfunction() {
System.out.println(i++); //This code is executed when transition is selected
}
…
public static void main(String[] args) {
OSMOTester tester = new OSMOTester(new ExampleModel());
tester.generate();
}
}
15
“Model is pure java code,
Which makes possible to use
Any java library”](https://image.slidesharecdn.com/w5-olli-pekkapuolitaivaleurostar2012final-141029101956-conversion-gate02/85/Olli-Pekka-Puolitaival-Model-Based-Testing-for-Integration-Testing-in-Real-Production-EuroSTAR-2012-15-320.jpg)
Olli-Pekka Puolitaival - Model-Based Testing for Integration Testing in Real Production - EuroSTAR 2012
- 2. Olli-Pekka Puolitaival • Lead quality engineer test automation • 5 yrs test automation • 4 yrs research Protecting the irreplaceable | f-secure.com
- 3. Index • Idea of model-based testing • Story of OSMO tester • How we test • Experiences & tips
- 4. MBT is new way to think • Traditional way to think • People do testing • Regression automation reduce peoples work time • New way to think • Smart people want to do testing in smart way • Computers are fast and hard worker, let them also try “stupid” things
- 5. 5
- 6. 6
- 7. 7
- 8. 8
- 9. 9
- 10. 10
- 11. 11
- 12. 12
- 13. 13
- 14. Why we made osmo? Traditional MBT tools • Hard to take in use • Closed • Limited extending • Advanced algorithms • Expensive Osmo tester • Easy start • Flexible • Easy to extend • Simple algorithms • Free (open source) 14
- 15. Osmo tester idea public class ExampleModel{ int i = 0; @Guard(“increase") public guardfunction() { return true; //Return true if this transition is allowed to execute } @Transition(“example") public void transitionfunction() { System.out.println(i++); //This code is executed when transition is selected } … public static void main(String[] args) { OSMOTester tester = new OSMOTester(new ExampleModel()); tester.generate(); } } 15 “Model is pure java code, Which makes possible to use Any java library”
- 16. Many additional features • Support online and offline MBT • Requirements tagging • Coverage calculations • Many test design algorithms, also manual drive • Coverage, time and length based stopping criteria • Self written adapters to the SUT • Custom test output formats • Easy to integrate with Jenkins • Model reuse and combining • Probability numbers support • Available: http://code.google.com/p/osmo/ 16 Easy to start and custom for your needs
- 17. Tips • Modelling • Good model is high level, modular and cover core functionality • Start from small and continue iteratively • Adaptation • Reuse code from test automation or development if possible • Running • Run tests always automatically • Fixed run length for verifying and ”unlimited” for finding some new • Reports • Report what you need to know, visual is mostly better • OSMO is not good if you need very sophisticated algorithms, but most people don’t need those 17
- 18. The domain is content cloud
- 19. Client smoke Our continuous integration Backend components Backend components Building Clients 19 Backend components Clients Client Integration test environment Unittest Clients Smoke test Regression test Backend integration Test Environment Performance test Performance Test Environment Client regression Chain of Jenkins jobs runs everything
- 20. Jenkins Android phone Android client CAN Solution Backend Current test setup “run once per hour” Selenium webdriver Firefox Web adapter Test status radiator Nativedriver client Nativedriver server Model OSMO Android adapter
- 21. What kind of model? • Idea of this testing • Do actions and verify that other clients data is same • Model includes actions which can be seen in other client like • Upload a file • Remove a file • Rename a file • Check quota • Check that file exist • Check that all files exist as supposed 21
- 22. Example test run 22 Osmo test engine Web page Android Client Login Upload file1.jpg Login Check file1.jpg Upload file2.jpg Remove file1.jpg Check that only file2.jpg exist Logout Logout
- 23. What are changing in test runs • New file set each time • New test accounts each time • Random amount of uploaded files • Randomly removed files • Randomly renamed files • Random checks 23
- 24. Experiences • Useful testing because • Tells well the status of core functionality • Is very close the new user use case • Test generation is easy to do with OSMO • Modeling is easy and support well iterative development • Test parameterization is easy • The challenges • Adapters are not easy to do and maintain • I keep my testing enough simple, then I don’t need too much maintenance • Automating test run was challenging in first time • Developers was helping me 24
- 25. Next steps • Look for new adapter for android • Extend testing on other platforms • Extend the power • Cover many environments • Many machines for testing • Run continuously several kind of test sets: • Short run for radiator • “infinite run” for finding weakest point 25
- 26. My MBT cooking receipt 1. Find out the core functionality 2. Make a simple model of that 3. Make adaptation for test running 4. Automate test runs 5. Run model continuously against SUT 6. Put the results visible in your room for seeing the “temperature” 7. Keep the radiator GREEN 26