All the world's a stage – the next step in automated testing practices
2 likes1,436 views
The document discusses advanced automated testing practices, emphasizing the importance of execution over ideas and the value of well-maintained test suites. It introduces the Screenplay Pattern as a user-centric model that enhances maintainability and code reuse, while also highlighting factors affecting test suite value, such as maintenance overhead. Key principles of software engineering, like separation of concerns and narrative coding styles, are presented to support better testing strategies.
All the world's a stage – the next step in automated testing practices
- 1. @JanMolak@wakaleo The next step in automated testing practices All the world’s a stage
- 2. @wakaleo @JanMolak John Ferguson Smart John Ferguson Smart “I help teams of smart people learn to work together more efficiently, to deliver better software sooner”
- 3. @wakaleo @JanMolak Jan Molak “Having great ideas is not enough, it’s the execution that matters” JanMolak [email protected] Jan Molak
- 4. @wakaleo @JanMolak Test Automation Do it well, or don’t do it at all
- 5. @JanMolak@wakaleo How much are your tests worth? Weekly cost vs weekly savings $0 $20,000 $40,000 $60,000 $80,000 $100,000 $120,000 $140,000 $160,000 $180,000 1/09/2015 1/10/2015 1/11/2015 1/12/2015 Test0suite0cost0vs0savings0(per0week) Cost0saved0par0week Test0suite0cost0per0week Team cost is constant Time saved increases with the number of executed tests
- 6. @JanMolak@wakaleo How much are your tests worth? $0 $200,000 $400,000 $600,000 $800,000 $1,000,000 $1,200,000 1/09/2015 1/10/2015 1/11/2015 1/12/2015 What0is0your0test0suite0worth? Test0suite0ROI Test0suite0cost Cumulated cost vs cumulated savings Cumulated value should increase faster than cumulated cost
- 7. @JanMolak@wakaleo Now factor in high maintenance 40-60% Typical maintenance overhead for poorly-written test suites
- 8. @JanMolak@wakaleo Now factor in high maintenance Number of tests Maintenanceoverhead
- 9. @JanMolak@wakaleo $0 $50,000 $100,000 $150,000 $200,000 $250,000 $300,000 $350,000 $400,000 $450,000 $500,000 1/09/2015 1/10/2015 1/11/2015 1/12/2015 1/01/2016 What0is0your0test0suite0worth? Test0suite0cost Test0suite0ROI Now factor in high maintenance High-maintenance costs reduce the number of tests a team can write High maintenance costs can cause your test suite to lose value
- 10. @wakaleo Narrative Coding Style Open-Closed Principle Single Responsibility Principle Separation of Concerns Software Engineering Principles can help you write more maintainable tests
- 11. @wakaleo Narrative Coding Style Open-Closed Principle Single Responsibility Principle Separation of Concerns Software Engineering Principles can help you write more maintainable tests
- 12. @wakaleo Narrative Coding Style Open-Closed Principle Single Responsibility Principle Separation of Concerns Software Engineering Principles can help you write more maintainable tests
- 13. @wakaleo Narrative Coding Style Open-Closed Principle Single Responsibility Principle Separation of Concerns Software Engineering Principles can help you write more maintainable tests
- 14. @wakaleo Narrative Coding Style Open-Closed Principle Single Responsibility Principle Separation of Concerns Software Engineering Principles can help you write more maintainable tests
- 15. @JanMolak@wakaleo Separation of Concerns - Layers Goals Tasks Actions What are you trying to achieve? What do you need to do to achieve this goal? What interactions with the system do you need for each task?
- 16. 16
- 17. @JanMolak@wakaleo Separation of Concerns - Layers Scenario: Add a new todo entry on the todo home page Given I am on the Todo application home page When I enter 'Buy some milk' into the New Todo field And I press ENTER Then the todo list box should contain 'Buy some milk'
- 18. @JanMolak@wakaleo Separation of Concerns - Layers Scenario: Add a new todo entry on the todo home page Given I am on the Todo application home page When I enter 'Buy some milk' into the New Todo field And I press ENTER Then the todo list box should contain 'Buy some milk' Scenario: Record a new todo action for future use Given I need to buy some milk When I add the todo action 'Buy some milk' Then 'Buy some milk' should be recorded in my todo list
- 19. @wakaleo Feature documentationDetailed feedbackHigh level feedback Communication and documentation
- 20. @wakaleo @JanMolak Serenity BDD “Because life is too short to h a v e t o m a i n t a i n u g l y automated test suites”
- 21. @wakaleo Living documentation Strong Selenium WebDriver support JUnit, Cucumber or JBehave A layered approach Serenity BDD
- 22. @JanMolak@wakaleo The Serenity layered architecture Goals Tasks Interactions Tests or scenarios “Steps” Page Objects or other technical components
- 23. @JanMolak@wakaleo The Serenity layered architecture Feature: Add new todos James needs to be able to jot down actions he needs to do as he thinks of them Scenario: Record a new todo action for future use Given James needs to buy some milk When James adds the todo action 'Buy some milk' Then 'Buy some milk' should be recorded in his todo list A goal A task @Steps ATodoUser james; @When(“^(?:.*) (?:adds|has added) the todo action '(.*)'$") public void add_the_todo_action(String actionName) { james.adds_an_action_called(actionName); } The task definition TodoPage onTheTodoHomePage; @Step public void adds_an_action_called(String actionName) { onTheTodoHomePage.addAnActionCalled(actionName); } An interaction
- 24. @JanMolak@wakaleo Serenity for Living Documentation Feature: Add new todos I need to be able to jot down actions I need to do as fast as I think of them Scenario: Record a new todo action for future use Given I need to buy some milk When I add the todo action 'Buy some milk' Then 'Buy some milk' should be recorded in my todo list
- 26. 26
- 27. @wakaleo Hard to apply to Single Page Apps Need modifications to extend Multiple Responsibilities Bloat Page Objects considered harmful
- 28. @wakaleo @JanMolak Introducing the Screenplay Pattern
- 29. @wakaleo Highly readable style Encourages reuse and maintainable, scaleable code Small, reusable interaction components User-centric The Screenplay Pattern
- 30. @JanMolak@wakaleo Origins of the Screenplay Pattern in Serenity Antony Marcano “Page Objects kinda suck. Why not try this pattern instead?” Agile Alliance Functional Test Tools Workshop 2007 2007 * * Based on actual events
- 31. @JanMolak@wakaleo Origins of the Screenplay Pattern in Serenity Antony Marcano 2007 2009 More people should know about this pattern JNarrate Andy Palmer Let’s write a reference implementation!
- 32. @JanMolak@wakaleo Origins of the Screenplay Pattern in Serenity Antony Marcano 2007 2009 “Coronation Street” rocks 2012 Andy Palmer Eureka! “Soap Opera Personas”
- 33. @JanMolak@wakaleo Andy Palmer Origins of the Screenplay Pattern in Serenity Antony Marcano 2007 2009 2012 Jan Molak You should call this “The Journey Pattern” Damn! Why didn’t we think of that!
- 34. @JanMolak@wakaleo Andy Palmer Origins of the Screenplay Pattern in Serenity Antony Marcano 2007 2009 2012 Jan Molak 2015 John Smart Serenity is cool So let’s make it happen! But it needs the Journey Pattern
- 35. @JanMolak@wakaleo Andy Palmer Origins of the Screenplay Pattern in Serenity Antony Marcano 2007 2009 2012 2015 John Smart Actually, the pattern has evolved a lot Jan Molak Let’s call it the ‘Screenplay’ pattern
- 36. @JanMolak@wakaleo The Screenplay Pattern - A User-Centric model Actors have goals I’d like to be able to recall all the things I need to do Meet James
- 37. @JanMolak@wakaleo The Screenplay Pattern - A User-Centric model Actors have abilities I can browse the web with my browser I can also query REST services
- 38. @JanMolak@wakaleo I’d like to be able to recall all the things I need to do The Screenplay Pattern - A User-Centric model Actors perform tasks I’ll add ‘Buy some milk’ to my todo list
- 39. @JanMolak@wakaleo I’d like to be able to recall all the things I need to do The Screenplay Pattern - A User-Centric model Actors may interact with the system to perform these tasks I’ll add ‘Buy some milk’ to my todo list Type ‘Buy the milk’ Press ‘ENTER’
- 40. @JanMolak@wakaleo The Screenplay Pattern in Serenity Actors have goals Feature: Add new todos James would like to be able to recall all the things he needs to do Scenario: Record a new todo action for future use Given James needs to buy some milk When James adds the todo action 'Buy some milk' Then 'Buy some milk' should be recorded in his todo list I’d like to be able to recall all the things I need to do
- 41. @JanMolak@wakaleo A User-Centric model Actors have abilities I can browse the web with my browser Actor james = Actor.named(“James"); @Managed WebDriver hisBrowser; … james.can(BrowseTheWeb.with(hisBrowser));
- 42. @JanMolak@wakaleo A User-Centric model Actors perform tasks I’ll add ‘Buy some milk’ to my todo list @Steps AddItem addATodoItem; … james.attemptsTo(addATodoItem.called("Buy some milk"));
- 43. @JanMolak@wakaleo A User-Centric model Actors perform tasks I’ll add ‘Buy some milk’ to my todo list @Steps AddItem addATodoItem; … james.attemptsTo(addATodoItem.called("Buy some milk")); actor.attemptsTo( Enter.theValue(thingToDo).into(ToDoList.NEW_TODO_FIELD), Hit.the(RETURN).keyIn(ToDoList.NEW_TODO_FIELD) ); Type ‘Buy the milk’ Press ‘ENTER’
- 44. @JanMolak@wakaleo A User-Centric model james.should(seeThat(TheItems.displayed(), hasItem("Buy some milk"))); using a Hamcrest matcher of a question regarding the state of the application Check the answer
- 45. @JanMolak@wakaleo A User-Centric model Actor Abilities has Tasks Actions performs made up of enable Questions asks Elements interact with about the state of Screen on a
- 47. 47
- 48. @wakaleo Focus on the user journey and a common business vocabulary Scales well to teams with varying levels of experience Good design principles leading to lower maintenance costs Code reuse within and across teams Key Benefits
- 49. @JanMolak@wakaleo References • https://github.com/serenity-bdd/serenity-web-todomvc-journey • “Beyond Page Objects: Next Generation Test Automation with Serenity and the Screenplay Pattern” - InfoQ • “Page Objects Refactored: SOLID Steps to the Screenplay/ Journey Pattern” - DZone • http://www.serenity-bdd.info
- 50. @JanMolak@wakaleo Want to learn more? Ask us about training and mentoring John Ferguson Smart wakaleo http://johnfergusonsmart.com Jan Molak http://janmolak.com JanMolak