Test Automation Day 2015 Keynote Alan Richardson - Practical Lessons Learned in Test Automation
13 likes4,133 views
The document discusses key lessons learned from test automation, highlighting the historical context and controversies surrounding automation in software testing. It emphasizes that automation is a process rather than a standalone tool, urging testers to learn programming and adopt a strategic approach to automate effectively. Key takeaways include starting small, prioritizing value, and solving problems quickly, along with the importance of hiring skilled personnel.
Test Automation Day 2015 Keynote Alan Richardson - Practical Lessons Learned in Test Automation
- 1. Practical Test Automation Lessons Learned Test Automation Day 2015 Alan Richardson CompendiumDev.co.uk @eviltester
- 2. Expect controversy when we automate in software testing. Automation has long been controversial. @eviltester2
- 5. “What we need is more automation.” Ford Motor company VP, Delmar S. Harder, Coined “automation” in 1948 1948 @eviltester5
- 6. Automation is not a ‘thing’ @eviltester6
- 7. Automation is a ‘process’ (automatization) @eviltester7
- 9. Automatization was hard to spell "...the author found automatization both awkward and - from the standpoint of his weak spelling - hazardous ... it was the ease of spelling that finally overcame the author's reticence to coin a new word" John Diebold, “Automation”, 1952 @eviltester9
- 10. The word “Automation” is... Norbert Wiener “hideous” Sir Anthony Eden “barbarous” @eviltester10
- 11. “Science”, May 6th, 1960 @eviltester11
- 12. “Science”, May 6th, 1960 'automatization', 'automata', 'strategy', 'machine', 'automatic', 'automated', 'programming' @eviltester12
- 14. “Testers don’t need to learn to program to create automation” Controversy. Argument. Conflict. Leads To @eviltester14
- 15. “Testers don’t need to learn to program, to program automata” You do need to learn to program to program Leads To @eviltester15
- 16. “Testers don’t need to learn to program, to define test data” (etc.) A clear strategy for automating against Leads To @eviltester16
- 17. Automating requires a change to your test process. Describe your process with the correct words. @eviltester17
- 18. When we tagged ‘automation’ on ● No time to automate ● Less ‘testing’ done ● No Maintenance of automata ● ‘automation’ justified by ROI ● ‘automation’ didn’t fit the metrics and tools @eviltester18
- 19. Natural to tag it on when... ● Requirements (xref to) ● Test Cases (lead to) ● Test Scripts (automated by) ● Automation @eviltester19
- 20. “Automate all the test scripts” “But we don’t have any test scripts” 20
- 22. "...automation makes it possible to do many things that could not be done without it..." John Diebold, Beyond Automation, 1964, pg 191 @eviltester22
- 23. How to start automating? @eviltester23
- 24. Start by figuring out ‘Why?’, then ‘Doing’ and ‘Learning’ @eviltester24
- 26. How to start automating? A structured approach for vendor procurement? @eviltester26
- 27. How not to start automating? A structured approach for vendor procurement @eviltester27
- 28. How not to start automating? ● Identify a set of tools ● Create objective feature assessment lists ● Add weightings to each objective ● Create RFx for tool vendors ● Await RFx Replies ● Review replies ● Choose Tool ● Evaluate and figure out how to use the tool@eviltester28
- 29. “Many managements genuinely seem to be looking at this subject as if they were walking around the edge of an ice cold swimming pool...” John Diebold, Beyond Automation, 1964, pg 66 @eviltester29
- 30. “...They realize that sooner or later they are going to have to jump in, but they try to postpone the leap with as much rationalization and fact-gathering as possible” John Diebold, Beyond Automation, 1964, pg 66 @eviltester30
- 31. Evaluate everything for a long time and end up with nothing @eviltester31
- 32. Evaluate everything for a long time and end up with nothing @eviltester32
- 33. How to start automating ● Why? ○ Because? ■ How? ● Experiment @eviltester33
- 34. Avoid the Danger of Ratholes. Understand your objectives @eviltester34
- 35. As a: ● Tester: ○ identify the areas to improve ● Manager: ○ prioritise and approve, ○ have a vision, ○ check in frequently ● Automator: ○ build small, solve fast, try multiple approaches @eviltester35
- 37. Ignore the Tools & Demos ● SSADM Test Tool ● Generic Test Case Management Tool ● Test Case Xrefer ● Metrics Collator ● Test Data Generator * 2 ● Keyword driven terminal emu automator ● Demos to ‘sell’ automation to clients @eviltester37
- 38. My First Automation was tools and frameworks @eviltester38
- 39. My First Automation was tools and frameworks @eviltester39
- 40. My first ‘good’ automation Automate the generation of ‘Scripts’ from ‘Paths’ to support interactive testing @eviltester40
- 41. Do start by solving problems @eviltester41
- 42. Don’t start by building tools or frameworks @eviltester42
- 43. Refactor to Flexible Abstraction Layers @eviltester43
- 44. Automate the easy stuff first ● start small ● the better you get, the easier things become, so start simple ● 80/20 : automate the 20%, gain 80% value @eviltester44
- 45. Do start with proof of concepts @eviltester45
- 46. Can we automate this system? ● can you execute some basic paths? ● check results where? ● individual control mechanisms? ● synchronisation methods? 46
- 47. Automate that which can be automated. Prioritize by value and ease. @eviltester47
- 48. How do you manage that which you do not know how to do? @eviltester48
- 50. “He must, as in the past, be so complete a master of the technical aspects of his profession that his subordinates will recognise his mastery, and be prepared to learn from him.” The Art of Leadership, Captain S.W. Roskill, 1964, pg 30
- 51. “The capability of a superintendent is eminently conspicuous in the selection of his attendants…” The Philosophy of Insanity, 1860 51
- 52. Lessons learned in recruitment ● Phone screen ○ cv, experience & philosophy ○ portfolio ● Face to face ○ pairing ○ review, fix, create, speculate Technical Knowledge Required @eviltester52
- 53. “Automation has turned out to be a much more complex and difficult problem than was originally thought.” John Diebold, Beyond Automation, 1964, pg 51 53
- 54. Lessons Learned: Do Stuff ● Automation is not a thing, it is a process ● Watch your language ● Solve problems quickly and maximize value ● Build abstractions, not tools and frameworks ● Experiment small and fast ● Hire well, based on a portfolio and demo @eviltester54
- 55. Blogs and Websites ● CompendiumDev.co.uk ● SeleniumSimplified.com ● EvilTester.com ● JavaForTesters.com ● Twitter: @eviltester Online Training ● Technical Web Testing 101 ○ Unow.be/at/techwebtest101 ● Selenium 2 WebDriver API ○ Unow.be/at/webdriverapi Books Selenium Simplified Unow.be/rc/selsimp Java For Testers leanpub.com/javaForTesters Alan Richardson uk.linkedin.com/in/eviltester http://compendiumdev.co.uk/contact