Automation Responsibility
0 likes•773 views
The document discusses the complexities of test automation within software projects, emphasizing the need for defined goals and responsibilities. It contrasts internal and external approaches to automation, highlighting issues like resource allocation and the necessity of quick updates to maintain test effectiveness. Various strategies for managing automated tests are presented, focusing on the involvement of testers versus developers and the importance of training and budget considerations.
Automation Responsibility
- 1. 14 June 2011 The Magazine for Professional Testers printed in Germany print version 8,00 € free digital version www.testingexperience.com Improving the Test Process ISSN 1866-5705 © iStockphoto.com/ jgroup
- 2. © iStockphoto.com/carlosphotos Automation responsibility by Zbyszek Moćkun We are in the project meeting and discuss how to speed up the On the other of the hand, we have maintenance, which contains time of software releasing. Suddenly someone from the end of activities like: run tests, report results, debug failed tests and up- the room shouted: “Let’s automate our regression tests”. The idea date automated scriptsif there were changes in the application. looks very interesting, but during further discussion about auto- There is no way to do this outside the project, so maintenance can mation we found several questions that need to be answered. Be- be considered as a type of sub-project (internal project) which fore we start to work on automation we definitely have to answer should be conducted within the budget and resources of the pro- one of these questions: Who should be responsible for automa- ject. It will be very difficult to make quick changes to scripts (con- tion and why? nected with new features/change requests to the application) as an external project. So the changes should be done as soon as The first step is the same as for any project - at the beginning one they were identified by one of the project team members. should define a goal. We need to know what we want to achieve. Do we want to automate only smoke tests (few tests that verify if Let’s summarize: Writing automated scripts is an external pro- everything is ok), check all important features (about 20% of cov- ject, and maintenance is internal. erage), or do we need to automate almost all of the functionality of our application (about 80%). Please note that we can automate It’s time to go back to the second part of the question - who should even 100%, but the costs of automation for the last 20% will be write the automated tests scripts, and who should be responsible very high and in most of cases it’s not worth to do this (manual for the maintenance? Should it be done by developers or testers? tests are cheaper). Below I will concentrate mostly on the most Does anyone know developers that like testing? I don’t think. It’s popular and the best approachinvolving 20 to 80 percent of au- only the one reason but really important (if people do not do what tomated tests. they like, they just change a work or their performance slowly decrease). Another reason against developers is that they would We must not forget about two resources that are always limited: have to learn about the test case management tools, where we time and budget. If we sum up those (refer to the goal from the keep test cases, learn about the environment and about the tool paragraph above) we realize something very important – auto- that we will be using for test automation. Learning new tools is mation is a project. It’s unique, and we move within a triangle a time consuming process and we have to be sure that our invest (scope, budget and time). returned in long term. Developers should rather concentrate on unit tests, which is their level. So, why should automated tests be Another question is - What kind of project is it? Is it external or written by testers? internal in accordance with the application (main project) that we want to test? Before I answer this question, let’s go back to au- Testers are specialized in application testing; they are trained tomation. The task of automating can be divided into two phases: and should have a sixth sense. They know the testing technics phase 1 - create new test scripts, and phase 2- maintenance (run and tools, so there is no need for additional training. In addition, and keep up to date). The question is - should we treat both of when they write the automated tests, they gather knowledge, them in the same way? not only about the automation tool but also about the tested application. Thanks to this they can help us if there will be more Of course not - Writing new test scripts can be outsourced to an work in the project (e.g., complicated features, need of manual external project. The author (person who writes automated test regression, a lot of maintenance work in automated tests) or as scripts) does not need to be involved in the application project a backup to testers to minimize the project risk. Learning auto- since he rewrites regression tests. If manual test scripts are well mation tools is not accomplished in just few days, sometimes it made, the author does not even need to know anything about the can take weeks. It’s crucial not to lose that knowledge. Developers application. He knows the steps and the expected results, which go to other projects, where automation skills may not be desired, should be enough to created automated test scripts. but testers’ knowledge will be needed again and again for writing 112 The Magazine for Professional Testers www.testingexperience.com
- 3. tests but also for maintenance. DEVELOPERS DEVELOPERS INTERNAL EXTERNAL If we summarize the above, we see that there are four different ways to write and maintain automated tests: 1. Developer Internal - This is a very interesting way, which I tried to introduce in one of my projects. The idea was that each developer delivers newly implemented features to- MA TESTERS TESTERS INTERNAL EXTERNAL gether with the automated scripts. Unfortunately, it failed IN CR for several reasons. We had to invest a lot of time in automa- TA EA N AN TE tion tool training (the cost was high as we had to train all developers in the team). Another issue that we faced is the CE decision on who would write the test scenarios that were to be automated. Developers, testers? In the case of developers, Figure: Automation responsibility square we would need to train them in how to write test scenarios and how to use the test case management tool. If we chose testers, we need to find a way for them to deliver scripts to Maintenance is connected with current application development developers (this idea requires them to create test scenarios where new features have an impact on existing scripts that are at the beginning of a sprint, before requirements go into de- used for regression tests. As testers (internal) know the test suites velopment). Another issue (I think, the most decisiveone for and application best they are the ones that can recognize points the failure) is whether automated tests are required for all that have to be changed and can introduce the changes quickly features. There are always other important things to do, like without the need to send them to an external team. It’s very irri- features that should have been finished yesterday, so writing tating to run test suites that we know have failed, because we are automated tests takes second placeafter a while and auto- waiting for changes to come from an external team. We should mation dies. It is for the same reason that this approach can’t note that failed tests which are waiting for update do not test be used for maintenance. anything, and we should cover this area by manual tests. Please 2. Tester Internal - The idea is that the same team executes note that it is very hard (and needs a lot of time) to find new er- functional, regression and other types of tests and spends rors among several already known. This is why fixing or updating any free time on automation. From our experience, however, test scripts immediately is very important. we know that there is no free time because there is always something to do. In this approach, a lot depends on the test- er’s personality. If he is “lazy”, he will automate the most fre- Employing on two teams (an external team which only writes quently executed tests only. So if we just want to have really new scripts and an internal team to test the application and poor automated smoke test suites, this approach is for you. maintain automated scripts – it can be even the same person For bigger projects it will never finished with success. who has divided task in time) results in a process for creating new ones, which is free of interference. We can achieve continu- As described above, this approach is best suited for mainte- nance of automated tests. Testers run test suites, report and ous gains of automated tests and coverage. If we sum this up, it debug failed test scenarios. As automated tests should be ru- gives us shorter regression time and lower budget for testers in nas often as possible (continuous integration approach) it is the internal project. very important to make changes quickly. Testers in an inter- nal team can do this very quickly because there is no need to An approach like this gives you clear division of responsibility, send anything to an external team, which always takes time. ensures continuous growth of coverage, and achieves the aim of In addition,the external team does not lose time on updat- a fixed budget without any risk to the application development ing existing tests and can instead create new ones for new project. features. 3. Developer External – The idea is that we create an external team composed of developers with a separate budget, with a > biography defined aim and resources. The use of developers takes over Zbyszek Moćkun all the minuses mentioned above for the ‘developer internal’ has more than 6 years of ex- project. The biggest factor is that developers are not for writ- perience in QA engineering for ing test scripts, but for writing the application. big as well as for small com- 4. Tester External - This idea combines all benefits from inter- panies. Currently he works nal tester internal and external developer external. Testers as Head of QA Practice for do what they are specialized in, it’s very easy to manage the Cognifide - digital technology budget, and we have a backup for testers. I have managed agency. several automation projects using this approach, and all of The author has been involved them finished with success. Of course, it is always nice to in several projects of different have a developer (it can be rotated role) assigned to an au- size and methodology where tomation team to resolve complicated programming tasks. he could work on or introduce Everyone knows that 80% of automated scripts can be done automation. His experience by copy & paste using already written code, so there is no was used to optimize the time spent on automation and need for special development skills (I believe that each tester getting maximum benefits. This is very important, espe- should have basic development skills). cially in Agile methodologies, which is the area in which he specializes. Why do we need two different approaches for creating and for maintaining automated test scripts? www.testingexperience.com The Magazine for Professional Testers 113