Karishma Kolli – Myth Busters on Test Automation

Editor's Notes

• #4: If we have 100% unit test coverage we don’t need any other tests. As long as our REST services pass we are good. UI testing is really easy! Load testing can be performed with any tools. Performance testing comes as package with load testing.
• #5: Every organization has different needs and there are ton’s of tools that can help you achieve.
• #6: Based on your application look and feel may be the most important thing for you but again – like some website for senior citizens. There are good tools out there like wave or puppeteer 508 compliance is important but again decide if it should be part of your functional testing. Don’t overload your suites with excess code and make it difficult to maintain. UI tests should be picked carefully as they are more expensive, the more UI tests you add the more time you add to build check
• #7: Don’t increase your run time by waiting on exceptional test cases. This scenario is very common with UAT (User acceptance testing) your test will fail if the browser fails to load the page after a certain set time. You cannot include insane amount of time before your tests timeout because this undermines the base rule of automation you cannot wait on one test case forever
• #8: Yes, choosing the right tools and having the right skills are first step towards automation. There are other steps that you should keep in mind to ripe the fruits of your automation effort. Pipeline – It is always better if you could make your tests part of pipeline and have them act as gates. Test environments – If it does not replicate your production env that you won’t be able to achieve your goal. Testing platform – Running on your on laptop could work in the beginning but as your test suite grows it becomes difficult it also depends on how you handle synchronization.
• #9: This is very effective in functional testing on UI - you still check the core functionality trough UI but all the sister steps helping achieve this are taken care by your API’s hence saving you valuable time and also makes your tests less flaky. We can have a condition to switch to UI if the API response is unsuccessful but again we need to have a manageable code. You won’t ship the product with failed API so fix it and then consume for testing.
• #10: Speed is definitely one of the key features of automation but a good automation candidate (test) should be a repetitive test. Manual QA may be faster but humans are error prone. Manual QA gets tired and bored of repetitive work. A test that gets tested only once is not an idle choice for automation. Its not a competition but being smart and using machines to do our boring work while we concentrate on creativity.
• #11: Manually running an automation suite on every build is not just clicking a button. If your suite is part of pipeline your life is easier. Multiple servers, your engineer should change the config based on requested server We often tend to forget tiny details associated with most of our tasks. You can automate all of these if you are sure that your environments are always stable and you would want testing on the same server, OS, browser, locale etc but that undermines the purpose of automation Someone has to manually, go trough the results.
• #12: SDET’s can be more closely related to developers than Testers.(of course they need the mind set of both) If you want to use existing resources and have an automation suite for testing your best bet will be a developer. If someone who is not familiar with code wants to be an SDET learning the language is important not just the testing framework used at organization. Adding test scripts to existing framework is different from actually working on the framework(fix something when it breaks, enhance reporting, adding more capabilities etc)
• #13: If you have a good test automation framework then it is mostly built in house In-grown test frameworks use a strong coding languages and a bunch of testing tools and plugins these have to be updated and maintained just as your actual code. You can use easy to use and maintain automation tools such as UFT(even this is not record and play back) if your automation needs are not crazy and you are willing to pay. Test code is no different than actual application code. it needs the same amount of attention and care.
• #14: Automation testing is typically used to build trust. that your application is not broken. Unit tests should be hand in hand with your application but system tests and UI tests have to be built on the feature so manual testing will be done by the time automation suite is ready(even for in-sprint automation test cases).
• #15: Even if your test frame work is not data driven, you can still efficiently use data to make the best use of your test cases. Once you have the automation suite it is very common to get excited to test every scenario with every possible data but that’s not the best idea. Choose your test data wisely, make sure you pick data that has various factors that you would like to test If you can be confident on your application only if you test every single value of data then your application is in serious trouble. Eg: Test that when vegetable is received from DB a sad face is shown next to it. And a smiley when fruit is received. You don’t need to test it for every vegetable or fruit but choose basic veg\fruit Check non-native veg\fruit Check tamato.
• #18: UI tests are everybody’s favorite but they are also the most expensive tests.
• #19: Automation testing is checking. It checks if the coded conditions are met. Automation may help reduce the number of manual QA. needed and provide more coverage but will not eliminate manual QA. There will always be scenarios which are not straight forward, which needs human judgement. Your code doesn’t get creative unless someone works on it and changes the logic but humans do it everyday.