Test Automation Frameworks Final

Editor's Notes

• #4: Structured testing experience: Created test plans, strategies, scripts, manual testing, results analysis, etc. Exposure to test tool: Has worked with (or at least seen) one of the contemporary tools and is familiar with their high-level capabilities such as object recognition, programming capabilities, etc.
• #7: Also called – architecture Used the word “code” intentionally. The test engine can be built utilizing one of the commercially available tools, or purchased via one of the
• #8: Or another way of saying “Can’t I just hit the record button?” Reliable (executes to completion with accurate results) Maintainable (test suite maintenance can be performed within available time windows while maintaining positive ROI) Scalable (test coverage can be expanded efficiently – with existing test resources while maintaining positive ROI)
• #10: This is pretty well accepted – whatever your approach, it better be modular and data-driven (which is a general term for test suite execution being governed at some level by an external data source)
• #11: In my experience the leading cause of automation failure is attempting to implement with resources that don’t have the skills and background. What’s more, you end up taking focus away from their strengths – software testing and domain expertise. Separating these two functions allows everyone involved to focus on their strengths If test def & execution are all rolled up into the same code, additional test cases must be added by the automation engineers. Your framework should have a simple test definition interface that testers can use to create their tests and scenarios.
• #12: The primary goal of building an automation architecture, rather than defaulting to record and playback, is to separate the process of test definition and test execution. This eliminates that need for a “super tester” that has both subject matter expertise AND automation expertise. If done correctly, the SME’s can define their test scenarios using a simplified instruction set (stored in an Excel spreadsheet or some other external data source). The test engine, built and maintained by a core group of automation experts, executes the scenarios and reports the results In this example, the test engine consists of reusable scripts and functions that “know how” to perform a common set of test actions such as logging in, navigation, data input, verification, etc. The definition of the tests which contains which actions to perform and what specific data should be used is built in an external data source.
• #20: Consistent means that if I have a “Create Order” business process – each time I execute that process as part of my testing I go through pretty much the same steps each time. Finite means I know how many and what the BP’s are before I start. End-to-end means I’m concerned with the result of the BP, not incremental, lower-level test cases like screen navigation, user interface state, etc.
• #25: Consistent means that if I have a “Create Order” business process – each time I execute that process as part of my testing I go through pretty much the same steps each time. Finite means I know how many and what the BP’s are before I start. End-to-end means I’m concerned with the result of the BP, not incremental, lower-level test cases like screen navigation, user interface state, etc.
• #30: Consistent means that if I have a “Create Order” business process – each time I execute that process as part of my testing I go through pretty much the same steps each time. Finite means I know how many and what the BP’s are before I start. End-to-end means I’m concerned with the result of the BP, not incremental, lower-level test cases like screen navigation, user interface state, etc.
• #33: Many times we see test scripts that were created directly from requirements – any analysis of the test conditions (i.e. things to be tested) is often treated as work product and not kept. However, because we have different types of requirements (e.g. functional, security, user interface, etc.) we have different types of test cases. These varying types of test cases are often jammed together in manual test scripts because we want to spend as little time as possible performing manual tests. <CLICK> Why go through a particular set of screens multiple times, when you can jam all of your testing related to a particular process into just one pass? There’s nothing wrong with that – it’s just efficient manual testing. However, it’s not structured very well for automation. Why? Because as we’ll discuss in more detail later, successful automation requires some type of reusable, data-driven architecture. This manual test script is not reusable for other testing purposes. <CLICK> However, if we extract the test cases from the test scripts and look at them as a whole, we can start to visualize what our automation approach might be. Why, because we start to see types of test cases that are similar and repetitive – a good indication of an automation candidate. Let’s look at this a little deeper on the next slide.
• #34: Many times we see test scripts that were created directly from requirements – any analysis of the test conditions (i.e. things to be tested) is often treated as work product and not kept. However, because we have different types of requirements (e.g. functional, security, user interface, etc.) we have different types of test cases. These varying types of test cases are often jammed together in manual test scripts because we want to spend as little time as possible performing manual tests. <CLICK> Why go through a particular set of screens multiple times, when you can jam all of your testing related to a particular process into just one pass? There’s nothing wrong with that – it’s just efficient manual testing. However, it’s not structured very well for automation. Why? Because as we’ll discuss in more detail later, successful automation requires some type of reusable, data-driven architecture. This manual test script is not reusable for other testing purposes. <CLICK> However, if we extract the test cases from the test scripts and look at them as a whole, we can start to visualize what our automation approach might be. Why, because we start to see types of test cases that are similar and repetitive – a good indication of an automation candidate. Let’s look at this a little deeper on the next slide.
• #35: Many times we see test scripts that were created directly from requirements – any analysis of the test conditions (i.e. things to be tested) is often treated as work product and not kept. However, because we have different types of requirements (e.g. functional, security, user interface, etc.) we have different types of test cases. These varying types of test cases are often jammed together in manual test scripts because we want to spend as little time as possible performing manual tests. <CLICK> Why go through a particular set of screens multiple times, when you can jam all of your testing related to a particular process into just one pass? There’s nothing wrong with that – it’s just efficient manual testing. However, it’s not structured very well for automation. Why? Because as we’ll discuss in more detail later, successful automation requires some type of reusable, data-driven architecture. This manual test script is not reusable for other testing purposes. <CLICK> However, if we extract the test cases from the test scripts and look at them as a whole, we can start to visualize what our automation approach might be. Why, because we start to see types of test cases that are similar and repetitive – a good indication of an automation candidate. Let’s look at this a little deeper on the next slide.
• #36: To highlight the importance of the capturing and tracking test cases separately I want to take a look at a typical manual test script. As I’ve highlighted, we often see that manual test scripts often contain many types of test cases. CLICK If we look at the manual steps we see that we have some business process level test case, user interface, input validation, etc. This doesn’t seem too bad from automation perspective, but we’re not looking at the big picture. CLICK If we take into consideration that we have dozens, hundreds, or even thousands of manual scripts – we quickly see that we’re going to have mess in terms of understanding how to approach automation. So how do we begin to untangle this mess? As we saw in the last slide, we need to extract, or “distill” our test cases into similar levels and types. CLICK Once we have grouped our test cases into similar levels and types, we can start to envision what automation approaches (i.e. architectures) we might want to use. CLICK As the slide indicates, you likely won’t use the same automation approach for all types of testing – most successful automation functions that we have seen have specific architectures adapted to specific testing needs. We’ll discuss automation architectures in a little more detail later in the presentation
• #44: Actual implementation specifics will depend on the capabilities of your test tool