Test data automation: delivering quality data at speed

Editor's Notes

• #7: 3
• #11: .
• #13: This is the broad process we are next going to break down step by step. It is a largely automated, end-to-end process for moving directly from requirements and change requests to automated tests, data, test cases, and clear specifications for development. All of these QA assets are built directly from the system designs, and therefore are maintained automatically as the requirements change. First, the latest requirements must be converted into a format that is not only clear and complete, but can be used further “right” to drive automated testing directly. This is possible with formal modelling, that unambiguously breaks a system down into its core logic. However, this modelling must be quick enough to occur within sprint, and importers and accelerators are therefore imperative to leverage existing design and test assets. Here we can see a range of accelerators and recorders: a UI recorder or scanner; a BDD importer; test case importers; business requirements and BPMN diagram importers. Once the designs have been converted rapidly to mathematically precise, unambiguous models, they can be provisioned to developers. The optimized assets needed to verify the code produced by development can be generated simultaneouslyfrom the models. QA can overlay additional test logic and data to the formal models, generating the test cases, data, and automated tests needed to validate that the requirements have been fulfilled in the code. Testing in this approach becomes a largely automated comparison of the business requirements to the developed code. Next, the tests need to be executed automatically across the multi-tier architecture common to modern systems. Testing must go beyond the UI, including back-end databases and the service and message layers. It must also span across all platforms users can reasonably be expected to use, for instance multiple browsers as well as on mobile. The reporting and operations tasks associated with test execution should also be automated, in addition to the execution itself. Testers should not be laboriously comparing actual to expected results, and nor do they have time in short iterations to update the numerous platforms found across a DevOps pipeline. Fortunately these rule-based tasks are ripe for automation, and process automation can be used to update JIRA and ALM tools, as well as communication across teams in Git, Trello, Slack, or whatever technologies they are using. Finally, when a change request does come in, it needs to be fed back into the same model from which test assets have been generated. That way, the rigorous tests can be continuously and automatically kept up-to-date. Impact analysis, meanwhile, notifies developers to the affect a change has had on the system, and where. They can then work to mitigate the risk of integration errors. We’ll now break this down step-by-step.
• #14: Include in the demo: Synthetic + Production -> A hybrid approach. Test data allocation – reservation/blocking data, self-service, on demand. Each test case needs multiple varieties of data. Data flow modelling Huw will use the HR Demo -Mask -Generate/augment -Model