Automating the testing of your bi solution with n bi
Download as PPTX, PDF1 like1,900 views
The document discusses the importance of automating testing in business intelligence (BI) solutions, highlighting various testing types including queries, ETL packages, and multidimensional models. It emphasizes the benefits of test automation such as increased frequency, consistency, and improved quality of results while introducing tools like NBI and Genbi for facilitating automation. The overall aim is to address the high failure rates of BI projects due to lack of agility and testing capabilities.
Automating the testing of your bi solution with n bi
- 1. Automating the testing of your BI solutions with NBi CĂ©dric L. Charlier
- 2. Agenda âą Why automating is a must? âą Tests with queries (Sql/Mdx/Dax) âą Tests with ETL packages (SSIS) âą Testing multidimensional and tabular models (SSAS) âą Automating the creation of test-suites
- 3. Whatâs automated testing? A piece of software written to reproduce steps/actions defined in a manual process
- 4. Benefits of automation Allow testing to happen more frequently Speed up testing to accelerate releases Improve test coverage Ensure consistency
- 5. Tests automation doesnât apply to Business Intelligence
- 6. Wednesday, November 24, 2012Est. 1869 Price 6d More than 33% of BI projects fail to deliver Lack of agility. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoqueCum sociis natoqueCum sociis natoque Low quality of deliverables Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturientAenean commodo ligula eget dolor. Aenean massa. Cum sociis natoqueCum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, Simply Too late. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, Member of the Asscoiated Press . Aenean commodo ligula eget dolor. Aenean. Aenean commodo ligula eget dolor. Aenhswse. Cejhciebce fcdcdcd. ILLUSTRATED WEEKLY NEWSPAPER
- 7. We havenât time to create these tests
- 8. Time to execute a test-suite 1.0 1.1 1.2 1.3 1.4 1.5 Elapsedtime Release Manual Automate Exponential slope Why? - New objects - Old objects
- 9. Time to write a test-suite 1.0 1.1 1.2 1.3 1.4 1.5 Elapsedtime Release Manual Automated With Framework Initial gap
- 10. No existing framework for test automation dedicated to BI ⊠Iâm not proficient enough in C#
- 11. This framework exists: NBi No C#, just xml syntax Open- source
- 12. Agenda âą Why automating is a must? âą Tests with queries (Sql/Mdx/Dax) âą Tests with ETL packages (SSIS) âą Testing multidimensional and tabular models (SSAS) âą Automate the creation of test-suites
- 13. Compare queries ⊠result-sets System-under-test Assert Previous release Two different releases of a same data warehouse Tabular model versus Csv fileDAX ⊠or any brilliant idea Cube and data warehouseMDX SQL
- 14. Demo: compare queriesâ result âą Compare cube/operational database âą 101 âą Compare cube/flat files âą Defaults and References âą Tolerance âą Intervals
- 15. Other testing goals for queries âą Performance of a query âą Executed in less than x seconds âą Cache management âą Syntax of a query âą Extract queries from reports and assemblies
- 16. Queryâs result and format âą Resultâs format âą Sample: all the cells have effectively 2 digits after the decimal separator and this separator is a comma.
- 17. Check calculations in your query âą Assert a logical test, row by row, for you result-set âą Sample: Calculate the total price from the unit price, quantity and discount percentage. âą If at least one row doesnât validate this logical test, the whole test shift to red
- 18. Agenda âą Why testing automation is a must? âą Tests with queries (Sql/Mdx/Dax) âą Tests with Etl packages (SSIS) âą Testing multidimensional and tabular models (SSAS) âą Automate the creation of test-suites
- 19. Demo SSIS âą Assert the execution status of an ETL âą Alternatively test performances
- 20. Inside this ETL Dimension City Dimension Scientist Factless fact Birth/Death insert insertUnique index violation
- 21. Setup and Cleanup âą Setup: âą Define the state of your testing environment before executing a test âą Cleanup: âą Remove consequences of your tests on the testing environment
- 22. Tasks and behaviors âą Available tasks âą Truncate/Load tables with csv files âą Run sql batch, exe or batches âą Run Etl packages âą Move or delete files âą Start/Stop Windows services âą Run-once: execute only once this set of tasks âą Parallel: run a set of tasks in parallel
- 23. Demo Setup âą Setup to register a clean state before testâs execution âą Assert databaseâs state after the execution of an ETL
- 24. Agenda âą Why automation is a must? âą Testing queries (Sql/Mdx/Dax) âą Testing Etl packages (SSIS) âą Testing multidimensional and tabular models (SSAS) âą Automate the creation of test-suites
- 25. Testing dimensionâs members âą âGermanyâ is a member of dimension âCountryâ âą âEuropaâ is not a member of dimension âCountryâ âą Dimension âCountryâ has more than 190 members. âą All members of dimension âCountryâ have at least 4 characters âą Members of hierarchy âMonthâ are ordered chronologically (and not alphabetically) âą âŠ
- 26. Testing structure ⹠Assert that a dimension, hierarchy, measure, ⊠is visible for end- user in a given perspective. ⹠Assert the existence of a relation between a measure-group and a dimension.
- 27. Agenda âą Why automation is a must? âą Tests with queries (Sql/Mdx/Dax) âą Tests with Etl packages (SSIS) âą Testing Multidimensional and tabular models (SSAS) âą Automating the creation of test-suites
- 28. Some tests are really repetitive It should be possible to automate the creation
- 29. It exists! Take a look at Genbi A template for the tests A set of test- cases
- 30. First steps with Genbi File loaded from Genbi Variables to be substituted (surrounded by $)
- 31. Demo: Genbi âą Load test-cases âą From a flat file âą Select a template âą Generate the test-suite âą Grouping option
- 32. Iâm scripting macros to automate the actions performed by genbi ⊠With genbiL!
- 33. Sample for genbiL Subject Action Parameters
- 34. Demo for genbiL âą Command line
- 35. Conclusions âą Benefits of test automation âą Save Time (Time-on-market) âą Better Quality (end-result) âą More confidence âą NBi âą Support large range of different tests âą No need of compiler (neither C#) âą Good feedback when tests are failing âą Open-Source âą Automate the creation of your test-suites (genbi / genbiL)
- 36. Read further ⹠Website: http://nbi.codeplex.com ⹠Download, documentation, bugs/Wishes list, support ⹠Blog: http://seddryck.wordpress.com ⹠Twitter: @Seddryck Partnership TestingSquad Rue des Déportés, 47 1200 Bruxelles phone : +32 470/84.07.47 email : [email protected] Web : www.testingsquad.eu