![Code changes: just-in-time resourcing TestCase>>runCase self resources do: [:each | self assert: each isAvailable description: 'Unavailable resource ', each name, ' requested by test ', self printString]. [self setUp. self performTest] sunitEnsure: [self tearDown]. TestSuite>>run | result | result := TestResult new. self resources do: [ :each | each isAvailable ifFalse: [^each signalInitializationError]]. [self run: result] sunitEnsure: [self resources reverseDo : [:each | each reset]]. ^result](https://image.slidesharecdn.com/justintimeresourcingross-090902112424-phpapp02/85/Just-in-Time-Resourcing-7-320.jpg)
![Code changes: 3-valued logic for ‘current’ TestResource class>>isAvailable current isNil ifTrue: [ self makeAvailable ]. ^ self isAlreadyAvailable TestResource class>>makeAvailable | candidate | current := false. "any object not nil and not an instance of me would do" self resources do: [:each | self assert: each isAvailable description: 'Unavailable resource ', each name, ' requested by resource ', self name]. candidate := self new. candidate isAvailable ifTrue: [current := candidate]. TestResource class>>isAlreadyAvailable ^current class == self](https://image.slidesharecdn.com/justintimeresourcingross-090902112424-phpapp02/85/Just-in-Time-Resourcing-8-320.jpg)
Just in Time Resourcing
- 1. ESUG – 1sep2009 - 0 Just-in-time Resourcing: fast tests in SUnit 3.2 and friends Niall Ross, Cincom 1 September 2009 Version 1.0 SUnit
- 2. SUnit Once upon a time there were three classes ... TestCase TestSuite TestResult ... and then there was a fourth TestResource This talk is about: (mainly) TestResources in SUnit 3.2 (briefly) SUnit status
- 3. TestResource is an optimisation Kent Beck’s rules of optimisation Rule 1: do it later Rule 2: see rule 1 eXtreme Programming practices Test-driven development “ But my tests are too slow.” Refactoring “ So refactor your tests to be fast.” enter TestResource and (somewhat later) explanations of it Make it run Make it right Make it fast
- 4. Problems with TestResource XP style : “Do it later” / “You won’t need it”: Every resource set up before any test is run If one resource of one test in a suite of 15,000+ fails … … the run does nothing – not what you want to see when next you look XP Style : refactoring + “last make it fast” MyTestCase>>setUp … self assert: databaseSession isOnline description: ‘not online’. Tests getting slow? Refactor to a TestResource. MyTestResource(Object)>>doesNotUnderstand: #assert:description:
- 5. Problems with TestResources Resources can compete with other resources: e.g. connect to one DB at a time, several DBs to test I coded the CompetingResource pattern: in SUnit 3.1 and earlier, not easy ! Stephane D and Martin K also had patterns – also not easy Resources can rely on other resources: Tests (and resources) can have ordered resources MyTestCase class>>resources ^Array wth: ConnectToDBResource with: AddTestDataToDBResource resource setUp (tearDown) not in order (reverse order) resource setUp / tearDown after resource that needs it
- 6. What has changed in TestResource Resources are made available just-in-time: first test that needs it prompts set up later tests that need it see it has (or failed to) set up tearDown guaranteed at end of run; can be done anytime resetting in a test’s tearDown trades performance for test isolation Resources understand #assert:… protocol setUp and isAvailable run inside the handler in end-run tearDown, #assert: is just better protocol for same behaviour Resource-processing is ordered a test’s resources setUp in order and tearDown in reverse order a resource’s resources setUp before it and tearDown after it
- 7. Code changes: just-in-time resourcing TestCase>>runCase self resources do: [:each | self assert: each isAvailable description: 'Unavailable resource ', each name, ' requested by test ', self printString]. [self setUp. self performTest] sunitEnsure: [self tearDown]. TestSuite>>run | result | result := TestResult new. self resources do: [ :each | each isAvailable ifFalse: [^each signalInitializationError]]. [self run: result] sunitEnsure: [self resources reverseDo : [:each | each reset]]. ^result
- 8. Code changes: 3-valued logic for ‘current’ TestResource class>>isAvailable current isNil ifTrue: [ self makeAvailable ]. ^ self isAlreadyAvailable TestResource class>>makeAvailable | candidate | current := false. "any object not nil and not an instance of me would do" self resources do: [:each | self assert: each isAvailable description: 'Unavailable resource ', each name, ' requested by resource ', self name]. candidate := self new. candidate isAvailable ifTrue: [current := candidate]. TestResource class>>isAlreadyAvailable ^current class == self
- 9. Class changes Once upon a time there were three classes ... TestCase, TestSuite and TestResult ... and then there was a fourth ... TestResource ... and now a fifth ... TestAsserter : abstract superclass of TestCase TestResource any user-created TestCase delegate class ( ... and that’s enough ! )
- 10. Any impact on Users ? Logging TestCase methods moved to the class-side #isLogging, and #failureLog (and #logFailure: is on both sides) (So, who here overrides #isLogging or #failureLog ?) Profiling a test… method’s time: no impact a test suite’s overall time: no impact a test’s time in #runCase: sometimes see resource time time moved from start of suite’s #run to start of (some) tests’ #runCase Any objections, voice them now !
- 11. SUnit 3.2 Make your tests run Make your tests right Make your tests fast (resources can help) Thanks to Yuri Mironenko, Dale Henrichs, James Foster, Tim MacKinnon for helping me port to Squeak, Gemstone and Dolphin.
- 12. SUnit and Friends SUnit: cross-dialect, backward-compatible, 3-5 classes Add-ons: SUnitXProcPatterns, SUnitResourcePatterns, etc. UIs: RBSUnitExtensions SUnitBrowser, Squeak TestRunner, etc. SUnitToo: VW-specific, experimental, 11 classes SUnit-Bridge2SU2 maps SUnit tests to SUnitToo tests Assessments: VW-specific, configurable, 40+ classes transparent bridges configurable for SUnit, SUnitToo, etc. GemStone’s test framework … SUnit wants ideas SUnit will remain cross-dialect, backward-compatible, small
Editor's Notes
- #2: 0 "Just-in-time resourcing: fast, flexible testing with SUnit and friends", Niall Ross, Cincom In the beginning was basic SUnit: just 3 classes. Then a fourth, TestResource, was added, for use when your tests became slow. However TestResource could create problems of its own and how to use it was not always clear. Switching code from test to resource was not as easy as it could have been. Niall will present the robust SUnit 3.2 implementation of resources, with examples. The talk will also cover some other SUnit topics - Basic SUnit did not handle tests that spawned subthreads. Niall will briefly reprise his StS talk on the SUnit add-on that handles this - SUnit tools and variants have appeared: Niall will look at some VW and VA tools, and how to combine using them with using basic SUnit.
- #3: Once upon a time there was Ward Cunningham. When he pair-programmed with people, they were much more productive. Then there was Kent Beck – who wondered why this was. Then there were three classes – and a papar – and a name: Sunit And then the millenium came and suddenly there were four classes. Actually, it wasn’t very sudden.
- #4: TestResource exists because test-driven development is only fast if your tests are fast. TestResource was deployed – and (rather later) explanations of how to use it were deployed. It was never a tricky pattern - it was in many ways a straightforward implementation of the singleton pattern, but it was just complex enough that people needed hints. Whereas some such implementations use a #defaultInstance or #current or whatever accessor and have users manipulate the singleton instance, TestResource>>current was mostly just expected to be called by the framework, while users simply referenced their TestResource subclasses, not those subclasses’ instances. This made a fair amount of sense – the intent I believe was to give users an easy and at the same time very safe (i.e. Minimal) interface – but the original Sunit Camp Smalltalk progenitors Jeff O’Dell and Joseph Pelrine and Sames were not in perfect unison on the point. In my refactoring of TestResource, I have been greatly helped by the fact that a many people have written ‘bad’ TestResource code – code that violates the pattern, that does things in bizarre and inefficient ways, etc., etc. When refactoring tests in a hurry, it was easy to guess wrong.
- #5: One resource fails => nothing in suite runs. “ Do it later / You won’t need it” is an XP maxim, but every resource that any test in a suite needs is set up on starting, before any test is run. That might just be a point of style but the following is really annoying: - You finish work for the day. Last thing, you hit the overall run of 15,000 tests or whatever and go home. TestResources are by definition slow so their set up has not finished when you leave. Next morning, you gather that one resource failed and nothing was done. :-/ Make it fast last => refactor to resource but resource DNU self assert Competing resources: my system can connect to one database at a time, my tests run against specific databases, my overall suite wants to run all my tests, my database connections are test resources.
- #6: Competing resources: my system can connect to one database at a time, my tests run against specific databases, my overall suite wants to run all my tests, my database connections are test resources.
- #7: The original status was that a resource was set up once for the whole run, the best case scenario for performance but the worst case scenario for possible interaction between tests. In Sunit 3.2, this remains the default choice. If the coder makes their test reset a resource in some circumstance, they are choosing to sacrifice some performance in exchange for isolating other tests from theirs.
- #8: The code that setUp all resources before the run starts, failing (and leaving the earlier ones setUp) if any could not be, is dropped . Instead, we add code so that each test asks whether its resources are available; as we shall see, this check is cost-free unless it is the first time of asking. As you can see from the deleted code, it used to be that only if all TestResources were set up would any of them be torn down. In 3.2, a resource that completes its own setUp and is available will be torn down even if another resource fails. This is still les rigorous than for tests: if a test enters setUp it will be torn down. Should we do the same for resources?
- #9: The three-valued logic is that current is - nil : no setUp attempt has yet been made in this run - false : a setUp attempt was made but failed - an instance of the resource : a setUp attempt was made and succeeded
- #10: Once upon a time there was Ward Cunningham. When he pair-programmed with people, they were much more productive. Then there was Kent Beck – who wondered why this was. Then there were three classes – and a papar – and a name: Sunit And then the millenium came and suddenly there were four classes. Actually, it wasn’t very sudden.
- #11: The original status was that a resource was set up once for the whole run, the best case scenario for performance but the worst case scenario for possible interaction between tests. In Sunit 3.2, this remains the default choice. If the coder makes their test reset a resource in some circumstance, they are choosing to sacrifice some performance in exchange for isolating other tests from theirs.
- #12: 6 Other changes: - fixes to ensure running single tests and debugging also calls resources. Put should:/shouldnt: into #deprecated protocol TestResource>>uninitialize / tearDown partners RestResource>>initialize / setUp More details package, class and method comments Things we did not change If a test enters setUp it wll be torn down. If a resource exits setUp and isAvailable, it will be torn down. Usd to be, if a resource failed, NIALL: MAKE CHANGE: - sunitVersion must return 3.2 not 3.1.1
- #13: The original status was that a resource was set up once for the whole run, the best case scenario for performance but the worst case scenario for possible interaction between tests. In Sunit 3.2, this remains the default choice. If the coder makes their test reset a resource in some circumstance, they are choosing to sacrifice some performance in exchange for isolating other tests from theirs.