Atdd half day_new_1_up
1 like•1,163 views
1. Acceptance Test Driven Development (ATDD) involves writing tests before code to validate requirements and prevent defects. The tests are written by a triad of customer, developer, and tester. 2. ATDD focuses on testing behavior and outcomes rather than internal code or data structures. Tests validate that the system meets requirements from an external perspective. 3. Acceptance tests follow a Given/When/Then structure where the Given sets up initial conditions, When performs an action, and Then verifies expected outcomes. ATDD shifts testing left in the development cycle.
Atdd half day_new_1_up
- 1. ATDD Acceptance Test Driven Development J a r e d R i c h a r d s o n A g i l e A r t i s a n s @ J a r e d R i c h a r d s o n 1
- 3. Jared Richardson CoAuthored GROWS Methodology Author Ship It! and Career 2.0 Screencast editor PragProg.com 2nd public signatory of the Agile Manifesto Started AgileRTP in 2007! Coach AgileArtisans.com 3
- 5. What would you say you do here? Roles? Responsibilities? Why are you here? What are your tests for this session? 5
- 6. Objectives Define ATDD Why requirements are not enough Who to involve when Turning requirements into tests What to test How ATDD can make your entire company more productive 6
- 7. Outline Terms and definitions Requirements and tests Software development ATDD examples Anatomy of a test Tables (Grid Driven Development) 7
- 8. Take Aways What’s a test? Who writes them? When are they written? What do they verify? Provide an example Primary benefit? Secondary benefits? 8
- 10. Background Lean-Agile Acceptance Test Driven Development by Ken Pugh 10
- 11. One Rule There are exceptions to every statement Except this one 11
- 14. Key Points Skills don’t map dilbert.com/strip/1990-09-13 14
- 15. Key Points Beginners need steps 15
- 16. What is ATDD? Acceptance Test Driven Development External view of the system Examines visible effects Inputs Outputs State changes External interfaces 16
- 17. Definitions Acceptance criteria -> General idea Acceptance test -> specific. Pass/fail. Implementation independent Triad -> Customer, developer, tester 17
- 18. Fast Car Who wants one? Criteria Closed course. Measure acceleration. Test Accelerate 0 to 60 in X seconds flickr.com/photos/vodcars/4132650949 18
- 19. Fast Car flickr.com/photos/fordeu/14296246942 0 to 60 in 30 seconds 0 to 60 in 5 seconds. Top speed is 61. 0 to 60 in 5. Top speed is 120. 60 to 120 in 5 min 0 to 60 in 5. 60 to 120 in 20 sec. Range of 1/4 mile. 19
- 20. 0 to 60 in 30 seconds 0 to 60 in 5 seconds. Top speed is 61. 0 to 60 in 5. Top speed is 120. 60 to 120 in 5 min 0 to 60 in 5. 60 to 120 in 20 sec. Range of 1/4 mile. Range of 1,000 miles. Cost 7 billion Fast Car flickr.com/photos/alpi-costerni/8161111001 20
- 21. Why ATDD? Rework down from 60% to 20% Workflows work 1st time Little room miscommunication Saves time Getting business rules RIGHT Tighter cross functional team integration Crisp visible story completion criteria Automation yields reduced testing time 21
- 23. Requirement Needed by a user to solve a problem Needed by a system to satisfy contract, standard, spec, regulation... 23
- 24. Requirements Include Functions (what software does) Constraints (limits) 24
- 25. Requirements Should not Include Implementation details Specific manifestation 25
- 26. A Brownie Spec Ingredient Parts by weight Sugar 23.0 Flour 21.0 Shortening 16.8 Nuts 16.0 whole eggs 13.0 Cocoa 5.5 Dextrose, anhydrous 4.4 Salt 26
- 27. Brownie Spec a. Whip eggs in large bowl on high speed until light and fluffy. b. Combine sugars, cocoa, salt, and leavening; add to beaten eggs, and whip on high speed until thick. c. Add shortening slowly while mixing on low speed. d. Scrape bowl and whip on high speed until thick. e. Mix flour, nuts, and flavors together and fold into batter; mix until uniform. f. Pour batter into pan at a rate that will yield uncoated brownies which, when cut such as to meet the dimension requirements specified in 3.4f, will weigh approximately 35 grams each. g. Bake at 3500F until done (45 minutes). h. Cut into 2 inch squares 27
- 28. Requirements and tests Inter-related Can't have one w/o the other Failing test is a requirement Passing test valid system spec 28
- 29. ATDD as a Prism Requirements Tests ATDD 29
- 31. Exercise What Issues? Current software dev process Write down your org’s issues Groups of 3 to 8 Pass to front 31
- 33. Customer Unit Product owner Business analysts Users QA 33
- 36. Traditional Value Stream Elicit& Requirements& Analyze& Requirements& Design&& Code& Test& Why go back? Deploy& 36
- 37. Requirements Write Code The Traditional Approach Demo Rewrite “Discussion” Typically a confrontation & argument The demo forces a discussion of the requirements An isolated event, primarily worked by the customer proxy 37
- 38. Demo Write Code Requirements The Agile Approach Rewrite Before significant time and $$ is invested Discuss Include the Golden Triad Dev QA Customer 38
- 40. Mistakes Feedback corrects misunderstandings Quick feedback is better than slow feedback 40
- 41. Desired Actual Slow Feedback Actual Fast Feedback 41
- 44. Two Types of Testing To find defects To prevent defects 44
- 46. preventing Defects Saves development time Saves QA time Saves money 46
- 47. Steps Author tests (write) customer, tester, developer together Connect to the system (automate) developer Run the tests (execute) developer, testers, customers, CI system Test% write% bind% run% 47
- 49. Test First Write the test first Write the code after One molds the other First in wins 49
- 50. Testing Strategies Meszaros, XUNIT TEST PATTERNS: REFACTORING TEST CODE, Fig 6.1 "Purpose of Tests" p. 51, © 2007 Pearson Education, Inc 50
- 51. Testing Focus 51
- 52. Verification vs Validation Verification Are the requirements right Focus of acceptance tests Validation Have we got the right requirements? Customer really do it this way? How do you test this? 52
- 55. Tests all the way down Requirement* Requirement** Test* Test* Requirement* Test* Test* Service* Implementa4on* Test* Customer Acceptance Tests Developer Acceptance Tests Service* Implementa4on* Requires services to implement 55
- 57. Temperature Input in Celsius Output in Fahrenheit How long to code? 57
- 58. Temperature F = C x 9/5 + 32 58
- 60. Temperature 60
- 61. Test Binning Celsius' Fahrenheit' Notes' 0" 32" 100" 212" Needed?" Celsius' Fahrenheit' Notes' *273.15" *459.67" Precision" Formula Tests Precision Tests Celsius' Fahrenheit' Notes' *273.15" *459.67" 0"Kelvin"" *273.151" Error" Below"0"Kelvin"" 500" 932" Maximum"–"Needed?" Limit Tests 61
- 62. Temperature Input in Celsius Output in Fahrenheit How long to code? 62
- 63. Is This A Good Spec? 63
- 64. Business Rules If a Customer Rating is Good and the Order Total is less than or equal $10.00, Then give no discount Otherwise give a 1% discount If Customer Rating is Excellent, Then give a discount of 1% for any order If the Order Total is greater than $50.00, Then give a discount of 5% 64
- 65. Rule Table => Test ? ? 65
- 66. Another Business Rule Email coupon codes If a customer enters the code, provide a 20% discount Discount an item or the order? 66
- 67. Implementation Testing script Program interface Xunit framework ATDD framework 67
- 68. Testing Script Tools for testers (Usually) GUI based 68
- 69. Testing Script Log in as a Customer who is rated Good Start an Order Add items totally $10.01 Complete order Verify $0.10 discount Repeat for 10 other similar cases 69
- 70. Program Interface Code something (CLI or GUI) Screen Discount Method Discount Percentage Screen Customer Type: Good Order Total: 10.01 Percentage: 1 % C:>DiscountPercentage Good 10.01 Percentage: 1% 70
- 71. Xunit Test class TestCase { testDiscountPercentageForCustomer() { SomeClass o = new SomeClass() assertEquals(0, o.computeDiscount(10.0, Good)); assertEquals(1, o.computeDiscount(10.01, Good)); assertEquals(1, o.computeDiscount(50.01, Good)); assertEquals(1, o.computeDiscount(.01, Excellent)); assertEquals(1, o.computeDiscount(50.0, Excellent)); assertEquals(5, o.computeDiscount(50.01, Excellent)); } } 71
- 72. Fit Test 72
- 73. Tables as Requirements and Tests Discount Rule Customer Rating Order Total Discount Percentage Good <= $10.00 0% Otherwise 1% Excellent Any 1% > $50.00 5% Discount))Test) Order)total) Customer)ra3ng) Discount)percentage?) $10.00% Good% 0%% $10.01% Good% 1%% $50.01% Good% 1%% $.01% Excellent% 1%% $50.00% Excellent% 1%% $50.01% Excellent% 5%% Requirement Test 73
- 74. Context 74
- 76. Another Context Diagram Process' User Another System Shared Database Commands Views Requests Responses Stores Retrievals 76
- 77. Engine Controller Engine& Controller& Engine Rotation Speed Gasoline Flow Control Spark Timing Acceleration / De-acceleration Air Flow Control 77
- 78. Embedded Contexts Sub$ System Two Input Output System One Internal I/O 78
- 79. Car Context Drive& sha*& & Acceleration De-acceleration Movement Trans- mission& Engine& & Wheels& & What to test? RPMs? Miles per hour 79
- 80. Software Context Core or API Developer Acceptance Test Customer Acceptance Test Middle-tier GUI What to test? Elements in an array? Behavior of the system 80
- 81. What To Test Behavior Actions Changes via the API via the same mechanism your GUI does Not elements in a data structure Not fields in the database 81
- 82. Use Cases and Tests 82
- 83. Use Case Template Name—Identifier to easily reference it by Description—Brief note Actor—Who initiates the use case Pre-conditions—What must be true before the use case is initiated Post-conditions—What’s true if the use case successfully executes Main course—Steps that show the sequence of interactions 83
- 84. Example: Part One Name—Check Out CD. Description—Check out a CD for a customer. Actor—Clerk. Pre-conditions—The customer has an identification. The CD has an identity. Post-conditions—The CD is recorded as rented. The rental contract is printed. Main Course: 1. The clerk enters the customer identification and CD identifier into the system. 2. The system records the information. 3. The system prints a contract that the customer signs. 84
- 85. Example: Part Two Exceptions: 1a. Customer identification is not recognized. Clerk repeats step 1. 1b. The customer violates the CD Rental Limit business rule. The clerk notifies the customer of the violation. The use case is abandoned. Business Rule: CD Rental Limit A customer can rent only three CDs at any one time. Alternatives 3a. The printer jams. The clerk fills out the contract by hand. The use case exits. 85
- 86. Acceptance Testing The Triad create acceptance tests for stories Tests from general (expected path) ... ... to specific (exceptions) 86
- 87. Examples Rent a CD (expected path) Bad Customer ID (Enter it wrong) CD Rental Limit (already has 3. Try 4.) Print jam (simulate by removing paper) 87
- 88. Test Anatomy 88
- 90. Test Flow Ini$al' System' State' Test' Setup' (Given))' Trigger' (When)' Assert' (Then)' Final' System' State'and' Output' Action or Event Expected System' State'and' Output' 90
- 91. Given/When/Then Given = Use case pre-conditions When = Main course (or exceptional course) Then = Use case post-conditions 91
- 92. Given/when/Then Given (Setup) Customer has ID (initial system state) CD has ID (initial system state) CD is not currently rented (initial system state) When (Trigger) Clerk checks out CD (action) Then (Verify) CD recorded as rented (final system state) Rental contract printed (output) 92
- 93. Given/when/Then Given Customer has ID CD has ID CD is not currently rented When Clerk checks out CD Then (Verify) CD recorded as rented Rental contract printed 93
- 95. Internal or External Persistence Output is response to change address Change address Send mail External repository for address Input for send mail 95
- 96. Internal or External Persistence How to test internal persistence? Through another action Through a “see-through” method APIs should be symmetrical Add/Delete Modify/View Change address Internal persistence of address (state) Send mail Addressed mail 96
- 97. Exercise Write questions (tests) for: “copying a file from one directory to another” Describe in terms of: Given: some state When: copy (file, source_directory, destination_directory) Then: expected state Source' Directory' Contents' Des/na/on' Directory' Contents' Copy' File'' Des/na/on' Directory' Contents?' Anything'else?'' X' X' X' Given When Then (Expected) 97
- 99. Example Source' Directory' Contents' Des/na/on' Directory' Contents' Copy' File'' Des/na/on' Directory'' Contents?' Source' Directory' Contents?' Content' Comparison?' Notes' X,#Y,#Z# D,E,f# X# D,E,F,X# X,Y,Z# (Des-na-on.X# ==#Source.X)# Standard# copy## X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X# ==#Source.X)# X# overwri?e n# X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X# ==# Des-na-on.X)# # X#not# overwri?e n# What about modification date and other attributes? Make more columns Given When Then (Expected) 99
- 100. Example Source' Directory' Contents' Des/na/on' Directory' Contents' Copy' File'' Des/na/on' Directory'' Contents?' Source' Directory' Contents?' Content' Comparison?' Notes' X,#Y,#Z# D,E,f# X# D,E,F,X# X,Y,Z# (Des-na-on.X# ==#Source.X)# Standard# copy## X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X# ==#Source.X)# X# overwri?e n# X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X# ==# Des-na-on.X)# # X#not# overwri?e n# What about modification date and other attributes? Make more columns Given When Then (Expected) 100
- 101. Example Source' Directory' Contents' Des/na/on' Directory' Contents' Copy' File'' Des/na/on' Directory'' Contents?' Source' Directory' Contents?' Content' Comparison?' Notes' X,#Y,#Z# D,E,f# X# D,E,F,X# X,Y,Z# (Des-na-on.X# ==#Source.X)# Standard# copy## X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X# ==#Source.X)# X# overwri?e n# X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X# ==# Des-na-on.X)# # X#not# overwri?e n# What about modification date and other attributes? Make more columns Given When Then (Expected) 101
- 102. Example Source' Directory' Contents' Des/na/on' Directory' Contents' Copy' File'' Des/na/on' Directory'' Contents?' Source' Directory' Contents?' Content' Comparison?' Notes' X,#Y,#Z# D,E,f# X# D,E,F,X# X,Y,Z# (Des-na-on.X# ==#Source.X)# Standard# copy## X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X# ==#Source.X)# X# overwri?e n# X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X# ==# Des-na-on.X)# # X#not# overwri?e n# What about modification date and other attributes? Make more columns Given When Then (Expected) 102
- 103. Tables 103
- 104. Tables Table-driven requirements can reduce ambiguity Tables can be tests Discovery Tables bring to light overlooked assumptions Draws out unstated requirements 104
- 107. Calc Table Example Discount Calculation Item Total Customer Rating Discount Percentage? $10.00 Good 0% Given When Then 107
- 108. Data ! Exists&(or&should&exist)&–&for&Given&and&Then& Title Data Value Name 1 Value Name 2 Value for 1 Value for 2 Another value for 1 Another value for 2 108
- 109. Data Table Example Customer Data Name ID James 007 Maxwell 86 Customer Data Name Begins with="’J" Name ID James 007 109
- 110. Text Requirements User can log in What are the rules? 110
- 111. Login Grid User Name Password Result Notes 111
- 112. Action Table Enter enters data into an entry field Press initiates a process, such as a Submit button Check sees if a result is equal to an expected value 112
- 113. Action Table Example Check Out CD Enter Customer ID 007 Enter CD ID CD2 Press Rent Check Rented True Can be just “When” or a stand-alone Given-When-Then Given When Then 113
- 114. Copy File Example Copy%a%file%(happy%path)%% ! Given&a&source&& ! and&target&directory& &&&&&& File system Directory Name Directory Contents A X,&Y,&Z& B D,E,F Copy File Enter Source Directory A Enter Target Directory B Enter File name X Press Copy ! When©&request&is& made& &&&&&& ! Then&file&is&copied& &&&&&& File system Directory Name Directory Contents A X,&Y,&Z& B D,E,F,X 114
- 115. The Action ! Can$drive$a$GUI$ ! Or$a$method$$ CopyFile(Directory source, Directory target, Filename file) 115
- 116. Business Rules Test Usually many One test per rule 116
- 117. Example Rates& Type& Days& Cost?& Regular( 2( $2( Golden(Oldie( 3( $1.50( Hot(Stuff( 6( $12( Hot(Stuff( 50(( IGBTYOT(( CD(Rental(Rates( Regular(($2(/(2(days(plus($1(/(day(( Golden(Oldie($1(/(2(days(plus($(.50(/(day( Hot(Stuff(($4(/(2(days(plus($2(/(day( 117
- 118. UI & Business Rules Email&Valida)on&(Middle/Tier)& Value&& Valid?&& [email protected], yes, someone@@somewhere.com, no, !#$%&*+7/=?^_,{|}[email protected], yes, Email Ajax call to middle-tier Submit, Middle-tier test 118
- 119. Alternatives Can use "script" in tables or separately 119
- 121. External Interfaces ! Connec&ons(to(external(systems(need(to(have(test(doubles( (mocks)( ! Random(events(may(need(to(be(simulated(( ! Test(doubles(give(repeatability(and(speed(( External system, device, service Random events Time 121
- 122. External Interface Example CD Rental System Credit Card Processor Sam’s Bank Charge Transfer To Confirmation or Denial Bank Statement 122
- 123. Example of a Credit Card Test Double CD Rental System Credit Card Processor Test Double Charge Confirmation or Denial 123
- 124. Example of a Email Test Double CD Rental System Email Server Test Double Email Confirmation or Denial Test verifies that email sent to proper address with proper contents 124
- 126. Complex Business Rule Field&One& Field&Two& Field&Three& Field&Four& Result&?& >&20& <&50&or&blank& >=100& Y& Allow& Otherwise&or& blank& DNC& DNC& DNC& Disallow& DNC& Otherwise& DNC& DNC& Disallow& DNC& DNC& Otherwise&or& blank& DNC& Disallow& DNC& DNC& DNC& N&or&blank& Disallow& A business rule determines whether a user is allowed to perform a certain operation Fields may contain values or be blank 126
- 127. Complex Business Rule Field&One& Result?& >&20& Allow& Blank& Disallow& Otherwise& Disallow& Field&Two& Result?& <&50& Allow& Blank& Allow& Otherwise& Disallow& Field&Three& Result&?& >=100& Allow& Blank& Disallow& Otherwise& Disallow& Field&Four& Result?&& Y& Allow& N& Disallow& Blank& Disallow& Otherwise& ??& 127
- 128. Complex Business Rule Field&One& Field&Two& Field&Three& Field&Four& Result&?& Allow& Allow& Allow& Allow& Allow& Disallow& DNC& DNC& DNC& Disallow& DNC& Disallow& DNC& DNC& Disallow& DNC& DNC& Disallow& DNC& Disallow& DNC& DNC& DNC& Disallow& Disallow& 128
- 129. Google Maps What can you test w/o a UI? 129
- 130. Google Maps No UI? Search phrase => List of places Place => Position Zoom Level => List of maps Position => List what's there 130
- 131. Test Evaluation 131
- 132. WARNING!! Acceptances tests do NOT replace interactive communication! They provide focus for the communication 132
- 133. Test Evaluation Remove redundant redundancy Business domain terms Shared between customer unit and developer/ tester units Avoid lots of input and output columns Break into smaller tables 133
- 134. Guidelines Develop tests & automation separately First, understand Second, automate (if appropriate) Automate tests for regression Run in continuous integration Cover 100% of function (if practical) 134
- 135. Separation of Concerns Separate business rules from how results of business rules are displayed Separate calculation of a business rule (such as a customer rating) from the use of that business rule (such as to give a discount) Separate each use case or step in a workflow Separate out validation of an entity from use of entity 135
- 136. Separation and Abstraction One story together: When the build fails, send an email to the administrator Separate and abstracted: When the build fails, generate an alert When an alert is generated, notify the responsible individual When the responsible individual is notified, use that individual’s notification preference (email) Test the notification itself works (e.g. email works) 136
- 137. What is a Good Test? Fails reliably (for a known reason) No false positives No other test covers the same area NOTE: Tests at the "top" often forced to assume business rules... aim for 1 failure at each level 137
- 138. Use Alternate Tables Sudoku Puzzle 1 4 7 2 5 8 3 6 9 4 7 1 5 8 2 6 9 3 7 1 4 8 2 5 9 3 6 Sudoku Solution 1 6 5 4 9 8 7 3 2 9 2 4 3 5 7 6 8 1 8 7 3 2 1 6 5 4 9 4 9 8 7 3 2 1 6 5 3 5 7 6 8 1 9 2 4 2 1 6 5 4 9 8 7 3 7 3 2 1 6 5 4 9 8 6 8 1 9 2 4 3 5 7 5 4 9 8 7 3 2 1 6 Create tables in alternative forms, if standard not appropriate Required(Performance(( Puzzle(Type( Time(to(Solve?( Notes( Easy% .1%seconds%% See%easy%determina2on% Impossible% 1%second% No%solu2on%% 138
- 139. Other uses for Tests Estimation Aids Degree of Doneness 139
- 140. Process note Requirement Testable Requirement Developable Requirement At least one example for each test Not schedulable Preferably all examples for all tests 140
- 141. process Note Code% write% run% bind% Code% write% run% bind% Write previous iteration Write within iteration 141
- 142. SCMDevelopers Watch Build Unit Tests Deploy to Dev Integration Tests First Look Continuous Testing On Demand Deploy to QA Manual Testing 142
- 143. Not the End.... The Beginning! 143
- 144. Objective Review Understand ATDD in context of SDLC Learn how to turn requirements into tests What is a good acceptance test? ATDD as a communication vehicle 144
- 145. Our Outline Terms and definitions Requirements and tests Software development ATDD examples Anatomy of a test Tables System boundary 145
- 146. Recap Primary goals Discover ambiguous requirements and gaps in requirements early on Create a record of business/development understanding Give feedback on quality Secondary goals Use acceptance tests as an executable regression test Measure your progress towards "done” completeness Measure the complexity of requirements. Use the tests as a basis for user documentation 146
- 147. Rules Convert requirements into tests Write tests before code Write tests in grids (spreadsheets) Never write code without tests Include the Golden Triad 147
- 148. Further Reading GROWS Method http://GrowsMethod.com BDD Behavior Driven Development (by Dan North) Cucumber http://cukes.info Tellurim (hosted Cucumber) http://te52.com Junit http://jaredrichardson.net/articles/junit-tutorial.html FitNesse fitnesse.org 148
- 149. Exercise You listed your development issues... Would ATDD help, hurt, or be neutral Why? Write down your answer and pass it up 149
- 150. Take Aways What’s a test? Who writes them? When are they written? What do they verify? Provide an example Primary benefit? Secondary benefits? 150
- 151. 151 151