SlideShare a Scribd company logo
Between Scrum and
Kanban – define test
process
             Zbyszek Moćkun

                  © 2010 Cognifide Limited. In commercial confidence only.
Agenda

1.   Methodologies
2.   Problems to overcome
3.   Workflows
4.   Issue management
5.   Taskboard
6.   Exploratory testing
7.   Metrics

                       © 2010 Cognifide Limited. In commercial confidence only.
Methodologies

Few words about Agile




                        © 2010 Cognifide Limited. In commercial confidence only.
Methodologies axis


Prescriptive                                                       Adaptive


R               X     S              K
U               P     c              a               Do
P              (13)   r              n               whatever
(120+)                u              b               (0)
                      m              a
                                     n
                      (9)
                                     (3)

                            Source: Kanban vs Scrum – how to make the
                            best of both (Henrik Kniberg)

                                                      © 2010 Cognifide Limited. In commercial confidence only.
Kanban

•    Visualize the workflow
•    Limit WIP
•    Measure the lead time
•   •Manage the lenght of the queue
      Coverage
•   •No fixed timeboxes iteration
      Do not duplicate tests
      during regression
•   •You can choose when:
      Test specific
    − functionality
       Planning
    − Release
    − Retrospective



                                      © 2010 Cognifide Limited. In commercial confidence only.
Scrum-inspired
• Do not limit yourself to one tool
• Mix and match tools as you need
• Dynamic changes in projects (suite your
  process to actual needs)
   • Coverage
   • Do not duplicate tests
• Retrospective – continuous improvements
     during regression
   • Test specific
• Agile manifesto (Individuals and
     functionality

  interactions over processes and tools)



                                   © 2010 Cognifide Limited. In commercial confidence only.
Problems

Problems that we had to overcome




                             © 2010 Cognifide Limited. In commercial confidence only.
Problems to be overcame

• Project status on Taskboard
    − Status of all tasks visualized on Taskboard
    − No need of additional reports
    − Visualize QA activities
      • Coverage
•   Exploratory testing management
     • Do not duplicate tests
       during regression
•   Ensurespecificall tasks are tested
     • Test
             that
•   Metrics: quality, testing velocity, ...
       functionality

•   One process for different methodologies: Scrum,
    Kanban, mixed


                                                    © 2010 Cognifide Limited. In commercial confidence only.
Workflows

Ensure a QA place in workflow




                                © 2010 Cognifide Limited. In commercial confidence only.
The meaning of „Done”

• When do we finish our work?
• What is the latest action?
• Who is responsible for the transfer to
  the state done?
• What is the client role?
• Visualize the done state
• Done for Scrum vs Done for Kanban




                                           © 2010 Cognifide Limited. In commercial confidence only.
QA States

There are no QA states in standard workflows
• assumption - close action is for client (sign off)
• Green states
Add qa states to your workflow
• QA queue, Testing in progress, Tested
• Green plus white states

   In                      Testing in
             Resolved                      Tested        Closed
Progress                   Progress




                                                       © 2010 Cognifide Limited. In commercial confidence only.
Keep the standards

• Do not change the workflow parts that customer use




    In                       Testing in
               Resolved                    Tested            Closed
 Progress                    Progress




                              Testing in
 In Progress   Implemented                 Resolved              Closed
                              Progress




                                                      © 2010 Cognifide Limited. In commercial confidence only.
Different tasks, different needs

• Client oriented – extended workflow
  − QA Engineer is not always a client
  − Client has to accept task solution
• Single role – simple workflow
  − Division of task on the parts by developer




                                                 © 2010 Cognifide Limited. In commercial confidence only.
Issues management

Make it simple




                 © 2010 Cognifide Limited. In commercial confidence only.
Issues by software lifecycle




                               © 2010 Cognifide Limited. In commercial confidence only.
Issues tracking

• Regression – Standard Issue Type
• Acceptance – Sub-Task Issue Type




                                     © 2010 Cognifide Limited. In commercial confidence only.
Story as a container

• All issues connected with story are reported as sub task
  (bug, improvement, question, task)
• All data in one place (historical data too)
• All sub tasks have to be implemented to send story to QA
• Story accepted only if no sub-issues left


                               Story




       Sub
                  Sub bug     Sub bug     Sub bug               Subtask
    improvment




                                                    © 2010 Cognifide Limited. In commercial confidence only.
Dev – QA cooperation




                       © 2010 Cognifide Limited. In commercial confidence only.
Quality levels

• Are all issues have to be resolved?
• Can we fix some of them later?
• Flexible or restricted process?




                                        © 2010 Cognifide Limited. In commercial confidence only.
Retest: whole story or sub-
issues only
• Switching between contexts require very good time
  managment skills
• Helicopter view
• Do not duplicate data (JIRA <->TCM)
• All failed issues attached to story
• QA Engineer can make decision
  −   Retest whole story
  −   Only failed issues
  −   Chosen parts (change propagation)
  −   Decision made on available data (fixed issues, introduced
      changes, developer output)



                                                           © 2010 Cognifide Limited. In commercial confidence only.
Taskboard

Integrate Workflow with Taskboard




                               © 2010 Cognifide Limited. In commercial confidence only.
Workflow on Taskboard
• One tool, all information, no additional reports
• Only the Standard Issue Type visible
 − Acceptance issues are hidden
• Sorted by rank (priority)
• Rejected issues again in To Do queue




                                                     © 2010 Cognifide Limited. In commercial confidence only.
Exploratory testing

Why scripted aproach doesn’t work




                              © 2010 Cognifide Limited. In commercial confidence only.
Scripted testing doesn’t work

• Status in two tools: JIRA and TCM
• Retesting dilemma
• Test Script creation in advance
 − boring and not efficient
   • Coverage
 − Can’tnot duplicate tests 100% coverage
   • Do write scripts for
     during regression
 − Requirements/Acceptance criteria updates
   • Test specific
 − No time for this phase (parallel testing)
    functionality

• Acceptance tests




                                               © 2010 Cognifide Limited. In commercial confidence only.
Scripting doesn’t work - reports
• Scripted testing report
 phase               All              Executed   passed                      failed
 Test design phase   80               0          0                           0
 First day           90 (4 updated)   20         16                          4
 Second day          95 (10)          75         65                          10
 Third day           120 (14)         75         73                          2
 Fourth day          125 (16)         110        98                          12

• Not all founded issues are against test case
• Test case number means nothing (James Bach)
• New test scripts added almost each day
• Old onces are updated

                                                          © 2010 Cognifide Limited. In commercial confidence only.
Exploratory testing
• Efficiency
 − Concentrate on functionality
 − Time management (quality level)
 − Critical issues are founded earlier
 − Changes are welcome
 − No test script creation in advance phase
 − Test scripts as output (for regression tests purpose)
• Learning phase
 − Simulate users (usability issues)
• Move responsibility on tester
 − Retesting dilemmas



                                                           © 2010 Cognifide Limited. In commercial confidence only.
Improve your ET skills




                         © 2010 Cognifide Limited. In commercial confidence only.
Metrics

Find what you need




                     © 2010 Cognifide Limited. In commercial confidence only.
Metric by actions




                    © 2010 Cognifide Limited. In commercial confidence only.
One measurement,
two metrics
• Acceptance rate
 − Acceptance actions/all qa actions
 − All QA actions = accept or reject
 − Measure quality of software send to QA
• QA velocity
 − Sum off all actions
 − Divide by number of QA Engineers assigned to project
 − QA velocity depend on software quality that comes to qa
 − Why testing takes so long? (Kaner’s blog)




                                                          © 2010 Cognifide Limited. In commercial confidence only.

More Related Content

What's hot (20)

PDF
Combining Performance Testing And Modelling For Easy Jet.Com V 1.0
dannyq
 
PPTX
Segue presentation 1 17-2013
Bill Roderick
 
PDF
Inverting The Testing Pyramid
Naresh Jain
 
PPTX
Testing Sap: Modern Methodology
Ethan Jewett
 
PDF
Shirly Ronen - Documenting an agile defect
AgileSparks
 
PPTX
Test pyramid agile pune conference
Sachin Natu
 
PPTX
How to bake in quality in agile scrum projects
Santanu Bhattacharya
 
PPTX
Qfd
mogambo
 
PDF
Karate API Testing-Complete Guidance by Testrig
PritiFGaikwad
 
PDF
QA is dead long live the new QA - Agile Dev and QA Conference Israel
Yuval Yeret
 
DOC
CV_VENUGOPAL B
Budhi Venugopal
 
PPTX
Quality Coding with Visual Studio 2012
Imaginet
 
PPT
2010 07-15 rbeem nahbrc quality management
Amber Joan Wood
 
PPTX
Quality Coding: What's New with Visual Studio 2012
Imaginet
 
PPTX
Quality Coding: What’s New with Visual Studio 2012
Imaginet
 
PDF
Change agile for XP Days 2012 benelux v1.0
Ben Linders
 
PDF
Agile Project Outsourcing - Dealing with RFP and RFI
Siddhi
 
PDF
Ravit Danino HP - Roles and Collaboration in Agile
AgileSparks
 
DOC
Pradeep CV
Pradeep Kulkarni
 
PDF
Qap White Paper
Geert Appeltans
 
Combining Performance Testing And Modelling For Easy Jet.Com V 1.0
dannyq
 
Segue presentation 1 17-2013
Bill Roderick
 
Inverting The Testing Pyramid
Naresh Jain
 
Testing Sap: Modern Methodology
Ethan Jewett
 
Shirly Ronen - Documenting an agile defect
AgileSparks
 
Test pyramid agile pune conference
Sachin Natu
 
How to bake in quality in agile scrum projects
Santanu Bhattacharya
 
Qfd
mogambo
 
Karate API Testing-Complete Guidance by Testrig
PritiFGaikwad
 
QA is dead long live the new QA - Agile Dev and QA Conference Israel
Yuval Yeret
 
CV_VENUGOPAL B
Budhi Venugopal
 
Quality Coding with Visual Studio 2012
Imaginet
 
2010 07-15 rbeem nahbrc quality management
Amber Joan Wood
 
Quality Coding: What's New with Visual Studio 2012
Imaginet
 
Quality Coding: What’s New with Visual Studio 2012
Imaginet
 
Change agile for XP Days 2012 benelux v1.0
Ben Linders
 
Agile Project Outsourcing - Dealing with RFP and RFI
Siddhi
 
Ravit Danino HP - Roles and Collaboration in Agile
AgileSparks
 
Pradeep CV
Pradeep Kulkarni
 
Qap White Paper
Geert Appeltans
 

Similar to Between Scrum and Kanban - define test process for Agile methodologies (20)

PPTX
Test Strategy
Zbyszek Mockun
 
PDF
Automation as a project - lessons learnt
Zbyszek Mockun
 
PPTX
Project quality (and test process) metrics
Zbyszek Mockun
 
PDF
Automated Exploratory Tests
Zbyszek Mockun
 
PPT
Test automation lessons from WebSphere Application Server
Robbie Minshall
 
PDF
Vaidyanathan Ramalingam Rca In Agile Conference Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
PDF
Vaidyanathan Ramalingam Waterfall Vs Agile Testing Conference Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
PDF
Vaidyanathan Ramalingam Trade Off Economics In Testing Conference Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
PDF
Vaidyanathan Ramalingam Agile Testing Leadership Lessons Softec 2 July2011
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
PDF
Vaidyanathan Ramalingam_Agile QA_SOFTEC_2_July2011_Silicon India Conf_Bangalore
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
PDF
Vaidyanathan Ramalingam_Testing in Agile_SOFTEC_2_July2011_Silicon India Conf...
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
PDF
Vaidyanathan Ramalingam Agile Conference Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
PDF
Vaidyanathan Ramalingam Rca In Testing Conference Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
PDF
Vaidyanathan Ramalingam Testing Checklist Conference Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
PDF
Vaidyanathan Ramalingam_Sprint Testing_SOFTEC_2_July2011_Silicon India Conf_B...
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
PDF
Vaidyanathan Ramalingam Silicon India Testing Conference 2 July2011 Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
PDF
Vaidyanathan Ramalingam Software Testing Eco System Conference Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
PDF
Vaidyanathan Ramalingam Agile Testing Conference Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
PDF
Vaidyanathan Ramalingam_Iterative Testing_SOFTEC_2_July2011_Silicon India Con...
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
PDF
Flexing your Agile Muscle - Agile Technical Concepts Explained
Sandy Mamoli
 
Test Strategy
Zbyszek Mockun
 
Automation as a project - lessons learnt
Zbyszek Mockun
 
Project quality (and test process) metrics
Zbyszek Mockun
 
Automated Exploratory Tests
Zbyszek Mockun
 
Test automation lessons from WebSphere Application Server
Robbie Minshall
 
Vaidyanathan Ramalingam Rca In Agile Conference Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
Vaidyanathan Ramalingam Waterfall Vs Agile Testing Conference Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
Vaidyanathan Ramalingam Trade Off Economics In Testing Conference Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
Vaidyanathan Ramalingam Agile Testing Leadership Lessons Softec 2 July2011
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
Vaidyanathan Ramalingam_Agile QA_SOFTEC_2_July2011_Silicon India Conf_Bangalore
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
Vaidyanathan Ramalingam_Testing in Agile_SOFTEC_2_July2011_Silicon India Conf...
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
Vaidyanathan Ramalingam Rca In Testing Conference Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
Vaidyanathan Ramalingam Testing Checklist Conference Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
Vaidyanathan Ramalingam_Sprint Testing_SOFTEC_2_July2011_Silicon India Conf_B...
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
Vaidyanathan Ramalingam Silicon India Testing Conference 2 July2011 Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
Vaidyanathan Ramalingam Software Testing Eco System Conference Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
Vaidyanathan Ramalingam Agile Testing Conference Speech
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
Vaidyanathan Ramalingam_Iterative Testing_SOFTEC_2_July2011_Silicon India Con...
Skills2Talent (Integrated Talent Management and L&D Software - Hire to ROI)
 
Flexing your Agile Muscle - Agile Technical Concepts Explained
Sandy Mamoli
 
Ad

More from suwalki24.pl (20)

PDF
La powiat sp
suwalki24.pl
 
PDF
Gimnazjada ms
suwalki24.pl
 
PDF
Igrzyska ms
suwalki24.pl
 
PDF
Czwartki maj
suwalki24.pl
 
PDF
Czworboj2013
suwalki24.pl
 
PDF
Suwałki Blues Festival 2013
suwalki24.pl
 
PDF
Gim plywanie
suwalki24.pl
 
PDF
Licea plywanie
suwalki24.pl
 
PDF
20130226 komunikacja miejska w suwałkach w 2012r.
suwalki24.pl
 
PDF
Plywanie gm fg 2013
suwalki24.pl
 
PDF
Plywanie sp fg2013
suwalki24.pl
 
PDF
20130225 wyniki konsultacji
suwalki24.pl
 
PDF
Szelment 2013 wyniki
suwalki24.pl
 
ODT
Ogloszenie o konkursie dla dzieci e deklaracje-ii 2013
suwalki24.pl
 
PDF
2013.02.01 list otwarty
suwalki24.pl
 
PDF
Woj.podlaskie
suwalki24.pl
 
PDF
Strategia rozwoju woj_podlaskiego
suwalki24.pl
 
PDF
Plakat miedzynarodowy turniej[1]
suwalki24.pl
 
PDF
Badminton spg ms
suwalki24.pl
 
La powiat sp
suwalki24.pl
 
Gimnazjada ms
suwalki24.pl
 
Igrzyska ms
suwalki24.pl
 
Czwartki maj
suwalki24.pl
 
Czworboj2013
suwalki24.pl
 
Suwałki Blues Festival 2013
suwalki24.pl
 
Gim plywanie
suwalki24.pl
 
Licea plywanie
suwalki24.pl
 
20130226 komunikacja miejska w suwałkach w 2012r.
suwalki24.pl
 
Plywanie gm fg 2013
suwalki24.pl
 
Plywanie sp fg2013
suwalki24.pl
 
20130225 wyniki konsultacji
suwalki24.pl
 
Szelment 2013 wyniki
suwalki24.pl
 
Ogloszenie o konkursie dla dzieci e deklaracje-ii 2013
suwalki24.pl
 
2013.02.01 list otwarty
suwalki24.pl
 
Woj.podlaskie
suwalki24.pl
 
Strategia rozwoju woj_podlaskiego
suwalki24.pl
 
Plakat miedzynarodowy turniej[1]
suwalki24.pl
 
Badminton spg ms
suwalki24.pl
 
Ad

Recently uploaded (20)

PDF
ARAL_Orientation_Day-2-Sessions_ARAL-Readung ARAL-Mathematics ARAL-Sciencev2.pdf
JoelVilloso1
 
PPTX
MENINGITIS: NURSING MANAGEMENT, BACTERIAL MENINGITIS, VIRAL MENINGITIS.pptx
PRADEEP ABOTHU
 
PPTX
How to Set Maximum Difference Odoo 18 POS
Celine George
 
PDF
The dynastic history of the Chahmana.pdf
PrachiSontakke5
 
PPTX
STAFF DEVELOPMENT AND WELFARE: MANAGEMENT
PRADEEP ABOTHU
 
PPTX
HYDROCEPHALUS: NURSING MANAGEMENT .pptx
PRADEEP ABOTHU
 
PDF
Dimensions of Societal Planning in Commonism
StefanMz
 
PPTX
Universal immunization Programme (UIP).pptx
Vishal Chanalia
 
PDF
CONCURSO DE POESIA “POETUFAS – PASSOS SUAVES PELO VERSO.pdf
Colégio Santa Teresinha
 
PDF
DIGESTION OF CARBOHYDRATES,PROTEINS,LIPIDS
raviralanaresh2
 
PDF
Isharyanti-2025-Cross Language Communication in Indonesian Language
Neny Isharyanti
 
PPTX
Stereochemistry-Optical Isomerism in organic compoundsptx
Tarannum Nadaf-Mansuri
 
PDF
Lesson 2 - WATER,pH, BUFFERS, AND ACID-BASE.pdf
marvinnbustamante1
 
PDF
Chapter-V-DED-Entrepreneurship: Institutions Facilitating Entrepreneurship
Dayanand Huded
 
PDF
LAW OF CONTRACT (5 YEAR LLB & UNITARY LLB )- MODULE - 1.& 2 - LEARN THROUGH P...
APARNA T SHAIL KUMAR
 
PDF
Knee Extensor Mechanism Injuries - Orthopedic Radiologic Imaging
Sean M. Fox
 
PDF
SSHS-2025-PKLP_Quarter-1-Dr.-Kerby-Alvarez.pdf
AishahSangcopan1
 
PPT
Talk on Critical Theory, Part One, Philosophy of Social Sciences
Soraj Hongladarom
 
PPTX
2025 Winter SWAYAM NPTEL & A Student.pptx
Utsav Yagnik
 
PPSX
HEALTH ASSESSMENT (Community Health Nursing) - GNM 1st Year
Priyanshu Anand
 
ARAL_Orientation_Day-2-Sessions_ARAL-Readung ARAL-Mathematics ARAL-Sciencev2.pdf
JoelVilloso1
 
MENINGITIS: NURSING MANAGEMENT, BACTERIAL MENINGITIS, VIRAL MENINGITIS.pptx
PRADEEP ABOTHU
 
How to Set Maximum Difference Odoo 18 POS
Celine George
 
The dynastic history of the Chahmana.pdf
PrachiSontakke5
 
STAFF DEVELOPMENT AND WELFARE: MANAGEMENT
PRADEEP ABOTHU
 
HYDROCEPHALUS: NURSING MANAGEMENT .pptx
PRADEEP ABOTHU
 
Dimensions of Societal Planning in Commonism
StefanMz
 
Universal immunization Programme (UIP).pptx
Vishal Chanalia
 
CONCURSO DE POESIA “POETUFAS – PASSOS SUAVES PELO VERSO.pdf
Colégio Santa Teresinha
 
DIGESTION OF CARBOHYDRATES,PROTEINS,LIPIDS
raviralanaresh2
 
Isharyanti-2025-Cross Language Communication in Indonesian Language
Neny Isharyanti
 
Stereochemistry-Optical Isomerism in organic compoundsptx
Tarannum Nadaf-Mansuri
 
Lesson 2 - WATER,pH, BUFFERS, AND ACID-BASE.pdf
marvinnbustamante1
 
Chapter-V-DED-Entrepreneurship: Institutions Facilitating Entrepreneurship
Dayanand Huded
 
LAW OF CONTRACT (5 YEAR LLB & UNITARY LLB )- MODULE - 1.& 2 - LEARN THROUGH P...
APARNA T SHAIL KUMAR
 
Knee Extensor Mechanism Injuries - Orthopedic Radiologic Imaging
Sean M. Fox
 
SSHS-2025-PKLP_Quarter-1-Dr.-Kerby-Alvarez.pdf
AishahSangcopan1
 
Talk on Critical Theory, Part One, Philosophy of Social Sciences
Soraj Hongladarom
 
2025 Winter SWAYAM NPTEL & A Student.pptx
Utsav Yagnik
 
HEALTH ASSESSMENT (Community Health Nursing) - GNM 1st Year
Priyanshu Anand
 

Between Scrum and Kanban - define test process for Agile methodologies

  • 1. Between Scrum and Kanban – define test process Zbyszek Moćkun © 2010 Cognifide Limited. In commercial confidence only.
  • 2. Agenda 1. Methodologies 2. Problems to overcome 3. Workflows 4. Issue management 5. Taskboard 6. Exploratory testing 7. Metrics © 2010 Cognifide Limited. In commercial confidence only.
  • 3. Methodologies Few words about Agile © 2010 Cognifide Limited. In commercial confidence only.
  • 4. Methodologies axis Prescriptive Adaptive R X S K U P c a Do P (13) r n whatever (120+) u b (0) m a n (9) (3) Source: Kanban vs Scrum – how to make the best of both (Henrik Kniberg) © 2010 Cognifide Limited. In commercial confidence only.
  • 5. Kanban • Visualize the workflow • Limit WIP • Measure the lead time • •Manage the lenght of the queue Coverage • •No fixed timeboxes iteration Do not duplicate tests during regression • •You can choose when: Test specific − functionality Planning − Release − Retrospective © 2010 Cognifide Limited. In commercial confidence only.
  • 6. Scrum-inspired • Do not limit yourself to one tool • Mix and match tools as you need • Dynamic changes in projects (suite your process to actual needs) • Coverage • Do not duplicate tests • Retrospective – continuous improvements during regression • Test specific • Agile manifesto (Individuals and functionality interactions over processes and tools) © 2010 Cognifide Limited. In commercial confidence only.
  • 7. Problems Problems that we had to overcome © 2010 Cognifide Limited. In commercial confidence only.
  • 8. Problems to be overcame • Project status on Taskboard − Status of all tasks visualized on Taskboard − No need of additional reports − Visualize QA activities • Coverage • Exploratory testing management • Do not duplicate tests during regression • Ensurespecificall tasks are tested • Test that • Metrics: quality, testing velocity, ... functionality • One process for different methodologies: Scrum, Kanban, mixed © 2010 Cognifide Limited. In commercial confidence only.
  • 9. Workflows Ensure a QA place in workflow © 2010 Cognifide Limited. In commercial confidence only.
  • 10. The meaning of „Done” • When do we finish our work? • What is the latest action? • Who is responsible for the transfer to the state done? • What is the client role? • Visualize the done state • Done for Scrum vs Done for Kanban © 2010 Cognifide Limited. In commercial confidence only.
  • 11. QA States There are no QA states in standard workflows • assumption - close action is for client (sign off) • Green states Add qa states to your workflow • QA queue, Testing in progress, Tested • Green plus white states In Testing in Resolved Tested Closed Progress Progress © 2010 Cognifide Limited. In commercial confidence only.
  • 12. Keep the standards • Do not change the workflow parts that customer use In Testing in Resolved Tested Closed Progress Progress Testing in In Progress Implemented Resolved Closed Progress © 2010 Cognifide Limited. In commercial confidence only.
  • 13. Different tasks, different needs • Client oriented – extended workflow − QA Engineer is not always a client − Client has to accept task solution • Single role – simple workflow − Division of task on the parts by developer © 2010 Cognifide Limited. In commercial confidence only.
  • 14. Issues management Make it simple © 2010 Cognifide Limited. In commercial confidence only.
  • 15. Issues by software lifecycle © 2010 Cognifide Limited. In commercial confidence only.
  • 16. Issues tracking • Regression – Standard Issue Type • Acceptance – Sub-Task Issue Type © 2010 Cognifide Limited. In commercial confidence only.
  • 17. Story as a container • All issues connected with story are reported as sub task (bug, improvement, question, task) • All data in one place (historical data too) • All sub tasks have to be implemented to send story to QA • Story accepted only if no sub-issues left Story Sub Sub bug Sub bug Sub bug Subtask improvment © 2010 Cognifide Limited. In commercial confidence only.
  • 18. Dev – QA cooperation © 2010 Cognifide Limited. In commercial confidence only.
  • 19. Quality levels • Are all issues have to be resolved? • Can we fix some of them later? • Flexible or restricted process? © 2010 Cognifide Limited. In commercial confidence only.
  • 20. Retest: whole story or sub- issues only • Switching between contexts require very good time managment skills • Helicopter view • Do not duplicate data (JIRA <->TCM) • All failed issues attached to story • QA Engineer can make decision − Retest whole story − Only failed issues − Chosen parts (change propagation) − Decision made on available data (fixed issues, introduced changes, developer output) © 2010 Cognifide Limited. In commercial confidence only.
  • 21. Taskboard Integrate Workflow with Taskboard © 2010 Cognifide Limited. In commercial confidence only.
  • 22. Workflow on Taskboard • One tool, all information, no additional reports • Only the Standard Issue Type visible − Acceptance issues are hidden • Sorted by rank (priority) • Rejected issues again in To Do queue © 2010 Cognifide Limited. In commercial confidence only.
  • 23. Exploratory testing Why scripted aproach doesn’t work © 2010 Cognifide Limited. In commercial confidence only.
  • 24. Scripted testing doesn’t work • Status in two tools: JIRA and TCM • Retesting dilemma • Test Script creation in advance − boring and not efficient • Coverage − Can’tnot duplicate tests 100% coverage • Do write scripts for during regression − Requirements/Acceptance criteria updates • Test specific − No time for this phase (parallel testing) functionality • Acceptance tests © 2010 Cognifide Limited. In commercial confidence only.
  • 25. Scripting doesn’t work - reports • Scripted testing report phase All Executed passed failed Test design phase 80 0 0 0 First day 90 (4 updated) 20 16 4 Second day 95 (10) 75 65 10 Third day 120 (14) 75 73 2 Fourth day 125 (16) 110 98 12 • Not all founded issues are against test case • Test case number means nothing (James Bach) • New test scripts added almost each day • Old onces are updated © 2010 Cognifide Limited. In commercial confidence only.
  • 26. Exploratory testing • Efficiency − Concentrate on functionality − Time management (quality level) − Critical issues are founded earlier − Changes are welcome − No test script creation in advance phase − Test scripts as output (for regression tests purpose) • Learning phase − Simulate users (usability issues) • Move responsibility on tester − Retesting dilemmas © 2010 Cognifide Limited. In commercial confidence only.
  • 27. Improve your ET skills © 2010 Cognifide Limited. In commercial confidence only.
  • 28. Metrics Find what you need © 2010 Cognifide Limited. In commercial confidence only.
  • 29. Metric by actions © 2010 Cognifide Limited. In commercial confidence only.
  • 30. One measurement, two metrics • Acceptance rate − Acceptance actions/all qa actions − All QA actions = accept or reject − Measure quality of software send to QA • QA velocity − Sum off all actions − Divide by number of QA Engineers assigned to project − QA velocity depend on software quality that comes to qa − Why testing takes so long? (Kaner’s blog) © 2010 Cognifide Limited. In commercial confidence only.

Editor's Notes

  • #7: Dynamic changes in projects (suite your process to actual needs) – ex. When client not deliver requriements/not answer on question you can’t plan sprint, so you can’t use scrum
  • #9: Musielismy sie zmierzyc z wizualizacja stanu projektu za pomoca taskboarda. Minimalizacja raportow QA, jak informowac o statusie testow na biezaca – integracja z taskboardemJak dobrze zarzadzac testami eksploracyjnymiJak zarzadzac statusami nowych funkcjonalnosciStatus pracy qa na taskboardzie – czyli w tym samym miejscu gdzie i status pracy reszty zespoluProject status on TaskboardStatus of all tasks visualised on Taskboard for client/ PMNo need of additional reportsQA ReportsRemove the need of qa reportsVisualistion issue state incliding testingVisualisation QA activitiesOne tool – how to repleace Test Case Management reportsHow to manage exploratory testingHow to manage issues status according to visualisationDefine QA role in projectsDefine the place of qa in our processOne process for different methodologies: Scrum, Scrum-approach, KanbanMeasure the quality and qa workScripting doesn’t work
  • #11: Kiedy konczymy prace – czy mozemy polegac na kliencie.W Scrum, aby skonczyc sprint wszystkie itemy musza trafic w stan Done, ciezko tutaj polegac na kliencie.Sprint konczy sie demo, po ktorym klient robi sign off, niestety czest jest niedostepny lub nie czeka z sing off, wiec chcemy sie pozbyc tej zmiennejdo czesci z funkcjonalnosci dodaje poprawkiStan done w naszym wypadku jest ostatnim na po prawej stronie w taskboard, aby latwo klient widzial co czeka na niego. Jesli klient naprawde wspolpracuje, to dodajemy oczywiscie kolumne sign off.W Kanbanie nie jest to juz konieczne, nie mamy ograniczenia czasoweg, mozemy spokojnie czekac
  • #12: Tutaj kilka slow dlaczego jest wazne aby qa byl kolejny stanem w etapie wytwarzania oprogramowani. Standrdowe workflowy niestety ich nie maja u musimy je edytowac
  • #13: Zmieniaj tylko te czesci ktore nie sa widocznie dla klientow, nie wymagaja ich udzialu jesli to mozliwe
  • #14: Jak rozroznic taski, dlaczego dla niektorych warto uproscic workflowy, punkt widzenia – czy mamy klienta dla taska czy niSingle – czyli bez klienta, jestesmy sami sobie klientem, np dzielimy prace na pewne kawalki – na przyklad takie jakie mozemy zrobic przez jeden dzien – planowanie pracy.- Client oriented – czyli takie ktore wymagaja wspolrpacy z roznymi rolami, nie zawsze to qa musi zaakceptowac taski, moze to byc technical lead, project manager, analityk, ...
  • #16: Obrazek do poprawy,Cykl zycia softwaru – blecy akceptacji podczas wytwarzania funkcjonalnosci, bledy regresyjne sa dla juz oddanej funkcjonalnosci
  • #17: Regresyjne – standard issue type (czyli dla rzeczy ktore juz skonczylismy – patrz slajd o Done)Akceptacyjne sub issue type, zawsze pod funkcjonalnoscia dla ktorej byly znalezione
  • #18: Sub issue as failed test scenario (usuniete, ale rowniez wazne)
  • #20: Dlaczego warto miec scisly proces i furtkeCzyli o wypinaniu i konwersji do standard issue typeesionCzy naprawde musimy wszystko naprawiac, moze czesc zechcemy naparwic pozniej, wiec zmienmy je z akceptance na reg
  • #21: Przypomnijmy ze story posiada zawsze cala historie pod soba- All failed issues attached to storyHelicopter view - Zawsze patzym an calosc a nie czastkeQa testuje tylko story, gdy juz cale jest naprawione, testownaie pomniejszych rzeczy nie ma sensu:Przelaczanie sie miedzy wieloma funkcjonalnosciami nie jest efektywnieWplyw wprowadzonych zmian na reszte funkcjonalnosciPotrzebujemy retetowac cala funkcjonalnosc, czy nie? Decyzja do qa co tak naprawde musimy powtorzyc
  • #23: Klienta interesuje ostateczny status, czy cos zostalo skonczone, lub w jakim miejscu sie znajduje, nie interesuje go ilosc skryptow testowych, ktore przeszy, ktore nie, tylko czy funkcjonalnosc jest gotowa czy nie. Praktycznie zaden PM czy klient mimo dostepu do narzedzi zarzadzania testami tam nie zagladal, zawsze wymagali oddzielnych raportow, na prosbe klienta, zawsze byl potrzebny test amnager ktory przedstawial dodatkowe stanyGoogle wygralo bo poprostu zwracala informacje ktore szukalismy. Tak samo u nas z taskboard. Jedno miejsce ktore dostarcza tego czego potrzebujemyAcceptance issues are hidden – widzimy jedynie ich liczbe dla konkretnergo taska - ale to juz konfiguracja
  • #25: Skrypting nie dziala, nie chcemy tak pracowac jednak:Musimy informowac jakos o statusie funkcjonalnosciZarzadzac testami automatycznymiZarzadzac testami regresjiDostarczac skrypty testowe dla klienta – wykonywane przez nich w czasie UATScripting nie sprawdza sie w testach akceptacyjnych, gdy poznajemy aplikacje – chcemy ja poznawac tak jak nasz uzytkownik, mamy problem ze znalezieniem statusu funkcjonalnosci bo jest rozbity, zawsze sa jakies bledy z jirze mimo 100% wykonalnosci testowBardzo wazne tutaj. Poniewz ccemy szybko wykrywac bledy, mamy testowanie ciagle, tak naprawde nie ma czasu na scripting creation in advance
  • #26: Raport wykoania testow nie mial sensu, nigdy nie bylismy w stanie napisac ile, ..Liczby klamalyUsuniete ze slajdow, ale warto wspomniec: -Infinite space in finite time - Manage your tests depends on timeMimo 100% egzekucji i tak trzba bylo sprawdzac status w 2 miejscach jira i qmetryOddzielne jiryStatus wymagan – ciezko bylo go znalesc, czy polaczyc go – JIRA – test Case managemen tool, zeby autoamtycznie sie updatetowalCo ma retestowc 0 tylko testy ktore poszly failedCo jesli zostala przepisana czesc funkcjonalnosci jeszcze raz wykonac testy ktore byly pass (jesli to zrobie z raportu wyjdzie ze nic nie zrobilem tego dnia)Opowiadanie czeczkaNot all issues assigned to test case4 failed tests – 6 issues assigned10 more issues in JIRA without test scenarion in test runAutomated tests in scope or not (CI)
  • #27: Ciag dalszy poprzedniego slajduTesty eksploracyjne okazaly sie lepsze bo daja wieksza mozliwosc zarzadzania testerom, szczegolnie przy akceptaycjnych, retescie, oszczedzaja czas, sa wydajniejsze, koncentruja sie na calej funkcjonalnosci, a nie konkretnym przypadku testowym. Ich wyniki sa tez bardziej miarodajne i mowia wiecej. Latwiej je tez planowac jesli jestesmy ograniczni czasem, wykryc najbardziej krytyczne bledy najszybciej. Czy tez uczyc sie z uzytkownikiemAgile manifesto – postawienie na czlowieka, jego wiedze i doswiadzcenie, to on podejmuje decyzje, no i mozliwosc zarzadzania czasem –dostosowanie poziomu testow do mozliwosci czasowych
  • #28: Heuristics, TestingDojos, SessionBasedTesting, CrossTesting, Checklists, RapidTesting, Tools, Context-Aware, DomainKnowledgeTesting Dojos – pair testing, work with othere, learn from themCross Testing – not pair testing but the idea that the same part should be tested by two testers / one after another. review other works, check what he find, talk and compare results. Different testers may be sensitive to differentfactors, or make different observations. The same tester may see different things atdifferent times or when intentionally shifting focus to different things.
  • #29: Inne spojrzenie na metryki - 2 lata temu jedna z przentacjina testwarez byla o metrykach, rpzeczytalem wiele artylulow, jednak nigdzie nie znalezlem nic o tych ktore sa dla mnie najwazniejsze.
  • #30: Qa do not improve quality, so measure on qa level .
  • #31: Screen z metryka – szybkosc qa zalezy od jakosci jaka dostajemhy. Jest to tak naprade suma wszelkich testow jak i retestow. Czyli jesli cos dostaniemy zlego, odrzucimy to kilka razy, to nasza efektywnosc jest roznie mierzonaManager – przetestowaliscie tylko jedno storyQA przetestowalismy 4 story (za jazdtm razem to samo), ale za kazdym razem dostawalismy je z bledem