QA Challenges in an Agile World

QA Challenges in an Agile
World
Presentation for Southern California
Quality Assurance Association
Presented by Yousef Abazari
(yousefabazari@yahoo.com)
November 17, 2015

What is covered in this presentation
 Introduction
 About me
 Brief overview of Agile
 Brief overview of Scrum
 Multi-sited scrum teams
 Scrum without test
automation
 Using waterfall methods in
Scrum
 Lack of code review
 Lack of unit testing
 Not capturing bugs in bug
reports
 Ambiguous user stories
 Testing a US without AC
 Limiting QA testing to AC
 Testing without TCs using AC
 Falling behind in testing US
 Q&A

Introduction
 I will go over some of the challenges that
software QA professionals encounter in the
context of a Scrum methodology while
performing their assigned scrum tasks
 I will provide some of the solutions (or
suggestive recommendations) to address
those issues

About me
 I have an Electrical Engineering degree from
McGill University (Montreal, Canada)
 I am an experienced software QA manager.
 I have managed QA teams at Accurate
Background (2y), Fujitsu(11y), Ericsson (3y),
Avocette (1y) and Infowave (<1y).
 My special interest is managing quality, starting
new QA teams, new test automation teams, and
defining a new QA process and methodology that
works best for a client.

What is Agile?
 Agile is a set of
principles and values
upon which a family of
incremental and
iterative software
development
methodologies are
based.
Agile Manifesto:
 Individuals and
Interactions over
processes and tools
 Working software over
comprehensive
documentation
 Customer collaboration
over contract negotiation
 Responding to change
over following a plan

What is Agile? – Continued
Agile is based upon:
 Adaptive planning
 Quick feedback cycles
 Evolutionary development
and testing
 Customer collaboration
 Encourage rapid & flexible
response to change
 Self organizing & Cross
functional teams
 Early delivery
 Continuous improvement
Some of the Software
Development Methodologies
that are based upon Agile
(note that this is not an
exhaustive list):
 Scrum
 Kanban
 Lean
 Extreme Programming
 Design Driven Development(D3)
 Test Driven Development (TDD)
 Crystal

What is Scrum?
According to Merriam-
Webster:
 In Rugby, a way of starting
play again in which players
from each team come
together and try to get
control of the ball by pushing
against each other and
using their feet when the ball
is thrown in between them
 A large group of people who
are close together in one
place
 It was discussed for the first
time in 1986 in an article (1)
 It is one of the most popular
Agile-based software
development methodologies
 Product or application is built
in a series of time-boxed
iterations called sprints
 Product is built using a
product backlog, which is a
prioritized list of features and
functionality updated
continuously
(1) Scrum’s origin goes back to a 1986 article published in Harvard Business Review by Takeuchi & Nonaka.
Jeff Sutherland and his team created the Scrum software development methodology in 1993 at Easel Corp.

What is Scrum? - Continued
 At times Scrum is used
synonymously to Agile
 Work is done by a small group
of 5-7 cross- functional (multi-
skilled), members sitting
physically next-to-one another
 At the end of each sprint, the
team has a potentially
(increment of) deployable
product
 Scrum activities are repeated
in cadence (flow of planned
activities repeated from one
sprint to another)
Scrum Roles;
- 1. Product owner
- 2. Scrum master
- 3. Team member (Dev, QA, DBA,
…)
Scrum Activities:
- 1. Sprint execution
- 2. Sprint Planning
- 2. Daily scrum
- 3. Sprint Review
- 4. Sprint Retrospective
- 5. Product Backlog grooming
- 6. Product (increment) deployment

What is Scrum? - Continued
Scrum artifacts:
- 1. Product Backlog
- 2. Product Backlog Items:
User Story (Feature or part
of a feature), Defect, Task,
Test task, Design spike,
Knowledge Acquision task
- 3. Sprint Backlog (a subset
of Product Backlog that is
being implemented in the
current sprint)
- 4. Shippable (increment of)
product
Scrum team participants and
corresponding numbers in a
team of 5-7 members:
- 1. Product owner (0.5-1)
- 2. Scrum master (0.5-1)
- 3. Developers (3-5)
- 4. Testers (1-2)
- 5. DBA (0-1)
It is important to keep the scrum
team size small and the number
of participants between 4-7 to
allow it to operate effectively.

Sprint Life Cycle Activities
Sprint
Planning
Acceptance
Testing
Reg /
Other QA
Testing (1)
Development
& Coding
Sprint Review
Sprint
Retrospective
Sprint
Deployment
Backlog
Grooming
Sprint Activities
(1) This can also be Single Sprint Level Acceptance Test (Product Increment Acceptance Test) and/or
Multi-Sprint Level Acceptance Test (Product Acceptance Test) and/or Non-Functional Acceptance Test.

P#1: Multi-sited scrum teams
 Problem: Placing members of the scrum
team physically in separate locations
(offshore/ onshore/ different
cities/buildings/floors) impacts the team’s
daily interaction, cohesion & communication
 Solution: Co-locate all team members in one
place, otherwise, plan regular visits, start
daily team chat board, daily calls, 1:1
messaging between team members, regular
updates to tickets’ progress on scrum board.
PO: Product Owner

P#2: Scrum without test automation
 Problem: Lack of smoke/ regression/
functional test automation impacts quality
and adequate test coverage of earlier user
stories implemented in this/previous sprints
 Solution: Automate AC for each US,
automate Smoke/Reg/Functional TCs,
Identify/reduce dependency of USs, Involve
developers in TA, Make TA a mandatory part
of Scrum activities, Create/run sprint-
level/multi-sprint-level E2E/Reg test cases
AC: Acceptance Criteria E2E: End-to-End Reg: Regression TA: Test Automation US: User Story

P#3: Using waterfall methods in Scrum
 Problem: Making USs available for testing at the
end of sprint, making many of US un-testable,
developing/ testing many (not 1 or 2) USs by one
person at the same time, lack of cross-functional
approach by scrum members, changing the scope of
a sprint mid-way, changing sprint length, Lack of pre-
defined cadence/rhythm for sprint activities, Not
applying the lessons learned (coming out of Sprint
Retrospective) from this sprint to the next sprint,
Lack of deployable product increment at the end of a
sprint that stops the team from getting the
customer/business stakeholder’s feedback.
US: User Story

P#3: Using waterfall methods in Scrum
 Solution: Avoid Waterfall method inclusion in Scrum if you can:
• Make user stories available for testing early in the sprint.
• Make all user stories testable by providing Acceptance Criteria.
• Each team member should only work on 1-2 US at the time and no
more.
• Each available scrum team member should help others performing
their tasks. QA help Developers and Developers helping QA, etc.
• Do not change the scope of a sprint mid-way.
• Do not change the sprint length while the sprint is in progress.
• Run sprint activities as planned (i.e. Sprint Planning on day 1, One
daily scrum in the morning, Sprint Retrospective on the last day of
sprint, etc.)
• Collect feedback from the team in Sprint Retrospective and apply
Lessons Learned to the next sprint. Target problems and not a person.
• Target to have a deployable product increment at the end of each
sprint in order to collect customer/business stakeholder’s feedback
and also identify issues earlier.
US: User Story

P#4: Lack of code review in Scrum
 Problem: Lack of code review (specially for complex
code being written/modified) results in build failures,
creating sub-optimal code, introducing catastrophic
run-time errors, adversely impacting existing code,
implementing user stories that go back between QA
and Dev multiple times, and impacting team velocity.
 Solution: Ensure senior developers review the code
created by junior or mid-level developers. Capture
review time in user story estimation during sprint
planning. You can add a state such as Ready for
Code Review on Scrum board. Capture review
comments. Avoid bottle necks for code review.

P#5: Lack of unit testing in Scrum
 Problem: Lack of unit testing results in build failures, creating
new code that does not work, bugs being uncovered later in the
process while testing a US, breaking existing functionality when
the code is changed later, burdening QA by making it busy
dealing with rudimentary bugs that should have been caught
earlier, forcing QA to expand the AC for the US.
 Solution: Write unit test for each public method, include the
unit test code with the base code and release it to the code
repository, make unit tests independent from one another,
consider unit test as an additional documentation of new code
base, run unit test before or as part of creating a new build &
ensure unit test covers both happy path and error scenarios.
When providing estimate for US, consider unit testing as well.
Code review is not a replacement for unit test and vice versa.
AC: Acceptance Criteria US: User Story

P#6: Not capturing bugs in bug
reports
 Problem: Communicating bugs (without capturing
them in bug reports) to developers verbally while
running acceptance criteria for user stories is a
common tendency among scrum participants. This
saves them a bit of time up front. However, creating
bug reports allows collecting QA metrics, provides a
history of an issue, allows issues to be viewed by
people outside the scrum team, allows bug reports to
be added to the product/sprint backlog and proves
that the QA analyst is doing his/her job right.
 Solution: Capture all bugs in bug reports and link
them to the user story if you can. While estimating a
user story, take into account the time needed to
open defects and to verify them later on.
Defining what the software should and should not do is a business decision (decided by the PO), not a technical decision.

P#7: Ambiguous User Stories
 Problem: The problem with implementing an ambiguous user
story is not knowing clearly what needs to be implemented.
This happens when a PO is not familiar with the functionality
being built, or doesn’t have clear business requirements to
create the user story, or lacks the time to investigate the
subject matter area to ‘cook’ the user story. Also, this happens
when there is not enough info provided by the customer about
a functionality.
 Solution: The scrum team should be courageous enough to
reject a US when it is not ready to be implemented or ask the
PO to provide more details/info. A good US has 5 (SMART) or
6 (INVEST) characteristics:
 SMART: Specific, Measurable, Achievable, Relevant, and
Time-boxed.
 INVEST: Independent, Negotiable, Valuable, Estimateable,
Small, and Testable.
An ambiguous US has no AC, lacks details, lacks user information, is not testable and/or lacks business value for the customer.

P#8: Testing a US without AC
 Problem: A user story without an AC is considered an
untestable US or ambiguous US. The problem with such
a US is that there is no written agreement on when it is
considered Done. In addition, it is harder to size a US
without AC. Lack of AC in a US makes it without
adequate content to work on.
 Solution: Make sure every US has one or more AC.
Acceptance Criteria within a US: defines what (and not
how it) is being implemented, identifies the boundary of a
US, provides the definition of Done for a US, allows QA to
test the US to see if “It Works as business Intended” and
not “I think it works or it works as coded”, and helps to
size a US. Use the format GIVEN, WHEN, THEN layout
to fill out the Acceptance Criteria information.
Acceptance Criteria are the functional/non-functional conditions/statements with a clear pass/fail result that are specified within a US that must be met
by an implemented software in order for the US to be considered “Done” (completed).

P#9: Limiting QA testing to AC
 Problem: There are many circumstances within which regular US
specific AC testing may not be adequate:
- Implementing a new US may break (under certain
circumstances) prior implemented user stories in this
sprint.
- Implementing a new US may break (under certain
circumstances) prior implemented user stories from
previous sprints
- When multiple scrum teams work on the same
application and their code can impact one another
- When scrum team is not performing in-scope
negative scenarios, boundary conditions and non-
functional testing (i.e. performance, stress, load,
soak, security, browser dependency testing)
- When scrum team is not performing Product level
acceptance testing.
AC: Acceptance Criteria US: User Story

P#9: Limiting QA testing to AC -
Continued
 Solution: In such cases additional testing is to be added in
the form of:
- Introducing test-only spikes or test-only USs with focus
on types/areas of testing that are missing.
- Introducing Hardening sprint(s) with focus on
types/areas of testing that are missing.
- Involving business SMEs and/or POs and/or QA in such
test activities
- Creating & running an auto smoke/regression test
- Use metrics to measure the success/impact of these
additional testing on quality improvement
- Additional Test spikes or USs or tasks can be performed
each sprint or the last sprint depending on the type of
testing being performed and scope of testing (To be
decided by the scrum team in Sprint Planning meeting)
AC: Acceptance Criteria PO: Product Owner SME: Subject Matter Expert US: User Story

P#10: Testing without TCs using AC
 Problem: Limit Scrum testing to AC provides a good coverage of a
product but under certain circumstances (listed below) it may not be
adequate:
- There are no detailed test steps, environment, data, setup,
precondition, post-condition and expected result information
that appears in a TC.
- There are no product level user stories and acceptance test
- Important business scenarios (E2E) that traverse over several
functional areas/applications are not captured in any user
stories
- Negative data/operation/flow/browser/functional scenarios are
not captured in any user stories.
- Non-functional test scenarios are not covered in any user
stories.
 Solution: Create appropriate test-only spikes/USs and include them
on the Scrum board during the Sprint Planning meeting. It is very
helpful to have a traceability matrix/page added to TCs mapping US to
TCs.
AC: Acceptance Criteria E2E: End-to-End TC: Test Case US: User Story

P#11: Falling behind in testing USs
 Problem: Members of the scrum team at times fall
behind in coding or testing USs due to various reasons
including incorrect estimation, unexpected issues, late
availability of many USs for testing, or interrupting
urgent issues that take scrum members’ time and
bandwidth.
 Solution: For non-urgent US, if you cannot complete a
US, complete it in the following sprint. However, in
most cases, less busier members of the scrum team
should pitch-in and help in coding or testing of the USs
which are at risk of not being completed. Remember
that the essence of Scrum is a cross-functional team
who is willing and capable of taking different roles and
helping members whenever needed.
US: User Story

Questions and Answers
 If you have any questions, please pose it now.

QA Challenges in an Agile World

More Related Content

What's hot (20)

Similar to QA Challenges in an Agile World (20)

QA Challenges in an Agile World