Post on 23-Aug-2020
transcript
BIO PRESENTATION
BETTER SOFTWARE CONFERENCE & EXPO 2005 September 22, 2005
Hyatt Regency San Francisco Airport San Francisco, California, USA
W15
September 21, 2005 3:00 PM
AGILE QA - AN OXYMORON?
Elisabeth Hendrickson Quality Tree Software, Inc.
Elisabeth Hendrickson Elisabeth Hendrickson is an independent consultant who specializes in software testing, quality assurance, and management. With more than twelve years of experience working with leading software companies, she is the founder of Quality Tree Software, Inc. An award-winning author, Elisabeth has published more than twenty articles and is a frequently invited speaker at major software conferences.
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 1
Agile QA—An Oxymoron?
Elisabeth HendricksonQuality Tree Software, Inc.
www.qualitytree.com
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 2
What Does “Agile” Mean?
Agile is more than a buzzword. It is a relentless focus on providing a continuous stream of business value, even in the face of constant change.– See the Agile Manifesto:
http://www.agilemanifesto.org– And the Agile Alliance:
http://www.agilealliance.org– And the Declaration of Interdependence
http://www.pmdoi.org/– And the Agile Project Leadership Network
http://www.apln.org
Agility is usually achieved by adopting one or more Agile methodologies.
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 3
Examples of Agile Methodologies
LeanLean manufacturing concepts applied to software development.
ScrumLightweight management framework.
CrystalLightweight set of development practices.
Extreme Programming (XP)
Rigorous set of practices designed to keep both the code and team agile.
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 4
Agile Synthesized
• Communication and collaboration
• Visible indicators• Disciplined development
practices
• Feedback• Whole team thinking• Short iterations• Low overhead, high
productivity
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 5
Calling It “Agile” Doesn’t Make It So
This is NOT Agile:
1. Compress the schedule2. Toss out the
documentation3. Code up to the last minute
The organization may gain short term speed but at the cost of long term pain.
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 6
How Traditional QA Practices Evolved
Analyze Design Code Test/Bug Fix
With great optimism and the best of intentions, The Project Plan is announced:
ReleaseRequirements handed off to Dev
Completed Code handed off to QA
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 7
How Traditional QA Practices Evolved
Analyze, Design, & Code Test/Bug Fix
Inevitably, The Project Plan is revised:
ReleaseCompleted Code handed off to Test
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 8
The Result: Practices Intended to Control ChaosTraditional QA practices attempt to manage the chaos (or at least avoid the blame):• “Last Defender of Quality” stance• Strict change management • Detailed preparation and up front planning• Heavyweight documentation suitable for outsourcing
the test effort• Strict entrance and exit criteria with signoffs• Heavyweight test automation focused on regression• Attempts at process enforcement
These practices are anti-Agile. Can QA adapt in an Agile environment?
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 9
Becoming Agile: Shifting Roles
“Fear not! I’ll protect
you!”
“Hey, would this help?
from last line of defense… …to team support
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 10
Necessary Conditions for Agile QA
1. The organization is willing to embrace agility as defined by the Agile Manifesto.Saying “Be More Agile” or “Go Faster” isn’t enough.
2. The whole team is responsible for quality, not just the testers or people with “Quality” in their title.Which are you more likely to hear: “How did you miss that bug?!?” or “How did we not catch that?”
3. Everyone tests, not just designated testers.Agile teams are “test infected.”
4. Managers focus on fixing problems, not blame.Agile practices don’t provide CYA paper trails and are unlikely to succeed in a high-blame, high-fear environment.
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 11
Becoming Agile: Two Key Principles
Increase the rate of feedback
Reduce waste
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 12
Traditional & Agile QA Contrasted
Get down from the pulpit and work with the team.
Create it, enforce it, bring out the Quality Stick if necessary.
Process
All levels, built by anyone, an integral part of the project.
System-level, built by tool specialists, created after the code is “done.”
Automation
It’s not a relay race. Collaborate.
Formal entrance and exit criteria with signoffs.
Handoffs
Only as much as absolutely necessary.
Verbose.Documentation
Plan as you go.Comprehensive up front design, including test design.
Planning
Accept it. Plan for it. Embrace it.
Manage & control it.Change
Agile QATraditional QA
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 13
Embrace Change: Plan for Maintainability
When creating test artifacts…
• Minimize duplication.Thought exercise: if a feature were removed from your software, how many artifacts would have to be updated?
• Use tools designed for change.Hint: if the vendor says “stabilize the interface first,” run away!
Chan
gePl
annin
gDo
cume
ntatio
nHa
ndoff
sAu
tomati
onPr
oces
s
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 14
Plan Ahead…But Not Too Far Ahead
A little planning is good. More is not better.• Plan for the current iteration.
Design tests for the features or stories to be done in the near term. Speculative planning means rework.
• Have a strategy that fits on one page.If it is still relevant in 6 months, it’s probably at the right level of detail.
• Keep an up-to-date, prioritized risks list.What kind of information are the testers looking for? The risks list covers it.
Chan
gePl
anni
ngDo
cume
ntatio
nHa
ndoff
sAu
tomati
onPr
oces
s
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 15
Favor Informal, Collaborative Tools
Databases
Gantt/PERT Charts
Comprehensive, Polished Documents Created from a Standard Template
Formal
Whiteboards
Sticky Notes
Index Cards
Wikis
InformalChan
gePl
anni
ngDo
cume
ntatio
nHa
ndoff
sAu
tomati
onPr
oces
s
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 16
Monitor Documentation Costs
How Much Does Documentation Cost?Informal polling of 162 software testers from 65 companies revealed that most spend more than 33% of their time documenting tests.
How Much Time Are We Spending on Test Documentation?
0510152025303540
0 10 20 30 40 50 60 70 80 90 100
% Time
# Te
ster
s R
epor
ting
Chan
gePl
annin
gDo
cum
enta
tion
Hand
offs
Autom
ation
Proc
ess
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 17
Keep the Documentation Simple
• Capture the essence, not the details.Step-by-step instructions cost time without providing value (usually).
• Point to other project documents.If it’s in the user guide, requirements, specs, etc., leave it there.
• Centralize generic tests in a checklist.Try this: count the number of times common condition, like invalid dates or null strings, are documented in the test docs. More than once is too many.
Chan
gePl
annin
gDo
cum
enta
tion
Hand
offs
Autom
ation
Proc
ess
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 18
Watch Verbose Wording
Which would you rather maintain?Choose the Import option from the File menu. A dialog titled “Import File” appears. Navigate to \\x\y\z\long.dat and click Open. A dialog titled “Importing…” appears with the current status, a progress bar, and a button labeled “Cancel.” Click on the Cancel button. Choose OK on the confirmation dialog. Verify that the import stops.
Or:Start a long import. Cancel it in the middle. Verify it stops.
Chan
gePl
annin
gDo
cum
enta
tion
Hand
offs
Autom
ation
Proc
ess
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 19
A Lightweight Example: WikisCh
ange
Plan
ning
Docu
men
tatio
nHa
ndoff
sAu
tomati
onPr
oces
s
Built in Twiki with the Table plugin installed. See http://www.twiki.org
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 20
Another Lightweight Example: Mind MapsCh
ange
Plan
ning
Docu
men
tatio
nHa
ndoff
sAu
tomati
onPr
oces
s
Built in MindManager. See http://www.mindjet.com
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 21
Reduce Handoffs: Integrate Test Efforts
Testing is not a phase. It’s a way of life.• Agile teams are test infected.
The question, “How will we test it?” is as important as “How will we build it?”
• Co-locate testers and programmers.But sitting side by side does not ensure communication.
• Track testing status and programming status all together.Show tests run-passed-failed together with features/stories done and left to do.
Chan
gePl
annin
gDo
cume
ntatio
nHa
ndof
fsAu
tomati
onPr
oces
s
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 22
Instruments of Agreement, not Instruments of Blame Ch
ange
Plan
ning
Docu
menta
tion
Hand
offs
Autom
ation
Proc
ess
• Hold end of iteration exploratory testing sessions to ensure shared understanding.
• Do a team victory dance whenever a feature/story is officially accepted.
Do away with the sign off sheet.
“OK , now everyone sign in blood.”
Instead, ritualize acceptance as a shared team activity.
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 23
Leverage Automation Investments
Automate everything you can, but invest wisely.• Collaborate with programmers on test
infrastructure code.The programmers have already automated the unit tests. Why not reuse the investment where possible?
• Use different types of automated tests for different purposes.Automated system tests should cover end-to-end sequences. Unit tests detect unintended change. Don’t substitute one for the other.
Chan
gePl
annin
gDo
cume
ntatio
nHa
ndoff
sAu
tom
atio
nPr
oces
s
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 24
Automation & Exploratory Testing
Use test automation to support earlyexploratory testing.
Traditional test wisdom says we can’t start testing a feature until it’s accessible from an external interface (like a GUI). But we don’t have to wait. Test automation can facilitate manual exploration.
Chan
gePl
annin
gDo
cume
ntatio
nHa
ndoff
sAu
tom
atio
nPr
oces
s
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 25
The Whole Team is Responsible for ProcessCh
ange
Plan
ning
Docu
menta
tion
Hand
offs
Autom
ation
Proc
ess
How Not To Improve The Process:
…furthermore, henceforth you shall
submit your unit tests for my review…
Don’t lecture or blame.Don’t rattle the Quality Process drums.
Instead, let the team take responsibility for process.And be a member of the team.
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 26
Agile Testers Wanted: A Job Description
Responsibilities:• Perform manual/exploratory tests on early-stage code• Create automated acceptance tests• Advise the team about overall risks and trends• Help the business stakeholders define acceptance criteria• Facilitate communication between the technical & business
stakeholdersQualifications:• Experience designing and executing tests• Scripting skills in at least one of the following languages:
Ruby, Perl, Python, or JavaScript. Experience programming in Java, C#, etc. a plus.
• Strong analysis skills• Ability to work in a team (bullpen) environment
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 27
QA Teams that Embrace Agility…
• Focus on providing value to our key stakeholders (both business-facing and technology-facing)
• Shift from being the last line of defense to providing an information service.
• Aggressively reduce time and resources spent on anything that does not directly contribute to providing information that helps move the project forward.
• Collaborate with programmers to improve testability and leverage test automation efforts.
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 28
Further Reading…
Beck, K. (1999). Extreme Programming Explained: Embrace Change. Addison-Wesley.
Cockburn, A. (2004). Crystal Clear: A Human-Powered Methodology for Small Teams. Addison-Wesley.
Crispin, L., & House, T. (2002). Testing Extreme Programming. Addison-Wesley.
Poppendieck, M. & Poppendieck, T. (2003). Lean Software Development. Addison-Wesley.
Schwaber, K. & Beedle, M. (2001). Agile Software Development with SCRUM. Prentice Hall.
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 29
Acknowledgements
Many thanks to early reviewers of the ideas presented here: Brian Marick, William Wake, Jonathan Kohl, Jeffrey Fredrick, Daniel Knierim, Marc Kellogg, Danny Faught, Ron Jeffries, Hubert Smits, Rob Mee, Sherry Erskine, Amy Jo Esser, Gunjan Doshi, Dave Liebreich, Janet Gregory, Chris McMahon