XBOSoft, Inc. All Rights Reserved. 1
Are You Making These 7 Metric Mistakes?And What to Do If You Are
with Mark Bentsen, ARGO Data
#XBOSoftWebinar
#TestingMetricMistakes
XBOSoftDedicated to software quality
improvementFounded in 2006
We speed products to market with our expert:
• Software QA consulting• Software testing
Global team with offices in San Francisco, Beijing and Amsterdam
“Thorough, accurate and fast”XBOSoft, Inc. All Rights Reserved. 2
House Rules Participants other than the speakers are muted Ask questions in the GoToWebinar control on the
R-side of your screen or through Twitter @XBOSoft Questions may be asked throughout the webinar -
we’ll try to answer them at the end You’ll receive info on recording after the webinar
XBOSoft, Inc. All Rights Reserved. 3
Event hashtags: #XBOSoftWebinar #TestingMetricMistakes
Meet Our Speakers
Philip LewCEO and Founder, XBOSoft• Relevant specialties and
passionso Software quality process,
evaluation, measurement and improvement
o Software quality in use / UX design
o Mobile User Experience and usability
o Cycling and travel
XBOSoft, Inc. All Rights Reserved. 4
Rachel GeibigMarketing Manager, XBOSoft• Emails you about our
industry hot topic webinars
• Invites you to download our latest trends in software testing white papers
XBOSoft Team
Guest Presenter
Mark Bentsen
XBOSoft, Inc. All Rights Reserved. 5
• Quality Assurance Manager, ARGO DataCTAL-Full, CSTE, PMP, ASQ CMQ/OE
• Leading cross-enterprise collaboration in the DFW area among Software Quality practitioners for the past seven years. Gathering corporately on a quarterly basis and on field trips to one another’s work places, the QA Trailblazers are pushing each other to expand the capabilities of the modern software testing organization.
• He is the QA Manager of ARGO Data, a software development company providing mission-critical and analytical solutions for financial services and healthcare.
• Mark & his wife Melissa are the President Couple of Better Marriages Texas and have been active in Marriage Enrichment since they said “I do” in 2001. Prior to working in technology, he worked in YWAM & Mercy Ships in Switzerland and Namibia.
• Contact: [email protected]
Agenda
XBOSoft, Inc. All Rights Reserved. 6
1. Why We’re Here2. 7 Metrics Mistakes3. What you can do about it!
Is There Ever TMI In Testing?
XBOSoft, Inc. All Rights Reserved. 7
• No• But There Is TMD… Too Much Data• Reporting too much data and not enough
information
What do we do?
XBOSoft, Inc. All Rights Reserved. 8
• Reduce Risk & Eliminate Waste
• Effective software testing teams:– Build confidence– Reduce “Risk & Surprises”– Detect defects early– Provide valuable information
What do we do?
XBOSoft, Inc. All Rights Reserved. 9
• Reduce Risk & Eliminate Waste
• Effective Quality Assurance teams:– Identify risks– Prevent defects– Focus on continuous improvement of SDLC quality– Guard the company brand
STEP ZEROCommunication is what the LISTENER does
XBOSoft, Inc. All Rights Reserved. 10
Communication Heuristics
XBOSoft, Inc. All Rights Reserved. 11
• Misconception is that status and metrics only go out in email.
• If you depend on email, you have one “where” and one “how” in your communication tool belt. There are a lot of other tools available to the wise test manager.
METRIC MISTAKES7
XBOSoft, Inc. All Rights Reserved. 12
1-Flash In The Pan
XBOSoft, Inc. All Rights Reserved. 13
Too ambitious in the beginning
• Symptom: Having trouble keeping metrics current and reported when they are needed.
• Principle: Slow is fast. • Solution: Make a sustainable decision up
front.• Then: Steadily report meaningful information.
2-Heavy Lifting
XBOSoft, Inc. All Rights Reserved. 14
Choosing to work hard instead of smart
• Symptom: Overly manual processes required to produce.
• Principle: Maintainability is a key quality attribute and a success criteria for process success.
• Solution: Build in telemetry.• Then: Automate, or automation assist.
3-Message In A Bottle
XBOSoft, Inc. All Rights Reserved. 15
Communicating in a vacuum, throwing the bottle into the ether and expecting the best
• Symptom: No one responds to status reports from testing.
• Principle: Communication is what the listener (recipient) does.
3-Message In A Bottle
XBOSoft, Inc. All Rights Reserved. 16
Communicating in a vacuum, throwing the bottle into the ether and expecting the best
• Solution: Know your audience. Who they are and what they care about.
• Then: Answer the questions they have, and address their concerns.
4-House on Fire
XBOSoft, Inc. All Rights Reserved. 17
Ongoing state of emergency
• Symptom: Poor reporting and insight into QA processes through reporting.
• Principle: Reporting should be part of every testing process.
4-House on Fire
XBOSoft, Inc. All Rights Reserved. 18
Ongoing state of emergency
• Solution: Validate you have meaningful metrics for each stage of your testing processes leading into a milestone.
• Then: Remove the mystery from what is taking place.
5-Mine Goes To Eleven
XBOSoft, Inc. All Rights Reserved. 19
Reporting is too complicated & cumbersome
• Symptom: No one understands the metrics or the implications, including the Test Analyst sending it out.
• Principle: Keep it simple.
5-Mine Goes To Eleven
XBOSoft, Inc. All Rights Reserved. 20
Reporting is too complicated & cumbersome
• Solution: Choose to be pragmatic over trying to be impressive.
• Then: Be consistent & crystal clear in your narratives.
6-Fish Out Of Water
XBOSoft, Inc. All Rights Reserved. 21
Answering the wrong questions
• Symptom: Suffering from “This is the way we always do it” mentality.
• Principle: Know your audience.
6-Fish Out Of Water
XBOSoft, Inc. All Rights Reserved. 22
Answering the wrong questions
• Solution: Answer questions that should be asked and avoid the status quo.
• Then: Follow-up in person with key stakeholders and adjust.
7-Rearview Mirror Driver
XBOSoft, Inc. All Rights Reserved. 23
Haphazardly moving forward without knowing what to expect in the days/weeks ahead.
• Symptom: You are spending all your time trying to find defects instead of preventing them.
• Principle: An ounce of prevention is worth a pound of cure.
7-Rearview Mirror Driver
XBOSoft, Inc. All Rights Reserved. 24
Haphazardly moving forward without knowing what to expect in the days/weeks ahead.
• Solution: Predict what your metrics will be in the next two weeks. Document it.
• Then: Compare reality with predictions. Adjust until forecasting accurately.
AND NOWWhat to do about it…
XBOSoft, Inc. All Rights Reserved. 25
Test Management Trifecta
XBOSoft, Inc. All Rights Reserved. 26
• What have you completed?• What did you learn?• What remains?
Tactical versus Strategic
XBOSoft, Inc. All Rights Reserved. 27
• Tactical– Standardize metrics across teams
• Strategic– Align to KPIs– YoY performance– Enterprise Quality Roadmap
BLUF
XBOSoft, Inc. All Rights Reserved. 28
• Bottom Line Up Front• Follow with a “Headline”• Impact to the triple constraint?
Less is More
XBOSoft, Inc. All Rights Reserved. 29
"I would have written you a shorter letter if I had more time."
- Blaise Pascal
Post your webinar questions on Twitter @XBOSoft
Registrants will receive an email with information on where to view the recording and slides from today’s webinar.
Join us to keep updated on all our webinars, reports and white papers:
facebook.com/xbosoft
+xbosoft
linkedin.com/company/xbosoft
Check out our blog: http://xbosoft.com/software-quality-blog/
Download our free white papers: http://xbosoft.com/knowledge-center/
Contact us with ideas for future webinars or questions regarding our services! [email protected]
Thank you!
Q+A
www.xbosoft.com