Reporting principles for every QA manager

transcript

Joel MontveliskyChief Solution

Architect

Simplify Test Management

PractiTest Webinar:Reporting Principles for Every QA Manager

What do we want to cover today?

• DDTM – Decision Driven Test Management

• The need for repetition & multiple reporting channels

• Numberless reporting

• Do’s & Don’ts for successful reporting

• Questions and AnswersSimplify Test Management

A little about me…Testing:

- Tester- QA Manager- Blogger- Speaker- Solution

Architect

Some numbers:- 1 wife- 3 kids- 1 dog & 2 fish- 18 years testing

PractiTest- Chief Solution Architect- Test Manager

About SaaS ALM Solution Simple & Intuitive – yet – Methodological & Flexible Integrates with Jira, Selenium, Pivotal Tracker, Jenkins,

Redmine, Bugzilla, QTP, Ranorex, SoapUI, etc… Best Darn Support in the Business

DDTMDecision-Driven

Test Management

Why Do We Test?

To provide our stakeholders with visibility

into the status of the product and process,

so they can make the correct decisions.

DDTM Decision-Driven Test ManagementPlanning and running your tests based on the information needed by your stakeholders to make their decisions.

DDTM –Reverse Engineering your Testing Process:

1. Identify your stakeholders and their decisions

2. Understand the information needed to answer questions

3. Plan your reports, metrics and statistics

4. Define the tests to capture data

5. Run your tests based on timelines required for data capturing

6. Continuously monitor for new stakeholders & their changing needs

A Word of Caution About Stakeholders’ Decisions

Stakeholders come in many types & sizes.

Decisions tend to happen at the beginning or the end of the project.

Decisions are not only based on user stories or requirements.

Each stakeholder will make different

decisions, at different times,

based on different information.

The Need for Repetition&

Multiple Channels

Question:

Which of the following reporting channels do you use?

Web-based Dashboards Live Monitors High or Low Level Testing Reports Emails with Status Updates Update Meetings Periodic Management Presentations Other

Need for Repetition(Theory of Effective Frequency)

Herbert E. Krugman (GE 1960’s):

“People need to be exposed 3 times to act on an

idea.”

1st time: What is it? (Curiosity)

2nd time: What of it? (Recognition)

3rd time: Take action. (Decision)

Using Multiple Channels

People absorb information differently:

Graphically Textually Statistically Comparatively

People access information via multiple media:

Computer Phone Verbally Presentations

It is more effective to combine multiplechannels & formats to communicate information

And so...In the case of Reports, More is actually Better.

More Repetitions x More Formats =

Better Results!!

Numberless Reporting

Question:

Do you use exact numbers as part of your reports?

e.g. Number of Tests, Number of Bugs, etc

ReminderDon’t make

me think!Cheers,

The Boss

If you already know what interests your stakeholders, why not provide them with the processed information?

Numberless Reporting - example

Version 7.2 Status Overview:

Feature A.1This

week Last

Feature B.2Feature C.3Feature D.4Platform E.5Platform F.6Stability & Load G.7

Overall project projectionStabilityNew features

Version Projection

Main risks Major delays on feature D.4 Lack of stability in Feature

C.3 4 Team members moving to

project XYZ next week

Color code the main areas (features?) of your project

List the most important things to know

Provide projections

Make sure people know where things are good and where they are not good

Be ready to provide numbers to back your numberless conclusions.

Reporting Do’s & Don’ts

Do 1: Always start from the bottom line then you can explain

Place the most important information on the top and center.

Add data to support your conclusions, but always at the end.

You telling the News, and not writing a Novel!

Do 2: Work S.M.A.R.T.

Simple

Measurable

Actionable

Repeatable

Timely

Do 3: Trust your gut

Do 4: Focus

Express your feelings even if you sometimes lack hard data to back them up.

Be concrete and focused on what you are trying to say.

Don’t beat around the bush.

Don’t 1: Don’t fall in love with your data

Don’t 2: Don’t be afraid to correct yourself

Don’t 3: Don’t be afraid to say I don’t know(and get back to people with the answer later)

Questions & Answers

About SaaS ALM Solution Simple & Intuitive – yet – Methodological & Flexible Integrates with Jira, Selenium, Jenkins, Redmine, Bugzilla,

QTP, Ranorex, SoapUI, etc… Best Darn Support in the Business

Thank You!joel@practitest.com

http://qablog.practitest.com

Reporting principles for every QA manager

Software