Stefaan Luckermans - Number for Passion, Passion for Numbers - EuroSTAR 2010

transcript

Numbers for Passion

Passion for Numbers

Stefaan Luckermans

Owner of

Father of Jordy and Ndu Husband of Thabile

Content Table

1. Rationale : Why metrics?

2. Resistance : Why metrics raise resistance?

3. Passion : Why metrics are linked to passion?

4. Support : Why metrics are accepted?

5. Benefit : Why are metrics in demand?

Rationale: Why metrics?

Management has set 3 priorities:

– More quality– Better planning of software development activities– Better/Proof of traceability that testing covers the requirements

Management likes balanced scorecards, KPI’s, etc.

Management wants to see one ‘score’ even for more complex matters. E.g. traceability is 94%.

Rationale: Why passion?

Management’s view on their testers:

– They are professionals/experts– Their intention is to do their best `– They know what to do– They are individualists but function well in a team– They want quality

Management says:

– ‘Our testers are passionate about their job’

Rationale: Why passion?

Management’s expectations:

– Testers must be professionals/experts– It is common sense that they do their best `– They must know what to do– They work for one company– It is normal they want quality

– Our testers must be passionate

Rationale: Why passion and numbers?

Management wants a number to confirm what they think, expect and say

Management wants a number to point out to them possible issues, even before these

issues become problems

Management wants a number to point out to them possible issues, even before these

issues become problems

Content Table

Resistance?

What is it with numbers ?

I like numbers why don’t you like numbers ?

Numbers mean different things to different people?

The psycho(logy) of numbers

Psychology of numbers?

Sentiment ?

Feeling ?

Reaction ?

Sentiment ?

Feeling ?

Reaction ?

TEST RESULTS

First of the class,

My parents will be pleased

Good enough for

me, but my

parents…

I did study,

Just don’t

understand it.

The teacher must

think I am stupid !!

I studied and did

better than last

time (40%).

I know the teacher

is proud of me !!

35%I did study,Just don’t

understand it.The teacher must

35%Does not reflect the

effort I put in studying

35%Why should I do the effort to study, it has

no result !!

35%I am not good in

doing tests !!

35%She will not let me

pass !!

35%Is reason enough to

stop studying

no result !!

35%I am not good in

doing tests !!

pass !!

stop studying

Content Table

NUMBERS ONLY RESULT IN RESISTANCE IF THE PEOPLE ARE PASSIONATE OF WHAT

THEY ARE DOING

no result !!

35%I am not good in

doing tests !!

pass !!

stop studying

Franklin D. Roosevelt

Men and women are

not prisoners of fate,

but only prisoners of

their own minds.

Why resistance?

Men and women are not

prisoners of fate, but only

prisoners of their own minds.

Why resistance?

But How can we support

them to change their

minds?

Men and women are not

prisoners of fate, but only

prisoners of their own minds.

Why resistance?

But How can we support

them to change their

minds?

Content Table

Support

Management wants evaluation

– Could be perceived as an accusation

Passionate testers want/need support

– If there was a way to find out themselves before management knew

Elias Canetti

People love as self-

recognition what

they hate as an

accusation

Elias Canetti (Bulgarian: Елиас Канети; 25 July 1905–14 August 1994) was a Bulgarian-born novelist and non-fiction writer of Sephardi Jewish

ancestry who wrote in German. He won the Nobel Prize in Literature in 1981

Support

Management wants evaluation

– Could be perceived as an accusation

Passionate testers /need support

– If there was a way to find out themselves before management knew

Why Tool?

Impersonal

Uniform

Best practices

Track history

Evidence

Plenty of triggers to make it a success

Triggers

POSITIVE

NEGATIVE

Triggers

POSITIVE

NEGATIVE

Just like Pavlov’s dogs, also

humans are conditioned to

react in a specific way on

certain signals, colors,

sounds, symbols, etc.

Excel tool

Traceability from test design to test cases to test execution

Traceability from defects

Test coverage of test design and test cases

Defect recognition/ Defect resolution

Based on one of the three priorities set by management

Traceability

Test setsTest execution

TDS can be traceable to multiple Test sets

Test sets can be traceable to multiple TDS

TDS can be

traceable to

multiple

TCS can be

traceable to

multiple

TCS can be traceable to nothing (TCS not

covering TDS and/or not used in a test set

TDS can be traceable to nothing (TDS not yet covered)

Test set can be

traceable to

multiple TCS

TCS can be

traceable to

multiple

Test sets

Traceability

Test sets

can be

traceable to

nothing

(empty test

Test setsTest execution

TDS can be

traceable to

multiple

TCS can be

traceable to

multiple

MA_1_1

MA_1_3

TDS can be traceable to multiple Test sets

Test sets can be traceable to multiple TDS

MA_1_2 MA_1_14

TCS can be traceable to nothing (TCS not

covering TDS and/or not used in a test set

MA_1_4

MA_1_10

TDS can be traceable to nothing (TDS not yet covered)

Test sets

can be

traceable to

nothing

(empty test

MA_1_11

Test set can

be traceable

to multiple

TCS can be

traceable to

multiple

Test sets

MA_1_5

MA_1_6 MA_1_7

MA_1_8MA_1_9

MA_1_12

MA_1_13

MA_1_14

Traceability

MA_1_2

MA_1_1

MA_1_3

MA_1_4

MA_1_5 MA_1_7 MA_1_8

MA_1_11

MA_1_12 MA_1_13MA_1_14

MA_1_14

Testers

Management

Excel assessment tool, only 5 numbers

Excel (self) assessment tool, 5

groups, 43 numbers

Excel self assessment tool, 5 groups, 43 numbers, Training

material, organization wide rollout

Content Table

Management

Have a SPOC to get a report + analysis

Think this tool is good enough

– It pushes people to work in a uniform way

– It allows to compare projects

– It allows to see progress over time

– It allows to see improvement over time

Test managers

Must include final evidence in test summary report

Can use the tool when they want

Have a SPOC when they encounter problems

Think this tool is not (good) enough

– Simplification – less numbers

– Not enough – more numbers

– Buttons for more support -> improvement lists

Test managers

Must include final evidence in test summary report

Can use the tool when they want

Have a SPOC when they encounter problems

Think this tool is not (good) enough

– Simplification – less numbers

– Not enough – more numbers

– Buttons for more support -> improvement lists

TOOL IS ALIVE

NO RESISTANCE

LOTS OF FEEDBACK

Feedback vs. Resistance

Feedback: use it, but

Traceability to bus. Requirements

Info on consistency of a test stage (test level)

Improvement lists

Changing baselines

Scope creep

Resistance : can’t use because

My project has no test stages(test levels)

I have no test design, nor test plan

Test cases in excel

Feedback vs. Resistance

Feedback: use it, but

Traceability to bus. Requirements

Info on consistency of a test stage (test level)

Improvement lists

Changing baselines

Scope creep

Resistance : can’t use because

My project has no test stages(test levels)

I have no test design, nor test plan

Test cases in excel

Training

What came out of it?

Excel – ms access – (.net, sql server)

Improvement lists are a fact

Data consistency check per test level

Traceability check from and to business requirements

(Baseline consistency check)

What came out of it?

Excel – ms access – (.net, sql server)

Improvement lists are a fact

Data consistency check per test level

Traceability check from and to business requirements

(Baseline consistency check)

Conclusion

General usable

Worth a try

Is free

Future 4 prototypes are ready

Documentation is being developed

Development of final tool has to start

Partners are needed/wanted

Open source : www.sweetthaste.org

Versions:

– In the cloud

– Desktop application single user

– Web application multi user

– Mobile : iPhone and iPad

Stefaan Luckermans - Number for Passion, Passion for Numbers - EuroSTAR 2010

Technology