How i Test (Failure) - Ben Simo

8/4/2019 How i Test (Failure) - Ben Simo

1/178


2/178


3/178

prepare to readfast

v1.2 FAILURE.QualityFrog.com

Now that I have your attention,


4/178


Youre testing

And an error message appears


5/178



6/178


Did you expect it?

Or are you surprised?


7/178


How does it make you feel?


8/178


What do you do?


9/178


Not ready reading drive A

Abort, Retry, Ignore, Fail?


10/178


Continue to the next test step


11/178

v1.2

FAILURE.QualityFrog.com


12/178

v1.2


Report a bug


13/178

v1.2



14/178

v1.2


Resolve unknown conflict


15/178

v1.2



16/178

v1.2


Find a file


17/178

v1.2



18/178

v1.2


Eliminate something


19/178

v1.2



20/178

v1.2


Request permission


21/178

v1.2



22/178

v1.2


Try again


23/178

v1.2



24/178

v1.2


Try something new


25/178

v1.2



26/178

v1.2


Seek help


27/178

v1.2



28/178

v1.2


Set a reference to an instance


29/178

v1.2



30/178

v1.2


lick OKC


31/178

v1.2


lick OK


32/178

v1.2



33/178

v1.2


Save


34/178

v1.2



35/178

v1.2


Enter a multiple of |ECL0


36/178

v1.2



37/178

v1.2


Exit


38/178

v1.2



39/178

v1.2


Reconnect


40/178

v1.2



41/178

v1.2


Declare success


42/178

v1.2



43/178

v1.2


Assign blame


44/178

v1.2



45/178

v1.2


Laugh


46/178

v1.2



47/178

v1.2


Restart


48/178

v1.2



49/178

v1.2


Guess


50/178

v1.2



51/178

v1.2


Gasp


52/178

v1.2



53/178


Request a raise


54/178



55/178


Swear


56/178



57/178


So


58/178


What do you do

when you encounter an error?


59/178


60/178


Oh, who am I?


61/178


I am Ben Simo

[email protected]


62/178


I use software

[email protected]


63/178


I test software

[email protected]


64/178


I question software

[email protected]

www.QuestioningSoftware.com


65/178


I code software

[email protected]



66/178


I code software

that helps test software

[email protected]



67/178


I talk about software

[email protected]



68/178


I write about software

[email protected]



69/178


I think about software

[email protected]



70/178


I like software


71/178


I like softwarethat serves people


72/178


I like softwarethat helps people


73/178


I dislike bad software


74/178


I dislike bad softwarethat frustrates people


75/178


I dislike bad softwarethat confuses people


76/178


I dislike bad softwarethat angers people


77/178


I dislike bad softwarethat discombobulates people


78/178


I sometimes like big words thatdiscombobulate people


79/178


But only for a moment


80/178


Discombobulate means toemotionally confuse


81/178


Discombobulate means tofrustrate


82/178


Discombobulate means tobewilder


83/178


84/178


Discombobulate means tobefuddle


85/178


86/178


Too many error messages

discombobulate people


87/178


Error messages should help people,not confuse them


88/178


You see,

Its not about the software


89/178


90/178


91/178


They fail to think abouthow they affect people


92/178


So what do I do when

I encounter error messages?


93/178


I test them


94/178


I test themusing the FAILURE mnemonic


95/178


96/178


97/178


98/178


The FAILURE mnemonics

guideword heuristics help me thinkabout testing error messages


99/178


100/178


I ask Functional questions like

Does the error detection

function as expected?


101/178


102/178


103/178


104/178


105/178


106/178


107/178



Do dialog buttons work?


108/178


109/178



Is it functional?


110/178


Only I dont stop at Functional


111/178


112/178


113/178


114/178


I ask Appropriate questions like

Is the error reporting accurate?


115/178



Are UI messages free of

inappropriate technical detail?


116/178



Is there a better way to handle

or report the error?


117/178



Is the error user friendly?


118/178


119/178


120/178


121/178



Is a user allowed to waste time

and effort only to be told thattheir work cannot be applied?


122/178


The I reminds me to ask

about the Impact of the error

FAILURE


123/178


I ask Impact questions like

Is the impact communicated?


124/178



Is the impact communicated

to the user?


125/178



Is the impact communicated

to others?


126/178


127/178



Is any impact to the users data

communicated?


128/178


129/178



Was a transaction only partially

completed?


130/178


131/178


I ask Log questions like

Should the error be logged for

support?


132/178




system administrators?


133/178




developers?


134/178




technical investigation?


135/178



Will log information beavailable if the user waits to

report a problem?


136/178



Are log messages standardizedto support automatedinformation mining?


137/178



Can I grep this error?


138/178



Do logs contain enough

information?


139/178



Do logs contain useless

information?


140/178



Is this an error that should not

be logged?


141/178



Is there too much logging?


142/178



Does excessive logging impact

performance?


143/178



Does excessive logging impact

disk space?


144/178



Does the logging help or

complicate error investigation?


145/178


The U reminds me to think about

the User Interface

FAILURE


146/178


I ask UI questions like

Is the error reported to the

user?


147/178



Does this error need to be

reported to the user?


148/178



What user activitycaused the failure?


149/178



What user activitycaused the failure?

Is this communicated to the user?


150/178



Who is the user?


151/178



Are UI messages worded for

the intended audience?


152/178



Is this message consistent with

the desired look and feel?


153/178



Is the UI reporting consistent

with reporting for other errors?


154/178



Is the UI reporting efficient?


155/178



Does the user need to click awayexcessive dialogs?


156/178



Is this the best way tocommunicate the error?


157/178



Could this error be handled

without involving the user?


158/178


The R helps me think about

error Recovery

FAILURE


159/178


I ask Recovery questions like

Is the user told how to recover

from the impact of the error?


160/178



Does the user need to contact

support?


161/178



Is support contact information

provided?


162/178



Does support know how to fix

the problem?


163/178



Does the software guide theuser through the recovery

process?


164/178



Or is the user left to figure it

out on their own?


165/178


The E leads me to think about

the Emotions of users

FAILURE


166/178


I ask Emotions questions like

What emotions are likely to be

raised by the error message?


167/178



Does the error reporting evoke

the desired user emotions?


168/178



Does this message enhance

user frustration?


169/178



Does this message quiet

user frustration?


170/178



Does this message encourageusers to use some other

product?


171/178



Does this message cause

confusion?


172/178


Asking these types of questions can

help turn FAILURE into success


173/178


The next time

you encounter an error message


174/178


The next time

you encounter an error messageTest it!


175/178


The next time

you encounter an error messageQuestion it!

F


176/178


Try the

FAILUREmnemonic

Functional

Appropriate

Impact

LogUI

Recovery

Emotions

F


177/178


And let me knowhow it works for

you

Functional

Appropriate

Impact

LogUI

Recovery

Emotions

F


178/178

Ben Simo

[email protected]

http://FAILURE.QualityFrog.com

Functional

Appropriate

Impact

LogUI

Recovery

Emotions

Date post:	07-Apr-2018
Category:	Documents
Upload:	varaprasad119
View:	223 times
Download:	0 times

How i Test (Failure) - Ben Simo

Documents