Heuristics, mnemonics and other Greek words in the exploratory testing of mobile apps

transcript

Software quality assurance days20 Международная конференция по вопросам качества ПОsqadays.com

Минск. 24–26 ноября 2016

Владислав РоманенкоIntellectsoft. Минск, Беларусь

Эвристики, мнемоники и другие греческие слова в исследовательском тестировании мобильных приложений

Тестирую мобильные и веб приложения

ISTQB Certified Test Analyst

QA-инженер в Intellectsoft

laechoppe@gmail.com

@laechoppe

Reasons forExperience-based testing:

- lack of time - lack of documentation - issues could hardly be located using more systematic approaches

Experience-based techniquesError guessing

Checklist-based testing

Exploratory testing

Error guessing

Checklist-based testing Checklist-based testing

Exploratory testing- tests are designed and executed concurrently- usually guided by a charter- lightweight documentation (if any)

+ more interesting+ more apparent bugs first+ good if time is limited+ works if docs are absent+ more checks due to variations of tests+ ….

Exploratory testing

- less experienced testers won't get as much as more experienced

- could be hard to reproduce the issues- not suitable if requirements are mandatory- hard to collect metrics- …

Heuristic - a set of research methods to facilitate the discovery of a previously unknown

Mnemonic - a learning technique, allowing to remember the sequence and structure of something.

Richard Of York Gave Battle In Vained

Mnemonic

Not Every Cat Eats Sardines (Some Are Really Yummy)

MnemonicNotEveryCatEatsSardines(SomeAreReallyYummy)

Goldilocks Principle - in any dataset there’s always a U-shaped distribution

Goldilocks Principletoo short text too long text just right text

SFDPOT (San Francisco Depot)

S StructureF FunctionD DataP PlatformO OperationsT Time

What files does it have? Can I download the app?Can I download an update?Does the app write to an SD card? What if the card is filled, removed or changed?

SFDPOT

Does the app perform the tasks or features it was designed to?Does the app or site prevent, block or not include features not intended on a mobile device?Does the app direct me to turn on necessary related services?

SFDPOT

What kinds of input does it process?What does its output look like?Does it come packaged with preset dataDoes the app integrate with any hardware such as bar code scanners for payment or other purposes?

SFDPOT

What operating systems does it run on?Does the environment have to be configured in any special way? Does it depend on third-party components?Does the app depend on any device settings?

SFDPOT

Where and how the app will be used?Does the app function when I am moving and traveling around (switch wi-fi <-> 3G)?What happens when a mobile interruption occurs such as an incoming call or text message?

SFDPOT

What happens if the time zone is switched?What happens when my location is switched?Does the device auto detect a change in time zones and the day and date?Does the app find time-related data based on my device time?

SFDPOT

SFDPOTIS StructureF FunctionD DataP PlatformO OperationsT TimeI Interface

Does the app work well on phones and tablets?Does the app work well when used with “one hand?”Is the mobile version consist or complementary to the desktop version?Is the app consistent with UI current constructs in the marketplace?

SFDPOTIS StructureF FunctionD DataP PlatformO OperationsT TimeI Interface

SFDPOTIES StructureF FunctionD DataP PlatformO OperationsT TimeI InterfaceE Errors

Does the app continue gracefully after an error?Is the message helpful, instructive?If a user provokes multiple errors in succession, does the app continue?Can I get to an error state that forces the app to shut down?What happens when I restart the crashed app – is previous data or state saved?Is there a method to send an error report?

SFDPOTIES StructureF FunctionD DataP PlatformO OperationsT TimeI InterfaceE Errors

I SLICED UP FUNI InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network

I InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network

I SLICED UP FUN

Start from

I SLICED UP FUN

Heuristics help to:

find out what to start testing fromremember the context of app usageavoid repeating the mistakes…

TO SUM UP:

эвристикамнемоникатехникатеориядиапазонсистемныйпараллельнометодсимвол

идеяэргономикателефонбиблиотекаграфикмикрофонпараметрриск…

Some of the Greek words:

http://www.rockynook.com/press-release-software-test-engineers-handbook-2nd-edition/

http://www.computing.dcu.ie/~ray/teaching/CA358/dorothy_graham.pdf

http://karennicolejohnson.com/2012/05/applying-the-sfdpot-heuristic-to-mobile-testing/

http://karennicolejohnson.com/2012/12/applying-the-sfdpot-heuristic-to-mobile-testing-adding-i/

http://karennicolejohnson.com/wp-content/uploads/2012/11/KNJohnson-2012-heuristics-mnemonics.pdf

https://www.utest.com/articles/rotation-tour-in-mobile-testing

http://www.kohl.ca/articles/ISLICEDUPFUN.pdf

http://www.satisfice.com/articles/sfdpo.shtml

http://michaeldkelly.com/blog/2005/9/20/touring-heuristic.html

https://goritskov.com/blog/heuristics-in-testing.html

Pictures from: mycutegraphics.com

References

laechoppe@gmail.com

@laechoppe

Heuristics, mnemonics and other Greek words in the exploratory testing of mobile apps

Technology