1

Hands-On Mobile App Testing

@dnlkntt

Who am I?

Daniel Knott

Senior Software Test Engineer Mobile

Working @ XING

Author, Blogger and Speaker

http://adventuresinqa.com

@dnlkntt

www.handsonmobileapptesting.com

Intro Beta Testing

Mobile Test Pyramid

Hands-On Mobile App Testing

F-Word

Numbers & Facts

{…}

Numbers & Facts

Numbers & Facts

* Running Android > 4.x** Running iOS 9

3+ Million Apps available in biggest app stores

100+ Billion App downloads

+24,000* Android Devices

15** iOS Devices

80% - 90% of apps are deleted after first usage

56% of apps that require a registration gets deleted

45% of apps are not well tested

Numbers & Facts

Average user checks device 150 times a day

User expect app loading times in 2 seconds

Mobile users have very high expectations

Users are on the move

Users have usually more than one device

Users are emotionally attached to apps

Users can leave bad reviews very quickly

Numbers & Facts

Simple UseableFastItKeep

K I F S U

F-Word

F-Word

http://opensignal.com/reports/2015/08/android-fragmentation/

F-Word

http://opensignal.com/reports/2015/08/android-fragmentation/

F-Word

Don‘t buy all devices

F-Word – Device Groups

Group 1, Prio A

• High End Devices• Dual/ Quad Core

CPU• RAM >2048MB• Retina, Full HD

Display >= 5“

Group 2, Prio B

• Mid-range devices• Dual Core CPU• RAM <=1024MB• Screen size < 5“• Software not older

1 year

Group 3, Prio C

• Small devices• Single Core• RAM < 512MB• Low screen size

and resolution• Older OS, older

browser

Based on your target group

F-Word – Open Device Labs

Rent devices for free

Shared community pool of smartphones, tablets or wearables

Donate devices to support the community

http://opendevicelab.com/

F-Word – Device Clouds

Use device clouds if special devices are required

Can be used for manual and automated testing

Lots of different provider on the market

Hands-On Mobile App Testing

Hands-On Mobile App Testing – Sample

Hands-On Mobile App Testing – Sample

Speed recording requires movement

Slope tracking requires movement

Current slope information require mobile data network connection

Connection on top of the mountain vs. In the valley

Offline behaviorMobile data roaming

Hands-On Mobile App Testing – Sample

Special clothes required for testing

Gloves

Goggles

Weather can have influence on device and running apps

Hands-On Mobile App Testing

Author: amateur_photo_bore, http://www.flickr.com/photos/andresv/233667770/

Author: Rain Rabbit, http://www.flickr.com/photos/37996583811@N01/8033259053/

Author, Daniel Sancho, http://www.flickr.com/photos/teclasorg/2852716491/

Hands-On Mobile App TestingAuthor: Surat Lozowick, http://www.flickr.com/photos/suratlozowick/4311105154/

Author: Surat Lozowick, http://www.flickr.com/photos/suratlozowick/4464587302/

Hands-On Mobile App Testing

Author: Theus Falcao, http://www.flickr.com/photos/theusfalcao/9563767145/

Android: $ adb shell monkey –p APP.PACKAGE.NAME –v 2000

iOS: UI AutoMonkey | Dixie | CrashMonkey

Hands-On Mobile App Testing

Author: minxlj, http://www.flickr.com/photos/minxlj/3270996158/

$ adb install –r YOUR_APP.apk Apple Configurator

Mobile Test Pyramid

Test Pyramid Manual Testing

Integration Tests

Unit Tests

UI Tests

Mobile Test Pyramid

End2End Tests(UI Tests)

Manual Tests

Beta Tests

Unit Tests

Beta Testing

Beta Testing

Try to release your app as early as possible

If possible use public beta testing with real users

It is easy to create your own beta testing channel

First try to establish beta testing within your company

Beta Testing – Android Approach

Google Play Developer Console offers stage rollout

Able to setup alpha and beta releases

4 ways to establish a beta release

1. Public Beta2. Closed Beta with Google Groups3. Closed Beta with Google+4. Closed Beta with eMails

Easy to integrate in the development lifecycle

Alpha releases can be used within the company

Beta Testing – Android Approach

Beta Testing – iOS Approach

iOS beta testing is not as easy as on Android

Beta testers must install TestFlight App to participate

Limited to 1,000 beta testers

Beta Tester can‘t join on their own. Invitation only

Beta App is valid for 30 days

Summary – Take Aways

Mobile users have high expectations - KIFSU

Group your devices | Device Labs | Cloud Testing

Test your app in the wild

Test the update process | Stress test your app

Keep hardware and sensors in mind

Remind the mobile test pyramid

Establish a beta testing program

Thank you for your attention

http://adventuresinqa.com

@dnlkntt

» Numbers:» http://offers2.compuware.com/rs/compuware/images/

Mobile_App_Survey_Report.pdf» www.gartner.com/newsroom/id/2665715

» F-Word:» http://opensignal.com/reports/2015/08/android-fragmentation/» http://testdroid.com/» https://testmunk.com/» https://saucelabs.com/» https://testobject.com/» https://aws.amazon.com/de/device-farm/

» Update Testing:» Apple Configurator:

https://itunes.apple.com/de/app/apple-configurator/id434433123?mt=12

Links & References

» Stress Testing:» https://github.com/jonathanpenn/ui-auto-monkey » https://github.com/Skyscanner/Dixie» https://github.com/mokemokechicken/CrashMonkey

» Pyramid: » https://stocksnap.io/photo/20D9C664B1

» Beta Testing:» https://support.google.com/googleplay/android-developer/answer/

3131213» https://developer.apple.com/library/prerelease/ios/documentation/Languag

esUtilities/Conceptual/iTunesConnect_Guide/Chapters/BetaTestingTheApp.html

» https://developer.apple.com/testflight/» Photos:

» https://stocksnap.io/

Links & References