About Me
TestObject GmbH Co-Founder I Sales
Mobile Quality Crew’s Co-Founder I Organizer Contact: @testobject I timo@testobject
Session
1) Mobile app testing: Why it is important
2) Challenges
3) Different testing approaches
4) Automation as key factor for success
Familiar?
48 % of users won‘t use an
app again after encountering a
major issue!
Source: Compuware Global Study Mobile Apps 2013
Better Find Bugs Before Users Do
Bad Quality = Less Revenue
Quality
Ratings
Downloads
Revenue
App Testing Challenges
Device Fragmentation
http://opensignal.com/reports/2015/08/android-fragmentation/
OS Fragmentation
http://opensignal.com/reports/2015/08/android-fragmentation/
Screen Size Fragmentation
http://opensignal.com/reports/2015/08/android-fragmentation/
Android iOS
... More Challenges
§ Mobile means your users are not in a protected environment
§ Network (Wifi, 3G, GPRS ...) § Sensors, Bluetooth ...
Learnings
§ Your users are not very patient
§ Quality has direct impact on apps‘ success
§ Challenges: Fragmentation and mobile
use of apps
§ Implication: Mobile QA is a central part of development
There must be an easy solution to handle this ?!
...nope!
What Do I Have to Test?
§ What to test: § Usability § Functionality § Performance § Edge Cases
§ Approx. 50 use cases (per platform) § Approx. minimum 20 target devices
Testing Framework
Development QA Production
Unit Tests
Functional Tests
Usability
Sanity
Monitoring
Automation
Manual Edge Cases
Checkups
Manual Approaches
§ Checkups, Usability, Edge Cases, Performance § Pros: You can test the actual user behavior
§ Cons: Time consuming, inefficient
Continuous Delivery = Continuous Testing
Build
Test
Deploy
Enhance
UI Automation Approaches
§ Regression, End-to-end, Integration § Pro: High test coverage, scalable, time-to-market,
releases resources for manual testing
§ Con: Initial investment, requires technical know-how, maintaining of scripts, no edge cases
Automated Testing
ROI
Investment
Test Cycles
Resources
UI Automation Components
§ CI & Build: Jenkins, Bamboo, Travis CI, Gradle, Ant, Maven
§ Scripting Tool: Proprietary or Open Source Tools § Infrastructure: e.g. Spoon, Selenium Grid,
SauceLabs, TestObject ...
UI Automation Frameworks
iOS Android Web Language
Calabash
x x - Ruby
Robotium - x - Java
Espresso
- x - Java
Appium
x x x Java, JavaScript, Python, Ruby ...
Code Example Appium
Example Test Execution
Learnings
§ Quality is crucial but mobile QA is complex
§ Different automation and manual testing approaches are required
§ For decent test coverage and time-to-
market UI test automation is key
Thank you!