the present and future of mobile automation

Jonathan Lipps • Director of Engineering • Sauce Labs

@AppiumDevs • @jlipps • @saucelabs

Mobile Testing Meetup • Seattle, WAApril 22, 2016

Jonathan Lipps • Director of Ecosystem & Integrations • Sauce Labs

@AppiumDevs • @jlipps • @saucelabs

Director of Engineering

Project Lead &Architect

Mobile is here to stay. So how do we scale mobile quality?

Automated testing is the solution for a fast dev cycle that maintains high quality

The Dev Cycle of Optimal Happiness[ ]

appium is the cross-platform solution for native and hybrid mobile automation

appium philosophy

appium aims to make mobile CI/CD possible by solving the problem of mobile app UI/functional testing

The appium philosophyTest the same app you submit to the marketplace Write your tests in any language, using any framework Use a standard automation specification and API Build a large and thriving open-source community effort

R1.

R2.

R3.

R4.

appium takes the best mobile automation tech and gives it to you in a Selenium WebDriver interface

appium lets you use the years of accumulated Selenium wisdom and best practices effectively for mobile

appium works with any language and any test runner / test framework, even custom ones

appium enables true cross-platform test-writing. One API for iOS, Android, and soon, Windows

appium is perfect for scaling your test concurrency with cloud solutions like Sauce Labs

appium present

Over 8,000 commits

3,300 stars2,000 forksOver 160 contributors

Over 2,100 pull requests Over 3,300 issues

Over 95 releases

Current Platform SupportiOS native, web, hybrid via Instruments + UIAutomation Android native via UiAutomator Android web, hybrid via UiAutomator + ChromeDriverAndroid native, hybrid via Selendroid (Instrumentation)

Current TeamAppium Core Team (5-10 people, Sauce Labs, FOODit, Opera, Bootstrap, Gamesys, etc…)Sometime contributorsIssue helpersAppium Discuss forum helpers

appium future

Future Platform SupportContinued support for the current platforms, plus:iOS native via XCUITest Android native, web, hybrid via UiAutomator2 (Eventually, older platforms will be deprecated)Windows native desktop via WinAppDriver

XCUITest (iOS)Apple’s Xcode 7+ UI test frameworkApple has not yet published documentationLot of reverse engineering by us / FacebookBeta is out!… still some unknowns

UiAutomator 2 (Android)Successor to Google’s UiAutomatorBased on InstrumentationMore stable, less flakeyAPI additions like waits, better support for using gestures with elements

WinAppDriver (Windows)For now, desktop-onlyMobile coming soonWorking with MicrosoftMJSONWP-based!

Future TeamAppium Core Team expands!Middle tier of trained contributors (anyone of you interested?)Issue triagers, not just helpersAppium Discuss forum helpers

appium takes the best mobile automation tech and gives it to you in a Selenium WebDriver interface

appium scale

Sauce Labs gives you browsers, mobile simulators, and real devices on-demand

appium is perfect for scaling your test concurrency with cloud solutions like Sauce Labs

appium best practicesWriting Robust Tests

“Robust” - Not flakey / no race conditions - DRY programming style - Test logic distinct from UI structure - Good organization (follows software dev best practice)

Wait for conditions wisely

Wait for conditions wisely

Use page/view object model

Use page/view object model

Use page/view object model

Building your app model

Building your app model

appium advanced topicsCross-Platform Tests

Cross-platform tests

Cross-platform tests - Always use accessibility ID strategy - Use a Page/View Object Model - Design apps for functional parity (not necessarily UI parity)

appium advanced topicsGestures

TouchAction API

MultiAction API

appium advanced topicsRunning Tests in Parallel

Run tests in parallel

Run tests in parallel - Both conceptual + technical requirements (no free lunch) - Eliminate test interdependencies - Suite-level ||ism vs. test-level ||ism - Pick a test runner with good support (e.g., Python’s nose)

appium advanced topicsDebugging Test Failures and Errors

Kinds of Errors - App errors (=> test failures, yay!) - Test errors - Client library errors - Appium server errors - Automation backend errors

Determining the kind of error - Test output / exception messages - Appium logs - Logcat / IOS logs - Screenshot / videos - driver.getPageSource() at failure point

appium advanced topicsSauce Labs Configuration

Test Configuration - set test names with {name: “xxx”} - put tests in a build with {build: “xxx”} - set pass/fail status

Sauce Temporary Storagecurl -u YOUR_USERNAME:YOUR_ACCESS_KEY \ -X POST \ -H "Content-Type: application/octet-stream" \ https://saucelabs.com/rest/v1/storage/YOUR_USERNAME/TestApp7.1.app.zip?overwrite=true \ --data-binary @apps/TestApp7.1.app.zip

https://docs.saucelabs.com/reference/rest-api/#temporary-storage

Sauce Connect - secure tunnel to your infrastructure - host your app without exposing it - host your stage website privately

Thanks!

http://appium.iohttps://github.com/appium/appium

@AppiumDevs • @jlipps • @saucelabs

Questions?

http://appium.iohttps://github.com/appium/appium

@AppiumDevs • @jlipps • @saucelabs