Three-tier Mobile Application Testing Framework:

1

Tools available in market

• Robotium• Fonemonkey• See Test• Test studio• Calabash• Keynote• Eggplant• Appium / Selenium

And many more…

2

What’s Calabash

A cross platform mobile automation testing tool for Android and iOS platforms

Technology

• Calabash Cucumber framework• Ruby development kit• Android SDK

Cucumber

• Cucumber is a tool for running automated acceptance tests written in a behavior-driven development (BDD) style

Advantages• Open-Source Tool .• Cross platform support

– write once run anywhere (IOS and Android)Based on the Cucumber framework

• Supports both devices and emulators.• Provides simple language for a non programmer to write test

scripts• library Support for all the basic events and movements on

the mobile • Supports Native/Hybrid apps ,browser based apps • thriving forum and Google Group: “Calabash Android”.

Comparative analysis

• Real device support which most of the tools doesn't support.

• Source code is not a must.• Alert box and toast notifications can be

automated which cannot be achieved by other tools.

Steps – Configure A Device

Steps – Configure an Emulator

Feature files and step definitions Feature: Register featureScenario: As a user I can Register into my appGiven I am about to RegisterAnd i enter login credentialsThen I press "Login“

Step definition :Given /^I am about to Register$/ doscreenshot_embed(:label => "Register") endWhen /^i enter login credentials$/ doquery("* id:'Username'",{:setText => “test"})query("* id:'Password'",{:setText =>“test"})end

Calabash Android Architecture

Calabash IOS Architecture

Generating Cucumber files

Calabash-android gen Will generate feature

Re-signing the app

Re-sign application with new signature

Calabash console

• Creates test server

Calabash Run

• Connects to the device/emulator and runs the automation script

Test reporting

calabash-android run (path-to-apk )--format html --out reports.html will output the test report in the HTML format

DEMO

Limitations

• Slower execution time- on both emulator or device – always installs the app first before starting each scenario.

• If a step fails then the subsequent tests in the scenario are skipped.

• still in its nascent stage. complex scenario library support is currently not available. – Need to code your way in Ruby.

• Need the source code of the app for identifying the ids of various elements.

• Browser based apps can be tested with web-view only• ID capture is missed sometimes due to faster script execution

19

Scenarios - yet to be tried

• Flash content on a browser based app• Recording test scenario provision