The material contained in this document is Confidential and Proprietary to InterraIT.
Turning Ideas Into Reality
Business Transformation - Innovative Technology Solution
InterraIT Testing & Quality
Assurance Services (iTQAS)
Interra Information Technology, Inc
25 Metro Dr., Ste 550, San Jose, CA 95110
Tel +1 408 451 1700 Fax +1 408 441 7495
www.InterraIT.com
InterraIT Confidential 2
iTQAS: InterraIT Testing & Quality Assurance Services
Methodology
Cross Application Testing
Product QA & Testing
Risk based Testing
Methodology
Critical Mass of Resources
Knowledge sharing
Resource flexibility
Access to in-depth expertise
Package Training / Certification
Team
Testing Center of Excellence (TCoE)
Subject matter experts
Latest versions of software
Develop solution frameworks
Dashboard Development & Tracking
Center of Excellence
iTQAS
Infrastructure
Test Labs
Virtual Test Labs ( Device Anywhere )
Robust IP protection mechanisms
Information security framework
Infrastructure
Test Suite Management: Test Link, Quality
Center, Rational Quality Manager
Test Lab Virtualization: VMWare
Memory Profiling Tools: JProfiler,
OptimizeIT, JConsole, YourKit
Defect Tracking: JIRA, ClearQuest, Remedy,
BugZilla, Rational Team Concert
Unit Testing & Code Coverage: JUnit/
HTTPUnit, NUnit, Eclipse, Clover
Cloud Provisioning: Amazon EC2
Cloud Management: Hyperic, Right Scale
Testing Tools: QTP, Silk Test, RIA Test,
Watir, Selenium, RTC, HTTP Analyzer,
Firebug, HTTP Fox, Robotium
Perl/ Java based Frameworks
Load Testing: Jmeter, Neoload,
TeamSuite
Tools
QA & Testing Tools Expertise
InterraIT Confidential 3
• Diversity of device environment
• Hardware and network related challenges
• Network Latency
Mobile Business Applications
• Resource intensive transactions
• Utilize corporate computing environment
• Integration with Existing systems
• Maintaining Security
Enterprise Application
• 400+ mobile network operators with different types of networks ( CDMA/GSM/Wi-Fi)
• Network Latency
Networking & Device Environment
• Mobile computing ecosystem consists of various OEMs with open standards, different application runtime
• Difference in screen sizes & resolution , rendering of images
Diversity in
Device
Behaviour
• Limited memory size & processing power
• Network landscape often act as data optimizers & restrict flow of information
• High Cost involved in buying multiple devices
Hardware Configuration & Networking
Challenges in Mobile Application Testing
InterraIT Confidential 4
Testing Strategy
InterraIT Confidential 5
Testing carried out at InterraIT
Type of Testing Manual Testing Automated Testing
Devices Emulators
Unit Testing Yes Yes No
Integration Testing Yes Yes No
System Testing Yes No No
Regression Testing Yes No Yes
Compatibility Testing Yes No Yes
GUI Testing Yes No No
Performance Testing Yes No Yes
Security Testing Yes No Yes
Synchronization
Testing Yes No No
InterraIT Confidential 6
Mobile Testing – Manual and Automation
Testing of Mobile Application is traditionally done by manual execution of test cases and visual
verification of results. But it is an effort intensive and time consuming process. Automating the
appropriate areas of a testing program can yield quantifiable results.
Emulators are beneficial for testing the features which are device independent, however the actual
devices should be used for validating the results.
Due to diversity in mobile hardware and platforms – GUI and compatibility testing to be done
along with functional testing
InterraIT Confidential 7
Testing Activities – Manual & Automation
Manual
Define objectives
Creation of Test Plan and Test Strategy
Creation of Test Cases
Execution of Test Cases
Reporting Defects
Creation of Test Reports and Defect Reports
Automated
Develop the framework to develop Automation
suite
Creation of Test Plan and Test Strategy
Develop high impact test cases based on scores
Use high level scripting functions which are not
dependent on menu structure of the device
Carried out using emulators & performance Testing
tools using PC
Reporting Defects
Creation of Test Reports and Defect Reports
InterraIT Confidential 8
Change &
Configuration
Management
Report defect
Defect Management
Report Status
Test Management
Application Under Test
GUI Interface
Non-GUI
Interface Database
Test Data
Database Spread sheet .CSV
Recording Script
Script Enhancement
Automated Script
Execution
Define Test Case
Review & Approval
Bu
siness F
un
ctio
ns
Pull Test
Data
InterraIT - Test Automation Architecture
InterraIT Confidential 9
NANT Automated Test Process for Windows
Scheduler
SVN
Automated Test Bed
Setup
NANT
Test Test Scripts
Test
Scripts
……
….
…..
Test
Scripts
……
….
…..
Test
Scripts
……
….
…..
Test
Scripts
……
….
…..
Test
Scripts
……
….
…..
Test
Scripts
……
….
…..
Ap
plicatio
n U
nd
er Test
Automated
HTML
Reports with
screenshots
of Errors
…..
…..
Release Candidate
InterraIT Confidential 10
Successful Mobile Engagements…
InterraIT Confidential 11
Field Service Mobility - Dell
Customer Background
Project Vision is to provide a web and mobile accessible application that will allow the field
technicians/engineers quick access to tools while on-site with customers. It supports
empowering the field technicians/engineers to resolve customer issues that drive repeat
dispatch (RD) tails. It also supports positively impacting customer resolution time and
customer experience (CE).
Automation using Scripts
Used Java and Eclipse framework for automation with Ant scripts( Android)
Used C# , Nunit with Nant Scripts for Windows 7
Importing Robotium wrappers for testing interactivity interactions.
Complete application flow was tested using UI controls
Android testing was carried out on multiple devices ( 6-7) covering OS 1.6 to 2.3
It was tried out on 2 major operators in India & 3 in US.
Testing Expertise
InterraIT’s Services
Black Box Testing
UAT Testing
Functional Testing
Logs Testing.
Performance Testing
Server Side Performance Testing
Client Side Performance Testing
Tools Used
Automation Tools
Robotium ( Android)
Junit
Jmeter for performance testing on server
Defect Tracking
Bugzilla
InterraIT Confidential 12
MyMazda Garage - Mazda
Customer Background
InterraIT adds value to My Mazda project by developing a native iphone application that
enables MNAO customers to access Service History ,Maintenance Schedule, Locate a Dealer,
Find my car ,Service Offers and Service Reminders .
Manual Testing:
1. UI & content, Testing
2. Functional Testing
Device Anywhere for testing and monitoring the usability, performance and availability of
the app.
Tested across 2major operators in India and 2in US.
Tested across multiple handsets iPhone 3G,3GS and 4G
Testing Expertise
InterraIT’s Services
Black Box Testing
UAT Testing
Functional Testing
UI & Content Testing
Performance Testing
Server Side Performance Testing
Load Testing
Tools Used
Automation Tools
JMeter for Server side Load Testing
SenTestingframework for Unit testing
Defect Tracking
Bugzilla
InterraIT Confidential 13
MyMazda Sales iPad - Mazda
Customer Background
InterraIT adds value to Mazda Sales I Pad application by developing a web based I Pad
application that enables MNAO Sales consultants and dealers to get video and textual
information during the various steps of the Sales process.
Manual Testing:
1. UI & content, Testing
Tested across 2 operators in US and 2 in India
Tested on Ipad1 & Ipad2 devices
Testing Expertise
InterraIT’s Services
Black Box Testing
UAT Testing
UI & Content Testing
Tools Used
Tools
iPhone SDK 3.1.3
XCode 3.2.4
Defect Tracking
Bugzilla
InterraIT Confidential 14
Market Street - USM
Customer Background
“Market Street” lets you build, save and alter any number of grocery list on your iPhone.
Application will also populate the weekly special offers for Shopping Store’s products if
applicable. Based on your requirement you can add items and quantity you want, in your
shopping cart. At any point of time you can also find out your current location & near by
Shopping Stores location at Google map. Application will also give privilege to browse
multiple recipes, based on different level of categories. Marking recipes into “Favorite
Recipes List” is also permitted.
InterraIT’s Services
Black Box Testing
UAT Testing
Functional Testing
Regression Testing.
Client side Performance Testing
Leaks
Activity Monitor
Time Profile
Network Connections
Tools Used
Tools
iPhone SDK 3.1.3
XCode 3.2.4
Defect Tracking
BugZilla
SenTestingframework
Automation from Instruments native application
Testing Expertise
InterraIT Confidential 15
GPS Navigation Application - TCS (Telecommunication Systems)
Customer Background
TCS (Formerly known as Networks In Motion, Inc. (NIM)) is a product base company. Atlasbook is
base product of TCS. Atlasbook is a GPS based Navigation application with different
features. The activities include feature development on BREW, porting, bug fixing for
different mobile and R&D as per the product and mobiles
Bugzilla is used for bug tracking & “Perforce” is used for the tracking of code check- ins.
For code review there is a specific owner from developer team to approve the code.
Car drive testing is done for testing the navigational feature
Testing out on different BREW devices from Samsung, ZTE, LG , Pantec and Casio etc.
Tested across more than 5 operators in US (Verizon, AT&T, US Cellular, MetroPCS, Cricket etc.)
Testing Expertise
InterraIT’s Services
Black Box Testing
UAT Testing
Functional Testing
Logs Testing.
Tools Used
For Non GPS testing Hopman Box is used to test various scenarios and
behavior of app.
Logging tool ( QALOG) which is responsible to log all the information in
encoded format which can be decoded on the server for analysis purpose.
Defect Tracking
Bugzilla
InterraIT Confidential 16
Successful Enterprise Engagements…
InterraIT Confidential 17
InterraIT – QA & Test Automation Role
Physical Appliance, Virtual Appliance, Cloud Testing
QA strategy document for every major release
Test Plan from functional specifications
Test cases from Test Plan – identified positive, negative and boundary
conditions
BFAT – Build Functional Acceptance Testing
Use of RTC, JIRA for bug tracking & QC/ RQM/ Testlink for test case
repository
Value added quality processes – bug tracking, classification – blocker, priority,
severity
Analysis of need for test Automation
Components – Studio, Connectors, Platform, WMC
Different automation techniques
Use of RTC, QTP, Perl, RIA Test
Development of a QA data warehouse for defect analysis
Comparison and consultancy on Automation Tools
Escape Analysis
Lab Virtualization (use of VMWare)
State of the art test labs
High Customer satisfaction and quality of support
Lower costs due to higher productivity of quality resources
Participating in version upgrades
Middleware
Databases
Flat-files
(FTP, HTTP,
E-mail)
XML /
Web Services
ERP/CRM
Custom
Applications
B2B
(EDI, XML, Excel)
On-demand
Applications
Tools Used
Quality Center, Testlink
QTP, RIA Test, Perl Harness
JIRA, Confluence
VMWare
Code Coverage on Clover
Code Profiling: Jprofiler, GDB, TotalView
(C++)
InterraIT’ Service Offerings
Achievements & Client’s Benefits
InterraIT Confidential 18
Cloud Based Application Testing Experience
• Leveraging our expertise in integration space to test integration
solutions on Amazon’s EC2 cloud
• On Premise & Cloud Integration
Testing SAAS Applications on Cloud.
Leveraging various cloud features such as Multi Tenancy, Multi
Environment to test real customer scenarios
Session Management
Security Testing using Key Authentication
Leveraging Secure Agent to connect to the servers behind the
firewall
Server Management for Cloud Instances using Hyperic
Lower cost of ownership
Reduced infrastructure management responsibility
Service delivered through a browser
Faster application rollout
Tools Used
Hyperic
Right Scale
Expertise on Cloud
Achievements & Client’s Benefits
InterraIT Confidential 19
Partner Tools QA & Test Automation - VMware
Workbench –VMware ships workbench (a Desktop Development Environment ) as
a Virtual Appliance in the Linux environment to its partners. The partners use the
workbench as tool for running their Hardware Certification Test Cases. Workbench
is build on Eclipse platform using Java SWT.
Hardware Compatibility Guide – Web HCL is a portal used by the VMware
partners to verify if their Hardware Products are compatible with the VMware
products. The portal is re-engineered to meet Web 2.0 standards and it uses LAMP
technology to overcome the usability and performance issues of the existing portal.
Project Background
InterraIT’s QA Services
White Box Testing
Automated JUnit test cases of the new features
Build Acceptance Testing
Black Box Testing
Integration Testing
Functional Testing of New Features
UI Driven End to End Use Case Scenarios
Performance Testing
Server Side Performance Testing
Client Side Performance Testing
Tools Used
Automation Tools
Window Tester Pro for White Box Testing
Silk Test 2009 for Functional Testing
JMeter for Performance Testing
Test Case Management , Defect Management &
Configuration Management Tools
TCRMS
Bugzilla
Perforce
InterraIT Confidential 20
QA Services for content Delivery - Knewco
Customer Background
InterraIT adds value to Knewco by automating the content discovery, phrase extraction
and Definition retrieval from their websites. It helps them to build an automation strategy
and manually test the rest of the chunks on their web application
Automation using Selenium:
Used Java and Eclipse framework for automation with Selenium
Injecting the java script code to Selenium code
Crawling URLs against each website & Listing concepts from each URLs
Fetching concept Definitions and checking any disambiguation
Manual Testing:
UI & content, Testing
Bubble & ads Testing
Testing Expertise
InterraIT’s Services
Black Box Testing
UAT Testing
Functional Testing
Logs Testing.
Multi Browser Testing
Performance Testing
Server Side Performance Testing
Client Side Performance Testing
Tools Used
Automation Tools
JMeter for Server side Performance Testing
Selenium for Functional Testing of URLs Crawling and concept
handling
HTTP Analyzer for Client side performance testing on IE
Firebug and HTTP Fox for Mozilla, Safari and Chrome browsers
Defect Tracking
Bugzilla
InterraIT Confidential 21
Project Release Cycle Management
Manual Testing:
Migration Testing
System Testing
Regression Testing
Tickets Resolution
Test Automation
Design Automation Framework
Automation Scripts Creation and Execution
Automation Execution Metrics
Test Reporting and Analysis
Client leveraged InterraIT’s strong expertise in AS400 and QTP automation
Migration from .Net 1.1 to .Net 4.0 tested successfully
Salesforce Test Automation using HP QTP-QC
Tools Used
Quick Test Pro
Quality Center
Clarity
InterraIT’ Service Offerings
Achievements & Client’s Benefits
QA Services for Online Marketing Company - Vertrue
InterraIT Confidential 22
May 2004 - Cast Iron & InterraIT relationship established with 2.x product version
July 2004 – ODC established in India with strong team on Integration skills
Team started with Testing / QA followed by development and Customer Support
InterraIT became an integral part of 3.x product line with support on 2.x
QA Automation kicked off with Perl Harness
InterraIT team expanded to include Professional Services engagement
Perl RT Harness expanded
Install / Upgrade & QTP Automation initiated
QA Automation expansion; CLI Automation initiated
Connector & WMC development, testing on RIA Test
Offshore Consulting work initiated
InterraIT team conducted Training
Key Partner on 4.x Product
Support Knowledgebase
3.x to 5.x version support
Management API Harness
2004
2005
2006
2007
2008
2009
5.X and 6.x Testing
Automation – Generic projects
Hyperic, RightScale
2010
IBM Cast Iron - InterraIT Engineering Relationship
Hardware
Appliance
based
Integration
Virtual
Appliance
Cloud
The material contained in this document is Confidential and Proprietary to InterraIT.
Turning Ideas Into Reality
Business Transformation - Innovative Technology Solution
Interra Information Technology, Inc 25 Metro Dr., Ste 550
San Jose, CA 95110
Tel +1 408 451 1700 Fax +1 408 441 7495
www.interrait.com
Thank You
InterraIT Confidential 24
Testing Methodology for
Mobile Applications
