Appium & Perfecto: A Perfect Match

Max Skybin, Director of Sales EngineeringNick Sanjines, Senior Sales Engineer

AGENDA

• Digital transformation context• Key enablers of digital quality

• What is Appium?• Appium’s strengths and limitations

• Who is Perfecto and how we work with Appium• How we improve and build on Appium’s core strengths

• Benefits of using Perfecto & Appium together• Demo

© 2016, Perfecto Mobile Ltd. All Rights Reserved.

05/01/2023 3© 2015,Perfecto Mobile Ltd. All Rights Reserved.

3

- Physical experience and interaction gives way to Digital

- Digital is an Omni-Channel strategy

- Mobile is the first screen among connected consumers

MobileSocial

Enterprise End User

Web

Internet of Things

The Digital Transformation

DIGITAL PLATFORMS ARE ALL MERGING INTO ONE EXPERIENCE

05/01/2023 4© 2015, Perfecto Mobile Ltd. All Rights Reserved.

One User Experience Across All Digital Platforms

Source: Google

Network QualityCellular, Wi-Fi

Interacting with sensors

Peak time -Millions of transactions Multiple devices

Specific profiles

USERS DEMAND EXCEPTIONAL DIGITAL EXPERIENCE REGARDLESS OF CONDITIONS

Varied Device Conditions

Background Apps Location

Immediate, reliable, secure 24/7 access to mobile devices and web-browsers across all

required platforms

Unattended parallel automation covering full

scope of functional and non-functional requirements

Real-time on-demand reporting with high-level view of full digital quality as well as

detailed drill-down into specific test results

KEY ENABLERS FOR EXCEPTIONAL DIGITAL QUALITY

© 2016, Perfecto Mobile Ltd. All Rights Reserved.

Enterprise-GradeDevice Lab

RobustAutomation

ComprehensiveReporting

Reduce device management spend and enable high levels of

automated testing

Avoid manual testing spend without compromising quality

Quickly identify quality issues earlier in SDLC, prioritise, triage and fix defects across web and mobile

WHAT IS APPIUM?

Appium is a tool for automating testing of applications on mobile devices. It was created as an open-source alternative to existing commercial tools.

05/01/2023 7© 2016, Perfecto Mobile Ltd. All Rights Reserved.

APPIUM’S BENEFITS

• Cross-platform• Write tests against multiple platforms (iOS, Android, Windows), using the same API

• Non-intrusive• Doesn’t require the app to be modified or recompiled• Doesn’t require access to application’s source code• Test what you ship to end-users

• Open• Automate using any language/framework• Based on industry-standard Selenium Web Driver• Open source project supported by community

05/01/2023 8© 2016, Perfecto Mobile Ltd. All Rights Reserved.

APPIUM’S LIMITATIONS

• Open Source Support Model• No official support when something goes wrong• Tends to take 1-2 months to support a new iOS version after GA release

• No OS-level or environmental controls• Testing outside the application context is impossible, which prevents automation of full

digital experience (2 factor authentication, widgets, interaction with other apps)• No environment controls are available to change GPS, network conditions, etc.• Object-level recognition only, no ability to visually validate user experience

• Unreliable automation• Defects in Appium/Selenium cause test failures• Parallel execution difficult to configure and sustain

• Negative productivity impact• No reporting capabilities – you have to build your own reporting• Lack of IDE plug-ins slows down script creation

05/01/2023 9© 2016, Perfecto Mobile Ltd. All Rights Reserved.

IMPACT: APPIUM DOESN’T ADDRESS THE FUNDAMENTAL PROBLEM OF PROVIDING AN ENTERPRISE-GRADE LAB

10

Real end user conditions

Complex hardware & software setup

Secure environments

On-demand access to mobile devices, browsers & wearables

24x7 continuous support

iOS 10 Beta 6/13

iOS G.A.9/13

10/11 - Appiumsupport for iOS 10

120 days without iOS 10 support!54% user adoption of iOS 10 during Appium support gap

IMPACT: LACK OF TIMELY OS SUPPORT IN APPIUM LEADS TO RISK OF RELEASING UNTESTED APPLICATIONS DURING WIDE OS ADOPTION

Appi

um C

anno

t Tes

tAp

pium

Can

Te

stIMPACT: LACK OF OS-LEVEL CONTROLS AND VISUAL VALIDATIONS LIMITS THE AMOUNT OF AUTOMATION COVERAGE

05/01/2023 12© 2016, Perfecto Mobile Ltd. All Rights Reserved.

Verify the user is able to enter only valid postal code • Verify the user is able to enter only valid post code

by checking input exists in DOM

• Verify the continue button is visible so the user can proceed to the next screen

• Verify the screen renders within acceptable time under WiFi, LTE and 3G connectivity

• Validate that when invalid postal code is entered, error message is colored red

IMPACT: POOR REPORTING MAKES TEST RESULT ANALYSIS AND DEFECT ISOLATION A PAINFUL EXERCISE

05/01/2023 13© 2016, Perfecto Mobile Ltd. All Rights Reserved.

WHO IS PERFECTO?

A Cloud Based Continuous Quality

Lab enabling application testing and

monitoring on real devices in real

end user conditions, for enterprises

adopting digital transformation and

agile delivery processes

INDUSTRY LEADERSHIP

PERFECTO OFFERS THE TESTING PLATFORM FOR COMPLETE DIGITAL COVERAGE

Digital Test Suite Reporting for Agile Teams

One ScriptFor All Digital Platforms

One Lab – Web & MobileOne Secure Cloud

Manual & Automated Testing

GLOBAL, ENTERPRISE-GRADE DIGITAL LAB

05/01/2023 17© 2015, Perfecto Mobile Ltd. All Rights Reserved.

OPEN AND INTEGRATED

18

Open & Integrated

XcodeSelenium

HOW PERFECTO EXTENDS APPIUM

ACHIEVE HIGH %OF AUTOMATION

SYSTEM LEVEL

CONTROL

IMAGE INJECTION

OCR & IMAGE OBJECT

RECOGNITION

RICH REPORTS

MIMIC REAL USER ENVIRONMENT

GPS INJECTION

NETWORK VIRTUALIZATI

ON

PERSONA-BASED

TESTING

HYBRID CLOUDGLOBAL

COVERAGE

GET ENTERPRISE

GRADE

ENTERPRISE SECURITY

24X7 SUPPORT

SCALE & FLEXIBLE

ROLES & PERMISSION

S

MAKING OPEN-SOURCE

ENTERPRISE READY

OPEN SOURCE

IOS & ANDROID

NATIVE/DOM OBJECT

RECOGNITION

CI SUPPORT

BENEFITS OF USING APPIUM WITH PERFECTO

• Reliable unattended parallel automation• Reuse the same Appium scripts and skill-set• Fixed Appium/Selenium defects that cause failed executions fixed• Unlimited parallel execution, no setup required

• OS-level and environmental controls• You can interact with anything on the screen testing real-user interactions• Test pop-ups, incoming calls, SMS, interruptions, etc• Full ability to test a variety of real user conditions

• Accelerate tester and developer productivity• Integrated reporting engine provides screenshots, videos, script and device

analytics, and more accelerating defect triage and giving executive visibility• IDE plug-ins for popular IDEs accelerate script development and debugging

• Enterprise Support Model• 24x7 support, global, secure, audited environments• OS Beta support. Same-day support for GA versions.

05/01/2023 20© 2016, Perfecto Mobile Ltd. All Rights Reserved.

Appi

um w

ith P

erfe

ctoA

ppiu

m O

nly

IMPACT: ACHIEVE COMPLETE TESTING COVERAGE WITH VISUAL ANALYSIS, OS-LEVEL TESTING, AND REAL END-USER CONDITIONS

05/01/2023 21© 2016, Perfecto Mobile Ltd. All Rights Reserved.

Verify the user is able to enter only valid postal code • Verify the user is able to enter only valid post code

by checking input exists in DOM

• Verify the continue button is visible so the user can proceed to the next screen

• Verify the screen renders within acceptable time under WiFi, LTE and 3G connectivity

• Validate that when invalid postal code is entered, error message is colored red

IMPACT: UNDERSTAND THE IMPACT OF REAL-WORLD CONDITIONS ON APPLICATION PERFORMANCE WITH PERSONA-BASED TESTING

22

Marketing / Business Business-Engineering alignment Automated lab configuration Communicate Risk in business terms

((( )))Name: SaraLocation: LondonAge: 25Device: NewSensitivity to UX: HighBehavior: Fashion magazine designer

Engineering

Name: SaraLocation: New YorkAge: 25Device: NewSensitivity to UX: HighBehavior: Fashion magazine designer

Data?

Test Cases

Name: SaraLocation: New YorkDevice: iPhone 7, iOS 10Carrier: BTNetwork: 3GBackground Apps: Instagram

IMPACT: UNDERSTAND HIGH-LEVEL DIGITAL QUALITY TRENDS AND QUICKLY NARROW DOWN PROBLEMATIC AREAS

05/01/2023 23© 2016, Perfecto Mobile Ltd. All Rights Reserved.

IMPACT: ACCELERATE DEFECT RESOLUTION BY ZOOMING INTO DETAILS OF EACH FAILED TEST EXECUTION WITH DEVICE VITALS, SCREENSHOTS AND VIDEOS

05/01/2023 24© 2016, Perfecto Mobile Ltd. All Rights Reserved.

PERFECTO PLATFORM

IN-HOUSE OR3RD PARTY LAB

APPIUM & SELENIUM

PERFECTO ENTERPRISE EXTENSIONS

AUTOMATION FRAMEWORK

AUTOMATION SCRIPTS

CONTINUOUS INTEGRATION

< 50% AUTOMATION

90%+ AUTOMATION

ACHIEVE HIGH BUSINESS VALUE

WITH PERFECTO & APPIUM

05/01/2023 26© 2016, Perfecto Mobile Ltd. All Rights Reserved.

DEMO