Mobile DevTest Dictionary -...

Mobile DevTest Dictionary1

GlossaryMobile DevTest Dictionary: Terminology from A to Z

February, 2016

Edition 1

Tools ............................. 5-8

Mobile App Styles............ 9-11

Objects Defined .............. 12-14

Software Testing Styles .... 15-24

Developer & Agile Terms ... 25-32

Common Testing Terms..... 33-35

Table of Contents

Mobile DevTest Dictionary 4

IntroductionAs a market-leader with experience spanning many industries and levels of mobiletesting maturity, Perfecto has unique insights into the most critical tools and trends.We’ve used this inside information as the source for our “Mobile DevTest Dictionary:Terminology from A to Z,” a go-to guide for developers, testers and practitioners to stayinformed on the resources they need to excel at their jobs.

The DevTest Dictionary is divided into six categories with terms covering automationand CI tools, testing styles, agile best practices, and more that we’ll be updatingregularly. Happy reading!


ToolsAppium

Calabash

Appium is an open source test automationframework for use with native and hybridmobile apps. It drives iOS and Android appsusing the WebDriver JSON wire protocol.All implementations of WebDriver thatcommunicate with the browser, orRemoteWebDriver server use a common wireprotocol. This wire protocol defines a RESTfulweb service using JSON over HTTP.

This software automation tool supportedby Xamarin is based on the BDD conceptand Cucumber tool. Calabash wasextended by LessPainful (acquired byXamarin) to mobile app testing based onthe Ruby language. Calabash tests can bedeveloped and executed on local devicesor remotely using the Xamarin test cloud.

Perfecto supports Appium test executionand extends Appium capabilities inside thePerfecto Continuous Quality Lab with featuressuch as visual object analysis and usercondition testing (location, networkcoverage, etc).

Perfecto supports Calabash testexecution.


CI Tools

Development Languages

Espresso

Maven

TeamCity, Bamboo, Gradle, Jenkins, TFS,BuildBot

C#, Java, Objective-C, Swift, Ruby, Python –Most of these are supported by Selenium.

The Espresso testing framework, available inthe Android Testing Support Library, providesAPIs for writing UI tests to simulate userinteractions within a single app. Espressotests can run on devices running Android2.2 (API level 8) and higher. A key benefit ofusing Espresso is that it provides automaticsynchronization of test actions with the UI ofthe app you are testing.

Apache Maven is a software projectmanagement and comprehension tool.Based on the concept of a project objectmodel (POM), Maven can manage a project’sbuild, reporting and documentation from acentral piece of information.

Jenkins is supported by Perfecto. The pluginis in the public Maven repository and in theJenkins official wiki page. Microsoft CI is alsosupported.

As part of Perfecto’s Remote WebDriverimplementation, Perfecto offers full supportfor the Selenium languages within Eclipse andVisual Studio IDEs.

Perfecto supports the development andexecution of Espresso Android UI tests on realdevices in its CQ Lab.

http://developer.android.com/tools/testing-support-library/index.html


Remote WebDriver RemoteWebDriver is a tool that helps executeSelenium and Appium tests. It is composed oftwo pieces: a client and a server. The client isyour WebDriver test and the server is simplya Java servlet, which can be hosted in anymodern JEE app server. The server will alwaysrun on the machine with the browser youwant to test.

Perfecto supports Remote WebDriver todevelop Selenium/Appium test code inany supported language.

Selenium Selenium is a suite of tools that automatesthe testing of web browsers and mobile appsacross various platforms. It has an add-onfor the Firefox browser for web automationrecording, and also mobile specific solutionssuch as MobileCloud WebDriver and Appium.The Selenium technology allows you toidentify objects from applications usingXPATH, CSS, Class Name and visual.

Perfecto supports the Seleniumtest framework for both mobile (iOSand Android) as well as desktop web testing.


WebDriver

Selenium Grid

Selenium WebDriver is the successor toSelenium RC (remote control). WebDriveraccepts commands (sent in Selenese, or via aClient API) and sends them to a browser. Thisis implemented through a browser-specificdriver (Android, Chrome, iOS), which sendscommands to a browser and retrieves results.In mobile, there are specific WebDriversfor Android and iOS. For desktop, there aredrivers for the relevant browsers (Chrome,Internet Explorer, FireFox, etc).

Selenium Grid supports distributed testexecution by allowing customers to runmultiple tests at the same time on different machines against different browsers and devices.

MobileCloud WebDriver is in phase out mode;we recommend customers use the RemoteWebDriver solution.


MobileApp Styles

App Instrumentation

Hybrid Application

The method of bundling a source code librarywithin an application, or changing its code toget native object support.

Hybrid apps run inside a native container,and leverage the device’s browser engine(but not the browser) to render the HTMLand process the JavaScript locally.

This test automation supporting techniqueallows dev and testers access to the app’snative object properties. It’s most commonlyused in iOS platform test automation.Perfecto provides an app instrumentationsolution to get fully native and DOM objectidentification for hybrid apps.

Web Application A web app is an app written in HTML/JavaScript that runs within the platformbrowser (Chrome, Safari, Android NativeBrowser) and requires constant networkconnectivity to access the server and retrievedata. On the other hand, native apps runlocally on the device platform and can belaunched independently from the browser.


Responsive Web App

Adaptive Web App

Responsive app design works on theprinciple that a single fluid website can lookgood on any device. Responsive websites usemedia queries, flexible grids, and responsiveimages to create a consistent user experience that adapts and changes across digital platforms such as mobile, web and IOT.

Like responsive, adaptive design attempts tooptimize the experience across platforms. But instead of one flexible design, adaptive design detects the device and other features, and then provides the appropriate feature and layout based on a predefined set of viewport sizes and other characteristics.

Perfecto enables customers to test RWD(responsive web design) through its SeleniumRemote WebDriver solution. Customers canbuild a Selenium Grid and execute Seleniumtests in parallel on desktop browsers andmobile devices.



Objects Defined

DOM Objects The list of objects for a mobile web/hybrid application on which the test automation developer will perform actions (press the buttons, etc.).

The DOM is represented as a tree of objectswhere each object is related to the others,making the path from one to the otherreasonable. Getting DOM objects analysisfrom the native browser (Safari) is not alwayseasy. Perfecto is among only a few vendorsproviding such support for both web andhybrid apps.

Perfecto supports OS level native objects for both iOS and Android. Such objects can be identified through XPATH, CSS or othermethods.

Native Objects These are mobile OS-specific objects that areidentified by the unique properties that theapp developer assigned to them. Forexample, XPATH retrieves and provides thenative objects properties from a mobile/hybrid app.


System Level Control The ability to interact with the mobileoperating system outside of the applicationunder test (change device settings, press theHome button , etc).

This is essential for various use cases, bettertest coverage and continuous integration.

Visual Objects Usually known as OCR (Optical CharacterRecognition) engines. OCR essentially uses asmart software engine that converts scannedimages of handwritten, typewritten orprinted text into machine-encoded text.

Perfecto supports visual object analysis aspart of its test automation solution.

XPATH XML Path Language is a query languagefor selecting nodes (i.e., locating data orobjects) from an XML document.

As part of its Selenium support, Perfectosupports XPATH object identification innative, web and hybrid apps.



SoftwareTesting Styles

Advanced functional testing

Availability

ATDD

The practice of adding simulated realworld conditions and out-of-applicationcontext events to functional test flows toensure that the application functions inreal world conditions.

Perfecto’s Wind Tunnel enables advancedfunctional testing on real devices.

Measurement of the availability of key stepsof the application.

Acceptance Test-Driven Development is atechnique used to bring customers into thetest design process before coding begins.It is a collaborative practice where users,testers and developers define automatedacceptance criteria. ATDD helps ensure thatall project members understand preciselywhat needs to be done and implemented.Failing tests provide quick feedback thatrequirements are not being met. The tests


ATDD Cont. are specified in business domain terms. Eachfeature must deliver real and measurablebusiness value. If a feature doesn’t traceback to at least one business goal, then itmay not be necessary.

Integration Testing

Longevity Testing

The phase in software testing in whichindividual software modules are combinedand tested as a group. It occurs after unittesting and before validation testing.Integration testing uses modules that havebeen unit tested, groups them in largeraggregates, applies tests defined in anintegration test plan to those aggregates,and outputs the integrated system as readyfor system testing.

The practice of testing the applicationbehavior along specified user flows over anextended period of time and iterations toensure application functionality over anextended period of usage. Typical aspects of such testing is memory leaks and battery drain.

As part of Perfecto’s performance testingand Wind Tunnel capabilities, customers canget insights into app CPU, memory and otherresource consumption.


Network Virtualization

Performance Optimization

Performance Testing

This technology enables pre-productionteams to model and simulate networkconditions as accurately as possible. Networkvirtualization provides predictability of theuser experience in real world conditions inproduction to optimize the applicationbehavior in such conditions.

The practice of analyzing the applicationdesign and behavior (availability andresponsiveness of KPIs), as well asunderlying data (vitals, network analysis)to ensure availability of key steps in theapplication.

The practice of measuring and optimizing the behavior of an application in terms ofavailability and responsiveness to ensureacceptable user experience in real-worldconditions. Performance testing can come in two flavors: single user performance test and load/stress test.

Perfecto offers NV capabilities as part of themanual, automation and performanceproducts in its private hosted clouddeployments.

Perfecto offers a full mobile performancetesting solution that can be leveragedeither through integration with Jmeter orHP LoadRunner.


RUM Monitoring

Single User Performance Testing

Service Virtualization

Real User Monitoring – A methodof monitoring production apps byembedding a piece of software (SDK)within the application that communicatesreal-time analytics data about the appfrom the device running back to themonitoring server.

The practice of adding simulated real worldconditions and out-of-application eventsto functional test flows to ensure theapplication’s responsiveness and availabilityin real world conditions.

A method to emulate the behavior of specificcomponents in heterogeneous component-based applications such as API-drivenapplications, cloud-based applications andservice-oriented architectures. It is usedto provide software development and QA/testing teams access to dependent systemcomponents that are needed to exercisean application under test (AUT), but areunavailable or difficult-to-access fordevelopment and testing purposes.

Using this method allows more use ofanalytics -based information. However, it’svery hard to pinpoint and identify the rootcause of the issues because the reports aresent from a user’s private device whereproduction teams do not have access.

Perfecto provides mobile SV through itspartnership with CA and the CA Lisa forMobile solution.


Single User Profiling

Synthetic Monitoring

A set of tests performed as part of the earlydevelopment stages, typically in-sprint,maybe nightly. Unlike load and stresstesting, typically done out of the sprint,the single user profiling increases softwarereliability and usability by testing thesoftware against various simulated real world conditions such as network conditions and application interruptions. The reportingincludes measuring device vitals and memory leaks as well as responsivenesson different platforms. With this practice,development teams can reveal performanceissues in advance and be more prepared forthe performance testing phase.

A monitoring method that’s performed in a “clean room environment” (lab) to detect, analyze and fix production issues as soon as they are identified. No app changes are required to execute synthetic monitoring scripts.

Perfecto supports this method of monitoringthrough integrations with HP BSM andAlertSite (SmartBear).


System Testing

Smoke Testing

Sanity Testing

System testing of software or hardware istesting conducted on a complete, integratedsystem to evaluate the system’s compliancewith its specified requirements. Systemtesting falls within the scope of black boxtesting, and as such, should require noknowledge of the inner design of thecode or logic.

Perfecto customers can develop system testsfor any of their mobile app styles in thePerfecto CQ Lab in the cloud.

Perfecto customers can develop smoke subsettests for any of their mobile app styles in thePerfecto CQ Lab in the cloud.

Perfecto customers can develop and executesanity tests for any of their mobile app styles inthe Perfecto CQ Lab in the cloud.

Preliminary testing to reveal simple failuressevere enough to reject a prospective softwarerelease. A subset of test cases that cover themost important functionality of a componentor system is selected and run to ascertain ifcrucial functions of a program work correctly.

Basic test to quickly evaluate whether aclaim or the result of a calculation canpossibly be true. Sanity Testing is a simplecheck to see if the produced material isrational (that the material’s creator wasthinking rationally). The point of a sanitytest is to rule out certain classes of obviouslyfalse results, not to catch every possibleerror. The advantage of a sanity test, over acomplete or rigorous test, is speed.


Security Testing

Regression Testing

Security testing is a process intended toreveal flaws in the security mechanismsof an information system that protects dataand maintains functionality as intended.Due to the logical limitations of securitytesting, passing security testing is not anindication that flaws don’t exist or thatthe system adequately satisfies the securityrequirements.

Typical security requirements may includespecific elements of confidentiality,integrity, authentication, availability,authorization and non-repudiation. Actualsecurity requirements depend on the securityrequirements implemented by the system.

A type of software testing that seeks to uncover new software bugs, or regressions, in existing functional and non-functional areas of a system after changes such as enhancements, patches or configuration changes have been made to them.

The purpose of regression testing is to ensure that changes such as those mentioned above have not introduced new faults. One of the main reasons for regression testing is to determine whether a change in one part of the software affects other parts of the software.

Perfecto provides various test automation solutions to run regression tests on real devices and browsers.


Unit Tests Unit testing is a software testing method bywhich individual units of source code andusage and operating procedures are testedto determine if they are fit for use.

Perfecto customers can integrate with variousIDEs such as Eclipse and Visual Studio todevelop Junit, TestNG and other unit testswithin the Perfecto CQ Lab in the cloud.

Regression Testing Cont. Common methods of regression testing includere-running previously completed tests andchecking whether program behavior has changed and whether previously fixed faults have reemerged. Regression testing can be performed to test a system efficiently by systematically selecting the appropriate minimum set of tests needed to adequately cover a particular change.

User Condition Testing

MAIN TRAITS:- Traveler who’s always connecting to new networks, Wi-Fi and carrier- Loyal Apple customer, relies on multiple devices- Big user of social, banking, and enterprise apps

TESTING REQUIREMENTS:- Test for newest Apple smartphones, tablets, and smartwatches- Prepare for apps conflicts, interruptions, alerts, and changing networks

Georgia, 47Corporate Road Warrior

The mimicking of an end-user environmentusing personas along with theirenvironment and behavior traits such as

network conditions, devices and locations, and apps running in background. These conditions are all embedded into a test automation scenario to enhance test coverage and increase app quality.

Perfecto offers user condition testing in its CQLab through the automation capabilities of its Wind TunnelTM product.


Vitals

Production Insights

Metrics indicating the state of the client/hardware/OS, as well as the environment.Examples include CPU, memory, datathroughout and HTTP errors. Vitals are usedto help shed light on the cause of a delayedresponse in the application.

Tests performed post production release ona continuous basis to provide key insightsaround response time, app availability,regressions and other issues.

Perfecto provides device related vitals such aslogs and CPU as well as app level vitals.

Can be performed through Perfecto’ssynthetic monitoring or test automationsolutions.

Mobile DevTest Dictionary 24Mobile Market Glossary24


Dev & AgileRelated Terms

BDD Behavior-Driven Development (BDD) is asoftware development process based on test-driven development. BDD allows DevTestteams to collaborate earlier in the SDLCaround functional and user experiencetesting of an app. The advantage of BDD isit makes development easier because itdescribes the business logic and outcomes of specific use cases.

Cucumber and Calabash are desktop/mobiletools supporting this technique.

Perfecto provides basic support forCalabash, a BDD related tool used bydevelopers and testers through the Rubydevelopment language.


Continuous Integration (CI) The software engineering practice ofmerging all code copies to a sharedmainline several times a day.

When embarking on a change, a developertakes a copy of the current code baseon which to work. As other developerssubmit changed code to the source coderepository, this copy gradually ceasesto reflect the repository code. Not onlycan the existing code base change, butnew code can be added as well as newlibraries and other resources that createdependencies and potential conflicts.

The longer a branch of code remainschecked out, the greater the risk ofmultiple integration conflicts and failureswhen the developer branch is reintegratedinto the main line.

Continuous integration involvesintegrating early and often to avoidthe pitfalls of “integration hell.” Thepractice aims to reduce rework and thusreduce cost and time.

Perfecto customers can implement theircontinuous integration process in the cloudusing Jenkins CI, Microsoft Visual Studio TeamServices or TeamCity.


Continuous Delivery (CD)

Continuous Quality

Is a software engineering approach in whichteams produce software in short cycles,ensuring that the software can be reliablyreleased at any time. The CD process helpsreduce the cost, time and risk of deliveringchanges and facilitates more incrementalupdates to applications in production.

Continuous Quality is a methodology forembedding quality activities into every stepof the SDLC process, from design throughbuild to production based on supportingprocesses, tools and lab infrastructure that’stailored for the intensity of the process.

Successful continuous quality acceleratestime to market, drives faster and morefrequent releases and helps reduce escapeddefects to production by managing risk in anautomated way as early as possible.

Perfecto enables Continuous Quality throughits CQ Lab in the cloud.


Continuous Quality Lab

Development Languages

IDE

The Continuous Quality Lab (CQ Lab) is therequired infrastructure and tools needed toconduct tests, measurements and analysis. Itallows teams to be fully integrated into theAgile / DevOps process and replicate the enduser conditions early in the life cycle. The CQLab supports 24/7 continuous operation witha production grade SLA, and consistencyacross the different phases of the DevTestlifecycle, minimizing the time it takes toresolve issues while speeding up time tomarket.

C#, Java, Objective-C, Swift, Ruby, Python.Most of these are supported by Selenium.

Integrated Development Environment, anenvironment used by software developers todevelop, debug, build and deploy theirapplication. The common IDEs in the mobilespace are Xcode for iOS apps (MACenvironment), Android Studio (Google Toolsuite), Eclipse (main IDE for Android), TFSand InteliJ.

The CQ Lab is Perfecto’s core offering.

As part of the Perfecto Remote WebDriverimplementation, Perfecto offers full supportfor all the Selenium languages.

Perfecto supports the Eclipse IDE andMicrosoft Visual Studio for test codedevelopment as well as the Perfectoweb-based proprietary IDE.


Kanban

SCM

Kanban is a method for managingknowledge work with an emphasis onjust-in-time delivery while not overloadingteam members. With this approach,the process—from definition of a task to itsdelivery to the customer—is displayed forparticipants to see and for team members topull work from a queue. Kanban in thecontext of software development can mean avisual process management system that tellsdevelopers what to produce, when toproduce it and how much to produce.

Source configuration manager is thetask of tracking and controlling changesin the software, part of the largercross-discipline field of configurationmanagement. SCM practices include revision control and the establishmentof baselines. If something goes wrong,SCM can determine what was changedand who changed it. If a configuration isworking well, SCM can determine how toreplicate it across many hosts.


Shift Left

Software Build

TDD

A trend where dev teams and testing teamscollaborate as one to deliver better softwarefaster through CI/Agile and early testingthat utilizes efficient test automation,performance and unit testing.

The result of a compilation of software code.In mobile, it can be an Android app (.APK) oriOS app (.IPA) that can be executedon the target platform. In other words, asoftware build is a binary artifact that can bedeployed and executed on the targetplatform (device, browser).

Test-Driven Development (TDD) is asoftware development process that relies onthe repetition of a very short developmentcycle. First, the developer writes an initiallyfailing automated test case that defines adesired improvement or new function; thenthe developer produces the minimumamount of code to pass that test; and finallyre-factors the new code to acceptablestandards.


Waterfall The waterfall model is a sequential designsoftware development processes in whichprogress is seen as flowing steadilydownwards (like a waterfall) through thephases of conception, initiation, analysis,design, construction, testing, production/implementation and maintenance.

REQUIREMENTSProduct Requirements Document

Software Architecture

Software

DESIGN

IMPLEMENTATION

VERIFICATION

MAINTENANCE


DevOps A culture or practice that emphasizesthe collaboration and communication ofboth software developers and other ITprofessionals to automate the processof software delivery and infrastructurechanges. The purpose is to establish aculture and environment where building,testing and releasing software can happenrapidly, frequently and more reliably.

DEVELOPMENT

OPERATIONS TESTING

DevOps


CommonTesting Terms

GitHub

Key PerformanceIndicators (KPI)

GitHub is a web-based Git repository hostingservice. It offers all of the distributedrevision control and source codemanagement (SCM) functionality of Git aswell as its own features. Unlike Git, whichis strictly a command-line tool, GitHubprovides a web-based graphical interface anddesktop as well as mobile integration. It alsoprovides access control and severalcollaboration features such as bug tracking,feature requests, task management and wikisfor every project.

Measurable metrics indicating theresponsiveness and availability of key stepsin the user flow, typically measured in aperformance test.

Developers use GitHub to get code samplesand learn about community best practices(including Selenium, Appium or othertechnology-based shared code).


MTTK

MTTR

Persona

Stack Trace

Mean Time to Know – This is a measurementof the time it takes a monitoring/ops/DevOps team to be notified about defects,performance degradation or outages.

Mean Time to Resolution is the time it takesfrom the moment an outage/productiondefect is identified until it gets resolved andfixed in production.

A detailed definition of an app user. Personasare defined by unique traits such as location,devices, networks, background apps, and more.

Programmers commonly use stack tracingduring interactive and postmortem debugging.Users may see a stack trace displayed as part ofan error message, which the user can thenreport to a programmer.

A stack trace allows tracking the sequence ofnested functions called—up to the point where

Perfecto offers production monitoring 24x7 toreduce the MTTK. Because monitoring teamsget immediate real-time alerts as issues areidentified, faster MTTK helps decrease theMTTR (Mean time to resolution).

Perfecto offers production monitoring withdebugging information, allowing teams toeasily pinpoint the root cause of the issueand resolve it.

Perfecto’s Wind Tunnel enables persona-based testing as part of its testautomation solution.


the stack trace is generated. In a postmortemscenario, this extends to the function where the failure occurred (but was not necessarily the cause). Sibling function calls do not appear in a stack trace.

Stack Trace Cont.

Stack Overflow A website that serves as a platform forprogrammers to become members and activelyparticipate. Members ask, edit and ratequestions and answers about programmingtopics in a fashion similar to a Wiki or Digg.Stack Overflow users can earn reputation points and “badges.”


About Perfecto:

Perfecto enables exceptional digital experiences. We help you transform your business and strengthen every

digital interaction with a quality-first approach to creating web and native apps, through a cloud-based test

environment called the Continuous Quality Lab. The CQ Lab is comprised of real devices and real end-user

conditions, giving you the truest test environment available.

More than 1,500 customers, including 50% of the Fortune 500 across the banking, insurance, retail,

telecommunications and media industries rely on Perfecto to deliver optimal mobile app functionality and end

user experiences, ensuring their brand’s reputation, establishing loyal customers, and continually attracting new

users. For more information about Perfecto, visit www.perfectomobile.com, join our community at

community.perfectomobile.com, and follow us on Twitter at @PerfectoMobile.


Seek Perfection

www.perfectomobile.com

Date post:	06-Feb-2018
Category:	Documents
Upload:	dangdiep
View:	215 times
Download:	1 times

Mobile DevTest Dictionary -...

Documents