Share this eBook!

2

TABLE OF CONTENT

THE IMPORTANCE AND DIFFERENCE FROM MOBILE APP TESTING 4

6 KEY AREAS IN MOBILE GAME TESTING 10

UI AND FUNCTIONALITY + IMAGE RECOGNITION 14

APPIUM EXAMPLE ON CLASH OF CLANS (WITH CODE) 16

GRAPHICS PERFORMANCE AND UX WITH TEST AUTOMATION 20

SIMPLE STEPS TO START DOING PERFORMANCE TESTING 25

TESTING FOR CONNECTIVITY 27

QUICK CHECKLIST FOR TESTING CONNECTIVITY 30

CONCLUSION 32

Share this eBook!

3

INTRODUCTIONW ith the proliferation of mobile gamers in the past few years, mobile gaming

industry (and game development) is booming rapidly. There comes the massive

emergence of small gaming studios, aiming to develop a million-download mobile game

to acquire a share of the lucrative gaming market.

We have seen here at Testdroid how certain Android and iOS games have been developed

from the scratch, tested, published, and how those games have quickly become highly

popular among gamers.

Given the experience of servicing mobile gaming developers for a long time, we would

like to share our insight and knowledge of the best practices when it comes to mobile

game testing. As this is typically very different compared to mobile app testing, here we

have compiled what we know into a thorough guide that discusses about how to win in

mobile game testing.

With this ebook in hand, you will become more familiar with the differences between

app testing and game testing and how to leverage test automation in your mobile game

testing.

Share this eBook!

4

CHAPTER 1

The Importance and Difference from Mobile App Testing

Share this eBook!

5

M obile game testing differs from the regular mobile app testing. Effective mobile

game testing derives from a well-structured and systematic approach, use of

test automation framework and seamless integration with your agile process. Naturally,

the simplistic view of testing is to identify bugs and help developers remove them.

However, to aim for efficient, result-driven testing it is mandatory to have test automation

as a part of the overall development process.

TWO FORMS OF MOBILE TESTING

Generally, there are two different forms of testing that can be categorized as Black-

Box testing and White-Box testing. Their definitions are well known in software testing

world but, process-wise, aren’t different with game testing. Only their goals are slightly

different in game testing:

Share this eBook!

6

Black-box testing – focuses on the functional

and overall playability aspects of the

game. In this type of testing, for example,

Methods in Mobile Game Testing the

graphical user interface, user experience

or any visual appearance is in the key role.

Menus, graphical elements, special effects,

animations, and the actual gameplay are

those under test with Black-box approach.

White-box testing – focuses on the

architectural, integration and systemic aspects of mobile game: how third-party

components, databases, social media/external entities, as well as graphics/game engines,

audio play and so on are integrated in to your game.

In the test automation context, black-box testing

focuses on controls, capabilities, features, gameplay,

and automation of the testing flow. For this, it is typically

recommended to build test scripts that know how

game is progressing, what kinds of faulty states could

there be, and also focus on image recognition. As many

graphics cannot be recognized as control mechanism

– buttons, menus, sliders etc. – image recognition will

help to identify those that are meant for controlling

(and not just as a graphical element).

Test automation with white-box approach focuses on testing the integration aspects of

the game e.g. how game utilizes social media and how integration works.

Share this eBook!

7

METHODS USED IN MOBILE GAME TESTING

The testing methodology and used forms of game testing aren’t too different from

application or other software testing. The most important ones when it comes to mobile

game testing are as follows:

Functional testing is probably the most common method in mobile game testing. Typically

functional testing is associated with manual testing and playing ‘game through’. However,

to be agile, all functional testing should be automated. Functional testing – with help

of test automation frameworks - requires some basic understanding of programming

and setting up environment for testing. Automated functional testing can reveal issues

related to user interface (and graphics), stability, game flow/mechanism, and integration

of graphics assets.

Share this eBook!

8

Compatibility testing is all about making that game compatible across different devices.

The typical misconception with compatibility testing is that it will be only done with the

final version of the game. If compatibility testing is done during

all development phases – known also

as smoke testing – it can help to reveal

incompatibility issues with any parts of

the game, its third-party components or

integrations with those actual devices that

end-users use. This is very important to get the

game working robust and well across all possible

device variants – hardware and software.

Performance testing is important for game testing.

Sluggish and badly performing games can ensure

that game won’t be successful, gets bad ratings and

won’t make its creator name. Related to performance

testing it is important to understand how used device

ecosystem varies and what are actual requirements of the

game for its users. Many games are by default developed

(and tested) on high-end devices, without any chance to get

decent performance level on low-end devices.

Localization testing becomes highly important when your game is

targeted for the global markets. The word ‘global’ means your game needs to be ‘local’ for

everyone. When your game titles, texts and content needs to be translated and tested with

devices in multiple languages, these types of tests can be easily performed automatically

(with help of cloud-based device access and test automation) to change the language of

devices, and even do it again with the same test runs. The problems layouts can be easily

picked from screenshots.

Share this eBook!

9

Load testing tests the limits of a

system, such as the number of

players on a server, the graphic

content on the screen (e.g. frames

per second, FPS), or memory

consumption (allocation and de-

allocation of it). Load testing can be

built seamlessly in test automation

context and heavy loads of data, usage or

interactions can be simulated with test scripts

and real content form backend servers. In this category

you can count in multi-player testing as it can involve several

different scenarios to be tested.

Regression testing needs to happen when anything changes in your software.

Most of the mobile games today have a server-client interaction, requiring a

login, uploading of data (e.g. results) and downloading of data (e.g. data, images).

If you’re developing these kinds of services around your game, users need to

ensure all their changes are done in code, to either server side or client side do

not break the functionality of the service. Automated testing is really the only

option in this case, while manual doesn’t cut it.

Share this eBook!

10

6 KEY AREASIN MOBILE GAME TESTING

#1: USER INTERFACE AND FUNCTIONALITY

The User Interface and its overall functionality have a direct impact on how successful

your game will be. These two things, including visual appeal and the gameplay are

the most important things to get right – and ensure device fragmentation doesn’t break

any of these. In the UI various things needs to be tested:

UI layouts and elements: All games are typically targeted for

maximal set of different screen resolutions and different types of

screens. Regression testing should be done each and every time

when UI layout changes to ensure that game works.

Menu structure and functions: Testing menu structured, functionality

and correct behaviour can be automated with instrumentation and

help of different test automation frameworks.

Screen orientation: Surprisingly this gets so wrong with many

games out there. For example, if screen orientation changes during

the game what happens, and what actually supposed to happen?

Does the game work well in both landscape and portrait modes?

Screen resolution: There are lots of screen resolutions especially on Android and typically

auto scale can help game developers. However, it is highly recommended to test your

game across these different resolutions to ensure that the graphics do not get stretched.

Share this eBook!

11

#2: GRAPHICS PERFORMANCE

Performance of Game on Real Device Performance needs to be consistent across all different

device variants that your game’s users have. Because of this it is highly recommended

that you use as many real devices for testing as possible. In order to determine how

well your game responds to various levels of usage, you should consider creating tests,

which last for hours to observe how well the game as well as how much of battery is

used during long usage. To determine if you game will be able to run successfully for a

sustained duration under heavy load, Load/Stress tests can be used. The performance

tests will measure how responsive your game can be on a real device.

For example, take a look at GameBench. It can help game developers to understand those

potential bottlenecks in performance, reveal unused potential with CPU and GPU resources,

and help to push the boundaries of Android gaming and visuals, without forgetting the

most important thing – a need to test the game on real devices.

Share this eBook!

12

#3: USABILITY AND USER EXPERIENCE (=GOOD ENTERTAINMENT)

Testing usability, navigation flow, and eventually what

user experience your game delivers to gamers simply

cannot be done on a desktop with a mouse and keyboard.

So forget emulators and use only real devices for testing.

And to get the best possible understanding of how great

usability and entertainment your game is providing, here

are two important things you should consider when

testing ‘user experience’:

User Interactions and Excellent Responsiveness – It is

important to test the game for its performance, as this

will make either the positive or negative user experience.

For example, performance lags are easy to expose using

actual devices.

Real occurring events on background –

Interrupts, battery consumption, how charger

effects to overall performance and usage. These

all have significant impact on user experience –

and good entertainment level.

#4: MULTI-PLAYER/USER FEATURES

Nowadays, multi-player/user feature is very

common in mobile games. A need to test multi-

player abilities is there and is naturally more

Share this eBook!

13

challenging and requires the other end to perform as a real counterpart. For example, typical

case is that the game communicates with backend server. In these cases, connectivity

is essential and the synchronization between backend and devices that need to get

information about the gameplay. There are tons of different scenarios you should be

testing and many of those will Social Integration severely impact the game’s experience,

resulting in negative feedback and gets uninstalled by users.

#5: SOCIAL INTEGRATIONS

Social network integration is also very

important part of your game. In many games,

it is essential to be able to share results

across your ecosystem, friends or just

for yourself. This should be thoroughly

tested in the context of real Android and

iOS devices, with different OS versions

and different device configurations,

to understand the functionality and

easiness-of-use.

#6: SECURITY AND LIABILITIES

Nearly all of the game developers use some open source components as part of their game.

This is highly recommended and well-accepted development practice as it offloads the

task of developing code for non-core functions in your mobile game. However, identifying

third-party code, its vulnerabilities, and license restrictions are very often neglected by

game developers.

Share this eBook!

14

CHAPTER 2

UI and Functionality +Image Recognition

Share this eBook!

15

Speaking of mobile game testing, it is a common misconception that manual testing

is the only way to go forward. In many cases it is the first thing to start with, but to

really get all issues spotted out and fixed before the game is published, test automation

built into the process, hammering each and every regression and advancement can yield

significant results when considering the customer-ready game.

Let’s face the fact: manual testing has still too many disadvantages as it’s way too time-

consuming, tedious, error-prone and not being able to thoroughly, systematically do the

testing of whole software entity. Manual testing is somewhat like scratching the surface

and you can’t really find out what’s going on underneath the UI. Besides, you would need

an army of programming-capable guys to debug all bits and pieces – something that test

automation can deliver you 24/7 – without any manual efforts.

UI AND FUNCTIONAL TESTING – WITH APPIUM

In a nutshell, Appium is a mobile UI testing framework

supporting cross-platform testing of native, hybrid

and mobile-web apps for iOS and Android. In

fact, Appium is a pretty good choice for mobile

games as in many cases those games tend

to be identical – or at least very similar – on

both platforms, Android and iOS – and the

same test script can apply for both. But there

are also some other reasons why Appium

is a great choice for mobile game testing.

For example, you can write tests using your

favorite development tools/environment and

programming languages, such as Java, Objective-C,

JavaScript, PHP, Ruby, Python, C# and so on.

Share this eBook!

16

Appium enables you to execute your tests on mobile device irrespective of the device OS.

This is because the framework is basically a wrapper that translates Selenium Webdriver

commands into UIAutomation (iOS) or UIAutomator (Android, API level>=17) or Selendroid

(Android, API level <=16) commands depending on the device type. For example, in the

context of Android, this is how Appium compares to other test automation frameworks:

EXAMPLE: USING APPIUM TO TEST CLASH OF CLANS (GAME BY SUPERCELL)

We took Supercell’s Clash of Clans as an example to give you a clear picture. A fantastic

game and I bet many of you have played it so you should be pretty familiar how the game

looks and so on. We’re also going to use Appium as a selected test automation framework

to basic clicking-through of Clash of Clans tutorial.

Share this eBook!

17

#### Example script that tests Clash of Clans tutorial first steps#### Works on different resolutions, both iOS and Android##import unittestfrom time import sleepfrom TestdroidAppiumTest import TestdroidAppiumTest, logfrom selenium.common.exceptions import WebDriverExceptionclass ClashOfClansTest(TestdroidAppiumTest): def setUp(self): # TestdroidAppiumTest takes settings (local or cloud) from environment variables super(ClashOfClansTest, self).setUp() def test_tutorial(self): driver = self.get_driver() # Initialize Appium connection to device sleep(10) # Wait that game loads # Use this to get detected screen hierarchy # print self.driver.page_source # Dismiss the in-app purchases dialog if it shows okayButton = None if self.isAndroid(): try: okayButton = driver.find_element_by_id(‘button3’) okayButton.click() sleep(5) except WebDriverException: log(“There was no in-app purchases dialog”) else: # iOS self.driver.implicitly_wait(5) # wait only 5 seconds to find it try: okayButton = driver.find_element_by_accessibility_id(‘Okay’) okayButton.click() # No need to sleep here since for iOS we wait the Game Center to popup... except WebDriverException: log(“There was no in-app purchases dialog”) self.driver.implicitly_wait(30) # Cancel iOS Game Center login if self.isIOS(): #print self.driver.page_source try: self.driver.implicitly_wait(5) cancelButton = driver.find_element_by_accessibility_id(‘Cancel’) log(“Canceling iOS Game Center login...”) cancelButton.click() sleep(2) except WebDriverException: log(“The Game Center login was not displayed”) self.driver.implicitly_wait(30) self.screenshot(“welcome-chief”) # Check that there is a goldmine on screen rect = self.find_image(“queryimages/tutorial_goldmine.png”, screenshot_match=”screenshots/goldmine_match.png”) self.assertIsNotNone(rect, “There should be a goldmine on screen in beginning of tutorial”) log(‘Gold mine found at %s %s! Tapping tutorial forward...’ % (rect[0], rect[1])) # Dismiss the bubbles self.tap_middle()

Share this eBook!

18

sleep(2) # second blabla self.tap_middle() sleep(2) # Goblin appears self.tap_middle() sleep(1) # Go to shop # NOTE: tap_image does also assert, fails test if target not recognized self.tap_image(“queryimages/shopbutton.png”) sleep(1) # Buy cannon self.screenshot(‘cannon’) self.tap_image(“queryimages/cannon.png”) sleep(2) # Place the cannon self.screenshot(‘place_the_cannon’) self.tap_image(“queryimages/place_the_cannon.png”, width_modifier=0.75) sleep(2) self.screenshot(‘finish_now’) # Use gem to finish right away self.tap_image(“queryimages/finish_now.png”) sleep(3) # Bring it on! self.screenshot(‘bring_it_on’) self.tap_image(“queryimages/bring_it_on.png”, height_modifier=0.75) sleep(10) self.screenshot(‘battle’) sleep(10) self.screenshot(‘end_of_battle’) # To be continued...if __name__ == ‘__main__’: unittest.main()

Let’s look some stages in this script:

1. It first figures out if test is executed either on Android (self.isAndroid()) or iOS. As you can see, it looks content different, on Android it is trying to find by element ID and on iOS by accessibility ID with description (‘Okay’). The same check happens for iOS Game Center login.

2. Screenshots are taken in various steps and stored in files entered as a parameter in a function call.

3. We do a check if “goldmine” exists on screen by comparing two pngs using self.find_image call. If these pictures match (=goldmine exists on screen), we’ll go forward with tutorial.

4. We proceed with the tutorial with the following steps: 1) Go to shop, 2) Buy cannon, 3) Place the cannon. The information about all these three items is stored in those pngs: shopbutton.png, cannon.png, place_the_cannon.png.

5. Finally, we finish the tutorial and start the battle! After battle, application is brought down.

Share this eBook!

19

Above is the programming and scripting. In the real practice, we used one iOS (iPhone

4S) and two Android phones (Samsung Galaxy S3 Mini and HTC One X) for this script. If

you are interested, you can take a look at the video at Example of Test Automation using

Appium.

HOW ARE WE USING IMAGE RECOGNITION?

This Appium + Clash of Clans example was the basic image recognition flow for enabling

mobile game to be tested on real devices, regardless of different OS (Android and iOS).

There are different ways to recognize content from images. For example, template matching

is a technique for finding small parts of an image, which match a template image. It

is actually very handy even for recognizing UI elements and graphics resized and/or

rotated in different form. This is one way how some Testdroid game developers use

image recognition today.

Let’s say the template image has some strong features – e.g. text easy to be abstracted

from the background content – a feature-based approach can be used.

In this example, “Button 1” text was resized and rotated (or if it is otherwise transformed)

this can be quickly and easily identified and further actions can be taken.

Share this eBook!

20

CHAPTER 3

Graphics Performance andUX with Test Automation

Share this eBook!

21

Good performance is very closely

related to good user experience.

User wants to see constant progress with

the game, do the smooth game-playing,

graphics performance needs to be up to

snuff and work across all different mobile

devices (phones, tablets etc.) the very same.

Mobile games require performance testing

before published to app markets. What type

of performance testing is needed varies by the

nature of game, graphics, gameplay, multiplayer

functionalities and many other things. The most

typical performance tests done for mobile games

are general tests performance to determine how

the game performs in terms of responsiveness,

stability and refresh rate under certain workload or

user interaction pattern.

The resource usage is important metric with mobile

games as the power of CPU and GPU and the size of memory varies device by device.

High-end devices are more capable to run anything whereas low-end devices can’t even

get the game up and running. This is not exceptional case that game is built for a high-

end device, published and then gets complained by thousands of users that it can even

get in the first loading state.

Remember! Use test automation and you’ll get your game’s performance tuned to its

best. Manually, you can’t get any sort of understanding what happens underneath the

UI/front-end/controls.

22

Share this eBook!

DIFFERENT TYPES OF PERFORMANCE TESTS

Performance test – this is the basic type of performance testing that provides information

how quickly something is executed, how much it can be scaled (in terms of more

graphics assets, or more connections/integrations) or how stabile the performance is

in general. Typical metrics from performance tests are CPU load, memory consumption

and frames per second (FPS). Based on this data game developers

should further improve the game to either scale down

Performance of Game certain functionalities, heavy

graphics assets or something else to ensure

game works on lower-end devices as well.

With performance tests it is easy to fine-

tune, further plan improvements on

graphics, gameplay and connections,

and to optimize game accordingly. One

interesting ‘benchmark’ for measuring

the user experience of your game

is Gamebench, the least cheatable

benchmark you can find out there now

for mobile games, hardware and so on.

Load testing – is somewhat the simplest form

of performance testing. When certain load is

exposed to the game and infrastructure the developer

gets understanding of where possible bottlenecks are and what

parts of game/infrastructure need optimization. Load testing can vary from basic server

upload/download function to database connection and queries, and different types of

inputs given to the game/infrastructure. During load testing developers typically get better

Share this eBook!

23

understanding of concurrency issues related to the game and how those could be optimized.

For example, if your game needs to pull something from back-end server and suddenly

300+ devices do that in the same WiFi network, you can get very invaluable information

about your games’ capability during this kind of scenario. Furthermore, load testing can be

you to understand how game needs to ‘compromise’ when certain operations cannot be

handled. Also, it typically gives great insights to hardware capabilities and how much real

resources (CPU, Memory

etc.) are needed for your

game and what happens

when those resources are

exceeded.

Stress testing – determines

if the system is able to

handle everything, how it

handles with data when

the game infrastructure

is overstressed, and how

far can stress testing

go before game stops

responding and how are

errors shown to users.

Stress testing is important in case of games especially if you game requires connections

to servers or needs to be connected to some other device. Typically not so many mobile

games communicate everything instantly online but rather in parts. This limits the recurring

use of networks, draining less battery and limits error scenarios, as your game doesn’t

need to be send/receive something all the time. The stress testing helps you to understand

what kinds of failures are the most valuable to plan for and what are the most common

hardware failures or issues that you need to anticipate.

Stress testing in progress.

Share this eBook!

24

One of the subcategories for stress testing is soak testing. It’s also known as endurance

testing and is used to understand if the game and infrastructure can sustain certain

continuous load set on it. When soak testing is done, typically CPU, Memory as well as

FPS and other important metrics are monitored to detect any potential leaks and issues.

The goal of soak testing is to discover how the system behaves under sustained use.

Some industry benchmarks can be seen as a soak tests, however, users should be aware

of cheatable nature of those.

The other type of stress test is called spike testing which is done

by suddenly increasing the load generated by a very large

number of users, data, inputs, or any other type of

stimuli. This is not continuous but periodical

and the thing is to find out if dramatic

change in load will cause any

errors or issues for the game

and its infrastructure. The goal

with spike testing is to determine

if performance will suffer, the

infrastructure will fail or how can

those error states be handled in case

of dramatic changes of inputs.

Share this eBook!

25

Simple Steps to Start Doing Performance Testing

1. Write and conduct performance tests in whatever types of tests make the best use for your game. Make sure you stress the game itself but all infrastructure related to it as well. This means back-end servers, integrations with social media, and any other third party or external entity. All these things have an impact on performance of your game and through the performance to its user experience. You can even use Record-Playback approach to generate your simplistic performance tests!

2. Input everything to logs! This is very important and will help you to further investigate what happens in your game – even you would be monitoring it in real-time during the testing. Logs will maintain all that information and you can easily compare those later – even build-by-build – to see how things have regressed while building a new version of game.

3. Use only real mobile devices – and as many as possible. You’ll get quickly understanding of how your game performs when you start doing basic smoke testing on hundreds of different devices. While running your game on those real devices, check at least these things:

• The game’s graphics and UI – does the game display/screen show properly (and similarly) on all these devices?

• Do the UI and layout and other graphical elements (toolbars, buttons, icons etc.)

display correctly (and the same way) on all devices?

Share this eBook!

26

• Is usability of controls (buttons, menus, boxes, what) identical in all devices? Some

OS versions have different GUI look’n’feel and controls so this important to check

across all possible devices out there.

• Is navigational / game progress flowing smoothly enough? How much delays are

there between “game stages”? Again, try with a high-end and low-end device to

understand the difference/gap between top and not performing device.

• Animations – are those flowing well? This is about to change with Android L for

native apps but doesn’t have an impact on mobile games. Remember! Animations

require some performance

capability from the device. It

is important to test this in the

context of low-end devices.

• Fonts – do the device where your

game will be executed have those

fonts you meant there to be – or

not?

• Screen Resolution – one old

root cause for fragmentation. Try

to support all possible screen

resolutions and have those tested

on real devices. Resolutions “are

not identical” when OS version

is different as for example

Kitkat handles overlapping UI

notifications differently than Jelly

Bean, Icecream Sandwich or

Gingerbread.

• Screen Orientation – what

happens – if anything – when user

rotates the screen? Is your game

meant to be rotatable?screen resolution problem.

Share this eBook!

27

CHAPTER 4

Testing for Connectivity

Share this eBook!

28

C onnectivity is one of the key aspects when you build mobile games. Typically, mobile

games communicate over the internet with various different entities; app markets

(e.g. Google Play), directly with other players and their devices, or your own back-end

servers. When it comes to mobile game development, both indies and major game studios

are struggling to make an ideal environment for testing massive multiplayer setup. This is

one of the areas where test automation can provide you significant benefits.

Test automation can help when

it comes to multiplayer testing,

cross-platform testing and general

connectivity testing - not only

spotting out those bugs and issues,

but also significantly reduces the

cost of failure, saves time through

its repeatability and earlier

verification, and leverages resource

productivity. Using test automation

for mobile game testing already in

the earliest phase of development

is a good way to gain those benefits

and effectively use resources.

But you also need a good

infrastructure for doing efficient

testing for connectivity of your game. Naturally, there are plenty of options where and how

testing will be conducted, but let’s take two the most common ones as an example – in-

house environment and cloud-based setups. For setting up an in-house test environment,

Share this eBook!

29

you will need servers, USB hubs, plenty of real mobile devices (phones, tablets etc.

whatever you plan to support with your game), appropriate speed WiFi, and so on. Here

is a good tip and checklist what you need if you built this internally.

Using a cloud-based setup is easy and effortless, and

you can use various different techniques to test

the connectivity aspects of your game. For

example, different types of performance

tests can be used. However, the most

important thing in cloud-based

testing is that you have plenty of

devices in use – as versatile set

of different OS versions, chipsets,

display sizes, memory and other form

factors. This enables the simultaneous

smoke and compatibility testing when you do connectivity

and performance testing for your game.

Whether deciding to go with either of these options, WiFi

infrastructure is another very important infrastructure area

that is often overlooked when creating large-scale mobile test automation environments.

You can get to about 10 maybe even 15 devices with any WiFi infrastructure without

any problems but as the number of devices in your WiFi network adds up, so do the

challenges. Those issues will start appearing when data is transferred. This gives the game

developer an excellent way to optimize the data passing, infrastructure software and

decide what data should be passed between servers and clients. Not everything needs

to be communicated/sent back and forth. In general, it is a good rule of thumb that if

WiFi cannot handle 15 devices sending all data between server and client (your game),

you should seriously consider optimizing the data passing.

Share this eBook!

30

Quick Checklist for Testing Connectivity

1. Optimize the data communication between servers and clients. You can set your

mobile game to fetch or pull a large amount of data from your back-end server.

Especially if you plan to build a MMO (massive multiplayer online) game you need to

optimize every piece of your data communication. Despite unlimited data plans are

pretty common nowadays, you should still for sake of

user experience try to optimize all possible data passing.

2. Test simultaneously on several devices. This is only

possible by using real devices to see if the game states

are properly synchronized on all devices and all device

resources (players/users in real life) have equal access

to all game resources and that there is no lags in game

response, inputs or outputs, for all players. Test automation

can be a huge help with this, as test scripts can be made

‘intelligent’ and play against each other. For example, with

image recognition feature you can put game playing against

another instance, on another devices. Take a look at here.

3. How does game behave when online and offline.

When internet is available game might work fine. If

your game is designed to work online all thev time,

you should plan how to fall back when device goes

Share this eBook!

31

offline. This can be easily ‘simulated’ on real environment with real devices for

example cutting down the WiFi or stop communicating with back-end server.

Typically today’s mobile games can be played offline and later on game status

with all details are synchronized with the game centers, servers and so on.

4. Check for interruptions. In addition to online/offline gameplay, there are many other

interruptions for games that can happen while game is played. For example, incoming calls,

device or server running out of memory, the bugs/failures in game itself causing a state

where gameplay doesn’t advance. Actually all these are related to connectivity testing as

for example network connection can switch (WiFi<->Cellular) and cause an interruption.

5. Test for cross-platform compatibility and possible differences. If your mobile game is

a cross-platform one (e.g. you have identical game for Android and iOS) naturally the

use of cross-platform test automation

frameworks is great news for you.

Regardless, if there are small differences,

you can still use these frameworks.

6. Test for location-based details. If your

game is using location-based services

(e.g. just pulling out the info in which

country you are in), you should test for

this feature. With test automation you

can quickly set up a device with a specific

language/country settings to match

localized device / environment / user.

7. Combine different forms of testing. While you are doing connectivity testing, you

can also do the instant smoke testing how your game works across different devices.

Again, with test automation this is easy, effortless and provides you quickly results on

how the game performs.

Share this eBook!

32

ConclusionM any things make a difference when mobile gamers select which game they download

from app market. Naturally, the reputation of any game developer is important, but

by far more important factor for the maximal exposure for hundreds of millions of users

are those user ratings and comments given in Google Play or App Store. Those either put

the game in front of those downloaders – or not.

If your game crashes, is otherwise buggy, doesn’t load quickly, or user experience isn’t up

to snuff, users will leave your game. And in most of the cases they go away angry, writing

negative (=justified) feedback about the game in reviews.

WHAT HELPS YOU BUILD BETTER GAMES?

In the conclusion section, we would like to share a list of insights and new aspects that

can help you build a more robust Android game for your customers.

1. Use Record & Playback approach to test your game

Game engines – such as Unity3D – are awesome and

enable you as a developer quickly to take the weight

off from developing graphic algorithms and routines to

focus on game and gameplay itself. Many of those game

engines and complementary tools provide primitive tools

to test your game before releasing it. However, not any

single of those game engines provide a test automation

Share this eBook!

33

approach to the problem, capability to record user interactions and then playback those

on real devices – the very same devices that those feature-savvy gamers use.

Testdroid Recorder has been very popular and useful among Android app developers,

and now supports record-playback approach with Unity3D games.

2. Test localization and make your game work every corner of the world

Diversity is a great thing, but even better in mobile app testing. For many mobile developers

it is not possible – not at least without travelling to specific country and acquiring devices

– to access those devices that are used in these different countries. For example Japan,

the most lucrative market for mobile games. Not the paid-apps, but the largest amount of

revenue is generated from freemium apps in Japan. This yields about serious game-playing

going on. And your game gets one shot – are

you ready to confront those gamers without

testing on their devices?

Well, the great news is that you can access all of

those popular Android devices at Testdroid Cloud,

instantly and any time of the day. Test localization

using real devices from these regions, and your

game will be surely more robust!

3. Test performance: app & activity launch

times – and response times

Consistent performance across all different software/hardware combinations that

those end-users use is naturally one of your top targets. As many things make an ideal

Android device for gamers – such as processor, GPU, display resolution, OS version,

OEM customizations, and dependencies to other SW/HW – not all those devices are

Share this eBook!

34

capable to run all games without problems. Typically, input, controls, graphics performance

among many other things either make your game usable across those devices, or not.

Make sure you conduct performance testing in earlier phase of the development to avoid

bottlenecks in your game!

With the help of Testdroid Cloud, you can instantly run your Android game on all relevant

devices and ensure 90-95% coverage of global Android users. Most likely, by using

Testdroid Cloud device roster, you are close to 100% when it comes to real gamers

arsenal of devices.

4. Testing payments, Google Play transactions

In-app purchases are in steep growth and every

game developer is looking this way to enable

payments through their app. Naturally, there are

easy ways to implement in-app purchases using

Google APIs and also some external providers can

help you to get up and running with this.

We’ve now opened a way to test payments at

Testdroid Cloud.

5. Make sure all works well: Layouts, rendering,

installations, and UX

Developing games for mobile is totally different ball game than developing for web or

even for consoles. Game engines and complementary tools provide an excellent way to

create your game on desktop and quickly test it on actual device. But again, performance

capability of device and many things associated with it can make your game run very

Share this eBook!

35

differently on those devices. That is not something that can be tested on emulated or

simulated environment.

All Testdroid products can help you to make your game super robust – and it has worked

for so many game developers building the most popular games for Android.

6. Test for security, liability and 3rd part component safety

Remember Heartbleed? Just

one example of why everything

about third party or open

source components included

in your game must be tested for

security. Security is important

but equally important are

liabilities and safety aspect of

your game.

With the help of Testdroid Cloud

and Appcheck you can quickly

check what sort of components

are involved and do you need

to act before publishing your

app at Google Play.

Share this eBook!

36

Thank you for reading this e-book!

ARE YOU ALREADY FAMILIAR WITH TESTDROID RECORDER? IF NOT CHECK IT OUT NOW. IT’S FREE.

This great tool will provide you a way to generate Robotium based scripts for your

Android apps and games. And, we’re supporting Android L, Unity3D and many other

things with it! What a great tool for Android game, app and service developers!

FREE