Post on 22-Jun-2020
transcript
Test Cases for Windows Applications Windows 10 Universal Windows Platform – Desktop and Mobile device families
Windows 8.1 Windows and Windows Phone
April 2016
Contents Introduction ........................................................................................................................................................................ 3
1. Launch-related Test Cases ......................................................................................................................................... 3
1.1 Low spec devices .................................................................................................................................................... 3
1.2 Launch on all required device families ............................................................................................................. 4
1.3 Multi-regional launch ............................................................................................................................................ 4
1.4 Offline launch .......................................................................................................................................................... 5
1.5 Launch time ............................................................................................................................................................. 5
2. Log In Test Cases .......................................................................................................................................................... 5
2.1 Offline log in ............................................................................................................................................................ 6
2.2 Invalid credentials .................................................................................................................................................. 6
2.3 All log in methods are functional ...................................................................................................................... 6
2.4 Switching away after logging in ........................................................................................................................ 7
2.5 Account creation ................................................................................................................................................... 7
2.6 Account switching ................................................................................................................................................. 7
3. Application Use Test Cases ........................................................................................................................................ 8
3.1 App Switching ......................................................................................................................................................... 8
3.2 Software Input Pad implementation ................................................................................................................ 9
3.3 Touch targets .......................................................................................................................................................... 9
3.4 Visual feedback ...................................................................................................................................................... 9
3.5 Gestures ................................................................................................................................................................. 10
3.6 Secondary tiles ...................................................................................................................................................... 11
3.7 Command bar with contextual commands ................................................................................................... 11
3.8 Application is stable ............................................................................................................................................ 12
3.9 Application performance is responsive ......................................................................................................... 12
3.10 Toast notifications are actionable ................................................................................................................. 13
3.11 Offline mid-use scenarios................................................................................................................................. 13
3.12 Saved status preservation ................................................................................................................................ 14
3.13 Portrait and landscape mode scenarios ...................................................................................................... 14
3.14 Core scenarios completed within the application .................................................................................... 15
3.15 In-App purchases............................................................................................................................................... 15
3.16 Geofencing .......................................................................................................................................................... 16
3.17 Cross-Platform adaptive layout ..................................................................................................................... 16
3.18 Application is feature complete ..................................................................................................................... 17
3.19 App Resume ........................................................................................................................................................ 17
3.20 Readability ........................................................................................................................................................... 18
3.21 Placeholder text and imagery ........................................................................................................................ 18
3.22 Application specific hardware ........................................................................................................................ 19
3.23 Cortana ................................................................................................................................................................ 19
3.24 Resuming after idling ....................................................................................................................................... 20
4. Phone-Only Test Cases ............................................................................................................................................ 20
4.1 Application resumes successfully from tombstoned state ....................................................................... 20
4.2 Back button for mobile ...................................................................................................................................... 21
4.3 Continuum for mobile ........................................................................................................................................ 21
5. Windows-Only Test Cases ....................................................................................................................................... 22
5.1 Split and windowed views and display resolutions ..................................................................................... 22
5.2 Application resumes from a suspended state ............................................................................................. 23
5.3 Back button for desktop ....................................................................................................................................24
5.4 Desktop and tablet mode switching ..............................................................................................................24
5.5 Multiple input methods ..................................................................................................................................... 25
5.6 Forward and backward navigation ................................................................................................................. 25
6. Social Networking Integration Test Cases ........................................................................................................... 26
6.1 Shared content ..................................................................................................................................................... 26
6.2 Share targets ......................................................................................................................................................... 26
7. Game Specific Test Cases ......................................................................................................................................... 27
7.1 Game leaderboard ............................................................................................................................................... 27
7.2 Game achievements and scores ..................................................................................................................... 27
7.3 Game multiplayer support ................................................................................................................................ 28
7.4 Roaming Game Saves ........................................................................................................................................ 28
8. Media Specific Test Cases ........................................................................................................................................ 29
8.1 Background audio playback .............................................................................................................................. 29
8.2 Background audio controls .............................................................................................................................. 29
8.3 Video pauses playback when the application is in the background ..................................................... 29
9. .......................................................................................................................................................................................... 30
10. WACK Test Findings ................................................................................................................................................. 30
11. Competitive Parity Test Cases ................................................................................................................................ 30
11.1 Feature parity – Competitive platforms ........................................................................................................ 30
11.2 Feature parity - Website ................................................................................................................................... 31
11.3 Performance parity............................................................................................................................................. 31
Introduction
Application functionality is the foundation of user satisfaction. The tests included here are designed to
test the most fundamental functional scenarios in Universal Windows Platform (Mobile and Desktop
device families) Store applications and games. These tests make no assumptions regarding an
application’s user interface and verify that an application’s behavior is consistent with user
expectations and the Windows design guidelines. The reader is encouraged to incorporate these test
cases into their software development projects to help ensure high user satisfaction – and high user
ratings.
1. Launch-related Test Cases
1.1 Low spec devices
The application should install and function properly on low spec devices.
Expected result:
The application should function properly on low spec devices.
Note: A ‘Low spec’ phone is defined as a 512MB Windows Phone 8.1 device, ex: Lumia 520, and/or a
1GB Windows 10 Mobile device, ex: Lumia 535.
1.2 Launch on all required device families
The application should install and launch properly on all device families the app is designed to run on.
With Windows 10 developers can write a single application that can be installed across a variety of
device families including Mobile, Desktop, and XBOX. By default, Windows 10 applications target all
device families. In the application manifest, this is called out as the ‘Universal’ Target Device Family.
Such applications will be tested against all device form factors where this application may be
deployed. Alternatively, developers can limit the device families for which an application can be
deployed. For example, if ‘Desktop’ is specified as the Target Device Family, that application can only
be installed on PC devices. When published, this application will only be available in the Store running
on PCs – it will not be available in the Phone store.
Steps to Reproduce:
1. Check the supported device families in the application’s manifest.
2. Attempt to install and launch the application on all required device families.
Expected result:
The manifest entry should be restricted only to supported devices when only one device is
supported.
More information:
For more information see DeviceTargetFamily.
Target Device Family Hardware
Desktop PCs and Tablets running Desktop OS SKU.
Mobile Phone and Tablets running Mobile OS SKU.
Team Surface Hub running Team OS SKU.
Universal All
1.3 Multi-regional launch
The application should launch properly on devices where the local username contains Unicode
characters.
Steps to Reproduce:
1. Launch the application on a device where the user local folder path has Unicode characters,
ex: c:\users\캔디\.
Expected result:
The application launches properly on devices where the local username contains Unicode characters.
1.4 Offline launch
The application should be stable and not crash or close unexpectedly if the device is offline. When the
application is offline, the application should warn the user for any requested action that cannot be
completed.
Steps to Reproduce:
1. Turn airplane mode on and wait 10 seconds.
2. Launch the application.
3. Perform an action that will cause a connection to be attempted.
Expected result:
The application will detect that the network is not available and alert the user.
Application should not crash and provide a relevant error message to user. Application should
not display the exception stack to the user.
The application can also implement an offline (cached) mode without alerting the user.
1.5 Launch time
The application should launch in a reasonable amount of time, and should display a loading or
progress indicator if the application takes a while to load.
Steps to Reproduce:
1. Launch the application.
2. Record the start-up time.
Expected result:
The application displays a loading or progress indicator if the application takes a while to load,
and the content loads in a reasonable amount of time.
2. Log In Test Cases
The following test cases apply only to applications that support log in.
2.1 Offline log in
When the application is offline and the user attempts to log in, the application should warn the user
of the offline status.
Steps to Reproduce:
1. Turn airplane mode on and wait 10 seconds.
2. Launch the application.
3. Attempt to log in with valid credentials.
4. Verify that an appropriate error message appears.
5. Turn airplane mode off and wait 10 seconds.
6. Return to the application.
7. Log in with valid credentials.
Expected result:
When attempting to log in without a network connection, an appropriate error message
should appear warning the user of the offline state.
When the network connection has been resumed, log in should work appropriately.
2.2 Invalid credentials
The application should warn the user appropriately when invalid credentials are entered.
Steps to Reproduce:
1. Launch the application.
2. Attempt to log in with invalid credentials.
3. Verify that an appropriate error message appears.
4. Log in with valid credentials, changing the casing in the username field.
Expected result:
When attempting to log in with invalid credentials, an appropriate error message should
appear warning the user that the credentials are invalid.
The username field should not be case-sensitive.
When correct credentials are entered after a failed attempt, log in should work appropriately.
2.3 All log in methods are functional
All available log in options should be fully functional and appropriately implemented.
Steps to Reproduce:
1. Launch the application.
2. Log in using all available options with valid credentials.
Expected result:
Application logs in when valid credentials are entered.
Social networking log in and/or sign up is processed using the services’ login page(s) and not
using application pages for this purpose.
2.4 Switching away after logging in
The application should load content successfully after switching away during the loading process.
Steps to Reproduce:
1. Launch the application.
2. Log in with valid credentials.
3. While the application is loading, switch away from the application.
4. Wait one second and return to the application.
Expected result:
Content should load successfully after returning to the application.
2.5 Account creation
The creation of accounts within the application (when available) should function properly.
Steps to Reproduce:
1. Launch the application.
2. Create an account if the option is available, entering valid information when prompted.
3. Exit the application.
4. Attempt to log in with the newly created account.
Expected result:
Accounts can be successfully created within the application when the option is available.
If an email verification is required, the application should state this and the verification email
should arrive within an hour of creating the account.
Username, email, and password requirements are made clear to the user.
2.6 Account switching
Users should be able to log into various accounts properly when the option is available.
Steps to Reproduce:
1. Launch the application and log into account #1.
2. Navigate through the application, saving information to the account whenever possible.
3. Log out and exit the application.
4. Launch the application and log into account #2.
5. Navigate through the application.
Expected result:
Logging into a second account should function properly after logging out of a first account.
Data from a first account should not appear when logged into a second account.
3. Application Use Test Cases
3.1 App Switching
When switching away from the application, network requests are cancelled. The application should
detect this situation and re-issue network requests when application is reactivated.
Steps to Reproduce:
3. Launch application.
4. Navigate through the application.
5. During loading (when loading indicator is present), switch away from the application.
6. Wait one second and return to the application.
Expected result:
Functionality completes successfully and data is downloaded without error or exception.
More information:
For more information on Fast App Switching in Windows Phone applications, see Get Ready for Fast
Application Switching in Windows Phone and App activation and deactivation for Windows Phone 8
For Windows applications, see Managing app lifecycle so your apps feel "always alive"
3.2 Software Input Pad implementation
The Software Input Pad (SIP) should be contextually-aware of the types of input allowed per input
field and it needs to be dismissed when input field is not selected on an application or when the
application is deactivated.
Steps to Reproduce:
1. Switch the device to Tablet mode and ensure a keyboard is not connected.
2. Launch the application.
3. Find an input textbox that allows alphanumeric characters.
4. Verify when this control has focus (is tapped with touch), the full keyboard and symbols are
displayed.
5. Find an input textbox that allows only numbers.
6. Verify when the control has focus (is tapped with touch), only the keypad SIP is displayed.
7. Move focus out of the input textbox (tap outside of the textbox with touch).
Expected result:
The SIP is dismissed when input controls lose focus.
Users can see what they are typing while they are typing it.
3.3 Touch targets
Touch targets should not be smaller than 7 mm or 26 pixels square.
Steps to Reproduce:
1. Launch the application.
2. Navigate through the application.
3. Identify actionable UI elements on the application.
Expected result:
Actionable UI elements in the application should be clearly visible and easily touchable by finger
without interfering with other neighboring touchable UI elements.
More information:
For more information on the Windows touch language, see Touch interaction design
3.4 Visual feedback
Visual feedback helps users recognize whether their interactions with your application are detected,
interpreted, and handled as intended.
Steps to Reproduce:
1. Launch the application.
2. Explore and navigate the application. Tap on each actionable UI element.
Expected result:
There is visual feedback for actionable or interactive UI elements.
There is no visual feedback for non-actionable or non-interactive UI elements.
Note: Acceptable visual feedback can be a slight change in color and/or movement of the
button when selected by touch or mouse (not when hovering over with a mouse.)
More information:
For more information on visual feedback, see Guidelines for visual feedback
Touch interaction guidance: Touch interactions for Windows
Responding to user interactions: Responding to user interaction
3.5 Gestures
Primary actions should be enabled by directly interacting with content. For example, tap to open,
swipe to select, pinch to zoom, and drag to pan/paginate. If a panorama or pivot control contains
other controls which use the same left / right flick gestures, this may result in a poor user experience
due to gesture overlaps. UI elements, controls or other parts of an application should not interfere
with Windows edge gesture.
Steps to Reproduce:
1. Launch the application.
2. Explore and navigate the pages and functionality of application with touch only.
3. Inspect the application’s panorama/pivot control for scrollable map, horizontal scroll viewer,
toggle switches, sliders, or other controls that use left / right flick (or pan) gestures.
4. Navigate to each page and test the edge gestures for commanding surfaces (Title bar, Task
bar, Action center, Task View). (For Windows 8 apps: app bar, navigation bar, charms bar,
recent app.)
Expected results:
The application should fully support touch input. For example, the user should be able to
navigate or interact with commands using touch input only.
Touch gestures should only be used for actions that match the Windows touch language.
Pivot and/or panorama controls do not have controls with competing gestures and can be
used reliably.
If the hub, panorama or pivot reacts by scrolling then the child control should not trigger.
Any UI elements, controls, or other parts of the application should not interfere with Windows
edge gestures.
More Information:
Gesture interaction guidance: Gestures, manipulations, and interactions
3.6 Secondary tiles
Secondary tiles should enable users to pin context specific content on the Start screen.
Steps to Reproduce:
1. Launch the application.
2. Explore the application and identify functionality (if any) where the user has the option to ‘Pin
to Start’.
3. Select ‘Pin to Start’ to create a secondary tile on the Start screen.
4. Launch the application by tapping the primary tile of the application on the Start screen or
tapping the application name from the application list page.
5. Navigate to a second or third level page in the application.
6. Tap the Start button to return to the Start screen.
7. On the Start screen tap the secondary tile of the application.
8. Exit the application.
9. Launch the application from the secondary tile.
Expected result:
Launching the application from the secondary tile results in displaying content related to the
secondary tile.
On pressing the Back button (if applicable), user exits the application or navigates to the main
page.
More information:
For more information on tiles refer below link: Secondary tiles overview (Windows Runtime apps)
3.7 Command bar with contextual commands
Command bar commands need to be functional as well as predictable and organized so users can
easily find them.
Steps to Reproduce:
1. Launch the application.
2. Explore and navigate the pages and functionality of application.
3. Select items with contextual commands.
Expected result:
All command bar buttons are functional, relevant, and contextual.
3.8 Application is stable
The application should be stable and not crash or hang. The application should not have functional
bugs that prevent completion of core scenarios.
Steps to Reproduce:
1. Launch the application.
2. Explore and navigate the pages and functionality of application.
Expected result:
Application should not crash or hang.
3.9 Application performance is responsive
The application must appear responsive to the user at all times. Visual feedback is needed to indicate
the application is not frozen. Load times may be longer on slow connections resulting in the
application appearing to be frozen if no visual feedback is provided.
Steps to Reproduce:
1. Launch the application.
2. Explore and navigate the pages and functionality of application.
3. Invoke functionality that requires time to load. E.g. network request to load data.
Expected result:
The application displays a loading or progress indicator for tasks that may take time to load,
and the content loads in a reasonable amount of time.
When the content has fully loaded, the loading indicator should no longer be present.
Application should be responsive and fluid.
Transition animations should be smooth.
The application should not display a blank screen, blank tiles, or empty white rectangles.
More information:
Guidelines for progress controls: Guidelines for progress controls
3.10 Toast notifications are actionable
Notifications should be used for time-sensitive or personally relevant notifications. Notifications are an
invite back into the application when it is not in the foreground. Tapping the notification should bring
the application to the foreground and in context to the notification.
Steps to Reproduce:
1. Launch the application.
2. Invoke functionality to create a toast or notification. E.g. push notification or scheduled
reminder.
3. Switch away from the application and/or close the application.
4. Wait for the notification.
5. Select the notification.
Expected result:
When selecting the toast or notification, the application should be brought to the foreground
in the context of the toast or notification.
Toast notifications should not be displayed when the application is in the foreground.
More information:
For more information see Guidelines for toast notifications and Guidelines for toasts and badges
3.11 Offline mid-use scenarios
When the system is in an offline scenario (no cell coverage, no WiFi, airplane mode, etc.), the
application should alert the user appropriately.
Steps to Reproduce:
1. Launch the application.
2. Explore and navigate the pages and functionality of application.
3. Turn airplane mode on and wait 10 seconds.
4. Return to the application.
5. Perform an action that will cause a connection to be attempted.
6. Verify that the application notifies the user of the offline state.
7. Turn airplane mode off and wait 10 seconds.
8. Return to the application.
Expected result:
The application will detect that the network is not available and alert the user.
Application should not crash and provide a relevant error message to user. Application should
not display the exception stack to the user.
The application can also implement an offline (cached) mode without alerting the user.
When resuming to the application in an online state after being previously using in an offline
state, the application should detect that the connection has been restored and work properly.
More information:
For more information, see Offline Sync for Mobile Services and Build offline apps with Azure Mobile
Services
3.12 Saved status preservation
Application progress and saved states should be maintained after navigating away, exiting, and
synching on another device.
Steps to Reproduce:
1. Launch the application and log in if available.
2. Explore and navigate the pages and functionality of application, saving whenever possible (for
games, clear 2~3 levels/stages/checkpoints).
3. Exit the application.
4. Launch the application (if applicable, launch on a second device and log into the same
account as in step 1).
Expected result:
The application should retain saved states.
The application should retain saved information across devices when logged into the same
account.
3.13 Portrait and landscape mode scenarios
The application should adapt properly to portrait or landscape view if supported.
Steps to Reproduce:
1. Rotate the device to portrait or landscape view.
2. Launch the application.
3. If the application supports both portrait and landscape view (layout changes), navigate to
each page in the application. Rotate the device between all supported orientations at each
page.
Expected result:
When launched, the application remains in the same orientation as the device (if the
application supports multiple orientations).
All application pages render properly and adapt to the views supported. Ex: no text
truncation, all buttons are accessible, etc.
Applications should not display a logo or error message when the view is changed.
Applications should not launch in portrait mode on a Desktop device unless the application is
launched in portrait mode.
3.14 Core scenarios completed within the application
All core application scenarios are completed without errors.
Steps to Reproduce:
1. Launch the application.
2. Navigate through the entire application.
Expected result:
All core scenarios are accessible from within the application, not from an external web
browser.
Note: It is acceptable for a privacy policy or a login page that redirects back to the application
to open in a web browser.
Note: It is acceptable for the application to contain an in-app web browser.
3.15 In-App purchases
In-App purchases should display the corresponding Store UI.
Steps to Reproduce:
1. Launch the application.
2. Locate an in-app purchase.
3. Cancel the purchase.
Expected result:
The item for sale in the application should correspond to the one found in the Store UI.
The Store UI should take no more than 5 seconds to display.
Cancelling a purchase should not charge the user or reward them with the item.
After June 29, 2015, third party solutions for selling digital content consumed in the
application are no longer permitted. Microsoft IAP must be used to sell digital items that are
consumed or used within the application. The use of an external web browser or in-app
webview is not acceptable.
3.16 Geofencing
When geofenced sections of the application are accessed outside of the targeted area, the
application should display an appropriate error message.
Steps to Reproduce:
1. Launch the application.
2. Locate or create the application scenario dependent on geofencing.
3. Perform an action that invokes the geofenced feature.
Expected result:
There should be an indication that the application is being used outside of the required region.
3.17 Cross-Platform adaptive layout
The application layout should adapt and display properly on all supported devices.
Steps to Reproduce:
1. Launch the application on all supported devices.
2. Navigate through the application and inspect the layout.
Expected result:
All application pages render properly and adapt across all supported devices. Ex: no text truncation,
all application buttons are accessible, no large blank areas, etc.
More information:
For more information see Introduction to Universal Windows Platform (UWP) applications for
designers and TargetDeviceFamily (Windows 10 Insider Preview).
3.18 Application is feature complete
The controls within the application should invoke the expected functionality. The application should
not display messages indicating features are ‘coming soon’ or do nothing in response to control
actions.
Steps to Reproduce:
1. Launch the application.
2. Navigate through the application, interacting with buttons and tiles throughout.
Expected result:
The controls within the application should work as expected and should not present “coming soon”
messages.
3.19 App Resume
App Resume allows the previous suspended instance of the application to be resumed when user re-
launches the application from the Start screen.
Steps to Reproduce:
1. Launch the application by tapping the primary tile of the application on the Start screen or
tapping the application name from the application list page.
2. Navigate to a second or third level page in the application.
3. Tap the Start button to return to the Start screen.
4. On Start screen, tap the primary Tile again to resume the application.
Expected result:
The application resumes to the same page as in step 2 and pressing the back button takes the user
through the correct sequence of pages.
Or
The application resumes at the main page (without the splash screen and loading indicator) and
pressing the Back key exits the application. Pressing the Back key should NOT navigate to another
page.
More information:
For more information see: App lifecycle - Windows app development
3.20 Readability
All content in the application should be easily readable and adapt to a phone set to both light and
dark themes.
Steps to Reproduce:
1. If testing on a phone, set the device to the dark theme.
2. Launch the application.
3. Navigate through the application, if an in-app light/dark theme is available switch between
the two.
4. If testing on a phone, set the device to the light theme.
5. Navigate through the application.
Expected result:
All content in the application displays properly (ex: no light text against white backgrounds, etc.)
3.21 Placeholder text and imagery
The application should not display placeholder text or imagery.
Steps to Reproduce:
1. Locate the application in the device’s list of applications.
2. Pin the application to the Start screen.
3. Adjust the tile size.
4. Launch the application.
5. Navigate through the application.
Expected result:
• Tiles should be unique to the application and not display the default ⊠, a Unity logo, or be
blank.
• Splash screen should reflect the brand of the application. The splash screen should not
display the default icon ⊠ or be blank.
• The application should not display developer or placeholder text.
• Messaging and/or imagery in the application should not be from a different version of the
application (ex: Mobile imagery or tutorial instructions on a Desktop application,
iOS/Android imagery or text, etc.)
3.22 Application specific hardware
The application should function properly with all supported hardware.
Steps to Reproduce:
1. Connect supported hardware (ex: Bluetooth earphones, fitness band, etc.).
2. Launch the application.
3. Navigate through the application, interacting with supported hardware as needed.
4. Disconnect or turn off the hardware.
Expected result:
• The application should connect with supported hardware and interact properly, registering
any changes made with the hardware in the application.
• When the hardware is disconnected, the application should not crash and should alert the
user that the hardware has been disconnected.
• The hardware device connection should not be intermittent and stay reliably connected to
the host device.
• In-application instructions on how to sync/connect the device to the application should be
provided if the connection is not done automatically.
3.23 Cortana
Actions performed using supported Cortana commands should execute properly.
Steps to Reproduce:
1. Launch the application (this will ensure the commands are populated).
2. Select the Cortana button in the lower left corner of the Windows taskbar (on Mobile devices,
launch the Cortana application).
3. Perform the command, “What can I say?”.
4. Scroll down through the list of applications and locate the application under test.
5. Select the application under test and perform each Cortana command listed with voice
(example: say “News, show me the latest stories”), ensuring that the application is running
before each command is spoken.
6. Close the application.
7. Perform each Cortana command from step 5 with voice, ensuring that the application is not
running when the commands are spoken (show all running apps by displaying the task
manager on Desktop or the running apps list by holding down the back key on Mobile).
Expected result:
Spoken commands should perform the action that is implied by the listed commands.
Cortana should not perform a Bing search or simply launch the application.
Commands should function when the application is running and closed.
More information:
• Cortana Voice Commands: http://aka.ms/cortanadocs
• Speech interactions in your app: http://aka.ms/speechdocs
• App Linking: http://dev.bing.cofm
3.24 Resuming after idling
Applications should support the application lifecycle and resume successfully after the screen is turned
off.
Steps to Reproduce:
1. Launch the application.
2. Navigate to a page other than the default landing page for the application. Change the state
of the page. E.g. fill in a text box.
3. Let the device go idle until the screen automatically turns off.
4. Turn the screen back on and resume use of the application.
Expected result:
• The application should be in the same state as when the application was suspended.
• Interaction with the application should resume as normal.
More information:
For more information see Application lifecycle overview
4. Phone-Only Test Cases
The following test cases apply only to Windows Phone applications.
4.1 Application resumes successfully from tombstoned state
Application should resume successfully from tombstoned state.
Steps to Reproduce:
1. Using a low-memory device if possible, launch a different application (ex: Internet Explorer).
2. Press the Start button.
3. Launch the application under test.
4. Perform some steps to get the application into a state with data present.
5. Run memory consumption application(s) to use all of the phone's memory (on 512mb devices
this is generally 5-6 games). This will cause application under test to tombstone.
6. Press the back button until the application under test is brought back.
Expected result:
The application should resume to the same page and state that it was in before entering
tombstoned state. There should be no data loss.
Application should not crash on resuming and user should be able to continue using the
application as normal.
More information:
For more information on tombstoning, see How to preserve and restore page state for Windows
Phone 8, App lifecycle - Windows app development, and ApplicationExecutionState enumeration.
4.2 Back button for mobile
Applications should properly navigate back through pages when using the back key.
Steps to Reproduce:
1. Launch the application.
2. Navigate to a second or third page within the application.
3. Press the back key until reaching the Start menu.
Expected result:
The application should not bring up the Start screen or exit the application unless back key is
pressed while on the main menu.
The application should navigate back through the previously selected pages in the correct
order and eventually arrive at the Start menu.
More information:
Guidelines for back buttons
4.3 Continuum for mobile
Applications should display properly on external monitors and support keyboard and mouse when
using the continuum dock.
Steps to Reproduce:
1. Launch the application.
2. Navigate to a second or third page within the application.
3. Dock the device.
4. Select the application from the application list on the external monitor and ensure that the
application resumes where it was in step 2.
5. Navigate through the application and check the functionality.
6. Undock the device.
7. Select the application from the application list and ensure that the application resumes where
it was in step 5.
8. Navigate through the application and check the functionality.
9. Exit the application by holding down the back key and selecting the x in the upper right
corner and re-dock the device.
10. Launch the application from the external monitor while docked.
11. Navigate through the application and check the functionality.
Expected result:
The application displays properly on larger external screens, ex: no truncation, stretched out
images, or large empty spaces.
The application functions properly with keyboard and mouse, including any login screens.
More information:
For more information on Continuum, see Optimizing Windows Apps for Continuum and Optimizing
apps for Continuum for phone
5. Windows-Only Test Cases
The following test cases apply only to Windows Client applications.
5.1 Split and windowed views and display resolutions
The application should support a variety of screen sizes and split view(s).
Steps to Reproduce:
1. Set display resolution to minimum recommended resolution of 800x600 (1024x768 if testing a
W8.1 application on a W8.1 device).
2. Ensure that the device is in Desktop mode if testing on a W10 device.
3. Launch the application.
4. Select the window button in the title bar to resize the application screen if testing on a W10
device.
5. On each page in the application, adjust the horizontal and vertical size of the application if
testing on a W10 device.
6. Inspect the layout and application functionality.
7. Exit the application.
8. Set display resolution to 1280x800 (1366x768 if testing a W8.1 application on a W8.1 device).
9. Switch the device to Tablet mode if testing on a W10 device.
10. Launch the application.
11. Split the screen by swiping down from the top of the screen.
12. On each page in the application, inspect the layout and application functionality.
Expected result:
All application pages render properly and adapt at minimum resolution and split and resized
view(s). Ex: no text truncation, all command and app bar buttons are accessible, etc.
Consumer non-game applications should not display a logo or error message (ex: "this app
does not support split screen view") in split screen view.
All non-game applications should support windowed views and not force the user to full
screen view.
When snapping an application, the currently viewable position and state of the application
should still be visible when the application is in split screen view and full screen view.
Navigation between pages is still functional when in split screen view.
More information:
For more information see: Defining layouts and views and Guidelines for snapped and fill views
5.2 Application resumes from a suspended state
Applications should support the application lifecycle and resume successfully from a suspended state.
Steps to Reproduce:
1. Launch the application.
2. Navigate to a page other than the default landing page for the application. Change the state
of the page. E.g. fill in a text box.
3. Switch to the desktop.
4. Open Task Manager (enable View | Status values | Show suspended state) and verify
application is suspended.
5. Switch back to the application.
Expected result:
Application should be in the same state as when the application was suspended.
More information:
For more information see Application lifecycle overview
5.3 Back button for desktop
Windows 10 applications should properly navigate back through pages when using the Windows 10
system back button.
Steps to Reproduce:
1. Set the device to Tablet mode.
2. Launch the application.
3. Navigate to the second or third page within the application.
4. Select the system back button located on the taskbar until reaching the Start menu.
Expected result:
The application should not bring up the Start menu unless it is a Windows 8.x application or if
the system back button is pressed while on the main menu.
The application should navigate back through the previously selected pages in the correct
order and eventually arrive at the Start menu.
More information:
Guidelines for back buttons
5.4 Desktop and tablet mode switching
Applications should seamlessly support switching between Desktop and Tablet modes.
Steps to Reproduce:
1. Set the device to Desktop mode.
2. Launch the application.
3. Navigate through the application and check the functionality.
4. Switch to Tablet mode.
5. Switch back to the application.
6. Navigate through the application and check the functionality.
7. Switch to Desktop mode.
8. Switch back to the application.
9. Navigate through the application and check the functionality
Expected result:
The application should not crash when switching between desktop and tablet mode.
The application should adapt to the change in screen format and handle the differing
functionality (Software Input Pad, split view, etc.)
5.5 Multiple input methods
The application should properly handle multiple input methods.
Steps to Reproduce:
1. Launch the application.
2. Navigate through the application using a mouse. Checking boxes, right clicking on items, and
dragging if possible.
3. Navigate through the application using touch (and Xbox controller and Pen if applicable).
Attempt to perform all of the same functionality as the mouse (ex: right-clicking highlighted
text displays an additional menu).
Expected result:
Application fully supports navigation and selection using a mouse and touch.
Listbox items should not change state if there are no actions associated with marking or
‘ticking’ the item.
More information:
For more information on Touch: Gestures, manipulations, and interactions Keyboard: Responding to
keyboard interactions Mouse: Responding to mouse interactions
5.6 Forward and backward navigation
There should be UI that clearly indicates how to backward navigate within the application.
Steps to Reproduce:
1. Launch the application in Desktop mode.
2. Navigate through the application.
3. Attempt to return to a previous page.
Expected result:
Applications should be easy to navigate (ex: in-app back buttons or menus on every page).
6. Social Networking Integration Test Cases
The following test case applies only to applications that have Social Network Service integration.
6.1 Shared content
Sharing should function properly when available.
Steps to Reproduce:
1. Launch the application.
2. Navigate through application where social networking or sharing tied features are provided
(also check the “Share” option located in the command bar or Share Charm if applicable.)
3. Verify that actions taken in the application are properly reflected on the social network or
target application.
Expected result:
The contents and status provided by the social network or target application function properly.
6.2 Share targets
Applications that are a share target should be able to have content shared to them from another
application.
Steps to Reproduce:
1. Launch the application.
2. Launch another application that is a share source.
3. Share to the application under test.
4. Verify that the actions taken in the application are properly reflected on the selected share
target.
Expected result:
The application in test appears as a share target when appropriate content is selected in
another application, ex.: a news article to be shared to the application in test Facebook.
The content shared using another application is properly shared to the application, ex.: no
references to a link that does not appear in the post, no cut off text, etc.
7. Game Specific Test Cases
Game applications may have special user scenarios where additional attention or different
interpretation of requirement is needed. The following test cases ensures applications meet general
quality bars commonly expected from game applications.
7.1 Game leaderboard
Implementation of a leaderboard is not required. If a leaderboard is implemented, ensure the
leaderboard is updated for single and multiplayer scenarios.
Steps to Reproduce:
1. Launch the application.
2. Play the game long enough to post to the leaderboard for both single and/or multiplayer
sessions if offered on the application.
3. Look at all the leaderboards in the game.
Expected result:
With an active connection to the game’s backend service, the application should post to the
leaderboard showing both the gamer info such as gamer name and the associated score that
is the same as the one from the game session played.
Without an active connection to the game’s backend service, the application should not show
stability issues when not posting to a leaderboard and should inform users about the
connection issue to refresh leaderboard when users access leaderboards.
7.2 Game achievements and scores
Implementation of achievements and score are not required. If implemented, posted achievements
and scores should be recorded correctly.
Steps to Reproduce:
1. Launch the application.
2. Play the game and earn as many achievements and high scores as possible.
Expected result:
The achievements and high scores are posted on the application and the application’s online
service accurately.
The achievements and high scores are earned while the device has no network connection
are saved accurately on the application and the application’s online service when connection
becomes available.
7.3 Game multiplayer support
Implementation of multiplayer support is not required. If implemented, multiplayer game functionality
should gracefully handle any player leaving the game.
Steps to Reproduce:
1. Launch the application.
2. Select multiplayer game mode and start the game playing.
Expected result:
The game is played in a multiplayer mode with valid user.
The application gracefully handles the case where other user(s) playing the game left the
multiplayer game session.
The application gracefully handles the case where the user lost network connection during
multiplayer game session.
7.4 Roaming Game Saves
UWP games that can be used across multiple device families should support roaming game saves.
Steps to Reproduce:
1. Launch the application on the first device family (ex: desktop).
2. Complete a level and/or earn an achievement.
3. Launch the application on the second device family (ex: mobile), ensuring that this device is
logged into the same Microsoft account as the device in step 1.
4. Complete another level and/or earn an achievement.
5. Launch the application on the first device family (ex: desktop).
Expected result:
The game should retain achievements and level completions across multiple device families.
8. Media Specific Test Cases
The following test cases only apply to applications that support background audio or play video.
8.1 Background audio playback
Media applications that play audio (music, radio, podcasts, etc.) should continue audio playback when
the application is not in the foreground.
Steps to Reproduce:
1. Launch the application.
2. Start audio playback.
3. Switch away from the application.
Expected result:
Audio continues to play after switching away from the application.
8.2 Background audio controls
Media applications that play audio (music, radio, podcasts, etc.) should support the system transport
control when the application is not in the foreground.
Steps to Reproduce:
1. Launch the application.
2. Start playback.
3. Switch away from the application.
4. Use hardware hot keys for volume to activate the system transport control.
5. Switch tracks or pause using the system transport control.
Expected result:
Audio playback should be controlled using the system transport control.
8.3 Video pauses playback when the application is in the background
Media applications that display video should pause playback when the application is not in the
foreground.
Steps to Reproduce:
1. Launch the application (in Tablet mode for desktop).
2. Start video playback.
3. Switch away from the application.
4. Switch back to the application.
Expected result:
Video playback should be paused when switching away from the application.
More information:
For more information, see Application lifecycle (Windows Runtime apps) [Activated and suspending
events]
9.
10. WACK Test Findings
11. Competitive Parity Test Cases
11.1 Feature parity – Competitive platforms
The application should have feature parity with competitive platforms (Android and iOS if applicable).
Steps to Reproduce:
1. Launch application on Windows, Android, and iOS devices (if applicable).
2. Check the application on Windows for feature gaps when compared to Android or iOS.
Features in Android or iOS versions not available in Windows version:
Expected result:
Non-OS specific features that are found in the Android or iOS versions of the application are also
found in the Windows version.
11.2 Feature parity - Website
The application should have feature parity with its respective website.
Steps to Reproduce:
1. Launch application on a Windows device and open the website the application is based on.
2. Check the application on Windows Phone for feature gaps when compared to the website.
Features in the web version not available in Windows version:
Expected result:
Features that are found on the website are also found on the Windows application.
11.3 Performance parity
The application should perform similarly to competitive platforms using comparably capable devices
(Android and iOS if applicable).
Steps to Reproduce:
1. Launch the application.
2. Record the start-up time for the initial launch.
3. Record the data loading time between a page or level.
4. Close the application.
5. Launch the application.
6. Record the start-up time for subsequent launches.
7. Observe the graphical performance and overall application responsiveness.
8. Repeat steps 1-7 on competitive platforms (Android or iOS).
Expected result:
Applications should start up within 5 seconds. Games within 10. If this is not the case then the
start-up time should be no more than the time found on competitive platforms.
Load times between pages should be no more than the time found on competitive platforms.
The graphics and responsiveness of the application is comparable across platforms.