!
!
W8#Session!6/24/2015! !11:30!AM!!!!!
“Usability#Testing#Goes#Mobile”##!
Presented#by:#!
Susan#Brockley#ExxonMobil#
#####
Brought#to#you#by:#####
##
340!Corporate!Way,!Suite!300,!Orange!Park,[email protected]!E!www.sqe.com!
!!!
!!
!!!!! !!!!!
With more than twenty-four years of software engineering experience at ExxonMobil, Susan Brockley is passionate about quality assurance, testing, and requirements management. Although her current title is business analyst in the IT Center of Expertise, Susan spends most of her time consulting on projects about properly scoping work, completing requirements, and engaging the customer. Her credentials―Certified Software Quality Analyst and Certified ScrumMaster―hang on the wall, but Susan prefers to roll up her sleeves and coach teams with whatever they need to begin a successful journey.
!
Susan Brockley
ExxonMobile
Usability Testing Goes Mobile June 2015
Susan Brockley
Introduction What is usability? What makes mobile usability different? What are some process changes should you consider? How can you transition usability into the mobile environment?
Agenda
3
Susan Brockley ExxonMobil Global Services Company • 25 years in software engineering (test and QA)
• Software Engineering Masters from UT Austin (2010)
• Certified Software Quality Analyst (CSQA) • QAI, since 2002
• Certified ScrumMaster
• Scrum Alliance, since 2010
Introduction
What is Usability?
5
[ISO 9241] “…the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” • Goals are task-oriented • Effectiveness - the ability to complete a task without errors
• Efficiency - completing the task quickly with minimal wasted effort
• Satisfaction – user’s perception of the product
Usability Basics
6
There are other important usability attributes. • Learnability – the time to accomplish a task without having prior
knowledge of the system
• Memorability – how well a user remembers the system after a period of inactivity
• Simplicity – how easy or difficult it is to complete a task
• Software Ergonomics – the physical workload to complete a task • Traditional input: keystrokes, mouse clicks, mouse movement • Mobile devices: screen gestures, time on device, posture, typing style, usage
Additional Usability Attributes
7
Traditional usability tests include these processes.
• Plan tests • Develop tasks • Decide where to hold the test • Establish a baseline
• Conduct tests • Conduct a pre-test interview • Observe user as they complete tasks • Conduct a post-test interview
• Analyze the data and report results • Compute metrics, summarize comments and survey results, make
recommendations • Share results with development team, management, and stakeholders
The Usability Testing Process
What Makes Mobile Usability Different?
9
Pull out your mobile device. Pair up with somebody else. • Informal poll - raise you hand…
• Who pulled out a phone? • Who pulled out a tablet? • Who pulled out something else? What was it?
• Within your pair, one of you start typing while the other(s) observe. Now switch. • How is viewing a mobile device different? • Were you able to read what the other person was typing? • How much of the person’s facial expression / body language were you able
to observe while you looked at the device?
Activity – Getting in a Mobile Mindset
10
Usability on a mobile device is still a planned activity to determine a product’s efficiency, effectiveness, satisfaction, and ergonomic impact.
• Requires up-front planning
• Involves five or so Testers in a one-hour test session
• Is best communicated to the project team in a summarized form
Mobile Usability – What Stays the Same?
11
Users expect more from mobile applications.
• Users expect that mobile applications will have better performance than their desktop equivalent
• Users expect a friendlier interface, no need to contact help
• Mobile applications will be updated more frequently
• Mobile devices will seamlessly integrate applications and components such as phone, message, camera, calendaring, etc.
So what?
• Communicate heightened user expectations to development team
What Makes Mobility Different? (1 of 8)
12
Mobile devices can be used anywhere and at any time.
• Mobile context – circumstances under which a mobile device is used • Location awareness – delivering information about physical location • Physical environment introduces additional challenges to mobility
• Sunlight/Glare • Water • Heat/cold • Sound
So what?
• Your testing environment just got bigger and more diverse
Why is Mobile Usability Different? (2 of 8)
13
Accessibility means making it easier for seniors or people with disabilities to use mobile devices. • Being able to change text size • Displaying high contrast themes • Enabling voice • Magnifying the screen • Displaying browser captions • Using intelligent agents (example: Siri) • Interfacing with peripherals (example: hearing aids)
So what?
• An application or web site is not usable unless it is accessible
What Makes Mobility Different? (3 of 8)
14
Affordance means making it clear what can be selected, tapped, or swiped.
• A button with slightly raised appearance suggests you can push it • An e-book suggests you can swipe a page to turn it
So what?
• Consistent affordance improves usability because the same gesture
means the same thing everywhere
What Makes Mobility Different? (4 of 8)
15
Power is now a part of the user experience.
• “…poor battery life contributes to customer dissatisfaction more than any other smart phone feature…” - Survey by J.D. Powers and Associates
• Mobile applications don’t drain the battery directly, they do so by using the devices hardware components
So what?
• Usability testing should measure power consumption to ensure there are no surprises
But wait!
• Consider power loss in context, never by itself
What Makes Mobility Different? (5 of 8)
16
User experience will vary with connectivity. • Intermittent connectivity introduces variability in performance and
reliability • …and, when it comes to mobile devices, performance matters
So what?
• Capture comments about performance and connectivity to quantify the user experience
But wait!
• Again, consider connectivity in context, never by itself
What Makes Mobility Different? (6 of 8)
17
Protective covers for mobile devices are common and necessary.
• If you loan mobile devices for testing, you may wish to protect them from damage and breakage
…however
• Covers may alter a user’s perception of the product
• Covers may also affect the way the mobile device is used in a normal setting (e.g. using a cover to prop a device is common)
So what?
• Note the use of protective covers and whether they alter user perceptions
What Makes Mobility Different? (7 of 8)
18
Caution your Testers about the risk of using devices in field exercises. • Pedestrian cell phone-related injuries have doubled since 2005 • 70% of adults have experienced digital eye strain • There is an increase in the number of deaths and injuries due to
distracted driving
• Encourage safe testing behavior • Discourage mobile use and driving or walking • Ask Testers to observe the 20-20-20 rule
• Take a break way from the mobile device every 20 minutes for 20 second to focus on something 20 feet away
• Remind Testers to take breaks and pay attention to posture • Avoid sitting in the same position for extended periods
What Makes Mobility Different? (8 of 8)
What Process Changes Should You Consider?
20
Field tests require additional planning considerations
• Do you loan devices for field tests or have Testers use their own? • What is the right number of Testers? • How long with the test last? • What data will you collect and how will Testers supply it?
But wait! Do you really need a field test?
• A major advantage of conducting field tests is that it takes context into
consideration, which is difficult to simulate in a laboratory
Planning Processes
21
Field test execution is very different from lab-based execution.
• Typically longer in duration
• Less structured and usually does not require tasks
• Collecting results is different • Direct observation • Daily diaries • Software capture • Survey
Execution Processes
22
Test results should always include the following:
• Tester demographics • Test environment (location, device type, connectivity, etc.) • Direct observation (posture, verbal remarks, task completion, timing) • Survey results
Field tests introduce a broader, less structured source of information • Make it clear which results were obtained in a lab setting, which ones
were obtained in the field • Correlate contextual information with Tester’s feedback to better
quantify field results • Example: poor performance observed more often with Wi-Fi than 3/4G
• Follow up with Tester to clarify remarks
Analysis and Reporting Processes
How do You Get Started?
24
• How many of you have performed traditional usability testing?
• How many of you now test or support mobile applications or devices?
• How many of you have a mobile test lab?
• How many of you have done field tests on mobile devices?
• How many of you have a centralized usability or UX/UI team?
• How many of your organizations require usability tests as part of the development process?
Activity – Organizational Usability Maturity
25
Each organization has a different starting point with respect to usability testing – whether traditional or mobile
• The goal is to mature as an organization in usability testing practices
Getting Started
26
Begin performing traditional usability studies on desktop applications. • Practice test planning, execution, and reporting • Find a development team who will work with you • Recruit people to help – Testers, Technical Coordinators, Observers • Conduct a retrospective to improve your process
And when you are ready • Conduct “real” tests and share your results outside your testing group
Getting Started – Step 1
27
• Transition to mobile usability studies in a laboratory setting. • Introduce mobile applications or devices into the usability testing process • Modify what you wish to capture • Practice mobile usability tests • Conduct a retrospective to improve your process
And when you are ready
• Conduct “real” tests and share your results outside your testing group
Getting Started – Step 2
28
Set up a mobile test environment. • Start simply – just controlling context, connectivity, and power is a big plus • Add loaner devices, protective covers, charging carts, testing software as
time and budget allows
Don’t forget about Mobile Device Management (MDM) • Administers mobile devices • Distributes applications • Sets configurations • Oversees physical device management
• Storage • Power • Check in/out • Protection
Getting Started – Step 3
29
Conduct mobile field studies. • Begin with a shadowing technique to observe Testers as they go through
their normal activities • Your first studies will provide valuable experience and validate your
assumptions about field tests
If you want to involve more Testers, try a group approach • Communicate objective, schedule, and desired results • Capture results using
• Surveys • Daily diaries • Specialized software • Group feedback sessions
• Thank Testers for their participation
Getting Started – Step 4
30
Consider creating a User Experience/User Interface (UX/UI) group. • A UX/UI group promotes and encourages software usability as an integral
part of the software development process
UX/UI governance groups:
• Provide projects with tools and processes to conduct usability tests • Stay connected with the industry and share knowledge • Offer training and coaching to develop usability practitioners and experts • (Optionally) steward user interface guidelines
Getting Started – Step 5
31
Require usability testing in the development process. • Transition from an ad-hoc, grass-roots movement into an organization that
consistently plans for usability in the development process
• If done properly, it touches • Project management methodology • Business analyst processes • Developer guidelines • Testing practices
Getting Started – Step 6
Questions?
Thank you