24-hour Days, 4 Languages:

Challenges of International

Usability Testing

•Founded 1967: Ann Arbor, Rochester NY, Palo Alto

•Early practitioners of user research

Contributed to the development of early usability testing methodologies that are standard today

•More than 150 clients; close to 2,000 projects

•Apps, websites, devices, other products for B2C/B2B

What makes a good localized

app?

•When do salespeople want materials in English,

and when in their local language?

•When do they search using English, and when in their local language?

6 1 1

6

1

4

2 1 0 2 4 6 8

10 12 14 16 18 20 22

English Local language

Other response N

um

be

r o

f P

art

icip

an

ts

Favored Language

Favored Search Language Do you prefer to search using English or

[language]?

French Speakers

Mandarin Speakers

Spanish Speakers

•Engaged bilingual partner researchers in France,

Chile, and China

•Coordinated with our client’s participant recruiters

•Created translation-ready research protocol in English

•Had the protocol translated into French, Spanish,

and Chinese

•Created data-logging template

•Had our partner researchers conduct pilot-test sessions in

the local language

Here’s What We Did

•Reviewed pilot session results with our client,

then adjusted the protocol

•Had our partner researchers conduct 7 or 8 sessions in the

local language

•Collected data logs in English from our partner researchers

•Arranged for translation of the session recordings into

English

•Wrote results report for presentation to our client, with

partner researcher input

More Details

Engaging Research Partners

• User research skills and experience

• Bilingual communication facility

• Availability for desired schedule

Coordinating Participant

Recruiting

Salespeople: sensitive recruiting

Indirect access and communication

Unexpected outcomes

Content similar to U.S. research projects

Needed to be foolproof

No U.S. slang, no ambiguities

Data-Logging Templates

Consistent reporting from

researchers using 3 different

languages

Sessions in native language, but

English data logs

Structured logs facilitated data

analysis

Pilot-Test Sessions

One in each language

Observed by client stakeholders

over WebEx

Each followed by debrief in English

with the partner researcher

At Last, the Sessions

7 – 8 sessions in each language

Over 15 time zones

Just-in-time recruiting

Arranged to translate all session recordings into English

5 cycles of report deliverables

15 graphs of participant preferences

Hundreds of participant quotations

Project management for a multiple-language usability test is hugely complex

Maintaining consistency of methodology in multiple countries is another challenge

International holidays differ and can impact schedules

A project in 5 time zones spanning 15 hours means you are never off-duty

Thank you!

Stephanie Rosenbaum

stephanie@teced.com Larry Rusinsky larry@teced.com