Testing Content: Early, Often, & Well

transcript

Testing Content

Early, Often, & Well

Colleen Jones colleen@content-science.com

Kevin O’Connor koconnor@userinsight.com

Testing design.

You’ve probably heard of it.

It’s good. But, it’s not enough.

We’ve talked about testing content for years, but as an industry...

Too many usability tests focus ONLY on finding information—not on how the information [content] itself works for people.

－ Ginny Redish

“ “

Content includes the text, graphics, video, & audio that make up an interactive experience.

－ Kristina Halvorson

“ “

What we mean by “content.”

THE SITUATION

CDC Travelers’ Health wanted to assess their content.

TREND: Content is critical to the health industry.

…providers of online health resources can increase the relevance & influence of their Web sites if they •  focus on site usability, •  bolster the credibility of their content, & •  bridge the gap between Web & point of care.

－ Who Cares About Online Health Content? Forrester Research

“ “

The site is a big opportunity to improve health decisions through content.

•  Top 5 most popular CDC websites

•  In 2009 •  9.5 million visits •  31 million page views

With such rich content on our site, I was concerned about the number of people who were calling because they couldn’t find vaccine information.

－ Kelly Holton Communication & Education Team Lead Travelers’ Health, CDC

“ “

I knew it was time to do something different to help drive our content strategy. I felt we needed to find out what users really think about & do with our content.

－ Kelly Holton Communication & Education Team Lead Travelers’ Health, CDC

“ “

CDC had clear objectives.

•  Gather feedback about different user groups’…

•  Content needs & preferences.

•  Perspectives on content quality, presentation, & format.

•  Create a content and usability testing model.

OUR APPROACH

Testing content helps you choose the right content direction.

We tested content iteratively.

Baseline Test Concepts Validation Test

BASELINE TEST

We focused on a critical content sample:

DESTINATIONS

We tested with these user groups.

Healthcare Professionals Doctors, nurses, & pharmacists who are in a posi3on to provide travel health-‐related informa3on & care (N=12)

Travelers

Consumers who do or do not research health-‐related informa3on before traveling to foreign countries (N=12)

Travel Professionals

Travel & tourism industry representa3ves (e.g., travel agents, tour organizers, etc.) (N=6)

Travel Medicine Professionals Doctors, clinic admins, nurses, & specialists provide travel health informa3on & care (N=6)

We observed & interviewed people one-on-one.

We needed a deep understanding of WHY.

Our protocol focused on key content questions…

Can users… •  Find & read the content they need?

•  Understand the content?

•  Act on the content?

We learned a lot.

Can they find & read it? NO •  Users couldn’t find vaccinations.

Do they understand it? PARTLY •  Users were not sure whether the content was specific to the

destination.

•  Users were distracted by & misunderstood "Travel Notices.”

Can & will they act on it? NO •  Users trusted CDC but…

•  Felt overwhelmed by the content.

•  Were confused about which vaccinations were required, necessary, or most relevant to their level of risk.

•  Were not clear on the next step they should take.

CONFUSING DATES RISK LEVELS?

AMBIGUOUS LANGUAGE

HARD-TO-READ FORMATS

PORTABLE FORMATS?

Lost in Content

We recommended focusing on travelers.

Travelers

Consumers who do or do not research health-‐related informa3on before traveling to foreign countries (N=12)

•  Other sections, such as Yellow Book, addressed travel medicine professionals better.

•  80 / 20 rule applies.

•  Health professionals & travel professionals act as catalysts or referrals.

THE CONCEPTS

Travelers need to decide… •  Whether they are at

•  What shots to get.

•  How to get shots.

•  How to avoid common diseases.

•  What health items to bring.

•  More

Design from the content out.

－ Jeffrey Zeldman

“ “ That’s exactly what we did.

We explored content…

•  Organization & priority

•  Layering & layout

•  Writing style & tone

•  Calls to action

5 organization schemes

Priority on vaccinations

Tone & language

Rhetorical / psychological framing

Layering / progressive disclosure

Decision table

Call to action

Then, we tried variations.

Concept 1 Concept 2

Layering & layout

•  Less intro instruction •  Tabs •  More explanation in

vaccination summary

• More intro instruction • Reveal • Less explanation in

vaccination summary

Calls to action •  Bold buttons • Contextual text

Style & tone •  More familiar • Less familiar

Does this content work better for travelers?

TESTING THE CONCEPTS

We tested 2 concepts + the original.

…by presenting users with alternative…solutions, subjective ratings are less prone to inflation & give rise to more & stronger criticisms.

−Getting the Right Design & the Design Right CHI 2006

The protocol was similar to the baseline.

•  Can they find and read it?

•  Do they understand it?

•  Will they act on it?

We didn’t expect a “winner,” just better feedback.

We followed best testing techniques.

•  Tested on a small scale.

•  Avoided order bias.

•  Followed a discussion guide.

•  Took time stamped notes & looked at patterns of behaviors.

The concepts tested better.

Can they find it? YES

Do they understand it? MOSTLY

•  Users understood the vaccination content but….

•  Asked for more explanation of the ratings.

•  Were still distracted by & misunderstood "Travel Notices.”

Can and will they act on it? YES

•  Users preferred more explanation in the decision table.

•  Users knew the next step they should take.

•  Despite tone & framing, users still trusted the content.

Preference for Concept

What’s Next

•  Testing •  Analysis •  Synthesis

1 Plan

•  Concept •  Testing

2 Create

• Refinement •  Testing

3 Deliver

•  Evaluation •  Quality Control

4 Govern

Process Inspired by Content Strategy for the Web

We’re not done, but we know we’re on the right track.

Testing Content: Early, Often, & Well

Business