IBIS-PH (Indicator-Based Information System for Public

Health) Usability: Fall 2009

Sam Vanous Ph.D. HMO Health Program Manager

Utah Department of HealthOffice of Health Care Statistics

Why?

• Supposing is good, but finding out is better.

- Samuel Clemens

• Know thy user, and you are not thy user.- Arnie Lund

• The most common user action on a Web site is to flee.

- Edward Tufte

Early Examples

• 1963: The first mouse was usability tested

• 1981: XEROX usability tested (PARC)

• 1984: It took clients over an hour to install and print a check with Quicken– Took it to the street

• 1987 a usability lab was created for Windows 2.0

Usability Testing

• What exactly is Usability Testing?– Representative users try to accomplish typical

tasks with the product, while staff watch, listen, and take notes.

• Very important: the product is being tested and not the users

• Usability testing allows designers to create products that are functional and intuitive for the users.

Usability Testing

• Why Usability Test?– Identify any usability problems– Collect data on participant’s performance– Determine participant’s satisfaction

• When should it be done?– Early and often

• Prototype Test Analyze• Repeat as often as needed

Methods

• 20 Participants– Users

• 11 light users• 6 medium users• 3 heavy users

– Recruitment• 16 users from UDOH• 4 users with no experience

Methods

• Script– Welcome– Computer/IBIS experience questions– Scenarios– Follow-up questions

Results

• Completion time

• Help (scenario 1)

• Publications (scenario 2)

• Home Page (scenario 3 - not discussed)

• Indicators (scenarios 4, 5, and 6)

• Query (scenario 7)

Completion Time: UDOH

Average Time for Participants to Complete Scenarios:Set 1 (Health Department Employees)

1.61.3

1.7

2.72.3

4.9

0.0

1.0

2.0

3.0

4.0

5.0

6.0

Scena

rio 1

: Help

Scena

rio 2

: Cur

rent

Issu

e

Scena

rio 4

: Diab

etes

Scena

rio 5

: Uta

hns S

mok

e

Scena

rio 6

: Sec

ond

Hand

Smok

e

Scena

rio 7

:YRBS

Minutes

Average Time

Completion Time: Non-UDOH

Average Time for Participants to Complete Scenarios: Set 1 (Non-Health Department Employees)

2.5 3.25

5.5 5.75

9.25

19

02

468

10

121416

1820

Scena

rio 1

: Help

Scena

rio 2

: Cur

rent

Issu

e

Scena

rio 4

: Diab

etes

Scena

rio 5

: Uta

hns S

mok

e

Scena

rio 6

: Sec

ond

Hand

Smok

e

Scena

rio 7

:YRBS

Minutes

Average Time

Scenario 1: Help

• Varied responses and navigation– Contents and Usage, About IBIS, Links, Help,

and Contact

• Text driven and overwhelming

• Suggestions: – First time user button with consolidated

information (contents, usage, and tutorials)

Home Page

Scenario 2: Publications

• Many users clicked News and Updates tab • Light users had most difficulty• When Publications tab chosen, task still was

difficult. • Search function: not as expected.• Suggestions: Organize Publications as well as

News and Updates to consolidate the two areas, or make them completely separate. A dedicated search engine for IBIS would be appreciated.

Publications

Scenarios 4, 5 and 6: Indicators

• Most found the information, but the manner in which they did it varied greatly

• Categorical index – very confusing• Alphabetical index – easier to understand

– Alphabet bar doesn’t pop, users scroll to correct letter

• Indicators are not organized by key word: smoking, diabetes, etc…

• Suggestions: clean language, change alphabet bar

Indicators

Scenario 7: Queries

• Designed to be the most difficult, especially for light users

• Many searched all over looking for the information: indicators (both indices), publications, queries

• Instead of searching for a target, they are exhausting all other options

• Survey list for queries posed the greatest problem, many did not know what was in each category

• If dataset given, query did not pose a problem• Suggestions: simplify language, or describe in more

detail (rollovers, pop-ups, etc…)

Queries

Outsider Perspective

• Used to see effects of language

• Indices were most troubling for those not familiar with health care

• Time was increased to accomplish tasks

• Indicators were rarely used, individuals did not know what they were

• Confirmed language is a barrier to ease of use

Post Questions

• Individuals are very pleased with IBIS

• Having all the information in one location that can be queried is useful

• The worst feature is the organization and difficulty finding what they are looking for