Tom Hume Human-Computer Interaction, assignment 1

Text

TitleX-Construction Lite: HCI evaluationAndy Keavey, Mariana de Rojas-Morao,

Merve Yildirim, Queen Ajifa Ododo, Tom Hume

Tom Hume Human-Computer Interaction, assignment 1

Text

Title

Screenshot of page from Android Marketplace

Tom Hume Human-Computer Interaction, assignment 1

Overview of gameplay

First screen of instructions (animates) Second screen of instructions

Game-play screen Results of play screen

Tom Hume Human-Computer Interaction, assignment 1

1. Expert review1.1. Cognitive walkthrough, written up1.2. Evaluation vs Nielsen heuristics

2. User testing2.1. Pre-test questionnaire2.2. User test of first level (video recorded)2.3. Interview with user whilst reviewing recording2.4. Post-test questionnaire

Evaluation approach

Tom Hume Human-Computer Interaction, assignment 1

User evaluation

8 test subjects recruited

(all Android owners, familiar with UI)

Tom Hume Human-Computer Interaction, assignment 1

Results of cognitive walkthrough

Issue # reportsZoom functions misused ✓ ✓ ✓Errors placing girders ✓ ✓ ✓Misunderstanding help messages ✓ ✓ ✓Unsure how to proceed ✓ ✓Game crashed on start ✓Problem starting play ✓Confused by pop-up adverts ✓ “I successfully drew some

girders, but I struggled a bit because they were going the wrong direction most of the time”

“Some of my girders went out of view at the top of the screen. I used the zoom out gesture consistent with Google Maps and could see a much better view of the task”

Tom Hume Human-Computer Interaction, assignment 1

Results of heuristic evaluation

Agreement DisagreementMatch between system & real worldConsistency & standardsFlexibility & efficiencyAesthetics & minimalism

User control & freedomError preventionRecognition over recall

Tom Hume Human-Computer Interaction, assignment 1

Seconds to complete first level

Results of quantitative data

0

75

150

225

300

0 1 2 3 4

# attempts to complete first level

Tom Hume Human-Computer Interaction, assignment 1

Results of questionnaire data

0

10

20

30

40

8 7 5 4 6 1 3 2

Age of test subjects

25%

75%

Female Male

0

2

4

6

8

1 2 3 4 5 6 7 8

Estimated vs actual fun

Estimated Actual

38%

63%

Android gamer Not android gamer

Tom Hume Human-Computer Interaction, assignment 1

Results of user testing

Issue # usersMisunderstood girder placement ✓ ✓ ✓ ✓ ✓ ✓Didn’t understand help/instructions ✓ ✓ ✓ ✓ ✓Mixed up replaying/restarting level ✓ ✓ ✓ ✓Couldn’t get the train started ✓ ✓ ✓Panned unintentionally ✓ ✓Couldn’t tell they were complete ✓Couldn’t find girder removal tool ✓Confused by an advert ✓Misunderstood tool instructions ✓Couldn’t find instructions again ✓

Tom Hume Human-Computer Interaction, assignment 1

Key issue: misunderstood girder placement

Tom Hume Human-Computer Interaction, assignment 1

Key issue: mixed up replaying/restarting level

Tom Hume Human-Computer Interaction, assignment 1

Good news• Most players complete the first level, first time• Most players enjoy the game more than they thought• Many expert evaluators are now regular players

Bad news• 75% players, 3/5 experts had issues with placing girders• 62% players don’t read help, 3/5 experts misunderstand it• 50% players mix up replaying and restarting the level• 37% players, 2/5 experts have trouble starting their train • 20% players pan unintentionally, 3/5 experts had zoom issues

Conclusions

Tom Hume Human-Computer Interaction, assignment 1

1. Warn players on the first level when they place a girder which doesn’t finish attached to an end-point;

2. Clarify language in help messages and screens;

3. Improve the iconography, labelling or placement of the “replay failure” and “restart level” icons to reduce confusion between them;

4. Teach players to use the “play” button by animating it on the first level when they have a workable bridge;

5. Allow pinch-and-zoom to zoom into the bridge during play.

Recommendations

Tom Hume Human-Computer Interaction, assignment 1

Critique of methods

Hard to record video in sunlight

Users were self-selecting, to a degree

Slight inconsistencies in questionnaire usage

Tom Hume Human-Computer Interaction, assignment 1

Questions?