Observing UsersPaul Bogen, LaShon Johnson, Jehoon Park

Overview• Introduction• Goals, questions and paradigms• How to observe• Data collection• Indirect Observation• Analyzing, interpreting, and presenting data

Introduction• What is Observation?

– Informing Design vs. Evaluation

• Aims of this Chapter– Why Observation?– How to Observe– How to use your observations– Issues with Observation

Goals, questions, and paradigms

• Why have a goal?• What and when to observe?• Approaches to observation

– “Quick and dirty”– Usability testing– Field studies– Ethnography

How to observe – Controlled

• Role of Observer– Predetermined Location– Testing Equipment– Informed Consent

• Think-aloud technique

How to Observer – Field Studies

• Practitioner’s Framework– The person– The place– The thing

How to Observe – Field Studies• Goetz and LeCompte

– Who is present?– What is happening?– When does the activity occur?– Where is it happening?– Why is it happening?– How is the activity organized?

How to Observe – Field Studies• Colin Robson

– Space– Actors– Activities– Objects

– Acts– Events– Goals– Feelings

How to Observe – Field Studies

• Field Checklist– State the initial goal and question clearly– Select a framework to guide your activity in the

field– Decide how to record events– Be prepared to go through your records asap

How to Observe – Field Studies

• Field Checklist cont.– Separate personal opinion from what

happened– Be prepared to refocus study– Think about gaining trust of those you observe– Think about how to handle sensitive issues

How to Observe – Field Studies

• Field Checklist cont 2.– Consider working as a team– Consider checking notes with the observed– Look at the situation from different

perspectives

How to Observe - Ethnography

• Need of acceptance

• Ethnography vs. Participant Observation– Open Interpretivist– Theoretical Underpinning– Biases

How to Observe – Ethnography• Ethnographic Checklist

– Identify a problem or goal and ask questions– Just be there to observe– Collect a variety of data– Move between broad picture and specific

questions– Use a holistic approach to data analysis

Data collection• Note plus still camera

• Audio recording plus still camera

• Video

• Issues– Equipment– Flexibility– Completeness– Disturbance to users– Reliability– Analysis– Feedback to designers

Indirect Observation

• Diaries

• Interaction logging

Analyzing the data• Kind of Data

– Qualitative that is interpreted– Qualitative that is categorized

• Looking for incidents or patterns• Categorizing• Analyzing discourse

– Quantitative that is collected

• Feeding the findings back into design