HCC 729 Field Data Collection 1...1 Human-Centered Compu/ng at University of Maryland, Bal/more...

Human-CenteredCompu/ngatUniversityofMaryland,Bal/moreCounty1

HCC 729�Field Data Collection 1

Human Centered Design Dr. Amy Hurst Dr. Aqueasha Martin


Homework Policy

Turn in homework to box before class. If you decide you need to update your assignment after class you have 24 hours.

•  Note that Box stores version histories so we will see if nothing was turned in until Wednesday


Anyone see a ✔+?

Aqueasha and I are going to try “Crowd sourcing” the ✔+s during class?

•  Did anyone’s partner go above and beyond the assignment’s requirements?

We will email students with individual feedback if they received a ✔–

•  Contact us during class critiques if you want additional feedback


Homework Critiques

Break into groups of 3


Questions for Critiques

Who is your client and why are you working with them?

•  What do you think your biggest challenge with your client will be?

How do your client’s usability concerns compare to your own? Competitive Analysis: Are there a lot of similar sites to your clients?

•  Why do you think this is? •  How does your client’s site compare?


Discussion

How did you gather this data from your client?

•  Email? •  Phone call? •  Face to face meeting?

How do you feel this approach worked? •  Pros and cons? •  How do you plan to communicate in the

future? Questions / Problems / Concerns?


Redesign

For both new and existing systems, first and foremost you must understand:

– Context – Context – Context – Context – Context – Context – Context!!!!!


What do we mean by context?

Who is using the system? Who is impacted by the system? Who controls the system? Where / when is the system used? Under what situations?


Data Gathering Methodology

Gathering and analyzing data of multiple types, from multiple sources, using multiple techniques One data set trades off the limitations of another

9



Document Review Direct Observation Guided observation (Think Aloud) Interviewing Focus Groups Survey Field Experimentation Lab Experimentation

10



Document Review Direct Observation: Contextual Inquiry Guided observation (Think Aloud) Interviewing Focus Groups Survey Field Experimentation Lab Experimentation

11


Reading Quiz


Break


Reading Quiz Discussion

What are the things we need to go in more depth with?


Posting Readings

Thank you for participating in the quiz. We have decided it is simplest for everyone if we post readings to Box .

•  We have removed the link from the blog. •  There is a link to box on the blog


Lecture

Field Data Collection 1



Document Review Direct Observation: Contextual Inquiry Guided observation (Think Aloud) Interviewing Focus Groups Survey Field Experimentation Lab Experimentation

17


Observation

Our primary manner of understanding the world around us. But not just looking, rather seeing… Doing this correctly requires significant training

18


What you can discover through observation?


Observation

What you can discover by observation: – Realistic system use in natural setting – What is actually done vs. what is described – Activities too routine to mention – Tacit vs. explicit processes / interactions – Collaborative elements –  Impact of environmental factors

– Additional interview candidates or documents

20


Observation Exercise

Carefully look around our classroom. Do not get up and move. If you were given a sticky note, also look for things that meet that criteria. Quickly, but legibly, write down what you see. 21



Pass your observation notes to someone on the other side of the classroom. In turn, read off findings pointing to item in room. “Boggle Rules” - As we read out items, cross off anything your informant also saw. Repeat. At end, circle anything that was uniquely seen. Return sheets.

22



Any differences from your own observations?

In what way? What could be the cause for this?

What was the impact of point of view?

Did the sensitizing concepts alter what you saw?

How could we capture everything? •  Aggregating everyone’s notes? •  Video observation?

23


Attention

https://www.youtube.com/watch?v=vJG698U2Mvo


Direct Observation

Selective attention Recall from 629… The Invisible Gorilla

25


What to Observe?

What Why When How Where Who …everything? But we’re not just cameras!

26


What are you observing?

First define what you are trying to observe! •  It is impossible to observe everything, must

give yourself focus

Ideally this should come from your client / research questions, prior work, or the literature

27


Where will observation be conducted?

In a usability lab – Experiment

In structured field setting

– Field experiment In natural field setting

– Field/naturalistic study

28


When to Observe?

Planned events – have a beginning, middle, and end (e.g. staff meetings) Representative sampling (e.g., retail hours) Exception based sampling (e.g., end of day inventory) Daily, weekly, monthly variations (e.g., fiscal close) Seasonal variations (e.g., holiday rush vs. mid-winter lull) Duration Adequate reduction in observer effects Theoretic saturation Longitudinal

29


Level of Immersion

How involved is the usability analyst? Instrumentation (system use logs) Casual observation (“looking around”) Direct observation (observation guide) Contextual inquiry (interactive, “props”) Participant observation (active involvement) Ethnography (immersive)

30


Observation Methods

Shadowing Participant/direct observation Think-aloud protocols (after midterm) Self-reports Looking around while interviewing (next week) Design choices

– overt vs. covert – emic vs. etic / insider vs. outsider

31


Elicitation: what will you do?

None, observe w/o interaction. Prompting (careful of leading/bias) Dialogue (e.g., expert/novice) Feign being the novice (deception?) Unobtrusive measures (environmental clues to what’s important)

32


How to conduct an observation


Observation Protocol

Defines scope of an observation – When/how it starts – When/how it ends

Scripts all interaction with subjects Defines what and how data is captured

– Recording – Data instrument – Notes

34


Observation Protocol

… and analyzed – Transcription norms – Analysis methods

Level of detail especially important in lab settings – Have to ensure sessions are carried out

consistently Documents design choices

– Setting –  Level of immersion – Sampling strategy

35


In-Person Capture Methods

Observer present – Jottings => field notes –  Image recording (still photography) – Audio recording – Video recording

36


Remote: Capture Methods

Instrumented software / logging behavior – High quality capture – Varied granularity: what utility?

Diary studies

– Capture in moment – Critical incident triggers – User initiated recording (audio/video) – Self-report problems

Experience Sampling Method

37


Issues with Recording

Intrusive Increase observer effects (blinking red light syndrome) Robust data, able to recreate actual occurrences verbatim Reduce observer bias, allow others to “code” the data Increased marketability of results - help others to “see” Difficult preparation for analysis (esp. transcription) NEVER rely solely on recording for capturing your data!

38


Ethics with Recording

Public vs. private interactions – know the law! Dilemmas of Internet study Institutional Review Board approval of protocol Informed consent

–  Ability to withdraw at any time –  Ability to stop recording at any time –  Ability to destroy recording

Confidentiality of any captured data Judging – what if you don’t like what you see? Deception – when, if ever, is it valid? Intrusive / unobtrusive – when, if ever, is it okay to interject?

39


For Next Time


Project Assignment

Observe at least two people using the existing site.

•  Cannot be your client or someone in this class

You are looking for the following information •  Things that work well •  Usability challenges •  Things users like •  Things users don’t like


Contextual Interview Steps1.  Conventional Interview

1.  Get to know the user 2.  Transition

1.  Explain the rules of a contextual interview 3.  Contextual Interview

1.  Observe and probe their work 4.  Wrap Up

1.  Feedback and interpretation

You will only do steps 1 - 3 this week


1. Contextual Interview

Pick at least 3 pieces of demographic information you want to know about your participant. Ask them if they have used this site before, and their experience interacting with it Ask them if they have used a similar site before, and their experience interacting with it


2. Transition

State the rules for the contextual interview •  Tell your participant how you will be

recording their interaction. •  Explain what kinds of interactions you want

the user to have •  Define your environment of partnership


3. Contextual Interview

Show user the interface and ask them to choose and complete a task (their choice) Gather your observations and maintain interaction Then wrap-up and thank your participant


A. Reflect on observation context

Describe your users •  What do you know about them?

Describe your observation environment •  date / time of day / duration / room

How did you introduce user to this method? How did you create environment of partnership? How did you create the right amount of focus for your participant?


B. Reflect on your findings

What did you observe? •  Write a bulleted list of observations •  Don’t think about redesigns / fixes.

•  Focus only on what you observed. •  Limit judgement


Reading Assignment

Usability Engineering Chapter 7: Field Methods

•  Posted to Box


Note…

Next week we will cover interviewing techniques and you will be asked to schedule an interview with your client. If your client is a busy person you might want to schedule a meeting sometime between 2/17 and 2/22.

Date post:	11-Jul-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times