CS 321: Human-Computer Interaction DesignSeptember 3, 2015

Interviews• User Research• Interview Methods

Chapter 7

• Interview Questions

Chapter 4

CS 321September 1, 2015Contextual InquiryPage 2

User Research

Quantitative Research

Qualitative Research• Uses metrics to

statistically gauge user behavior

• Predicts future behavior via interpreted observations

• Yields objective data that is often open to subjective interpretation

• Yields subjective data that is often presented as objectivity• Techniques include

focus groups, market demographics, and usability testing

• Techniques include contextual inquiry and ethnographic interviewing

CS 321September 1, 2015Contextual InquiryPage 3

Stakeholder InterviewsTo understand the basic context in which a

proposed software design is needed, a good place to begin is to interview

stakeholders.

• Basic product vision• Budget & schedule• Technical

constraints• Intended users• Competitive

alternatives

Stakeholders might include management

and marketing personnel, as well as

key technical and customer support

individuals

Sought information includes:

CS 321September 1, 2015Contextual InquiryPage 4

Subject Matter Expert InterviewsSubject matter experts, proficient at the

processes being planned for the software in question, may provide useful, if biased,

data to the software design team.Warnings about subject matter experts:• Experts tend to

provide ideas beneficial to experts, not to intermediates or beginners

• Experts are more helpful at identifying problems than at designing practical solutions

• Experts are usually premium resources, able to provide only limited access time to designers

CS 321September 1, 2015Contextual InquiryPage 5

Ethnographic InterviewsBorrowing terminology from the field of

cultural anthropology, design researchers use ethnographic interviews to establish

rapport with interviewees, in hopes of eliciting deeper, more extensive replies.After observing that only 10% of its registered

users were sellers, eBay conducted ethnographic interviews to determine the root causes of seller dissatisfaction with eBay and what might be done

about it.

TRADEOFF #4:Personalization vs. Cost/ROI

A completely customizable system is very desirable to sellers, but the cost of developing such a system might yield inadequate benefits,

especially in view of the difficulty it might cause for buyers.

TRADEOFF #2:Buyer vs. Seller Experience

To improve the experience of buyers, sellers must take and include photos,

compose detailed and attractive descriptions, and accurately categorize all

sale items.

TRADEOFF #1:Ease-of-Use vs. Security

To open an eBay account, sellers must supply specific bank and

credit card information, a potential security risk.

TRADEOFF #3:System vs. Seller

DecisionsDefault values (payment

services, shipping recommendations,

description formats) are used in the system unless the seller expressly cancels

them.

CS 321September 1, 2015Contextual InquiryPage 6

Persona HypothesisBy studying the results of stakeholder and

subject matter expert interviews, the design team can develop a persona

hypothesis, a theory concerning what types of users will utilize the planned software.Persona

Considerations:• Different user roles with respect to the planned system• Behavioral differences, motivations, and preferences• Varying expertise levels

• Work environment (e.g., company size, security concerns, technical savvy)

CS 321September 1, 2015Contextual InquiryPage 7

StorytellingRather than relying on a fixed set of prepared questions, ethnographic

interviewers encourage users to tell detailed stories about relevant work,

including both typical and untypical details.Other Ethnographic No-Nos:• Don’t seek the user’s

solution ideas• Don’t let users wallow in technical details

• Don’t conduct the interview away from where the action is• Don’t ask leading questions, by design or by accident

CS 321September 3, 2015InterviewsPage 8

Interview Methods

Open-EndedConversational, using questions that have answers without predetermined format and content.Benefit:Unexpected topics are often considered.

Drawback:Time-consuming and hard to analyze.

StructuredPredetermined questions, often requiring precise answers (e.g., multiple choice).Benefit:Very easy to analyze responses by grouping similarities.

Drawback:Clearly defined goals and precise wording are necessary.

Semi-StructuredMixes open and closed questions, with a preplanned script and additional probing.Benefit:Improves analysis capability while opening the door to additional insights.

Drawback:Probing and prompting may introduce bias.

GroupA facilitator guides and prompts discussion from a representative focus group.Benefit:Low cost, quick results, believable findings.

Drawback:Groups are hard to schedule and to manage.

CS 321September 3, 2015InterviewsPage 9

Unfortunately, interviewees don’t necessarily tell the truth during interviews.

Self-MisperceptionInterviewees might be deluding themselves concerning what they do and how well they do it.

Eager To PleaseInterviewees might be

trying to say what they think the interviewer

wants to hear. Try To Remember…Interviewees might not accurately recall what they’ve done or how long it usually takes.

Interviewee Honesty

CS 321September 3, 2015InterviewsPage 10

General Guidelines for Interview Questions• Avoid lengthy questions that are difficult to

remember.• Split compound questions into separate questions.

“How does this game compare with others you’ve played?”“How do you like this game?”“Have you played other games?”“If so, how did you like those games?”

• Avoid jargon that the interviewee might not understand.• Avoid leading questions.

“How is this game superior to others you’ve played?”• Avoid closed questions that permit one-word

answers.• Be alert to unconscious biases.

CS 321September 3, 2015InterviewsPage 11

Types of QuestionsBy using different types of questions, various views can be obtained and the information obtained may be validated.Descriptive

Questions“Explain how you would use this application in your daily routine.”Tends to produce a summaryLacks detailsProvides a good overview

Follow-Up Questions“You said this pull-down menu was confusing. What aspects of the menu make it confusing?”

Can foster antagonismHelps provide specific details

Experience Questions“Describe a feature that you found particularly memorable in this interface.”May decrease the interviewee’s responsivenessCan help direct the focus of the interview

Comparison Questions“You said that the gameplay here is really different from Doom. What are some of the differences?”

May narrow the focus of the interviewProvides the user with a basis for description

CS 321September 3, 2015InterviewsPage 12

Interview OrganizationTo facilitate the interview process and to put the interviewee at ease, the following

interview structure is suggested:1.IntroductionInterviewer introduces self; explains interview purpose; reassures interviewee concerning ethical issues (privacy, etc.); mentions recording (if necessary)2.Warm-Up SessionBegin with easy, non-threatening questions (e.g., demographic info)

3.Main SessionPresent questions in a logical sequence, with difficult questions at the end

4.Cool-Off PeriodA few easy questions to defuse tension, if any has arisen

5.Closing SessionThank the interviewee, switch off recorder, put notebook away