Needs Analysis: Design through Discovery

2/12/04 Slide 1SBC Laboratories

Needs Analysis:Design through Discovery

Philip Kortum, Ph.D.

Robert Bushey, Ph.D.

SBC Laboratories

Human Factors


Today’s Agenda

Who we are (5 min) Basic needs analysis techniques (60 min) Virtual Lab tour (10 Min) Break (10 min) Needs analysis in high-data environments (70 min) Conclusions (10 min)


Who is SBC?

Short Answer:•We’re the phone company

Long Answer:•2nd largest telecommunications providers the US

•Wireless•Long Distance•Data•Residential/Business phone service

•Over 60 million access lines •$41 Billion in Revenue (# 27 on the Fortune 500)


Who is SBC Laboratories?

SBC Laboratories is the Applied Research and Development arm of SBC.

About 200 Advanced-degreed Scientists and Engineers work at SBC’s laboratories in Austin and the Bay area in California.

BroadbandArchitecture,

Infrastructure, & Services

IntelligentNetworks

InformationTechnology

Internet WirelessSystems

Technology Focus Areas


SBC Laboratories Human Factors Engineering

Stages in the Product and System Development/ Acquisition Cycle

Human FactorsActivities

ConceptGeneration

High-LevelDesign/RFI

DetailedDesign/RFP

Developmentand Testing

Release Evaluation

User NeedsAssessment

UserProfile

CompetitiveAnalysis

System/UserModelingUsability

SpecificationUser Interface

DesignUsability

EvaluationUser Interface

Design GuidelinesProduct

Integration


Where today’s talk fits in this class

From Mayhew, 1999


What is Needs Analysis?

Needs analysis is the method of uncovering user requirements

through direct or indirect interaction with that user


Why do we perform needs analysis?

We are not the user Users may not know what they really need

» User is unable to articulate the need» True needs are often masked

Good match between needs and implementation leads to superior efficiency and usability


How is Needs Analysis different than Task Analysis?

Needs Analysis – trying to uncover the underlying motivations

Task Analysis – trying to uncover the procedural steps


How Is Needs Analysis Different Than Task Analysis?

Task Analysis» Setting up a VCR

» Programming a VCR

Needs Analysis» The ways a person uses a VCR and

the reasons behind that use

– What they record

– What they do with what they record

– Time-shifting

– Remote control use use


How Is Needs Analysis Different Than Marketing?

They both deal with the user They both try to determine what the user

‘needs’ Some of the techniques are similar Some of the information collected is the same


How Is Needs Analysis Different Than Marketing?

Marketing

Demographics» Where people live, shop» How much they make» Physical attributes

Purchasing behavior patterns

» What people will buy» Price points

Needs Analysis

User behavior patterns» What people do

What users know Users environment Users mental models


Needs vs. Marketing ExampleA device that records off of the television

Marketing How much money do the

users have to spend? What other similar devices do

they have? What kinds of technology do

they own? How much do they use these

technologies

Needs Analysis Why would they want to use

an image recorder? How do they use current

devices? When do they use such

devices? What other technology

interactions are there?


A Simple (if unfair) Characterization:

Marketing deals with wants

Needs deals with needs


A word about Semantics….

Many techniques are similar and are called different things by those who use them (even within the usability community)

Good practitioners almost always use a mix of techniques, and often call these mixtures by a new name.


Needs Analysis

Doesn’t have to ‘big sky’

Can be very focused and limited in scope


Basic Types of Needs Analysis

Questionnaires Interviews Observational Research Hybrid methodologies


Questionnaires

“A set of questions for obtaining statistically useful or personal information from users”

Webster


Questionnaire Advantages

Easy to administer» Inexpensive» Can be administered by untrained staff

Can be administered remotely Can have a relatively large number of

questions and data samples Can more easily target highly specific users Can branch to reliably capture pertinent

information Data can be simple to code


Questionnaire Disadvantages

Difficult to construct a good test instrument Must know a priori exactly what you want

to ask » No follow-up questions

Question intent may be open to interpretation» Can result in unknown bad results

Open ended responses highly variable Open ended questions are difficult to code Validation/verification difficult if study is

not closely controlled


Questionnaire Disadvantages: Fixes

Difficult to construct a good test instrument» Follow scientifically designed questionnaire guides

(Kirakowsi, Gillham, USARI)» Test for validity, reliability, repeatability

Must know a priori exactly what you want to ask » Conducted pilot tests to try out the questions» Conduct interviews to help build a reasonable first pass» Use domain experts to help capture detail questions



Question intent may be open to interpretation» Conduct pilot tests to check for weaknesses» Use experts and novices to examine interpretation

differences» Use unambiguous, quantifiable, anchored language


Ambiguous Question Example

How bad was your last car accident?Really bad bad not really bad not bad at all

2 respondents, who both dented the front bumper slightly, no injury

Respondent 1: ‘Really Bad’

“I’m only 16, it’s my first accident and it was my dad’s new sports car, and I wasn’t on the insurance for the car. I’m going to be grounded for life!

“Well, compared to that 23 car roll-over collision I had with that semi last month, this was nothing. Not to mention that I got the collision waiver on the rental!

Respondent 2: ‘Not bad at all’

Class: How could we fix this?



Open ended responses highly variable» Provide hints, examples to guide the responses» Use multi-part questions that are more precise

Open ended questions are difficult to code» Use more choices, or more questions, to remove need

for open ended questions» Focus on variables of interest and use ‘other’ to capture

low probability events that are not pertinent» Devise keyword codebooks to help code open ended

data



Validation/verification difficult if study is not closely controlled» Key code questionnaires to match user and instrument» Lock-out multiple take attempts» Ask qualifying questions as part of a pretest, or branch


Interviews

“A meeting at which information

is obtained from a person” Webster


Interviews

Generally classified into two types:•Structured•Unstructured


Interview Advantages

Small numbers of interviews can be performed quickly and inexpensively (although not always)

Can be conducted remotely Can be conducted in groups or

individually Can be conducted with little notice, if

needed Can change the interview for different

levels of users


Interview Advantages

Open for opportunistic data discovery Get non-verbal clues to help guide the

interview Good technique for ‘fishing expeditions’ Useful for gathering preliminary

information to guide later needs analysis


Interview Disadvantages

Hard to conduct a ‘good’ interview» Easy to lead the respondent, with both verbal and non-

verbal cues» Easy to unknowingly ask loaded questions

– “How does it feel to be ugly?”

Harder still to conduct a good group interview» herd mentality, strong leader

Hard to stay ‘up’ and focused for a large number of interviews



Data can be difficult to code and quantify Can be large differences between

interviewers Expensive and time consuming to conduct

large numbers of interviews Usually a low upper limit to the number of

questions



Interview Example



Interview data may not match reality

What was said What the data suggests

Reps make 1 offer per call

‘order takers’ are the least desirable

Taking lots of calls is what makes customers happy

Almost no reps make 1 offer, some do 0 some do 2+

Order takers are good - they drive accessibilityand are among top $ performers

No supporting data

Question

How many offers do reps make per call?

What kind of rep is the least desirable?

What makes the customer happy?


Interview Disadvantages: Fixes

Hard to conduct a ‘good’ interviews» Use scientifically developed methods (i.e. Weiss’s

Learning from Strangers)» Pre-develop a core set of questions» Manage group dynamics» Use hypothetical questions to elicit more detailed

answers

Hard to stay ‘up’ and focused for a large number of interviews» Limit the number of interviews» Use tag teams, alternate sessions» Use well trained ‘people persons’



Data can be difficult to code and quantify» use core set of questions » use data code books to help quantify responses» use affinity diagrams to aid in coding

Can be large differences between interviewers» use single interviewer» use visible or hidden teams» conduct pilot training to level the interviewers



Expensive and time consuming to conduct large numbers of interviews

Usually a low upper limit to the number of questions» target exact population» supplement with other techniques» use remote techniques» use core question set, based on pre-test

Interview data may not match reality» Always verify data


Observational Research

“An act of recognizing and

noting a fact or occurrence…” Webster


Observational Research

Observation doesn’t have to occur in real time

Observation doesn’t have to be visually based» audio recordings

» diaries» telemetry (GPS, web logs, biometrics, equipment monitors) (e.g. car

snooper)


Observational Advantages

Good for understanding complex needs Results are not defined by the design of the

method Capable of capturing unknown or undocumented

behaviors Gather data on specific artifacts Can uncover inter-dependencies If documented appropriately, can review

observations as many times as necessary Can parse expert/novice distinction


Observational Disadvantages

Some phenomenon is too infrequent to catch

Heisenberg Principle (i.e. driving tester) Can be more difficult to obtain a

reviewable record Desired behavior may be interspersed

with other unrelated activity Slow, expensive to collect data Can easily get ‘off-target’ Data set can be overwhelming


Observational Disadvantages: Fixes

Some phenomena are too infrequent to catch» use high sample rates to find low probability events» ‘Create’ low probability (disaster training)» sample in areas known to have these events» be lucky

Heisenberg Principle (i.e. driving tester)» observe for extended periods of time » use remote measuring techniques» use low/no interaction models» clearly understand the cost-benefit of direct interaction» work in the users environment



Can be more difficult to obtain a reviewable record» use alternate recording technologies» work in teams» Use advanced behavioral software (e.g. Noldus Observer)

Desired behavior may be interspersed with other unrelated activity» use specific artifact techniques» use remote data collection techniques to time compress» don’t assume that unrelated activity is actually unrelated (e.g.

using the restroom while waiting)



Slow, expensive to collect data» use remote or automated collection where possible» understand required sample size» carefully select person/situation/setting to be observed

Can easily get ‘off-target’» use pilot observations to help identify key behaviors» be open to the fact that off-target behaviors may be

pertinent



Data set can be overwhelming» use data reduction/consolidation techniques

– training samples– affinity diagrams– physical flow/sequence structures


Observational Research ExampleWatching Television


Observational Research ExampleWatching Television

Nathen, et al, 1985


Hybrid Methodologies

Ethnography Contextual Inquiry Empathic Design Participatory learning

– actors– rotational management assignments


What happens when needs isn’t done?


What happens when needs isn’t done?


Questions?

Date post:	20-Jan-2016
Category:	Documents
Upload:	cicero
View:	37 times
Download:	0 times

Needs Analysis: Design through Discovery

Documents