RUMM Handout or Final Paper

8/8/2019 RUMM Handout or Final Paper

1/20

Information Architecture Summit 2004 Uzanto Consulting1

Rapid User Mental Modeling (RUMM)

at eBay.com: A Case Study

Jonathan Boutelle

Rashmi Sinha

& Kirsten Swearingen

Uzanto Consulting

Larry Cornett

&

Ido Dan

eBay, Inc.


2/20


Background to Information Architecture Redesign of

eBay.com

eBay website had grown in organic fashion, leading to asomewhat inconsistent user experience

Although much design and usability work is done on site itself,this project was first effort to delineate user mental models forthe entire website.

Basic eBay facts Worlds largest online marketplace

16 million listings on any given day (as of Q2, 2003)

Almost 34 million active users worldwide

Buyers and sellers

New and experienced Occasional and full-time

Individuals and corporations

Different countries and cultures


3/20


Challenges

New IA would need to: Support sites vision of global economic democracy

Serve a large and diverse user population with varying goals and

mental models

Be future-oriented, allowing for planned change and growth

Occur through evolution not revolution -- must accommodatecurrent users who are very used to the site as it exists today

Senior management at eBay needed to see proof of the

need for a drastic restructuring.

First phase of project (background research) to becompleted in only 6 weeks


4/20


Overview of Design Research Methodology

Stage 1: Explore

Interviews and free-listing with

users and stakeholders

Stage 3: Verify / Refine

Large-sample closed

card-sorting study

Rapid UserMental Modeling

Stakeholders

User mental models

Preliminary IA

Stage 2: Understand

Open card-sorting study

site.com

TopCategory 3

TopCategory 1

TopCategory 2

Sub cat1

SubCat2

SubCat3

xx

SubCat1

Subcat4

Subcat3

Subcat2

xx

xx

xxx

SubCat3

SubCat5

SubCat4

SubCat6

SubCat4

xx

xxx

xx

xxx

xxx

xxx

SubCat4

SubCat5

SubCat1

SubCat2

xxx

xx

xx

Verified categorization

Scope of content& functionality


5/20


Our Approach to Design Research: Rapid User

Mental Modeling (RUMM)

Rigorous, three-stage design research methodology to layfoundation for IA redesign

Bring together established techniques in a novel way

Three stages: Exploration Understanding Verification

Individual components complement and build upon each other

Process moves from open-ended exploration to focused

information-gathering and hypothesis testing

Addresses individual and group differences in a systematic way

Produces reliable results (can be reproduced)

Uses mix of qualitative and quantitative methods

Qualitative: rich, open-ended understanding of users and domain

Quantitative: generates recommendations firmly grounded in data

(can help make case to management)


6/20


Towards Persuasive, Defensible Design

Research Important to create not just good design, butsuccessful design:

Serves user

Satisfies the business goals

Is accepted by management Results of design research phase must provide

rich picture of user, but also be persuasive anddefensible. Defensible: firmly grounded in data, can stand up to

scrutiny

Persuasive: persuades stakeholders that your solutionmakes sense and will provide benefit to the business


7/20


Stage 1: Explore Domain Through Stakeholder

Interviews

Goals: Explore domain in open-ended fashion

Identify both current and planned content andfunctionality

Method: Interviews with eBay users about specific

incidents of site usage

Interviews with stakeholders: top management,members of design team etc.

Identify problems, constraints, and specific plansfor future

Generate list of current tasks and Horizon Tasks(tasks planned for future)

Stakeholder analysis

Task list development


8/20


Stakeholder Interviews

Spoke to 16 eBay managers and designers Asked 3 questions:

What do you imagine a user doing on the eBay site two

to five years from now that they cant do today?

What one problem, if it were solved today, would causea large increase in the number of transactions executed

on the eBay platform?

What user population is least well served by the current

site?


9/20


Purpose of Stakeholder Analysis

Gain valuableperspectives for design research: Insure that design aligns with business strategy.

Tap into a companys existing knowledge in a

systematic fashion.

Ensure success of project by getting input at theresearch phase from those who have ability to say yes

or no.

Helps stakeholders feel involved in and understand the design

research and process.

Generate task list: helps map the scope and

boundaries of the information domain


10/20


Generating the Task List Using Free-Listing

Decided not to conduct inventory of site Incredibly time-consuming

Might tie new IA too closely to current sites design

Focused on tasks, since all eBay content is in context of

completing tasks:

Pay for an item

List an item

Generated initial list of 300 tasks by using free-listing

exercises with:

Users Designers

Stakeholders (especially important for identifying future tasks)


11/20


The Free-Listing Exercise

This simple, semi-structured method helps: Determine the scope of a domain

Gain some insight into how the domain is structured

Example: List all the animals you can think of

Results: Indices of salience:

Frequency

Rank

Indirect indices of similarity:

Co-occurrence. How often are cat and dog both mentioned?

Average rank distance. Are cat and dog mentioned togethermore closely than cat and horse?

In this case, we asked List all the tasks at eBay.com aperson can do / might be able to do in the future.


12/20


Refining the Task List

Consolidated task list, removing redundant items Divided tasks into groups:

Current Tasks (can be done on existing eBay website):

Core Tasks form core functionality of website

Boundary Tasks mark boundary of functionality offered

Horizon Tasks

Might be done on future versions of website

Important for creating a future-oriented IA

Selected representative items from each group

Result: comprehensive list of 100 tasks marking scopeand boundary of what users can accomplish on the eBay

site.

We were ready to begin studying user mental models!


13/20


Stage 2: Understand User Mental Models

Goals: Gain further insight into usermental models and identify top-level

categories for site content and

functionality

Method: open card sorting

Used list of 100 tasks (including 20 Horizon

Tasks) generated in previous stage.

28 participants, representing the main types of

eBay users (sellers and buyers; new and

advanced users).

22 completed card-sorts individually

6 worked as a group to create a structure (mix of

buyers, sellers, new, and advanced)


14/20


Identifying the Top Levels of the Hierarchy

Cluster analysis used to generateaggregate mental maps.

These were directly used to create

the hierarchical structure.

Looked at mental maps for fourmajor segments.

Identified the inconsistencies between

the mental maps of different groups.

Understood how to reconcile those to

create a new map. Result: preliminary IA, including 5

top-level categories that accounted

for all site content and functionality.

Bu d i ita l

Bid promo. option

Reverse auction

Negotiate price

Offer for unsold item

EsTrk car payment

Get buying tools

Contact seller

Contact buyer

Fn accessories

Fn items by seller

Search & browse

Fn BIN items

Fn items nearby

Fn specific item

Fn types of items

Browse items

Fn proFn services

View item desc.

Translate listingFn eBay internatLrn

Save items

Task Y

Task U

Task O

Task F

Task R

Task S

Task D

Task L

Task M

Task A

Task K

Task Ge

Task PTask X

Task B

Task Z

Task N

Dendrogram- User group !

Bu d i ita l

Bid promo. option

Reverse auction

Negotiate price


EsTrk car payment

Get buying tools

Contact seller

Contact buyer

Fn accessories

Fn items by seller

Search & browse

Fn BIN items

Fn items nearby

Fn specific item

Fn types of items

Browse items

Fn proFn services

View item desc.


Save items

Task Y

Task U

Task O

Task F

Task R

Task S

Task D

Task L

Task M

Task A

Task K

Task Ge

Task PTask X

Task B

Task Z

Task N

Bu d i ita l

Bid promo. option

Reverse auction

Negotiate price


EsTrk car payment

Get buying tools

Contact seller

Contact buyer

Fn accessories

Fn items by seller

Search & browse

Fn BIN items

Fn items nearby

Fn specific item

Fn types of items

Browse items

Fn proFn services

View item desc.


Save items

Task Y

Task U

Task O

Task F

Task R

Task S

Task D

Task L

Task M

Task A

Task K

Task Ge

Task PTask X

Task B

Task Z

Task N

Bu d i ita l

Bid promo. option

Reverse auction

Negotiate price


EsTrk car payment

Get buying tools

Contact seller

Contact buyer

Fn accessories

Fn items by seller

Search & browse

Fn BIN items

Fn items nearby

Fn specific item

Fn types of items

Browse items

Fn proFn services

View item desc.


Save items

Task Y

Task U

Task O

Task F

Task R

Task S

Task D

Task L

Task M

Task A

Task K

Task Ge

Task PTask X

Task B

Task Z

Task N

Dendrogram- User group !

Aggregate maps fromcluster analysis


15/20


Stage 3: Verify and Refine 5-Category

Scheme Goals:

Verify that the five top-level groupings cover all 100

tasks/concepts.

Identify tasks/concepts that do not fit in, as well as

where users do expect to find such information. Examine differences in groupings among various user

groups.

Further develop IA for eBay site.

Establish method for future verification and refinementof eBay IA.


16/20


Stage 3: Verify and Refine 5-Category

Scheme Method: closed card sorting

Large sample study with more than 1,000 participants

representing all segments of eBay users

Study conducted online.

Participants asked to categorize tasks to one of the fivetop-level categories (or to select Other if it didnt

belong)

Result: site-map showing the structure for the

main parts of the eBay site.


17/20


Analysis of Closed Card Sort Results

Category-belonging scores: Percent of users who think a task belongs to a particular category

Lack of consensus or high scores in Other category may signal a

design issue

Category Memberships across all participants

Task N

Category

1

Category

2

Category

3

Category

4

Category

5

Category

6

Task 1 870 48% 7% 21% 14% 3% 6%

Task 2 991 8% 1% 85% 1% 2% 3%

Task 3 959 7% 0% 86% 2% 1% 4%

Task 4 918 0% 0% 87% 3% 6% 4%

Task 5 929 0% 2% 83% 6% 6% 2%

Task 6 896 46% 5% 7% 10% 27% 5%

Task 7 908 55% 4% 15% 9% 11% 6%

Task 8 893 35% 2% 49% 1% 1% 12%

Task 9 870 1% 1% 67% 1% 3% 27%

Task 10 993 76% 1% 19% 0% 1% 4%

A color-coded visualization of closed card sorting data


18/20


Reconciling Different Mental Models

Results suggeststructural possibilities--they donotprescribe one solution

In both open and closed card sorting, found

differences in mental models

Reconcile through design, if possible (revisitdendograms)

The most important, contentious issues may ultimately

require a decision driven by company brand and

business goals (revisit stakeholder analysis)


19/20


20/20


Impact of Project

Helped convince management that current site did notmatch user mental models and that IA redesign wasneeded.

Provided basis for a long-term redesign plan to implementa structured site-wide model.

Provided immediate guidance to near-term projects inorder to begin moving the site in the long-term direction.

Broadened eBay designers perspective on overall designdirection and goals by providing: Access to user mental models

Site-map for all content and functionality of eBay.com

Cleared the road towards a more consistent and unifieduser experience.

Date post:	10-Apr-2018
Category:	Documents
Upload:	suraj3974
View:	219 times
Download:	0 times

RUMM Handout or Final Paper

Documents