Date post: | 19-Dec-2015 |
Category: |
Documents |
View: | 218 times |
Download: | 1 times |
1
Designing Information Architecturefor Search
Marti Hearst
University of California, Berkeleywww.sims.berkeley.edu/~hearst
NSF CAREER Grant, NSF9984741
Tutorial: SIGIR 2001
2
Outline
1. Motivation2. Search Interfaces:
1. Web search vs Site Search2. Search UIs: What works; what doesn’t
3. Methodology1. Information Architecture Defined2. Faceted Metadata3. Integrating Search into IA via Faceted Metadata
4. Results of Usability Studies5. Tools6. Conclusions
3
Contributors to the Research
• Dr. Rashmi Sinha• Graduate Students
– Ame Elliott– Jennifer English– Kirsten Swearington– Ping Yee
• Research funded by – NSF CAREER Grant, NSF9984741
4
Motivation and Background
5
Claims
• Web Search is OK– Gets people to the right starting
points
• Web SITE search is NOT ok• The best way to improve site
search is– NOT to make new fancy algorithms– Instead …
6
The best way to improve search:
Improve the User Interface
7
Recent Study by Vividence Research
• Spring 2001, 69 web sites– 70% eCommerce– 31% Service– 21% Content– 2% Community
• The most common problems:53% had poorly organized search results32% had poor information architecture32% had slow performance27% had cluttered home pages25% had confusing labels15% invasive registration13% inconsistent navigation
8
Vividence findings: effects on users
• Poorly organized search results– Frustration and wasted time
• Poor information architecture– Confusion– Dead ends– "back and forthing"– Forced to search
9
Vividence findings: effects on users
• Cluttered home pages– Creates disinterest– Wastes time– No contrast: everything has equal weight– Don’t know where to start– Failure to engage– No call to action– Failure to establish navigation– Layout reflects company organization chart– Investor centeredness
10
Vividence findings: characteristics
• Inconsistent Navigation– Primary navigation bar is, in fact, really
secondary– Un-scalable designs– Poor transitions between company divisions– "Junk Drawer" navigation bars– Random links– Shoe-horned functions– Heavy need to hit the "back-button"
11
Vividence Study
• Breakdown of most common search problems– 41% - of searches encountered no problems– 20% - had search problems not named below– 14% - of searches were not “advanced” enough– 12% - did not organize results well– 10% - of searches yielded inaccurate/unrelated results– 9% - were too slow– 8% - of searches had insufficient instructions– 7% - engine was too difficult to locate– 7% - of searches produced too few results– 7% - of searches were too limiting– 3% - of searches produced an error message– 3% - were too difficult to use
12
Other Relevant Studies
• Commercial studies (are not usually scientific, do not supply full details)– CreativeGood.com Holiday 2000 ecommerce
report– UIE, and Jared Spool’s talks:
http://world.std.com/~uieweb• Scientific studies (often less relevant to real web situations)
– Many papers from the CHI proceedings http://www.acm.org/dl/
– Papers from Human Factors and the Web http://www.optavia.com/hfweb/
– See the extensive bibliography from my textbook chapter (in this package).
13
The Philosophy
• Information architecture should be designed to integrate search throughout
• Search results should reflect the information architecture.
• This supports an interplay between navigation and search
• This supports the most common human search strategies.
14
The Approach
• Assign faceted metadata to content items
• Allow users to navigate through the faceted metadata in a flexible manner
• Organize search results according to the faceted metadata so navigation looks similar throughout
• Give previews of next choices• Allow access to previous choices
15
Advantages of the Approach
• Supports different task types– Highly constrained known-item
searches use one interface– Open-ended, browsing tasks use
another interface– Both types of interface use the same
underlying structure– Can easily switch from one interface
type to the other midstream
16
Advantages of the Approach
• Honors many of the most important usability design goals– User control– Provides context for results– Reduces short term memory load– Allows easy reversal of actions– Provides consistent view
17
Advantages of the Approach
• Allows different people to add content without breaking things
• Can make use of standard technology
18
Web Search vs. Site Search
19
Web Search is Working!
Survey finds high user satisfactionStudy by npd group
http://www.searchenginewatch.com/reports/npd.html
20
Why is Web Search Working?
• Web Search is Successful at Finding Good Starting Points (home pages)
• Evidence: – Search engines using
• Link analysis• Page popularity• Interwoven categories
– These all find dominant home pages
21
22
23
Organizing Search Results:What works, What Doesn’t
• There is a lot of prior work on this– Cha-Cha (Chen et al. 1999)
– Scatter-Gather clustering (Cutting et al. 93, Hearst et al. 1996)
• Becoming more prevalent in web search too.– Teoma– Vivisimo– Northern Light
24
Putting Results into Clusters
25
Drilldown – what does it mean?
26
Vivisimo – same idea
27
28
Yahoo lists category matches
29
Web Search Results Grouping
• Drill down one category • Cannot mix and match categories• Not clear if it is useful or not
– Can help differentiate different meanings of the same word.
• But …what about site search?
30
If Web search engines are providing source selection …
… what happens when the user gets to the site?
Follow Links … or …
Search
31
Following Hyperlinks
• Works great when it is clear where to go next
• Frustrating when the desired directions are undetectable or unavailable
Site SearchIs not getting good reviews
32
An Analogy
text searchhypertext
33
Analogy
• Hypertext: – A fixed number of choices of where to go next; – A glance at the map tells you where you are;– But may not go where you want to go.
• To get from Topeka to Santa Fe, may have to go through Frostbite Falls
• Site Search:– Can go anywhere;– But may get stuck, disoriented, in a crevasse!
34
Goal: An All-Tertrain Vehicle
• The best of both techniques– A vehicle that magically lays down
track to suggest choices of where you want to go next based on what you’ve done so far and what you are trying to do
– The tracks follow the lay of the land and go everywhere, but cross over the crevasses
– The tracks allow you to back up easily
35
Organizing Search ResultsWhat works; what doesn’t
36
What works, what doesn’t
• There is negative evidence for– Clustering– Fancy visualizations
• There is positive evidence for– Grouping into meaningful, consistent
categories– Relevance feedback
• Depends how you do it– Showing similar items
37
Koh
on
en F
eatu
re M
aps
on
Text
(fro
m C
hen e
t al.,
JAS
IS 4
9(7
))
UWMS Data Mining Workshop
Study of Kohonen Feature Maps
• H. Chen, A. Houston, R. Sewell, and B. Schatz, JASIS 49(7)
• Comparison: Kohonen Map and Yahoo• Task:
– “Window shop” for interesting home page– Repeat with other interface
• Results:– Starting with map could repeat in Yahoo
(8/11)– Starting with Yahoo unable to repeat in map
(2/14)
UWMS Data Mining Workshop
Study (cont.)
• Participants liked:– Correspondence of region size to #
documents– Overview (but also wanted zoom)– Ease of jumping from one topic to
another – Multiple routes to topics– Use of category and subcategory
labels
UWMS Data Mining Workshop
Study (cont.)
• Participants wanted:– hierarchical organization– other ordering of concepts (alphabetical)– integration of browsing and search– corresponce of color to meaning – more meaningful labels– labels at same level of abstraction– fit more labels in the given space– combined keyword and category search– multiple category assignment (sports+entertain)
41
Visualization of Clusters
– Huge 2D maps may be inappropriate focus for information retrieval • Can’t see what documents are about• Documents forced into one position in
semantic space• Space is difficult to use for IR purposes• Hard to view titles
– Perhaps more suited for pattern discovery• problem: often only one view on the
space
42
Summary: Clustering(Based on other studies as well)
• Advantages:– Get an overview of main themes– Domain independent
• Disadvantages:– Many of the ways documents could group
together are not shown– Not always easy to understand what they mean– Different levels of granularity
• Probably best for scientists only• Take heart – there is good evidence for
organizing via categories!
43
The DynaCat System
• Decide on important question types in an advance– What are the adverse effects of drug D?– What is the prognosis for treatment T?
• Make use of MeSH categories• Retain only those types of categories
known to be useful for this type of query.
Pratt, W., Hearst, M, and Fagan, L. A Knowledge-Based Approach to Organizing Retrieved Documents. AAAI-99: Proceedings of the Sixteenth National Conference on Artificial Intelligence, Orlando, Florida, 1999.
44
DynaCat
45
DynaCat Study
• Design– Three queries– 24 cancer patients– Compared three interfaces
• ranked list, clusters, categories
• Results– Participants strongly preferred categories– Participants found more answers using
categories– Participants took same amount of time with
all three interfaces
46
Cha-C
ha (
intr
anet
searc
h)
Cha-Cha: A System for Organizing Intranet Search Results, by Chen, Hearst, Hong, and Lin, Proceedings of 2nd USENIX Symposium on Internet Systems, Boulder, CO, Oct 1999. cha-cha.berkeley.edu
47
Cha-C
ha (
intr
anet
searc
h)
48
How People Search
49
The Standard Model
• Assumptions:– Maximizing precision and recall
simultaneously– The information need remains static– The value is in the resulting
document set
50
“Berry-Picking” as an Information Seeking Strategy (Bates 90)
• Berry-picking model– Interesting information is scattered
like berries among bushes– The user learns as they progress, thus – The query is continually shifting
51
A sketch of a searcher… “moving through many actions towards a general goal of satisfactory completion of research related to an information need.” (after Bates 89)
Q0
Q1
Q2
Q3
Q4
Q5
52
Search Tactics and Strategies
– Marcia J. Bates, Information Search Tactics, Journal of the AmericanSociety for Information Science, 30, 4, 1979
– Marcia J. Bates, Where should the person stop and the informationsearch interfaces start?, Information Processing \& Management, 26,
5,1990
Marcia J. Bates, The Berry-Picking Search: User Interface Design, UserInterface Design, Harold Thimbleby, Addison-Wesley, 1990
– Marcia J. Bates, The design of browsing and berrypicking techniquesfor the on-line search interface, Online Review, 1989, 13, 5,407—431.
– Vicki L. O'Day and Robin Jeffries, Orienteering in an informationlandscape: how information seekers get from here to there,
Proceedings of ACM INTERCHI '93, April, Amsterdam, 1993
53
Tactics vs. Strategies
• Tactic: short term goals and maneuvers– operators, actions
• Strategy: overall planning– link a sequence of operators together
to achieve some end
54
An Important Strategy
• Do a simple, general search– Gets results in the generally correct area
• Look around in the local space of those results
• If that space looks wrong, start over– Akin to Shneiderman’s overview + details
• Our approach supports this strategy– Integrate navigation with search
55
Term Tactics
• Move around a thesaurus– Look at category labels– Look at related terms– Look at parent terms– Look at child terms
• In older literature, refers to navigating the thesaurus itself, as opposed to the items themselves.
56
Source-level Tactics
• “Bibble”:– look for a pre-defined result set – e.g., a good link page on web
• Survey:– look ahead, review available options– e.g., don’t simply use the first term or first
source that comes to mind
• Cut:– eliminate large proportion of search domain– e.g., search on rarest term first
57
Source-level Tactics (cont.)
• Stretch– use source in unintended way– e.g., use patents to find addresses
• Scaffold– take an indirect route to goal– e.g., when looking for references to
obscure poet, look up contemporaries
58
Monitoring Strategies
• Check– compare original goal with current state
• Weigh– make a cost/benefit analysis of current or
anticipated actions
• Pattern– recognize common strategies
• Correct Errors• Record
– keep track of (incomplete) paths
59
Additional Considerations(Bates 79)
• Need a Sort tactic• When to stop?
– How to judge when enough information has been gathered?
– How to decide when to give up an unsuccesful search?
– When to stop searching in one source and move to another?
60
Information Architecture
61
A Taxonomy of WebSites
low
low
high
high
Complexity of Applications
Complexity of Data
From: The (Short) Araneus Guide to Website development, by Mecca, et al, Proceedings of WebDB’99, http://www-rocq.inria.fr/~cluet/WEBDB/procwebdb99.html
Catalog Sites
Web-based Information
Systems
Web-Presence
Sites
Service-Oriented
Sites
62
A View of Website Design
• Information design– structure, categories
of information
• Navigation design– interaction with
information structure
• Graphic design– visual presentation of
information and navigation (color, typography, etc.)Information Architecture
From Sitemaps, Storyboards, and Specifications: A Sketch of Web Site Design Practice as Manifested Through Artifacts. M.W. Newman and J.A. Landay. In proceedings of Designing Interactive Systems: DIS '00. August 17-19, 2000.
63
A View of Information Architecture
• Content Items +• Information Structure +• Navigation Structure +• Layout
64
Content Items
– The information items that the site is designed to show the user.
– Individual content items can be considered leaves in a tree, or base-level items.
– Aggregates of individual (base-level) items can be considered to be content items.
– This definition is especially relevant for catalog-style sites, for example:
• Image collection• Product selling• Collection of articles on some topic (medical, legal)• Collection of information about some entity (IRS, Park
Service)
65
Information Structure
• Independent of the website.• A set of descriptors which are used to
characterize the content of a website.• Consists primarly of a category structure
and a set of textual labels.• The categories can have flat, hierarchical,
faceted or network structure.• The textual labels include alternative
ways of expressing the same concepts (synonyms).
66
Navigation Structure
• Defined in terms of the website.• Site level:
– The paths connecting content items throughout the site.
• Page level: – The link from one page to others.
67
Exam
ple
fro
m W
alm
art
.com
68
Content
Navigation Structure
69
Related Items
• Often are content items• Related to the target by some
shared information structure• The particular related items that
are shown are revealed through the navigation structure
70
The Information Structure
• Consists of a set of descriptors for the content items
• Can’t really see it directly, since it is independent of web site description
• Can see parts of it in the navigation structure
71
A View of Information Architecture
Content Items
Information Structure
Start with an information structure (categoriesand labels) and a set of content items.
72
A View of Information Architecture
Content Items
Information Structure
Each content item is assigned some descriptors from the information structure.
Prod: CampingBrand: REIMaterial: NylonSize: 4-person
73
Navigation structure links items or groups of items.
74
Navigation Structure Differs from Information Structure
• Example:– Part of the info structure is the
product hierarchy.– Some products are assigned more
than one spot in the hierarchy (e.g., sports and games), thus forming a tree structure
– Navigation structure shows a progressive disclosure of the hierarchical structure only.
75
Navigation Structure Differs from Information Structure
• Example:– Main navigation structure is the
product hierarchy.– However, “lateral” links are shown
from product leaf nodes to other nodes• (e.g., from a tent to a flashlight and a
sleeping bag)
76
Navigation Structure Differs from Information Structure
• The differences can be much more profound
• Examples:– Show only main product categories at
top levels– After a search, show links according
to brands of items, but only those brands that make sense for the items retrieved by the search.
77
“Breadcrumbs”
• A navigation technique for showing either history or contextualizing hierarchy via hyperlinks.
• Two main types:– Hierarchy without history:
• Search results at walmart.com
– History across facets (without hierarchy):• Epicurious path recording.
78
An Important IA Trend• Generating web pages from databases• Implications:
– Web sites can adapt to user actions– Web sites can be instrumented – “An essential feature of a design
environment is to give authors the possibility of evaluating the current network against the final adaptive system.”
» Petrelli, Baggio, & Pezzulo, Adaptive Hypertext Design Environments: Putting Principles into Practice, AH 2000
79
Faceted Metadata
80
Metadata: data about dataFacets: orthogonal categories
Time/Date Topic RoleGeoRegion
81
Faceted Metadata: Biomedical MeSH (Medical Subject Headings)www.nlm.nih.org/mesh
82
Mesh Facets (one level expanded)
83
Using Mesh Facets
• Some stats:– >18,000 labels– avg depth: 4.5, max depth 9– ~8 labels/article on average
• How to go from the information structure to the navigation structure?
84
Using faceted metadata incorrectly
• Yahoo uses faceted metadata poorly in both their search results and in their top-level directory
• They combine region + other hierarchical facets in awkward ways
85
Yahoo’s use of facets
86
Yahoo’s use of facets
87
Yahoo’s use of facets
88
Yahoo’s use of facets
Where is Berkeley? College and University > Colleges and Universities >United States > U > University of California > Campuses > Berkeley
U.S. States > California > Cities >Berkeley > Education > College and University > Public > UC Berkeley
89
However, Yahoo does use some metadata well• Yahoo restaurant guide combines:
– Region – Topic (restaurants) – Related Information
• Other attributes (cuisines)• Other topics related in place and time
(movies)
90
Green: restaurants & attributes
Red: related in place & time
Yellow: geographic region
91
Combining Information Types
• Region– State
• City
• A & E– Film– Theatre– Music– Restaurants
• California• Eclectic• Indian• French
Assumed task: looking for evening entertainment
92
Other Possible Combinations
• Region + A&E• City + Restaurant + Movies• City + Weather• City + Education: Schools• Restaurants + Schools• …
93
Bookstore preview combinations
• topic + related topics• topic + publications by same author• topic + books of same type but related
topic
94
Problems with Metadata Usage
• Standard approaches– Paths are hand-edited, predefined– Not well-integrated with search– Not tailored to task as it develops– Not personalized– Not dynamic
95
Questions we are trying to answer
• How many facets are allowable?• Should facets be mixed and
matched?• How much is too much?• Should hierarchies be progressively
revealed, tabbed, some combination?
• How should free-text search be integrated?
96
Recipe Collection Examples
97
From soar.berkeley.edu (a poor example)
98
99
From www.epicurious.com (a good example)
100
101
102
103
Epicurious Metadata Usage
• Advantages– Creates combinations of metadata on the fly– Different metadata choices show the same
information in different ways– Previews show how many recipes will result– Easy to back up– Supports several task types
• ``Help me find a summer pasta,'' (ingredient type with event type), • ``How can I use an avocado in a salad?'' (ingredient type with dish
type), • ``How can I bake sea-bass'' (preparation type and ingredient type)
104
Metadata usage in Epicurious
PrepareCuisineIngredient Dish
Recipe
105
Metadata usage in Epicurious
PrepareCuisineIngredient Dish
PrepareCuisineDishISelect
106
Metadata usage in Epicurious
PrepareCuisineIngredient Dish
I >
Group by
PrepareCuisineDish
107
Metadata usage in Epicurious
PrepareCuisineIngredient Dish
PrepareCuisineDishI >
Group by
108
Metadata usage in Epicurious
PrepareCuisineIngredient Dish
PrepareCuisineDishI >
Group by
PrepareCuisineISelect
109
Recipe Information Architecture
• Information design– Recipes have five types of metadata
categories• Cuisine, Preparation, Ingredients, Dish,
Occasion• Each category has one level of
subcategories
110
Recipe Information Architecture
• Navigation design– Home page:
• show top level of all categories
– Other pages:• A link on an attribute ANDS that attribute
to the current query; results are shown according to a category that is not yet part of the query
• A change-view link does not change the query, but does change which category’s metadata organizes the results
111
Metadata Usage in Epicurious
• Can choose category types in any order• But categories never more than one
level deep• And can never use more than one
instance of a category– Even though items may be assigned more
than one of each category type
• Items (recipes) are dead-ends– Don’t link to “more like this”
• Not fully integrated with search
112
Epicurious Basic Search
Lacks integration with metadata
113
114
Information previews
• Use the metadata to show where to go next– More flexible than canned hyperlinks– Less complex than full search
• Help users see and return to what happened previously
• Reduces mental work– Recognition over recall– Suggest alternatives
115
The Importance of Information Previews
• Jared Spool’s studies (www.uie.com)
• More clicks are ok if– The “scent” of the target does not
weaken– If users feel they are going towards,
rather than away, from their target.
116
Problem with Metadata Previews as Currently Used
– Hand edited, predefined– Not tailored to task as it develops– Not personalized– Often not systematically integrated
with search, or within the information architecture in general
117
Putting it Together
118
Desiderata for Objects inInformation-Seeking Workspaces
• Structured• Fractal• Queriable• Navigable• Historical• Similarity Engine Compatible• Contextualized• Other
From Furnas, G., and Rauch, S., Considerations for information environments and the NaviQue workspace. In Proceedings of DL 98. Pittsburgh,PA, June, 1998.
Search Usability Design Goals
1. Strive for Consistency2. Provide Shortcuts3. Offer Informative Feedback4. Design for Closure5. Provide Simple Error Handling6. Permit Easy Reversal of Actions7. Support User Control8. Reduce Short-term Memory Load
From Shneiderman, Byrd, & Croft, Clarifying Search, DLIB Magazine, Jan 1997. www.dlib.org
120
Analogy: Chess
• Chess is characterized by a few simple rules that disguise an infinitely complex game
• Another intriguing characteristic: the three-part structure– Openings: many strategies, new ones all the
time, many books on this– Endgame: well-defined, well-understood– Middlegame: nebulous, hard to describe
• Our thought: search is similar and the middlegame is critically underserved.
121
Chess-based view of Info Architecture
• The Opening:– Usually exposes top-level hierarchy or
top-level facets (or both)– Usually also has a search component– This is also the place to expose the
main tasks that can be accomplished on the site
122
The Opening
123
The Opening
124
The Opening
125
The Opening
126
The Opening
127
Chess-based view of Info Architecture
• The Endgame:– Has become rather well-established in
shopping sites– Penultimate page: shows a list of
items– Leaf node:
• Shows one content item in detail• Lateral links
– To similar items (same facet)– To other items that go with it (other facets)
128
The Endgame – Penultimate Pages
129
The Endgame – Penultimate Pages
130
The Endgame – Leaf Nodes
131
Chess-based view of Info Architecture
• The Middlegame:– Hardest to describe/understand– The “berry-picking” part of supporting
search– Issues:
• How to progressively expose hierarchies?• How to show multiple facet choices?• How to integrate with search results?• How to show history / retain context?
132
Sophisticated Middlegames
zdnet.comContinued next slide
133
Sophisticated Middlegames
134
Sophisticated Middlegames
Walmart.comContinued next slide
135
Sophisticated Middlegames
136
Sophisticated Middlegames
137
Online Grocery Shopping Examples
• In each case, note– Chess analogy
• What is the opening?• What is the endgame?• How is the middlegame handled?
– How are search results integrated?– How is hierarchical drill-down
revealed?– Are multiple facets allowed?
138
Grocery shopping example
Homerun.com
139
Grocery shopping example
Homerun.com
140
Grocery shopping example
Homerun.com
141
Grocery shopping example
peapod.com
142
Grocery shopping example
peapod.com
143
Grocery shopping example
peapod.com
144
Grocery shopping example
webvan.com
145
Grocery shopping example
webvan.com
146
Summary: Grocery Shopping Examples
• A good opening seems to make a big difference
• Familiar metadata helps make the task easier
• Middlegame hierarchy exposure– One uses cascading menus– Two use webpage-based drilldown
• Two use metadata to organize search results– But don’t use metadata creatively– Could organize by recipe, etc.
147
Medical Text Example
– Allow user to select metadata in any order
– At each step, show different types of relevant metadata, • based on prior steps and personal
history, • include # of documents
– Previews restricted to only those metadata types that might be helpful
148
Ecommerce Examples
• E-commerce sites are farther ahead than information collection sites
• However, their problem is usually easier– Single facet often works fine– Categories are familiar to users– Collections are often much smaller
• How to move this to large sites containing more abstract information?– Image collections?– Text collections
149
Current Search Approach
Can a metadata preview approach do better?
150
Asthma > Steroids
1. A steroid-induced acute psychosis in a child with athsma.2. Management of steroid-dependent asthma with methotrexate.
1. A steroid-induced acute psychosis in a child with athsma.2. Management of steroid-dependent asthma with methotrexate.
Steroids•Pregnanes• Pregnadienes (5)• Prednisone (5)• Pregnenes• Budesonide (4)• Corticosterone (3)
Other Views• Admin & Dosage (50)• Drug Effects (20• Therapeutic Use (25)• Risk Factors (4)• More …
User Preferred• Musculoskeletal (4)•Drug Resistance (6)
•All Categories (99)
99 Documents: [Sort by author] [Sort by popularity] [Sort by Steroids] [Cluster]
1. Effect of short-course budesonide on the bone turnover of asthmatic children.2. Effect of prednisone on response to influenza virus vaccine in asthmatic children.…
1. Effect of short-course budesonide on the bone turnover of asthmatic children.2. Effect of prednisone on response to influenza virus vaccine in asthmatic children.…
151
Asthma > Steroids > Admin & Dosage
1. Dosage levels for asthmatic steroids: A survey.1. Dosage levels for asthmatic steroids: A survey.
Steroids•Pregnanes• Pregnadienes (3)• Prednisone (5)
Related Categories•Inhalators (40)•Emotional Effects (25)•Preferred Suppliers (30)
User Preferred• Musculoskeletal (0)•Drug Resistance (2)
•All Categories (50)
50 Documents: [Sort by author] [Sort by popularity] [Sort by Dosage] [Cluster]
1. Optimal dosage levels for prednisone in the treatment of childhood asthma.2. …
1. Optimal dosage levels for prednisone in the treatment of childhood asthma.2. …
152
Other paths: back up and go forward
Asthma > Steroids > Budesonide > Huang
Asthma > Huang > Budesonide
Asthma > Steroids
Asthma > Steroids > Budesonide
153
Advantages of the Methodology
• Supports different types of information seeking tasks
• Uses interface idioms known to be usable for general users
• Flexible content entry and update– Allows for non-experts to add new content
independently– Makes use of standard DBMS technology
154
Advantages of the Methodology• Systematically integrates search:
– search results reflect the structure of the info architecture
– search results retain the context of previous interactions
– search results preview next choices
• Gives user control– Over order of metadata use– Over when to navigate vs. when to search
• Allows integration with advanced methods– Collaborative filtering, predicting users
preferences
155
Advantages
• Users have a feeling of control• Users can predict what will happen
– Not true of statistical ranking or clustering
• Adding new items to the system changes the behavior in understandable ways
• Users have flexibility – In ordering of operations– In combining of operations
156
Usability Study: epicurious
157
Epicurious Usability Study
• 9 participants so far– Independent Variables:
• 1) Epicurious Interface (Basic vs. Enhanced vs. Browse) • 2) Task type (known-item search vs. browsing for inspiration) • 3) Degree of constraint of query • 4) Number of results required (1 vs. many)
– Dependent Variables: • 1) Time to find satisfactory recipe(s)• 2) Navigation path (backtracking, starting over, revising
queries)• 3) Satisfaction with results of search • 4) Satisfaction with individual system features (e.g.
breadcrumbs, query previews, refine by hyperlinks)• 5) Likelihood of using each interface in the future.
158
Epicurious Usability Study
• Participants were asked to:– Do 3 pre-specified searches in advance– In the lab:
• Specify a cooking scenario of interest to them
– Search for 3 recipes for this recipe– Search for each recipe using each of the interfaces
• Complete several structured tasks
– Along the way, answer questions about• Getting closer or farther away from goal• Satisfaction with search results• Satisfaction with the interace
159
Usability Study: Preliminary Results, Preference Data
Site Basic Enhanced BrowseTotal "Very Likely" to Use: 7 2 4 7
Total "Likely" to Use: 0 1 1 0Total "Not Likely" to Use: 2 6 4 2
PERCENTAGES Basic Enhanced BrowseVery Satisfied 32 43 35
Satisfied 50 43 52Middle 9 4 4
Dissatisfied 9 9 9
160
Usability Study: Preliminary Results, Preference Data
FavoriteSubject_JG: EnhancedSubject_NS: EnhancedSubject_SP: Browse
Subject_RM: Browse
Subject_LA: Enhanced
Subject_MC: BrowseSubject_MW: BrowseSubject_NM: EnhancedSubject_CG: Browse
Query previews and navigation. Options to refine by course or season. Choose how you view the results
Searching within made all the difference. I could see how many results I was getting in each Very specific. I can choose more than 1 detail with search for recipe I'm looking for.Likes the way it narrows things down. And it gives you the numbers.
Found it simpler, more readable. Helped you hone in on the season.Liked the kid friendly, low fat optionWhy?
Can narrow down when you're stuck. You can always refine [your search].
Allowed me to make specific selections. I liked Browse too. Gave lots to choose from. Depends on what you’re looking for that day
Can limit and unlimit and limit again in a different way. Prioritize your criteria--change the first thing I clicked and go in a different direction. Easy to back up.
161
Usability StudyPreliminary Results: Feature Preference
Subject_SP Subject_RM Subject_LA Subject_MC Subject_MW Subject_NM Subject_CGQuery previews
Having a complete list of ingredients (enhanced and browse)
enhanced - 0, browse - +1
Search within resultsRefine using hyperlinks (browse)Set all criteria from one screen (enhanced)"May include" and "must include" options"All words," "any words," "exact phrase," and "Boolean" options
KEY+1 - Helpful0 - Not helpful-1 - InterferedDN - Didn't notice
162
Usability StudyPreliminary Results: Quantitative
Search Metrics Basic Enhanced Browse
No Facets / Keywords per search
Total per search 4.3 7.7 ~2.11
During Step 1 of Search 2.3 always 1
For Each Step of Search 2.2 3.5 ~1
Ave. times 0 Results
During Step 1 0.0% 15.8% 0.0%
Overall 12.2% 31.4% 0.0%
Ave. Time ( in seconds)
Time per Search 98.6 130.3 108.11
Time per Step 50.9 60.6 41.38
Ave. No Steps 1.86 2.1 2.81
Median No Results
Overall (excluding 0 results) 186 17 704
in Step 1 13 10 579
In Final Step 32 14 13
163
Usability StudyPreliminary Results: Constraint-based Preferences
# of Results High LowEnhanced (LA) Browse (LA)Enhanced (MC) Browse (MC)Browse (MW) Browse (MW)Enhanced (NM) Enhanced (NM)Basic (CG) Browse (CG)Enhanced (LA) Browse (LA)Enhanced (MC) Browse (MC)Enhanced (MW) Browse (MW)Enhanced (NM) Enhanced (NM)Enhanced (CG) Browse (CG)
Constraint
1 result needed
Many results needed
164
Observed patterns of use of epicurious metadata browse interface
• choosefacet• refine• refine• back• scan focus
• choosefacet• refine• back refine• scan focus
• choosefacet• refine• refine• scan focus
• choosefacet• refine• refine• back refine• scan focus
• choosefacet• refine• refine• searchword
• choosefacet• searchword• scan searchword• back refine• scan focus
• choosefacet• refine• back back refine• refine• refine
• choosefacet• refine• refine• searchword• scan
• choosefacet• refine• refine• back refine
• choosefacet• scan focus
165
Usability Study Results: Summary
• People liked the browsing-style metadata-based search and found it helpful
• People sometimes preferred the metadata search when the task was more constrained – But zero results are frustrating– This can be alleviated with query previews
• People dis-prefer the standard simple search
• More study needed!
166
Application to Image Search
167
Image Search: What is the task?
• Illustrate my slides? – “Find a crevasse”– Keyword match works
pretty well
• Find inspiration for an architectural design? – Needs richer search
support
168
Faceted Metadata for Image Collection
Planalto Palace Parti Communiste Francais Pantheon
Oscar Neimeyer Oscar Neimeyer Jacques-Gabriel Soufflot
20th Century 20th Century 17th & 18th C.
Brasilia Paris Paris
Stone Curvilinear Stone
Image:
Architect:
Period:
Location:
Concept:
169
Planalto Palace Parti Communiste Francais Pantheon
Oscar Neimeyer Oscar Neimeyer Jaques-Gabriel Soufflot
20th Century 20th Century 17th & 18th C.
Brasilia Paris Paris
Stone Curvilinear Stone
Image:
Architect:
Period:
Location:
Concept:
Faceted Metadata for Image Collection
170
SPIRO Query Form (Original)
171SPIR
O q
uery
on
Su
bje
ct:
chu
rch
172
Pilot Study
• Architecture task:– Emphasize images over text– Use hypertext-style interface as a
reasonable baseline for comparison– Find out how much choice is too
much– Find out whether explicit metadata is
better than implicit more-like-this
173
Evaluation Methodology
• Solicit feedback from architects to determine if faceted metadata is helpful and how to present it
• Informal evaluation of paper prototype• Informal study of a crude live version
– 1 hour one-on-one with 9 architects /grad students, 2 tasks (audio recorded) and a survey
174
Results of a pilot study with Archictects: Metadata is Helpful
• Very positive feedback about the general approach– All 9 participants named the
metadata in the search results area as their favorite aspect of Flamenco
• Metadata was successful at giving hints about where to go next– Perceived as useful “These are places
I can go from here.”
175
Results: More Metadata Please• Participants asked for more metadata
– Although there were complaints about the contents of the metadata, users still wanted more
• Longer lists of options (more hints)• Users wanted more control to make very specific searches
• Half the participants requested the ability to control order of results with metadata– Juxtapose visible images 2 different ways:
• Overview (one image from each project) vs. like together ( all images of a project next to each other)
– Different than ranking for text retrieval (precision, recall), but ordering does matter
176
Results: Complaints
• The UI was not successful at clarifying searching within results vs. starting a new search– Only 2 of the 9 participants understood the
distinction without discussion – but they want to do both
• The 1/3 of the participants who couldn’t find a treasure hunt image felt that Flamenco was slow– Corroborates findings that perceived system
speed is about finding what you want (Spool ‘00)
177
New Developments
• A new, sophisticated implementation• Richer, hierarchical, cleaned up
metadata• Usability Study contrasting four
versions:– Single search form– Multiple facet search form– Yahoo-style directory-based – Faceted interface with query previews
• Results TBA
178
179
180
181
Tools
• Our system (all open-source)– Mysql (has a text search component)– Python 2.2– Python-mysql– Webware (python application server)
• Earlier attempt– Cold fusion – not flexible enough, not
enough of a programming language
182
A new term: “Parametric Search”
• From an XML glossary– "A search request submitted to a
search or database engine delivered with consideration for the metadata of the underlying dataset.”
– www.sla.org/chapter/ctor/courier/v37/v37n1.pdf
183
Commercial Tools
• This list is NOT comprehensive• These are NOT recommendations• General Search
– Inktomi Search/site (formerly Infoseek ultra)
• Specializing in Online Catalogs– Dieselpoint– Requisite– Saqqara
• Question-answering– Askjeeves– Primus (formerly Answerlogic)
184
“Parametric” Search• A survey of sites using parametric search:
– http://www.amp.com/search/default.asp (see product family search)– http://ebiz.zilog.com/– http://www.sears.com (Dieselpoint)– http://dieselpoint.com/flashlink.htm (for Dieselpoint 2.0 demo)– http://www.findmro.com (Requisite's BugsEye)– http://www.cypress.com (Saqqara's one step)– http://infineon-tech.sacosnet.de/search/index.htm– http://www.idt.com/tools/parametric.html– http://www.ti.com/sc/docs/psheets/parms/uarts.htm#parms– http://www.gensemi.com/search/productsearch.htm– http://www.usa.samsungsemi.com/search/– http://www.gearfinder.com– http://www.mysimon.com/category/index.jhtml?c=babydiaperingbathing
Site list courtesy Mark Detweiler
185
“Parametric” Search Usage• Goal is to focus on product group for comparison
shopping.• Common Procedure
– Begin with a list of product "families" or groups. – User selects a category, and is prompted to
• 1) select a sub-category from a list of hyperlinks or • 2) select search parameters using a form
– If the number of results is too big, the system may prompt the user to refine the search further.
– When an acceptable number of results is returned, the user sees a list of products which can be:
• 1) sorted by various criteria • 2) selected for display in a comparison table • 3) viewed individually with more detail.
186
“Parametric” Search as used on these Sites
• Observations:– Only one facet (appropriate for
products?)– No query previews– Breadcrumbs rare– Many allow sorting by attribute to
facilitate comparison– “Others like this” simply moves up
the hierarchy
187
Summary and Conclusions
188
Summary• Web site search needs improvement
– Users want more organized results– Our approach: integrate navigation with search
• Metadata is being mixed and matched in interesting ways, but there are no guidelines on what works– We are investigating how to design websites
containing large sets of items
• Preliminary results indicate that metadata organization is useful in some situations– Depends on the type of search need
189
Advantages of the Methodology
• Supports different types of information seeking tasks
• Uses interface idioms known to be usable for general users
• Flexible content entry and update• Systematically integrates navigation &
search• Gives user control• Allows integration with advanced
methods
190
Summary
Our research goals– Systematically determine what works, with the
following emphases: • Task-centric• Integrate metadata with search• Dynamic previews• Easily retrace steps
– Develop recommendations that reflect both the task structure and the richness of the information structure
– In future: integrate with more sophisticated displays
191
Some Unanswered Questions
• How best show combinations of facets that consist of large hierarchies?
• How to use faceted metadata to expand (as opposed to refine)?
• How to integrate with relevance feedback (more like this)?
• How to incorporate user preferences and past behavior?
• How to combine facets to reflect tasks?
192
Thank you!
bailando.sims.berkeley.edu/flamenco.html
For more information: