Flamenco Image Browser:Using Metadata to Improve Image Search During
Architectural Design
Ame ElliottGroup for User Interface Research (GUIR)
& Dept. of ArchitectureUniversity of California, Berkeley
Flamenco Image Browser 2
Problem Statement
Architects begin new projects by looking at images of prior projects How are elementary schools put together? What’s new in patio materials?
Current image search interfaces are not helpful for image browsing in the early phase of design Rely on well-formulated textual queries Ignore context of architecture
Flamenco Image Browser 3
Evaluation
Prototyping Tools .
Scope of the ProjectThe Flamenco Image Browser is one customization of a toolkit for domain-specific search (the Flamenco projects) The Flamenco Image Browser is one component of my dissertation
Understanding Architects’ Work Practices(ethnography, task analysis, interviews)
Hardware: Digital Desk
Software: Image Search UI
Quantitative
Qualitative
Flamenco Image Browser 4
Evaluation
Prototyping Tools .
Scope of the ProjectThe Flamenco Image Browser is one customization of a toolkit for domain-specific search (the Flamenco projects) The Flamenco Image Browser is one component of my dissertation
Understanding Architects’ Work Practices(ethnography, task analysis, interviews)
Hardware: Digital Desk
Software: Image Search UI
Quantitative
Qualitative
Flamenco Image Browser 5
Flamenco Image Browser
Interface to a collection of architectural images using metadata about the images to support search Metadata about the images providesQuery previews giving hints about possible
next actionsWays of broadening or refining search
terms
Flamenco Image Browser 6
Why Use Metadata?
Open questions about its utility How to use metadata not well understood
Not an obvious thing to do for images Contrasts with automated content analysis Defies conventional wisdom
If a picture is worth 1,000 words, let the pictures speak for themselves
Professional architects have different needs than curators
The thumbnail is the whole document, not like text search using a title to stand in for a long article
Flamenco Image Browser 7
Faceted Metadata
Planalto Palace Parti Communiste Francais PantheonOscar Neimeyer Oscar Neimeyer Jacques-Gabriel Soufflot20th Century 20th Century 17th & 18th C.Brasilia Paris ParisStone Curvilinear Stone
Image:Architect:Period:Location:Concept:
Flamenco Image Browser 8
Planalto Palace Parti Communiste Francais PantheonOscar Neimeyer Oscar Neimeyer Jaques-Gabriel Soufflot20th Century 20th Century 17th & 18th C.Brasilia Paris ParisStone Curvilinear Stone
Image:Architect:Period:Location:Concept:
Faceted Metadata
Flamenco Image Browser 9
Evaluation MethodologySolicit feedback from architects to determine if faceted metadata is helpful and how to present it Not evaluating if the current metadata in the system is the
right metadata
Lo-Fi evaluation of paper prototype 1 hour one-on-one with 3 professional architects Walk-through interactions on a paper computer, users think-
aloud (audio recorded), questions about the experience
Informal study of live version 1 hour one-on-one with 9 architects /grad students, 2 tasks
(audio recorded) and a survey
734 images of France in the entire collection
100 images of France in the 1,000 results for stone
Flamenco Image Browser 12
Results of Lo-Fi StudyThe difference between searching within results and changing to a new search confused users The “breadcrumb” did not help users orient themselves when
browsing results Home >> Gehry, Frank >> Berlin
Home >> Berlin Users were surprised at the outcome and could not anticipate
which they were getting (refine or expand).Image search is exploratory, not reductive Specific textual queries not important
“If I knew what I wanted to type in the box I’d just go get it myself.” Expanding search more important than refining
is different than
Flamenco Image Browser 13
Make iterative changes to the alternatives to identify useful aspects of the UI – not statistical analysis Learn to what extent the metadata is useful for searching How much text is too much? What kinds of queries will the users do?
Explore how to clarify searching within results vs. starting a new search
Goals for Study of Live Web Site
Flamenco Image Browser 14
Informal Study of Live VersionComparison- a form-based UI to the same collection of imagesTask 1 – Free form search Participants told they are helping to test an image search engine Asked to talk about a project they’re working on and something they’d like to find Let them go at it and try to find images they’re interested in – they can ask any
questions they want but no formal instructions given Task 2 – Treasure hunt Participants given paper copies of 3 images (but no metadata) and told to find them
(range from easy to hard)
Exit survey
Flamenco Image Browser 17
About the SystemWeb browser used to view Cold Fusion generated pages, queries made to a mySQL database36,000 images from the UC Berkeley Architecture Slide library5 facets of metadata about each image Image title, architect, period, location, and concept Shallow hierarchy for results in architect, period, and location Concept facet is from the Getty Art and Architecture thesaurus - a
40,000 term hierarchical thesaurus of descriptors of the built environment – this hierarchy not tested in the live version
Building MaterialsMasonry Finishing Materials
Stonework Brickwork Shingles
Flamenco Image Browser 18
Results: Metadata is HelpfulVery positive feedback about FlamencoAll 9 participants named the metadata in
the search results area as their favorite aspect of Flamenco
Metadata was successful at giving hints about where to go nextPerceived as useful “These are places I
can go from here.”
Flamenco Image Browser 19
Results: More Metadata PleaseParticipants asked for more metadata Although there were complaints about the contents of the
metadata, users still wanted more Longer lists of options (more hints) Users wanted more control to make very specific searches
Half the participants requested the ability to control order of results with metadata Juxtapose visible images 2 different ways:
Overview (one image from each project) vs. like together ( all images of a project next to each other)
Different than ranking for text retrieval (precision, recall), but ordering does matter
Flamenco Image Browser 20
Results: ComplaintsThe UI was not successful at clarifying searching within results vs. starting a new search Only 2 of the 9 participants understood the
distinction without discussion – but they want to do both
The 1/3 of the participants who couldn’t find a treasure hunt image felt that Flamenco was slow Corroborates findings that perceived system speed
is about finding what you want (Spool ‘00)
Flamenco Image Browser 21
Enthusiasm for Metadata
Contradicted my suspicions that a sledgehammer and pick- through-the-rubble approach would be preferable No-one thought there was too much text Adding more text preferable to adding another row
of images
Flamenco Image Browser 22
Conclusions
Metadata is useful for exploratory tasks Good at giving hints about where to go next
Architects want to use metadata to get more control Of results display Build complex queries
Sometimes the right word is worth more than many pictures
Flamenco Image Browser 23
Contributions
Increase understanding of what makes a good search interface Provide insights about how image search differs from text search
Flamenco Image Browser 24
Acknowledgements
Thanks to Marti Hearst my advisor Andy Edmonds for excellent coding Andy Chou for help with the database the Berkeley Museum Informatics Project for
access to the image collection the Flamenco project team for search ideas and the members of GUIR for helpful feedback
Thanks!
Flamenco Image Browser 26
Flamenco Image Browser 29
Related Work
“Berrypicking” model of information seeking (Bates)Information scent (Pirolli)Query previews (Plaisant & Shneiderman et al)QBIC & Blobworld (content analysis)
Flamenco Image Browser 30
Architects’ Image Use Practices
Architects rely on personal image collections and libraries of print mediaImages emerge as relevant due to their juxtaposition with other images