1

Project CLiMB

Computational Linguistics for

Metadata Building

Using Computational Linguistic Techniques

to Harvest Image Descriptors

Columbia UniversityFunded by the Andrew W. Mellon Foundation

2002-2004

2

Session Outline

Subject Access in a Digital Age: A Sisyphean Task

Angela Giral - Columbia University

CLiMBing the Mountain: Automating Subject Access for Image Professionals

Roberta Blitz - Columbia University

A New Vista? Preview and Feedback for the CLiMB ToolKit

Rebecca Passonneau - Columbia University

3

Session Outline

Subject Access in a Digital Age: A Sisyphean Task

Angela Giral - Columbia University

CLiMBing the Mountain: Automating Subject Access for Image Professionals

Roberta Blitz - Columbia University

A New Vista? Preview and Feedback for the CLiMB ToolKit

Rebecca Passonneau - Columbia University

4

Angela Giral – Columbia University

A Sisyphean Task

Subject Access in a Digital Age:

5

Thank you!

Any further questions?

www.columbia.edu/cu/cria/climb

6

Session Outline

Subject Access in a Digital Age: A Sisyphean Task

Angela Giral - Columbia University

CLiMBing the Mountain: Automating Subject Access for Image Professionals

Roberta Blitz - Columbia University

A New Vista? Preview and Feedback for the CLiMB ToolKit

Rebecca Passonneau - Columbia University

7

Roberta Blitz – Columbia University

Automating Subject Access

for Image Professionals

CLiMBing the Mountain:

8

Presentation Outline

1. Introduction to CLiMB

2. Two examples with images from theNorth Carolina Museum of Art

3. CLiMB Subject Access Terms

9

CLiMB: Interdisciplinary Research at

Columbia University

• CRIA (Center for Research on Information Access)• Libraries• Computer Science Department

Funded by the Andrew W. Mellon Foundation2002-2004

10

CLiMB Project Members

Judith Klavans, PI

Stephen Davis

Angela Giral

Patricia Renfro

Bob Wolven

Roberta Blitz

Rebecca Passonneau

Veronika Horvath

David Elson

11

Problems in Image Access

Cataloging digital images

Traditional approach:

manual expertise labor intensive expensive

Can automated techniques help?

12

Can we harvest image descriptors?

13

CLiMB will identify and extract• proper nouns• terms and phrases

from text related to an image:

By September 14, 1908, the basis of the Greenes' final design had been worked out. It featured a radically informal, V-shaped plan (that maintained the original angled porch) and interior volumes of various heights, all under a constantly changing roofline that echoed the rise and fall of the mountains behind it. The chimneys and foundation would be constructed of the sandstone boulders that comprised the local geology, and the exterior of the house would be sheathed in stained split-redwood shakes.

— Edward R. Bosley. Greene & Greene. London: Phaidon, 2000. p.127.

CLiMB Technical Contribution

14

CLiMB Overall Goals

The essence of CLiMB: • Use scholars themselves as “catalogers” by

employing scholarly publications• Enhance existing descriptive metadata

The CLiMB project:• Research: Development of richer retrieval

through increased numbers of descriptors• Practice: Development of CLiMB ToolKit

15

• Image collection

• Associated text

• Target object identification (TOI)

• CLiMB ToolKit

• Evaluation

Squeezing Metadata out of Scholarly Texts

16

CLiMB Collections• Greene & Greene Architectural Records, Avery Architectural and Fine Arts Library

Columbia University

• Chinese Paper Gods Collection, C.V. Starr East Asian Library

Columbia University

• Digital Images from the North Carolina Museum of Art

17

Greene & Greene Architectural Records and Papers Collection

Drawings and ArchivesAvery Architectural and Fine Arts LibraryColumbia University Libraries

18

NYDA.1960.001.00023

All Saints Episcopal Church (Pasadena, Calif.). Alterations1902-1903

19

Greene & Greene Catalog RecordAuthor: Greene & Greene.Title: [Mrs. Dudley P. Allen house, 1188 Hillcrest Avenue (Pasadena,

Calif.). Alterations.]Residence of Mrs. Dudley P. Allen, 1188 Hillcrest Ave., Pasadena,

Cal. [graphic] : Alteration / Greene & Greene, Architects. Published: [1917]

Physical Details: 4 sheets : various media ; 87.8 x 57.3 cm. (34 5/8 x 22 5/8 in.)Location: Columbia University, Avery Architectural Drawings

Other Authors: Greene, Charles Sumner, 1868-1957. Greene, Henry Mather, 1870-1954.

Subjects: HousesAlterationsArchitecture--Designs and plans--United States.Mrs. Dudley P. Allen house, 1188 Hillcrest Avenue (Pasadena,

Calif.)

Component Item: [1] Item no. NYDA.1960.001.03224. [AVERYimage]. Electric lighting -- floor plan, part plan of basement : Sheet no.

Component Item: [2] Item no. NYDA.1960.001.00073. [AVERYimage]. [Electric lighting] floor plan, part plan of basement.

20

• Bosley, Edward R. Greene & Greene. London : Phaidon, 2000.

• Current, William R. Greene & Greene: architects in the residential style. Fort Worth [Tex.] : Amon Carter Museum of Western Art, [1974]

• Makinson, Randell L. Greene & Greene: architecture as fine art. Salt Lake City : Peregrine Smith, c1977.

• Makinson, Randell L. Greene & Greene: the passion and the legacy. Salt Lake City : Gibbs and Smith, c1998.

• Smith, Bruce. Greene & Greene masterworks. San Francisco : Chronicle Books, c1998.

• Strand, Janann. A Greene & Greene guide [Pasadena, Calif. : G. Dahlstrom, 1974]

Greene & Greene Bibliography(associated texts)

21

22

  

C.V. Starr East Asian Library, Columbia University

Chinese Paper GodsAnne S. Goodrich Collection

24

Pan-hu chih-shenGod of tigers

25

Title: Chuang gong chuang mu [graphic].

Published: [193-]

Physical Details: 1 print : wood-engraving, color ; 34 x 30 cm.

In: Anne S. Goodrich Collection.

Location: Columbia University, C.V. Starr East Asian Library (CJK)

EAX GAC 1 no. 16

Subjects: Gods, Chinese, in art.

Folk art--China.

Genre Or Form: Woodcuts--Chinese.

Notes: Date according to time period Anne S. Goodrich collected prints in

Beijing.

Record ID: NYCP02-F20

Chinese Paper Gods Catalog Record

26

Chinese Paper Gods Bibliography (associated texts)

• Day, Clarence Burton. Chinese peasant cults : being a study of Chinese paper gods. Taipei : Ch'eng Wen Pub. Co., 1974.

• Goodrich, Anne Swann. Peking paper gods : a look at home worship. Nettetal : Steyler Verlag, 1991.

• Laing, Ellen Johnston. Art and aesthetics in Chinese popular prints: selections from the Muban Foundation collection. Ann Arbor, MI : Center for Chinese Studies, University of Michigan, c2002.

27

HEADING: Nezha (Chinese deity)Used For/See From: Daluoxian (Chinese deity)

Jinhuan Yuanshuai (Chinese deity)Jinkang Yuanshuai (Chinese deity)Li Nezha (Chinese deity)Luoche Taizi (Chinese deity)Ne Zha (Chinese deity)Nezhataizi (Chinese deity)No-cha (Chinese deity)Nuozha (Chinese deity)Tailuoxian (Chinese deity)Taizi Yuanshuai (Chinese deity)Taiziyeh (Chinese deity)Yühuang Taizi (Chinese deity)Zhongtan Yuanshuai (Chinese deity)

Search Also Under:  Gods, Chinese

Chinese gods: selection from LC Authority File

28

• Image collection

• Associated text

• Target object identification (TOI)

• CLiMB ToolKit

• Evaluation

Squeezing Metadata out of Scholarly Texts

29

Target Object Identification (TOI)

• Define based on institutional needs

• Varies from collection to collection– Greene & Greene – Project Names– Chinese Paper Gods – God Names– North Carolina Museum – Artist / Work Names

• Compile authority list

30

Deriving TOIs from creator & title lists

31

North Carolina Museum of Art Museum Catalog

(Associated Text)

Images

(Catalog Records)

North Carolina Museum of Art: Handbook of the Collections. Ed. Rebecca Martin Nagy. Raleigh, NC: North Carolina Museum of Art, Hudson Hills Press, 1998.

32

North Carolina Museum of Art Museum Catalog

(Associated Text)

Images

(Catalog Records)

North Carolina Museum of Art: Handbook of the Collections. Ed. Rebecca Martin Nagy. Raleigh, NC: North Carolina Museum of Art, Hudson Hills Press, 1998.

• village of Cebolla • adobe Church of Santo Niño• rusted tin roof• window

33

Georgia O'Keeffe (American, 1887-1986)

Cebolla Church, 1945

Oil on canvas, 20 1/16 x 36 1/4 in. (51.1 x 92.0 cm.) Purchased with funds from the North Carolina Art Society (Robert F. Phifer Bequest), in honor of Joseph C. Sloane, 72.18.1

North Carolina Museum of Art<http://ncartmuseum.org/collections/highlights/20thcentury/20th/1910-

1950/038_lrg.shtml>

34

MARC format

100 O’Keeffe, Georgia, ≠d 1887 -1986. 245 Cebolla church ≠ h [slide] / ≠ c Georgia

O’Keeffe.260 ≠c2003300 1 slide : ≠ b col.500 Object date: 1945.500 Oil on canvas.500 20 x 36 in.535 North Carolina Museum of Art ≠ b Raleigh, N.C.650 Painting, American ≠ y 20th century.650 Women artist ≠ z United States 650 Church buildings in art.

35

Cebolla Church, 1945Oil on canvas, 20 1/16 x 36 1/4 in. (51.1 x 92.0 cm.)Purchased with funds from the North Carolina Art Society (Robert F. Phifer Bequest), in honor of Joseph C. Sloane, 72.18.1

Driving through the New Mexican highlands near her home, Georgia O'Keeffe would often pass through the village of Cebolla with its rude adobe Church of Santo Niño. The artist was moved by the poignancy of the little building: its sagging, sun-bleached walls and rusted tin roof seemed so typical of the difficult life of the people.

When O'Keeffe came to paint the church she addressed it directly, emphasizing its isolation and stark simplicity. Literally formed out of the earth, the building affirms the permanence and the hard, defiant patience of the people. For O’Keeffe, it symbolized human endurance and aspiration. "I have always thought it one of my very good pictures", she wrote, "though its message is not as pleasant as many others".

And the question remains: What is that in the window?

36

MARC format with CLiMB subject terms100 O’Keeffe, Georgia, ≠d 1887 -1986. 245 Cebolla church ≠ h [slide] / ≠ c Georgia O’Keeffe.260 ≠c2003300 1 slide : ≠ b col.500 Object date: 1945.500 Oil on canvas.500 20 x 36 in.535 North Carolina Museum of Art ≠ b Raleigh, N.C.650 Painting, American ≠ y 20th century.650 Women artist ≠ z United States 650 Church buildings in art.

CLiMB New Mexican highlands CLiMB village of Cebolla CLiMB adobe Church of Santo NiñoCLiMB sagging, sun-bleached walls CLiMB rusted tin roofCLiMB isolationCLiMB human enduranceCLiMB window

37

Joseph Cornell (American, 1903-1972)

Suzy's Sun (For Judy Tyler), 1957

Mixed media construction, 10 3/4 x 15 x 4 in. (27.3 x 38.1 x 10.2 cm.)Purchased with funds from the State of North Carolina, 78.1.1

North Carolina Museum of Art<http://ncartmuseum.org/collections/highlights/20thcentury/20th/1950-

2000/030_lrg.shtml>

38

VRA Core 3.0Record Type=workType=shadow box Title=Suzy's Sun (For Judy Tyler)Measurements.Dimensions=10 3/4 x 15 x 4 in. (27.3 x

38.1 x 10.2 cm.) Material.Medium=mixed mediaCreator.Personal name=Cornell, JosephCreator.Role=artistDate.Creation=1957Location.Current Repository=Raleigh (NC, USA),

North Carolina Museum of ArtID Number.Current Accession=78.1.1Subject=assemblages (sculpture)

39

Suzy's Sun (For Judy Tyler), 1957Mixed media construction, 10 3/4 x 15 x 4 in. (27.3 x 38.1 x 10.2 cm.)Purchased with funds from the State of North Carolina, 78.1.1

Joseph Cornell fabricated shadow boxes and filled them with objects collected both by chance and choice. With invention and insight, he teased themes from these unlikely groupings, most often revolving around time and memory.

In Suzy’s Sun (for Judy Tyler), the sun (a cutout from an antipasto tin) and the sea (an implied presence) speak eloquently of life cycles and passing time. Equally potent symbols, driftwood and the infinitely spiraling seashell readily bring to mind the tides on which they ride, summoning a universal metaphor for the ebb and flow of life itself. In small details-a postage stamp showing a multi-masted schooner, the collaged word "hotel"-Cornell uses the romantic notion of travel to far-off lands as additional commentary on one’s passage through life.

Cornell dedicated this box to an actress. Judy Tyler had just achieved a certain celebrity when she was killed in an automobile accident. "Suzy" probably refers to the artist’s assistant, Suzanne Miller. The sun, designated as Suzy’s, presides over the box, as a life-sustaining force counteracting the finality of death.

40

Title=Suzy's Sun (For Judy Tyler)Creator.Personal name=Cornell, JosephSubject=assemblages (sculpture)

CLiMB=shadow boxes CLiMB=spiraling seashell

CLiMB=sun CLiMB=tidesCLiMB=time CLiMB=metaphorCLiMB=memory CLiMB=ebb and flow of lifeCLiMB=cutout CLiMB=postage stampCLiMB=antipasto tin CLiMB=multi-masted schoonerCLiMB=sea CLiMB=collaged word hotelCLiMB=life cycles CLiMB=travelCLiMB=passing time CLiMB=actressCLiMB=driftwood CLiMB=Judy Tyler

CLiMB=Suzanne Miller

VRA Core 3.0 with CLiMB subject terms

41

CLiMB Metadata Can Be Employed in Standard Catalog Formats

USMARC: MARC 21 Concise Format for Bibliographic Data(http://lcweb.loc.gov/marc/bibliographic/ecbdhome.html)

Fields for CLiMB metadata:650 - SUBJECT ADDED ENTRY--TOPICAL TERM (R)653 - INDEX TERM--UNCONTROLLED (R)

VRA Core Categories, Version 3.0: A project of the Visual Resources Association Data Standards Committee

(http://www.vraweb.org/vracore3.htm)

Field for CLiMB metadata: SUBJECT

Dublin Core Metadata Element Set, Version 1.1: Reference Description(http://dublincore.org/documents/dces/)

Field for CLiMB metadata: SUBJECT AND KEYWORDS

42

Possible controlled vocabularies• *AAT

Getty Art & Architecture Thesaurus

• TGN Getty Thesaurus of Geographic Names

• ULAN Union List of Artist Names

• LC NAFLibrary of Congress Name Authority File

• LCSHLibrary of Congress Subject Headings

• TGM IThesaurus for Graphic Materials: Subject Terms

• TGM II Thesaurus for Graphic Materials: Genre and Physical Characteristic Terms

Getty

LC

43

CLiMB terms Controlled vocabulary

New Mexican highlands uplands (LCSH) used for highlands New Mexico (LCSH / TGN)

village of Cebolla villages (AAT / LCSH / TGM I) Cebolla (TGN)

adobe Church of Santo Niño adobe (AAT) churches (AAT) adobe churches (LCSH) Santo Niño (TGN)

sagging, sun-bleached walls walls (AAT / LCSH / TGM I)

rusted tin roof roofs (AAT / LCSH / TGM I) rust (AAT) tin (AAT)

isolation solitude (TGM I) used for isolation

human endurance [no matches]

window windows (AAT / LCSH / TGM I)

Controlled vocabulary matches

44

Thank you!

Any further questions?

www.columbia.edu/cu/cria/climb

45

Session Outline

Subject Access in a Digital Age: A Sisyphean Task

Angela Giral - Columbia University

CLiMBing the Mountain: Automating Subject Access for Image Professionals

Roberta Blitz - Columbia University

A New Vista? Preview and Feedback for the CLiMB ToolKit

Rebecca Passonneau - Columbia University

46

Rebecca Passonneau – Columbia University

Preview and

Feedback for the

CLiMB ToolKit

A New Vista?

47

The CLiMB ToolKit

• Software prototype• For large image collections• Semi-automated metadata

– Subject access terms– Human intervention at all steps

• Iterative development cycle

48

Presentation Outline

1. Process flow in ToolKit

2. Prerequisites to use ToolKit

3. Demonstration

49

• Image collection

• Associated text

• Target object identification (TOI)

• CLiMB ToolKit

• Evaluation

Squeezing Metadata out of Scholarly Texts

50

CLiMB TOOLKIT: Process Flow

1. Load Text

2. Load TOI List

3. Analyze Text

5. Review

4. Select Subject Access Terms

51

Prerequisites and Resources

1. Image collection

2. Minimal-level catalog records

3. Texts about the images (associated texts)• Scholarly Monographs• Museum Catalog

4. TOI list of images• NOW: CLiMB format• FUTURE: Authoritative list of images in your format

5. [ Controlled Vocabularies ]

52

The Visible CLiMB ToolKit

• Web Browser

• Help Menus

• Projects

A Graphical User Interface (GUI):

53

The Invisible CLiMB ToolKit

• TOI List

• Texts

• Text analysis tools to

find terms for TOIs

• Controlled Vocabularies

• Rulesets

Tools and Resources:

54

• Image collection

• Associated text

• Target object identification (TOI)

• CLiMB ToolKit

• Evaluation

Squeezing Metadata out of Scholarly Texts

55

CLiMB and the Image Professional:

Do you like what you see?• Overall look and feel

• Help and Documentation

• Text Loader

• Other functions

Do you like what you don’t see?• Automatic Proposals for Subject Access Terms

Iterative Development and Evaluation

56

CLiMB ToolKit Demo

http://climb.cs.columbia.edu/demo/

57

Thank you!

Any further questions?

www.columbia.edu/cu/cria/climb