2003.09.02 - SLIDE 1IS 202 - FALL 2003

Lecture 03: Categorization

Prof. Ray Larson & Prof. Marc Davis

UC Berkeley SIMS

Tuesday and Thursday 10:30 am - 12:00 am

Fall 2003

SIMS 202:

Information Organization

and Retrieval

Credits to Marti Hearst and Warren Sack for some of the slides in this lecture

2003.09.02 - SLIDE 2IS 202 - FALL 2003

Today

• Review of Last Time

– What Is Information?

• Categorization

• Discussion Questions

• Action Items for Next Time

2003.09.02 - SLIDE 3IS 202 - FALL 2003

Assignment 1 - Discussion

• What is information, according to your background or area of expertise?

2003.09.02 - SLIDE 4IS 202 - FALL 2003

What Is Information?

• Relating data to a context (“situational interpretation”)

• Anything that is important to anyone (“significance”)

• World data information knowledge

• Requires community of interpretation

• All information is dependent on context

• Capable of being recorded and stored and transmitted (also in physical form – e.g., fossils)

• Information must be recorded

• Information is a record of something that can be reused

• Information is a commodity

2003.09.02 - SLIDE 5IS 202 - FALL 2003

What Is Information?

• Negentropy

• Potential energy to become knowledge

• Potential for it to be built upon

• Questions– Does information have to be related to “true”

data?– Can information be downgraded to data if it is

forgotten?

2003.09.02 - SLIDE 6IS 202 - FALL 2003

Human Communication Theory?

Destination

Noise

Source DecodingEncoding

Message Message

Channel

2003.09.02 - SLIDE 7IS 202 - FALL 2003

The Conduit Metaphor

• Language functions like a conduit, transferring thoughts bodily from one person to another

• In writing and speaking, people insert their thoughts or feelings in the words

• Words accomplish the transfer by containing the thoughts or feelings and conveying them to others

• In listening or reading, people extract the thoughts and feelings once again from the words

2003.09.02 - SLIDE 8IS 202 - FALL 2003

Toolmakers’ Paradigm

2003.09.02 - SLIDE 9IS 202 - FALL 2003

How Much Information Today?

• See report by Hal Varian and Peter Lyman http://www.sims.berkeley.edu/research/projects/how-much-info/

• Total annual information production including print, film, magnetic media, etc.– Upper Bound 2,120,539 Terabytes (1012 bytes)– Lower Bound 635,480 Terabytes– I.e., between 1 and 2 Exabytes per year (1018 bytes)

• How do we organize THIS?

2003.09.02 - SLIDE 10IS 202 - FALL 2003

Categorization

09/02/2003 Categorization

09/04/2003 Knowledge Representation

09/09/2003 Lexical Relations and WordNet

09/11/2003 Metadata Introduction

09/16/2003 Controlled Vocabulary Introduction

09/18/2003 Thesaurus Design and Construction

2003.09.02 - SLIDE 11IS 202 - FALL 2003

Foucault on Borges

• This passage quotes “a certain Chinese encyclopedia” in which it is written that ‘animals are divided into: (a) belonging to the Emperor, (b) embalmed, (c) tame, (d) suckling pigs, (e) sirens, (f) fabulous, (g) stray dogs, (h) included in the present classification, (i) frenzied, (j) innumerable, (k) drawn with a very fine camelhair brush, (l) et cetera, (m) having just broken the water pitcher, (n) that from a long way off look like flies.’– Michel Foucault, The Order of Things, 1970

2003.09.02 - SLIDE 12IS 202 - FALL 2003

Yahoo! Categorization

2003.09.02 - SLIDE 13IS 202 - FALL 2003

Yahoo! Categorization Detail

2003.09.02 - SLIDE 14IS 202 - FALL 2003

Why Study Categorization?

• Categorization is central to how we organize information and the world

• Categorization is a core cognitive process

• In recent years, centuries-old views of categorization have been revised

• Understanding how people categorize can help us design information systems that do a better job at organization and retrieval

2003.09.02 - SLIDE 15IS 202 - FALL 2003

Why Read Lakoff?

• Very influential figure in recent thinking about human categorization, metaphor, and cognition

• Provides summary of historical work and develops syncretic model of cognition and categorization

• Clear explanations using examples

• Professor at UC Berkeley (Department of Linguistics)

2003.09.02 - SLIDE 16IS 202 - FALL 2003

George Lakoff

• Lakoff’s research covers many areas of Conceptual Analysis within Cognitive Linguistics– The nature of human conceptual systems, especially metaphor

systems for concepts such as time, events, causation, emotions, morality, the self, politics, etc.

– The development of Cognitive Social Science, which applies ideas of Cognitive Semantics to the Social Sciences

– The implications of Cognitive Science for Philosophy, in collaboration with Mark Johnson, Chair of Philosophy at the University of Oregon

– Neural foundations of conceptual systems and language, in collaboration with Jerome Feldman, of the International Computer Science Institute, seeking to develop biologically-motivated structured connectionist systems to model both the learning of conceptual systems and their neural representations

– The cognitive structure, especially the metaphorical structure, of mathematics, in collaboration with Rafael Núñez

2003.09.02 - SLIDE 17IS 202 - FALL 2003

George Lakoff

• Selected publications– Metaphors We Live By (with Mark Johnson) Univ. of

Chicago Press. 1980.– Women, Fire, and Dangerous Things. University of

Chicago Press. 1987.– More Than Cool Reason. (with Mark Turner) Univ. of

Chicago Press. 1989.– Moral Politics. University of Chicago Press. 1996.– Philosophy in The Flesh. Basic Books, 1999.– Where Mathematics Comes From: How the Embodied

Mind Brings Mathematics into Being. (with Rafael Núñez). Basic Books. 2000.

– Moral Politics: How Liberals and Conservatives Think. Second Edition. University of Chicago Press, 2002.

2003.09.02 - SLIDE 18IS 202 - FALL 2003

Objectivist Views

• Thought is mechanical manipulation of symbols• The mind is an abstract machine• Symbols get their meaning from correspondences to the external

world• Symbols are internal representations• Abstract symbols stand in correspondence with the external world

independent of the interpreting organism• The human mind is a mirror of nature• Human bodies play no role in characterizing concepts• Thought is abstract and disembodied• Exclusively symbolic machines are capable of thought• Thought can be broken down into simple “building blocks”• Thought is defined by mathematical logic

2003.09.02 - SLIDE 19IS 202 - FALL 2003

Experientialist Views

• Thought is embodied• Thought is imaginative• Thought has gestalt properties• Thought utilizes basic-level categorization and basic-

level primacy• Thought uses prototypes and family resemblances as

organizing structures• Conceptual structure can be described using cognitive

models that have the above properties• The theory of cognitive models incorporates what was

right about the traditional view of categorization, meaning, and reason, while accounting for the empirical data on categorization and fitting the new view overall

2003.09.02 - SLIDE 20IS 202 - FALL 2003

Central Conceptual Issue

• Do meaningful thought and reason concern merely the manipulations of abstract symbols and their correspondence to an objective reality, independent of any embodiment (except, perhaps, for limitations imposed by the organism)?

• Do meaningful thought and reason essentially concern the nature of the organism doing the thinking—including the nature of its body, its interaction in its environment, its social character, and so on?

2003.09.02 - SLIDE 21IS 202 - FALL 2003

Categorization

• Classical categorization– Necessary and sufficient conditions for

membership– Generic-to-specific monohierarchical structure

• Modern categorization– Characteristic features (family resemblances)– Centrality/typicality (prototypes)– Basic-level categories

2003.09.02 - SLIDE 22IS 202 - FALL 2003

Defining Category Membership

• Necessary and sufficient conditions– Every condition must be met– No other conditions can be required

• Example: A prime number:– An integer divisible only by itself and 1.

Source: Webster's Revised Unabridged Dictionary, © 1996, 1998 MICRA, Inc.

• Example: mother– A woman who has given birth to a child.

2003.09.02 - SLIDE 23IS 202 - FALL 2003

Defining Category Membership

• Necessary and sufficient conditions for Mother?– mother(A,B) -> female(A), gave-birth-to(A,B),

same-species(A,B)

• What about– Birth mother vs. adoptive mother– Surrogate mother– Transgenic mother

2003.09.02 - SLIDE 24IS 202 - FALL 2003

Can Category Membership Be Defined?

• What are the necessary and sufficient conditions for something to be a game?

• Famous example by Wittgenstein– Classic categories assume clear boundaries

defined by common properties (necessary and sufficient conditions)

• How do we categorize games?

2003.09.02 - SLIDE 25IS 202 - FALL 2003

Definition of Game

• Counterexample: “Game”– No common properties shared by all games

• Card games, ball games, Olympic games, children’s games

– Competition: ring-around-the-rosy– Skill: dice games– Luck: chess

– No fixed boundary to category• Can be extended to new games (e.g., video

games)

• Alternative notion of category membership– Concepts related by family resemblances

2003.09.02 - SLIDE 26IS 202 - FALL 2003

Properties of Categorization

• Family resemblance– Members of a category may be related to one

another without all members having any property in common

• Instead, they may share a large subset of traits• Some attributes are more likely given that others

have been seen

– Example: feathers, wings, twittering, ...• Likely to be a bird, but not all features apply to

“emu”• Unlikely to see an association with “barks”

2003.09.02 - SLIDE 27IS 202 - FALL 2003

Properties of Categorization

• Example: Prime numbers– Definition: An integer divisible only by itself and 1– Examples: 2, 3, 5, 7, 11, 13, 17, …

• A very clear-cut category. Or is it?– Can one number be “more prime” than another?

• Centrality– Some members of a category may be “better

examples” than others, i.e., “prototypical” members• Example: robins vs. chickens vs. emus

2003.09.02 - SLIDE 28IS 202 - FALL 2003

Properties of Categorization

• Characteristic features– Perceived degree of category membership

has to do with which features help define the category

– Members usually do not have ALL the necessary features, but have some subset

– Those members that have more of the central features are seen as more central members

– People have conceptions of typical members

2003.09.02 - SLIDE 29IS 202 - FALL 2003

Testing for Centrality/Typicality

• Ask a series of questions, compare how long it takes people to answer– True or false:

• An apple is a fruit• A plum is a fruit• A coconut is a fruit• An olive is a fruit• A tomato is a fruit

• Rosch and Mervis– The more features a fruit shares with the other fruits,

the more typical a member of the class it is

2003.09.02 - SLIDE 30IS 202 - FALL 2003

Characteristic Features

• Is a cat on a mat a cat?

• Is a dead cat a cat?

• Is a photo of a cat a cat?

• Is a cat with three legs a cat?

• Is a cat that barks a cat?

• Is a cat with a dog’s brain a cat?

• Is a cat with every cell replaced by a dog’s cells a cat?

2003.09.02 - SLIDE 31IS 202 - FALL 2003

Properties of Categorization

• Basic-level categories– Categories are organized into a hierarchy

from the most general to the most specific, but the level that is most cognitively basic is “in the middle” of the hierarchy

• Basic-level primacy– Basic-level categories are functionally primary

with respect to factors including ease of cognitive processing (learning, reasoning, recognition, etc.)

2003.09.02 - SLIDE 32IS 202 - FALL 2003

Basic-Level Categories

• Brown 1958, 1965, Berlin et al., 1972, 1973• Folk biology:

– Unique beginner: plant, animal– Life form: tree, bush, flower– Generic name: pine, oak, maple, elm– Specific name: Ponderosa pine, white pine– Varietal name: Western Ponderosa pine

• No overlap between levels• Level 3 is basic

– Corresponds to genus– Folk biological categories correspond accurately to

scientific biological categories only at the basic level

2003.09.02 - SLIDE 33IS 202 - FALL 2003

Psychologically Primary Levels

SUPERORDINATE animal furniture

BASIC LEVEL dog chair

SUBORDINATE terrier rocker

• Children take longer to learn superordinate categories above the basic level

• Superordinate categories above the basic level are not associated with mental images or motor actions

2003.09.02 - SLIDE 34IS 202 - FALL 2003

Basic-Level Categorization

• Perception– Overall perceived shape– Single mental image– Fast identification

• Function– General motor program

• Communication– Shortest, most commonly used and contextually neutral words– First learned by children

• Knowledge Organization– Most attributes of category members stored at this level

2003.09.02 - SLIDE 35IS 202 - FALL 2003

Middle-Out Categorization

• Top down– Object

• Writing implement– Pen

• Bottom up– Sanford Uniball Black Pen

• Ink Pen

– Pen

• Middle out– Writing implement

• Pen– Ink Pen

2003.09.02 - SLIDE 36IS 202 - FALL 2003

Summary

• Processes of categorization underlie many of the issues having to do with information organization

• Categorization is messier than our computer systems would like

• Human categories have graded membership, consisting of family resemblances– Family resemblance is expressed in part by which subset of

features is shared– It is also determined by underlying understandings of the world

that do not get represented in most systems

• Basic-level categories, as well as subordinate and superordinate categories, seem to be cognitively real and therefore important in the design of information organization and retrieval systems

2003.09.02 - SLIDE 37IS 202 - FALL 2003

Discussion Questions (Lakoff)

• Margaret Tong on Lakoff– If categorization is embodied, i.e., is a consequence

of bodily experience, and if there is a pool of information so large that must be categorized by a computer (beyond human capacity to categorize), then does the computer incorporate ‘bodily experience’? If so, how? If not, does it have to rely on the so called classical view of categorization?

– The objects under study by various researchers are mostly physical, such as trees, birds and colors. Would the same theory apply if the entity to categorize is information, which is somewhat intangible?

2003.09.02 - SLIDE 38IS 202 - FALL 2003

Discussion Questions (Lakoff)

• Carolyn Cracraft on Lakoff– Do the existence of prototype members offer

any support for the conduit theory of language discussed last week? For instance, does the fact that diverse peoples across many cultures will all select focal blue as the best example of their word for blue imply that there really is a transmittable idea contained in that word?

2003.09.02 - SLIDE 39IS 202 - FALL 2003

Discussion Questions (Lakoff)

• Carolyn Cracraft on Lakoff– In the discussion of basic-level categories, Lakoff

opens with Brown, whose examples of basic names determined by distinctive actions seem to fall at the “life form” level - flower, ball, cat, etc. Then he attempts to slide seamlessly into the discussion of Tzeltal plant classification, where the basic level is the “genus” level – oak, maple, etc. (or, to relate to the earlier examples, rose, baseball, Persian). It seems to me, though I’ve not experimented, that children actually learn Brown’s life-form-level words before the genus-level (tree before oak, flower before rose). So what is the basic-level category? Does it really exist, and is it predictable?

2003.09.02 - SLIDE 40IS 202 - FALL 2003

Discussion Questions (Lakoff)

• Carolyn Cracraft on Lakoff– In terms of our real concerns, i.e., organization of information in

a library or database, it seems that prototype theory is readily applicable but basic-level theory is less so. I can see where, given a question like “How did the Egyptians build the pyramids?”, people would categorize potentially relevant information and feel that some pieces of information were better than others (like in Barsalou’s ad-hoc categories referred to at the bottom of page 45). But I’m not sure I understand Lakoff’s claim that basic-level categories can be extended from the physical world into “event categories” by way of metaphor (bottom of page 47). What does he mean by event categories? Would these event categories have relevance to questions of information retrieval or abstract knowledge organization like the one posed above?

2003.09.02 - SLIDE 41IS 202 - FALL 2003

Discussion Questions (Lakoff)

• Simon King on Lakoff– Does experientialism completely invalidate

objectivism as a model of cognition? Even if we accept prototype theory and believe that human thought is not based on rigid classical categorization, isn’t possible that humans find it simpler to internally represent categories by their prototypical members and that this is simply a cognitive shortcut? The same thoughts could be formed by manipulating categories, even though this is not how humans think. Does it even make sense to think about cognition outside of our human context?

2003.09.02 - SLIDE 42IS 202 - FALL 2003

Discussion Questions (Lakoff)

• Simon King on Lakoff– Does objectivism preclude imagination and creativity?

If thought is atomistic does this mean that there can be no intuition or ‘leaps’ of logic? Lakoff states “every time we categorize something in a way that does not mirror nature, we are using general human imaginative capabilities.” Does this mean that imagination can be considered a form of logical error or a mistaken internal representation of the world? Is imagination a requirement of ‘thought’ or can some organism or system be said to think if it operates on logic alone?

2003.09.02 - SLIDE 43IS 202 - FALL 2003

Next Time

• Knowledge Representation

2003.09.02 - SLIDE 44IS 202 - FALL 2003

Homework (!)

• Read the handouts – “The Vocabulary Problem in Human-System

Communication” (G. W. Furnas, T. K. Landauer, L. M. Gomez, S. T. Dumais)

– “Commonsense-Based Interfaces” (M. Minsky)

– “CYC: A Large-Scale Investment in Knowledge Infrastructure” (D. B. Lenat)