transcript
- Slide 1
- Cluster-based models of belief networks, social networks, and
cultural knowledge Josh Tenenbaum, MIT 2007 MURI Annual Meeting
Work of Charles Kemp, Chris Baker, Tom Griffiths, Pat Shafto,
Vikash Mansinghka, Dan Roy
- Slide 2
- Goal Algorithmic tools for uncovering structure in belief
networks, social networks, and joint structure (social-belief
networks). Why? Joint social-belief structure ~ culture Algorithms
let us map cultural knowledge quickly and semi-automatically,
detect changes and track dynamics.
- Slide 3
- Approach Data Peoples beliefs about properties of objects
Relations between people Peoples beliefs about relations between
objects (or people). Representation: cluster-based models Clusters
of things: categories Clusters of people: social groups Clusters of
people who share similar beliefs about clusters of things (or
people): cultural groups
- Slide 4
- Approach Learning: Bayesian inference from data Relational
models: analyze arbitrary relational databases of beliefs, not just
a single matrix Nonparametric models: automatically determine
complexity of representations Hierarchical models: multiple levels
of structure Nested models: structures with structure Result: a
flexible toolkit that goes qualitatively beyond standard
algorithms. e.g., ability to discover cultural groups characterized
by a shared understanding of the environments relational
structure.
- Slide 5
- Talk outline Classic cluster-based methods New methods
Clustering with arbitrary relational systems Hierarchical
relational clustering Cross-cutting clustering with nested models
Cross-cutting relational clustering Application to Guatemala data
from Atran & Medin
- Slide 6
- Classic cluster-based methods Belief networks: mixture
models
- Slide 7
- Classic cluster-based methods Belief networks: mixture
models
- Slide 8
- Classic cluster-based methods Social networks: block
models
- Slide 9
- Classic cluster-based methods Cultural knowledge (joint
social/belief structure): cultural consensus model Not cluster-
based. SVD on matrix of people x questions
- Slide 10
- Problems with classic methods No principled tools for
discovering different cultural groups characterized by different
belief networks. CCM not intended to find cultural groups, but
rather to uncover (and test for) shared knowledge and
authoritativeness in a single cultural group. Test theory without
an answer key Can only represent simple forms of knowledge that fit
into a single two-mode matrix. Cultural knowledge is often much
richer.
- Slide 11
- Talk outline Classic cluster-based methods New methods
Clustering with arbitrary relational systems Hierarchical
relational clustering Cross-cutting clustering with nested models
Cross-cutting relational clustering Application to Guatemala data
from Atran & Medin
- Slide 12
- people social relation Alyawarra tribe, central Australia
(Denham) 104 individuals 27 binary social relations 3 attributes:
kinship class, age, sex (used only for cluster validation, not
learning) people attributes Clustering arbitrary relational
systems
- Slide 13
- Infinite relational model (IRM) discovers 15 clusters
Clustering arbitrary relational systems
- Slide 14
- International relations circa 1965 (Rummel) 14 countries: UK,
USA, USSR, China, . 54 binary relations representing interactions
between countries: exports to( USA, UK ), protests( USA, USSR ), .
90 (dynamic) country features: purges, protests, unemployment,
communists, # languages, assassinations, .
- Slide 15
- Slide 16
- concept predicate Learning a medical ontology Data from UMLS
(McCrae et al.): 134 concepts: enzyme, hormone, organ, disease,
cell function... 49 predicates: affects(hormone, organ),
complicates(enzyme, cell function), treats(drug, disease),
diagnoses(procedure, disease)
- Slide 17
- Learning a medical ontology e.g., Diseases affect Organisms
Chemicals interact with Chemicals Chemicals cause Diseases
- Slide 18
- Hierarchical relational clustering
- Slide 19
- Slide 20
- Models so far all learn a single system of clusters. We would
like to be able to discover multiple cross-cutting systems of
clusters. Within an individuals mind: multiple mental models of a
single complex domain. Across individuals in a population: multiple
cultural groups with different characteristic mental models.
Cross-cutting clustering with nested models
- Slide 21
- Conventional mixture model Cross-cutting clustering with nested
models
- Slide 22
- CrossCat model Cross-cutting clustering with nested models
- Slide 23
- Experimental tests of CrossCat = Stimuli: Task: Repeated free
sorting
- Slide 24
- Experimental tests of CrossCat Results: Conventional mixture
model CrossCat model Human frequency
- Slide 25
- Experimental tests of CrossCat Results: Conventional mixture
model CrossCat model Human frequency
- Slide 26
- Nested relational model: Cross-cutting clustering with nested
models people relation Infinite relational model: people relation
people relation
- Slide 27
- Talk outline Classic cluster-based methods New methods
Clustering with arbitrary relational systems Hierarchical
relational clustering Cross-cutting clustering with nested models
Cross-cutting relational clustering Application to Guatemala data
from Atran & Medin
- Slide 28
- Culture and cognition in Guatamela (Atran & Medin) Subjects
12 native Itza maya 12 immigrant Ladino 12 immigrant Qeqchi maya
Questions Does plant i help animal j? Does animal j help plant i?
animal plant 36 people 2 directions
- Slide 29
- Discovering cultural groups with the IRM animal plant 36 people
PA+
- Slide 30
- Cultural knowledge across groups animal plant 24 people (Itza,
Ladino) 2 directions
- Slide 31
- Ground Truth ecology
- Slide 32
- Cultural knowledge across groups Itza Ladino PA+AP+
- Slide 33
- I1 I2 I3 I5 I7 I8 I9 I10 I12 L1 L2 L3 L4 L5 L6 L7 L8 L9 L10 L11
L12 I6 I11 Q3 Q6 Q8 Q9 Q10 Q11 Q12 Q1 Q2 Q4 Q5 Q7 I4 Discovering
cultural groups with the nested IRM Data: PA+ Nesting structure
Cluster people Cluster plants within people Cluster animals within
plants and people Clusters of people found:
- Slide 34
- I1 I2 I3 I5 I7 I8 I9 I10 I12 ciricote ramon chicozapote
stranglerfig allspice coatimundi paca whitelippedpeccary
crestedguan ocellatedturkey greatcurassow tinamou spidermonkey
howlermonkey kinkajou pigeon bat 0.63 chachalaca squirrel agouti
parrot toucan scarletmacaw 0.8 amapola guano yaxnik broompalm
chachalaca coatimundi paca collaredpeccary whitelippedpeccary
crestedguan ocellatedturkey squirrel greatcurassow tinamou agouti
parrot kinkajou toucan boa ferdelance pigeon scarletmacaw bat 0.4
jabin madrial pukte watervine ceiba xate santamaria killervines
manchich corozo chapay pacaya herb grasses jaguar paca
collaredpeccary whitelippedpeccary margay mountainlion 0.59
chachalaca paca crestedguan ocellatedturkey squirrel greatcurassow
tinamou agouti parrot toucan boa ferdelance pigeon scarletmacaw
0.15 whitetaileddeer tapir redbrocketdeer boa ferdelance 0.98
agouti armadillo 0.39 mahogany cedar cordagevine kanlol chaltekok
0.004 jaguar boa laughingfalcon 0.03
- Slide 35
- L1 L2 L3 L4 L5 L6 L7 L8 L9 L10 L11 L12 I6 I11 ciricote ramon
chicozapote stranglerfig coatimundi paca collaredpeccary
whitelippedpeccary ocellatedturkey squirrel greatcurassow agouti
parrot spidermonkey howlermonkey kinkajou scarletmacaw 0.77 toucan
0.8 mahogany guano chachalaca coatimundi paca collaredpeccary
whitelippedpeccary crestedguan ocellatedturkey squirrel
greatcurassow agouti parrot spidermonkey howlermonkey kinkajou
toucan ferdelance pigeon scarletmacaw bat 0.4 jabin cedar madrial
pukte watervine ceiba allspice santamaria killervines broompalm
chapay herb grasses paca collaredpeccary crestedguan
ocellatedturkey greatcurassow tinamou armadillo margay mountainlion
pigeon 0.25 greatcurassow pigeon bat 0.22 whitetaileddeer tapir
redbrocketdeer ferdelance 0.76 boa 0.86 yaxnik cordagevine kanlol
chaltekok 0.028 chachalaca ocellatedturkey squirrel parrot toucan
scarletmacaw 0.27 bat 0.57 crestedguan chachalaca whitetaileddeer
armadillo jaguar boa laughingfalcon 0.41
- Slide 36
- Q3 Q6 Q8 Q9 Q10 Q11 Q12 ciricote ramon chicozapote watervine
cordagevine corozo spidermonkey howlermonkey 0.4 amapola
stranglerfig broompalm jaguar chachalaca whitetaileddeer
whitelippedpeccary crestedguan ocellatedturkey greatcurassow
tinamou parrot tapir mountainlion spidermonkey howlermonkey
kinkajou redbrocketdeer toucan boa ferdelance laughingfalcon
scarletmacaw pigeon 0.14 herb grasses whitetaileddeer
collaredpeccary ocellatedturkey greatcurassow armadillo ferdelance
pigeon 0.17 paca 0.26 jabin mahogany cedar guano madrial pukte
yaxnik ceiba xate allspice santamaria killervines manchich kanlol
chaltekok chapay pacaya 0.01 Redbrocketdeer boa 0.32
- Slide 37
- Q1 Q2 Q4 Q5 Q7 ciricote pukte watervine killervines
spidermonkey howlermonkey toucan 0.2 amapola mahogany cedar ramon
chicozapote madrial stranglerfig yaxnik jaguar chachalaca paca
crestedguan ocellatedturkey squirrel greatcurassow tinamou parrot
spidermonkey howlermonkey toucan pigeon laughingfalcon scarletmacaw
0.39 grasses broompalm collaredpeccary whitelippedpeccary boa
ferdelance 0.35 allspice cordagevine manchich kanlol chaltekok
chapay 0.01 squirrel 0.1 ceiba jaguar ocellatedturkey squirrel
parrot toucan pigeon 0.37 jabin guano santamaria corozo peca
collaredpeccary whitelippedpeccary agouti 0.3 herb xate pacaya
whitetaileddeer tinamou parrot armadillo tapir redbrocketdeer
pigeon 0.18
- Slide 38
- Discovering cultural groups with the nested IRM Data: AP+
Nesting structure Cluster people Cluster plants within people
Cluster animals within plants and people Clusters of people found:
L2 L3 L6 L7 L10 L11 L12 I1 I2 I3 I4 I5 I6 I7 I8 I9 I11 I12 L4 L5 L8
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 L1 L9 I10
- Slide 39
- Conclusions A flexible toolkit for statistical learning about
cultural knowledge and cultural groups. Relational models: analyze
arbitrary relational databases of beliefs, not just a single matrix
Nonparametric models: automatically determine complexity of
representations Hierarchical models: multiple levels of structure
Nested models: structures with structure Can automatically discover
important qualitative structure in real-world data (Atran &
Medin).
- Slide 40
- Ongoing and future work More flexible nested structures More
dynamic data and analyses Second-generation Guatemala data
Political data sets: voting records, international relations More
structured representations necessary to capture cultural stories:
grammars, logical schemas (c.f. Forbus, Richards, Atran) people
plantsanimals directionality
- Slide 41
- The end
- Slide 42
- Slide 43
- Discovering structure in relational data 3 9 1 13 5 11 7 14 2
10 6 12 4 8 15 3 9 1 13 5 11 7 14 2 10 6 12 4 8 15 3 9 1 13 5 11 7
14 2 10 6 12 4 8 15 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 2 3 4 5 6
7 8 9 10 11 12 13 14 15 InputOutput person TalksTo(person,person)
person
- Slide 44
- O z Infinite Relational Model (IRM) 3 9 1 13 5 11 7 14 2 10 6
12 4 8 15 0.9 0.1 0.9 0.1 3 9 1 13 5 11 7 14 2 10 6 12 4 8 15 3 9 1
13 5 11 7 14 2 10 6 12 4 8 15
- Slide 45
- Model fitting
- Slide 46
- a(3). a(9). a(1). a(13). a(5). a(11). b(7). b(14). b(2). b(10).
b(6). c(12). c(4). c(8). c(15). r(X,Y) a(X),a(Y). (0.0) r(X,Y)
a(X),b(Y). (0.9) r(X,Y) c(X),a(Y). (0.95)... r(3,7). r(1,10).
r(2,4)... The concepts discovered by the IRM can serve as
primitives in complex logical theories (cf. clustering approaches
to predicate invention, e.g., Craven and Slattery (2001) or
Popescul and Ungar (2004)) 3 9 1 13 5 11 7 14 2 10 6 12 4 8 15 3 9
1 13 5 11 7 14 2 10 6 12 4 8 15
- Slide 47
- Related Work Relational models Sociology: Wang and Wong (1987);
Nowicki and Snijders (2001) Machine learning: Taskar, Segal and
Koller (2001) Wolfe and Jensen (2004) Wang, Mohanty and McCallum
(2005) Nonparametric Bayesian models Ferguson (1973); Neal (1991)
Nonparametric Bayesian relational models Carbonetto, Kisynski, de
Freitas and Poole (2005) Xu, Tresp, Yu, Kriegel (2006)
- Slide 48
- Optimization (or inference) Global proposals Split and merge
clusters Local proposals Re-assign one entity to best cluster based
on current assignments of all other entities (i.e., Gibbs sampling)
Both cognitively plausible and computationally reasonable.
- Slide 49
- Slide 50
- O z Infinite relational model (IRM) 3 9 1 13 5 11 7 14 2 10 6
12 4 8 15 0.9 0.1 0.9 0.1 3 9 1 13 5 11 7 14 2 10 6 12 4 8 15 3 9 1
13 5 11 7 14 2 10 6 12 4 8 15
- Slide 51
- O z Infinite relational model (IRM) 3 9 1 13 5 11 7 14 2 10 6
12 4 8 15 0.9 0.1 0.9 0.1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 2 3
4 5 6 7 8 9 10 11 12 13 14 15
- Slide 52
- Independent symmetric beta priors on : Chinese Restaurant
Process over z: Goal: Infer Infer (integrating out to reduce space
of unknowns) Generating and z
- Slide 53
- Global-local search process
- Slide 54
- Joint modeling of belief systems and social systems animal
plant person helps(plant,animal,person judging) Data from Atran and
Medin
- Slide 55
- Slide 56
- ItzaLadinos
- Slide 57