Date post: | 22-Dec-2015 |
Category: |
Documents |
View: | 216 times |
Download: | 1 times |
TAMARA NICOL MEDINAJOHN TRUESWELL
LILA GLEITMANUniversity Of Pennsylvania
JESSE SNEDEKERHarvard University
Society for Research in Child DevelopmentApril 2, 2009, Denver, CO
Using Extra-Linguistic Cues to Identify Good Word Learning
Instances
Just look at the world!
Observe physical and temporal contingencies between words and objects. (At least, for physically observable objects.)
Experimental evidence supports ease of mapping Fast mapping (e.g., Carey, 1978; Mervis & Bertrand,
1994; Behrend et al., 2001; Jaswal & Markman, 2001) Cross-situational word learning (e.g., Yu & Smith,
2007; Smith & Yu, 2008; Vouloumanos, 2008; Xu &Tenenbaum, 2007)
It’s Not that Easy! (Augustine, Locke, Quine, Gleitman, Fodor, Siskind, etc.)
Reference problem Book? Cat? Shoes? Chair? Cheerios? Cup?
Rug? Pants? Head? Hand? …Frame problem
Dog or Puppy? Hand or Finger? Red or Ball?
Naturalistic learning conditions Medina, Trueswell, Snedeker, & Gleitman. (2008). When the shoe
fits: Cross-situational word learning in realistic learning environments. BUCLD.
How do learners narrow down the possibilities?
Linguistic context (Landau & Gleitman, 1985; Gleitman, 1990; Gillette, Gleitman, Gleitman, & Lederer, 1999)
Learning biases Whole object constraint (Markman, 1989) Mutual exclusivity (Markman & Wachtel, 1988;
Markman, Wasow, & Hansen, 2003)
Social-attentional cues (e.g., Baldwin 1991, 1993; Tomasello & Akhtar, 1995; Bloom, 2002; Behne, Carpenter, & Tomasello, 2005)
Social-Attentional Cues
Nonverbal cues can reduce the range of possible interpretations.
Direction of speaker eye-gaze (Baldwin, 1991, 1993; Trueswell & Gleitman, 2003; Nappa, Wessel, McEldoon, Gleitman, & Trueswell, 2009)
Joint attention: Occurs naturally when parent and child are focused on the same thing at the same time (Baldwin, 1991; Bruner, 1978)
~ 70% of mothers’ utterances (Collis, 1977; Harris, Jones, & Grant, 1983; Tomasello & Todd, 1983)
Positively associated with early vocabulary acquisition (Tomasello & Todd, 1983; Harris, Jones, Brookes, & Grant, 1986; Tomasello, Mannle, & Kruger, 1986; Akhtar, Dunham, & Dunham, 1991)
Quality of Learning Instances(Baldwin, 1991)
But what about the lack of perfect contingency between word and referent?
Quality of Learning Instances(Baldwin, 1991)
But what about the lack of perfect contingency between word and referent?
Follow-in labeling: eye gaze, voice direction, and body posture oriented at object child is currently focused on
16-19 mo-olds mapped correctlyDiscrepant labeling: eye gaze, voice direction,
and body posture directed at a hidden (but previously seen) object, while infant is focused on another object
Infants did not map the word to the focused-object.
Social-attentional cues in interaction
(Frank, Goodman, & Tenenbaum, In Press)
Rollins corpus (CHILDES): mom and baby (6 mo)
Social-attentional cues Infant: Hands, Mouth (infant only), “Touch”, Looking
(direction of eye gaze) Caregiver: Hands, “Touch”, Looking (direction of eye
gaze)
Cross-situational word-learning model successfully discovered the mappings between words and objects.
Social-attentional cues in interaction
(Frank, Goodman, & Tenenbaum, In Press)
Rollins corpus (CHILDES): mom and baby (6 mo)
Social-attentional cues Infant: Hands, Mouth (infant only), “Touch”, Looking
(direction of eye gaze) Caregiver: Hands, “Touch”, Looking (direction of eye
gaze)
Cross-situational word-learning model successfully discovered the mappings between words and objects.
Social-attentional cues in interaction
(Frank, Goodman, & Tenenbaum, In Press)
Rollins corpus (CHILDES): mom and baby (6 mo)Social-attentional cues
Infant: Hands, Mouth (infant only), “Touch”, Looking (direction of eye gaze)
Caregiver: Hands, “Touch”, Looking (direction of eye gaze)
Cross-situational word-learning model successfully discovered the mappings between words and objects.
Joint attention? Follow-in?What would interaction look like if child were
initiating actions?
Our Goals
Look at a representative sample of parent-child interactions.
Explore the conditions under which word meaning is transparent (or not) from extra-linguistic cues alone: Presence (or absence) of cues Timing and coordination of cues
Joint attention? Follow-in?
Selection of Stimuli
Large video corpus of parent-child interactions in natural settings (home, outdoors, etc.) Snedeker, J. (2001). Interactions between infants (12-15
months) and their parents in four settings. Unpublished corpus.
Selection of Stimuli
Word learning “norming” study Gertner, Y., Fisher, C., Gleitman, L., Joshi, A., & Snedeker, J. (In
progress). Machine implementation of a verb learning algorithm. Adapation of Human Simulation Paradigm (Gillette, Gleitman,
Gleitman, & Lederer, 1999; Snedeker and Gleitman, 1999) Randomly selected six instances of highly frequent content
words. Each instance was edited into a 40-second “vignette”. Sound turned off.
Visual context only cue to word meaning, placing viewers in the situation of the early word learner.
Utterance of target word (at 30 sec) indicated by a BEEP.
Guess the “mystery” word in each vignette.
Two types of vignettes “High Informative” – vignettes guessed by >50% of
participants “Low Informative” – vignettes guessed by <33% of
participants
Selection of Stimuli
Two types of vignettes “High Informative” – vignettes guessed by >50% of
participants Rare (only 7% of vignettes). All basic level objects.
“Low Informative” – vignettes guessed by <33% of participants
Stimuli for Current Study 8 nouns: bag, ball, book, horse, necklace, nose, phone,
shoe One HI vignette One LI vignette
Selection of Stimuli
N = 12 (ages 3;1 to 5;4)Modified for fun!
Shorter vignettes with funny noises “What do you think the parent said?” Celebratory animation
Pilot Study: Children
Extra Linguistic Cue Coding
Is the target object visible in the scene (on screen)?Is the child moving or reaching towards the object?Is the child handling the object?Is the child looking at the object?Is the child looking at the parent?Is the parent moving or reaching towards the
object?Is the parent handling the object?Is the parent looking at the object?Is the parent looking at the child?
Presence of Target Object
0
10
20
30
40
Presence of Object
Ave
rage
Tot
al D
urat
ion
of C
ue (
sec)
HI
LI
Error bars reflect Standard Error of the Mean.
Cue Occurrence at Word Onset
0%
50%
100%
Child Parent
% O
ccur
renc
e at
Tar
get W
ord
HI
LI
*2 = 7.27
*2 = 4.00
*2 = 4.27
Joint Attention at Word Onset
Child Looking at and/or Handling ObjectAND Parent Looking at Object
0%
50%
100%
Joint Attention
% O
ccur
renc
e at
Tar
get
Wor
d
HI
LI
*2 = 4.00
What is the timing of cues?
Follow-in?Parent refers to object under child’s focus of
attention.
First onset of cues relative to word onset.
First Onset of Cues
Child Looking at Object t(1,12)=1.56, p=0.14
Child Moving/Reaching Toward Object t(1,12)=2.05, p=0.06
Child Handling Object t(1,12)=2.96, p=0.01
Parent Handling Object t(1,8)=1.09, p=0.31
0 10 20 30 40
LI
HI
Info
rma
tivity
of V
ign
ette
s
Time (sec)
Error bars reflect Standard Error of the Mean.
First Onset of Cues
0 10 20 30 40
LI
HI
Info
rma
tivity
of V
ign
ette
s
Time (sec)
Parent Looking at Child t(1,13)=0.54, p=0.59
Parent Looking at Object t(1,12)=0.08, p=0.93
Child Looking at Parent t(1,6)=0.02, p=0.98
Parent Moving/Reaching Toward Object t(1,7)=0.15, p=0.89
Error bars reflect Standard Error of the Mean.
Differentiating HI and LI Vignettes
High Informative Follow-In: Utterance of target word immediately
after first onset of child’s shift in focus towards object.
Joint Attention: Co-occurring high rates of child’s attention to object and parent’s attention to child and object.
Low Informative Delayed follow-in. Low joint attention.