Cue Reliability and Re-Weighting in Word Recognition

Goals of Study

General Approach

Conclusions

Listeners are influenced by the distributions of cues in the current input Listeners can dynamically re-weight cues in response Listeners selectively re-weight cues (i.e., don’t converge to 50/50 responses) Important for future cue integration experiments: most standard balanced designs create conflict

References & Acknowledgments

[1] Connine, Blasko & Hall (1991) JML; [2] Szostak & Pitt (2013) APP; [3] Bicknell et al (in revision) [4] Idemaru & Holt (2011) JEP:HPP This work was partially funded by NSF NRT #1449828 (W.B.) and NICHD R01 HD075797 (T.F.J.).

Manipulate acoustic cues (VOT) and semantic cues (biasing context) in a sentence (see [1,2,3]) Task: did you hear “tent” or “dent”? Mechanical Turk subjects (N = 106) VOTs: 10, 30, 35, 40, 50, 85ms 7 sentence frames repeated for each semantics, distance, & VOT combination = 168 total trials (no fillers)

1Department of Brain & Cognitive Sciences, University of Rochester 2Department of Computer Science, University of Rochester

Wednesday Bushong1 and T. Florian Jaeger1,2

dent-biasing/tent-biasing semantic context

/d/-like VOT /t/-like

Results

0.00

0.25

0.50

0.75

1.00

20 40 60 80Voice Onset Time (ms)

Pro

port

ion

Res

pons

es /t

/ Semantic CueDent-biasingTent-biasing

Responses by Cue Conditions

Semantic & acoustic cues are integrated: categorization responses in both groups (collapsed in graph) are sensitive to both VOT and biasing context.

0.0

0.5

1.0

1.5

2.0

10 30 35 40 50 85VOT (ms)

Sem

antic

Cue

Effe

ct (

log-

odds

)

GroupHigh-ConflictLow-Conflict

Semantic Cue Effect by Group

High-Conflict group down-weights effect of semantic cue on their responses over course of experiment while Low-Conflict stays constant

Changes in Cue Weights Over Time

0

1

2

3

-1 0 1Trial (Scaled)

Pre

dict

ed s

eman

tic e

ffect

(log-odds)

GroupHigh-ConflictLow-Conflict

Semantic Cue(a)

20

40

60

80

-1 0 1Trial (Scaled)

VO

T (m

s)

-1

0

1

GroupDifference(log-odds)

Acoustic Cue(b)

Little change in weighting of VOT over time in either group

High-Conflict Group Low-Conflict Group

10 30 35 40 50 85 10 30 35 40 50 850

10

20

<- More /d/-like More /t/ like -> <- More /d/-like More /t/-like ->

Voice Onset Time (ms)

Num

ber o

f Tria

ls

Semantic Cue Dent-biasing Tent-biasing

Distribution of Cues by Exposure Group

...the ?ent in the fender/forest...

High-Conflict group shows a smaller effect of semantic cue on responses

Beginning Middle End

20 40 60 80 20 40 60 80 20 40 60 80

-5.0

-2.5

0.0

2.5

5.0

VOT

Est

ima

ted

/t/ r

esp

on

se

(log-odds)

GroupHigh-ConflictLow-Conflict

Listeners integrate top-down and bottom-up cues in speech perception But cues are distributed differently in different contexts (e.g., between speakers) Do listeners adaptively change cue weightings given new exposure distributions?

Why Do We See This Re-Weighting

Pattern? Listeners could reduce cue conflict by downweighting either VOT or semantics. Why do listeners downweight semantics? Use of multiple acoustic cues can show similar effects: when correlations between two acoustic cues are perturbed, listeners down-weight the less reliable cue [4] This could suggest that listeners might consider semantic information to be less reliable than acoustic features (at least in word recognition tasks)

Manipulation

High-Conflict Group: VOT & semantic cues perfectly uncorrelated (i.e., very /d/-like stimuli paired with tent-biasing semantics as much as dent-biasing semantics) Low-Conflict Group: VOT & semantic cues are correlated in the expected direction

How can we measure relative reliability of semantic & acoustic features to word recognition? Do listeners rationally re-weight cues given evidence over time?

Future Work

Smaller semantic effect strikingly consistent across subjects

0

2

4

6

8

0.0 2.5 5.0Empirical semantic effect (log-odds)

# S

ubje

cts

GroupHigh-ConflictLow-Conflict