This work was supported by SSHRC grant 435-2016-0747 to Meghan Clayards. Participants • 54 monolingual speakers of Canadian English Dual task • 2AFC + visual search • 2AFC (head or had): 5 spectral (TANDEM-STRAIGHT [7]) x 5 duration steps (PSOLA in Praat) • Visual search: A black diamond is present? Gradiency in phoneme categorization • Visual Analogue Scaling (VAS: heed—hid): 7 spectral x 7 duration steps Cognitive abilities • Working memory (Backward Digit Span, Reading Span), Inhibitory control (Stroop, Go/No-go) [8] Do listeners show adaptive strategies for phonetic categories in the face of cognitive load? If so, what makes some listeners better adaptors? Speech perception under cognitive load • Speech perception is an inherently attention demanding process and limited attentional resources have been shown to disruptive effects on speech perception [1, 2]. Cognitive abilities in speech perception • Cognitive abilities (e.g. inhibitory control, working memory) play a role in speech perception in adverse conditions [3, 4]. Gradiency in phoneme categorization • Listeners who have more gradient categorization patterns are more sensitive to acoustic-phonetic details [5, 6]. References: [1] Mattys, S. L., & Wiget, L. (2011). Effects of cognitive load on speech recognition. Journal of Memory and Language, 65(2), 145–160. [2] Mitterer, H., & Mattys, S. L. (2017). How does cognitive load influence speech perception? An encoding hypothesis. Attention, Perception, & Psychophysics, 79(1), 344–351. [3] Janse, E., & Adank, P. M. (2012). Predicting foreign-accent adaptation in older adults. Quarterly Journal of Experimental Psychology, 65(8), 1563–1585. [4] Tamati, T. N., Gilbert, J. L., & Pisoni, D. B. (2013). Some factors underlying individual differences in speech recognition on PRESTO: A first report. Journal of the American Academy of Audiology, 24(7), 616– 634. [5] Kapnoula, E. C., Winn, M. B., Kong, E. J., Edwards, J. R., & McMurray, B. (2017). Evaluating the sources and functions of gradiency in phoneme categorization: An individual differences approach. Journal of Experimental Psychology: Human Perception and Performance, 43(9), 1594–1611. [6] Kong, E. J., & Edwards, J. R. (2016). Individual differences in categorical perception of speech: Cue weighting and executive function. Journal of Phonetics, 59, 40–57. [7] Kawahara, H., Takahashi, T., Morise, M., & Banno, H. (2009). Development of exploratory research tools based on TANDEM-STRAIGHT. Proceedings of Asia-Pacific Signal and Information Processing Association, 2009 Annual Summit and Conference. pp. 111–120. [8] Mueller, S. T., & Piper, B. J. (2014). The psychology experiment building language (PEBL) and PEBL test battery. Journal of Neuroscience Methods, 222, 250–259. [9] Heald, S. L. M., & Nusbaum, H. C. (2014). Speech perception as an active cognitive process. Frontiers in Systems Neuroscience, 8. RQ1: Listeners flexibly adapted to unfamiliar vowels by up-weighting a secondary cue when a primary cue is not informative. A: Primary use of spectral differences at Baseline, B1: Up-weighting of the duration cue when the spectral cue is not informative at Exposure, : Differences in proportion /æ/ responses (330ms – 130ms) RQ2: Individuals varied widely in categorization gradiency and this variability was related to their use of a secondary cue, but the link between gradiency and adaptation was not found. RQ3: Individual differences in inhibitory control correlated with the amount of adaptation. Results Proportion /æ/ responses A baseline1 exposure baseline2 Block Diff. /æ/ responses (long-short) -50 Categorization gradiency (VAS slope) Gradiency and secondary cue use 179 179 179 179 179 179 179 179 179 179 179 179 179 179 180 180 180 180 180 180 180 180 180 180 180 180 180 180 182 182 182 182 182 182 182 182 182 182 182 182 182 182 183 183 183 183 183 183 183 183 183 183 183 183 183 183 185 185 185 185 185 185 185 185 185 185 185 185 185 185 191 191 191 191 191 191 191 191 191 191 191 191 191 191 85 85 85 85 85 85 85 85 85 85 85 85 85 85 151 151 151 151 151 151 151 151 151 151 151 151 151 151 196 196 196 196 196 196 196 196 196 196 196 196 196 196 200 200 200 200 200 200 200 200 200 200 200 200 200 200 148 148 148 148 148 148 148 148 148 148 148 148 148 148 204 204 204 204 204 204 204 204 204 204 204 204 204 204 208 208 208 208 208 208 208 208 208 208 208 208 208 208 209 209 209 209 209 209 209 209 209 209 209 209 209 209 205 205 205 205 205 205 205 205 205 205 205 205 205 205 baseline2 0.1 0.2 0.00 0.25 0.50 0.75 1.00 Proportion /æ/ responses Duration 130 ms 330 ms Inhibitory control and perceptual adaptation 0 0 40 80 120 0 40 80 120 0 40 80 120 0 40 80 120 0 40 80 120 0 40 80 120 count A Individual differences in plasticity in speech perception under cognitive load CHSCOM2019 Linköping, Sweden June 9-12, 2019 Donghyun Kim, Meghan Clayards Contact: [email protected]1. How and to what extent are speech perception abilities modulated by cognitive load? 2. Do individuals differ in their use of acoustic cues in phonetic categorization under cognitive load? 3. Are individual cue weighting strategies under cognitive load related to individuals’ cognitive abilities and gradiency in phoneme categorization? Research Questions RQ1: Listeners overall showed an increased reliance on the primary (spectral quality) and the secondary cue (duration) under cognitive load. RQ2: There were considerable differences across individuals in the effect of cognitive load on perceptual cue weighting. RQ3: Individual differences in adaptive cue weighting strategies under cognitive load were linked to cognitive abilities (but were not linked to gradiency in phoneme categorization). Individuals with better inhibitory control showed more adaptive spectral change. Methods Background Results Increased cue weights under cognitive load may be interpreted as an active cognitive process [9] Some listeners showed an increased (decreased) reliance on spectral quality whereas others showed an increased (decreased) reliance on vowel duration under cognitive load. No Load Cognitive Load 120 180 240 300 360 120 180 240 300 360 /ɛ/ 50% /æ/ Duration step (ms) Spectral step (/ɛ/ to /æ/) 0.25 0.50 0.75 /æ/ response Average /æ/ responses by condition A No Load Cognitive Load 1 2 3 4 5 1 2 3 4 5 0.00 0.25 0.50 0.75 1.00 Spectral step (/ɛ/ to /æ/) Proportion /æ/ responses Duration 120 ms 180 ms 240 ms 300 ms 360 ms Cue weighting by condition B 24 28 56 78 132 142 151 167 181 187 222 224 234 237 239 244 246 248 249 250 251 252 253 254 255 256 257 259 260 261 262 263 264 265 266 267 268 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 0 1 2 -2.5 0.0 2.5 Working memory composite score Duration cue weight difference (Cognitive load - No load) Duration cue change and working memory 24 28 56 78 132 142 151 167 181 187 222 224 234 237 239 244 246 248 249 250 251 252 253 255 256 257 259 260 261 262 263 264 265 266 267 268 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 -2 0 2 0 2 4 Go/No-go score Spectral cue weight difference (Cognitive load - No load) Spectral cue change and Go/No-go No Load Cognitive Load 120 180 240 300 360 120 180 240 300 360 /ɛ/ 50% /æ/ Duration step (ms) Spectral step (/ɛ/ to /æ/) 0.00 0.25 0.50 0.75 1.00 /æ/ response Duration weight increase (Listener 234) A No Load Cognitive Load 120 180 240 300 360 120 180 240 300 360 /ɛ/ 50% /æ/ Duration step (ms) Spectral step (/ɛ/ to /æ/) 0.00 0.25 0.50 0.75 1.00 /æ/ response Duration weight decrease (Listener 278) B No Load Cognitive Load 120 180 240 300 360 120 180 240 300 360 /ɛ/ 50% /æ/ Duration step (ms) Spectral step (/ɛ/ to /æ/) 0.00 0.25 0.50 0.75 1.00 /æ/ response Spectral weight increase (Listener 261) A No Load Cognitive Load 120 180 240 300 360 120 180 240 300 360 /ɛ/ 50% /æ/ Duration step (ms) Spectral step (/ɛ/ to /æ/) 0.00 0.25 0.50 0.75 1.00 /æ/ response Spectral weight decrease (Listener 267) B /ɪ/ 50% /i/ 80 125 170 215 260 305 350 Duration steps (ms) Spectral steps (/ɪ/ to /i/) Visual Analog Scaling (VAS) /ɛ/ 50% /æ/ 120 180 240 300 360 Duration steps (ms) Spectral steps (/ɛ/ to /æ/) Dual-task heed hid Individuals with better working memory showed more adaptive duration change. Individual differences in adaptive cue weighting strategies under cognitive load, which may be interpreted as an active cognitive process, were linked to listeners’ cognitive abilities.
Transcript
This work was supported by SSHRC grant 435-2016-0747 to Meghan Clayards.
Participants
• 54 monolingual speakers of Canadian English
Dual task
• 2AFC + visual search
• 2AFC (head or had): 5 spectral (TANDEM-STRAIGHT [7]) x 5 duration steps (PSOLA in Praat)
• Working memory (Backward Digit Span, Reading Span), Inhibitory control (Stroop, Go/No-go) [8]
Do listeners show adaptive strategies for phonetic categories in the face of cognitive load? If so, what makes some listeners better adaptors?
Speech perception under cognitive load
• Speech perception is an inherently attention demanding process and limited attentional resources have been shown to disruptive effects on speech perception [1, 2].
Cognitive abilities in speech perception
• Cognitive abilities (e.g. inhibitory control, working memory) play a role in speech perception in adverse conditions [3, 4].
Gradiency in phoneme categorization
• Listeners who have more gradient categorization patterns are more sensitive to acoustic-phonetic details [5, 6].
Research Questions
References: [1] Mattys, S. L., & Wiget, L. (2011). Effects of cognitive load on speech recognition. Journal of Memory and Language, 65(2), 145–160. [2] Mitterer, H., & Mattys, S. L. (2017). How does cognitive load influence speech perception? An encoding hypothesis. Attention, Perception, & Psychophysics, 79(1), 344–351. [3] Janse, E., & Adank, P. M. (2012). Predicting foreign-accent adaptation in older adults. Quarterly Journal of Experimental Psychology, 65(8), 1563–1585. [4] Tamati, T. N., Gilbert, J. L., & Pisoni, D. B. (2013). Some factors underlying individual differences in speech recognition on PRESTO: A first report. Journal of the American Academy of Audiology, 24(7), 616–634. [5] Kapnoula, E. C., Winn, M. B., Kong, E. J., Edwards, J. R., & McMurray, B. (2017). Evaluating the sources and functions of gradiency in phoneme categorization: An individual differences approach. Journal of Experimental Psychology: Human Perception and Performance, 43(9), 1594–1611. [6] Kong, E. J., & Edwards, J. R. (2016). Individual differences in categorical perception of speech: Cue weighting and executive function. Journal of Phonetics, 59, 40–57. [7] Kawahara, H., Takahashi, T., Morise, M., & Banno, H. (2009). Development of exploratory research tools based on TANDEM-STRAIGHT. Proceedings of Asia-Pacific Signal and Information Processing Association, 2009 Annual Summit and Conference. pp. 111–120. [8] Mueller, S. T., & Piper, B. J. (2014). The psychology experiment building language (PEBL) and PEBL test battery. Journal of Neuroscience Methods, 222, 250–259. [9] Heald, S. L. M., & Nusbaum, H. C. (2014). Speech perception as an active cognitive process. Frontiers in Systems Neuroscience, 8.
RQ1: Listeners flexibly adapted to unfamiliar vowels by up-weighting a secondary cue when a primary cue is not informative.
A: Primary use of spectral differences at Baseline, B1: Up-weighting of the duration cue when the spectral cue is not informative at Exposure, B2: Differences in proportion /æ/ responses (330ms – 130ms)
RQ2: Individuals varied widely in categorization gradiency and this variability was related to their use of a secondary cue, but the link between gradiency and adaptation was not found.
RQ3: Individual differences in inhibitory control correlated with the amount of adaptation.
1. How and to what extent are speech perception abilities modulated by cognitive load?
2. Do individuals differ in their use of acoustic cues in phonetic categorization under cognitive load?
3. Are individual cue weighting strategies under cognitive load related to individuals’ cognitive abilities and gradiency in phoneme categorization?
Research QuestionsRQ1: Listeners overall showed an increased reliance on the primary (spectral quality) and the secondary cue (duration) under cognitive load.
RQ2: There were considerable differences across individuals in the effect of cognitive load on perceptual cue weighting.
RQ3: Individual differences in adaptive cue weighting strategies under cognitive load were linked to cognitive abilities (but were not linked to gradiency in phoneme categorization).
Individuals with better inhibitory control showed more adaptive spectral change.
Methods
Background
Results
Increased cue weights under cognitive load may be interpreted as an active cognitive process [9]
Some listeners showed an increased (decreased) reliance on spectral quality whereas others showed an increased (decreased) reliance on vowel duration under cognitive load.
No Load Cognitive Load
120 180 240 300 360 120 180 240 300 360
/ɛ/
50%
/æ/
Duration step (ms)
Spec
tral s
tep
(/ɛ/ t
o /æ
/)
0.250.500.75
/æ/ response
Average /æ/ responses by conditionA
No Load Cognitive Load
1 2 3 4 5 1 2 3 4 50.00
0.25
0.50
0.75
1.00
Spectral step (/ɛ/ to /æ/)Pr
opor
tion
/æ/ r
espo
nses
Duration120 ms180 ms240 ms300 ms360 ms
Cue weighting by conditionB
24
28
56
78
132
142
151
167
181
187
222
224
234
237
239
244246
248
249
250
251252
253
254
255256 257
259
260
261
262
263
264
265
266
267
268
270
271
272
273
274
275 276
277
278
279
280
281
282
283
284
285
286
0
1
2
-2.5 0.0 2.5Working memory composite score
Dur
atio
n cu
e w
eigh
t diff
eren
ce(C
ogni
tive
load
- N
o lo
ad)
Duration cue change and working memory
24
28
56
78
132
142
151
167
181
187
222
224
234
237
239
244246
248
249
250
251
252
253
255
256
257
259260
261
262
263 264
265
266
267
268
270
271
272
273274
275
276277
278
279
280
281
282
283
284
285
286
-2
0
2
0 2 4Go/No-go score
Spec
tral c
ue w
eigh
t diff
eren
ce(C
ogni
tive
load
- N
o lo
ad)
Spectral cue change and Go/No-go
No Load Cognitive Load
120 180 240 300 360 120 180 240 300 360
/ɛ/
50%
/æ/
Duration step (ms)Sp
ectra
l ste
p (/ɛ
/ to
/æ/)
0.000.250.500.751.00
/æ/ response
Duration weight increase (Listener 234)A
No Load Cognitive Load
120 180 240 300 360 120 180 240 300 360
/ɛ/
50%
/æ/
Duration step (ms)
Spec
tral s
tep
(/ɛ/ t
o /æ
/)
0.000.250.500.751.00
/æ/ response
Duration weight decrease (Listener 278)B
No Load Cognitive Load
120 180 240 300 360 120 180 240 300 360
/ɛ/
50%
/æ/
Duration step (ms)
Spec
tral s
tep
(/ɛ/ t
o /æ
/)
0.000.250.500.751.00
/æ/ response
Spectral weight increase (Listener 261)A
No Load Cognitive Load
120 180 240 300 360 120 180 240 300 360
/ɛ/
50%
/æ/
Duration step (ms)
Spec
tral s
tep
(/ɛ/ t
o /æ
/)
0.000.250.500.751.00
/æ/ response
Spectral weight decrease (Listener 267)B
/ɪ/
50%
/i/
80 125 170 215 260 305 350Duration steps (ms)
Spec
tral s
teps
(/ɪ/
to /i
/)
Visual Analog Scaling (VAS)
/ɛ/
50%
/æ/
120 180 240 300 360Duration steps (ms)
Spec
tral s
teps
(/ɛ/
to /æ
/)
Dual-task
heed hid
Individuals with better working memory showed more adaptive duration change.
Individual differences in adaptive cue weighting strategies under cognitive load, which may be interpreted as an active cognitive process, were linked to listeners’ cognitive abilities.
