The impact of auditory ecology on hearing-aid design and testing
Karolina Smeds, PhD
Director of research
Widex ORCA Europe, Stockholm, Sweden
www.orca-eu.info
Auditory Ecology describes the relationship between the acoustical environments experienced in everyday life and the perceptual demands of different people in these environments.
Gatehouse interpreted by Søgaard Jensen 2005
Why is it useful to focus on auditory ecology?1. Development of hearing aids2. Evaluation of hearing aids (development and clinically)3. Fitting and fine-tuning hearing aids4. Demonstration of hearing aid features5. …
Auditory Ecology describes the relationship between the acoustical environments experienced in everyday life and the perceptual demands of different people in these environments.
Gatehouse interpreted by Søgaard Jensen 2005
Why is it useful to focus on auditory ecology?1. Development of hearing aids2. Evaluation of hearing aids (development and clinically)3. Fitting and fine-tuning hearing aids4. Demonstration of hearing aid features5. …
A journey towards a stronger focus on auditory ecology
A 10 year thought process…
Widex ORCA Europe
Widex HQ
Outline
1. Noise reduction in hearing aids ”NR”
2. Realistic signal-to-noise ratios ”SNR”– Example: noise reduction study
– Example: speech testing
3. Common Sound Scenarios ”CoSS”
4. Summary
NR SummaryCoSSSNR Speech testNR
”How does Widex’ SII-based Noise Reduction system, the Speech Enhancer, compare to other noise reduction systems?”
ISAAR. Binaural processing and spatial hearing. Helsingør, Denmark: 445-452.
NR SummaryCoSSSNR Speech testNR
Method
• 12 modern hearing aids
• Three audiograms
• Long-term average test box gain
measurements
• Speech in stationary speech-shaped
noise
– Signal-to-noise ratios (SNRs) varied from
+6 to -12 dB in 3-dB steps (plus one
condition with “pure speech”)
• Measurements with NR ON and OFF
• Gain reduction =
Gain(NR_ON) – Gain(NR_OFF)
• Reduction contours to facilitate comparisons
NR SummaryCoSSSNR Speech testNR
Pure speecch
Decreasing SNR
Increased gain
Effect of noise reduction in 12 hearing aids
-8
-8
-6
-6
-6
-6
-6 -4
-4-4
-4
-4 -2
-2-2
-2
00
0
0 0
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
-14
-12
-12
-12
-10-1
0
-10
-10
-8
-8
-8
-8
-8
-8
-8
-6-6
-6
-6
-6
-4
-4
-4
-4
-4
-4
-2
-2
-2-2-2
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
-10
-10-10
-8
-8
-8-8
-6
-6
-6-6
-4
-4
-4-4
-2
-2-2-2
0
0
0
00
0
0
0
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
-4
-4
-2
-2
-2-2
-4
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
-6
-6
-4
-4
-4
-4
-2
-2
-2
-2 0
0
0
0
00
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
-2-2
-2
-2
-2
-2
-2
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
-4
-4
-4
-2-2-2
-2
-2
-2 -2
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
-4
-4
-4
-4
-2
-2
-2-2
-2
-2
-2
-2
-2
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
-10
-10
-10
-10
-10
-8-8
-8-8
-6
-6
-6-6
-4
-4
-4-4
-2
-2
-2
-2-2
0
0
0
0
2
2
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
-8-8
-6
-6
-4
-4
-4
-4
-2-2
-2
-2
-2
0
0
0
0
2
2
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
-4
-4
-4
-4
-4-4-4
-2-2
-2
-2
-6
-6
-6
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
-2
-2
-2-2-2
-2
-2
-2
-2
-2
0
0
0
0 2
-2
-2
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
SNR
(d
B)
0.5 21 4
∞630
-3-6-9
-12
0.5 21 4
∞630
-3-6-9
-12
0.5 21 4Frequency (kHz)
∞630
-3-6-9
-12 0.5 21 4Frequency (kHz)
∞630
-3-6-9
-12
0.5 21 4
∞630
-3-6-9
-12
0.5 21 4
∞630
-3-6-9
-12
0.5 21 4Frequency (kHz)
∞630
-3-6-9
-12
0.5 21 4
∞630
-3-6-9
-12
0.5 21 4Frequency (kHz)
∞630
-3-6-9
-12
0.5 21 4
∞630
-3-6-9
-12
0.5 21 4
∞630
-3-6-9
-12
0.5 21 4
∞630
-3-6-9
-12SN
R (
dB
)SN
R (
dB
)
LKJI
HGFE
DCBA
NR SummaryCoSSSNR Speech testNR
Learnings and new questions
• Large difference in NR characteristics– as a function of frequency
– as a function of SNR
• Hearing (speech) in background noise is oftendescribed as the most difficult listening situation(e.g., Kochkin, 2010)
• What are the SNRs in the those situations?– Is it appropriate for a NR system to be active at positive SNRs?
– Is it enough if a NR system is active only at negative SNRs?
• There was a lack of studies describing realistic SNRs– Pearsons et al. 1977
– Teder 1990
-10
-10-10
-8
-8
-8-8
-6
-6
-6-6
-4
-4
-4-4
-2
-2-2-2
0
0
0
00
0
0
0
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
-10
-10
-10
-10
-10
-8-8
-8-8
-6
-6
-6-6
-4
-4
-4-4
-2
-2
-2
-2-2
0
0
0
0
2
2
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
-2-2
-2
-2
-2
-2
-2
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
-10
-10
-10
-10
-10
-8-8
-8-8
-6
-6
-6-6
-4
-4
-4-4
-2
-2
-2
-2-2
0
0
0
0
2
2
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
NR SummaryCoSSSNR Speech testNR
”What are the SNRs in real life?”
NR SummaryCoSSSNR Speech testNR
Recorded data
Wagener, Hansen & Ludvigsen, JAAA 2008– 20 experienced and satisfied HA users
– Encouraged to record ”situations in daily life” for 3–4 days
– Lab evaluation of own recordings under headphones 3 w – 4 m after recordings
– Participants recorded relevant everyday situations
– Situations of varying difficulty
NR SummaryCoSSSNR Speech testNR
Manual SNR estimation method
NR SummaryCoSSSNR Speech testNR
Example: In a car
Speech +Noise
Noiseonly
NR SummaryCoSSSNR Speech testNR
Manual SNR estimation method
135 recordings
minus 7 recordingsdue to recording artifacts
minus 5 recordingsdue to dissimilarnoise characteristics
minus 22 recordingsno
minus 14 recordingsno
no
NR SummaryCoSSSNR Speech testNR
Summary results
-12 recordings due to category mismatch
In total 75 recordings included
NR SummaryCoSSSNR Speech testNR
Summary results
NR SummaryCoSSSNR Speech testNR
Results Babble
NR SummaryCoSSSNR Speech testNR
Results Car
NR SummaryCoSSSNR Speech testNR
Results Kitchen
NR SummaryCoSSSNR Speech testNR
Discussion and Conclusion
Main result
Estimated SNRs generally positive (very few situations with negative SNRs)
DiscussionBehavioural
– Avoidance of too difficult situations?
– Raised vocal effort when talking to hearing-impaired person?
Methodological
– Short recording time (3-4 days)
– Short excerpts of speech and noise used for analysis
– Estimation bias, dissimilar noise characteristics
Important result?
NR SummaryCoSSSNR Speech testNR
Short-term gain reduction HA F
Ga
in r
eduction (
NR
ON
–N
R O
FF
) (d
B)
Frequency (kHz)
SNR: +6 dB SNR: +3 dB SNR: 0 dB
SNR: -3 dB SNR: -6 dB SNR: -9 dB
-6
-6
-4
-4
-4
-4
-2
-2
-2
-2 0
0
0
0
00
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
NR SummaryCoSSSNR Speech testNR
Gain
reduction
(NR
ON
–N
R O
FF
) (d
B)
Frequency (kHz)
Short-term gain reduction
SNR = +6 dB SNR = +3 dB SNR = 0 dB
SNR = -3 dB SNR = -6 dB SNR = -9 dB
-10
-10
-10
-10
-10
-8-8
-8-8
-6
-6
-6-6
-4
-4
-4-4
-2
-2
-2
-2-2
0
0
0
0
2
2
Frequency [Hz]
SNR [dB]
250 500 1000 2000 4000 6300-12
-9
-6
-3
0
+3
+6
ISTS
HA I
NR SummaryCoSSSNR Speech testNR
Speech testing – Psychometric function
• Describes performance as a function of some parameter– Presentation level (SPL)
– Signal-to-noise ratio (SNR)
NR SummaryCoSSSNR Speech testNR
Speech testing at fixed SNRs
NR SummaryCoSSSNR Speech testNR
Floor effects
Ceiling effects
10 test persons
Adaptive speech tests
To avoid floor and ceiling effects: Adaptive speech tests
Search for instance 50% correct by – Adjust the noise level (constant speech level)
– Adjust the speech level (constant noise level)
0
50
100
SNR
Perc
enta
ge
corr
ect
(%
)
NR SummaryCoSSSNR Speech testNR
10 test persons
On potential disadvantages: Test participants will perform the test at different SNRs
This is a potential confounding factor when interpreting the results of signal-processing studies
Smeds, Leijon, Wolters,
Hammarstedt, Båsjö,
Hertzman, JASA 2014
Example
NR SummaryCoSSSNR Speech testNR
Learnings and new questions
• SNR determined for various background noises
• SNRs positive and medians in 5-10 dB range, but reported to vary in difficulty
• ”Kitchen” shared some acoustical characteristics, but also context (limited visual cues and/or moving targets)
• A purely acoustical characterization of a listening situation is not enough to fully describe the situation!
NR SummaryCoSSSNR Speech testNR
”How can listening situations be described in order to understand why they differ in difficulty despite similar acoustics – at leastsimilar SNRs?”
NR SummaryCoSSSNR Speech testNR
Aim and goal
Aim
• Investigate the acoustic environments and listening situations people encounter
Goal
• Deliver a structured framework of common sound scenarios1. Development of hearing-aid signal-processing features
2. Evaluation of such features in realistic laboratory tests
3. Demonstration of feature effects to hearing-device wearers
NR SummaryCoSSSNR Speech testNR
Method
1. General literature search: identify research expertise in the area of acoustic environments and everyday listening
2. Specific literature search: find studies for data extraction
3. Data were then extracted and categorized in a systematic way
4. A set of common sound scenarios was selected
NR SummaryCoSSSNR Speech testNR
1. General literature search
• Soundscape ecology, soundscapes, or acoustic ecology
NR SummaryCoSSSNR Speech testNR
ISO-12913-1 Sound Scape Framework
Intentions
Activities
Tasks
Experiences
Attitudes
NR SummaryCoSSSNR Speech testNR
”Making senseof sound”
Specific literature search
NR SummaryCoSSSNR Speech testNR
Only from the ”audiological field”
Questionnaire
Ecological Momentary Assessment (EMA)
Recordings
Dosimeter
Specific literature search
NR SummaryCoSSSNR Speech testNR
7 published, 3 unpublished papers
Common Sound Scenarios
Common– Happens often (everyday scenarios, rather than on extraordinary ones)
– General and widespread (recognized and shared by many people)
Sound– Main focus
– Physical and psychoacoustical characteristics
Scenario– Sum of physical environment and context
– Emphasizes a hypothetical aspect, indicates something created theoretically (with intentions and tasks as building blocks)
NR SummaryCoSSSNR Speech testNR
170 situations from the 10 studies
Work process
NR SummaryCoSSSNR Speech testNR
CoSS Framework
Intention
Scenario #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 #13 #14
Occurrence
Difficulty
Importance
Scenario
Convers
atio
nat
hom
e
Convers
atio
non
metro
Meetin
gin
an
offic
e
Car
ride
with
fam
ily
Phone
call
at
hom
e
Mobile
call
in th
estre
et
Lectu
re
At
aconcert
Watc
hin
gT
V
Lis
tenin
g to
car
radio
Vacuum
cle
anin
g
City
walk
Rela
xin
gw
itha
book
Rela
xin
gon tra
in
Non-specific
Task 2 people2 people
More than Through
deviceLive sounds
Two people having a
conversation
Several people
having a shared
conversation
Two or more people
having a shared
conversation through
a communication
device
Focused listening to
sound without being
able to control the
sound source
Focused listening to
sound while being
able to control the
sound source
Conscious or
unconscious
screening of sound of
relevance to current
activity
surroundings
Unconscious
perception of
environmental
sounds, without
relevance to current
activity
media device listening
MonitoringThrough Passive
Speech communication Focused listening
NR SummaryCoSSSNR Speech testNR
Discussion
Literature search– Few studies
– All studies from North America and North-Western Europe
– Only one study included NH informants
CoSS validity– Dillon et al (1999) COSI: 1 770 participants.
– EuroTrak (EHIMA, 2015): Importance to hear well in predefined situations.
– Dahlquist et al (2015): CoSS concept tried, 20 participants, 2-week field trial using a diary.
Current work on– Detailed prose descriptions of the sound scenarios
– Detailed technical descriptions (SPL, SNR, reverberation…)
NR SummaryCoSSSNR Speech testNR
Learnings and new questions
• People can categorize their listening situations in the CoSS categories
• The categorization seems useful when interpreting comments from hearing-aidusers’ real life
HA setting preference per CoSS category
NR SummaryCoSSSNR Speech testNR
Setting A
Setting B
Ecological MomentaryAssessment (EMA)
Learnings and new questions
• People can categorize their listening situations in the CoSS categories
• The categorization seems useful when interpreting comments from hearing-aidusers’ real life
• How can we combine context and acoustics to get– a broad picture of peoples’ auditory reality
– in order to create best hearing-aid settings?
NR SummaryCoSSSNR Speech testNR
We’ll be back!
Summary of presentation:Why is this research important?
1. Development of hearing aids
2. Evaluation of hearing aids
NR SummaryCoSSSNR Speech testNR
1. Development of hearing aids
We need a thorough understanding of peoples’ real life!
Examples:– A noise reduction system should work in commonly experienced ”noisy” situations
Frequency (kHz)
SNR: +6 dB SNR: +3 dB SNR: 0 dB
SNR: -3 dB SNR: -6 dB SNR: -9 dB
NR SummaryCoSSSNR Speech testNR
1. Development of hearing aids
Development of hearing aids: We need a thorough understanding of peoples’ real life!
Examples:– A noise reduction system should work in commonly experienced ”noisy” situations
– All development focus cannot be geared towards difficult speech-in-noise situations
30%
23%
47%
CoSS Intention Category Distribution
SpeechCommunication
Focused Listening
Non-specificSmall internal ORCA Europestudy, reported at poster session yesterday
NR SummaryCoSSSNR Speech testNR
1. Development of hearing aids
Development of hearing aids: We need a thorough understanding of peoples’ real life!
Examples:– A noise reduction system should work in commonly experienced ”noisy” situations
– All development focus cannot be geared towards difficult speech-in-noise situations
– Hearing aid programs could still be useful in order to deal with peoples’ varying intentions
Sit and relax”Isn’t this lovely?”
Leaves rustlingDistant trafficFootsteps on gravel
Children playing
Monitoring the kids”Did she hurt herself?”
NR SummaryCoSSSNR Speech testNR
Solve difficult problem”Aren’t the kids noisy?”
2. Evaluation of hearing aids
We need to evaluate hearing aids in an ecologically valid way!
Examples:– Ecological momentary assessments: Paired comparisons between two settings
NR SummaryCoSSSNR Speech testNR
2. Evaluation of hearing aids
We need to evaluate hearing aids in an ecologically valid way!
Examples:– Ecological momentary assessments: Paired comparisons between two settings.
– Laboratory tests: Need appropriate acoustical settings for the feature/system being evaluated
NR SummaryCoSSSNR Speech testNR
2. Evaluation of hearing aids
We need to evaluate hearing aids in an ecologically valid way!
Examples:– Ecological momentary assessments: Paired comparisons between two settings.
– Laboratory tests: Need appropriate acoustical settings for the feature/system being evaluated
NR SummaryCoSSSNR Speech testNR
2. Evaluation of hearing aids
We need to evaluate hearing aids in an ecologically valid way!
Examples:– Ecological momentary assessments: Paired comparisons between two settings.
– Laboratory tests: Need appropriate acoustical settings for the feature/system being evaluated
NR SummaryCoSSSNR Speech testNR
2. Evaluation of hearing aids
We need to evaluate hearing aids in an ecologically valid way!
Examples:– Ecological momentary assessments: Paired comparisons between two settings.
– Laboratory tests: Need appropriate acoustical settings for the feature/system being evaluated
NR SummaryCoSSSNR Speech testNR
2. Evaluation of hearing aids
We need to evaluate hearing aids in an ecologically valid way!
Examples:– Ecological momentary assessments: Paired comparisons between two settings
– Laboratory tests: Need appropriate acoustical settings for the feature/system being evaluated
– Laboratory tests: Need to test also other intentions than focused listening using artificial tasks
Intention
Scenario #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 #13 #14
Occurrence
Difficulty
Importance
Scenario
Convers
atio
nat
hom
e
Convers
atio
non
metro
Meetin
gin
an
offic
e
Car
ride
with
fam
ily
Phone
call
at
hom
e
Mobile
call
in th
estre
et
Lectu
re
At
aconcert
Watc
hin
gT
V
Lis
tenin
g to
car
radio
Vacuum
cle
anin
g
City
walk
Rela
xin
gw
itha
book
Rela
xin
gon tra
in
Non-specific
Task 2 people2 people
More than Through
deviceLive sounds
Two people having a
conversation
Several people
having a shared
conversation
Two or more people
having a shared
conversation through
a communication
device
Focused listening to
sound without being
able to control the
sound source
Focused listening to
sound while being
able to control the
sound source
Conscious or
unconscious
screening of sound of
relevance to current
activity
surroundings
Unconscious
perception of
environmental
sounds, without
relevance to current
activity
media device listening
MonitoringThrough Passive
Speech communication Focused listening
NR SummaryCoSSSNR Speech testNR
Aligned focus on Auditory Ecology at Widex
Auditory Ecology
Best HAs for end-users’
real-life needs
Develop-ment
EvaluationEvidence
Dispensing(HA fittingand fine-tuning)
Marke-ting