1

Perception of pitchAUDL4007: 26 Feb 2015. A. Faulkner.

See Plack CJ “The Sense of Hearing” Lawrence Erlbaum,2005 Chapter 7

or Moore, BCJ “Introduction to the Psychology ofHearing, Chapter 5”.

2

Definitions

Perception: Pitch is the perceptual property of sound thatconveys melody

Acoustics: Pitch is closely related to frequency andperiodicity

Pitch is a perceptual property of periodic and approximatelyperiodic sounds – these have spectra that contain harmonics of acommon fundamental frequency.

Pitch should be distinguished from “timbre”, which is a perceptualquality relating to the sharpness of dullness of a sound. Timbre ismainly related to spectral shape

The pitch of a sound is defined, for the purposes of measurement,as being equivalent to the frequency of a simple sine wave that hasthe same pitch as the sound. Hence pitch is expressed in Hz.

3

Singing voice {

4

Why is pitch important?

• In speech

– Pitch variations signal differences between child, adultmale and adult female speakers.

– Pitch variation conveys intonation, which indicateslexical stress and aspects of syntax.

• e.g. it’s raining? “checking” question usually shows final pitchrise

• No I mean the BLUE shirt! – emphasis on BLUE would lead topitch rise

– In tone languages, pitch movement is lexicallycontrastive

5

hemp

horse

scold

mother

6

Importance of pitch: 2

• Music

• Separating sources of sound

– Pitch is rather like a carrier frequency that we can tune in to

• Much studied in examining roles of spectral and temporalcoding and processing in hearing

7

Auditory coding of frequency and pitch

Information in spectral/place and time domains

Theories of pitch perception have been largely concernedwith contrasting the contributions of spectral and temporalcues to the perception of pitch.

– Place representation - pitch is related to place of basilarmembrane vibration

– Temporal representation - neural firing patternpreserves periodicity of the signal

8

Place and time coding of sine-wavefrequency

Place of maximum response varieswith frequency

Pitch Discrimination for sinewaves

Practiced listenerscan hear differencesof less than 1 Hz for a200 Hz sinusoid(precision better than0.5%)

At 1000 Hz,differences of 2 Hzcan be detected(precision of about0.2%)

NB Scales here chosento fit data to straightline: square root(F) andthreshold frequencydifference on a log scale

200 Hz – differences of 1, 2,4, 8, 16 Hz1k Hz, differences of 2, 4, 8,16, 32, 64 Hz.

10

Can we account for pure tone discriminationon the basis of place cues?

Deriving excitation patterns for a 1 kHzsinusoid from filter frequency responses

Note shallower slope to lower frequencies (left) forfrequency responses

300 Hz frequency 1900 Hz

FIlter responses with centre frequenciesrunning from 1400 – 600 Hz

1400 Hz

Excitationlevel for 1kHz toneby centrefrequencyof filter

Acoustic frequency (Hz)

Filter centre frequency (Hz) = place on BM

Shift of excitation pattern with change of frequency

Overall patternof excitationover filter centrefrequency

Response toone frequencyin a series offilters

Acoustic frequency (Hz)

Filter centre frequency (Hz) = place on BM

14

Excitation pattern coding offrequency difference

Intensitydiscriminationthresholds are about1 dB.

At 1000 Hz forexcitation levels todiffer by 1dBrequires a frequencydifference of about10 Hz – yet we canhere a frequencydifference of 2 Hz.

15

What other cues are there?

A just detectable pitch change at 3 kHz and below leads to achange in excitation level that is too small to be detected.

Therefore - acuity for pitch differences for low frequencysinusoids cannot be explained by place cues.

16

Neural temporal coding

Interval histogram from recordings of auditory nerve responses to 1100 Hz sinewave. The common intervals are at 1/1100 seconds, 2/1100 seconds, etc.

17

Synchrony of nerve firing times to sine-wave period:very precise up to about 1.5 kHz – then declines andis lost at 5 kHz and above – so timing cues to pitch

decline in accuracy above 1.5 kHz

18

19

What about effects of duration?

• If pitch discrimination is based on timeintervals between nerve firings then as moreintervals occur, discrimination is likely tobe more accurate in a way that depends onthe statistics of timing of nerve firing,

20

Effects of duration on spectrum

• But duration also affects spectrum, andhence place coding - width of excitationpattern grows with inverse of duration

21

Effects of duration on sine wave spectrum –

spectrum spreads at shorter durations whichlimits place coding of pitch

Sequence from 2 to 128 cycles

Effects of signal duration: place vs. temporal codingof sine wave frequency

Data from Moore (1972, 1973)

Above 4 kHz there are only placecues – duration has a relatively smalleffect which can be explained by thespectral spread arising for shortertones.

Below ~ 4 kHz, pitch discriminationfor longer signals is too fine to beexplained by place (shifts inexcitation pattern)

Effects of signal duration (differentcurves) are larger at low frequencies.They cannot be explained by spreadof excitation pattern but can beexplained by statistics of temporalcoding which depends on number ofinter-spike intervals.

23

Relative discriminability of pitch

Typically pitchdiscrimination isexpressed relativeto frequency.Expressed this waythe relativeDifference Limenfor Frequency(DLF) is smallestat 2 kHz.

24

Coding pure tone frequency

• Only by place of excitation above 4 kHz

• Place information not good enough at lowerfrequencies – coding is dominated bytemporal coding below ~ 1.5 kHz

• Between 1.5 and 4 kHz both types of cueare available.

25

Pitch of complex sounds

• A complex harmonic sound such as a pulse trainhas a pitch that is equivalent to that of a sinusoidat the fundamental frequency (F0) of the pulsesignal.

• This information is present in the acoustic signalboth in the spectrum, as the frequency of thecomponent at F0, and in the time domain, as theperiod of the pulse train.

26

27

Ohm’s other law:

“Every motion of the air, then, which corresponds to acomposite mass of musical tones, is, according to Ohm’sLaw, capable of being analysed into a sum of simplevibrations, and to each such simple vibration corresponds asimple tone, sensible to the ear, and having a pitchdetermined by the periodic time of the correspondingmotion of the air.”

(Helmholtz, 1885; “On the Sensations of Tone”

28

Auditory filter bandwidth increases with frequency (whileharmonics are evenly spaced). For F0 of 200 Hz, bandwidth

exceeds harmonic spacing above about 1.6 kHz

29

ANexcitation

pattern

Cochlear frequency selectivity and resolution of harmonics

Cochlear Place

CochlearFilter Bank

CF

ResolvedHarmonics

UnresolvedHarmonics

Harmonic Number (F/F0)

F0

x

Missing-F0harmoniccomplex

tone

30

Excitation patterns: complex sounds

Lower harmonics are clearly resolved – For 200 Hz F0, above1.6 kHz filter bandwidth is wider than 200 Hz spacing betweenharmonics and these higher harmonics are not resolved.

Similar limits apply at other F0s

31

Classical Place account of pitch

• Pitch of a complex sound determined by positionof peak in excitation pattern due to basilarmembrane response to fundamental frequency (F0)component

32

The missing fundamental• Schouten (1938, 1940)

made a crucial test of theplace theory that is basedon Ohm’s Law

• He presented a pulsesignal, with a completeharmonic series. A placeaccount would claim thatthe pitch is due to thelowest frequencycomponent, at thefundamental frequency.

• This signal is compared toa signal modified toremove the fundamentalfrequency component.According to place theory,the pitch should change

33

For most listeners, pitch is unaffected by deletion ofharmonic at fundamental frequency

Schouten called this “residue pitch” – attributing thelow pitch percept to the periodicity shown in theauditory nerve response to the unresolved higherharmonics

Audio demonstration from “Audio Demonstrations on Compact Disc (ASA1989).

The first sound is a 200 Hz harmonic complex tone comprising the 1st 10harmonics. Succeeding sounds have the 1st, 1st and 2nd, 1st thru 3rd, and then 1st

thru 4th harmonics deleted.

34

Higher harmonics are closelyspaced relative to filterbandwidths and are notresolved. The filter outputshows the fundamentalperiodicity of the pulse train

Lower harmonics arecompletely resolved (1st 5 to 8harmonics depending on F0)

Auditory frequency analysisof a pulse train

35

Role of auditory non-linearity?

• Additional frequency components are introducedwhen a signal is passed through a non-linearsystem – for harmonic complex tones this couldinclude a distortion component at F0.

• Can a component introduced at the fundamentalfrequency explain “The case of the missingfundamental”?

36

Is distortion product responsible for lowpitch?

• Patterson (1976) Low frequency noise will maska distortion component at F0 – (e.g. a differencetone arising from two adjacent harmonics)

– but LF noise does not mask the low pitch at F0

– therefore the low pitch is not due to distortion

Audio demo – A simple melody is heard played by a series of sine wavesand complex tones comprising 3 higher harmonics with the same F0 as thesine wave. Both the sine and complex tones sound the same melody. Then alow pass noise is added – this masks the sine wave and would mask anyauditory distortion product at F0. The low pitch is still heard from thecomplex tones.

37

Contributions of resolved and unresolved harmonics

The pitch of the residue suggests that higher unresolved harmonics areimportant in determining the pitch of complex tones. Both Ritsma andPlomp in 1967 published studies that challenged this.

Plomp used stimuli in which the higher and lower harmonics were shiftedin frequency in opposite directions. E.g., Harmonics 1 to 4 were shifteddown by 10% and harmonics 5 upwards were shifted up by 10%.

38

Contributions of resolved and unresolved harmonics

Generally, and especially in thespeech F0 range, it isharmonics 4 to 8 that dominatepitch

At very high F0 – above 1.5kHz, the fundamental frequencycomponent is dominant.

Contributions of unresolvedhigh harmonics never dominateover contributions of resolvedharmonics.

39

Resolved harmonics produce higher precisionof pitch than unresolved harmonics

(Bernstein &Oxenham, 2003)

Harmoniccomplex tone, 12

successiveharmonics

Resolved Unresolved

Better

Worse

ALSOPitch discrimination forcomplex tones generallybetter than for the sine

wave at F0 (Henning andGrosberg, 1968)

40

The filter output shows thefundamental periodicity – weakcue to pitch

Lower harmonics arecompletely resolved – theirfrequencies coded in time (atdifferent places) are primarycues to pitch –DOMINANTAND MOST PRECISE

Harmonic at fundamentalfrequency not a necessary cuefor pitch

Where are cues to pitch?

Pitch cues across frequency inspeech

1000 Hz

2000 Hz

3000 Hz

4000 Hz

42

Primary cues for pitch of complexsounds

• Pitch is mostly effectively determined by temporally-encoded representations of the frequencies of resolvedharmonics (temporal code needed to explain the precisionof pitch discrimination)

• The temporal encoding of F0 from the unresolved higherharmonics is not a primary cue

• Nor is the harmonic component at F0 except when F0 > 1.5kHz.

43

Pitch without spectral information• White noise that is amplitude

modulated at rates up to 1000 Hzhas a weak pitch (Burns andViemeister, 1976). The spectrumof the noise is flat, and onlytemporal cues to pitch are present

• E.g, below shows white noise(lower trace) amplitude modulatedby half-wave rectified sine wave

Purely temporal pitches, although weak, canconvey melody information for rates up to 300 or500 Hz - but very weak above 200 Hz.

Monaural temporal pitch is perceived from thetemporal nerve firing pattern, which will beaffected by amplitude modulation.

Also DICHOTIC temporal pitches –where a pitch is heard that changeswith inter-aural phase.

Unmodulatednoise Noise amplitude modulated by sine wave

gliding from 40 up to 100 Hz (left) anddown from 100 to 40 Hz (right)

100 Hz am noise

44

Current theories of pitch perception• Pitch perception is based on the pattern of

information over a range of frequencies. Themajor contributing information is the

frequencies of the dominant resolvedharmonics.

• This information is conveyed in the temporalfiring pattern of the auditory nerve acrossfrequency channels.

• Pattern processing identifies intervalsbetween nerve firing that are common acrossfrequency channels. For a series of resolvedharmonics, nerve firings show a related seriesof time intervals

• Periodicity information from higher

frequency unresolved harmonics or fromthe modulation envelope of noise is anothersource of input to this pattern processing, butis a relatively weak cue.

Figure from Moore and Glasberg (1986)

45

Auditory nerve responses to pulse train: nerves responding toresolved harmonics show periodicity of each harmonic

46

Summary: Simple signals

• While pitch is broadly correlated withperiod, human pitch processing is complex

• Sine waves up to a few kHz - pitch istemporally coded – place coding is too poorto account for performance

• Sine waves above 4 kHz, only place cuesare present to code sine wave frequency

47

Summary: Complex signals• The period indicated by temporal cues alone from

unresolved high harmonics in a single auditory filter cansignal pitch at F0.

– And a weak pitch can be heard from purely temporal cues withamplitude modulated noise

• However, pitch of complex tones is dominated by resolvedharmonics (range 4 to 8 for F0 in speech range). Here pitchprocessing depends on pattern extraction operating on timeintervals between nerve firings

48

How might impaired hearing affectpitch perception?

• Wider auditory filters due to OHC damage– Fewer harmonics resolved

• Impaired temporal coding– Would limit phase-locking and hence temporal

coding of frequency

Today’s lab

Measuring fundamental frequency fromwhistles and speech.

Aimed only at Audiology students

49