8/6/2019 Feature Analysis

1/19

Musical Featuresfrom Spectral Analysis

Xavier Serra

Music Technology Group

Universitat Pompeu Fabra, Barcelona

1 Xavier Serra

8/6/2019 Feature Analysis

2/19

Xavier Serra2

Index

Sound and Music features Feature correlations Feature detection and extraction

Segmentation Segment features Instrument models Musical features

8/6/2019 Feature Analysis

3/19

Xavier Serra3

Sound and music features

Basic Sines+Residual parameters Instantaneous frequency and amplitude of partials Instantaneous spectrum of residual

Instantaneous features Fundamental frequency Amplitude and spectral shape of sinusoidal component Amplitude and spectral shape of residual Degree of harmonicity Noisiness

Spectral tilt Spectral centroid

Segment features

8/6/2019 Feature Analysis

4/19

Xavier Serra4

Loud piano note

Soft piano note

Fundamental frequencyFundamental frequency

Amplitude

Spectral shape

Spectral shape

Amplitude

Feature correlations

8/6/2019 Feature Analysis

5/19

8/6/2019 Feature Analysis

6/19

Xavier Serra6

Amplitude of sinusoids:

Amplitude of residual:

Fundamental frequency:

8/6/2019 Feature Analysis

7/19

Xavier Serra7

Spectral shape of residual:Spectral shape of sinusoids:

f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11

2f0 2f1 2f2 2f3 2f4 2f5 2f6 2f7 2f8 2f9 2f10 2f11

2f0 2f1 2f2 2f3 2f4 2f5 2f6 2f7 2f8 2f9 2f10 2f11

Octave transposition

8/6/2019 Feature Analysis

8/19

Xavier Serra8

Harmonic distortion:

Noisiness:

8/6/2019 Feature Analysis

9/19

Xavier Serra9

X. Serra

inharmonicity in differentpiano strings (Fletcher 91)

inharmonicity in differentpianos (Fletcher 91)

8/6/2019 Feature Analysis

10/19

Xavier Serra10

Centroid:

Spectral tilt:

where:

8/6/2019 Feature Analysis

11/19

Xavier Serra11

Spectral tilt as a function of amplitude(Benet and Rodet, 89)

8/6/2019 Feature Analysis

12/19

Xavier Serra12

unconstrainedconstrained

Segmentation

8/6/2019 Feature Analysis

13/19

Xavier Serra13

AttackStart

AttackEnd

DecayEnd

DecayStart

8/6/2019 Feature Analysis

14/19

Xavier Serra14

X. Serra

frequency articulation

amplitude articulation

8/6/2019 Feature Analysis

15/19

Xavier Serra15

musical script

constrained segmentation

8/6/2019 Feature Analysis

16/19

Xavier Serra16

Segment features

Mean and variance of frame attributes Amplitude of sinusoidal and residual amplitudes Fundamental frequency Spectral shape of sinusoidal and residual component

Vibrato

8/6/2019 Feature Analysis

17/19

Xavier Serra17

X. Serra

steady state withoutvibrato

steady state withvibrato

8/6/2019 Feature Analysis

18/19

Xavier Serra18

X. Serra

steady state withvibrato

steady state with extractedvibrato

8/6/2019 Feature Analysis

19/19

Xavier Serra19

Instrument features

Spectral Shape Models (formants, average shape, ...) Phase Models (constant, formant-based, ...) Frequency Models (harmonic model, piano model) Vibrato Models Articulation Models (frequency, amplitude functions) Residual Models Brightness-Loudness model (amplitude versus spectral

tilt)