2018 FallCTP431: Music and Audio Computing

Sound Synthesis (Part 1)

Graduate School of Culture Technology, KAISTJuhan Nam

Outlines

• Signal model (analog / digital) – Part 1- Additive Synthesis- Subtractive Synthesis- Modulation Synthesis- Distortion Synthesis

• Sample model (digital) – Part 2- Sampling Synthesis- Granular Synthesis- Concatenative Synthesis

• Physical model (digital) – Part 2- Digital Waveguide Model

Signal Model

• Modeling the patterns of musical tones using elementary waveforms- Time domain: ADSR- Frequency domain: spectrum

• Types of signal models- Additive synthesis: a set of sine waveforms - Subtractive synthesis: sawtooth, square waveforms + filters- Frequency modulation synthesis: a pair of sine waveforms - Distortion synthesis: sine waveforms + nonlinear units

• These techniques date back to the analog age

Additive Synthesis

• Synthesize sounds by adding multiple sine oscillators- Also called Fourier synthesis

OSC

OSC

OSC

.

.

.

Amp (Env)

Amp (Env)

Amp (Env)

.

.

.

+

Telharmonium

• Additive synthesizer using electro-magnetic “tone wheels” (Cahill, 1897)- Transmitted through telephone lines - Subscription only but the the business failed

Tone wheel

Hammond Organ

• Drawbars- Control the levels of individual tonewheels

Theremin

• A sinusoidal tone generator- Two antennas are remotely controlled to adjust pitch and volume

Theremin ( by Léon Theremin, 1928)

Theremin (Clara Rockmore)https://www.youtube.com/watch?v=pSzTPGlNa5U

Sound Examples

• Web Audio Demo- http://femurdesign.com/theremin/- http://www.venlabsla.com/x/additive/additive.html- http://codepen.io/anon/pen/jPGJMK

• Examples (instruments)- Kurzweil K150- https://soundcloud.com/rosst/sets/kurzweil-k150-fs-additive

- Kawai K5, K5000

Subtractive Synthesis

• Synthesize sounds by filtering wide-band oscillators- Source-Filter model

5 10 15 20−60

−50

−40

−30

−20

−10

0

10

20

Frequency (kHz)

Mag

nitu

de (d

B)

5 10 15 20−60

−50

−40

−30

−20

−10

0

10

20

Frequency (kHz)

Mag

nitu

de (d

B)

0 0.5 1 1.5 2 2.5x 104

−60

−50

−40

−30

−20

−10

0

10

20

Frequency (kHz)

Mag

nitu

de (d

B)

Source Filter Filtered Source

FilterOscillators Amp

50 52 54 56 58 60−0.4

−0.2

0

0.2

0.4

time−milliseconds

amplitude

Moog Synthesizers

MiniMoog (1970)

Moog Synthesizers

• Architecture

Envelope

Envelope LFO

Wheels Slides PedalPhysical Control

FilterOscillators Amp

Keyboard

Audio Path

SoftControl

Parameter = offset + depth*control (e.g. filter cut-off

frequency)(static value) (dynamic value)

Parameter Parameter Parameter

“Switched-On-Bach” by Wendy Carlos(1968)

Oscillators

• Classic waveforms

• Modulation- Pulse width modulation- Hard-sync- More rich harmonics

0 50 100 150 200−2

0

2

5 10 15 20−60−40−20

020

Frequency (kHz)

Mag

nitu

de (d

B)

−6dB/oct

0 50 100 150 200−1

0

1

5 10 15 20−60−40−20

020

Frequency (kHz)M

agni

tude

(dB)

−6dB/oct

0 50 100 150 200−2

0

2

5 10 15 20−60−40−20

020

Frequency (kHz)

Mag

nitu

de (d

B)

−12dB/oct

Sawtooth TriangularSquare

Amp Envelop Generator

• Amplitude envelope generation- ADSR curve: attack, decay, sustain and release- Each state has a pair of time and target level

Note On Note Off

Attack Decay Sustain

Release

Amplitude(dB)

Examples

• Web Audio Demos- http://www.google.com/doodles/robert-moogs-78th-birthday- http://webaudiodemos.appspot.com/midi-synth/index.html- http://aikelab.net/websynth/- http://nicroto.github.io/viktor/

• Example Sounds- SuperSaw- Leads- Pad- MoogBass- 8-Bit sounds: https://www.youtube.com/watch?v=tf0-Rrm9dI0- TR-808: https://www.youtube.com/watch?v=YeZZk2czG1c

Modulation Synthesis

• Modulation is originally from communication theory- Carrier: channel signal, e.g., radio or TV channel - Modulator: information signal, e.g., voice, video

• Types of modulation synthesis- Amplitude modulation (or ring modulation)- Frequency modulation

• Decreasing the frequency of carrier to hearing range can be used to synthesize sound - Generate new sinusoidal components- Modulation is non-linear processing

Ring Modulation / Amplitude Modulation

• Change the amplitude of one source with another source- Slow change: tremolo- Fast change: generate a new tone

OSC

OSC

(1+ am (t))Ac cos(2π fct)am (t)Ac cos(2π fct)

Amplitude Modulation

xCarrier

Modulator

OSC

OSC

xCarrier

Modulator

+

Ring Modulation

Ring Modulation / Amplitude Modulation

• Frequency domain - Expressed in terms of its sideband frequencies- The sum and difference of the two frequencies are obtained according to

trigonometric identity- If the modulator is a non-sinusoidal tone, a mirrored-spectrum with regard

to the carrier frequency is obtained

fc+fmfcfc-fm

am (t) = Am sin(2π fmt))

carriersidebandsideband

Examples

• Tone generation- SawtoothOsc x SineOsc- https://www.youtube.com/watch?v=yw7_WQmrzuk

• Ring modulation is often used as an audio effect- http://webaudio.prototyping.bbc.co.uk/ring-modulator/

Frequency Modulation

• Change the frequency of one source with another source- Slow change: vibrato- Fast change: generate a new (and rich) tone- Invented by John Chowning in 1973 à Yamaha DX7

Ac cos(2π fct +β sin(2π fmt))

β =Amfm

Index of modulationOSC

OSC

Carrier

Modulator

frequency

Frequency Modulation

• Frequency Domain- Expressed in terms of its sideband frequencies- Their amplitudes are determined by the Bessel function- The sidebands below 0 Hz or above the Nyquist frequency are folded

y(t) = Ac Jk (k=−∞

k=−∞

∑ β)cos(2π ( fc + kfm )t)

fc+fmfc fc+2fm fc+3fmfc-fmfc-2fmfc-3fm

carriersideband1sideband1

sideband2sideband2sideband3sideband3

Frequency Modulation

• Bessel Function

Jk (β) =(−1)n (β

2)k+2n

n!(n+ k)!n=0

∞

∑

0 50 100 150 200 250 300 350−0.5

0

0.5

1

beta

J_(k

)

CarrierSideband 1Sideband 2Sideband 3Sideband 4

The Effect of Modulation Index

0 500 1000 1500 2000−1

0

1

Time (Sample)

Ampl

itude

0.2 0.4 0.6 0.8 1−60−40−20

020

Frequency (kHz)

Mag

nitu

de (d

B) Beta = 0

0 500 1000 1500 2000−1

0

1

Time (Sample)

Ampl

itude

0.2 0.4 0.6 0.8 1−60−40−20

020

Frequency (kHz)

Mag

nitu

de (d

B) Beta = 1

0 500 1000 1500 2000−1

0

1

Time (Sample)

Ampl

itude

0.2 0.4 0.6 0.8 1−60−40−20

020

Frequency (kHz)

Mag

nitu

de (d

B) Beta = 10

0 500 1000 1500 2000−1

0

1

Time (Sample)Am

plitu

de

0.2 0.4 0.6 0.8 1−60−40−20

020

Frequency (kHz)

Mag

nitu

de (d

B) Beta = 20

fc = 500, fm = 50

FM Synthesizer

Yamaha DX7 (1983)

Examples

• Web Audio Demo- http://www.taktech.org/takm/WebFMSynth/

• Sound Examples- Bell- Wood - Brass- Electric Piano- Vibraphone

Non-linear Synthesis (wave-shaping)

• Generate a rich sound spectrum by distorting sine waveforms using non-linear transfer functions

• Also called “distortion synthesis”

0 50 100 150 200−1

0

1

Time (Sample)

Am

plitu

de

5 10 15 20−60−40−20

020

Frequency (kHz)

Mag

nitu

de (d

B)

0 50 100 150 200−1

0

1

Time (Sample)

Am

plitu

de

5 10 15 20−60−40−20

020

Frequency (kHz)

Mag

nitu

de (d

B)

−1 −0.5 0 0.5 1−1

−0.5

0

0.5

1

Time (Sample)

Ampl

itude

x’=gx: g correspond to the “gain” of the distortion

Distortion Transfer Function

• Examples of transfer function: y = f(x)- y = 1.5x’ – 0.5x’3

- y = x’/(1+|x’|)- y = sin(x’)- Chebyshev polynomial: Tk+1(x) = 2xTk(x)-Tk-1(x)

T0(x)=1, T1(x)=x, T2(x)=2x2-1, T2(x)=4x3-3x