stand B-7

Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music GenerationHao-Wen Dong and Yi-Hsuan YangResearch Center of IT Innovation, Academia Sinica

Gz

noise

True/Fake

Synth Pad

x1 or x1

x2 or x2

xM or xM

Drums

Piano

…

~

~

~

D

Generator DiscriminatorDrums

Piano

Guitar

Bass

Ensemble

Reed

Synth Lead

Synth Pad

Target outputs - Multi-track piano-rolls Convolutional Generative Adversarial Networks

CNNs + GANsmulti-trackpolyphonic

stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang

Drums Piano Guitar Bass Strings

stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang

bass linechords

drum patterns

Drums Piano Guitar Bass Strings

stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang

Gz

noise

D True/Fake

Synth Pad

x1 or x1

x2 or x2

xM or xM

Drums

Piano

…

~

~

~

Generator Discriminator

real-valued

stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang

Generator Refiner Discriminator

Gz

noise

x1

x2

xM

…

R

R

R

1

2

M

…

ˆ

ˆ

ˆ

D True/Fake

Synth Pad

x1 or x1

x2 or x2

xM or xM

Drums

Piano

…~

~

~

real-valued binary-valued

stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang

Generator Refiner Discriminator

Gz

noise

x1

x2

xM

…

R

R

R

1

2

M

…

ˆ

ˆ

ˆ

D True/Fake

Synth Pad

x1 or x1

x2 or x2

xM or xM

Drums

Piano

…

~

~

~

real-valued binary-valued

Generator’s outputs Real data

MuseGAN real-valued binary-valued

BinaryMuseGAN binary-valued binary-valued

stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang

Generator Refiner Discriminator

Gz

noise

x1

x2

xM

…

R

R

R

1

2

M

…

ˆ

ˆ

ˆ

D True/Fake

Synth Pad

x1 or x1

x2 or x2

xM or xM

Drums

Piano

…

~

~

~

Deterministic Binary Neurons (DBNs)𝐷𝐷𝐷𝐷𝐷𝐷 𝑥𝑥 = 𝑢𝑢 𝜎𝜎 𝑥𝑥 − 0.5

Stochastic Binary Neurons (SBNs)𝑆𝑆𝐷𝐷𝐷𝐷 𝑥𝑥 = 𝑢𝑢 𝜎𝜎 𝑥𝑥 − 𝑣𝑣 , 𝑣𝑣~𝑈𝑈 0, 1

stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang

shared/privatedesign

shared/private design

Generator Refiner Discriminator

onset/offset stream

chroma stream

Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang stand B-7

MuseGAN’s output

(real-valued)

hardthresholding

Bernoullisampling

many overly-fragmented notes

Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang stand B-7

BinaryMuseGAN(+DBNs)MuseGAN’s output

(real-valued)

hardthresholding

Bernoullisampling

many overly-fragmented notes

less overly-fragmented notes

stand B-7Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation Hao-Wen Dong and Yi-Hsuan Yang

MuseGAN BinaryMuseGAN

Come to learn more and listen to the demos!

(different training strategies)