+ All Categories
Home > Documents > Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Date post: 04-Jan-2016
Category:
Upload: adrian-copeland
View: 214 times
Download: 0 times
Share this document with a friend
34
Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian
Transcript
Page 1: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Machine Learning for Computer graphics

Aaron HertzmannUniversity of Toronto

Bayesian

Page 2: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Computers are really fast

• If you can create it, you can render it

Page 3: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

How do you create it?

Digital Michaelangelo Project

Steven Schkolne

Page 4: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Two problems

• How do we get the data into the computer?

• How do we manipulate it?

Page 5: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Data fitting

Pj ;t jjmj(qt) à sj ;tjj

2

+ w1P

j ;t jjqt à qtà1jj2

+ w2P

j ;t jj:::jj2

Page 6: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Key questions

• How do you fit a model to data?– How do you choose weights and

thresholds?– How do you incorporate prior

knowledge?– How do you merge multiple

sources of information?– How do you model uncertainty?

Bayesian reasoning provides a solution

Page 7: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Talk outline

• Bayesian reasoning• Facial modeling example• Non-rigid modeling from video

Page 8: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

What is reasoning?

• How do people reason?• How should computers do it?

Page 9: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Aristotelian Logic

• If A is true, then B is true• A is true• Therefore, B is true

A: My car was stolenB: My car isn’t where I left it

Page 10: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Real-world is uncertain

Problems with pure logic• Don’t have perfect information• Don’t really know the model• Model is non-deterministic

Page 11: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Beliefs

• Let B(X) = “belief in X”,• B(¬X) = “belief in not X”

1. An ordering of beliefs exists2. B(X) = f(B(¬X))3. B(X) = g(B(X|Y),B(Y))

Page 12: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Cox axioms

R.T. Cox, “Probability, frequency, and reasonable expectation,” American J. Physics, 14(1):1-13, 1946

p(A) =P

fB=bigp(A;B)

p(A;B) = p(AjB)p(B)

p(TRUE) = 1

Page 13: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

“Probability theory is nothing more than common sense reduced to calculation.”

- Pierre-Simon Laplace, 1814

Page 14: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Bayesian vs. Frequentist

• Frequentist (“Orthodox”):– Probability = percentage of

events in infinite trials

• Medicine, biology: Frequentist• Astronomy, geology, EE,

computer vision: largely Bayesian

Page 15: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Learning in a nutshell

• Create a mathematical model• Get data• Solve for unknowns

Page 16: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Face modeling

• Blanz, Vetter, “A Morphable Model for the Synthesis of 3D Faces,” SIGGRAPH 99

Page 17: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Generative model

• Faces come from a Gaussian

p(SjSö;þ) = (2ù)djþjp 1 eà(SàSö)Tþà1(SàSö)=2

p(fSigjSö;þ) =Q

i p(SijSö;þ)

Learning

argmaxSö;þ p(Sö;þjfSig) Si

Page 18: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Bayes Rule

p(modeljdata) / p(datajmodel)p(model)

P(A;B) = p(AjB)p(B)

= p(BjA)p(A)

P(BjA) = p(AjB)p(B)=p(A)

Often: p(modeljdata) / p(datajmodel)

Page 19: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Learning a GaussianargmaxSö;þ p(Sö;þjfSig)

Sö P

i Si=Nþ

Pi(Si à Sö)(Si à Sö)T=N

= argmaxSö;þ

Qi p(SijSö;þ)

= argmaxSö;þ p(fSigjSö;þ)

= argmaxSö;þ

Qi (2ù)djþjp 1 eà 2

1(SiàSö)Tþà1(SiàSö)=2

Page 20: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Maximization trick

• Maximize <-> minimize

p(x)à lnp(x)

Page 21: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Fitting a face to an image

Generative model

p(SjSö;þ) = (2ù)djþjp 1 eà(SàSö)Tþà1(SàSö)=2

p(I jS;ú;û2) = (2ù)djþjp 1 eà jjIàRender(S;ú)jj2=2û2

I = Render(S;ú) + n

Page 22: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Fitting a face to an image

Maximize

jjI à Render(S;ú)jj2=2û2 + (S à Sö)Tþà1(S à Sö)=2

+ 2N ln2ùû2 + 2

1ln(2ù)djþj

à lnp(S;újI ;Sö;þ;û2) =

p(S;újI ;Sö;þ;û2)

minimize

Page 23: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Why does it work?p(fxigjmodel) =

Qp(xijmodel)

p(x|model)

s:t:R

p(xjmodel) = 1

Page 24: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

General features

• Models uncertainty• Applies to any generative model• Merge multiple sources of

information• Learn all the parameters

Page 25: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Caveats

• Still need to understand the model

• Not necessarily tractable• Potentially more involved than

ad hoc methods

Page 26: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Applications in graphics

• Shape and motion capture• Learning styles and generating

new data

Page 27: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Learning Non-Rigid 3D Shape from 2D Motion

Joint work with Lorenzo Torresani and Chris

Bregler (Stanford, NYU)

Page 28: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Camera geometry

Orthographic projection

p ;̀t = RtS ;̀t + Tt

p ;̀tS ;̀t

Page 29: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Non-rigid reconstruction

Input: 2D point tracksOutput: 3D nonrigid motion

pt = RtSt + Tt

Totally ambiguous!

Page 30: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Shape reconstruction

Least-squares version

pt = RtSt + Tt

minimizeP

t jjpt à RtSt à Ttjj2 + wP

t jjSt à Söjj2

Page 31: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Bayesian formulation

p(R;S;T;Sö:þjP)/ p(PjR;S;T)p(SjSö:þ)

maximize

p(StjSö;þ) = (2ù)djþjp 1

eà(StàSö)Tþà1(StàSö)=2

p ;̀t = RtS ;̀t + Tt + n n ø N(0;û2)

Page 32: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

How does it work?p(R;S;T;Sö:þjP)/ p(PjR;S;T)p(SjSö:þ)

àP

t lnp(PtjRt;St;Tt) àP

t lnp(StjSö;þ)

=P

t jjpt à RtSt à Ttjj2=2û2 + 2N lnû2

+P

t 21(St à Sö)Tþà1(St à Sö) + 2

T lnjþj

Maximize

Minimize

(actual system is slightly more sophisticated)

Page 33: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Results

Least-squares, no reg.

Least-squares, reg.

Gaussian

LDS

View 2

Input

Page 34: Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Conclusions

• Bayesian methods provide unified framework

• Build a model, and reason about it

• The future of data-driven graphics


Recommended