Monte Carlo Methods 1

Monte Carlo Methods

T-61.182 Special Course In Information Science II

Tomas Ukkonen

tomas.ukkonen@iki.fi

Monte Carlo Methods 2

Problem

1. generate samples from given probability distribution P(x)

2. estimate

The second problem can be solved by using random samples from P(x)

xdxPxxE )()()]([

r

rxR

)(1

Monte Carlo Methods 3

Why sampling is hard?

densities may be unscaled:hard to know how probable a certain point is when the rest of function is unknown

curse of dimensionality

Monte Carlo Methods 4

Brute force method

why don’t just calculate expected value directly

problem grows exponentiallyas the function of dimension d

number states to check grow exponentially

i

ipZ *

Zpp ii /*

Monte Carlo Methods 5

Brute force method, cont.

going through most of the cases is likely to be unnecessary

high-dimensional, low entropy densities are often concentrated to small regions

Monte Carlo Methods 6

Uniform sampling

for small dimensional problems

Just sample uniformly and weight with

Required number of samples for reliable estimators still grows exponentially

x )(xP

HNR 2min

Monte Carlo Methods 7

Importance sampling

idea: approximate complicated distribution with simpler one

only works when correct shape of distribution is known

doesn’t scale to high dimensionseven when approximation is almostright

)(

)(*

*

r

rr xQ

xPw

r r

r rr

w

xw )(

Monte Carlo Methods 8

Rejection sampling

Alternative approximation based sampling method

sample uniformly from(x,u) = (x,cQ(x)) and reject samples where u > P(x)

doesn’t scale to high dimensions

Monte Carlo Methods 9

The Metropolis-Hastings method

The previous approaches didn’t scale to high dimensions

In Metropolis algorithm sampling distribution depends on samples sampled so far

Monte Carlo Methods 10

The Metropolis-Hastings, cont.

A new state is drawn from distributionand accepted with a certain probability which guarantees convergence to the target density

The method doesn’t depend on dimensionality of a problem, but samples are correlated and a random walk based moving is slow

);'( txxQ

Monte Carlo Methods 11

Gibbs sampling

a special case of the metropolis method where only single dimension is updated per iteration

useful when only conditional densities are known

one dimensional distributions are easier to work with

)....|( 111 Niii xxxxxp

Monte Carlo Methods 12

Gibbs sampling, cont.

Monte Carlo Methods 13

Slice sampling

a newer method which is combination of rejection, Gibbs and Metropolis sampling

still a random walk method but with a self tuning step length

Monte Carlo Methods 14

Slice sampling, cont.

faster integer based algorithm has been also developed

Monte Carlo Methods 15

Slice sampling, cont.

Monte Carlo Methods 16

Slice sampling, cont.

Monte Carlo Methods 17

Practical issues

Hard to know for certain when Monte Carlo simulation has converged

Caculating normalization constant

allocation of computational resources:one long simulation or more shorter ones?

)(1

)( * xPZ

xP

Monte Carlo Methods 18

Practical issues II, cont.

Big Models Metropolis method & Gibbs sampling- update variables in batches

How many samples- how much accuracy is needed?- typically 10-1000 samples is enough

Monte Carlo Methods 19

Exercises & References

exercise 29.4. exercise NN.N.

David J.C. Mackay: Information Theory, Inference, and Learning Algorithms, 2003