Post on 31-Dec-2015
description
transcript
Markov Chain Monte Carlo Methods
By Marc Sobel
Exploring Monte Carlo
A biulding in the country of Monte Carlo:
Monte Carlo Goals 1. Simulate observations from a density f(x)
(having the property that it integrates to 1). We would like to visit sites in the proportion given by the density.
2. Estimate expectations of the form:
by
With the x’s simulated under the density p.
( ) ( ) ( ) I g g x p x dx
1
1( )
n
iig X
n
Monte Carlo Theorem
Monte Carlo Theorem The accuracy of the sampling sum depends only
on the variance of ‘g’ and not on the dimensionality of the space. The Variance takes the form (where sigma-squared depends on g).
So what’s the problem. Suppose p(x) is known up to a constant of proportionality and
We could try to approximate the expectation by uniform sampling of the X’s over their range.
2 / n
1000 x
1
1
1( ) ( )
ˆ( )1
( )
n
i ii
n
ii
g X p XnI g
p Xn
Monte Carlo Example But in high dimensional spaces, this
frequently doesn’t work well, since we have to pick an enormous number of uniform pts to get good accuracy: Suppose you wanted to map out and study the depths in a lake. Assuming the depth (viewed as a function of position) is quite complicated, uniform sampling will miss the important contours. Basically, the set of possible depth functions (i.e., the state space) is very large, so uniform sampling will tend to miss important parts of it.
Importance Sampling is (sometimes) the answer!! Design a distribution h, from which we can sample, which
captures the contours of ‘p’ properly. The estimate is then,
The quantities i=1,…,n are the
non-normalized weights attached to calculating the given expectation.
If they vary too little, the expectation tends to be inaccurate. If they vary too much, the sum may not converge.
1
1
( )1( )
( )ˆ( )( )1
( )
ni
ii i
ni
i i
g Xp X
n h XI g
p X
n h X
( )
( )i
ii
p Xw
h X
Why Importance Sampling works
Why does importance sampling work?
1
1
( ) ( )1( ) ( )( )
( )( )ˆ( )( )1 ( )
( )( ) ( )
= g(x)p(x)dx
ni
ii i
ni
i i
g X g xp x h x dxp X
h xn h XI g
p X p xh x dx
n h X h x
Project
Could we assume h is a Gaussian? But suppose the underlying distribution is multimodal. Show that things go wrong when one variance is more than twice the others. Hint: Calculate the variance of the weights when the numerator p(x) is a mixture of gaussians and the denominator h is one gaussian from the mixture.
Importance Sampling (using independent variables) does not always work in high dimensions
Suppose x is the uniform distribution in a 1000 dimensional unit sphere. We approximate it by putting together 1000 independent gaussians each having variance σ2. The mean of the importance sampling distribution of r2=Σx2 is 1000σ2 with sd √2000σ2 . So, the typical weight will be:
Note that this means that weights with +√2000 will dominate all the rest.
1000( ) 1000 20002 exp
( ) 2
P x
Q x
An example of importance sampling
Statisticians make frequent use of the t distribution with density in settings with large errors.
‘ν’ is the degrees of freedom, ‘θ’ is the mean, and ‘σ’ is the standard deviation.
1
2
, , 2
2
112( )
( )12
f xx
Importance Sampling Example
Can we use the normal to generate t distributions. Assume θ=0; σ=1;
ν=4 degrees of freedom. Create standard normal rv’s z1,…,zn. Create (normalized weights)
Give weights wi to zi (i=1,..,n). The resulting sample has a t distribution.
4( ); (i=1,...,n)
( )i
ii
t zw
z
Matlab Code for importance sampling
zz= normrand(0,1,500,1); w= tpdf(zz,4)./normpdf(zz); ww = w/sum(w); tt= randsample(zz,500,true,w)
Histogram of generated rv’s
The real t histogram (with 4 dfs)
Rejection Sampling is
sometimes the answer!!
Rejection Sampling Our goal is to simulate from f.
Instead we simulate from (a dominating distribution) Q. Q is dominating in the sense that there is a constant c with the property that cQ>f. For each simulated point xi, we also simulate a uniform variate ui
between 0 and 1. If ui<f(xi)/cQ(xi) then we accept the point; otherwise we reject it.
Rejection Sampling
Why does rejection sampling work? Use Bayes theorem – a constant problem-solver:
Rejection Sampling:The Key
Say f is the target density, q is the simulating density, and c is the constant:
( )| ( )
( )( )|
( )
( )( )
( ) = ( )
f xP U x q x
cq xf xP x U
cq x C
f xq x
cq xf x
C
An Example of rejection sampling We want to simulate a normal random variable
truncated to be above 3. We use a dominating exponential distribution:
With constant c=2/[3(1-Φ(3))√2π]. So, we create Exponential random variables Xi and uniforms Ui,
rejecting when Ui< φtrunc(Xi )/cΕΧΡ1.5(Xi).
2
31
exp22 2 3 3
exp for x>31 (3) 2 23 1 (3) 2
xx
x
Matlab code for rejection sampling in generating a truncated normal num=0; c=3/(2*sqrt(2*pi)); for ii=1:1000 zz=0; while zz<3, zz=exprnd(2/3); end w=normpdf(zz)/(c*exppdf(zz,2/3)); if(rand<w) num=num+1; ss(num)=zz; end end ss
Truncated normal simulated by exponential: acceptance rate=20%
A real histogram for truncated normal
Problems with rejection sampling If the constant ‘c’ is too large, (as will be
the case in high dimensions) rejection sampling fails because the acceptance rate is so small.
If the contours of the ground truth distribution don’t correspond to those of the proposal distribution (e.g., one is multi and the other is unimodal), then rejection sampling fails to properly capture those contours. (Possible Project: Construct an example of this)
Rejection Sampling in High Dimensions The ground truth space is 1000 dimensional
(iid) gaussian space with component means 0 and standard deviation σP. If we use rejection sampling to simulate it using gaussians with component means 0 and standard deviations σQ =.99σP. Using the fact that,
We see that to use rejection sampling, we need to use c= 20,000. This means that the acceptance rate is about 1/20000=.00005.
1000
1 0 40 0( )1.010 2.314*101
( )P
Q
Q x
P x
Metropolis and the Hydrogen Bomb Metropolis received his B.Sc. (1937) and Ph.D. (1941) degrees in experimental physics
at the University of Chicago. Shortly afterwards, Robert Oppenheimer recruited him from Chicago, where he was at the time collaborating with Enrico Fermi and Edward Teller on the first nuclear reactors, to the Los Alamos National Laboratory. He arrived in the Los Alamos, on April 1943, as a member of the original staff of fifty scientists. After the World War II he returned to the faculty of the University of Chicago as an Assistant Professor. He came back to Los Alamos in 1948 to lead the group in the Theoretical (T) Division that designed and built the MANIAC I computer in 1952 and MANIAC II in 1957. (He chose the name MANIAC in the hope of stopping the rash of such acronyms for machine names, but may have, instead, only further stimulated such use.) From 1957 to 1965 he was Professor of Physics at the University of Chicago and was the founding Director of its Institute for Computer Research. In 1965 he returned to Los Alamos where he was made a Laboratory Senior Fellow in 1980.
Metropolis contributed several original ideas to mathematics and physics. Perhaps the most widely known is the Monte Carlo method. Also, in 1953 Metropolis co-authored the first paper on a technique that was central to the method known now as simulated annealing. He also developed an algorithm (the Metropolis algorithm or Metropolis-Hastings algorithm) for generating samples from the Boltzmann distribution, later generalized by W.K. Hastings.
Metropolis-Hastings to the rescue!!
We need to do ‘smart’ (rather than dumb) monte carlo. Smart monte carlo takes account of the past where importance and rejection sampling do not. For Metropolis Hastings we need a proposal density Q(xnew ,xcurrent) which tells us how to move from current point xcurrent to the new point. Suppose P is the ground truth density (from which we would like to sample). We simulate a point using the proposal distribution and ‘accept’ it only if it is ‘likely’ – not ‘more likely’. Basically, MH is a hill climbing algorithm.
Hill Climbing for Academics You want to explore a hill. You are particularly
interested in the hill’s terrain near the top. After each step, you analyze the terrain at that point.
Greedy algorithm: After each step. Propose a path which always goes up the hill. Note that you get stuck at local peaks where the only way to go is down.
Metropolis: At each step, propose a path; always
take it if it is going up, but sometimes take it even if it is going down. In this way, you can break out of local peaks.
Good versus Evil!!!
Metropolis Hastings greedy algorithms
Formal Metropolis Hastings Our goal is to visit sites in proportion to their
posterior probability --- Metropolis only moves to a new position depending on its likelihood: the probability of moving is:
new
new
current
new
Probability of going from x to xProbmove(x | ) 1
Probability of going from x to x
Q(x | ) | =1
Q(x | ) |
currentnew current
current
new new
current current
x
x x Y
x x Y
current
new
Q(x | ) | =1
Q(x | ) |new new new
current current current
x Y x x
x Y x x
Metropolis Explanation
The ratio measures the real
Likelihood ratio but if it is very likely that we visit the new value xnew, then we have to discount it’s likelihood; we do this by using the factor,
|
|new
current
x y
x y
|
|current new
new current
Q x x
Q x x
Proof that Metropolis works We show why it works. To do this, we show
‘the equation of balance’:
This shows that,
Therefore the resulting x’s are distributed according to the steady state distribution of the chain.
current( | )Probmove(x | x ) ( | )Pr ( | )new new current new currentx y x y obmove x x
( | ) Pr ( | )( | )new new current current currentx y obmove x x x y dx
Proof (continued) We show balance:
And this is the same with the other side of the equation, by symmetry.
current
new
new current
Probmove(x | )*( | )
Q(x | ) |=Q(x |x ) 1 |
Q(x | ) |
=Q(x | ) | Q(x | ) |
new current current
new newnew current current
current current
current current new new
x x y
x x Yx Y
x x Y
x x Y x x Y
Example: Simulating Multivariate Gaussian variables using independent Gaussians
Suppose we would like to simulate values from
We use the proposal:
Z’s are standard normals. The determinant is designed to correspond to the ground truth.
1 .9( ) exp (1 / 2) ; =
.9 1f x
-1x'Σ x
1 2[1] [1] .4 * ; x [2] [2] .5 *new current new currentx x z x z
The move Probability The move probability is:
So generate a vector using independent normals, then calculate the move probability, and move if it is realized, otherwise, don’t.
1
1
'exp
2Pr 1
'exp
2
new new
current current
x x
obmovex x
A Picture of the data
Devise a Metropolis Hastings algorithm to do rejection sampling when there is no dominating
distribution available
Suppose we want to simulate from a complicated density f(x) and instead simulate from a density h(x) which does not dominate f (i.e., there is no c with the property f(x)<ch(x)). First, choose a c for f(x)<ch(x) for some x. Next, we calculate that, ( )
| ( )( )( )
|( )
( ) ( ) ( )( ) &if 1
( ) ( ) =
( ) &otherwise
f ZP U Z y h y
ch Zf ZP y U
ch Z d
f y f y f yh y
cdh y cd ch y
h y
d
Move probabilities calculated to satisfy balance
If x,y∈C={u: f(u)/ch(u)<1} α(y|x)=1 and α(x|y) [f(y)f(x)/cd]=
[f(x)f(y)/cd], so α(x|y)=1. If x ∉ C, but y ∈ C. Then α(x|y) =1
because x has high f value. So, α(y|x) [f(x)f(y)/cd]= [f(y)h(x)/d] or α(y|x)=[ch(x)/f(x)]; If x∈C, but y∉C. Then α(x|y) =[ch(y)/f(y)];
Probmove continued If x ∉ C, but y ∉ C, suppose α(x|y)=1; α(y|x) [f(x)h(y)/d]= [f(y)h(x)/d] or α(y|x)=[f(y)h(x)/h(y)f(x)]∧1 Example: Use this to simulate from a
t distribution with 4 degrees of freedoms
by simulation from a standard normal r.v. Constant c=1.2.
Explanation: Basic code. Define the set C as: Start with x. Then if x,yεC, move to y. If yεC but x not in C, move to y with
probability, cφ(x)/t4(x). Else y=x. If xεC but y not in C, move to y with
probability, cφ(y)/t4(y). Else y=x. If neither x nor y is in C, move to y with
probability, min{t4(y)φ(x)/[φ(y)t4(x)],1}. Else y=x.
4( ): 1
( )
t rC r
c r
Matlab code for % compute a t distribution using a normal x=0; y=0; c=1.2; suc=0; % construct 5000 variables for ii=1:5000 % simulate x,y and determine whether x,y are in C. nx=normpdf(x,0,1); ny=normpdf(y,0,1); tx=tpdf(x,4); ty=tpdf(y,4); if(tx<1.2*nx) x1=1; else x1=0; end if(ty<1.2*ny) y1=1; else y1=0; end % distinguish move probabilities based on occupancy if(x1==1 & y1==1) alph=1; end if(x1==1 & y1==0) alph=(1.2*normpdf(y,0,1))/tpdf(y,4); end if(x1==0 & y1==1) alph=(1.2*normpdf(x,0,1))/tpdf(x,4); end if(x1==0 & y1==0) alph=(tpdf(y,4)*normpdf(x,0,1)/... (tpdf(x,4)*normpdf(y,0,1))); end if(rand<alph) tt(ii)=y; suc=suc+(1/5000); else tt(ii)=x; end x=tt(ii); y=normrnd(0,1); ss(ii)=y; end
T distribution with 4 degrees of freedom simulated from a normal versus normal histogram
Gibbs Sampling
Gibb’s sampling is a special case of Metropolis where we always move. For k dimensional data it is the composition of k MH algorithms; the i’th is:
α(x[1],…,x[i],y[i],x[i+1],…,x[n] |x)=δy[i],x[i]g(y[i]|x)
Gibbs Sampling Example
Simulate a normal random variable truncated above 5. Add a variable z and define the joint distribution,
g(x,z)=constant δx>5δ0≤z≤exp{-(x-μ)2
/2σ2
}
The marginal distribution of x is normal with mean mu and variance sigma-squared truncated above 5. We apply Gibbs sampling by sampling x conditional on z and then z conditional on x.
Gibbs Sampling
(x|z)= {5,3+√-2σ2log(z)} This comes from solving the equation, z< exp{-(x-μ)2/2σ2}; x>5 (z|x)={0,exp(-(x-μ)2/2σ2)} This comes from fixing x and ‘solving
for z’.
Matlab Code for truncated normal using Gibbs Sampling Mulower is the truncation point, mu is the normal mean, and
sigma is the normal sd. function tnp=truncpos(mulower,mu,sigma) z=normpdf(mulower,mu,sigma)*sigma*sqrt(2*pi)*.2; if(z<.0000000001 & mulower<mu) tnp=mu; return; end if(z<.0000000001 & mulower>mu) tnp=mulower; return;
end for ii=1:10000 if(z>1) continue; end tnp=unifrnd(mulower,mu+sqrt(-2*sigma^2*log(z))); z=unifrnd(0,normpdf(tnp,mu,sigma)*sigma*sqrt(2*pi)); end
Simulation of a normal random variable with mean 3 and variance 1 truncated to be above 5 (simulation of 500 values).
Example: Simulating Multivariate Gaussian variables using Gibbs Sampling
Suppose we would like to simulate values from
We use the proposal: basic code is:
Z’s are standard normals.
1 .9( ) exp (1 / 2) ; =
.9 1f x
-1x'Σ x
2[1] | [2] .9* [2] .19 ; [2] | [1] [2] .19 *new current current new current currentx x x z x x x z
Definition of Simulated Annealing Simulated annealing (SA) is a generic probabilistic meta-algorithm for the
global optimization problem, namely locating a good approximation to the global optimum of a given function in a large search space. It was independently presented by S. Kirkpatrick, C. D. Gelatt and M. P. Vecchi in 1983, and by V. Černý in 1985. It originated as a generalisation of a Monte Carlo method for examining the equations of state and frozen states of n-body systems, invented by N. Metropolis et al in 1953.
The name and inspiration come from annealing in metallurgy, a technique involving heating and controlled cooling of a material to increase the size of its crystals and reduce their defects. The heat causes the atoms to become unstuck from their initial positions (a local minimum of the internal energy) and wander randomly through states of higher energy; the slow cooling gives them more chances of finding configurations with lower internal energy than the initial one.
By analogy with this physical process, each step of the SA algorithm replaces the current solution by a random "nearby" solution; if the new solution is better, it is chosen, whereas if it is worse, it can still be chosen with a probability that depends on the difference between the corresponding function values and on a global parameter T (called the temperature), that is gradually decreased during the process. The dependency is such that the current solution changes almost randomly when T is large, but increasingly "downhill" as T goes to zero. The allowance for "uphill" moves saves the method from becoming stuck at local minima—which are the bane of greedier methods.
Simulated Annealing for Academics You want to explore atomic phenomena at low
energies by slowly freezing settings – fast freezing sometimes fails to show what’s happenning.
Metropolis: At each step, propose a path; always move if the new point is more likely, but sometimes take it even if it is less likely. In this way, you can break out of local peaks.
Simulated Annealing: Begin as above. Eventually, propose paths which (more and more) tends to move to points which are more likely and tends to never move to points which are less likely. Note that since this doesn’t happen for awhile, you avoid early local peaks.
Simulated Annealing If we are applying a Metropolis Hastings
algorithm use move probability,
Where ‘Tn’ is a temperature parameter which is decreasing down to 0 at an appropriate rate. Note that for small temperatures the move probability tends to 1 if the posterior probability ratio is >1 and it tends to 0 if the posterior probability ratio is <1.
(1/ )current
new
|Q(x | )Probmove(x | ) 1
Q(x | ) |
nTnew newnew
new oldcurrent current current
Y x xxx
x Y x x