Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
MARKOV CHAIN MONTE CARLOMETHODS
Hedibert Freitas Lopes
The University of Chicago Booth School of Business5807 South Woodlawn Avenue, Chicago, IL 60637
http://faculty.chicagobooth.edu/hedibert.lopes
1 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Outline
1 Historical facts
2 MH algorithmsSpecial casesRandom walk Metropolis
3 Simulated annealing
4 Gibbs sampler
5 Example ix. simple random effects model
6 References
2 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Historical facts
Dongarra and Sullivan (2000) list the top algorithms with thegreatest influence on the development and practice of scienceand engineering in the 20th century (in chronological order):
• Metropolis Algorithm for Monte Carlo
• Simplex Method for Linear Programming
• Krylov Subspace Iteration Methods
• The Decompositional Approach to Matrix Computations
• The Fortran Optimizing Compiler
• QR Algorithm for Computing Eigenvalues
• Quicksort Algorithm for Sorting
• Fast Fourier Transform
3 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
70s and 80s
Metropolis-Hastings:Hastings (1970) and his student Peskun (1973) showed thatMetropolis and the more general Metropolis-Hastings algorithmare particular instances of a larger family of algorithms.
Gibbs sampler:
Besag (1974) Spatial Interaction and the Statistical Analysis of Lattice Systems.Geman and Geman (1984) Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images.Pearl (1987) Evidential reasoning using stochastic simulation.Tanner and Wong (1987). The calculation of posterior distributions by data augmentation.Gelfand and Smith (1990) Sampling-based approaches to calculating marginal densities.
4 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
MH algorithms
A sequence {θ(0), θ(1), θ(2), . . . } is drawn from a Markov chainwhose limiting equilibrium distribution is the posteriordistribution, π(θ).
Algorithm
1 Initial value: θ(0)
2 Proposed move: θ∗ ∼ q(θ∗|θ(i−1))
3 Acceptance scheme:
θ(i) =
{θ∗ com prob. α
θ(i−1) com prob. 1− α
where
α = min
{1,
π(θ∗)
π(θ(i−1))
q(θ(i−1)|θ∗)q(θ∗|θ(i−1))
}
5 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Special cases
1 Symmetric chains: q(θ|θ∗) = q(θ∗|θ)
α = min
{1,π(θ∗)
π(θ)
}2 Independence chains: q(θ|θ∗) = q(θ)
α = min
{1,ω(θ∗)
ω(θ)
}where ω(θ∗) = π(θ∗)/q(θ∗).
6 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Random walk Metropolis
The most famous symmetric chain is the random walkMetropolis:
q(θ|θ∗) = q(|θ − θ∗|)
Hill climbing: when
α = min
{1,π(θ∗)
π(θ)
}a value θ∗ with higher density π(θ∗) greater than π(θ) isautomatically accepted.
7 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Example iv. RW Metropolis
0 2 4 6 8
02
46
θθ1
θθ 2
0 2 4 6 8
02
46
θθ1
θθ 2
0 2 4 6 8
02
46
θθ1
θθ 2
0 2 4 6 8
02
46
θθ1
θθ 2
q(θ|θi ) ∼ N(θi , 0.25Σ2).8 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Example iv. Ind. Metropolis
−2 0 2 4 6 8 10
02
46
θθ1
θθ 2
−2 0 2 4 6 8 10
02
46
θθ1
θθ 2
−2 0 2 4 6 8 10
02
46
θθ1
θθ 2
●
●●●
●●●●
●●●
●●●●●●●●●●●●●●●●●
●
●●●●●●●●●
●●●●●●●
●●
●●
●●●●●●●●●●●●
●●●●●●●●●
●
●
●●●●●
●●●●●●●
●●●●●●●●
●●
●●● ●●●●
●●●
●
●●●●
●
●●●
●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●
●●●●●●
●●
●●
●●●
●●●
●
●●●●●
●●●●●
●
●●●●●
●●● ●
●●
●●● ●●●●●●●●●●
●●
●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●
●
●●●●●●●
●●●●●●●●●●●
●●●●●●● ●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●
●●●
●●●●●
●
●
●
●●●●
●●
●●●●
●●●●●●●●●
●●●
●●
●
●●●●●
●
●●●●●●●●
●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●
●●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●
●●●●
●●
●●●
●●●●●●●●
●●
●
●
●●
●
●●●●●●
●●●
●●●●●●●
●●●
●●
●
●●●
●●●
●●●●
●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●
●●
●●●●
●
●
●●●●
●●●●●●
●●●●●●●●●
●●●
●●●●●
●
●●●●●●●●●●●●●●●●
●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●●●●●●
●●●●●●●●
●●●●●
●●●●●●●
●●●●
●
●●
●
●
●●●●●●●●●●
●●●●●●●●
●●●●●
●●
●●●●●●●●
●
●
●
●
●●●●●●●●
●●
●
●●●●●●●
●●
●●●●●●●
●●
●
●●● ●
●●●●●●●
●●
●
●●●●
●●
●●●●●
●●●●●●
●
●●●●●●
●●●●●●
●●●
●
●●●●●
●●●●●●
●●
●●●●●
●●●●●●●●●●
●●●
●●●●●
●●●
●
●●●●●●●●●●
●●
●
●
●
●●●●●
●●●●
●●●
●
●●●
●
●●
●●●●●●●●●
●
●●●
●●●●
●●
●
●●●
●●
●●●●●●●●●●● ●●●●●●
●●
●
●●●
●●●
●●●●●
●
●
●●●
●●●●●●●●●●●
●
●●●●●●●
●●●●
●●●●●
●
● ●
●●
●
●
●
●
●●●●●●●
●●●
●
●●
●
●●●
●●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●●●●●●●●●
●●●
●●
●●●●●●●●●●
●
●●●●●●●●●
●●
●
●●●●●
●●●●
●●●●●●●●●
●●
●
●●●●
●●●●●●●
●●
●●●●●●●●
●●●
●●●●●●●●●●●●●●●●
●●●●
●●●
●●
●●●
●●●
●●
●●●●●●●●●
●●●●●
●●●●
●
●●
●●●●●
●●
●●●●●●●
●●●●●●●●●●
●●●●
●
●●●●●●●●
●●●●●●●●●●●●●
●
●●●
●●●●●●●●●●●
●
●●●●●●●
●●●●
●●●●
●●●●●
●●●
●●●●●
●●
●●
●
●●
●
●●
●●●●●●●●●●●●●
●●●●●●●●●
●●●●
●
●●●●●●●●
●●●
●●●
●●
●●●
●●●●
●●●●●●
●●●
●●●
●
●
●●●●●●●●●
●●
●●
●●●●●
●
●●●
●●● ●
●●●
●●
●●
●●●●●●●●●●●●●●●●●●
●●
●●●●●●●
●●●●
●●●●
●●●●●●
●●
●
●●●●
●●
●●●
●
●●
●●
●●●●●●
●●●●●●●●●●●
●
●
●●●
●
●
●●
●●
●
●●
●●●●●●●●●●●●●●
●●●●●●
●●●●
●
●●
●
●●●●●
●●●●●●●
●
●●●●●●●●●●●●●●●
●
●
●●
●
●●●
●
●
●●●
●●●●●
●●●●●●●●●●● ●●●●
●●●
●
●●●●●●●●
●●●●●●
●
●●●●●
●●●●●●●●●
●●
●●●●●●
●●●●●●●●●●●
●●
●●●●
●
●●●
●●●
●●
●
●
●●●●●●●●●●
●●●●●●
●●
●
●●
●
●●●●
●●●●●●
●
●
●●●
●●●●●
●●●●●●●●●●●●●
●●
●●●●●●
●●
●
●●●●●●●●●●●
●●●●●●●●●●●●
●●●
●●●●●●●●●●
●●●●
●
●●●●
●●●
●●
●●●
●●●●●●
●●●
●●●●●
●●●
●●●
●
●●
●●●●●●
●
●
●
●●●●
●●●
●●
●●●●●●●
●●●●●●
●●●●
●●●●●●●●●●●●●●●
●●
●●
●
●●●●
●●●●●
●●
●●●●
●●
●●●●●
●●●
●●●●●●●●
●●
●●
●●●
●●
●●●●●●
●●●●●●●●●●●
●●●●
●●●●●●●●●●●●
●●●
●●●
●●●●
●
●
●●●
●
●●●●●●●
●
●●●
●●●●●●●●●●●●●●●●●●●
●
●
●●●●●●●●●●
●●●●●
●●●●
●●●
●●●
●●
●●●●●
●●●
●
●●●
●●● ●●●●●●●●●● ●
●●●
●●●
−2 0 2 4 6 8 10
02
46
θθ1
θθ 2
q(θ) ≡ qSIR(θ) ∼ N(µ,Σ).9 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Example iv. Autocorrelations
Iterations
θθ 1
0 500 1000 1500 2000
02
46
8
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
Iterations
θθ 2
0 500 1000 1500 2000
02
46
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
Iterations
θθ 1
0 500 1000 1500 2000
−2
02
46
810
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
Iterations
θθ 20 500 1000 1500 2000
02
46
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
10 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Example v. RW Metropolis
−5 0 5 10 15
−10
−5
05
1015
θθ1
θθ 2
−5 0 5 10 15
−10
−5
05
1015
θθ1
θθ 2
11 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Example v. Ind. Metropolis
●
●●●
●●●●●
●●
●●
●●●●●●
●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●
●●●●●●●●●●●●●●●●●●●●
●● ●●●●●●
●●●●
●●●●
●●●●●●●●●●●
●●●
●●●●●●●●●
●●●●●●
●●●●●●
●●
●●●●●
●●●
●
●●●●●●●●●●
●●●●●●●●●●
●
●
●●
●
●●●●●●●●●●●●●●
●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●
●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●● ●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●●●●
●
●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
● ●●●●●●●●
●●●●●●●
●●
●
●●●●●●●●
●●
●●●
●●●●●●●●●●●
●●●
●●●●●●●
●●●●●●●●●●●●●
●●●●●●●●
●
●●●●
●●
●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●
●●
●●●●
●
●
●●
●●
●●●●
●●
●
●●●●●●
●●
●●●
●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●
●●●●●●●
●●●●●●●●
●●●●●
●●●●●●●●●
●●●●●
●
●
●●●●●●●
●●●
●●●●●●●
●
●●●●●
●●
●●●●●●●●●
●●
●
●●●●●●●●
●●
●●●●●●●●
●●
●●●●●●●●●●●●●
●
●●●●●●●
●●
●●●●●
●●●●●●●
●●●●●●●
●●●●●●●●●●●●
●●●●
●●●●●●●●●●●
●
● ●●●●●
●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●●●
●
●●
●●●●●●●●●●
●●●●●●●
●●
●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●
●
●
●●●●●●●●●●
●●●●●
●●●●●
●
●
●
●
●●●●●●●
●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●
●●
●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●
●●●
●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●
●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●●●●●●●●●●
●●●●●●●●
●
●●●●●●●●●●●●
●●●
●●●●●●●
●●
●
●●
●
●●●●●●●●●●●●●●●
●●●●●
●●●●
●●●●
●
●●●●●●●●
●●●
●●●
●●
●●●●●●●
●●●●●●●●●●●●
●
●●●●●●●●●●
●●
●●●●●●●●●●●●●
● ●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●
●●● ●●●●●●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●●
●
●
●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●
●●
●
●●●●●●●●●●●●●●●●●●●
●
●●
●●●●●
●●●●●●●●●●●
●●●●●
●●●●●●●●●●●●●
●●
●●●●
●●
●
●●
●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●
●●●●●●
●●●●●●●●●●●
●●●●
●●
●●●●●●
●
●
●●●●●●●●●●●●●●●●
●
●●●●●●
●●●●
●●●●
●●●●
●●●●●●●●●
●●●●●●●
●●●
●●●●●●●● ●●
●●●●●●●●●●●●●●●●●●●●
●●
●
●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●●●●
●●●●●●●●
●●●
●●●●●●●●●●●●●●
●●●●●●
●
●●●●●●●●●●
●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●●●● ●
●●●
●●●
−5 0 5 10 15
−10
−5
05
1015
θθ1
θθ 2
●
●●●
●●●●●
●●
●●
●●●●●●
●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●
●●●●●●●●●●●●●●●●●●●●
●● ●●●●●●
●●●●
●●●●
●●●●●●●●●●●
●●●
●●●●●●●●●
●●●●●●
●●●●●●
●●
●●●●●
●●●
●
●●●●●●●●●●
●●●●●●●●●●
●
●
●●
●
●●●●●●●●●●●●●●
●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●
●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●● ●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●●●●
●
●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
● ●●●●●●●●
●●●●●●●
●●
●
●●●●●●●●
●●
●●●
●●●●●●●●●●●
●●●
●●●●●●●
●●●●●●●●●●●●●
●●●●●●●●
●
●●●●
●●
●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●
●●
●●●●
●
●
●●
●●
●●●●
●●
●
●●●●●●
●●
●●●
●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●
●●●●●●●
●●●●●●●●
●●●●●
●●●●●●●●●
●●●●●
●
●
●●●●●●●
●●●
●●●●●●●
●
●●●●●
●●
●●●●●●●●●
●●
●
●●●●●●●●
●●
●●●●●●●●
●●
●●●●●●●●●●●●●
●
●●●●●●●
●●
●●●●●
●●●●●●●
●●●●●●●
●●●●●●●●●●●●
●●●●
●●●●●●●●●●●
●
● ●●●●●
●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●●●
●
●●
●●●●●●●●●●
●●●●●●●
●●
●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●
●
●
●●●●●●●●●●
●●●●●
●●●●●
●
●
●
●
●●●●●●●
●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●
●●
●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●
●●●
●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●
●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●●●●●●●●●●
●●●●●●●●
●
●●●●●●●●●●●●
●●●
●●●●●●●
●●
●
●●
●
●●●●●●●●●●●●●●●
●●●●●
●●●●
●●●●
●
●●●●●●●●
●●●
●●●
●●
●●●●●●●
●●●●●●●●●●●●
●
●●●●●●●●●●
●●
●●●●●●●●●●●●●
● ●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●
●●● ●●●●●●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●●
●
●
●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●
●●
●
●●●●●●●●●●●●●●●●●●●
●
●●
●●●●●
●●●●●●●●●●●
●●●●●
●●●●●●●●●●●●●
●●
●●●●
●●
●
●●
●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●
●●●●●●
●●●●●●●●●●●
●●●●
●●
●●●●●●
●
●
●●●●●●●●●●●●●●●●
●
●●●●●●
●●●●
●●●●
●●●●
●●●●●●●●●
●●●●●●●
●●●
●●●●●●●● ●●
●●●●●●●●●●●●●●●●●●●●
●●
●
●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●●●●
●●●●●●●●
●●●
●●●●●●●●●●●●●●
●●●●●●
●
●●●●●●●●●●
●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●●●● ●
●●●
●●●
−5 0 5 10 15
−10
−5
05
1015
θθ1
θθ 2
12 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Example v. Autocorrelations
Iterationsθθ 1
0 500 1000 1500 2000
45
67
89
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
Iterations
θθ 2
0 500 1000 1500 2000
02
46
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
Iterations
θθ 1
0 500 1000 1500 2000
−2
02
46
810
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
LagA
CF
Iterations
θθ 2
0 500 1000 1500 2000
02
46
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
Iterations
θθ 1
0 500 1000 1500 2000
−5
05
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
Iterations
θθ 20 500 1000 1500 2000
05
10
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
Iterations
θθ 1
0 500 1000 1500 2000
−5
05
10
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
Iterations
θθ 2
0 500 1000 1500 2000
05
1015
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
13 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Example vi. tuning selection
Tthe target distribution is a two-component mixture ofbivariate normal densities, ie:
π(θ) = 0.7fN(θ;µ1,Σ1) + 0.3fN(θ;µ2,Σ2).
where
µ′1 = (4.0, 5.0)
µ′2 = (0.7, 3.5)
Σ1 =
(1.0 0.70.7 1.0
)Σ2 =
(1.0 −0.7−0.7 1.0
).
14 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Target distribution
θθ1
θθ 2
0.1
0.2
0.3
0.3
0.4 0.4
0.5
0.6
0.7
0.8
0.9
−2 0 2 4 6
02
46
8
−20
24
6 0
2
46
8
0.0
0.2
0.4
0.6
0.8
θθ2
θθ1
15 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
RW Metropolisq(θ, φ) = fN(φ; θ, νI2) and ν =tuning.
●
●●●●
●●
●
● ●
●●●●
●●
●●●●●●●
●●●●●
●
●●●●
●●
●
●●●
●● ●●●●
●
●●
●●●
●●●●●●
●●
●●
● ●●●
●●
●●●
●●●
●●●
●
●●
●
●
●●●●
●
●
●●●●●●
●●●
●
●●
●
●
●●●
●
●●●●●
●
●●●
●●●
●●
●
●●●
●●●●
●
●
●●●●●●
●● ●●●
●●
●●●
●●●●●●●
●●
●●●●●●
●
●●
●● ●●
●●●
●●
●
●●●●
●
●
●●● ●●●
●●
●●●●● ●●●●●●
●
●●
●●
●●
●●● ●
●
●
●●
●
●●●
●●●●●●
●●●●
●●●
●●
●●
●●●●
●●●
●●
●●●
●
●●●●
●●
●●●●●
●●●●●●●●●
●●●●●●
●●●
●●●●
●●
●●
●●
●●
●●
●●●
●●●● ●●●●
●●
●●●● ●●●●
●●
●●
●
●●●●
●●●
● ●●●
●●●●
●
●●●
●●
●●●
●
●●
●
●●● ●●
●
●●
●●●●
●
●●●
● ●●● ●●
●●
●●●●●● ●
●●●
●●●●
●●●●●●●●● ●●
●●●
●●●
●●● ●
●
●●
●●
●
●●●●●
●●
●●●●●
●●
●
●●●
●●●
●●●●●●●
●●●
●●
● ●●●
●●●●
●●●●
●●
●
●
●●●
●●●
● ●● ●
● ●●
●●
●●●
●●●
●●●
●● ●
●●●●●●●●
●●
●●●
●●●●●
●●● ●●●●●●
●●
●●●●
●
●●●●
●●●●●●
●●
●
●●
●●●●
●●●●
●
●●
●●●
●●
●●●●●
●●
●●● ●
●
●
●●●●● ●●●
●
●●●●●
●●
●●●
●●●●●● ●●●●
●
●●
●●
●●●●
●●●●
●●●●●●
●●●
●●
●●
●●
●
●●●●
●●
● ●●●●
●●●
●●● ●●
●●● ●
●●
● ●●●
● ●
●●●
●●
●●●
●
●
●●●
●
●●●
●●
●
●
●
●●
●● ●●●●● ●
●
●●●
●●
●●
●●●●●
●●●●●●●●●
●●
●●●●●●●
●●●● ●●
●●
●●
●●
●●●
●●●●●●
●●●
●●●
●●●●● ●●
●
●
●●●●
●●●●●●●●●●
●●●●●
●●●●●
●●●
●
●
●●● ●●●●
●
●
●
●●●
●●●●
●
●
● ●
●●
●●
●
●●●●●
●● ● ●●
●●
●●
●●●●
●●
●
●● ●●
●●●●●
●●
●●●●●●●
●
●●●
●●
●●
●●●●
●●●
● ●●●●
●●
●
●●
●●
●●
●●●●
●●●●
● ●
●●●●
●●
●●
●●●
●●●●●
●
●●
●●●● ●●
●●
●●
●
●●●●●
●●●
●●●
●
●
●
●●●●●●●
●●●
●●●●●●●●●
●
●●
●●●
●
●●
●●●
●●●●●●
●●●
●●●
●●
●●●●●●●
●
●
●
●
●●●●●●●
●●●●●●●●●●
●●●●●
●●
●●●●
●●●●
●●●
●●●
●●●●●
●
●
●●●
●●
●●●●●
●●
●●● ●●
● ●●●●
●● ● ●
●●
●● ● ● ●
●
●●●
●●
●●●●●●●
●●
●
●●
●● ●
●
●●●●
●●●
●●
● ●●
●●●●●
● ●●
●●●
●●●●
●●
●●●
●●● ●●
●●●●● ●
●●●●●
●●●
●● ●●●●
●
●●●●
●
●
●●
●●●●●
●●
●●
●
●●
●●
●●●●
●●
●●●●
●●
●●
●●
●●●
●
●●
●
●●●●
●
●●●● ●
●●
●
●●●●
●●●●
●●
●●●
●●
●●●●●●
●●
● ●●
●●●●●●
●●●
●●
● ●● ●●●●
●●
● ●
●
●●●●●●●
●
●● ●
●●●●●
●●●●
●●●
●●●●
●●
●
●
●● ●●
●●●
●●
●
●●●●●●
●● ●
●●●
●●
●● ●
●●●●●
●●●●●
●●●
●●●●
●●●●●●
●●
●●●
●●● ●
●●●
●●●●●●
●●
●
●●●●
●●
●●●●
●●●
●●●●●●
●●●
●●●
●
●●
●
●●
●● ●●●●
●●●●
●
●●●
●●●●
●●
●●●
●●●● ●●●●●
●●●
●●●●●
●●●●
●●●●
●
●●●●●
●●
●● ●●●
●
●●● ●● ●
●●
●
●●●
● ●●●●●●
●●
●●●
●●●●
●●
●●
●●●●●
●●●●●●
●●
●●●
●●●●●●
●●●
●●●
●●●●●
●●●
●●●●●● ●●●
●●●●●
●●●●●●
●
●●●●●●
● ●
●●●●
●
●●
●●●●●●●●
●●●●●
●●● ●●●
●●
●●●●●●
● ●
●●
●
●
●●
●●●●
●●●●
●
● ●●●● ●
●
●●●
●●
●
●
●●●●
●●●●
●●● ●
●
●●●
●● ●●
● ●●●●
●●
●●●●●●
●●●●
●
●●●●
●
● ●●● ●
●●●
●
●●
●
●●●
●
●●
●●●●
●●●
●
● ●●●
●●
●●●●
●●●●
●●●●●
●●
●
●●●●
●●●
●●
●
●●
●●●
●●
●●●●
●●
●● ●
●●
● ●●●●●
●
●
●●●●
●
●●●
●●
●●●●●●
●●●●●
●●●●●
●●●●●
●●
●●●●●●
● ●
●●
●●●●● ●
●
●
●●●
●
●●●●●●
●●●
●●●●
●●●●
●●
●●●●
●●●
●●
●●
●●●●●
●●
●
●●
●●●
●●
●
●●●●
●●
● ●●● ●●
●●
●●●●
●●
●
●● ●●
●●
●
●●●
●●●●●●●●●
●
●●
●●●
●●●●●
●
●●
●●●●●●●●
●●●●●●● ●●
●●●
●●●●
●●●
●●●●
●
●●
●●
●●●●
●●●
●●●●●●
●●●
●●●●●
●
●●●
●●●
●
●●●
●●●
●
●●●
●●●●●
●●
●●●
●
● ●
●
●●●●
●●●
●
●●●
●●●●●●
●●●●●●●●
●●
●●●●●●●
●
●
●●● ●●●●●
●●●
●
●●
●●●● ●●●●
●●●●
●●●
●●●
●●● ●●
●●●
● ●
●●
●●
●
●
●●●●●●
●●
●●●
●●
●
●●
●●●●●●●●
●●
●●
●
●●●●
●●●
●●
●● ●●● ●
●
●●
●●
●●
●●●●
●●●●●
●●●
●●
●●
●●● ●●●
●●●●●
●●
●
●●●●●●●●
●●●●
●●
●●●
●●
● ●
●● ●●
●●●
●●●●●●
●●●●●
●● ●
●●●●●●
●●
●
●
●●●●●●
●●●●
●●●●
●●●●●
●●
●
●●●●
●●
●●●●●
●
●
●
●●●●●●●●
●● ●
●●●
●
●●●●●●
●●●●
●●●
●●●●●●
●●
●
● ● ●●●●●
●
●
●●
●● ●
●●●
● ● ●●●●●●●
●●●●●
●● ●●
●●
●
●●●●
●
●●
●
●●
●●●
●●●
●●●●●
●
●●●
●●●●
●●
●●●
●●● ●
●●●●
●●●
●● ●●
●●●●
● ●●
●
●●●
●●●
●●●
●●
●●
●●●●
●●●
● ●●●●●●●
●●●●●●●
●●
●●
●●
●●
●●
●●●
●●● ● ●
●●●●
●
● ●
●●
●●●●●●
● ●●●●●●
●●●●●●●
●●
●●
●
●●●
● ●●●
●
●● ●●●●
●
●●●●
●●
●●●●
●●●
●●
● ●●●
●●
●●
●●●
●● ●●●
●●●
●
●●
●●●
●
●●● ● ●●●●●
● ●●●●●●●
●●
●● ●●●●
●●●●
●●
●●●
●●
●
●● ●●●●●●
●●
●●●
●●●●
●●●●● ●●●●
●
●
●
●●
●
●●
●●●●●
● ●●●
●●
●
●●●●●●
●●●●●●●
●●
●●
●●●●● ●●●●● ●●
●●
● ●●●
●●
●●
●●
●
●
●●●
●●●● ●●
●●●●
●
●● ●●●● ●
●●●
●●● ●●
●●
●●●
●
●●
●● ●
●● ●●● ●●
●●●●●
●●●
●●
●●
●●●
●●●
●
●●●●
●●●
●●●●●●●●
●●●●
●● ● ● ●
●●●●
●●●●
●
●
●●
●
●●●
●●●
●● ●●●●
●●●
●●
●●●
●●●●● ●●
●
●
●●
●●●
●●
●●●●●
●●
●●●●●
●●
●●●●●
●●●●
●●●●
●
●●
●
● ●● ●
●●
●
●●
●●
●●●●●
●●
●
●●●●●
●●●●
●●●●
●
●●●
●●●●●
●●
●● ● ●● ●●●●●●
●●●
●●
●●●
●●●●
●●●
●
●●
●●●●
●
●●
●●●
●
●●
●●●●●
●●
●●●●
●●
●●●
●●● ●●
●●
●
●
●●●● ●●●●
●●●●●●●
●●●●
●●●●
●●●
●●●
●●●●
●●
●●●●●●●●
●
●●●
●●●● ●
● ●●●●●
●●
●
●●●●●●●
●
●●
●●●
●●●●
●●●
●●●●●
●●●●●●●
●●
●
●
●
●●●●●●●
●● ●●●●
●●
●●●●●●●
●●●●●●
●● ●●●● ●● ●
●●●●
●●●●●
●●●
●●
●●●●
●●●
●● ●●●●
●●● ●●
●●
●
●●●
●●
● ●
●●
●●
●●
●●●●
●
●
●●●●
●
●● ●●
●
●●●
●
●●
●●
●
●
●
●●●●●●
●●●●●
●
●●●
●●
●●
●●●
●●●
●●●●●●
●●●
●●●●●●●●
●●
●
●●
●●●●
●●●
●●●●
● ●●
●●●●
●●●
● ●●●
●●●●●
●●●
●●
●●●
●●●●●
●●●●●
●●● ●●● ●
● ●●
●●●●●●●
●●●●
●
●●●
●●
●●●●●●●
●●
●●●
●
●
●●●
●
●●
●●
●
●●●●●
●●●●
●●●
●●
●●●●
●● ●●●●
●
●●
●●●●
●●●●
●●●●●●●●●
●
●
●●
●●
●●●
●
●
●●● ●●●
●●●●●●●
●●●
●●
●
●●●●●●
●●
●●●
●●●●
●●●
● ●●●●●●●
●
●
●●
●●●●
●●●●
●●●
●●● ●● ●●●
●●●
●●
●●●●●●
●●●
●
●● ●●
●●
●●●
●●●
●●●
●●●●
●●● ●●
●
●●●●
●●
●●●
●●
●
●
●
●●●●●●
●●
●●
●●●
●
●● ●
●●
●
●
●●●●
●
●●
●
●●●●●●
●●●●●●●
●●●●●●
●●
●●●
●●
●
●●
●●●
●●●
●●●
●●
●
●●●●●
●●
●
●
●●
●●
●●●●●●●● ●●
●●●
●●●
●●●●
●●●●
●●●●
●●●●●
●●●
●●
●● ●
●
●●
●● ●●●
● ●●●●●●
●●
●● ●●
●●
●●●●●●●
●●●
●●
●
●●
●●
●●●●
● ●●●●●●●●
●●●●●
●●●●
●●●
●●●
●
●●●
●●
●●●●●
●
●●●
●
●●●●●
● ●
●●●●
●
●●●
●●
●
●●● ● ● ●
●●●●●
●●
● ●●●●
●●●
●●
●●
●●●●
●●●●
●●●●
●●●
●●●
●●●
●●●
●●●
●●●●
●
●●
●●
●●●●
●●
●●
●● ●●●●●
●●
●●●
●●
●
●
●●●●●●●
●●●
●
●●●
●●●
●● ●
●●●
●●●
●●
●●
●●●●
●
●●●●●
●●
●●● ●
● ●●●●
●●●●● ●●
●● ●●
●
●●
●●
●●
●●
●●
●●●
● ●●●●
●●
●●
●●
●●●●●●●
●●
●●● ●●●●●●
●●
●●●
●●●●●
●●●●● ●
●●
●●●
●●
● ● ●
●●
●●●●
●●●
●●
●●●
● ● ●
●●●
● ●●
●●●
●●
●●
●●●●●
●●●●●●
●●
●●● ●● ●
●
●●●●●●
●●
●●●
●●●
●●●
●●●●●●
●●●●●●
●●
●●●●●●●
●●●●●●●●
●●●●●●●●
●●● ●
●
●●●
●●●●●●●
●●●●●
●●●●
●●
●
●●●●
●●● ●●
●●●●●●●●●●● ●
●●●●
●●● ●
●
● ●●
●●●●●●●●
●●●●
●●●
●
●●
●●●●
●●●●●●●●●
●●●●●
●●
●●
●●●
● ●●● ●
●
●●
●●
●● ●
●●●●●
●●●●
●●●●
●●●●
●● ● ●●●●● ●●●
●●●
●●●●●
●●
●●●●●●
●●●●●
●●●●
●● ●●●●●●●●●●
● ●●
●●●
●●●●
●●●●●
●●●●
●●●
●●●●
●● ●
●
●●
●●
●●●
●●●
●
●
●●●●●
●●
●●●
●●
●●●●●
●●●●
●●●●
●
●●●●●●
●●●●●
●
●●
●●●● ●
●●
●●●●●
●●●●●●
●●
●●●● ●●
●
●
●●●●
●●●
●●●
●●●●
●●●
●●
●●●
●
●●
●●●●●
●●
●●●● ●
● ●● ●●●●●
●
●●
●●
●●●●
●●
●●●
●●
●●
● ●●
●●●
●●●●
● ●●
● ●
●●
●
●●
●●
●
●●●
●
●●
●●●
●●
●
●
●●●
●●●●
●●●●●●
●
● ●●●
●●●●
●
●●●●●
●
●●●
●●
●
● ●●●●
●
●●
●
●
●●
●●●
●●
● ●
● ● ●●●●●●●●●
●●●●●
●●●
●●●● ●●●
●
●
●●●●
●●●
●
● ●
●
●●●●●●
●●●
●
●● ●
●●●
●●
●●●
●●●
●●
●●
●
●
●●
● ●●●●●●●●
●●
●
●●●
●●●●
●
●●
●
●
●● ●
●●●●●
●● ●●●●
●●
●
● ●
●●●●●
●
●●●
●●●
●●●●●
●●
●
●●●●●●●●
●
● ●●●●●●●
●
●●
●●●●
●●●
●●
●●
●●●●
● ● ●
●
●●●●
●●●
●●
●●●●
●●●●●
●●●●
●●●
●●●●●
●●●●●●
●●●●
●● ●●●●
●●
●●●●●●●
●●●●●●
●
●
●
● ●●
●
●●
●●●●
●●●● ●
●●●●
●●●
● ●
●●
●●●●●
●●●●●
●●
●●
●●●
● ●● ●
●●
●●●
●●●
●●
●●●
●●●
●●●●
●●●●●●
●●●
●●●●●●●●●
● ●●
●●●
●●●●
●●●
●●●
●●●●
●●● ● ●●
●
●●
●●●●
●●●
●●
●●●●
●●●●●●●
●●●
●●●
●●●●●
●●●
●●● ●●
●●●
●●●●
●●●●●
●● ●●●●● ●
● ●●●●
●●● ●●
●●●●
●●●●●
● ●●●
●
●●
●●
●●
●●
●●
●
●●
●●●
●
●
●●●●●●
● ●●●●●
●●●●
●●●
●●
●●
●●●●●●
●●
●●
●●●
●●●●●● ●●●●●●●●●
●●●●●
●●
●●
●●●●
●●●●●●●●●●
●
● ●● ●
●●
●●
●●●
●●
●●●
●●● ●●●●●
●
●
●
●● ●●●
●●
●
●●●●●
●
−2 0 2 4 6
02
46
8
tuning=0.01 Initial value=(4,5)
Acceptance rate=93.8%
●●●●●●●●●●●●
●●
●
●
●
●●
●●
●●
●
●●●
●
●
● ●●●●
●●●●
●
●
●●
●●●●
●
●●
●
●
●●
●●
●●
●
●
●
●●●●
●●
●
● ●
●●●
●●
●●●●
●●
●●
●●
●●●●
●●●●
●
●
●●●●●●●
●●
●
●●
●
●
●
●●
●●
●●
●●
●●●
●●●●●
●●●
●●●
●
●●●
●●
●●
●
●●●●●
●●●●
●
●●●●
●
●
●●●●●
●●●●
●●●
●●
●
●●
●●
●●●●●
●●●● ●
●
●
●●●●
●●●●
●●●
●
●
●
●
●●●●●
●
●●●
●●●
●
●●
●●
●
●
●
●●
●●
●
●
●●
●●
●
●
●●●
●●●●●●
●●●
●●
●●●●●●
●●●●●●
●●●●●●●
●
●
●●●●●
●●●●●
●●
●
●
●●
●●
●●●●
●●
●●●
●●●●●
●
●●●●●●
●●
●
●●
●
●●●
●●●●
●●●●●●●
●
●
● ●●●
●●●
●●
●
●●
●●●
●
●
●
●●
●
●
●●
●●
●●● ●●●
●●●
●●●●●●●
●●●
●
●
●●●
●●●●●●●●●
●
●●●
●
●●●
●●●
●
●
●
●●●●
●
●●
●●
●●
●●●
●●
●●
●●
●●●
●
●●
●●
●
●●
●●●●
●
●
●●
●
●●
●●●●●●●
●●●
●●●●●●
●
●● ●●●
●
●●●●
●
●
●●
●●●
●
●
●
●
●
●●●●
●●
●
●●●●●●●
●
●●●
●
●●
●●
●●●●●●●●●●●●●
●
●
●
●●
●
●●
●●●●
●●●
●
●●●●
●
●●
●
●●●●●●
●
●●
●●
●
●●●●●●●
●●●
●
●●●●
●●●
●●●●
●
●●
●●
●
●●● ●●●●●●●●
●●
●●●●●
●●●●
●
●●
●●●
●
●
●●●
●●●●●
●●
●●●●●
●●●●●●●●●●●●
●
●
●●
●
●
●
●
●●●
●●
●●
●●
●
●
●
●
●
●●
●●
●●●●
●
●●
●
●
●
●●
●●●●●●●●●●●●
●
●
●●●●●
●●●
●
●●
●●
●
●●●
●
●●
●●●
●
●●●
●●●
●●●●
●●●
●●
●●●●
●●●●
●
●●
●●●●●
●●●●●●●●
●●●
●
●
●●
●
●●
●●
●●●●●●
●
●●●●●
●●●●●●
●
●
●●●●●
●●●●
●
●
●●●●●●
●●
●●
●●●●●●
●●●●
●●●●
●●●●●●●●
●●●
● ●●
●●●●●●
●●
●
●●●
●●
●●●●
●●
●●●●● ●●●●
●●●●●
●●●
●
●
●●
●
●
●
●
●●
●
●
●●
●●●●
●●
●●●●
●
●●●●●
●
●
●●●
●
●●
●●●
●
●●●
●●
●
●●●●●
●
●●
●●
●
●
●●
●●●
●●●
●●
●●
●●●●
●●●
●●
●
●●●●●●●
●●
●
●
●●
●
●
●●●
●●
●●●●●
●●●●
●●●●●●●
●●
●●●
●
●●●●●
●
●●●●●●
●●●
●●
●●
●
●●●●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●●●●
●●
●●
●●●
●●● ●
●●●●
●
●
●●●
●●●●
●●●●
●
●
●
●
●
●●
●●●●●●
●
●
●●●●●●●
●
●●
●●
●●●
●●●●
●
●
●
●●●
●
●●●
●●●●
●●●●
●
●●●●●
●●
●●●●●
●●●●●●
●●●●●●●
●●
●●●●
●●●●●
●
●
●●
●●●
●
●
●●●●●
●
●●
●
●
●
●
●●●
●
●
●
●
●●
●●●●
●
●
●
●
●●●●●●
●
●
●●●●
●●
●●
●●
●●●
●●●●●●
●●●●●
●●●
●●
●
●●
●●
●●●
●
●●●●●●●
●●●
●
●
●●●
●
●
●
●
●
●
●●
●●●●●
●●
●●
●●
●●
●●●
●
●●●
●
●●
●
●●●●●●●●
●●●
●●
●●●
●●●●●
●●●
●●●
●
●●●
●●
●●●●
●
●
●
●
●●●●●●●●
●●●
●●●
●●●●●●●●●● ●
●
●●●●●
●●
●●●
●
●●
●
●●
●
●
● ●
●●
●●●
●●
●
●
●●
●
●
●
●
●●●●
●●
●
●
●●●
●●
●●
●●
●
●●●
●●●●●●●
●
●●●●●
●
●
●
●●
●●●
●
●
●
●
●
●●
●
●●
●●●
●
●
●●● ●
●
●●●●●
●●
●●
●●●●●
●
●●
●
●
●
●●●
●
●
●●●●●
●●●
●
●●
●
●
●●
●
●●
●●
●●●●
●●●●●
●●●●
●●●●
●●
●●
●
● ●●●
●●●●●●●●●●●
●
●●●●●●
●
●●●
●●
●
●●●●●●●●
●
●●●●
●●
●●
●●
●●●
●
●●●●
●●
●
●●
●
●●
●
●●●●●●
●
●
●●●●●●
●●●
●●●●
●●●●
●●●
●
●●●●●●●●
●●
●
●●●
●●●●●
● ●●●
●●
●
●●●
●●
●●●
●●●
●●●
●● ●
●●
●●●●●
●●●●
●●●●●
●●●●
●
●●●●
●●
●
●●●
●●●●●●●●
●
●●
●
●●●
●
●
●
●●●●
●
●●
●●
●
●●●●●●●
●●
●●●
●●
●
●
●
●●●●●
●●●●●
●
●●●
●●
●●●●
●
●●
●●●●●
●●●●●●
●
●
●
●●●●●●
●●
●
●
●●
●
●●●●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●●
●
●●
●
●●
●
●
●●
●●●
●●
●
●●
●
●●
●
●●
●
●●●
●
●
●●
●
●●●
● ●
●
●●●● ●●
●●●●●●
●●●
●●●●
●●
●●
●●
●●
●●
●●●
●
●
●
●
●●●
●●●●
●
●
●●
●
●●●●
●
●
●
●●
●●●
●
● ●
●●
●
●● ●
●●●
●●●
●
●
●●
●●●
●●
●●●●●●
●
●●●●●
●
●
●
●
●
●
●●●●
●
●●●●
●
●
●●●●
●●●●●
●
●●●
●
●●●●●
●●
●●
●●
●●●●●
●
●●●
●
●●
●
●
●●●●
●
●●●
●
●●●●●
●●●
●●●
●●
●●
●●
●
●
●●●●●●●●
●
●●●●
●●
●●●
●
●
●●
●●●●●●●●
●●●●
●●●●●●●
●●●
●
●●●●
●●●●●
●●●●●●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●●
●●
●
●●
●
●●●●●
●●
●●●●●●●
●●
●
●
●
●
●●●●●●
●
●●●
●●
●
●
●
●●●●●●●
●●●
●
●● ●
●●
●
●●●●
●●
●●
●●●
●●
●●
●●
●●●●●●●
●●
●●
●
●
●
●●
●
●●●
●
●
●
●
●
●●●
●●
●●●
●●
●
●
●●●
●●
●
●●
●●
●●●●
●●●●●
●
●
●●
●●●●●●●
●●
●
●
●●
●●●●
●
●●●●●●●●
●●
●
●●●●
● ●
●●
●
●
●●●●●
●●●●
●
●
●
●
●
●●
●
●
●
●●●●●
●●
●
●
●●●●●●●●
●●
●
●●●●
●●●●●●●
●●●
●●
●
●●●●
●●●●
●
●●
●●●
●●
●●
●
●●
●●
●
●●
●●
●
●●●
●●●
●●●●
●● ●
●●
●●●
●
●
●●●●●
●
●●●●●●●
●●●
●●
●●●●
●
●
●
●
●
●
●
●●
●
●●●
●●●
●
●
●●
●
●
●●
●●●●●●●●
●
●
●
●●
●●●●
●●
●
●
●
●●●●●
●
●●●
●●
●●●●
●●
●●
●●
●
●●●
●●●
●
●
●●●●●
●
●
●●●●●●●●●●
●●
●
●●●
●
●●
●●
●
●●●●●● ●
●●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●●
●
●
●
●●●
●●
●●●
●
●●●
●●●●
●●
●
●
●
●
●
●
●●
●
●●●
●
●
●●●●
●●●
●●●
●
●
●●
●●
●●
●●●●
●
●
●
●●
●●●
●●●●●●●
●
●
●●
●
●
●
●
●
●●●
●●●●●
●
●●
●●●●
●
●
●●
●●●●
●●●●●
●
●
●
●●●
●
●●●●●
●●
●●●●
●
●
●
●●
●●●●●●
●●●●●●
●●●
●●●●●
●
●●●●
●
●
●●
●●
●
●
●●●
●●
●
●
●●
●
●●●●
●●●
●●●●
●●●
●●
●●●●
●●●
●
●●●
●
●
●●●
●●
●●
●
●●●●●●
●●●●●●●●
●●
●●
●●
●●
●
●
●
●●
●● ●●
●
●
●
●●
●●●●
●
●
●
●●●
●●
●
●●●●●●
●●
●●
●
●●
●●
●●●
●●
●
●
●
●
●
●
●●●●●●
●●●
●●
●
●●●●
●
●
●●●●●●
●
●
●
●●●●
● ●
●●
●●●●●●●●●
●●●●●●
●
●
●●
●●●●●●●●●
●
●
●
●
●
●
●
●●●
●●
●●●●
●●●
●●
●●
●
●●
●
●●●●●
●
●●
●
●●●●
●
●
●●
●●
●
● ●●
●●●●●●
●●●●
●●
●●●
●
●●●●●
●
●●
●
●●
●
●
●●●●●
●●●●●●●
●
●●
●●●●
●
●●●
●●●●
●
●●●
●●●
●●●●
●
●
●●
●
●
●●●
●●●●●
●
●●
●●
●●
●
●
●●
●
●●●●●
●●
●● ●●
●●●●●
●●●
●
●●
●
●
●●●
●●
●
●●●
●
●●●
●
●●
●
●●
●●●●●●
●●
●●
●●●
●
●●●●
●●
●●●
●
● ●●
●●
●
●●●●●
●●
●●
●
●●●
●
●
●●●
●
●●
●●●
●●●●●●●
●●
●
●
●
●●●
●
●●●●
●●
●●●
●
●●●●
●●●●
●●●
●
●●●
●●●●
●●●●●●●
●●●
●●
●
●
●
●●●
●●
●●●
●
●
●● ●●●●
●●●●
●●
●●
●●●
●●●●
●
●●
●●
●●
●
●●●●
●● ●
●
●●
●
● ●●
●●
●●●●●●●
●●●●
●●
●●
●
●●
●
●
● ●
●●●●●●
●
●
●●●
●●
●●●
●●
●●
●
●
●
●●
●●●●●●●●●●●
●
●●●●
●
●●
●
●
●●
●
●●
● ●
●
●●●●
●
●
●●
●●● ●●
●
●
●
●
●
●
●
●
●●
●
●●●●●
●
●
●●●●●
●●●●
●
●
●●●●
●
●
●
●●●●●●
●
●
●
●●●
●
●
●
● ●●●
●●●●
●
●●●●
●
●
●
●●●●●●●
●
●
●●
●
●●●
●
●
●●
●
●
●
●●
●
●●
●
●●
●
●
●●●●●
●
●●
●
●●●
●●●
●
●●●●●●
●
●●●●
●●
●
●
●●●●
●
●●●
●
●
●●●
●●●●
●
●
●●
●
●
●●●●
●
●
●●●
●●
●
●●
●●●●
●
●●●
●●●●
●●●
●●
●
●
●●●●●●
●
●●●
●●
●●●
●
●●●●
●●
●●
●●
●●
●
●●
●●
●●●●
●●
●●
●
●●●
●●
●●●●●
●●●
●●
●
●
●
●●
●●●●
●●
●●
●
●
●●●●●
●
●●●●●●
●●
●●●
●
●
●●
●
●
●●●
●
●
●●●
●●
●
●●●
●●●
●
●●
●
●
●●
●
●●●
●●●●
●●●●
●
●
●●●●●
●●●●●●●●
●
●●
●
●
●
●
●
●
●●●
●
●●●●
●●
●●●●
●
●●
●●●●
●●●●●
●
●●●
●●●
●●●●●
●●●
●
●
●●
●
●
●●
●●●●●●●
●●●●●●●●
●●●●●●
●
●
●●●●●●●●
●●●●
●●●●
●●
●●●●●●●
●●
●●●
●
●
●●
●
●
●●●
●
●
●
●
●●
●
●
●
●●
●
●●●●
●
●
●● ●
●●●●●
●●●●
●●
●●●●●●●
●●●●
●●●
●● ●
● ●
●
●
●
●●● ●
●●●
●
●●
●●
●●
●●
●
●
●●
●●
●
●●
●
●●
●●●
●●●
●●
●
●●
●●
●
●●●●●●●
●●●
●●
● ●●●●●●●
●●
●●●
●●●●
●●
●●●
●
●
●
●●●
●●●●●
●
●
●
●●
●●●●●●
●
●
●●
●●
●
●●
●
●
●●
●●●●
●
●
●
●●●●●
●
●●
●
●
●●●●
●●●
●●
●●
●●
●
●
●
●
●● ●●
●●●
●●●
●●
●●●●●●●●●●●●
●●●
●
●●
●●
●●●●
●
●
●
●
●●
●●
●
●
●
● ●●
●●
●●●●●●●●
●
●
●
●●●●●
●
●
●●●●
●●●●●●
●
●
●
●●
●●●
● ●●
●●
●
●●
●
●
●●
●●●●●●
●
●●
●●●● ●●●
●●●●
●
●●●●●●
●●
●
●●
●●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●●●
●●●●●●
●
●
●
●●●●●
●●
●
●
●
●
●●
●●●●
●●●● ●●●●●
●
●
●●
●●
●●
●
●●●●
●●●
●
●●
●
●●●●●
●●
●
●●●
●
●●
●
●●●
●
●
●●●●●
●●
●●
●●
●
●
●
●
●●●
●●
●
●
●
●●●
●●
●●
●
●●●
●●●●
●●
●●●●●
●●
●
●
●
●●
●●
●●●
●●●●
●
●●●●
●●
●●●
●●
●●●
●●
●
●
●●
●●●●●
●
●
●
●●●
●
●●●●●●
●
●●●●●
●
●●
●●●●
●●
●●●●●●●●
●
●●●
●
●●●●
●●
●●●
●
●●
●
●●
●
●●●
●●●
●●
●●●●
●●●
●●●
●
●
●
● ●●●
●
●
●●●
●●
●
●●●
●●
●●
●●●●
●
●●●●●
●●
●
●●●
●
●
●●
●●●●
●●●
●●●
●
●●●
●●●
●●●
●●●●●●●●●●●
●●●● ●●
●●●●●
●●●●
●
●
●
●●
●●●
●
●●
●●●●
●
●●●●●
●●
●●
●
●●●
●
●
●●
●●●
●●
●
●
●
●●●●●●●●●●●●●●
●●
●●
●●
●
●●●●●
●
●●
●
●
●
●●●
●●
●
●●●
●●
●●
●
●
●
●
●
●
●●
●●
●●●●●
●●●
●●●●●
●●
●●●●
●●●●
●
●●●●
●
●
●●
●
●
●●
●●
●●●
●●●
●●●
●
●●●●●●
●
●●●
●●
●●●
●●●
●●●
●
●●●●●●●
●●
●
●●
●●
●
●●
●● ●●●●●
●● ●
●●●●
●●●●
●
●●
●●
●
●●
●●
●
●●
●
●
●●
●
●
●●
●
●
●●●●
●●●●●
●●●●
●●● ●
●●●●●
●●●
●
●
●●●●
●
●●
●●●●●
●
●●●●●
●●●
●● ●●
●●
●
●
●●
●
●●●
●
●
●●●●●
●●●
●●●●
●●
●
●●
●
●●●●●●●●
●●●
●●●●
●
●
●●●●
●●
●
●
●●●●
●
●●●●
●
●●●
●
●
●●
●●●
●
●●●
●●●
●●●●●●
●●●●●
●●●
●
● ●●
●●●●●●●●●
●●●●
●
●●
●●
●
●●●
●●●●
●●
●
●
●
●
●●
●●
●
●●●●
●●●
●●●●●
●●●●●●
●●
●
●●
●●●
●
●●●●
●
●●●
●
●
●
●
●
●
●●●●● ●
● ●
●●●●
●●
●●●
●●
●●●●●
●
●●●
●
●●●●●●●●●●●●
●●●
●
●●●●
●●●●
●●
●●
●
●●●
●●● ●
●●●
●●
−2 0 2 4 6
02
46
8
tuning=1 Initial value=(4,5)
Acceptance rate=48.5%
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
−2 0 2 4 6
02
46
8
tuning=100 Initial value=(4,5)
Acceptance rate=2.4%
●●●●●●●●●
●
●
●
●●●●●
●●●●
●●●
●●
●●●●●●●
●
●●
●●●
●
●●
●●●●●
●●●●
●●●●
●●
●
●●
●
●●●
●●●●
●●●●●
●●●●●●
●●●●
●
●●●● ●●●●
●●●
●
●●
●●●
●●●●●●●●●
●●
●●●
●●●
●
●
●●●
●●
●●●●●
●●
●●●●●
●
●● ●●
●●● ●●
●●●●●●●●● ●
●
●●●●●
●●●●●
●●
●●●●●
●●
●●
●
● ●●
●●●
●●●●●
●●●●●
●●●●
●●●●
●● ●
●●●●
●●●●
●●
●
●●
●●●●
●●●●
●●●
●● ●
●●●
●
●●●●
●●●●
●●
● ●●●● ●●
●●●
●●
●●●●
●
●
●
●
●
●●●●●
●
●
●●●●
● ●●
●●●●●
●●
●●●●●●
●
●
●●
●●
● ●●
●●●
●●
●●●●
●●●●●
●
●●
●●●
● ●
●●●●
●● ●●●
●●●●
● ●●●
●●
●●●●●●
●
●●
●●
●●●●●●
●● ●●●●
●●●●● ●
●
●●●●●●
●●
●●●
●●●
●●
●
● ●●●●●●
●
●
●●
●●
●●● ●●
●●●
●●●
●●
●●●
●
●●●
●●
●●
●
●●●
●●● ●
●●●●●
●● ●
●●●●●●●●
●●● ●
●
●●●●
●
●●●●
● ●●●
●●●
●●● ●
●● ●●
●●●
●●●● ●●●
●●●●
●●
●●●●
●●●●
●● ●
●●●●
●●●●●●
●●●
●●●●●●
●●●●●●●●●●
●●●
●●
●
●●●●●
●●●
●●
●●●●
●●
●●
●
●●●● ●●●●●
●●●●
●●●●
●●●●●
●●●
●
●●●●●●
●●●●
●●●●
●● ●●
●●●●
●●●●
● ●●●
●●●●●
●
●●●● ●●
● ●●●● ●●
●●
●●
●
●●●
●
●●●●
●●●
●●●● ●●
●●●●●
●● ●
● ●● ●●●●● ●
●
●●●●●●●●
●●●
●●
●●●●●●●●●● ●
●●
●●
●●●●
●● ●●
●●
●●●
●●●●●
●● ●●
●●●●●●
● ●●●●●●
●●●
●●
●
●
●●
●
●
●●●●
●
●
● ●●●●●●●
●●●
●●●●●
● ●● ●●
●
●●●● ●
●
●●●●●●
●
●●
●
●●
●●
●●●●●●●●
●●
●●●●
●●●
●●●
●●●●●● ●
●● ●
●●●
●●●
●●●●●●●
●●
● ●●●●
● ● ●
●●●●●●●●
●●
●●●●●
●
●●
●●● ●
●●●●●●●●
●●●
●
●●●●
●
●●●● ●●●
●●●●●●
●●●●
● ●
●
●●●
●●●
●
●
●●●●● ●●● ●
●●●
●● ●●
●● ●
●●●
●●●●●●●●●●●
●● ●
●●
●●●●
●●
●●●
●●●●
●
●
●●●
●●
●●●
●
●●
● ● ●
●●
●
●●
● ●●
●● ●●●
●●
●●
●●● ●● ●●●
●●●
●●
●
●●
● ● ●●●
●
●●●●
●●● ●
●●
●●●●● ●●●
●●
●●●
●●●●●● ●●●●●●
●●●
●●
●●●
●●●
●●
●●
●
●●
●●
● ●●● ●●●●●
●●
● ●●●●●
●●●
●●●●
●
●
●●
●●
●●●
●●
●●
●●
●●●●●
●●
●●
●●
●●●
●●●
●
●●
●●●●●
●●●
●●
●●
●●●●●
●● ●●
●
●● ●●●●●●●●
● ●●
●●
●
●●●●
●●
●●
●●
●
●●● ●●●●
● ● ●●
●●
●
●●●
●●●●●
●●●●
●●
●
●●
●●●●●●●
●
●●●●
●●
●●●●
●● ●
●●●●●●
●●●
●●
●●●
●
●●●
●●
● ●●●●
●●●●●●●
●●
●●
●●●●
●
●●●●● ●●●
●● ●●
●●●
●
●
●● ●
●
●●
●
●●
●●●
●
●●●
●●
●
●●●
●
●
●●●
●●●●●
●
●●●●●
●
●●●●●●
● ●● ●
●●
●●
●●
●●
●●
●●●●
● ●● ●
●●●●●●
●● ● ●●●
●●●●●
●●
● ●●●
●
●
●●●
●●●●●
●
●●●●●●
●●●●●
●
●●●●●●●
●●●
●●
●●●●●
●●●
● ●●
●●●
●●●
●●
●●
●●
●
●●●●
●●●●●●
●●
●●● ●
●●●
●● ●●
● ●
●●●●●
●
●●●
●●●●
●●●
●●● ●●
●●●
●●
●●●●
●●●
●●
●●●●
●●●●
●●● ●
●
●●
●●●
●●●●
●●●●
●●●
●●●
●
●●
●●●●
●●
●●
●●●●●
●●
●●●●
●●●●●
●
●●
●●●●
● ●●●
●
●●
●
●
●●
●●●
●●
●●●
●●●●
●
●
●
●●●●●
● ●●●
●●
●●●
●● ●
●●●
●●●
●●
●●●
●●
●●●
●●
●●●●
●●●●
●
●
●●
●●● ●●●
●●● ●●●●
●●●●●●●●●
●
●
●
●●●●●●●
●●●●●
●●
●●
● ●●
●●●●
●●●●● ●●
●
●
●●
●●●●●
●●●●●●
●●●●●●
●●
●●
●●●●
●●
●●●
●●●●
●●
●●
●
●
●
●●●●
●
● ●●
●●●●
●
●
●●
● ●●
●●
●● ●●●
●
●●
●●
●● ●●
●●
●
●●●
●●●
●
●
●●●
●● ●●●
●●●●●
●●
●●●
●●●
●●●
●●
●●
●
●●●
●● ●●●●●●
●●
●● ●●●●
●●
●●
●●●
●
●●●●●●
●●
●●●
●●●
●
●
●
●●●
●●●●●● ● ●●
●●●
● ●●●
●●●
●●
●
●●●●●●
●●● ●●●●●●
●
●●●●
●●
●
●●●
●●●●●●
●●●
●●●●
●●●
●● ●
●●●● ●
●●●
●
●●● ●● ●●
●●●●
●●●●
● ●
●●●●
●●●● ●
●
●●
●
●●
●●● ●●●●● ●
●●●●●●●
●●
●
●●
●●●●
●●●●●
●●●●●
●●
●
●●
●●
●●●●●●●
●●
●
●●●
●
●●●
●●
●●●
●●
●●●●
●●●●●●●●● ●●●●●
●●●●●●
●●●●
●●●●
●
●●
●●
●●
●●
●●
● ●
●● ●
●
●
●
●●●●●
● ●●●
●●●●
●● ●
●●●
●●●●●●
●
●●●
●●●
●●●
●●●●●
●●●
●●
●●●
●●
●● ●●●●
●●●●●●
●●●
●●
●●●●
●●●●
●●
●●●●●
●●●●●
●●
●●●
●●
●●●
●
●●
●●
●●●●●
●●●
●● ●●
●●
●●
●●●●●
●● ●
●●
●●
●●●●
●●●●
●
●●
●
●●●
●●●●●●
●
●●●●
●●●●
●
●●●●●
●●
●●●●
●●●●●
●●
●●●●
● ●●●
●
● ●●
●●●
●●●●●●
●● ●
●●●
●● ●
●●●
●●
●●
●●
●●●
●●●
●●●
●
●●●●●
●
●●●●● ●●
●●
●●●●●
●●
● ●●
●
●●●●
●●●
●●●●
●●●●
●●
●●
●
●●
●●●
●●
●●●
●●●● ●●●●●●
●
●●●
●●
●●●●
●●
●● ● ●
●●●●●●●●
●●●
●●● ●●
●●●●
●●
●●
●
●●
●●●
●
●●●●●●●●● ●
●●●●
●
●
●●●
● ●●●
● ●●
●●●●
●●●●●
●
●●
●●●●●●
●●
●●●
●●
●●
●●
●●
●●●●
●●●●
●
●
●●●
●●
●●●
●●● ●
● ●●● ●
●●●●
●●●
●●
● ●●●
● ●●●
● ●●●●
●●●●
●●●
●
●
●●
●●●●●
●●● ●●●
●●
●
●●●
●●
●
●●
●●
●●
●●●●
●●● ●●
●●●●●
●●●●●●
●●●
●●●●●● ●
●●●
●●●
●●●●
● ●
●●
●●
●
●●
●●●●
●●
●●●● ●
● ●
●●●
●●●●●
●●●
●
●
●
●●●●●
●●
●●
●●●
●
●●●
●
●
●●
●●
●●●●●●
●
●●
●●●●
●
●●●●
●●●●
●●● ●●
●
●●●●●●
● ●●●
●● ●●●●
●●●●
●●
●●
●●●●
●
●●●●●●●●●
●●●●
●●●
●
●●●
●●●●
●●
●●●●● ●
●●
●●●
●
●●●
●
●●●●●●
●●●●●●
●●●
●●
●●●●●●●●●● ●● ●●
●●●●
●
●●●●
●●●●●●●●
●●●●●
●●
●●● ●●●
●●●●●●
●
●● ●●●
●●
●●●
●●●
●●●●
●●●
●●●●●●
● ●● ●●
●●
●●● ●●●●●●
● ●●
●●●●
●●●● ●
●
●●●
●●
● ●●
●●
● ●●●●●●●
● ●●●
●●
●
●● ●●
●●●
●●●●
●●●
●● ●●
●●
●●
●●
●●●●●
●●
●
●●
●●●● ●●●●
●●
●●●
●●●●
●
●●●●●
● ●●●
●●●●●●
●●●●
●●
●●
●
●● ●●● ●●
●●
●●●●●●
●●●
●●●●
●●●●●●
●●● ●●
●●●
●●●
●●●
●
●●●●●●●
●●
●
●
● ●●● ●
●●●
●●
●●●●
●●●●
●●●●
●●●● ●
●●●●
●●
●●
●
●●●●● ● ●●●●
●●●
●●●
●●●
●●●
●
●
●●●●●●●●●●
●
●●● ●●
●● ●●
●●
●●●●
●●●●●
●
● ●●
●● ●
●●● ●●●
●●●
●●
●●●●●●
●●●
●●
●●●
●
●
●●●●●
●●●
●
●●●●
●●
●●●● ●●●●
●●●●
●
●●●●●●
●●
●●●●●
●●
●●●
●
●●●
●●●
●
●●●
●●●●●
●
●●●●●
●● ●●
●●●●●●
●●●●●● ●
●●●●●
●
●● ●●
●●●●
●●●
●
●●●●●●
●
●●● ●●●●
●●● ●● ●
●●●
●
●● ●
● ●●●
●●●
● ●●●●●●●●
●●
●
●
● ●
●
●
●●●●● ● ●●
●●
●●
●●
●●●●
●●●
●●●●●●
● ●●●●
●●● ●
●●
●●●●● ●
●
●
●●●●●●
●●●● ●
●
● ●●●●●●
● ●●●●●●●
●
●●● ●
●
● ●
●●●●
●●●
●
●●●●
●●●
●●●●
●
●●●●
●●
●●●
● ●●●●●●
● ●
●●●●●●
●●
●●●●●●●●
●●
●●●
● ●●●●● ●●
●
●●●●
●●●●
●
●
●●●
●
●
●●●
●●
● ●
●● ●●●
●●
●●
●●●●●
●●
●●●
●●●●
●●
●●
●●
●●
●
●●
●● ●●●●
●●●
●●
●●
●
●
●
● ●●●
●●
●●●
●●●●●●● ●
●
●●●●●●●
●●●
● ●●●●●
●●
● ●●
●●
●●
●●●●●
●●
●●
●●●●
●●
●●●●
●●
●●
●
●●●●
●●●●
●●●
●
●●
●
●●● ●
●●●
●●●
●●●
●●
● ● ●●●●●●●●●
● ●●●
●●●●
●●●●●●●●
●●●●
●
●●●●●●●●
● ●
●●●●●●●
●●● ●
● ●●●●●●●●●
●
●●●
●●
●
●●●
●●
●
●●
●●
●●●
●●●
●●●
●●
●
●
●
●●● ●●
●● ●●●
●●
●●●●●
●●
●●●●●
● ● ●
●●
●
●●
●●●● ●●●●
● ●●●●●
●●
●●●●
● ● ●●●●●
●●●● ●
●●
●●●●●●
●●●●
●●●●●●
●●●
●●●●●
●●●●
●●
●●●●●●
●
●●●● ●●
●●
●●●
●●●●
●
●●
●●
●●
●●
●●●●
●●●
●●
●
●●
●
●●●
●●
●●
●●
●
●●● ●●
●●●
●
●●●
●
●●
●●
● ● ●●
●●●●●●
●
●●●●
●●●●
●
● ●
●●
●●
●●●
●
●●●●
●●●
●●
●●●
●
●●●
●●●●●
●●●●
●●●
●●●●
●●●
● ●●●
●●●
●●●●
●●
●●●●
● ●●●●● ●●
●●●
●●●●●●
●●
●●●
●
●●
●● ●
●●●●
●●●
●●●●●
●
●●●
●●●●
●●●●●● ●●
●●● ● ●●●
●●●
●●
●●● ●●●
●●
●●●●●●
●●●●
●●●●
●●
●●
●●
●●●●
●●
●●
●●●●
●●●
●●●●●
●
●●
●●●●●●
●●●●●●
●●●●
●
●● ●●●●
●●●●
●●●
●●● ●●●
●●●●●
●●
●●
●● ●●●
●●● ●●●
●●●●
●●●●
●●
●●●● ●
●
●●●●
●●●●●
●●●●
●●
● ●●
●
●●
●●●●
●●●
●●
●
●●
●
●● ●
●●
●●
●
●
●●●●●●●
●●●● ●
●●●●●●
●● ●●●
●●●●●●
●●●
●
●●●●●
●●●●
●●●●●●●●
●●
●●
● ●●●
●●● ● ●
●●●
●●
●
●
●●
●●●
●●
●
●●
●●
●●●
●●●●●
●●
●●
●
●●●●
●
●●●
●●●●●●
●●
●● ●●
●●
●●
●
●
●●●● ●
● ●●●●●●
●●
●
●●
●
●●●●●
●
●●●
●●●●
●●
●●●●
●●● ●
●
●●
●●●● ●
●●
●●
●●●
●
●●●●●
●●●
●
●●
●
●●●●●
●●
●●●●●
●●●
●●
●●●
●●●●● ●
●●●●●
●●
● ●●● ●● ●●
●●●
●●●●
●
●●●●●
●●●●●
●●
●●●●●
●
●
●
● ●●●●●●
●●●
●●
●●●●
●●
●●●●●
●●●●
●●
●●●●●●
● ●●●
●
●●●●●●●
●●●●
●●
●●
●●● ●●● ●
●●●●●● ●
●●●●●●●● ●
●●
●
●●●●
●●●●● ●●●●
●●●●
●●● ●●
● ●●
●●
●●●
●●
●●
●●● ●
●●
●●●●
●●
●● ●
●●
●
●●●●●
●
●●●●●
●●
●●●
●
●
●
●
●●
●●●●●
●●
●●●
●
●●●●●●●
●●●●
●●●
● ●●●●●
●●
●●●●
●●●●
●● ●
●●●●●
●●●●●●●●●●●
●
●
● ●●●●●
●●●
●●●
●●
●●
●●
●● ●●
●
●●●●●●●●●
●
●●
●●●
●● ●●●●●●
●●●
●●●● ●
●●●
●●
●●●
● ●●●
●●
●●
●●●●●●●●●
● ●●
●
●
●●●
●●●
●
●●
●●●●
●●●
●●●
●●●●●●●
●●
● ●
●
●●●● ●
●●
●●●
●●●●
●●●●●
●●
●●●●●
●
●
●●●●
●
●●
●●●●●●
●●●●
●●●
●
●
●
●● ●●
●●●
●●● ●● ●●●
●●●
●●
●●●●●●
●●
●●●●●●
●● ●●●
●●●●
●●●
●
●●●
●●● ●
●●●
●●●●●● ●● ●
●●●
●●●● ●●
●●●●●
●●●● ●●● ●
●●
●●●●
●●●●●
●●●●●
●●●●● ●
●
●●●
●●●●
●● ●●●
●
●
● ●●●●
●
−2 0 2 4 6
02
46
8
tuning=0.01 Initial value=(0,7)
Acceptance rate=93.3%
●
●● ●●●●●
●
●
●●
●
●
●●
●●●
● ●●●●●●●
●●
●
●
●●
●●●●●●●●●
●
●
●●
●●
●●
●
●
●●●
●●●
●
●
●●
●●
●
●●●
●
●●
●
●
●
●●●●●
●
●●
●●●
●●●●●
●●●
●
●
●
●●●●●●●
●●●
●
●●
●
●●
●
●
●
●
●
●●●
●●
●●●
●●
●●●●●
●●●●
●●
●
●
●●
●
●
●
●●●●
●
● ●●●●
●●●●
●
●●●●●
●●●●●●
●●●
●●●●
●●●
● ●
●
●●●
●●●●●
●●●●
●
●●●●
●●●
●●
●
●●●
●
●●●●
●
●●●
●
●
●
●●
●
●●●●
●●●
●
●●●
●
●
●●●●●●●●
●
●
●●●
●●●●●●●●
● ●
●
●
●●●●●●●
●
●●
●●●●●●●
●●●
●
●●
●●
●
●●
●●●
●
●
●●●
●
●
●●●
●●●●
●●
●●●
●●●●
●●●●●●
●
●●
●●
●
●
●●
●
●●
●●●●●
●●
●●●
●●●
●
●
●●
●
●
●
●●
●●●●●
●
●●
●●
●●●
●
●●●
●●●
●
●
●●●●
●
●●●●●●
●●
●
●●
●
●
●●●
●●●
●●●
●●
●●●●●●
●●
●
●●
●
●
●
●●●●●
●
●
●●●
●●
●
●●●●
●●●
●●
●
●
●
●●●
● ●
●●
●●●
●●●●
●●●●●●●●●●
●●●
●●●●
●●
●●●
●●●
●
●●
●●●●●
●
●●●●●●
●●
●●
●
●
●●●●●
●●
●●
●●●
●●●●
●●
●
●
●
●
●●●
●●
●●
●
●
●●
●●
●● ●●
●●●
●●●
●●●
●●
●●●●●●●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●●●
●●●●●●●●
●
●
●●
●
●●●
●
●
●
●
●●
●●●●
●●●●●●●●
●
●●●
●
●●
●●●●●
●●●●
●●
●
●
●
●●
●
●●
●
●
●●
●●●●
●●●
●●●
●
●●●●●
●
●
●●
●
●
●●●
●●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●●●●●
●●
●
●
●
●●
●
●
●
●●●●
●
●● ●
●
●●
●
●
●●
●
●
●●
●●
●●●●●
●
●●
●●●●
●●●●●●
●●●●●
●●●
●●●
●
●
●●●●●
●
●
●
●●●●
●●●● ●
●
●●●
●
●●
●●
●●●●
●●
●●●●
●●●
●
●●
●
●●●●●
●●
●●
●●●●●●●
●●●●
●
●
●●●
●
●●
●●●●
●
●
●
●●●●
●
●
●
●●●
●●
●●
●
●●
●●●●●●●
●
●
●
●●●●●●
●●
●●●
●●
●
●●●●
●●●●●
●●
●●
●
●
●
●●●
●
●●
●
●●●●●●
●
●
●●●●●
●
●
●●●●
●
●
●●●●●
●●●
●
●
●
●●●
●●
●●●●
●●●●
●●●
●●
●●●
●
●
●
●●●
●●●●●●●
●
●
●
●
●●
●●●●●●
●
●●●
●
●
●●●
●●●
●●●●●
●●●●●●●● ●
●●
●●●●●
●●●●●●●
●●●●●●
●●●●
●●●●●
●
●●
●
●●
●●
●●
●
●
●
●●
●●●●
●●●●●●
●●
●●●●
●
●
●
●●●●●
●●
●●
●● ●
●
●●●●●
●●●
●●●●
●
●●
●
●●
●●●●
●
●
●●
●●
●● ●
●●●●
●
● ●●
●
●●●●●
●●
●
●
●●●●
●●●●
●●●
●●●●●●●●●●
●
●
●
●●
●
●●
●●●●
●
●●●
●●
●●●●●●●●●●
●
●●●
●●●●●
●
●
●●
●
●●
●●●●●
●●●●
●
●
●
●
●●
●●
●●●●●
●●
●
● ●
●●
●
●●●●
●●●●
●●●●
●
●
●
●
●●●●
●●●●●●
●●●
● ●●●●
●
●
●
●●●●
●
●
●●●●● ●
●
●
●
●●●●
●
●
●
●
●●●
●●●●●●●
●●
●
●
●●
●
●●
●●●
●●
●
●
●
●
●●
●
●●●
● ●
●●
●●
●●●
●●
●
●●●●●●●●●●●●
●●
●●
●
●
●
●
●●●
●●●●●●
●●●●●
●
●●●●
●
●
●●
●●
●●
●●●●
●
●●●●
●
●
●
●●●
●
●●
●●●● ●●●●●
●
●●●
●●
●
●●●●
●● ●●●
●●
●●●
●
●
●●●●●
●
●●●●●
●●●●
●●●●
●
●
●
●●
●
●●
●
●●
●●● ●●
●●
●
●●
●
●●●●●●
●
●●
●●●●●●
● ●●
●●
●●●●●●●●●●
●●●●●●●●
●
●●●
●
●●●●
●
●
●●
●●●
●
●
●●
●
●
●●
●
●
●
●
●●●
●
●●●●●
●●●
●
●
●
●●
●●●●
●
●●●●●●●
●●
● ●
●●
●
●●
●●●
●
●
●
●
●
●
●
●●●●
●
●
●●
●
●
●
●
●●
●●●●
●
●●
●●●
●
●● ●●●●●●●●●●●●
●
●
●●●
●●●●●●●●●●●
●
●
●●●●
●
●
●●●●
●●●
●●●●●
●●
●
●●●●●●●
●●
●
●●●●
●●●
●●●●●
●●●●
●●●
●●●●●●
●●
●● ●●●●●●
●
●
●
●
●●●●●●●●
●●●●●
●●
●
●
●
●●●
●●●
●
●
●●●●● ●
●
●●●
●●●
●
●●●
●●
●●●
●●
●●●●●●
●●●
●●
●
●●
●
●●
●
●●●●
●●
●●
●
●●●
●●●
●●●●●●●
●
●●
●●
●●
●●
●
●●●
●
●●●
●●
●
● ●
●●●
●●●●●
●
●●
●
●●
●●
●●●
●●●●●●●
●●●
●
●●
●
●
●
●● ●●
●●
●●
●
●●
●●
●
●●●●
●
●
●●●
●●●● ●●
●●●●●
●
●●●
●
●●
●
●●●●●
●
●●● ●
●
●
●●
●●
●●
●●●●●●
●●
●●
●●●●
●●●●●
●
●●●●●
●●●●●●
●
●
●●●●●●●●● ●
●●●●●
●●●
●●
●●●●●●
●●
●●●●●
●●
●
●●●●●●
●●●●
●
●
●●●●
●●
●●●●●●
●●●●●
●
●
●●●
●
●
●●
●●
●
●●
●
●●
●
●●
●
●●●●●●
●
●●
●
●●●●●
●
●
●●●●
●●●
●●●●
●
●
●
●
●●●
●●●
●●●●●
●
●●
●
●
●●●
●
●●
●●
●
●
●
●
●
●●●●●
●●
●●●
●●●
●
●
●
●
●
●●●
●
●●●●●
●
●●●
●●●●
●●
●●
●
●
●●●●●●●●●
●●
●
●
●●●●●●
●●●●●
●
●
●
●●●
●●●
●●●●●
●●●
●●
●●●●
●●
●
●●●
●●●●
●
●
●●●●●
●●●
●
●●●●●●●●●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●●●●●●
●
●●
●
●
●●
●
●●
●
●●
●●
●
●
●●●●●●
●●●
●●
●●
●●
●●
●●●●●●
●
●●●●●
●●●
●
●
●
●
●●●
●●
●●●●
●●
●
●●
●●●●●
●●
●●
●
●
●
●
●●
●●●●
●●●
●●
●
●
●●
●●●
●
●
●●
●●●●●
●
●
●●
●●
●
●
●
●●
●
●
●●●
●
●
●
●●●●
●●
●
●
●●
●
●●
●●●
●
●
●
●●
●●●
●
●
●●●
●
●
●●
●
●●●●●●●●●
●●●
●●●●●
●
●●●●●
●●
●
●●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●●●●
●●
●
●●
●
●
●●
●●
●●
●●
●●●
●
●
●●●
●
●●●
●●
●
●
●●
●●●
●●
●●●●●
●
●
●
●●●
●●
●●●
●
●
●●●
●
●●●●●●●●●
●
●●
●
●
●
●●
●●●●
●●●●●●
●
●●
●
●●
●
●●
● ●●
●
●●●
●
●●●
●●●●●
●
●
●
●●
●●
●●●
●
●●●●●
●
●
●●
●
●●
●●●
●
●●●
●●
●
●●
●
●
●●●
●
●●
●●●●●●
●●●
●●
●
●
●
●●●●
●●
●●●
●
●●
●●
●●
●●
●
●●●●●●●
●
●
●●●
●●●
●●●●
●
●●●●
●
●●
●●●●●
●
●●●●●
●●●
●●●●
●
●●●
●
●●
●
●
●●
●●
●●●
●
●
●●●
●●●
●●●●
●●
●●
●●
●●
●●
●●●
●
●●
●
●●●●●
●●
●
●●
●●
●●
●●●●
●
●●
●
●●●●●●●●
●●●●●
●●●
●
●●
●●●●
●
●●
●
●●
●
●●
●
●●
●●
●
●●
●
●
●
●●
●●●
●
●
●●
●●●
●
●●●
●●●
●●●
●
●●●
●●
●
●
●
●
●
●●
●
●●
●●●●●
●●
●●
●●●
●●●●
●
●
●●●
●●●
●●●●●
●●●
●●●●●●●
●●●● ●
●●●●●
●●●●●
●
●
●
●
●●●●
●●
●●●●
●●●●●●●●●●
●
●
●●
●
●
●●
●
●
●●
●●●
●●●● ●
●
●●●●●●
●
●
●●
●
●●
●●●
●●
●●●
●●●
●●
●●●●
●●
●
●●●
●
●
●
●
●●●●
●●●●
●
●●●●
●
●●●●
●
●●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
●●●
●●
●
●●●
●●
●●●
●●●●
●
●●●●●●●
●
●●
●●●●
●
●●●
●
●●●●
●
●●
●
●
●●
●●●●
●
●
●
●●●●
●
●
●●
●
●●
●●●
●●●●
●
●
●
●
●●●
●
●
●●●●
●
●●
●●
●
●●●
●●●●
●●●●●
●●●
●
●
● ●●●●
●●●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●●●●●
●
●
●
●
●
●●
●
●●●●
●●●
●●●●
●●
●
●
●
●●
●
●
●
●
●●● ●
●●
●●●●
●●●
●●●●●
●●●●●●
●●
●●●●●●●
●●●●●●●●
●●●●
●●●●
●
●●●●
●
●●
●●
●●
●●●
●
●●
●
●●
●●
●●
●
●●●●●
●●
●●
●●●●●
●
●●
●●●
●
●
●●
●● ●●●●
●●
●●
●
●●●●●
●●
●
●●
●
●●
●
●
●
●●●
●●
●
●
●●●●●
●●●●
●
●
●
●●
●●
●●●●
●
●
●
●●●●
●
●
●
●●
●
●●●●●●
● ●
●● ●●●●
●●●●
●●
●
●
●●●●●●●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●●
●●●●
●●
●●
●●●●●●
●
●●●
●
●●●
●●●
●●●
●●
●
●●●●
●●
●●●
●
●
●●●●
●
●
●
●●
●
●
●●●●●●
●●
●●●●●●●
●●●
●
●
●
●
●●●●
●
●
●●
●
●●
●●
●
●
●●●●●
●●●
●
●●●
●●●●●
●●●
●
●●
●●●●
●●●●
●
●
●●●
●
●●●●●
●
●●
●
●●●●●●●●●●
●●
●
●
●●●
●●
●●
●
●●
●
●
● ●●●●
●
●
●
●●●
●●
●●
●●●●●
●●
●●●●●
●●●
●
●●●●●●
●●●
●
●
●●●●
●●
●●●●●
●
●
●
●
●●
●●
●●●●
●
●●●●
●
●●
●●●
●●
●
●●●●●
●●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●●
● ●
●
●●
●●●
●●●
●
●
●
●●●●
●●●
●
●
●●●
●●
●●●●●●
●●●
●
●
●
●
●●●●
●
●
●●●●●●
●
●●
●●
●
●●
●
●●●●
●●●
●●●
●●
●●
●●
●●
●
●●
●
●●●
●●●
●●●●●●●●
●●●●
●
●●●
●●●
●
●●
●●●
●
●
●
●
●●●
●●●●●
●●●●●
●●●
●
●●●
●
●●●
●
●●●●
●
●●●●●●
●●
●●●●
●●●●●●●●●●●
●●
●
●●
●●
●●●
●
●
●●●●●●●●
●
●
●
●●●
●
●
●
●●●●●
●●
●●
●
●
●
●●●
●
●●●
●●●
●
●
●
●●
●
●●
●●●●●
●
●●
●
●●
●
●●●
●●
●●●
●●●
●●●●
●
●●●
●●●●●●●●●●●
●●●●●●
●
●●●
●
●●
●●●
●
●●●
●●
●●●●●●●
●
●
●
●●
●●
●
●
●●●●
●●
●●●●●●●●
●●
●
●●
●●
●●●●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●●●
●●
●
●●
●●●●
●●
●●
●
●●●●●●●●●●●●●●
●
●●●●
●●●
●●●
●●●●●●●
●
●●
●●
●●●
●●●
●
●●●
●
●●●●
●
●●
●●
●●
●●●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●●
●
●●
●●●●
●●
●●
●
●●●●●●
●●●●●
●●
●
●●●●●●
●●●● ●●
●
●●
●●
●
●
●●
●
●
●●●●●
●●
●●
●
●●●
●
●
●●●●●●●●
●●●●●●●●
●
●●●
●●●●
●
●
●●●
●
●
●
●●
●●
●●
●
●●●
●●●●●●●●●
●●●
●
●●●●
●
●●●●
●
●
●
●
●
●
●
●
●●
●
●●●
●●
●●●
●●
●●●●
●●
●●●●
●●●●●●●●
●
●●●
●
●●●●●●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●●
●
●
●●●
●●
●
●●●
●●●
●●●
●
●
●●
●●●●●●●
●
●●
●●●
●● ●●●
●●●
●
●●
●
●
●●●●●●
●●
●●●●●
●●
●●
●●●
● ●
●●
●●●●
●●●●
●●●
●●
●●●●
●●
●●●●●●●●●
●
●●●●
●
●
●
●●●
●
●
●
●●●
●
●●
●
●
●●
●
●●●●●
●●
●
●
●●
●
●
●
●●●●
●●●●●●
●●
●●
●●
●●●●
●●●●
●
●●●●●
●●●●
●
●
● ●
●●●●●●
●
●● ●●
●
●
●●●
●●
●●●●
●
●●
●●
●●●
●●●●●
●
●●●●
●●
●
●
●●●●●
●●●●●●
●●●
●●
●
●
●
●●●●
●●●●●●
●
●●
●●●
●●●●●●
●
●●●
●●
●●●
●●
●●●●●●
●●●●●
●●●●●●●
●
●
●
●●
●●●●●●●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●●●●●
●●
●
●
●●
●●●●●
●
●
●●
●●●
●●
●
●●●
●
●
●
●●●
●●
●
●●●●●●
●
●●
●
●●●●●●●●
●
●
●●
●●
●●
●
●●
●
●
●
●●●●●● ●●●●
●
●●●●●●●●●
●
●
●
●●
●● ●●●●
●
●
●
●
●●●
●●●●
●●
●●
●●
●●
●
●●●●●●
●
●●●●●●
●
●
●
●●
●
●●
●
●●
●
●●
●●●●
●
●●●●●●●●
●●●
●●●
●●●●
●●●
●●
●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●●
●
●●●●●
●
●
●
●●
●●●●●●●
●●
●
●
●
●
●●●
●●●
●
●●
●●●●●●
●
●
●●●●●●●●●●●●●●●●
●●●●●●
●●
●
●
●●●
●
●●●●
●
●●
●
●●●
●
●●●●●
●●●●●●●
●
●●●●
●●
●
●
●●●
●●
●●
●●
●
●●●
●●
●
●●
●●●●
●
●●●
●●●●
●
●●●●
●
●●
●●●●
●
●●●●●
●
●●●
●●●
●●●
●●●
●●
●●
●
●●
●●
●
−2 0 2 4 6
02
46
8tuning=1
Initial value=(0,7) Acceptance rate=49.3%
●●●
●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●● ●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●
−2 0 2 4 6
02
46
8
tuning=100 Initial value=(0,7)
Acceptance rate=2.4%
16 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Autocorrelations
0 50 100 150 200
0.0
0.2
0.4
0.6
0.8
1.0
lag
Tuning=0.01 Initial value=(4,5)
0 50 100 150 200
0.0
0.2
0.4
0.6
0.8
1.0
lag
Tuning=1 Initial value=(4,5)
0 50 100 150 200
0.0
0.2
0.4
0.6
0.8
1.0
lag
Tuning=100 Initial value=(4,5)
0 50 100 150 200
0.0
0.2
0.4
0.6
0.8
1.0
lag
Tuning=0.01 Initial value=(0,7)
0 50 100 150 200
0.0
0.2
0.4
0.6
0.8
1.0
lag
Tuning=1 Initial value=(0,7)
0 50 100 150 200
0.0
0.2
0.4
0.6
0.8
1.0
lag
Tuning=100 Initial value=(0,7)
17 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Independent Metropolisq(θ, φ) = fN(φ;µ3, νI2) and µ3 = (3.01, 4.55)′.
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●
●●
●
●
●
●●●
●●
●●●●
●
●
●
●●
●
●
●●●●●●●●●●●●●●
●
●
●●
●●●
●●●●
●
●
●●●
●
●●●
●
●●●●●
●●
●●●●●
●●
●●
●
●●
●
●
●●●
●
●●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●●●
●●
●●●
●●●
●
●
●
●
●●
●●●●
●
●●●
●●
●
●●
●●
●●●●
●●
●
●
●●●●
●
●
● ●
●●●●●●
●
●
●
●
●●
●
●
●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●
●
●●●●
●●●
●●●●●●●
●
●
●●
●●●●
●●●
●
●●
●●
●● ●
●●●●
●
●
●
●
●
●●●●
●
●●
●●●
●
●
●
●●●●●
●
●●
●●●
●●●
●
●●
●
●
●●
●●●
●
●●
●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●
●●●●●
●
●
●●●
●
●●
●
●
●●
●●●
●
●
●
●●●●●●●●●●●●●●●●●●
●●●
●
●●●●●
●
●
●●●
●
●●
●●●●
●
●●●●●●
●●
●
●
●●
●
●●
●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●●●●
●
●
●
●
●
●●●●●●●
●●●●
●
●●●●● ●●●●●●
●●●●
●
●●●●
●●●
●
●●
●
●
●
●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●●
●
●
●●●●●
●●●
●
●●●●
●
●●●
●
●
●
●●
●
●
●●●●●●
●
●
●
●●●
●●●
●
●●
●
●●●
●
●
●
●
●●
●
●
●●
●●●
●
●●
●
●
●
● ●●●●
●
●●●
●●
●
●
●●
●
●
●●●
●
●●
●
●●
●
●●
●●
●
●●
●●
●●
●●
●●
●
●
●
●●●
●
●●
●
●
●●●●
●
●●
●
●●
●
●
●●
●
●●
●●●●●●●●●●
●
●●●●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●●
●
●●
●●
●
●
●●●
●●●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●●●
●
●●●
●
●●
●
●
●●
●
●●●●●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●
●●●
●
●●
●●
●
●
●
●
●
●
●●●●●●
●●
●●
●
●
●●●
●
●●●●●●
●●●●●●●
●
●
●
●
●
●
●●
●●●●
●●●●●●
●
●
●●●●●●●●●●
●
●
●●●
●●●●●●●●●●●●
●●●
●●
●
●●●●●●●●●●
●●
●●●●
●
●
●
●●●●●●●●●
●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●
●●●
●
●●
●●
●
●●
●
●●
●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
−2 0 2 4 6
02
46
8
tuning=0.5 Initial value=(4,5)
Acceptance rate=9.9%
Acceptance rate=9.9%
●●●●●●●●●●●●●
●
●
●
●●●●●●
●●
●●●●●●
●●
●●
●●●●●
●●●●●●●●
●●●
●
●●●●
●
●●●●
●●●●
●
●●●●●●
●●●
●●●
●●●
●
●●
●
●●●●
●●●●
●●
●●●
●
●●
●
●
●●●●
●●
●
●●
●●●●●
●
●●●
●●●●●
●●●●●
●●
●●●●
●●●●
●
●●
●●●●
●●
●●
●●●●●●
●
●
●●
●●●●●●●●
●●●●
●
●●●●●●
●●
●●
●
●●●●●●●
●●
●●
●●●●●●●●●●●
●●●●●●●●
●●●●
●●●●●
●●
●
●●●●●●●●●●●●
●●●●●●●●●
●
●●●●●
●●●
●●●●●●●●
●●
●
●
●●●●●●●●●●
●
●●●
●●●●
●
●
●●●●●●
●●●
●●●
●
●●●
●●●●●●●●
●
●●
●
●●●●●●●●●●●●
●
●
●●
●
●●
●
●
●●
●●
●
●●●●
●●●●●●●●●
●●●
●●
●●●●
●●●●●●●
●●●
●
●●
●
●
●
●●●●●●●●
●
●●●●
●●●●●●●
●●●
●●●●●
●●●
●●●●●●●●●●●
●●●●
●●●●●●●●●
●
●●●●●●●
●●●●●●
●●
●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●
●●
●●●●
●●●●●●●●●
●
●●●●●●●●●●
●●●●●●●
●●●●
●●
●●●●●●●
●●
●●●●●
●
●●●●●●
●●
●●●●●●●●
●●●●
●
●●●
●
●●
●
●●
●
●
●●●●●●●●
●●●●●●●
●●●●
●●●●●●●●
●●
●●●●●
●●●●●●●●●●
●●
●●●●●●●●●●●●●●
●●●●●●●
●●●●●●
●●
●●●●●●●●●●●●●●●●
●
●●
●●●
●●●●●●
●
●
●●●
●●●●●
●●●●●●●●●●
●
●●●●●●●●
●
●
●
●●●
●●
●●
●
●●
●●
●●
●●●●●●
●●●●
●
●●●●●●●●●●●●●
●●
●●●●●
●●●●●●●●
●●●
●
●
●●
●
●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●
●●
●●
●●●●●●●●
●●
●●●●●●●●●●●●
●●●
●
●●
●●●●●●
●●●
●●●●●
●●●●●●●●●●●
●●●●●
●●●●
●●●
●●●●●●
●
●●
●●
●●
●
●●●●●●●●●●
●●
●●●●●
●●●
●
●
●●
●●
●
●●●
●
●
●●●●●●
●
●
●●●●
●●
●●●●
●
●●●
●●●●
●●●●●●●●
●
●●
●
●
●●●
●
●●●
●
●●●●
●●
●●●●●●●●●●●●●●●●●●●
●●●●●
●
●●●●●●●●●●●●●●●
●●●●●
●●●●
●
●●
●●
●●●●●
●
●●●
●●●●●●●
●●●●
●
●●●●
●
●●
●●●
●
●
●●●
●●●
●
●●●●
●
●
●●
●
●
●
●●
●
●●●
●
●
●●●●●●●●●●●
●●●●
●
●
●●●
●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●
●●●●●●●●●●●
●● ●●●
●●●●●●●
●●●●●●●
●
●
●●●
●
●●●
●
●●●
●●
●●●
●
●●●●
●●●●●●
●●●●●●●
●●●●●●●●
●●
●●
●●●●●●●●●
●
●●
●●●●●●●●
●●
●●
●
●●●
●●●●●●●●●●
●●
●●
●
●
●●
●
●
●●
●
●
●●
●
●●●●●●●●●
●●●●●●
●●
●●●
●●●●
●●●●
●●●●●
●●●
●●
●
●●●●●
●
●●
●●●●●●
●●●●
●
●●●●
●
●●●
●
●●●●●●●●●●●●●●●●●●
●●
●
●●
●
●●
●
●●●
●
●●●●●●●●
●
●
●
●
●●
●
●
●
●
●●●●●●
●●●●
●
●●
●●●
●
●●●●
●●
●
●●
●●●●● ●
●●●
●
●●
●●●●●
●
●●●●
●●●●
●●●●
●●●●●●●●
●●
●
●
●●●●●●●●
●
●●●●
●
●
●●●●●●●●●●
●
●●●●●●
●
●●●●
●●● ●●
●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●
●
●●●●●
●●●●●
●●●●●
●●
●●
●●●●●●●●●
●●
●●
●
●●
●
●●●●●●
●●●
●●●●●●
●●●●●
●●●
●
●●
●●●
●●●●●●●
●●●●●●●●●
●
●●
●
●
●
●●●●
●●●
●
●●
●●●●●●●●
●
●●●●●●●
●
●●
●●●●
●●●●
●
●●
●●●
●●●●●●●●●●●
●●●●●●●
●●●
●●
●
●●●●●●●
●●●
●●●●●●
●●
●
●
●●●●●●●●●●●●●●●●
●●●●●
●
●●●●●●●
●●●●●
●●
●●●
●●●
●●●●●●●
●
●●●●●●●●●●
●●●●●●●
●●●●
●●●●●●
●
●●
●● ●●●
●
●●●●●●●
●
●
●●●●●●
●●
●●●
●●
●●
●●●
●
●●
●
●●
●
●●
●●●●●●●
●●
●●
●
●
●
●●●●
●●●●●●●●●●●●●●●●●●
●●
●●
●●●●●●
●
●
●●
●●●
●●●●●●●
●
●
●●●●●●●●
●●●●
●
●
●●●
●
●
●
●●●●●●●
●
●
●●●●●
●●●●●●●●●●●●●●●●●●
●●
●●●●
●●●
●●●●
●●
●●●
●●●●●
●●
●●●●●●●●●
●
●●●●
●●●●
●
●
●●●
●●
●●●●
●●
●●
●●
●
●●●●●
●●●
●●●
●
●
●●
●
●●●●●●●●●
●
●
●
●●
●
●
●●●●●
●
●
●
●●●●●●
●● ●●
●●●
●
●●●
●●●●
●●
●●●●
●●
●
●
●
●●●●●●●
●
●
●
●
●
●●● ●
●
●
●●●
●●
●●
●●
●
●●●●
●●●●●●●●●●●●●●●
●●
●●
●
●●●●
●●●
●●●●●●●
●●
●
●●●●●●●●●●●●
●
●●
●
●
●●
●
●
●
●●
●●●
●●
●
●
●●
●●●●
●●● ●●
●●●●●
●●
●
●●
●●
●
●●
● ●●●●●●
●●●●●
●
●●●
●●
●
●●●●●●
●●
●
●●●
●●
●
●
●●●●●●●●
●●
●
●
●●●●●
●●●
●●
●●●●●
●●
●●●●●●
●
●●
●●
●●●●●
●●
●●●
●
●
●●●
●●
●●
●●●●●●●●
●
●●
●●●●●●
●●●
●●
●●●●●●●●●●●●
●
●●●●●●●
●●●
●●●●●●
●●●
●
●
●●
●●●●●
●●●
●●●
●●
●●●●●●●
●●
●●●●●●
●
●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●
●●●
●●●●●●●
●
●● ●●●●●●●●
●●●●●●
●●●
●●●●
●●●●●●●●●●
●●●
●●●●●●
●●
●
●●
●●●
●●
●
●
●●
●●●●●●●●
●●●●●●●
●●●●
●●●
●●●●●●●●
●●●●●●●
●●●●
●●●
●
●●●●
●
●●
●●●
●●●●
●●●●●●●
●
●
●
●●●●●●●
●
●●
●●
●●●●●●●●●●●
●●●
●●●●●●●●●●
●●●●●●
●●●●●●
●●●
●●
●
●
●●
●●●●●●
●●
●●
●
●
●●●●
●●●●●●●●
●●●●
●●●●
●●●●
●●●●
●●
●●
●
●●
●
●●●
●●
●
●
●●●●
●
●●●●●●
●●●●
●
●●●●
●
●
●
●●●●
●●●
●●●●
●●
●●
●●●●●
●●
●●
●●●●
●
●●●●
●●●●●●●
●
●●
●●●●●●
●●● ●●●●●●●●●
●●
●●●●●●
●●●●●●
●
●●
●●
●●
●
●
●●●●
●●
●
●●●●
●●●
●●●
●●●●●●●
●●●●●●●●●●●●●●●
●
●
●●●●
●●●
●●
●●●●
●●
●●
●●●●●
●
●●●●
●
●●
●
●●●●●●●●●
●
●
●●
●●●●●
●● ●
●●●
●●●●●●●●●●●●●●●
●●●●●●●●●●
●
●●●
●
●●●●
●●●●●●●●
●●●●●●●●●
●
●●●●●●●●●●●
●●●●●●●●●●●
●●●
●●●●●●●●
●●●●●
●●●
●●
●
●
●
●●●●●
●●●●●●
●●●●●●●●●
●●●●●●
●●
●●●
●●●
●●
●●●
●
●
●●
●●●●
●
●●
●●
●●●●●
●
●●●
●
●●●
●●●●●●
●●
●●
●
●●
●●●●●
●●●●●●●●●●
●●●
●●
●●
●
●●●●●●●●
●
●
●●●●●●●
●
●●●
●●●
●●●
●●
●●
●
●●
●
●
●●
●●●
●●●●●●
●●●●
●●●●●●
●●●●●
●●●●●●●
●●●●●
●
●
●●●●
●●
●●
●
●
●
●●●●
●●●●
●●●●
●●●
●●
●●
●●●
●●
●●●●●●
●●
●●●
●
●
●●●
●
●●●
●●
●●●●
●●
●
●●●●●●●●●●●●●●●●●
●●●●●●
●
●●●●●
●
●
●
●●
●
●●●
●●●●●●●●●●●●●●●●●●●●
●
●
●
●●●●●●●●●
●●●●●
●●●
●●
●●●
●●●
●
●●●●
●●
●●●●●●●●●●●
●●
●●
●
●
●●●●●●●●●●
●
●
●●●●●
●●
●●
●●
●●
●●
●●
●●
●
●●●
●●
●●●
●●
●●
●●●●●●●●
●●
●
●●●●●
● ●●
●●●●
●
●●●
●●●●●●
●●●●●
●●●●●●●●●●●●
●●●●
●●●
●●●●●
●●
●●●
●●●●●
●●●●●●●
●
●●●●
●
●●
●●●●●●●●●●●●
●●
●●●●
●
●●
●●●●
●
●
●●●
●
●
●●●
●
●●●●●●●●●
●●
●●●
●●
●●●
●●●●
●
●●●●●
●
●
●●●
●●●●
●
●●●
●
●●●●●●●●●●
●
●●●
●
●●●●●●●
●
●●●●●●●●
●●
●
●●
●●
●●●
●
●●●●●
●●●●●
●
●
●●●
●●●
●●
●●●
●
●
●●●●●●●●●●
●●●●●
●●●
●●●●
●●●●
●●
●●●●●
●
●●
●
●
●
●●●●
●
●●●●
●
●●●●●●●●
●●
●●●●
●
●
●
●●
●●●
●●
●
●●
●
●
●●
●●●●●
●
●●●
●●
●●●●
●
●
●●●
●●●
●●●●●
●●●●●●
●●●●●●●●●●●● ●●
●●●●
●●●●●●●●●●
●●●●
●●●
●
●●●
●●
●●●●●●
●●
●●
●●
●●●●●●●
●●●●●●●●
●●●●
●
●●●
●●●●●●●●●●●
●
●
●
●●
●●
●●
●●
●●
●
●●●●●
●●●●
●●
●●
●●●●●●●●
●●●●
●●●●
●●
●●●●●●●●
●●●●●●●●
●●●
●
●●●
●
●
●●●●
●
●●
●●
●●
●●
●●●●
●●●●●●
●●●●●●●
●●
●●●●●●●●●●●●●●●●●●●●●
●
●
●●●●
●●●●
●●
●●
●●
●●
●●●●
●
●
●
●●●●
●●●
●
●●●●●●
●
●
●
●
●●●●
●●●●●
●●●●●
●●●●●●
●
●
●●●●
●●
●
●
●●●●
●●
●●●●●●
●
●●●●●●
●●●●
●
●●●●
●●●
●●●
●●●●
●●●●●●
●●
●●●
●●●●●●●●
●
●
●●●
●
●
●
●●
●●
●
●●●●●●●●
●
●●
●●●
●
●
●●●●
●●
●
●
●●●●●●●●●
●●
●●●●
●●●●●●●
●
●●●●●
●
●
●
●●●●●
●
●●
●
●
●
●●●●●
●●●
●
●
●
●
●●
●●●●
●
●●●●●●
●
●●
●
●●●
●●
●
●
●
●●●●●●●●●●●●
●●●●●
●
●
●
●●●
●●●●●●●●●
●
●●●●
●●
●●●
●●
●
●
●
●
●●
●●
●●●●
●●
●
●●●●●●●●●●●●●●●
●●●●●
●
●
●●
●●
●●●●●●●●●●
●●●●
●●
●●
●●
●●●●
●●●●●●●●●●●●
●●●
●●●●●●
●●●●
●●
●
●
●●●●●●●●●●●●●●●●●●●●
●●●●●
●●
●
●●●
●
●●●●●●●●●
●
●●
●
●●
●●●●●●●●●●
●
●●●●
●●
●●●●●
●
●
●●●●●●
●
●●●
●●●●
●
●●●●
●
●●
●●
●●●●●●●●
●●
●●
●●
●●
●●●●●●●●●
●
●
●
●
●●●●●
●●
●●●●
●●
●●●●●●●●●●●●●●●●●●●●● ●
●●●
●●●●●●●●●●●●●●●●
●●●●●●●
●●
●●●
●●●●●●●●●
●
●
●●●●
●
●
●●●●●
●
●●
●●●●●●●●●●●●
●
●●●●●●●●
●
●
●
●
●●●●●●●●●●● ●
●
●●
●
●●●●●●●
●●
●●●
●●
●●●●●●●●
●●●●
●
●●
●
●●●
●●●●
●
●●
●●
●
●●●●
●●●●
●●●
●●●
●
●
●●●●●●
●●●●
●●●●●●●●●
●
●●●●●●●●●●
●
●
●●
●
●●●●
●
●●
●●
●
●●●●●●
●●●●●●●
●●●
●●●●●●
●●●●●●
●
●●●
●●●●●●●●
●●
●●●●
●●●
●●●●●●●● ●
●
●●
●●●●●●●●●●●
●
●●●●●●●●
●●
●●
●●
●
●●●
●●●●●●●●
●●●●●●
●
●●●●●●●●●
●●●●●●●●
●
●●●●●●
●●
●●●●
●●●
●●●●●●●●●
●●
●●●●
●
●●
●●●●●●●
●
●
●
●●
●
●
●●
●●●●●
●●
●●
●
●●●●●●●●●
●
●●●●●
●●● ●●
●
●●●
●●
●●●
●●●●●●●●●●●●
●●●●●
●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●
●
●
●●●
●
●●●●●
●
●
●●●●●
●●
●●●●
●
●●●
●
●
●●●●●
●●●●●●●●●●
●●●●●
●
●●●●
●●●
●●●
●●●●●●●●●
●
●●
●
●
●
−2 0 2 4 6
02
46
8
tuning=5 Initial value=(4,5)
Acceptance rate=30.9%
Acceptance rate=30.9%
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●● ●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●●●●●●
●●●●●●●
●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●●
●●●●
●●●●●
●●●●●●
●
●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●●●●
●●●
●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●● ●●●●●●●●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●●●●
●●
●●●●
●
●●●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●
●●●●●●●●●●●
●●●
●●●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●●●●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●●●●●●●●
●
●●●
●●●●●●●●●●●
●●●●●●●●●●
●●●●●
●●●●●●●●●●●●●●
●●●●●
●●●●●●●●●
●●●●●
●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●
●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
−2 0 2 4 6
02
46
8
tuning=50 Initial value=(4,5)
Acceptance rate=5%
Acceptance rate=5%
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●●
●
●●●●●●
●
●
●
●
●●●
●●●●●●●●●●
●●
●
●●
●
●
●●●
●●
●●
●●●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●●●●●●●●
● ●
●
●
●
●
●
●
●
●●●●
●●
●
●●
●●●
●●●
●●
●
●●●●
●
●●●●●●●●
●
●
●
●●
●
●●
●●●●●●●
●●
●
●
●●●
●
●● ●
●
●
●
●
●●●●
●
●
●
●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●
●●●
●
●
●●●●
●●●●●●●●
●
●●
●
●
● ●
●●
●●●
●
●
●●●●●
●●●●●●●●●●
●
●●
●●●
●
●●
●
●●
●●
●● ●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●
●
●●
●●
●●
●
●
● ●●
●
●
●●
●●
●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●
●●
●
●
●
●●●●●●
●●●●●
●
●●
●
●
●●
●●●●●●
●●
●
●●●●●●●●●●
●●●
●●
●●
●
●
●●●●
●●
●
●
●
●●
●
●
●
●
● ●
●
●●
● ●
●
●
●●●●●●●
●
●●●●●
●●
●
●●●●
●
●
●●●●●●●●
●●●
●
●
●●●●●●●●●
●
●●
●
●
●●●●
●●
●●●
●
●●
●●●
●●●●●●●
●●
●●
●
●●●●●●●●●●●●●●●●●●●●
●●●
●●●●●●●●●●●●●
●
●
●
●
●●●●●●●●●
●
●
●●●●
●
●
●●●
●
●
●●●
●
●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●●●●
●
●
●●
●●
●●
●●●
●
●●●
●●
●
●
●●
●●●
●●●●●●●●●●●●●●●●●●●●●●
●
●●●
●
●
●●●
●
●●●
●●
●
●●
●
●
●●●●
●●●●
●●
●
●
●
●●●●
●●
●
●●●
●●●
●
●
●
●●
●●
●
●
●●●●
●
●
●
●●●●●
●
●●
●
●
●
●●
●●●●●
●
●
●●
●
●●
●●●●
●
●
●●●●
●●●●●●●●●●
●●●●●●●●
●●●●●
●●●
●●●●●
●●●
●
●●●●●
●●●
●
●
● ●●●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●●
●●●●●●●
●
●
●
●
●●●●●●
●●
●
●
●●
●●
●●●●●●●
●●●●
●●●●●●●●●
●
●
●
●●●
●
●●
●
●
●●
●
●
●
●●
●●●●
●●●
●
●
●
●
●
●
●●●●●●●●●●●●●●
●
●●
●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●●
●●
●●●●
●●
●
●●●●
●●●●
●
●
●
● ●
●●●
●●●●●●
●
●●●
●
●●●●●
●●●●●
●
●●●●●●
●
●●
●
●
●
●●●●●●●●●●●●●●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●●●●●●●●●
●●
●
●
●
●●●●●●
●
●●
●●●●●
●
●
●●●
●
●●●●
●●
●●
●
●
●●
●●
●
●
●
●
●
●●
●●●●●●
●
●
●
●
●●
●
●●
●●●●●●●●●●●
●●
●
●
●●
●
●
●●
●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●● ●
●●
●●●
●●●●●●●
●
●
●
●●
●●
●●●●●●●●●●●●●●●●●●●●
●●
●
●●
●●
●
●
●
●
●●●●●●●●●●●
●●●●
●
●
●
●
●
●
●●
●
●
●●●●
●
●●●
●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●
●●
●●●●●
●●
●
●●
●●●●
●●●
●●
●
●
●
●●●●●●●
●●●●●●●●●
●
●
●●
●
●
● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●●
●●
●
●
●●
●●
●●●●
●
● ●●●●●
●●●
●●
●●
●
●● ●●●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●●●●●●●●●●●
●
●
● ●
●
●
●●●●●
●●
●
●
●●
●
●●
●●●●●●●●
●
●●
●●●
●●●●
●●
●
●
● ●●●●
●
●
●●
●●
●●
●
●
●
●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●
●
●
●●
●●●●
●●
● ●●●●
●
●
●
●●●●●●●●●●●
●●●
●●●●
●●●
●
●
●●●
●●
●
●●
●●
●●
●
●
●●
●●●●●
●●●●●
●
●●●
●●
●●●●●●●
●●●●●●●
●
●
●
● ●
●
●●●●
●●●●●●●●●●●●
●
●
●
●
●●●●
●●● ●
●
●
●
●●
●
●●●
●
●
●
●●●
●●●●●
●
●●●●
●●●
●
●●●
●
●
●
●
●
●
●●● ●
●
●
●
●●●●●
●
●●● ●●●●●●●●●
●●●●●●
●
●
●●
●●
●
●
●●●●●●●
●
●
●
●●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●
●●●
●
●
●●
●●
●●●
●
●●●
●●●●
●●●●●●●●●●●
●●●●
●
●
●
●
●●
●
●
●●
●
●
●●●●●●●
●●●●●●●●●●●
●●●●●
●●
●
●●
●●●
●●●
●●●●
●
●●
●
●
●
●●
●●●
●
●
●
●
●●●
●●●●●
●
●●●
●
●
●
●
●●●●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●●●●
●
●
●
●●
●
●
●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●
●●
●●●●●●●●
●●●●●●●●
●●●
●●
●●
●
●
●
●●
●
●
●●●●●
●
●
● ●
●
●
●●●●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
●
●●●●
●●●●●
●●●●●●●●●●●●●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●●
●●●●●
●●
●●●●
●
●
●●
●●●●●
●
●●●
●
●
●
●
●
●●
●●●
●●
●●
●
●●●●●●
●
●
●●●
●●
●
●●●
●
●●
●●
●●●
●
●● ●●●●●●●●●●●●●●●●●●●
●●
●
●●●●●●●●●●●●●●
●
●
●●●●●●●●●●●●
●●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●
●
●●●
●●●●●●●●●●●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●●●●
●
●●●●●●
●
●
●●●●●
●
●
●
●
●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●
●●●●●
●●●
●●
●
●
●
● ●●●● ●●
●
●●
●
●●●
●
●
●
●
●
●
●●
●●●
●
●●●●●●●●●●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●
●●●
●●●●●●●●●
●
●
●
●●●
●●●
●
●
●●●
●●
●
●
●
●●●●
●●●
●
●
●●
●
●●●
●
●
●●
●
●●
●
●
●●●
●
●●●●●
●●●●
●
●●●●●●●
●●
●
●
●
●
●●
●
●●●
●
●●
●●●
●●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●●●
●
●●
●
●●●●●
●
● ●●●●
●●●●●
●●●●●
●
●
●
●
●
●
●
●●●●●●●●●
●●●
●
●
●
●
●
●
●●●●●
●
●●●●
●
●●●●●
●●●●●●
●●
●●
●●
●●
●
●
●
●●●●●
●●
●●
●
●●
●●●
●●
●
●
●●
●
●
●
●●
●●●●●
●●●●●●●●●
●●
●
●●
●
●
●
●●
●
●
●
●●●
●●
●
●●
●●
●●●
●
●
●●●
●
●●●
●●●●●●●●●●●●●●●●
●
●●●●●●●●●●
●
●●●
●●●
●●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●●●●●●
●●●●●
●
●
●●
●●
●
●
●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●
●●●●●●●
●●●●●●
●
●●
●
●●
●
●●●
●
●
●●
●
●●●●●●
●
●●
●
●
●●
●
●
●
●●●●
●
●●●
●
●
●
●
●
●
●
●
●
●●●●●●
●●● ●
●●●
●
●●●●●●●●●●●●●●●●●
●●●
●
●●
●
●●●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
● ●
●●●●●●
● ●
●●●
●
●
●
●●●●●●●●●●●●
●
●
●
●●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●● ●
●
●
●●●
●●●
●●
●
●
●
●
●●●
●
●
●
●●●
●
●●
●●●
●
●●●●●●
●●●
●
●●
●
●●●
●
●
●
●
●
●●
●●
●●
●●●
●
●●●●
●●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●
●
●●●●
●
●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●
●●●
●●
●
●
●
●●
●●
●●
●●
●
●
●●
●
●●
●
●●●
●
●●●
●
●●
●
●●●●
●
●●
●●●
●
●●
●●
●●●
●●
●
● ●●●●●●●●●●
●●
●
●
●●
●●●
●●
●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●
●●●
●
●●
●●●●
●
●
●●●
●●●●●●
●
●
●●
●●
●
●●●●●●●●●●●●●●●● ●●
●
●
●●●
●
●
●●
●
●
●
●●●●●●●●●●●●●●●●
●●
●
●●●
●
●
●
●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●●●
●●
●
●
●
●
●
●●●●●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●●●●
●●
●●
●●●
●●
●●●●
●●
●●
●
●●
●
●
●●
●
●
●
●
●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
−2 0 2 4 6
02
46
8
tuning=0.5 Initial value=(0,7)
Acceptance rate=29.4%
Acceptance rate=29.4%
●
●●
●
●●●●●
●●●●●
●●●●●
●
●●●●●●●●●●
●
●
●●
●
●
●
●
●●
●
●●
●●●
●
●●
●
●●
●
●●
●●●●●●●●●●
●●
●
●
●●
●
●
●
●●●●
●●●●
●●
●
●●●●●
●
●●
●●●●●
●●●●●●●●●
●●
●
●●●
●●●
●●●
●●●
●●●●●
●●
●●●●●●●●●●●●
●●●●
●●●
●●
●
●●●●●●●●●●
●●●●
●●●●●●
●●●
●
●●●●
●
●●●
●●●●●●●
●●
●●●●●
●●●●
●●●●●●●●●●●●●●
●
●●
●
●
●●●●●●●●●
●●●
●
●●●
●●
●●●●●●●●●●●●●
●
●●●●●●●●
●●●●●●●
●
●●●●●
●●
●●
●
●
●●
●●
●●●
●●●
●
●●●●
●
●●
●
●
●●●
●●●●●●
●
●●●●●
●
●●●●●●●●
●●
●
●●●
●
●●●
●●●
●●●●●●●●●
●●●
●
●●
●●●
●
●
●
●●●●●
●
●●
●●
●●●●
●
●●
●
●●●●●●●●●●●●●●●●●●●● ●●
●●●●●●●●●●●●
●
●●
●●●●●●●●
●
●
●●●●●●
●
●●
●●
●●●●
●●
●●●●●
●
●
●
●
●●●●●●●
●●
●●●
●●
●●●●●●●●●
●●●●
●●●●
●●●●●
●●●
●●
●●●●●●
●●
●●●
●●●
●
●●
●●●●
●
●●●●●●●●
●●●●●
●
●
●●●●●●
●●●●
●●●
●● ●
●●
●
●●●
●●●●●●●
●
●●●●●●●●●
●●●
●●
●●
●
●●
●●●●●●●●
●●●● ●●●●●●
●●●●●●
●●●
●●●●
●
●
●●●
●●●●●
●●●●●●●● ●
●●●●●
●●●●
●●●●●
●●
●●●
●
●●●●●●●
●●●
●●
●●●●●●●●●●●●
●●●●
●●●●
●●
●●●●●●●
●
●●
●
●
●●●●●●●
●
●●●●●
●
●●●●●
●●
●
●
●
●●●●
●●●●
●●
●
●●●
●
●●●
●●
●
●●
●●
●
●
●●
●●●●
●●
●●●●
●●●●
●●●●
●
●●
●
●●●●
●●●
●
●
●●●●
●●●●
●●●●●●
●
●
●●
●●●●
●●
●
●●●
●
●●
●●●●
●●●●●
●●
●●●●●●●●●●
●
●●●
●●●
●●●●
●
●●●
●●●●●●●●●
●
●●●●●●●●●●
●●●●
●●●●
●●●●●●●●
●●
●●
●
●
●●
●●●●
●
●
●●●●●
●●●●
●
●
●
●
●●●●●●
●●●●●●
●●●●●
●●●●●
●
●●
●●●●●
●●●●●
●
●●●
●
●●●●●
●●
●●
●●●
●●●●●
●●●●●●●●●●
●●
●●●●●●●●●●●●●●●●●
●●●●
●
●●●
●●●●●●●●●●●●●●
●
●
●●●●
●●●●●●●●●●●●●●●●●●
●
●●●●
●●
●
●
●●
●●●●●
●
●●●●●●●●●
●●●●
●●
●● ●
●●●●●●●●●●●●●
●●
●●●●●●●
●●●●●●●
●●
●●
●●●●●●●●●●●●
●●●
●●●●●
●
●
●
●●
●
●
●
●
●●●
●
●●
●●
●●●●●
●●●●●●●●●●●●
●
●●
●
●●●●
●
●●●●
●●
●
●●
●
●
●
●●●
●●●●●
●
●●●
●●●
●●●
●●
●
●
●
●●
●●
●
●●●
●
●
●
●
●●●●●
●●●●●
●●●
●●
●●
●●●●●
●
●●●
●●●●
●
●●●
●
●
●●
●
●●● ●
●●●●●
●●●●●●
●●●●●
●●●
●●●
●●●●●●●●●●
●●●●●●
●●
●●●●●●
●●●●
●●●
●●●●
●●●●●●
●●●
●●
●
●
●●●●
●
●●
●
●
●
●●●
●●●●●
●●●
●●
●
●●●●●●●
●●
●
●
●●●●
●
●●●●●●
●●
●●
●
●●●●●●●●●●
●●●●●●●
●
●●●●
●
●
●●●●●
●●●
●●
●
●●●
●
●●●●●
●
●●●●●●●●●●●●●●●●●
●
●
●● ●●
●●
●●
●
●●●
●●
●
●●
●
●●●●
●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●
●
●●●●●●●●●●●●●
●●●●●●●●
●●●●
●●●●●●
●
●●
●●●
●●●●
●
●
●●●
●●
●●●●
●●
●
●●●
●●●●
●●●
●●●
●●●
●
●
●●●
●
●●●
●●
●●
●●●●●●●●
●●
●
●
●●●●
●●●●
●●●
●●
●●●
●●
●●●●●●
●
●
●●
●●●●●●●●●●●●●●●●●
●●
●
●●●●●●●●
●●●●●●
●●●●●●
●●
●
●●●●●
●
●
●●●●●●●●●●
●
●●
●●
●●●●●
●
●●●●●●●●●
●●●●●●●●●
●
●●●●
●●
●●●
●
●●●●●●●●
●●●
●●●●●●
●●●●●●●●
●
●●●●●●●●●●
●●●●●●●
●
●●
●●
●●
●●
●●●
●●●
●
● ●●●●●●●●
●●
●●
●
●
●
●●●
●●●●●
●●●●●
●
●●
●
●
●●●
●
●
●●●●●
●●
●
●●
●●●●●●●
●
●●
●●
●●●●
●
●
●●●
●
●
●●●●●
●
●●
●●●
●●
●●
●●●
●●
●●
●●
●●●
●●
●
●●●●
●
●
●●●●●●●
●●●●●●●●●●●
●●●●
●●
●
●
●●
●●●
●●
●
●●●●●
●
●●●●●●●
●●●●●●●
●
●●●
●●●
●
●●●●●●●●●
●●●●●
●●
●
●●
●●●●●●●●
●●
●●
●●
●●●●●●●
●●●●●●●
●●●●●●●
●●
●●●●
●
●
●●
●
●●●●●●●●●●●●●●●●
●
●
●
●
●●●
●●
●●●●
●●●
●
●●●●
●
●●●●
●●●
●
●●
●●●●●
●●●●●
●●●
●●
●
●●
●
●●●
●
●●●●
●●●
●●●●
●●●●●●●●
●
●●
●●●
●
●
●
●●
●●
●●●
●
●●
●●
●●
●
●●●●●
●●●●●●●●●●●●●
●●●●●●●●●●
●●● ●
●●●●●●●
●●●●●●●●●●●
●
●●●●●●●
●●●
●●●●
●
●●●●●●●
●●●
●●
●
●●●●●
●●●
●●●●●●
●●●●
●
●
●●●●●●●●●
●
●
●●
●●●
●
●●●●
●●●●●
●●●
●
●●●●●●●●●●●●●●●
●●●●
●●
●●●●●●●●
●●●●●●
●●●
●
●
●●
●●●
●●
●
●●●
●●
●●●●●●
●●
●●●●
●●●
●
●●●●●●●
●●●●●●
●●●●●
●●
●●●●
●●●●●●●●●●●
●●
●●●●●●●●●●●●●●●
●
●●●
●●●
●●●
●●
●●
●
●
●
●●●
●●●
●●
●●●●●●●●●●●●
●●●
●●
●●● ●
●●●
●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●
●●
●●●●●●●
●
●
●●●●●●●●●●
●●
●●
●●●●●●●
●●
●
●●●
●●●
●●●
●
●●
●●●●
●●●●●
●
●●●●●
●
●
●
●●●●●●●●●●●●●
●●●●●●●
●●●
●
●●
●●●●●
●
●●●●●
●●
●
●●●● ●●
●
●
●●●●●
●●●
●
●●●●●●
●●●●●●●●●●●●●●●
●●●●
●●●●●●
●
●
●●●
●●
●●●●●●●
●●●
●
●●●●●
●
●
●
●
●
●●●●
●●●●●●●●●●●●●●
●●
●●●●●●
●●
●●
●●
●●
●●●●●●
●●●●●●
●●●●
●
●●●
●
●●
●
●●●●●●
●●●●
●
●●●
●●●
●●●
●
●
●
●●●
●●●
●
●●●
●
●
●●●●●●●●●●●●●●●●● ●●●
●●●●
●●
●●●●●●●●
●●●
●
●
●
●
●●
●●●●●●●●●●●●●●●●
●●●
●
●
●
●●
●●●●●
●●
●
●●●●●●●●●●●●●
●
●●●●●●●●
●
●
●●●●
●
●
●●●●●
●●●●●●
●●
●
●
●●
●●●
●●●●●●●●●
●●●●●
●●●●●●●
●●●●●●
●●●●●●●
●●
●
●
●
●●●●●●
●●
●●●●
●
●
●●
●
●
●●●
●●●
●●●●
●●
●●
●●●
●●●
●●●●●●●●●●
●●●●●●●●●●●●●●
●●●
●
●
●●●●●
●
●●
●
●●
●
●
●●●●
●●●●
●
●
●●
●●●
●●●●●
●
●●●●
●
●●●●
●●●
● ●●●●●●
●
●●●●●●
●●●
●●●●
●
●
●
●
●●●●●
●
●●●●
●●
●
●●
●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●
●●●●●●●●●●●●●●
●●
●●●
●●
● ●
●
●
●●●●●●
●●●●●
●●
●●●
●●●
●●●
●●●●●●●●●
● ●
● ●●●●●●●
●
●●●
●●●●
●
●
●
●
●
●●●
●●●●●●●●●●●●●●●●●●
●●●●●●
●●●
●●
●●●●●●●
●●●●
●●
●
●●
●●●●●
●●
●●●●●●
●●
●
●
●●
●●●
●●
●●●
●●●
●●
●
●●
●●
●●
●●●
●●●●
●
●●●
●●
●●●
●
●
●
●●
●●
●●●●●
●●●
●
●
●
●● ●●●●
●●●●●●●●●●●●●●
●●●●●●
●●
●
●●●●● ●●●●
●
●●●●●●●●●
●●●
●
●
●●●●
●●
●●
●●
●
●●
●
●●●●
●
●●
●●●
●
●
●●
●●●●
●
●
●●●
●
●●
●●
●●
●●
●
●●
●●●
●●●
●
●●●
●●●●●●
●●
●●●●●●
●●●●●●
●●●
●●●●
●●●● ●
●●●●
●●
●
●●●●
●
●
●●●●●●●
●●●●●●●●●
●
●
●●
●●
●●
●●●●●
●●●●●●●●
●
●
●
●●
●●●
●●●●●●
●
●
●
●
●
●●●●●
●
●●
●●●●●●●●●●
●●
●●●●●●●●●●
●●●●●
●●
●●●●●●●
●●
●●●●●●●●●
●●
●●●●●
●
●
●●
●●
●●
●
●
●●●●
●
●●
●
●●
●●●● ●●●●●
●●●●●●●
●●●
●●
●●●
●●●●●
●
●
●●●
●●●
●●●●●
●
●●
●●●
●●●●
●
●
●
●
●●●●
●
●
●●●●●
●
●
●●●●
●●●
●●
●
●
●●●
●
●
●
●●●●●●●●
●●●●●
●●●●●●
●●
●
●●
●
●●●●
●●●●●●●●●●
●●
●●●●
●
●
●
●●●●●●
●●●●●●
●●●
●●●
●●●●●
●●●●●
●
●●●●●●●●●●●
●●
●●●●●●
●●●●
●
●●●●●●●●●●●
●
●
●
●●●
●
●●
●
●●
●●●●●
●●●
●●●●●●●
●●
●
●
●●●●●●●
●●●●●
●
●●●●●●●●
●●●●●
●●●
●
●●
●●●●●●
●●
●
●●●●●
●
●
●●●●●●●●
●●
●●
●●●●
●●●
●●
●●●●
●
●
●●●●●●
●●
●●
●
●●●
●
●●
●●
●●●●●●
●●
●
●●●●
●
●●●●●
●●
●
●●
●●●●●●●
●●●●●●●●●●●●
●
●●
●●
●●
●●
●●
●
●
●●●
●
●
●●●●●●●
●●●●
●●●
●●
●●
●●
●
●●
●
●●
●●
●●●●
●●●
●●●●●●
●●
●●●●●●●
●●●●●●
●●
●
●●●●●●
●●
●●●●●●
●●●
●●
●●●●●●●●●●●●●●
●
●●
●
●
●●●●
●●
●●●●●●
●●●
●●
●●●●●
●●●●●●●
● ●
●●●●●●●●
●
●
●●●●
●
●
●
●●
●●
●
●●
●
●
●
●
●●●
●●●●●
●●●●●●
●●●●●●
●●
●●●●●●●●●●●●●●
●●●●●
●●
●●
●
●●●●●●●
●●●
●●●
●●●●●●●●
●●
●●●●●●●●●●●●●●●●●
●●●
●
●●●
●●●●●
●●●●●
●
●●●●
●●●●●●●●●●
●●
●●●●●
●
●
●
●●
●●●●●
●
●●●
●
●●●●
●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●
●●
●●●●
●●●
●●
●●
●
●
●
●●
●●
●●●●●
●
●●●
●
●●
●
●
●●
●
●●
●
●
●●●●
●
●●●
●
●
●●●●●●●●●●●●
●
●
●●
●●●●
●●
●
●●●●
●●●●
●
●●●
●
●●
●●●●
●●●●
●
●●●●●●●
●●
●●●●●●●●●●
●
●●●
●●
●●●●●
●●●●●●●●●●●
●
●
●●●
●●●●●●●●●
●●●●●●●●●●
●
●
●●
●●
●●●●
●
●●●
●●●
●
●●
●●●●
●●●
●●
●
●●●●
●●●●●●●●●●●●●●●●●●● ●●●●●●●
●
●
●●●
●
●●●●
●
●●●
●●●
●●●
●
●●●
●
●
●
●●●●●●●●●
●●
●
●
●●
●●●●●
●●●●●
●●
●●●●●●●●●
●●●●●
●●●
●●●●●
●
●
●
●●●●
●
●
●●●●●
●●●
●●
●
●
●●●●●●●●●●●●
●●
●●●●●●
●●●
●●
●●
●●●●●●●●●
●
●●●●●
●●●
●
●●
●●●●● ●
●●●●●
●●●●
●
●●●●
●
●●●●
●●
●●●●
●
●
●●●●
●●●●
●●
●
●●●●
●●●●●●●●●●●●●
●
●
●
●
●●●●●●●●●●●●●●●●●●●
●●●●
●●●●●
●
●
●●●●
●●
●●
●●●
●
●●●
●
●●●
●
●●●●●●●●●●●●●●●●●●●●
●
●
●
●●●●●
●●
●●●●●●
●
●●
●
●●●●●
●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●●●●
●●●
●●●●●●●●●●
●
●●●●●●●●●●●
●
●●●●
●●●●●●●●
●●●●●
●
●
●●●●
●
●●
●●●
●
●●●●
●
●
●●●●●●
●●
●●●
●●●
●
●●●
●●●●●●●●●
●●
●●
●●●●
●●●
●●●●●
●●●●●●
●
●●●●●●●●●●●●
●
●●
●●●●●●
●●
● ●●
●●●●●●●●●●
−2 0 2 4 6
02
46
8tuning=5
Initial value=(0,7) Acceptance rate=32.2%
Acceptance rate=32.2%
●●●
●●●●●●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●
●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●
●●●●●●●●●●●●●●●●●●●●●
●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●
●●●●●●●●
●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●
●●●●
●●●●
●●●●●
●●●●●●●●●●●●
●●●●
●●
●
●
●●●●●●●●●●●●●●●●
●●●●
●●●●●
●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●
●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●●●
●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●●
●
●●●●●●●●●
●●●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●●
●
−2 0 2 4 6
02
46
8
tuning=50 Initial value=(0,7)
Acceptance rate=4.3%
Acceptance rate=4.3%
18 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Autocorrelations
0 50 100 150 200
0.0
0.2
0.4
0.6
0.8
1.0
lag
Tuning=0.5 Initial value=(4,5)
0 50 100 150 200
0.0
0.2
0.4
0.6
0.8
1.0
lag
Tuning=5 Initial value=(4,5)
0 50 100 150 200
0.0
0.2
0.4
0.6
0.8
1.0
lag
Tuning=50 Initial value=(4,5)
0 50 100 150 200
0.0
0.2
0.4
0.6
0.8
1.0
lag
Tuning=0.5 Initial value=(0,7)
0 50 100 150 200
0.0
0.2
0.4
0.6
0.8
1.0
lag
Tuning=5 Initial value=(0,7)
0 50 100 150 200
0.0
0.2
0.4
0.6
0.8
1.0
lag
Tuning=50 Initial value=(0,7)
19 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Simulated annealing
Simulated annealing1 is an optimization technique designed tofind maxima of functions.
It can be seen as a M-H algorithm that tempers with the targetdistribution:
q(θ) ∝ π(θ)1/T
where the constant T > 1 receives the physical interpretationof system temperature, hence the nomenclature used(Jennison, 1993).
The heated distribution q is flattened with respect to π and itsdensity gets closer to the uniform distribution, which isparticularly relevant for the case of a distribution with distantmodes.
By flattening the modes, the moves required to coveradequately the parameter space become more likely.
1Kirkpatrick, Gelatt and Vecchi (1983)20 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Example vii: Nonlinear surface
Assume that the goal is to find the mode/maximum of
π(β1, β2) ∝4∏
i=1
e(β1+β2xi )yi
(1 + eβ1+β2xi )5,
with x = (−0.863,−0.296,−0.053, 0.727) and y = (0, 1, 3, 5).
The simulated annealing algorithm is implemented for fourinitial values:
(5, 30) (−2, 40) (−4,−10) (6, 0)
and two cooling schedules:
Ti = 1/i and Ti = 1/[10 log(1 + i)].
The proposal distribution is q(β|β(i)) = fN(β;β(i), 0.052I2).21 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Ti=1/i
ββ1
ββ 2
−4 −2 0 2 4 6
−10
010
2030
40
ββ1
Iterations
0 1000 2000 3000 4000 5000
−4
−2
02
46
ββ2
Iterations
0 1000 2000 3000 4000 5000
−10
010
2030
40
Ti=1/[10log(i+1)]
ββ1
ββ 2
−4 −2 0 2 4 6
−10
010
2030
40
ββ1
Iterations
0 1000 2000 3000 4000 5000
−4
−2
02
46
ββ2
Iterations
0 1000 2000 3000 4000 5000
−10
010
2030
40
Newton-Raphson mode: (0.87, 7.91).
Ti = 1/i : mode is (0.88, 7.99) when (β(0)1 , β
(0)2 ) = (5, 30).
22 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
3000 3500 4000 4500 5000
0.82
0.86
0.90
0.94
ββ1
Iterations
3000 3500 4000 4500 5000
7.7
7.9
8.1
ββ2
Iterations
23 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Gibbs samplerTechnically, the Gibbs sampler is an MCMC scheme whosetransition kernel is the product of the full conditionaldistributions.Algorithm
1 Start at θ(0) = (θ(0)1 , θ
(0)2 , . . .)
2 Sample the components of θ(j) iteratively:
θ(j)1 ∼ π(θ1|θ(j−1)
2 , θ(j−1)3 , . . .)
θ(j)2 ∼ π(θ2|θ(j)
1 , θ(j−1)3 , . . .)
θ(j)3 ∼ π(θ3|θ(j)
1 , θ(j)2 , . . .)
...
The Gibbs sampler opened up a new way of approachingstatistical modeling by combining simpler structures (the fullconditional models) to address the more general structure (thefull model).
24 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Example viii: Bivariate normal
Assume that the target distribution is the bivariate normal withmean vector and covariance matrix given by
µ =
(µ1
µ2
)and Σ =
(σ2
1 σ12
σ12 σ22
),
respectively.
In this case, the two full conditionals are given by
θ1|θ2 ∼ N
(µ1 +
σ12
σ22
(θ2 − µ2), σ21 −
σ212
σ22
)and
θ2|θ1 ∼ N
(µ2 +
σ12
σ21
(θ1 − µ1), σ22 −
σ212
σ21
)
25 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
µ1 = µ2 = 0σ2
1 = σ22 = 1
σ12 = −0.95
θθ1
θθ 2
−4 −2 0 2 4
−4
−2
02
4
26 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
−4 −2 0 2 4
−4
−2
02
4
θθ1
θθ 2
27 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
−4 −2 0 2 4
−4
−2
02
4
θθ1
θθ 2
28 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
−4 −2 0 2 4
−4
−2
02
4
θθ1
θθ 2
−4 −2 0 2 4
−4
−2
02
4
θθ1
θθ 2
−4 −2 0 2 4
−4
−2
02
4
θθ1
θθ 2
−4 −2 0 2 4
−4
−2
02
4
θθ1
θθ 2
0 10 20 30 40
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
0 10 20 30 400.
00.
20.
40.
60.
81.
0
Lag
AC
F
0 10 20 30 40
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
0 10 20 30 40
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F0 5 10 15 20 25 30
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
Middle frame: Based on M = 21, 000 consecutive draws.Bottom frame: Based on M = 1000 draws, after initialM0 = 1000 draws and saving every 20th draws.
29 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●● ●●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
−4 −2 0 2
−2
02
4
θθ1
θθ 2
θθ1
Den
sity
−4 −2 0 2 4
0.0
0.1
0.2
0.3
0.4
θθ2
Den
sity
−4 −2 0 2 4
0.0
0.1
0.2
0.3
0.4
30 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Simple random effects model
Let response (dependent variable) yi be linearly related toregressor (explanatory variable) xi in the following way:
yi = βxi + θi + εi εi ∼ N(0, σ2)
where θi is the random effect modeled by
θi ∼ N(µ, τ2)
with εi and θi uncorrelated and xi known.
The parameters of the model are
(β, σ2, µ, τ2) and θ = (θ1, . . . , θn).
31 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Prior and simulated data
We assume the following independent prior distributions:
β ∼ N(b0,C0) σ2 ∼ IG (a, b)
µ ∼ N(µ0,V0) τ2 ∼ IG (c , d)
We simulated n = 300 pairs (yi , xi ) based on β = 5, σ2 = 0.2,µ = 0, τ2 = 0.1 and xi ∼ U(0, 1).
In this case, the hyperparameters are a = 6,b = 1, c = 3,d = 0.2, b0 = 0, C0 = 100, µ0 = 0 and V0 = 100, such thatE (σ2) = 0.2, E (τ2) = 0.1 and V (σ2) = V (τ2) = 0.01.
32 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Full conditional distributions
(σ2|β, θ, y , x) ∼ IG(a + n/2, b +
∑ni=1(yi − βxi − θi )2/2
)(τ2|θ, µ) ∼ IG
(c + n/2, d +
∑ni=1(θi − µ)2/2
)(β|θ, σ2, x , y) ∼ N(m,C ), where C = 1/(1/C0 +
∑ni=1 x2
i /σ2)
and m = C (b0/C0 +∑n
i=1(yi − θi )xi/σ2)
(µ|θ, τ2) ∼ N(m,C ), where C = 1/(1/V0 + n/τ2) andm = C (µ0/V0 +
∑ni=1 θi/τ
2)
For i = 1, . . . , n, (θi |yi , xi , β, σ2, µ, τ2) ∼ N(mi ,C ), where
C = 1/(1/τ2 + 1/σ2) and mi = C (µ/τ2 + (yi − βxi )/σ2)
33 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Posterior predictive
Let γ = (β, σ2, µ, τ2), D = (y , x), y = yn+1, x = xn+1 andθ = θn+1. Then
p(y |x ,D) =
∫p(y |x , θ, γ,D)p(θ|γ,D)p(γ|D)d θdγ
=
∫p(y |x , θ, γ)p(θ|µ, τ2)p(γ|D)d θdγ
=
∫fN(y |βx + θ, σ2)fN(θ|µ, τ2)p(γ|D)d θdγ
with a Monte Carlo approximation given by
p1(y |x ,D) =1
M
M∑m=1
fN(y |β(m)x + θ(m), σ2(m))
where θ(m) ∼ p(θ|µ(m), τ2(m)) (random effects distribution)and γ(m) ∼ p(γ|D) (MCMC draws), for m = 1, . . . ,M.
34 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Posterior predictive mean
Similarly, a MC approximation to the mean of the posteriorpredictive is given by
E1(y |x ,D) =1
M
M∑m=1
E (y |x , θ(m), γ(m))
=
(1
M
M∑m=1
β(m)
)x +
(1
M
M∑m=1
θ(m)
)= βx + ¯θ
35 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Posterior predictive draws
Draws y (m), for m = 1, . . . ,M, from the posterior predictivecan be obtained from draws γ(m) and θ(m) by simply samplingy (m) from N(β(m)x + θ(m), σ2(m)).
An alternative MC approximation to E (y |x ,D) is given by
E2(y |x ,D) =1
M
M∑m=1
y (m).
It is easy to show that the MC variance of E1 is at most aslarge as the variance of E2 (Rao-Blackwell result) -Raoblackwellization.
Similarly, an alternative MC approximation to p(y |x ,D) isgiven by the histogram (kernel approximation) of the drawsy (m), namely p2(y |x ,D).
36 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Posterior summaryInitial values: β = 5.0, µ = 0, θ = θtrue
MCMC set up: (M0, L,M) = (50000, 50, 5000)
beta
iterations
0 1000 2000 3000 4000 5000
4.8
5.0
5.2
5.4
0 5 10 15 20 25 30 35
0.0
0.4
0.8
Lag
AC
F
Den
sity
4.8 5.0 5.2 5.4
01
23
sig2
iterations
0 1000 2000 3000 4000 5000
0.05
0.15
0.25
0 5 10 15 20 25 30 35
0.0
0.4
0.8
Lag
AC
F
Den
sity
0.05 0.10 0.15 0.20 0.25 0.30
02
46
8
mu
iterations
0 1000 2000 3000 4000 5000
−0.
3−
0.1
0.1
0 5 10 15 20 25 30 35
0.0
0.4
0.8
Lag
AC
F
Den
sity
−0.3 −0.2 −0.1 0.0 0.1
01
23
45
6
tau2
iterations
0 1000 2000 3000 4000 5000
0.05
0.15
0.25
0 5 10 15 20 25 30 35
0.0
0.4
0.8
Lag
AC
F
Den
sity
0.05 0.10 0.15 0.20 0.25
02
46
810
37 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Predictive: p1(y |x , D)
y
−20
24
68
x
0.2
0.4
0.6
0.8
predictive
0.0
0.2
0.4
0.6
x
y
0.0 0.2 0.4 0.6 0.8 1.0
−2
02
46
8
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●●
●●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
● ●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●● ●●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●● ●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
38 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Predictive: pi(y |x , D) for i = 1, 2
x=0.001
y
Pre
dict
ive
−2 −1 0 1 2
0.0
0.2
0.4
0.6
x=0.334
y
Pre
dict
ive
0 1 2 3
0.0
0.2
0.4
0.6
0.8
x=0.667
y
Pre
dict
ive
2 3 4 5
0.0
0.2
0.4
0.6
x=1
y
Pre
dict
ive
3 4 5 6 7
0.0
0.2
0.4
0.6
E1, E2, p1 and p2 (histogram).39 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Monte Carlo errorE1 and E2 are computed based on 50 sets of 100 MCMC drawsfor each value of x . The picture plots the standard deviationsover the 50 sets.
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
0.0 0.2 0.4 0.6 0.8 1.0
0.02
0.04
0.06
0.08
0.10
x
MC
sta
ndar
d de
viat
ion
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
40 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Book references
1 Ripley (1987) Stochastic Simulation. Wiley. (Chapters 5 and 7).
2 Thisted (1988) Elements of Statistical Computing. Chapman & Hall/CRC. (Chapter 5).
3 Gentle (1998) Random Number Generation and Monte Carlo Methods. Springer. (Chapter 5).
4 Liu (2001) Monte Carlo Strategies in Scientific Computing. Springer. (Chapters 2, 5 and 6).
5 Robert and Casella (2004) Monte Carlo Statistical Methods. Springer. (Chapters 3, 4 and 7).
6 Givens and Hoeting (2005) Computational Statistics. Wiley. (Chapters 6, 7 and 8).
7 Gamerman and Lopes (2006) MCMC: Stochastic Simulation for Bayesian Inference. Chapman &Hall/CRC. (Chapters 3, 5 and 6).
8 Rizzo (2008) Statistical Computing with R. Chapman & Hall/CRC. (Chapters 5 and 9).
9 Rubinstein and Kroese (2008) Simulation and the Monte Carlo Method. Wiley. (Chapters 5 and 6).
10 Robert and Casella (2010) Introducing Monte Carlo Methods with R. Springer. (Chapters 3, 6 and7).
41 / 42
Historicalfacts
MHalgorithms
Special cases
Random walkMetropolis
Simulatedannealing
Gibbs sampler
Example ix.simple randomeffects model
References
Classical papers1 Metropolis and Ulam (1949) The Monte Carlo method. JASA, 44, 335-341.
2 Metropolis, Rosenbluth, Rosenbluth, Teller and Teller (1953) Equation of state calculations by fastcomputing machines. Journal of Chemical Physics, Number 21, 1087-1092.
3 Hastings (1970) Monte Carlo sampling methods using Markov chains and their applications.Biometrika, 57, 97-109.
4 Peskun (1973) Optimum Monte-Carlo sampling using Markov chains. Biometrika, 60, 607-612.
5 Besag (1974) Spatial interaction and the statistical analysis of lattice systems. Journal of the RoyalStatistical Society, Series B, 36, 192-236.
6 Geman and Geman (1984) Stochastic relaxation, Gibbs distributions and the Bayesian restoration ofimages. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721-741.
7 Jennison (1993) Discussion of the meeting on Gibbs sampling and other Markov chain Monte Carlomethods. JRSS-B, 55, 54-6.
8 Kirkpatrick, Gelatt and Vecchi (1983) Optimization by simulated annealing. Science, 220, 671-80.
9 Pearl (1987) Evidential reasoning using stochastic simulation. Articial Intelligence, 32, 245-257.
10 Tanner and Wong (1987) The calculation of posterior distributions by data augmentation. JASA, 82,528-550.
11 Geweke (1989) Bayesian inference in econometric models using Monte Carlo integration.Econometrica, 57, 1317-1339.
12 Gelfand and Smith (1990) Sampling-based approaches to calculating marginal densities, JASA, 85,398-409.
13 Casella and George (1992) Explaining the Gibbs sampler. The American Statistician,46,167-174.
14 Smith and Gelfand (1992) Bayesian statistics without tears: a sampling-resampling perspective.American Statistician, 46, 84-88.
15 Gilks and Wild (1992) Adaptive rejection sampling for Gibbs sampling. Applied Statistics, 41, 337-48.
16 Chib and Greenberg (1995) Understanding the Metropolis-Hastings algorithm. The AmericanStatistician, 49, 327-335.
17 Dongarra and Sullivan (2000) Guest editors’ introduction: The top 10 algorithms. Computing inScience and Engineering, 2, 22-23.
42 / 42
