Computational Physics
Feb 21, 2019
http://hadron.physics.fsu.edu/~eugenio/comphy/
Adaptive, Adaptive, Multi-Dimensional, & Multi-Dimensional, &
Monte Carlo IntegrationMonte Carlo Integration
Computational Physics
Feb 21, 2019
http://hadron.physics.fsu.edu/~eugenio/comphy/
Adaptive, Adaptive, Multi-Dimensional, & Multi-Dimensional, &
Monte Carlo IntegrationMonte Carlo Integration
Series Integration
f(x)
xxa xb
w=x b−xa/N
xa xb
f(x)
x
4 ∑k=0
N −1
f xaw k1/2
I N =w6 f xa f x b2 ∑
k=1
N −1
f x ak w
∫x a
x b
f x dx = I N O w4∫xa
xb
f x dx = I N O w2
2 =115∣I N 2
− I N 1∣
error on 2nd integration
2 =13∣I N 2
− I N 1∣
error on 2nd integration
I N = w2∗ f x a f xb2∑
k=1
N −1
f xak w
trapezoidal rule Simpson's rulereview from last lecture
Series Integration
f(x)
xxa xb
w=x b−xa/N
xa xb
f(x)
x
4 ∑k=0
N −1
f xaw k1/2
I N =w6 f xa f x b2 ∑
k=1
N −1
f x ak w
∫x a
x b
f x dx = I N O w4∫xa
xb
f x dx = I N O w2
2 =115∣I N 2
− I N 1∣
error on 2nd integration
2 =13∣I N 2
− I N 1∣
error on 2nd integration
I N = w2∗ f x a f xb2∑
k=1
N −1
f xak w
trapezoidal rule Simpson's rulereview from last lecture
Adaptive IntegrationIntegration with just enough steps, N, to achieve the accuracy we want
Decide on the integration accuracy Evaluate the integral with a small number of steps N1
Then double the number N2 = 2N1, & evaluate integral again
Calculate the error on I2
use
If error is to within accuracy then we are finished,
otherwise repeat doubling of integration steps until desired accuracy is reached
ϵ2 = 13∣I 2−I 1∣trapezodial or ϵ2 = 1
15∣I 2−I 1∣Simpson
Adaptive IntegrationIntegration with just enough steps, N, to achieve the accuracy we want
Decide on the integration accuracy Evaluate the integral with a small number of steps N1
Then double the number N2 = 2N1, & evaluate integral again
Calculate the error on I2
use
If error is to within accuracy then we are finished,
otherwise repeat doubling of integration steps until desired accuracy is reached
ϵ2 = 13∣I 2−I 1∣trapezodial or ϵ2 = 1
15∣I 2−I 1∣Simpson
Adaptive IntegrationDoubling the number of integration steps with half the calculations
½ 1 1 1 ½
½ 1 1 1 1 1 1 1 ½
doubling effectively adds an additional set of points halfway between the previous points
I N = w 12
f xa12
f xb∑k=1
N −1
[ f x ak w ]
Adaptive IntegrationDoubling the number of integration steps with half the calculations
½ 1 1 1 ½
½ 1 1 1 1 1 1 1 ½
doubling effectively adds an additional set of points halfway between the previous points
I N = w 12
f xa12
f xb∑k=1
N −1
[ f x ak w ]
Adaptive IntegrationDoubling the number of integration steps with half the calculations
½ 1 1 1 ½
½ 1 1 1 1 1 1 1 ½
doubling effectively adds an additional set of points halfway between the previous points
adaptive trapezoidal rule integration(extra points are the odd points)
I N = w 12
f xa12
f xb∑k=1
N −1
[ f x ak w ]
I 2=12
I 1+ w2∗ ∑k odd
1... N 2−1
f ( xa+ k w2)
ϵ2 = 13∣I 2− I 1∣N 2=2N1 w2=
w1
2
Adaptive IntegrationDoubling the number of integration steps with half the calculations
½ 1 1 1 ½
½ 1 1 1 1 1 1 1 ½
doubling effectively adds an additional set of points halfway between the previous points
adaptive trapezoidal rule integration(extra points are the odd points)
I N = w 12
f xa12
f xb∑k=1
N −1
[ f x ak w ]
I 2=12
I 1+ w2∗ ∑k odd
1... N 2−1
f ( xa+ k w2)
ϵ2 = 13∣I 2− I 1∣N 2=2N1 w2=
w1
2
Adaptive IntegrationDoubling the number of integration steps with half the calculations
½ 1 1 1 ½
½ 1 1 1 1 1 1 1 ½
doubling effectively adds an additional set of points halfway between the previous points
I 2=12
I 1+ w2∗ ∑k odd
1... N 2−1
f ( xa+ k w2)
I 2 = 12
I 1 −13
w2 ∑k oddk=1
N 2−1
f (xa+ w2 k ) adaptive trapezoidal rule integration
(extra points are the odd points)
adaptive Simpson's rule integration (extra points are the new mid points minus odd endpoint point values)
I N =w6 f xa f x b2 ∑
k=1
N−1
f x ak w4 ∑k=0
N−1
f x awk1 /2
I N = w 12
f xa12
f xb∑k=1
N −1
[ f x ak w ]
ϵ2 = 13∣I 2− I 1∣ ϵ2 = 1
15∣I 2− I 1∣
+ 23
w2 ∑k=0
N 2−1
f (xa+ w2(k+ 1/2))
N 2=2N1 w2=w1
2N 2=2N1 w2=
w1
2
Adaptive IntegrationDoubling the number of integration steps with half the calculations
½ 1 1 1 ½
½ 1 1 1 1 1 1 1 ½
doubling effectively adds an additional set of points halfway between the previous points
I 2=12
I 1+ w2∗ ∑k odd
1... N 2−1
f ( xa+ k w2)
I 2 = 12
I 1 −13
w2 ∑k oddk=1
N 2−1
f (xa+ w2 k ) adaptive trapezoidal rule integration
(extra points are the odd points)
adaptive Simpson's rule integration (extra points are the new mid points minus odd endpoint point values)
I N =w6 f xa f x b2 ∑
k=1
N−1
f x ak w4 ∑k=0
N−1
f x awk1 /2
I N = w 12
f xa12
f xb∑k=1
N −1
[ f x ak w ]
ϵ2 = 13∣I 2− I 1∣ ϵ2 = 1
15∣I 2− I 1∣
+ 23
w2 ∑k=0
N 2−1
f (xa+ w2(k+ 1/2))
N 2=2N1 w2=w1
2N 2=2N1 w2=
w1
2
Integrals over infinite ranges
∫0
∞
f x dx
Solve by changing variables: z= x1x
x= z1−z
& dx= dz1−z 2
∫0
∞
f x dx = ∫0
1 11−z 2 f z
1−z dz
Integrals over infinite ranges
∫0
∞
f x dx
Solve by changing variables: z= x1x
x= z1−z
& dx= dz1−z 2
∫0
∞
f x dx = ∫0
1 11−z 2 f z
1−z dz
Integrals over infinite ranges
∫0
∞
f x dx
solution is to change variables: z= x1x
x= z1−z
& dx= dz1−z 2
∫0
∞
f x dx = ∫0
1 11−z 2 f z
1−z dz
We can make two changes of variables: y = x - a & z = y/(1+y)
to calculate: ∫a
∞
f xdx = ∫0
1 11−z 2 f z
1−z a dz
Integrals over infinite ranges
∫0
∞
f x dx
solution is to change variables: z= x1x
x= z1−z
& dx= dz1−z 2
∫0
∞
f x dx = ∫0
1 11−z 2 f z
1−z dz
We can make two changes of variables: y = x - a & z = y/(1+y)
to calculate: ∫a
∞
f xdx = ∫0
1 11−z 2 f z
1−z a dz
Multi-Dimensional Integration
∫xa
xb
∫ya
yb
f x , ydy dx
∫xa
xb
f xdx
f x=∫ya
yb
f x , ydy
For N points in each integral calculation there are N2 calculations
Solve by Series Integration
“Divide and Conquer”
TrapezoidalSimpson's Rule
Multi-Dimensional Integration
∫xa
xb
∫ya
yb
f x , ydy dx
∫xa
xb
f xdx
f x=∫ya
yb
f x , ydy
For N points in each integral calculation there are N2 calculations
Solve by Series Integration
“Divide and Conquer”
TrapezoidalSimpson's Rule
Multi-Dimensional Integration
I=∫0
1
dx1∫0
1
dx2 ...∫0
1
dx12 f x1, x2, ... x12
For 100 steps in each integration there are 10012 =1024 calculations
Example: Atomic Physics
3 Dimension/electron * 4 electrons = 12 Dimensions
Assuming 1 Giga evaluations/sec It would take over 107 years!!!!
4Be
Multi-Dimensional Integration
I=∫0
1
dx1∫0
1
dx2 ...∫0
1
dx12 f x1, x2, ... x12
For 100 steps in each integration there are 10012 =1024 calculations
Example: Atomic Physics
3 Dimension/electron * 4 electrons = 12 Dimensions
Assuming 1 Giga evaluations/sec It would take over 107 years!!!!
4Be
Monte Carlo Integration
Monte Carlo methods provide an alternative method of calculating an integral.
Using Random Numbers to Solve Integrals
Monte Carlo Integration
Monte Carlo methods provide an alternative method of calculating an integral.
Using Random Numbers to Solve Integrals
Random NumbersPseudo-Random Numbers
The numbers are pseudo-random in the sense that they are generated deterministically from a seed number, but are distributed in what has statistical similarities to random fashion.
Random NumbersPseudo-Random Numbers
The numbers are pseudo-random in the sense that they are generated deterministically from a seed number, but are distributed in what has statistical similarities to random fashion.
Random NumbersPseudo-Random Numbers
The numbers are pseudo-random in the sense that they are generated deterministically from a seed number, but are distributed in what has statistical similarities to random fashion.
Python modules for generating random numbers: ● import random:
● a smaller set of functions for random numbers● import numpy.random:
● a more complete set of utilities with many generating functions along with array manipulation capabilities
Random NumbersPseudo-Random Numbers
The numbers are pseudo-random in the sense that they are generated deterministically from a seed number, but are distributed in what has statistical similarities to random fashion.
Python modules for generating random numbers: ● import random:
● a smaller set of functions for random numbers● import numpy.random:
● a more complete set of utilities with many generating functions along with array manipulation capabilities
Random NumbersPseudo-Random Numbers
The numbers are pseudo-random in the sense that they are generated deterministically from a seed number, but are distributed in what has statistical similarities to random fashion.
Python modules for generating random numbers: ● import random:
● a smaller set of functions for random numbers● import numpy.random:
● a more complete set of utilities with many generating functions along with array manipulation capabilities
Numpy.random and random modules uses a Mersenne Twister algorithm to generate pseudorandom numbers which has become the generator of choice for serious physics calculations.
Let's always use the NumPy random module
Random NumbersPseudo-Random Numbers
The numbers are pseudo-random in the sense that they are generated deterministically from a seed number, but are distributed in what has statistical similarities to random fashion.
Python modules for generating random numbers: ● import random:
● a smaller set of functions for random numbers● import numpy.random:
● a more complete set of utilities with many generating functions along with array manipulation capabilities
Numpy.random and random modules uses a Mersenne Twister algorithm to generate pseudorandom numbers which has become the generator of choice for serious physics calculations.
Let's always use the NumPy random module
NumPy's random module
import numpy as np
>>> np.random.seed(136)The seed is an integer value. Any program that starts with the same seed will generate exactly the same sequence of random numbers each time it is run. This is useful for debugging programs but otherwise not needed as the seed is uniquely set each time the program executes.
Note: one could also import via:import numpy.random as rnd>>> rnd.seed(136)
NumPy's random module
import numpy as np
>>> np.random.seed(136)The seed is an integer value. Any program that starts with the same seed will generate exactly the same sequence of random numbers each time it is run. This is useful for debugging programs but otherwise not needed as the seed is uniquely set each time the program executes.
Note: one could also import via:import numpy.random as rnd>>> rnd.seed(136)
NumPy's random moduleGenerate random integers in the range [min, max)
>>> np.random.randint(5,10)8
Generates a single random number in [0.0, 1.0)>>> np.random.random()0.70110427435769551
Generate an array of random numbers in the interval [0.0, 1.0) >>> np.random.rand(1)array([ 0.73549029])
>>> np.random.rand(5)array([ 0.6652181 , 0.58861746, 0.8514131 , 0.68607923, 0.8785746 ])
>>> np.random.rand(2,3)array([[ 0.81698429, 0.632073 , 0.10512043], [ 0.88226248, 0.47654622, 0.45082853]])
NumPy's random moduleGenerate random integers in the range [min, max)
>>> np.random.randint(5,10)8
Generates a single random number in [0.0, 1.0)>>> np.random.random()0.70110427435769551
Generate an array of random numbers in the interval [0.0, 1.0) >>> np.random.rand(1)array([ 0.73549029])
>>> np.random.rand(5)array([ 0.6652181 , 0.58861746, 0.8514131 , 0.68607923, 0.8785746 ])
>>> np.random.rand(2,3)array([[ 0.81698429, 0.632073 , 0.10512043], [ 0.88226248, 0.47654622, 0.45082853]])
NumPy's random modulehpc-login% pydoc numpy.random ... ...DESCRIPTION ======================== Random Number Generation ======================== ... ...
==================== ========================================================= Univariate distributions ============================================================================== beta Beta distribution over ``[0, 1]``. binomial Binomial distribution. chisquare :math:`\chi^2` distribution. exponential Exponential distribution. f F (Fisher-Snedecor) distribution. gamma Gamma distribution. geometric Geometric distribution. gumbel Gumbel distribution. hypergeometric Hypergeometric distribution. laplace Laplace distribution. logistic Logistic distribution. lognormal Log-normal distribution. logseries Logarithmic series distribution. negative_binomial Negative binomial distribution. noncentral_chisquare Non-central chi-square distribution. noncentral_f Non-central F distribution. normal Normal / Gaussian distribution. pareto Pareto distribution. poisson Poisson distribution.
...
...
NumPy's random modulehpc-login% pydoc numpy.random ... ...DESCRIPTION ======================== Random Number Generation ======================== ... ...
==================== ========================================================= Univariate distributions ============================================================================== beta Beta distribution over ``[0, 1]``. binomial Binomial distribution. chisquare :math:`\chi^2` distribution. exponential Exponential distribution. f F (Fisher-Snedecor) distribution. gamma Gamma distribution. geometric Geometric distribution. gumbel Gumbel distribution. hypergeometric Hypergeometric distribution. laplace Laplace distribution. logistic Logistic distribution. lognormal Log-normal distribution. logseries Logarithmic series distribution. negative_binomial Negative binomial distribution. noncentral_chisquare Non-central chi-square distribution. noncentral_f Non-central F distribution. normal Normal / Gaussian distribution. pareto Pareto distribution. poisson Poisson distribution.
...
...
Generating random distributions
>>> import numpy as np>>> import matplotlib.pyplot as plt>>>>>> plt.hist(numpy.random.normal(size=10000), bins=numpy.linspace(-3,3,100))>>> plt.show()
hpc-login-24 % pydoc numpy.random.normal
Generating random distributions
>>> import numpy as np>>> import matplotlib.pyplot as plt>>>>>> plt.hist(numpy.random.normal(size=10000), bins=numpy.linspace(-3,3,100))>>> plt.show()
hpc-login-24 % pydoc numpy.random.normal
Monte Carlo Integration
Two of the most common ways to employ Monte Carlo techniques are
Monte Carlo Sampling method
Monte Carlo Mean-Value method
Monte Carlo Integration
Two of the most common ways to employ Monte Carlo techniques are
Monte Carlo Sampling method
Monte Carlo Mean-Value method
Monte Carlo Sampling Method
Box off the region of integrationcalculate the area of the box
Randomly place points in the boxCount # of points in the box vs # under the
functionAreaf(x) =Areabox * Nf(x)/Nbox
f(x) = -xπ + π xfmax
xmax x
f(x)
Nf(x) = #
Nbox = # + #
“random marks”
Monte Carlo Sampling Method
Box off the region of integrationcalculate the area of the box
Randomly place points in the boxCount # of points in the box vs # under the
functionAreaf(x) =Areabox * Nf(x)/Nbox
f(x) = -xπ + π xfmax
xmax x
f(x)
Nf(x) = #
Nbox = # + #
“random marks”
Monte Carlo Mean-Value Method
Integration using the Monte Carlo method is done by averaging the value of the function at randomly selected points within the integration interval
I=∫a
b
dx f x=b−a⟨ f ⟩
⟨ f ⟩≃1N ∑
i=0
N
f x i
Statistical Error f= f / Nstandard deviation of mean
I~ f
“The Work Horse”
f(x)
x
<f>
Monte Carlo Mean-Value Method
Integration using the Monte Carlo method is done by averaging the value of the function at randomly selected points within the integration interval
I=∫a
b
dx f x=b−a⟨ f ⟩
⟨ f ⟩≃1N ∑
i=0
N
f x i
Statistical Error f= f / Nstandard deviation of mean
I~ f
“The Work Horse”
f(x)
x
<f>
fmax
xmax x
f(x)import numpy.random as rndnUnder = 0 for _ in range(nPoints): x = xMin + (xMax - xMin)*rnd.random() fRand = fMin + (fMax - fMin)*rnd.random() if fRand < f( x ): nUnder += 1 print(“Integration by Samples = ”, \ fMax*(xMax - xMin) * nUnder/nPoints)
import numpy.random as rndsum = 0for _ in range(nPoints): x = xMin + (xMax - xMin)*rnd.random() sum += f( x ) print(“Integration by Mean-Value = ”, \
(xMax – xMin) * sum/nPoints )
Example: MC integration
f(x) = -xπ + πx
<f>
xmax x
f(x)
xmin
xmin
∫0
eln (π)(π−1)
π x−xπ
fmax
xmax x
f(x)import numpy.random as rndnUnder = 0 for _ in range(nPoints): x = xMin + (xMax - xMin)*rnd.random() fRand = fMin + (fMax - fMin)*rnd.random() if fRand < f( x ): nUnder += 1 print(“Integration by Samples = ”, \ fMax*(xMax - xMin) * nUnder/nPoints)
import numpy.random as rndsum = 0for _ in range(nPoints): x = xMin + (xMax - xMin)*rnd.random() sum += f( x ) print(“Integration by Mean-Value = ”, \
(xMax – xMin) * sum/nPoints )
Example: MC integration
f(x) = -xπ + πx
<f>
xmax x
f(x)
xmin
xmin
∫0
eln (π)(π−1)
π x−xπ
Multi-Dimensional Monte Carlo
I=∫a
b
dx∫c
d
dy f x , y≃b−ad−c∗ 1N ∑
i
N
f x i , y i
Easy to generalize mean-value integration to many dimensionsMulti-Dimensional Monte Carlo
I=∫a
b
dx∫c
d
dy f x , y≃b−ad−c∗ 1N ∑
i
N
f x i , y i
Easy to generalize mean-value integration to many dimensions
Multi-Dimensional Monte Carlo
I=∫a
b
dx∫c
d
dy f x , y≃b−ad−c∗ 1N ∑
i
N
f xi , y i
Easy to generalize mean-value integration to many dimensions
import numpy.random as rnddef integrateMC(func, dim, limit, N=100): I,sum = 1/N,0 for n in range(dim): I *= (limit[n][1] - limit[n][0]) for k in range(N): x = [] for n in range(dim): x += [limit[n][0] + (limit[n][1] - limit[n][0])*rnd.random()] sum += func(x) return I*sum
def f(x): return np.sin(x[0] * x[1]**2)
dim,limit = 2,[[0,np.pi],[0,np.pi]]print(integrateMC(f, dim, limit))
main part of program
integrand function
multi-dimensional Monte Carlointegration implementation
f (x , y )=sin(xy2)
Multi-Dimensional Monte Carlo
I=∫a
b
dx∫c
d
dy f x , y≃b−ad−c∗ 1N ∑
i
N
f xi , y i
Easy to generalize mean-value integration to many dimensions
import numpy.random as rnddef integrateMC(func, dim, limit, N=100): I,sum = 1/N,0 for n in range(dim): I *= (limit[n][1] - limit[n][0]) for k in range(N): x = [] for n in range(dim): x += [limit[n][0] + (limit[n][1] - limit[n][0])*rnd.random()] sum += func(x) return I*sum
def f(x): return np.sin(x[0] * x[1]**2)
dim,limit = 2,[[0,np.pi],[0,np.pi]]print(integrateMC(f, dim, limit))
main part of program
integrand function
multi-dimensional Monte Carlointegration implementation
f (x , y )=sin(xy2)
Multi-Dimensional Integrationvia MC mean value
I=∫0
1
dx1∫0
1
dx2 ...∫0
1
dx12 f x1, x2, ... x12
For N=106 random points in the MC integration there are ~106 calculations
Example: Atomic Physics
3 Dimension/electron * 4 electrons = 12 Dimensions
Assuming 1 Giga evaluations/sec It would take ~ 10-3 sec
4Be
≃1−0 12∗1N ∑
i
N
f x1,i x2,
i x12i
compare to 107 yrs
Multi-Dimensional Integrationvia MC mean value
I=∫0
1
dx1∫0
1
dx2 ...∫0
1
dx12 f x1, x2, ... x12
For N=106 random points in the MC integration there are ~106 calculations
Example: Atomic Physics
3 Dimension/electron * 4 electrons = 12 Dimensions
Assuming 1 Giga evaluations/sec It would take ~ 10-3 sec
4Be
≃1−0 12∗1N ∑
i
N
f x1,i x2,
i x12i
compare to 107 yrs
Monte Carlo Error
Monte Carlo error is StatisticalError decreases as 1/ N
Mean-Value Integration is an ordinary statistical mean
x=1N ∑
i=1
N
xi
The error on the mean is the standard deviation of the mean (SDOM)
x= x
NWith the standard deviation σξ defined as usual
x= 1N −1∑ xi−xb
2
Note: ∑ [x i−x 2]=[∑ x i2]−N x2 is very useful in computing SDOM!
Monte Carlo Error
Monte Carlo error is StatisticalError decreases as 1/ N
Mean-Value Integration is an ordinary statistical mean
x=1N ∑
i=1
N
xi
The error on the mean is the standard deviation of the mean (SDOM)
x= x
NWith the standard deviation σξ defined as usual
x= 1N −1∑ xi−xb
2
Note: ∑ [x i−x 2]=[∑ x i2]−N x2 is very useful in computing SDOM!
Monte Carlo Error∫0
∫0
sin x∗y2dx dy=1.9051±0.0006
Mean-value integration error decreases linearly
with 1/√N
SDOMAVG
Averaged over 20 samples for each value of number of
Monte Carlo samples
nSamples, n = 10000, 20 While nSamples < MaxSamples vSum, vSumSq = 0.0, 0.0 for _ in range(n): v = integrateMC(func, nDim, limits, nSamples) vSum += v vSumSq += v*v avg = vSum/n SDOM = sqrt(1/n) * sqrt( 1/(n - 1) * (vSumSq - n*avg*avg) ) nSamples *= 2
Monte Carlo Error∫0
∫0
sin x∗y2dx dy=1.9051±0.0006
Mean-value integration error decreases linearly
with 1/√N
SDOMAVG
Averaged over 20 samples for each value of number of
Monte Carlo samples
nSamples, n = 10000, 20 While nSamples < MaxSamples vSum, vSumSq = 0.0, 0.0 for _ in range(n): v = integrateMC(func, nDim, limits, nSamples) vSum += v vSumSq += v*v avg = vSum/n SDOM = sqrt(1/n) * sqrt( 1/(n - 1) * (vSumSq - n*avg*avg) ) nSamples *= 2
See python code at http://hadron.physics.fsu.edu/~eugenio/comphy/examples/mcint.pySee python code at http://hadron.physics.fsu.edu/~eugenio/comphy/examples/mcint.py
Let's get working on #5
MONTE CARLO
Let's get working on #5
MONTE CARLO