Random Number Generation
Fall 2012
By Yaohang Li, Ph.D.
Review• Last Class
– Variance Reduction• This Class
– Random Number Generation– Uniform Distribution– Non-uniform Distribution Random Number Generation– Assignment 3
• Next Class– Quasi-Monte Carlo
Random Numbers• Application of Random Numbers
– Simulation• Simulate natural phenomena
– Sampling• It is often impractical to examine all possible cases, but a
random sample will provide insight into what constitutes typical behavior
– Numerical analysis– Computer programming– Decision making
• “Many executives make their decisions by flipping a coin…”
– Recreation
Natural Random Number
• Natural Random Numbers– No two snowflakes are the same
– Sources
• White Noise
• Water Molecule Distribution
• etc.
– Generation
• Measurement
• Irreproducible
• Errors
Pseudorandom Number Generators
• Pseudorandom Numbers– Using a Mathematical Formula
– Deterministic
– Behave like real random numbers
• Comments– There is no “perfect” pseudorandom number generator
– We should never completely trust results from a single pseudorandom number generator
– Good random number generators are hard to find
Middle-square method
• Developed by von Neumann• Procedure
– 4 digit starting value is created
– Square and produce 8-digit number
– Get the middle 4 digits as the result and seed for next number
• Problems– What if the middle 4 digits are 0
– Forsythe found that the sequence may stuck in 6100, 2100, 4100, 8100, 6100, …
• Not a good generator
Quality of Pseudorandom Numbers
• Uniformity• Randomness• Independence• Reproducibility• Portability• Efficiency• A sufficiently long period
Generating Uniform Random Numbers
• Generating Uniform Random Numbers– Uniform distribution on [0,1)
– Generation
• Un=Xn/m– Xn: Random number Integer
– m: Max(Xn)+1: Usually the word size of a computer
– Un: Uniform real random number at [0,1)
Linear Congruential Method
• Linear Congruential Method– Most commonly used generator for pseudorandom numbers
• m: modulus• a: multiplier• b: additive constant
– Period• m constrains the period
– max period: 2m-1– m is usually chosen to be either prime of a power-of-two
)(mod1 mbaxx nn
Shift-Register Generators (SRG)
• Shift-Register Generators– based on the following recursion
– ai and xi are either 0 or 1
– Comments
• The recursion produces only bits
• Incorporate these bits into integers
),2(mod1
0
k
iinikn xax
Lagged-Fibonacci Generators
• Lagged-Fibonacci Generators (LFG)– Additive Lagged-Fibonacci Generators
– Multiplicative Lagged-Fibonacci Generators
• Comments– LFG has a much longer period than LCGs
– (2k-1)2m-1
kjxxx mknjnn ),2(mod
kjxxx mknjnn ),2(mod
Spectral Tests
Spectral Tests
Inversive Congruential Generators
• Inversive Congruential Generators (ICGs)– Recursive ICGs
– Explicit ICGs
– Advantage of ICGs
• ICGs do not fall in hyperplanes
),(mod1 mbxax nn
)(modmbanxn
)(mod1 mcc
Combined Generators
• Combined Generators– Combining different recurrences can increase the period length
– Improve the structural properties of pseudorandom generators
– Construct a new random sequence
• exclusive-or operator
• addition modulo
• addition of floating-point random numbers modulo 1
– x, y
• Different random number sources
,nnn yxz
Parallel Random Number Generators
• Requirements of Parallel Random Numbers– Every random number sequence generated on each processor
should satisfy the requirements of a good sequential generator.
– The parallel generator must be reproducible both on different machines and on the same machine with a different partitioning of the processing resources.
– The parallelly generated random streams must be uncorrelated and must not overlap.
– The parallel generator should work for an arbitrary, but perhaps bounded, number of processors.
Parallel Random Numbers Generations (Leapfrog)
• Leapfrog
Parallel Random Numbers Generations (Sequence Splitting)
• Sequence Splitting
Parallel Random Numbers Generations (Sequence Splitting)
• Random Tree Method– Also called
parameterization method
SPRNG
• http://sprng.cs.fsu.edu
Random Choices from a Finite Set
•A random integer X between 0 and k-1
– U is a random number uniformly distributed in [0,1)
•A more general case
kUX
1...,
,
0,
21
2112
11
Uppifpx
ppUifpx
pUifx
X
kk
Inverse Function Method
• Cumulative Distribution Function– Most real-valued distribution may be expressed in terms of its
distribution function F(x)
• Inverse Function Method– X=F-1(U)
– Now the problem reduces to how to evaluate the inverse function F-1()
Interesting Trick
• Generating the random samples of F(x)=x2
• Inverse Function Method– X=U-1/2
• A short cut method– If X1 is a random variable having the distribution F1(x) and if
X2 is a random variable having the distribution F2(x)• max(X1, X2) has the distribution F1(x)F2(x)• min(X1, X2) has the distribution F1(x)+F2(x)-F1(x)F2(x)
– Then X=max(U1, U2) has the distribution of F(x)=x2
– Hard to believe that max(U1, U2) and U-1/2 have the same distribution
Normal Distribution• Polar Method
1. Generate two independent random variables, U1 and U2
2. Set V1=2U1-1, V2=2U2-1
3. Set S=V1*V1+V2*V2
4. If S>=1, return to Step 1
5. Set X1 and X2 according to the following two equations
S
SVX
S
SVX
ln2
ln2
22
11
Acceptance-Rejection Method
• Desired pdf– Suppose we bound the desired
probability distribution function to sample from a box
• Algorithm1. Generate a random variable x from
U(0,1)
2. Generate another random variable y from U(0,1)
3. If x<f(y)/fmax then return y
4. else repeat from step 1
Acceptance-Rejection Method Example
• Determine an algorithm for generating random variates for a random variable that take values 1, 2, …, 10 with probabilities 0.11, 0.12, 0.09, 0.08, 0.12, 0.10, 0.09, 0.09. 0.10, 0.10 respectively
• Acceptance-Rejection Method– u1=U(0,1), u2=U(0,1), c=max(p())=0.12– Y=floor(10*u1+1)– while (u2>p(Y)/c)
• u1=U(0,1), u2=U(0,1)• Y=floor(10*u1+1)
– output Y
Analysis of Acceptance-Rejection Method
• Advantage– Acceptance-Rejection Method can fit in different pdfs
– popularly used in complicated probability geometry
• Disadvantage– Inefficient if the volume of the region of interest is small
relative to that of the box
• most of the darts will miss the target
Exponential Distribution
• F(x)=1-e-x/
– Logarithm method (inverse function method)
– X=-lnU
Shuffling Algorithm
• Let X1, X2, …, Xt be a set of t numbers to be shuffled1. j=t
2. Generate U
3. Set k=floor (jU)+1
4. Exchange Xk with Xj
5. Decrease j by 1. If j>1, return to step 2
Summary• Random Numbers– Uniform Random Numbers
• Generation of Uniform Random Numbers– Natural Random Number Generators– Pseudorandom Number Generators
• Pseudorandom Number Generators– Requirement of Pseudorandom Number Generators– LCG– LFG– SRG– ICG– Combined Random Number Generators
• Parallel Random Number Generators– Requirement of Parallel Random Number Generators– Techniques for Parallel Random Number Generators
• Leapfrog• Sequence Splitting• Random Tree
Summary• Numerical Distribution
– Random Choices from a finite set– General methods for continuous distributions
• inverse function method• acceptance-rejection method
– Distributions• Normal distribution
– Polar method• Exponential distribution
– Shuffling
What I want you to do?
• Review Slides• Review basic probability/statistics concepts