1
Chapter 05
RANDOM NUMBER
GENERATOR
1
Reading
Required
Lemmis Park, Discrete Event Simulation - A First Course,
Chapter 2: Random Number Generation
Optional
Harry Perros, Computer Simulation Technique - The
Definitive Introduction, 2007
Chapter 2 and Chapter 3
2
Goals of Today
Understand the fundamental concepts of Random
Number Generators (RNGs)
Lean how to generate random variate for
standard uniform distributions in the interval [0,1]
Learn how to generate random variate for any
probability distribution
Outline
Why Random Number Generators?
Desired Properties of RNGs
Lehmer’s Algorithm
Period and full-period RNGs
Modulus and multiplier selection (Lehmer)
Implementation - overflow
3
Why RNG?
Random Numbers (RNs) are needed for doing simulations
Generate random arrivals (Poisson), random service times (Exponential)
Random numbers must be Independent (unpredictable)
Two types of random number generators:
True RN Generator: generated from real physical random phenomenon (e.g.
Measure the values of “white noise” in electronic circuits)
Pseudo RN Generator (P-RNG): It uses computational algorithms that produce
long sequences of apparently random results, which are in fact completely
determined by a shorter initial value, known as a seed or key.
Computer Simulations uses pseudo-random number generators are
used (They are not really random, but their output “looks random”)
Problem Statement of RNGs
Problem: We want to create a function:
u = rand();
that
produces a floating point number u, where 0<u<1
AND
any value strictly greater than 0 and less than 1 is
equally likely to occur
(Uniform Distribution in ]0,1[ )
0.0 and 1.0 are excluded from possible values
4
Problem Statement of RNGs
This problem can be simply resolved by
the following technique: For a large integer m, let the set m = {1, 2, … m-1}
Draw in an integer x m randomly
Compute: u = x/m
m should be very large
Our problem reduces to determining
how to randomly select an integer in m
Lehmer’s Algorithm
The objective of Lehmer’s Algorithm is to generate a
sequence of random integers in m:
x0, x1, x2, … xi, xi+1, …
Main idea: Generate the next xi+1 value based on
the last value of random integer xi
xi+1 = g(xi)
for some function g(.)
5
Lehmer’s Algorithm
In Lehmer’s Algorithm, g(.) is defined using two fixed parameters
Modulus m : a large, fixed prime integer
Multiplier a : a fixed integer in m
Then, choose an initial seed
The function g(.) is defined as:
The mod function produces the remainder after dividing the first argument by the second
More precisely: where is the largest integer n such that n≤ x
modg x a x m
0 mx
modu
u v u vv
x
Observations
The mod function ensures a value lower than m is always produced
If the generator produces the value 0, then all subsequent numbers in the sequence will be zero (This is not desired)
Theorem
if (m is prime and initial seed is non-zero) then the generator will never produce the value 0
In this case, the RNG produces values in m = {1, 2, … m-1}
1 modi ix a x m
6
Observations
The above equation simulates drawing balls from an urn
without replacement, where each value in m represents a ball
The requirement of randomness is violated because successive draws are
not independent
Practical Fact: The random values can be approximately
considered as independent if the number of generated random
variates (ball draws) is << m
The Quality of the random number generator
is dependent on good choices for a and m
1 modi ix a x m
The Period of a Sequence
Consider sequence produced by:
Once a value is repeated, all the sequence is then repeated itself
Sequence: where
p is the period: number of elements before the first repeat
clearly p ≤ m-1
It can be shown, that if we pick any initial seed x0, we are guaranteed this initial seed will reappear
1 modi ix a x m
0 1, ,..., ,...,i i px x x x i i px x
7
Full Period Sequences
[LP] Discrete-Event Simulation: A First Course by L. M. Leemis and S. K Park, Prentice Hall, 2006, page. 42 Theorem 2.1.2
If and the sequence is produced by the
Lehmer generator where m is prime,
then there is a positive integer p with such that:
are all different
and
In addition,
0 mx 0 1 2, , ,...x x x
1 modi ix a x m
1p m
0 1 1, ,..., px x x
, 0,1,2,...i i px x i
1 mod 0m p
Full Period Sequences
Ideally, the generator cycles through all values
in m to maximize the number of possible
values that are generated, and guarantee
any number can be produced
The sequence containing all possible numbers is
called a full-period sequence (p = m-1)
Non-full period sequences effectively partition
m into disjoint sets, each set has a particular
period.
8
Modulus and Multiplier Selection Criteria
Selection Criteria 1: m to be “as large as possible”
m = 2i - 1 where i is the machine precision (is the largest possible
positive integer on a “two’s complement” machine)
Recall m must be prime
It happens that 231-1 is prime (for a 32 bit machine)
Unfortunately, 215-1 and 263-1 are not prime ;-(
Selection Criteria 2: p gives full-period sequence (p = m-1)
For a given prime number m, select multiplier a that provide a full
period
Algorithm to test if a is a full-period multiplier (m must be prime):
Modulus and Multiplier Selection Criteria
Criterias
m to be “as large as possible”
p gives full-period sequence (p = m-1)
p = 1;
x = a; // assume, initial seed is x0=1, thus x1=a
while (x != 1) { // cycle through numbers until repeat
p++;
x = (a * x) % m; // careful: overflow possible
}
if (p == m-1) // a is a full period multiplier
else // a is not a full period multiplier
Algorithm for finding if p is full-period
multiplier
9
Other Useful Properties
Theorem 2.1.1[LP]: If the sequence x0, x1, x2, … is produced by a
Lehmer generator with multiplier a and modulus m, then
Note this is not a good way to compute xi!
Theorem 2.1.4[LP, p. 45]: If a is any full-period multiplier relative
to the prime modulus m, then each of the integers
is also a full period multiplier relative to m if and only if the
integer i has no prime factors in common with the prime factors of
m-1 (i.e., i and m-1 are relatively prime, or co-prime)
0 mod 0,1,2,...i
ix a x m i
mod 0,1,2,..., 1i
ma m i m
Other Useful Properties
// Given prime modulus m and any full period multiplier a,
// generate all full period multipliers relative to m
i = 1;
x = a; // assume, initial seed is 1
while (x != 1) { // cycle through numbers until repeat
if (gcd(i,m-1)==1) // x=aimod m is full period multiplier
i++;
x = (a * x) % m; // careful: overflow possible
}
Generate all full-period multipliers
10
Implementation Issues
Assume we have a 32-bit machine, m=231-1
Problem
Must compute
Obvious computation is to compute first, then do mod
operation
The multiplication might overflow, especially if m-1 is large!
First Solution: Floating point solution
Could do arithmetic in double precision floating point if
multiplier is small enough
Double has 53-bits precision in IEEE floating point
May have trouble porting to other machines
Integer arithmetic faster than floating point
moda x m
a x
Implementation Issues: Mathematical Solutions
Problem: Compute without
overflow
General Idea: Perform mod operation first,
before multiplication.
Suppose that (not prime)
We have:
Thus, No overflow
moda x m
m a q
mod mod moda x m a x a q a x q
mod 1x q q
mod 1a x q a q a q m
11
For the case, m is prime, so let
q = quotient; r = remainder
Let
It can be shown that (Page 59, Lemis/Park Textbook)
and
m a q r
modx
x a x q rq
x a xx
q m
moda x m x m x
Implementation Issues: Mathematical Solutions
0 if
1 if -
m
m
xx
x
Random number variants
Discrete random variables
Continuous random variables
Next
12
The Concept
Goal: create random variates
for random variables X of a
certain distribution defined with
its CDF
Approach: Inverse cumulative
distribution function
CDF F(x)=u
0 1F x u
Cumulative Distribution Function
Generate a random variable
Determine x such that
0 1F x u
0 1u
1F x u F u x
u1
X1Exp
u2
X1Poisson
X2Exp X2Poisson
Generating Discrete Random Variates
Cumulative Distribution Function of X:
F(x) = P(X≤x)
x
F(x)
1.0
1 2 3 4 5
0.8
0.6
0.4
0.2
x f(x)
1 0.2
2 0.3
3 0.1
4 0.2
5 0.2 u = F(x)
0.5 = F(2)
x
u
Inverse Distribution Function (idf) of X:
F*(u) = min {x: u < F(x)}
x
F(x)
1.0
1 2 3 4 5
0.8
0.6
0.4
0.2
x = F*(u)
2 = F(0.5)
x
u
Random variate generation:
1. Select u, uniformly distributed (0,1)
2. Compute F*(u); result is random variate with
distribution f()
f(x=2)
13
Discrete Random Variate Generation
Uniform (a,b): equally likely to select an integer in interval [a,b]
;
if 1 return 0;
else return 1;
u random
u p
Bernoulli(p):
Return 1 with probability p,
Return 0 with probability 1-p
Geometric(p):
Number of Bernoulli trials
until first ‘0’)
;
return log 1.0 log ;
u random
u p
1kf x p p
;
return 1 ;
u random
a u b a
Exponential Random Variates
Exponential distribution with mean
;
return - log 1 ;
u random
u
