Post on 04-Jun-2018
transcript
8/13/2019 Lecture 8 - Random Number Generation
1/42
Random Number Generation
Dr. Akram Ibrahim Aly
Copyright Akram I. Aly 2012
Lecture (8)
8/13/2019 Lecture 8 - Random Number Generation
2/42
8/13/2019 Lecture 8 - Random Number Generation
3/42
Applications of Random Numbers Random numbers have a lot of real world
applications; e.g.: sampling
Simulation: generate event times and randomvariables in simulation statistical inference
random search optimization lotteries, gambling
8/13/2019 Lecture 8 - Random Number Generation
4/42
Properties of random numbers A sequence of random numbers R1, R2, , Rn
must have 2 important statistical properties: Uniformity Independence
8/13/2019 Lecture 8 - Random Number Generation
5/42
Uniformity Each random number is an independent sample
drawn from a continuous uniform distribution between 0 and 1: R i [0,1]
otherwise ,0
10 ,1)(
x x f
21
2)(
1
0
1
0
2 x xdx R E
12
1
4
1
3
1
2
1
3)(
21
0
1
0
32
xdx x RV
x
f ( x
)
0
1
PDF:
8/13/2019 Lecture 8 - Random Number Generation
6/42
Independence Independence : the probability of observing a
value in a particular interval is independent ofthe previous value drawn.
8/13/2019 Lecture 8 - Random Number Generation
7/42
Generation of Pseudo-RandomNumbers
Pseudo means fake
Pseudo, because we utilize known methods togenerate numbers which removes the potential fortrue randomness .
Pseudo-random numbers: Deterministic sequence ofnumbers with a repeat period but with the appearanceof randomness (if you dont know the algorithm).
8/13/2019 Lecture 8 - Random Number Generation
8/42
Generation of Random Numbers Goal: To produce a sequence of numbers in [ 0,1 ]
that simulates, or imitates, the ideal properties ofrandom numbers (RN). Important considerations in RN routines:
Fast Portable to different computers Have sufficiently long cycle
Replicable Closely approximate the ideal statistical properties of
uniformity and independence.
8/13/2019 Lecture 8 - Random Number Generation
9/42
Techniques for Generating RandomNumbers
Mid-square method Linear Congruential Method Combined Linear Congruential Method
Tausworthe generators (self study) Extended Fibonacci generators (self study)
8/13/2019 Lecture 8 - Random Number Generation
10/42
Mid-Square Method Proposed by Von Neumann and Metropolis in
the 1940s1. Start with an initial d-digit integer seed X 0 2. Square X
0 3. Take the middle d- digits as the next four-digit
number, X 1
4. Place a decimal point at the left of X 1 to get firstrandom number R 15. Square X 1 and continue process
8/13/2019 Lecture 8 - Random Number Generation
11/42
SEED Random number seed -initial random number
used to generate the next random numberwhich is in turn transformed into the new seedvalue.
8/13/2019 Lecture 8 - Random Number Generation
12/42
Example Let X 0 = 5497
X1 = (5497) 2= 30 2170 09 X 1 = 2170 R 1 = 0.2170 X2 = (2170 )2= 04708900 X 2 = 7089 R 2 = 0.7089 X3 = (7089) 2= 50253921 X 3 = 2539 R 3 = 0.2539
...and so on...
8/13/2019 Lecture 8 - Random Number Generation
13/42
Example 2: Poor Mid-SquareGenerator
Let X 0 = 5197
X1 = (5197) 2= 27008809 X1 = 0088 R1 = 0.0088 X2 = (0088) 2= 00007744 X2 = 0077 R2 = 0.0077 X3 = (0077) 2= 00005929 X3 = 0059 R3 = 0.0059
X4 = (0059) 2= 00003481 X4 = 0034 R4 = 0.0034 Once zeros appear, they are carried in subsequent
numbers.
8/13/2019 Lecture 8 - Random Number Generation
14/42
Example 3: Poor Mid-SquareGenerator
Let X 0 = 6500
X 1 = (6500) 2= 42250000 X1 = 2500 R1 = 0.2500 X2 = (2500) 2= 06250000 X2 = 2500 R2 = 0.25000
Cannot choose a seed that guarantees that the
sequence will not degenerate and will have a long period.
8/13/2019 Lecture 8 - Random Number Generation
15/42
Advantages and disadvantages ofthe MID-SQUARE method
Advantages of the mid-square method Rather simple to implement
Disadvantages of the mid-square method
Difficult to choose initial seed that will give goodsequence Strong tendency to degenerate fairly rapidly to
zero
8/13/2019 Lecture 8 - Random Number Generation
16/42
More Examples on the mid-squareMethod
8/13/2019 Lecture 8 - Random Number Generation
17/42
17
The Mid-product Method Improved Variation of Mid-square method
Example: Start with two arbitrary 3-digit numbers
Z0 = 123 Z 1 = 456
Z0Z1 = (123)(456) = 56088 U 1 = 0.608 Z 2 = 608Z1Z2 = (456)(608) = 277248 U 2 = 0.772 Z 3 = 772
Z2Z3 = (608)(772) = 469376 U 3 = 0.693 Z 4 = 693
8/13/2019 Lecture 8 - Random Number Generation
18/42
Linear Congruential Methodby D. H. Lehmer, 1951
To produce a sequence of integers, X 1 , X 2 , between 0 and
(m-1) by following a recursive relationship:
If c 0, the form is called the mixed congruentialmethod.
If c=0, the form is known as the multiplicativecongruential method
8/13/2019 Lecture 8 - Random Number Generation
19/42
Linear Congruential Method The random integers are being generated in the
interval [0, m-1], and to convert the integers torandom numbers: For [0,1[
For [0,1]
,...2,1 im
X R ii
,...2,1 1
im X
R ii
8/13/2019 Lecture 8 - Random Number Generation
20/42
Example Use X0= 27 , a = 17 , c = 43 , and m = 100 .
The Xi and Ri values are: X 1= (17*27+43) mod 100 = 502 mod 100 = 2,R1= 0.02; X 2= (17*2+32) mod 100 = 77, R2= 0.77 ; X 3= (17*77+32) mod 100 = 52, R3= 0.52;
8/13/2019 Lecture 8 - Random Number Generation
21/42
21
Example For example,
Starting with x 0=5:
The first 32 numbers obtained by the above procedure10, 3, 0, 1, 6, 15, 12, 13, 2, 11, 8, 9, 14, 7, 4, 5;10, 3, 0, 1, 6, 15, 12, 13, 2, 11, 8, 9, 14, 7, 4, 5.
By dividing x's by 16:
0.6250, 0.1875, 0.0000, 0.0625, 0.3750, 0.9375, 0.7500, 0.8125, 0.1250,0.6875, 0.5000, 0.5625, 0.8750, 0.4375, 0.2500, 0.3125;0.6250, 0.1875, 0.0000, 0.0625, 0.3750, 0.9375, 0.7500, 0.8125, 0.1250,0.6875, 0.5000, 0.5625, 0.8750, 0.4375, 0.2500, 0.3125.
8/13/2019 Lecture 8 - Random Number Generation
22/42
Selection of LCG Parameters The selection of the values for a , c , m , and X 0
affects the statistical properties and the cyclelength. Most natural choice for m is one that equals to
the capacity of a computer word. m = 2 b (binary machine), where b is the number of
bits in the computer word.
m = 10 d (decimal machine), where d is the numberof digits in the computer word.
8/13/2019 Lecture 8 - Random Number Generation
23/42
23
Selection of LCG Parameters The modulus m should be large
All generated numbers, X i , are between 0 and m-1 ,the period can never be more than m
For mod m computation to be efficient, m must be a power of 2 Mod m can beobtained by dropping the leftmost binary
digits in X i+1 and using only the b rightmost binary digits
8/13/2019 Lecture 8 - Random Number Generation
24/42
8/13/2019 Lecture 8 - Random Number Generation
25/42
Example Using the multiplicative congruential method, find
the period of the generator for a = 13, m = 26, and X 0
= 1, 2, 3, and 4. The solution is given in next slide. When the seed is 1 and 3, the sequence has period 16.
However, a period of length eight is achieved whenthe seed is 2 and a period of length four occurs whenthe seed is 4.
8/13/2019 Lecture 8 - Random Number Generation
26/42
Random Numbers 26
SolutionPeriod Determination Using Various seeds
i X i X i X i X i
0 1 2 3 41 13 26 39 522 41 18 59 363 21 42 63 204 17 34 51 45 29 58 236 57 50 437 37 10 478 33 2 359 45 7
10 9 2711 53 3112 49 1913 61 5514 25 1115 5 1516 1 3
8/13/2019 Lecture 8 - Random Number Generation
27/42
27
Selection of LCG Parameters forMaximum Period
If c 0, the maximum possible period m
is obtained if and only if: Integers m and c are relatively prime, that is, have nocommon factors other than 1
Every prime number that is a factor of m is also afactor of ( a-1)
If integer m is a multiple of 4, a-1 should be amultiple of 4
Notice that all of these conditions are met if m=2 b ,a = 4k + 1 , and c is odd.
8/13/2019 Lecture 8 - Random Number Generation
28/42
Selection of LCG Parameters forMaximum Period
For m a power of 2, say m = 2 b , and c = 0 , the
longest possible period is P = m / 4 = 2 b-2
, which isachieved provided that the seed X 0 is odd and themultiplier, a , is given by a = 3 + 8k or a = 5 + 8k , for
some k = 0, 1,...
8/13/2019 Lecture 8 - Random Number Generation
29/42
8/13/2019 Lecture 8 - Random Number Generation
30/42
30
Example
Using a seed of x 0=1 5, 25, 29, 17, 21, 9, 13, 1, 5,
Period = 8 = 32/4 With x 0=2, the sequence is
10, 18, 26, 2, 10,
Here, the period is only 4
8/13/2019 Lecture 8 - Random Number Generation
31/42
31
Example Multiplier not of the form 8i 3:
Using a seed of x 0=1, we get the sequence 7, 17, 23, 1, 7, The period is only 4
8/13/2019 Lecture 8 - Random Number Generation
32/42
8/13/2019 Lecture 8 - Random Number Generation
33/42
33
Multiplicative LCG with m 2k
Modulus m = prime number With a proper multiplier a , period = m- 1 Maximum possible period = m If and only if the multiplier a is a primitive root of the modulus m
a is a primitive root of m if and only if a n mod m 1
for n = 1, 2, , m- 2
8/13/2019 Lecture 8 - Random Number Generation
34/42
8/13/2019 Lecture 8 - Random Number Generation
35/42
35
Full period generators A generator that has the maximum possible period, m,
is called a full-period generator Examples:
8/13/2019 Lecture 8 - Random Number Generation
36/42
Combined Linear CongruentialGenerators,LEcuyer[1988]
Reason: Longer period generator is needed because of the increasing complexity ofstimulated systems.
Approach: Combine two or more
multiplicative congruential generators.
8/13/2019 Lecture 8 - Random Number Generation
37/42
8/13/2019 Lecture 8 - Random Number Generation
38/42
Combined Linear CongruentialGenerators
The maximum possible period is:
8/13/2019 Lecture 8 - Random Number Generation
39/42
8/13/2019 Lecture 8 - Random Number Generation
40/42
Example The algorithm (contd)
Step 4:Return
Step 5: Set j = j+1 , go back to step 2
Combined generator has period: (m11)(m21)/2 ~2x10 18
8/13/2019 Lecture 8 - Random Number Generation
41/42
8/13/2019 Lecture 8 - Random Number Generation
42/42
Random-Numbers Streams A single random-number generator with k
streams can act like k distinct virtual random-number generators
To compare two or more alternative systems itis advantageous to dedicate portions of the
pseudo-random number sequence to the same
purpose in each of the simulated systems.