Fast Inversion of Normal CDF

John D. Cook

What we’re computing

Basic assumptions

Five or six significant figures is adequate for many applications

Memory is fast and plentiful

Low-level bit operations are faster than arithmetic

Basic Approach

Tabulate values at sample points and use low-order approximations to fill in

Take advantage of binary representation of floating point numbers to decide where to sample

Change variables to avoid arithmetic

Big clever idea

Extract sample points based on the exponent and mantissa

The extraction is extremely fast

It is biased to sample more frequently near zero, exactly where cPhi-1 needs more sampling

IEEE Floating Point Representation (conceptual)

x = +/- 2e m, 1 <= m < 2

Bit 1: sign bit +/-

Bits 2 through 9: exponent e

Bits 10 through 32: mantissa

IEEE Floating Point Representation (details)

Bit 1: 0 for positive, 1 for negative

Bits 2 through 9: exponent of 2 biased by 127 (values 0 through 255 correspond to actual exponents -127 through 128)

Bits 10 through 32: mantissa minus 1(leading bit always 1, so don’t store)

Starting Point for Marsaglia’s algorithm

Represent numbers byu = 2-k (2-1 + 2-6 j + 2-24 m)

0 <= k < 32, 0 <= j < 320 <= m < 224

k = 126 – e, where ‘e’ is the exponent representation bits

j = first five mantissa representation bits

m = last 18 mantissa representation bits

Sample Points

Tabulate cPhi-1 at points corresponding to m = 0, i.e. at 32 possible values of i and j, a total of 1024 points.

Use quadratic Taylor approximation based at these points

A fixed number samples per exponent samples more finely near zero, just where cPhi-1 needs more samples

Clever indexing

Conceptually, we have a matrix A[i][j] of tabulated values

This requires two calculations to find indices –one for i and one for j – and two operations to lookup values

Combine into a single index n = 992-32k + jthat can be extracted directly by one bit manipulation: bits 2 through 14 minus 3040

Polynomial evaluation

Taylor approximation:t = h B(k,j)x = A(k,j) -0.5 t + 0.125 A(k,j)t2

Rescale B’s by square root of 8:t = h B’x = A – c t – A t2 [ c = sqrt(2) ]

Horner’s method:x = A – t(c – A t)

C++ Implementation

double NormalCCDFInverse(double x)

{

float f1 = (float) x;

unsigned int ui;

memcpy(&ui, &f1, 4);

int n = (ui >> 18) - 3008;

ui &= 0xFFFC0000;

float f2;

memcpy(&f2, &ui, 4);

double v = (f1-f2)*B[n];

return A[n] - v*(sqrt2 - A[n]*v);

}

Fine Print

This algorithm only valid for p <= 0.5

For p > 0.5, use cPhi-1(p) = -cPhi-1(1-p)

Phi-1(p) = cPhi-1(1-p)

Algorithm not valid for p < 2-33

The maximum error is 0.000004, which occurs near p = 0.25

Implementation

double PrivateNormalCCDFInverse(double x) …

double NormalCDFInverse(double x) {

return (x > 0.5)

? PrivateNormalCCDFInverse(1.0-x)

: PrivateNormalCCDFInverse(x);

}

double NormalCCDFInverse(double x) {

return (x > 0.5)

? -PrivateNormalCCDFInverse(1.0-x)

: PrivateNormalCCDFInverse(x);

}

References

Rapid evaluation of the inverse of the normal distribution functionby Marsaglia, Zaman, and MarsagliaStatistics and Probability Letters19 (1994) 259 – 266

Notes

This talk presented June 6, 2001

http://www.JohnDCook.com