Fast coputation of Phi(x) inverse
0 likes940 views
This document summarizes an algorithm for the fast inversion of the normal cumulative distribution function (CDF) based on Marsaglia's method. It represents numbers in a way that allows sample points of the inverse CDF to be extracted via bit manipulation for very fast evaluation. The algorithm tabulates sample values and uses quadratic interpolation between them, indexing the values in a way that can be computed with one bit operation. This allows for inverse CDF computation in less than 5 CPU instructions on most machines.
![Clever indexing
Conceptually, we have a matrix A[i][j] of
tabulated values
This requires two calculations to find indices –
one for i and one for j – and two operations
to lookup values
Combine into a single index
n = 992-32k + j
that can be extracted directly by one bit
manipulation: bits 2 through 14 minus 3040](https://image.slidesharecdn.com/fastphiinverse-100510105727-phpapp02/85/Fast-coputation-of-Phi-x-inverse-10-320.jpg)
![Polynomial evaluation
Taylor approximation:
t = h B(k,j)
x = A(k,j) -0.5 t + 0.125 A(k,j)t2
Rescale B’s by square root of 8:
t = h B’
x = A – c t – A t2 [ c = sqrt(2) ]
Horner’s method:
x = A – t(c – A t)](https://image.slidesharecdn.com/fastphiinverse-100510105727-phpapp02/85/Fast-coputation-of-Phi-x-inverse-11-320.jpg)
![C++ Implementation
double NormalCCDFInverse(double x)
{
float f1 = (float) x;
unsigned int ui;
memcpy(&ui, &f1, 4);
int n = (ui >> 18) - 3008;
ui &= 0xFFFC0000;
float f2;
memcpy(&f2, &ui, 4);
double v = (f1-f2)*B[n];
return A[n] - v*(sqrt2 - A[n]*v);
}](https://image.slidesharecdn.com/fastphiinverse-100510105727-phpapp02/85/Fast-coputation-of-Phi-x-inverse-12-320.jpg)
Fast coputation of Phi(x) inverse
- 1. Fast Inversion of Normal CDF John D. Cook
- 3. Basic assumptions Five or six significant figures is adequate for many applications Memory is fast and plentiful Low-level bit operations are faster than arithmetic
- 4. Basic Approach Tabulate values at sample points and use low-order approximations to fill in Take advantage of binary representation of floating point numbers to decide where to sample Change variables to avoid arithmetic
- 5. Big clever idea Extract sample points based on the exponent and mantissa The extraction is extremely fast It is biased to sample more frequently near zero, exactly where cPhi-1 needs more sampling
- 6. IEEE Floating Point Representation (conceptual) x = +/- 2e m, 1 <= m < 2 Bit 1: sign bit +/- Bits 2 through 9: exponent e Bits 10 through 32: mantissa
- 7. IEEE Floating Point Representation (details) Bit 1: 0 for positive, 1 for negative Bits 2 through 9: exponent of 2 biased by 127 (values 0 through 255 correspond to actual exponents -127 through 128) Bits 10 through 32: mantissa minus 1 (leading bit always 1, so don’t store)
- 8. Starting Point for Marsaglia’s algorithm Represent numbers by u = 2-k (2-1 + 2-6 j + 2-24 m) 0 <= k < 32, 0 <= j < 32 0 <= m < 224 k = 126 – e, where ‘e’ is the exponent representation bits j = first five mantissa representation bits m = last 18 mantissa representation bits
- 9. Sample Points Tabulate cPhi-1 at points corresponding to m = 0, i.e. at 32 possible values of i and j, a total of 1024 points. Use quadratic Taylor approximation based at these points A fixed number samples per exponent samples more finely near zero, just where cPhi-1 needs more samples
- 10. Clever indexing Conceptually, we have a matrix A[i][j] of tabulated values This requires two calculations to find indices – one for i and one for j – and two operations to lookup values Combine into a single index n = 992-32k + j that can be extracted directly by one bit manipulation: bits 2 through 14 minus 3040
- 11. Polynomial evaluation Taylor approximation: t = h B(k,j) x = A(k,j) -0.5 t + 0.125 A(k,j)t2 Rescale B’s by square root of 8: t = h B’ x = A – c t – A t2 [ c = sqrt(2) ] Horner’s method: x = A – t(c – A t)
- 12. C++ Implementation double NormalCCDFInverse(double x) { float f1 = (float) x; unsigned int ui; memcpy(&ui, &f1, 4); int n = (ui >> 18) - 3008; ui &= 0xFFFC0000; float f2; memcpy(&f2, &ui, 4); double v = (f1-f2)*B[n]; return A[n] - v*(sqrt2 - A[n]*v); }
- 13. Fine Print This algorithm only valid for p <= 0.5 For p > 0.5, use cPhi-1(p) = -cPhi-1(1-p) Phi-1(p) = cPhi-1(1-p) Algorithm not valid for p < 2-33 The maximum error is 0.000004, which occurs near p = 0.25
- 14. Implementation double PrivateNormalCCDFInverse(double x) … double NormalCDFInverse(double x) { return (x > 0.5) ? PrivateNormalCCDFInverse(1.0-x) : PrivateNormalCCDFInverse(x); } double NormalCCDFInverse(double x) { return (x > 0.5) ? -PrivateNormalCCDFInverse(1.0-x) : PrivateNormalCCDFInverse(x); }
- 15. References Rapid evaluation of the inverse of the normal distribution function by Marsaglia, Zaman, and Marsaglia Statistics and Probability Letters 19 (1994) 259 – 266
- 16. Notes This talk presented June 6, 2001 http://www.JohnDCook.com