Revisions to the Spectral Test and the Lempel-Ziv Compression Test in the NIST Statistical Test Suite
National Institute of Information and Communications Technology, JAPAN
Song-Ju Kim and Ken Umeno （ ChaosWare In
c. ）
It is well known that the NIST Statistical Test Suite was used in the evaluation of the AES candidate algorithms.
It is also world-widely used by external audiences in the evaluation of their Pseudo Random Number Generators.
The NIST Statistical Test Suite
Random Excursions Variant16Random Excursions15Cumulative Sums14
Approximate Entropy13Serial12
Linear Complexity11Lempel Ziv Compression10
Universal9Overlapping Template Matching8
Non-overlapping Template Matching7Discrete Fourier Transform6
Binary Matrix Rank5Longest Run4
Runs3Block Frequency2
Frequency1Test NameNumber
Random Excursions Variant16Random Excursions15Cumulative Sums14
Approximate Entropy13Serial12
Linear Complexity11Lempel Ziv Compression10
Universal9Overlapping Template Matching8
Non-overlapping Template Matching7Discrete Fourier Transform6
Binary Matrix Rank5Longest Run4
Runs3Block Frequency2
Frequency1Test NameNumber
“A Statistical Test Suite for Random and Pseudorandom Number Generators for Cryptographic Applications”
National Institute of Standards and Technology(2001)
http://csrc.nist.gov/rng/
OUTLINE On the NIST Statistical Test Suite Test Results (AES, SHA-1, and MUGI) Checking of the Uniformity of P-values Corrections to the Spectral (DFT) Test Corrections to the LZC Test Summary
The test procedure A set of sequences, each of length n, is produ
ced from the selected generator. Each statistical test evaluates the sequence a
nd returns one or more P-values. If the P-value ≥ α(=0.01), then we call the se
quence “success”. 1. Checking of the success rate. 2. Checking of the uniformity of the distributio
n of P-values.
What is p-value? P-value: the probability that a perfectrandom number generator would have produced a sequence less random than the sequence that are tested.
1. The checking of the success rate
The range of acceptable proportions:
※ (μ±3σ)/m : 99.73% range of binomial
distribution, where μ= m (1 – α) and σ= m α(1- α). α=0.01: significance level
m)1(31
Success Rate (Example)
Key 1
Key 4
2. The checking of the uniformity of the P-values distribution
The interval [0,1] is divided into 10 sub intervals, and the p-values that lie within each sub-intervals are counted (F i).
p-value of p-values: IGMC( 9 / 2, χ2 / 2 )
where IGMC(n, x) = and
The test passes if p-value of p-values ≥ 0.0001
10
1
2
2
10
)10
(
i m
mFi
dtn x
nt te
1
)(1
Uniformity of p-values (Example)
Key 1 (fail)
Key 4 (pass)
The parameters we used
n=10,
α=0.01,
6
1000 samples
20000Block Frequency
10Approximate Entropy10Serial
500 (5000)Linear Complexity
7(1280)
Universal(Initialization Steps)
9Template Matching
BLOCK LENGTHTEST NAME20000Block Frequency
10Approximate Entropy10Serial
500 (5000)Linear Complexity
7(1280)
Universal(Initialization Steps)
9Template Matching
BLOCK LENGTHTEST NAME
10 keysх1000 samplesх10^6 (sequence length) total 10^10 bit
Test Results AES (OFB)
Lempel-Zivpass１０Lempel-Zivpass９
passpass８passNOTM, OTM７
Lempel-ZivCUSUM６Lempel-ZivNOTM(2)５
passpass４passREX３passpass２passpass１
UniformitySuccess RateKey
Lempel-Zivpass１０Lempel-Zivpass９
passpass８passNOTM, OTM７
Lempel-ZivCUSUM６Lempel-ZivNOTM(2)５
passpass４passREX３passpass２passpass１
UniformitySuccess RateKey
Test Results SHA-1
Lempel-Zivpass１０passpass９passNOTM８passNOTM(2)７passNOTM, REX, REXV６
Lempel-Zivpass５FFTNOTM(2)４passNOTM(2)３
Lempel-Zivpass２passpass１
UniformitySuccess RateKey
Lempel-Zivpass１０passpass９passNOTM８passNOTM(2)７passNOTM, REX, REXV６
Lempel-Zivpass５FFTNOTM(2)４passNOTM(2)３
Lempel-Zivpass２passpass１
UniformitySuccess RateKey
Test Results MUGI
FFTpass１０passNOTM９passpass８passpass７passpass６passNOTM５passpass４
Lempel-ZivLempel-Ziv３Lempel-Zivpass２
passNOTM１UniformitySuccess RateKey
FFTpass１０passNOTM９passpass８passpass７passpass６passNOTM５passpass４
Lempel-ZivLempel-Ziv３Lempel-Zivpass２
passNOTM１UniformitySuccess RateKey
If we focus on the uniformity of P-values, only the DFT test and LZC test are failed frequently.
If we choose the sample size m greater than 10000, we cannot find any PRNG that pass these two test.
P-value of P-values (SHA-1)
These distributions of P-values indicates a apparent deviation from randomness although we use a well-known good PRBG (SHA-1)
This observation suggests that the test settings in these two tests are not accurate.
The DFT testtest description (NIST document) The zeros and ones of the input sequence are conv
erted to values of -1 and +1. Apply a DFT on X to produce: S=DFT(X). Calculate M=modulus(S’), where S’ is the substring
consisting of the first n/2 elements in S. Compute T= : the 95% peak height threshold v
alue. Compute N0 = 0.95n/2. Compute N1 = the actual observed number of peak
s in M that are less than T. Compute P-value =
n3
2/)05.0)(95.0(01
nNNd
2|| derfc
The probability distribution (SHA-1)
n3
300,000samples
n995732274.2
2npq
4npq
The LZC test test description (NIST document)
Parse the sequence into consecutive, disjoint and distinct words that will form a “dictionary” of words in the sequence.
ex. 0|1|00|01|000|11|011|
Compute P-value =
2221 W obserfc
The probability distribution (SHA-1)
09.69588
574336518.752 L
42178447.722 R
Despite the best fitting of the distribution, the uniformity of P-values cannot be improved.
This is because the distribution of the number of words is too narrow.
In other words, a variety of the appeared P-values is limited.
The effect of discreteness
Because the variety of appeared P-values is too scarce in centered bins, we never get the uniformity of P-values in this situation.
The histogram of P-values always has some biases even if we use good PRNG.
However, these biases are always the same if we use good PRNG.
Checking of Uniformity (LZ)
10
1
2
2
10
)10
(
i m
mFi
10
1
22 )(
i iSmSimFi
.0924485.0,1028565.0
,1098615.0,0858035.0
,0911150.0,1369235.0
,0844650.0,1076910.0
,0791270.0,1097085.0
109
87
65
43
21
SSSSSSSSSS
P-value of P-values (before)
P-value of P-values (after)
Summary We corrected two points for DFT test. (1) the threshold T (2) the variance of the theoretical distribution
We corrected two points for LZ test. (1) setting of standard distribution (asymmetric) which
has no algorithm dependence. (2) re-definition of the uniformity of P-values.
42 npq2
2 npq
n3 n995732274.2
