Much ado about randomness. What is really a random number?

transcript

Much Ado About Randomness

Aleksandr Yampolskiy, Ph.D.

Randomness

Random number generation is easy to get wrong.

In Theory• Cryptography is based on random

numbers:– Secret keys must be random– Random bits are needed for public-key

encryption, signatures, SSL, etc.• A common assumption in theory is a

random oracle model, where all parties have access to a perfect random source [BR92].

• Under this assumption, many crypto tools can be proven formally secure.

In Practice

• Kaminsky bug used to poison DNS caches.• Debian OpenSSL versions <0.9.8g-9

generated weak SSL keys.• Some Majordomo versions were susceptible

to subscribing victim to thousands of mailing lists.

• Kerberos 4 secret keys could be guessed in a few seconds.

• Netscape 1.1 generated SSL keys using time and process ID as seed; easily guessable and breakable.

Example #1

- unsigned char magic_cookie[LEN]; - srand(time(NULL)); - for (int i=0; i<LEN; i++) magic_cookie[i] = rand()

& 0xFF;

X Windows “magic cookie” used a weak LCG generator and was guessable in X11R6.

Example #2

protected void doStart(){//… _random=new java.util.Random();

_random.setSeed(_random.nextLong()^System.currentTimeMillis()^hashCode()^Runtime.getRuntime().freeMemory()); _sessions = new MultiMap();

//...}

Jetty 4.2.26 used java.util.Random to generate predictable session ID which could be brute-forced.

Example #3• global variable seed; • function RNG_CreateContext()

– (seconds, microseconds) = time of day; – pid = process ID; ppid = parent process ID; – a = mklcpr(microseconds); – b = mklcpr(pid + seconds + (ppid << 12)); – seed = MD5(a, b);

• function mklcpr(x) // simple scrambler– return ((0xDEECE66D * x + 0x2BBB62DC) >> 1);

• function MD5(x) // secure hash function

In 1996, two UC Berkeley students reverse exploit Netscape 1.1

Lessons Learnt

• Numbers, used to derive session IDs and keys, weren’t truly random!

• Seeds must be unpredictable– 128 bit sequences are sufficient– All possibilities equally likely– Best seeds are truly random

• PRG (pseudorandom number generator) must be secure– No detectable pattern– Even if attacker guesses some pseudorandom bits, no

correlation to other bits.

Two Types of Randomness

• Truly random number generator– Radioactive decay.– Disk timing.– Fair coin flip.– Randomness inherent in PC disk IO, thread scheduling, etc..

• Pseudo-random number generator (aka computational)– Generate a small “truly random” seed.– Stretch into a larger pseudo-random sequence.

• Fact: In practice, we use pseudo-random number generators.

What is (Pseudo)-Random?

random seed

pseudorandom string

impossible w.r.t. computationally unbounded observer

possible w.r.t. computationally bounded observer if the PRG is “hard to invert” relative to the observer

01010111001…

What is (Pseudo)-Random? (cont.)

random seed

pseudorandom string random string

look indistinguishableto any efficient observer

Definition [Blum-Micali-Yao]: PRG is a polytime function whose output is indistinguishable from random by any efficient observer

01010111001… 11010011010…

Find programs with weak PRG Find programs with weak PRG

Break-in Break-in

Attacking Weak PRGs

Guess the initial seed of a PRG

Guess the state of a PRG

Finding programs with weak PRG• If source code is available, grep for weak API calls.• If only a binary is available, reverse engineer the

program or grep for weak system calls.• For client programs, use Stompy or Ent to analyze

output’s randomness quality.• For web-based programs, use BurpSuite or

WebScarab proxy to analyze session ID randomness.

• Google Hacking for weak session IDs.

Finding programs with weak PRG (cont.)

• High-entropy session ID generators use things like:• java.security.SecureRandom (Java)• System.Security.Cryptography.RNGCryptoServiceProvider (.NET)• /dev/urandom, /dev/(s)random (if the latter, look for exhaustion attacks!)• OpenSSL’s RAND bytes• hardware security module.

• It’s pretty easy to quickly identify weak, low-entropy session ID generation in the code. They use

• the time and date• a random static string in the source code• the output of C library rand, the output of java.util.Random• small (32 bits or less) numbers• a cryptographic hash (like MD5) of anything low in entropy to generate

their session IDs.

Know Weak API

•The weak API generally use insecure constructions such as LCG, LSFR, Mersenne twister, etc.

•The strong API may use DES, SHA-1 based PRNG, Blum-Blum-Shub, etc.

Reverse Engineering The Binaries

• Many Java programs mistakenly use java.util.Random to generate session IDs instead of java.security.SecureRandom

root# javap -c BadRandom | grep RandomCompiled from "BadRandom.java"public class BadRandom extends java.lang.Object{public BadRandom(); 0: new #2; //class java/util/Random 4: invokespecial #3; //Method java/util/Random."<init>":()V 24:invokevirtual #9; //Method java/util/Random.nextInt:()I

Reverse Engineering the Binaries

• Similarly, C/C++ programs use rand() or random() instead of reading from /dev/random.

• Note /dev/random blocks and /dev/urandom doesn’t but may produce randomness of worse quality.

root# strings bad_random__gmon_startlibc.so.6_IO_stdin_usedsrandtimeprintf…

root# nm bad_random | grep rand U rand@@GLIBC_2.0 U srand@@GLIBC_2.0

Analyzing the Output Without Binaries

• Compute entropy of the stream• Count number of characters in each position• FIPS 140-2 statistical PRNG tests

– Monobit test: Are there as many 1’s as 0’s?– Runs test: Are the number of runs (sequences of only 0’s

or 1’s) as expected for random numbers?– Maurer’s test: Can the sequence be compressed?– Next-bit test: given m bits of the sequence, predict (m+1)st

bit• Just compress the data using WinRAR

java.util.Random

•The java.util.Random PRG is really a linear congruential generator (LCG) where x(n+1) = axn + b (mod m) for large constants a, b and moduli n,m

• synchronized protected int next(int bits) { seed = (seed * 0x5DEECE66DL + 0xBL) & ((1L << 48) - 1); return (int)(seed >>> (48 - bits)); }

Entropy of java.util.Random• In practice things aren’t that bad.• For 20,000 samples, the entropy of java.util.Random and

java.security.SecureRandom streams is almost identical.• For both, 14.2877123795 bits of entropy.• They also pass all FIPS 140-2 tests.

• For 200,000 samples, java.security.SecureRandom has slightly more entropy than java.util.Random, but is it significant?• For java.util.Random, we get 17.6095804744 bits of entropy• For java.security.SecureRandom, we get 17.6096204744 bits of

entropy

Is java.security.SecureRandom that much worse than java.util.Random?

• Folklore says that it is. But it really depends on OS:

• OpenSolaris (SunOS 5.11) : 67.9 slower • Windows XP, 64.5 times slower • Windows 7, 24.5 times slower • MAC OS X, Leopard: 25.1 times slower

Hacking Java bytecode to use SecureRandom• Java.security.SecureRandom inherits from java.util.Random

and has all its methods• ASM bytecode manipulation framework: http://asm.ow2.org/ • Replace Random with SecureRandom in the bytecode

public class ChangeMethodCallAdapter extends MethodAdapter {

@Override public void visitMethodInsn(int opcode, String owner, String name, String desc) { System.out.println("ChangeMethodCallAdapter(): opcode=" + opcode + ",owner=" + owner + ",name=" + name + ",desc=" + desc); if ("java/util/Random".equals(owner)) { mv.visitMethodInsn(opcode, "java/security/SecureRandom", name, desc); } else { mv.visitMethodInsn(opcode, owner, name, desc); } }

gilt-ml-ayampolskiy:ClassTransformer ayampolskiy$ javap -c API | grep Random 8: new #5; //class java/util/Random 12: invokespecial #6; //Method java/util/Random."<init>":()V 27: invokevirtual #7; //Method java/util/Random.nextInt:(I)I

gilt-ml-ayampolskiy:new ayampolskiy$ javap -c API | grep Random 8: new #28; //class java/util/Random 12: invokespecial #31; //Method java/security/SecureRandom."<init>":()V 27: invokevirtual #35; //Method

java/security/SecureRandom.nextInt:(I)I

Google Hacking

• Know the common session cookie names (SESSIONID,JSESSIONID,PHPSESSID,PHPSESSIONID, etc.)

• Google for the cookie names: inurl:"?sessionid=”• Even better, try googling session IDs with non-

random sequences “66”, “128”: inurl:”?sessionid=128”

• How about “lang:java java.util.Random session”

Testing Randomness of Client Programs

• Fourmilab’s entropy tests: http://www.fourmilab.ch/random/

• Stompy (session stomper): http://lcamtuf.coredump.cx/stompy.tgz . Seems to be too “optimistic”

“We could not arrest or charge this suspect because technically, no offence was being committed as there was no legislation in place to say that the act being committed was criminal. So, we had to let him go,” said Sergeant Jemesa Lave of the Fiji Police Cyber Crime Unit.

Amazon.com experiment

• Amazon.com uses a session-id, a 17-digit random number- is a persistent cookie that expires after 7 days. It is set the first time you reach Amazon. Its value does not change after you log in, nor when you switch users.

Testing Randomness of Web-Based Programs

• Several nice GUI tools to analyze session IDs for common problems ( WebScarab, BurpSuite, SPI Cookie Cruncher,Foundstone CookieDigger, etc)

• Test alphabet distribution, average bits changed, FIPS tests, etc.

WebScarab – Predictable Cookies

Entropy is a measure of uncertainty regarding a discrete random variable. For many purposes, the Shannon entropy is the only measure needed. Shannon entropy is defined byShannon (4.1)has the unit bits.

Not amazon.com

WebScarab – amazon.com

Burpsuite - amazon.com

BurpSuite – amazon.com

Typical amazon.com session-id 180-3029497-6907862

BurpSuite – amazon.com

Conclusion

• Use good seeds and strong PRNGs.• Know what the strong API for generating secure

random numbers are (SecureRandom, /dev/random)

• Try out Stompy, Ent, WebScarab, BurpSuite.• Happy hacking!

Questions, Comments?

Much ado about randomness. What is really a random number?

Technology