Mountain Video

Showed video of mountain landscape generated by the 4k file on this page:

http://pouet.net/prod.php?which=52938

A Random Talk About Random

Dave Feinberg

Which has more information?

The outcome of 2 coin flips: heads heads

The outcome of 3 coin flips: tails tails heads

Which has more information?

The outcome of 10 coin flips: "HHTTHTHTTT"

A strand of 10 DNA bases: "ATTGACATGG"

10 decimal digits: "7523104698"

Units of Information

bit = binary digit a single "0" or "1" outcome of 1 binary decision like a coin flip ten coin flips -> ten bits

Bits

How many bits do I need to store the outcome of one home pregnancy test?

A home pregnancy test reveals 1 bit of information. It may be the most important life-changing bit of information, but it's still just 1 bit.

One Itty Bitty Guy

Representing in Binary "HHTTHTHTTT"

"H" = 0 "T" = 1

0011010111

How can we represent "ATTGACATGG" in binary?

Representing in Binary 2 bits to represent 1 DNA base

"A" = 00 "C" = 01 "G" = 10 "T" = 11

ATTGACATGG 00111110000100111010 How many bits?

Representing in Binary

bits per symbol =log2(possible symbols)

4 symbols: ACGT

log24 = 2 bits per symbol

Representing in Binary How much information in

"7523104698"? 10 possible symbols log210 = approx 3.32 bits per symbol 10 decimal digits = approx 33.2 bits Bogus! How many bits does it really take to

represent 10 possible symbols?

decimal digits in 4 bits each

0 = 0000 5 = 0101 1 = 0001 6 = 0110 2 = 0010 7 = 0111 3 = 0011 8 = 1000 4 = 0100 9 = 1001

Seems wasteful. Can we do better?

decimal digit = 3 or 4 bits 0 = 000 5 = 101 1 = 001 6 = 1100 2 = 010 7 = 1101 3 = 011 8 = 1110 4 = 100 9 = 1111

What number is "0101011100"? On average: 3.4 bits per digit.

Which contains most information?

Outcome of 6 coin flips: "HHTHTT"

3 DNA bases: "ATG"

2 decimal digits: "74"

Information

more bits = more information

Right?

Memorizing

Volunteer to memorize 8 bits 00011110

Volunteer to memorize 50 bits 00000000000000000000000000000000

000000000000000000

Which is easier to memorize?Which contains more information?

Memorizing 0000000000000000000000000000000000000

0000000000000

Another volunteer to memorize 50 bits 0010001100111010011011000101000110001

0101001111001

Which is easier to memorize?Which contains more information?Why?

Information:another definition

Amount of information = length of shortest program that outputs those bits.

Write a program to print

00000000000000000000000000000000000000000000000000

for i in range(50):print "0",

Write a program to print

00100011001110100110110001010001100010101001111001

print "00100011001110100110110001

010001100010101001111001"

Memorizing Programs

print "00100011001110100110110001

010001100010101001111001"

is harder to memorize than

for i in range(50):print "0",

Therefore

00100011001110100110110001010001100010101001111001

contains more information than

00000000000000000000000000000000000000000000000000

Information

More random = more information

Pi and Information

3.14159265358979323846264338327...

How much information is stored in the digits of pi?

Calculating Pi

pi = 4/1 - 4/3 + 4/5 - 4/7 + ...

Calculating Pi

sign = 1n = 1pi = 0while True:

pi = pi + sign * 4.0 / nprint pisign = 0 - signn = n + 2

Information in Pi The digits of pi are output by a

short program.

Therefore, pi does not contain very much information.

(Then what does???)

Programs That Print Numbers

Write a program to print0.1

print "0.1"

Programs That Print Numbers

Write a program to print0.11111...

print "0.",while True:

print "1",

Programs That Print Numbers

Write a program to print0.121212...

print "0.",while True:

print "12",

Programs That Print NumbersWrite a program to print

0.12112111211112111112...

print "0.",ones = 1while True:

for i in range(ones):print "1",

ones = ones + 1print "2",

Programs That Print Numbers

Although 0.121121112... requires an infinite number of decimal digits,

It can be printed by a program of finite length.

Programs That Print Numbers

Are there any numbers that cannot be printed by a computer program?

Yes!

Programs That Print Numbers A number that can be printed by a

computer program is called a computable number.

A number that cannot be printed by a computer program is called an uncomputable number.

Are there a lot of uncomputable numbers?

The Number Line If I point to a

random point on the number line, what is the probability that I will point to an integer?

A rational (fraction)? A computable

number?

-1 0 1 2

Information and Science more random = more information. randomness = disorder. In science, what do we usually call the

measure of the amount of disorder in a system?

information = randomness = disorder = entropy

2nd Law of Thermodynamics: In a system, a process that occurs will tend to increase the total entropy of the universe.

Does this tell us anything about information?

Something Different ...

We'll come back to random/information ...

Let's make some pictures

Rewrite rule: Add square to long side.

Tiling Squares

Another Redraw Rule

Shrink and arrange 3 copies

Redrawing ...

Redrawing ...

Redrawing ...

Redrawing ...

Redrawing ...

Fractals A fractal is:

"a rough or fragmented geometric shape that can be split into parts, each of which is (at least approximately) a reduced-size copy of the whole"--a property called self-similarity.--Wikipedia

Fractals are recursive structures.

Triangles?

Why is the sierpinski triangle full of little triangles? Where did the triangles come from?

Squares ...

Shrink and arrange 3 copies

Squares ...

Shrink and arrange 3 copies

Squares ...

Shrink and arrange 3 copies

Squares ...

Shrink and arrange 3 copies

Squares ...

Shrink and arrange 3 copies

Squares ...

Shrink and arrange 3 copies

Fractal Antennas

(Gratuitous connection to science)

Some self similar fractal shapes have a property of "frequency invariance"—the same electromagnetic properties no matter what the frequency. --Wikipedia

Another Redraw Rule

Connect midpoints

Another Redraw Rule

Connect midpoints

It makes a disco floor!

Another Redraw Rule

Randomly move midpoints and connect.

Another Redraw Rule

Randomly move midpoints and connect.

Another Redraw Rule

Randomly move midpoints and connect.

Another Redraw Rule

Randomly move midpoints and connect.

Another Redraw Rule

Randomly move midpoints and connect.

Another Redraw Rule

What does thislook like?

Mountains I Generated

Connections

Approximate fractals are easily found in nature. These objects display self-similar structure over an extended, but finite, scale range. Examples include clouds, snow flakes, crystals, mountain ranges, lightning, river networks, cauliflower or broccoli, systems of blood vessels and pulmonary vessels, coastlines, tree branches, galaxies, etc. (borrowed from Wikipedia)

Fractals

All these natural phenomena are fractal-like.

What does that tell us about nature?

Sandwich Video

http://www.colbertnation.com/the-colbert-report-videos/340908/july-07-2010/thought-for-food---kentucky-tuna---grilled-cheese-burger-melt

Started at 4:50.(Not everything earlier is

appropriate.)

The Mountain Video

Was produced from somebody's 4 kilobyte computer program.

How large is 4 kilobytes?

Is there a lot of information in that video?

Determinism The computer is deterministic.

Follows rules, step by step.

Does that mean a program does the same thing every time, given the same input?

Can a computer behave randomly?

Cellular Automata A kind of toy universe.

Download and run: CellularAutomata.jar

Try running "rule 18" one generation at a time. Can you tell what it's doing?

Rule 18

For each cell, look at the 3 cells on the row immediately above it (immediately above, above-and-to-the-left, and above-and-to-the-right).

If the middle is white and either the left or the right is black (but not both), then this cell will become black. Otherwise, it will be white.

Now You Try

On a sheet of graph paper, put a black mark in the middle square along the top of the page. This top row is the first generation. Use the rule to fill in the black marks for the 2nd generation, 3rd generation, and so on.

What do you see?

Cellular Automata

In CellularAutomata.jar, look at rule 18.

How many bits does it take to identify a particular rule?

How many possible rules are there?

Why is it called rule 30?

Rule 30 Run rule 30 on random start. It looks very random. Where did that

randomness come from? Run rule 30 from a single center point. It still looks very random. Where does the randomness come from? Read off the sequence down the middle

column:00100011001110100110110001010001100010101001111001

Familiar? How much information is in this sequence?

Pseudo-random Rule 30 exhibits pseudo-randomness. Generated by an algorithm, but chaotic

output. Fools us into thinking it's random. Computers use pseudorandom number

generators to produce "random" behavior.

An Exercise

Start with a number from 1 to 10.

Multiply by 7, and find remainder when divided by 11. That's the next number.

Repeat.

An Exercise

(7n) mod 11

Let's start with 4.What sequence do we get? 4, 6, 9, 8, 1, 7, 5, 2, 3, 10, 4, ...

Does it look random?

Pseudo-Random "(7n) mod 11" exhibits pseudo-randomness.

A deterministic algorithm with chaotic output.

Fools us into thinking it's random.

Computers use pseudo-random number generators to produce "random" behavior.

An Exercise

"(7n) mod 11" generated: 4, 6, 9, 8, 1, 7, 5, 2, 3, 10, 4, ...

What if I want to generate a longer pseudo-random sequence?

Pseudo-Random (7n) mod 101

Starting from 4: 4, 28, 95, 59, 9, 63, 37, 57, 96, 66, 58, 2, 14, 98, 80, 55,

82, 69, 79, 48, 33, 29, 1, 7, 49, 40, 78, 41, 85, 90, 24, 67, 65, 51, 54, 75, 20, 39, 71, 93, 45, 12, 84, 83, 76, 27, 88, 10, 70, 86, 97, 73, 6, 42, 92, 38, 64, 44, 5, 35, 43, 99, 87, 3, 21, 46, 19, 32, 22, 53, 68, 72, 100, 94, 52, 61, 23, 60, 16, 11, 77, 34, 36, 50, 47, 26, 81, 62, 30, 8, 56, 89, 17, 18, 25, 74, 13, 91, 31, 15, 4, ...

This gives us 100 numbers, in the range 1 to 100.

Suppose we want 100 numbers in the range 0 to 9 ...

Pseudo-Random

Print ones digit only: 4, 8, 5, 9, 9, 3, 7, 7, 6, 6, 8, 2, 4, 8, 0,

5, 2, 9, 9, 8, 3, 9, 1, 7, 9, 0, 8, 1, 5, 0, 4, 7, 5, 1, 4, 5, 0, 9, 1, 3, 5, 2, 4, 3, 6, 7, 8, 0, 0, 6, 7, 3, 6, 2, 2, 8, 4, 4, 5, 5, 3, 9, 7, 3, 1, 6, 9, 2, 2, 3, 8, 2, 0, 4, 2, 1, 3, 0, 6, 1, 7, 4, 6, 0, 7, 6, 1, 2, 0, 8, 6, 9, 7, 8, 5, 4, 3, 1, 1, 5, 4, ...

Pseudo-Random "7n mod 101" gives us the same numbers

in the same order every time.

If I want different behavior, I need to start at a different place in the sequence.

How do I choose which number to start from?

Pick a random one to start with???

Pseudo-Random

I look at the current time to decide where to start.

Why don't I just look at the time for every value?

Keno

Anyone know how to play the casino game Keno?

Numbers 1 to 80. You pick 20. The casino chooses 20. The prize depends on how many you get correct.

Keno In April 1994, college student Daniel Corriveau

won $600,000 CAD playing keno at the Casino de Montréal.

He picked 19 of the 20 winning numbers. What are the odds of that? 1 in 3 thousand million million. If you play once a minute, you would pick 19-

out-of-20 about every 5.6 billion years. Daniel Corriveau picked 19 out of 20, 3 games

in a row. How did he do that?

Keno Keno is run on a computer. The numbers are

generated by a pseudo-random number generator.

When the machine is turned on, it uses a clock chip to determine where to start in the pseudo-random number sequence.

24-hour casinos only turn on the Keno machine once ever.

The clock chip for a Keno machine is sold separately.

Therefore, some casinos stopped buying clock chips.

Keno But this particular casino wasn't open

24 hours. So its Keno machine started once a day. Without the clock chip, it always started

at the same point in its pseudo-random number sequence.

So the sequence of winning numbers was the same every day.

Daniel Corriveau got to keep his winnings.

True Randomness?

If this is pseudorandomness, what processes result in true randomness?

Is flipping a coin random or pseudorandom?

Food For Thought

Consider the complex arrangements of atoms that make up the people and things you see. Could some of that complexity have arisen from simple rules, too?

Food For Thought

Is natural selection a set of simple rules that gets applied over and over again?

How about the laws of physics?

Big Ideas more random = more information

Complexity and pseudo-random behavior can arise from deterministic processes, where we repeatedly apply simple rules.