1

Computer Science 129

Science, Computing and Society

Week 4

Chapter 3

2

GRE/GMAT WORDS

• Homogeneous – common origin

• Loquacious – talkative

• Pragmatic – practical values and results

• Volatile – explosive, unstable

3

Chapter 3

INFORMATION IS PHYSICAL

Edward Fredkin:“the universe is a computer”

Consider flight simulators

Fredkin believes computer running the program of our universe is not located in our universe

4

Chapter 3

Strange theory….but….

Processing information is a common feature of the universe

Brains do it!

All communication is just information processing

5

Chapter 3

The main idea of this is:

• Information is something real and important in the description of nature

• Information is not abstract

• Information is physical

6

Chapter 3

Consider Complex systems: the universe is full of them

Chemicals in our bodies - Black holes –Summer thunderstorms

All are information-processing mechanisms

They acquire information about the environment and use it to decide what to do next.

7

Chapter 3

Information processing systems are essentially “observers”

But, in classical physics, observers are irrelevant

Laws of nature should be unaffected by observation – they are there whether anyone sees them or not

8

Chapter 3

19th century Physicist James Maxwell proposed a hypothetical observer known as his demon

This demon related to the story of Thermodynamics

9

Chapter 3

The 1st and 2nd Laws of Thermodynamics

Thermodynamics – study of energy (heat) change (Empirical Laws)

1st - Total energy of a system plus the surroundings is constant – or The Law of Conservation of Energy

2nd – Entropy (disorder) of the universe increases – energy can’t even break even

10

Chapter 3

The Second Law

requires any use of energy to produce some “waste heat” that is too degraded to do further work

Order gives way to disorder - or entropy

11

Chapter 3

James Maxwell – known for explaining Electricity and Magnetism using mathematics (Maxwell’s Equations)

Maxwell believed the 2nd Law was a statistical law – based on probabilities

Consider speed of molecules (temperature)

12

Chapter 3

In measuring temperature

you can precisely measure the average speed of the molecules

not the exact speed of any one molecule

13

Chapter 3

Maxwell’s demon – 2 rooms with same temperatures – door between is controlled by demon – demon lets fast moving molecules into one room and slow into the other

One room hot, other very cold – demon could heat or air-condition a house

Demon gets useful energy from waste heat – breaking 2nd Law

Or did it?

14

Chapter 3

No – it didn’t

The Demon , must be made, then it must find out how fast a molecule is going – measuring it, all of which requires energy – more energy that it could get out of the system

http://maxwells-demon.freeonlinegames.com/

15

Chapter 3

Shannon’s Entropy

Shannon’s math measured information in bits

(information theory : 21=1 bit and two possibilities, 22=2 bits and 4 possibilities, 23=3 bits and 8 possibilities and so on)

16

Chapter 3

• Remember our base 2 system?

• 1 bit = 2 possibilities ; zero=0, one=1

• 2 bits = 4 possibilities ; zero=00, one=01, two=10, three=11

• 3 bits = 8 possibilities ;Zero=000, one=001, two=010, three=011, four=100,

five=101, six=110, seven=111

17

Chapter 3

Entropy in computer science measures uncertainty

Using bits to quantify weather uncertainty:

18

Chapter 3

In the morning there are the following possibilities (pretend these are like math givens. There are only these possibilities, no more, no less):

Sunny all day Cloudy then Sunny

Cloudy all day Sunny then Storm

Storm all day Cloudy then Storm

Sunny then Cloudy Storm then Cloudy

8 possibilities require 3 bits (23=8) or 3 bits of uncertainty

19

Chapter 3

Jill is the weather person on channel ZHer weather forecast is essentially:GoodBadThis can be represented with 1 bit (21=2)If we have 3 bits of uncertainty and 1 bit of

information – missing 2 bits – even if she is always right, we still have more uncertainty than we do information

20

Chapter 3

Shannon’s equations look very similar to the 2nd Law of Thermodynamics

This raises the question: Is there a real connection between the entropy of physics and the entropy of information?

21

Chapter 3

Information is always embodied in some physical representation – ink on paper – holes in punch cards, magnetic patterns on a floppy disk – arrangement of atoms in DNA

Information is Physical

22

What You Should Know

• WHAT IS THE FIRST LAW OF THERMODYNAMICS?

• WHAT IS THE SECOND LAW OF THERMODYNAMICS?

23

What You Should Know

SHORT ESSAY:

EXPLAIN MAXWELL’S DEMON AND HOW DOES MAXWELL’S DEMON RELATE TO THE SECOND LAW OF THERMODYNAMICS?

24

What You Should Know

HOW DOES INFORMATION THEORY WORK?

25

What You Should Know

SHORT ESSAY:

EXPLAIN HOW SHANNON’S ENTROPY WORKS. GIVE EXAMPLES.