Lecture 07 Analog-Digital Claude Shannon, 1916-2001...

Lecture 07 Analog-Digital

Computation

1Discrete Structures - 2015

Alan Turing, 1912-1954

Claude Shannon,1916-2001

The Most Important Man you have Never Heard of


Claude Elwood Shannon 1916 - 2001

Claude Shannon’s Master Thesis at MIT (1937)


• A switch between two terminals is a device that is either closed (connecting the terminals), or open (disconnecting the terminals)

- For example, a light switch opens and closes the flow of current from a source to your light bulb

Basic Problem Studied in the Thesis


Closed switch.Current flows.

Open switch.Current doesn’t flow.

• A switch network is a network of switches between a set of input terminals and a set of output terminals

- The switches are set from the outside

• Example: two light switches for the same light bulb, so that the light is on if at least one of the switches is on



• Light bulb should be on iff exactly one of the switches is on.

Another Example


Evil twin of the original. Does exactly the opposite of what its counterpart switch does.


• Such switch networks are not restricted to “light bulbs”

- Telephone switches

- Industrial control circuits

- Or… on any processor in any of your devices

• Shannon asked himself:

- How to “analyze” such circuits efficiently➡ Given a circuit, how do we make sure that it “computes”

what it is supposed to

- How to “synthesize” such circuits efficiently➡ Given a computation problem, how do we come up with

such circuits

• … And provided solutions in his thesis.


• Such circuits are essentially the same as compound propositions

• Any circuit can be composed of the AND, the OR, and the NEGATION circuits

• Representations are not unique, and can be optimized to include a small number of such elementary “gates”

Basic Insights


X

Y

X _ Y

X Y

X ^ Y

X

¬X

Switch is ON = True = 1Switch is OFF = False = 0

• Any truth table can be implemented with - NOT:

- AND:

- OR:

• Hand-waving proof is on the right

• Example suffices for now - For every row with value True,

create formula that is True in that row, and False everywhere else

- Take the “OR” of the formulas

• Enormously important in computer architecture

Logical Completeness


¬^_

?T T T F

T T F T

T F T T

T F F F

F T T F

F T F T

F F T T

F F F F

p q r

(p ^ q ^ ¬r)(p ^ ¬q ^ r)

(¬p ^ q ^ ¬r)(¬p ^ ¬q ^ r)

True if and only if p is True, q is True, r is False. Why?

(p ^ q ^ ¬r) _ (p ^ ¬q ^ r) _ (¬p ^ q ^ ¬r)

_(¬p ^ ¬q ^ r)

• is complete!- Suffices to write the AND and the

OR in terms of NAND and the negation

NAND and NOR


NAND(x, y) = ¬(x ^ y)

NOR(x, y) = ¬(x _ y)

(NOR,¬)

(NAND,¬)

• is complete!- Suffices to write the AND and the

OR in terms of NAND and the negation

x _ y = ¬((¬x) ^ (¬y)) = NAND(¬x,¬y)x ^ y = ¬(¬(x ^ y)) = ¬NAND(x, y)

x ^ y = ¬((¬x) _ (¬y)) = NOR(¬x,¬y)

x _ y = ¬(¬(x _ y)) = ¬NOR(x, y)

You will learn this concept much more precisely in your “Computer Architecture” classes (CS-208, CS-307)

Logic Gates


NOT

AND

NAND

NOR

XOROR

Addition of two one-bit integers: a + b = v + 2*c

Synthesis, According to Shannon


0 + 0 = 0 + 2*00 + 1 = 1 + 2*01 + 0 = 1 + 2*01 + 1 = 0 + 2*1

a b v c

0 0 0 0

0 1 1 0

1 0 1 0

1 1 0 1

v = (¬a ^ b) _ (a ^ ¬b)

= NAND(NAND(a,¬b),NAND(¬a, b))

c = a ^ b

= NOR(¬a,¬b)

NAND

NAND

NOT

NOT

NAND

NOR

a

b

v

c

(a+2b)+(c+2d) = u + 2v + 4w = (a+c) + 2(b+d)

Addition of two 2-bit Integers


a

b

uc

d

NAND

NAND

NOT

NOT

NAND

NOR

NAND

NAND

NOT

NOT

NAND

NOR

NAND

NAND

NOT

NOT

NAND

NORNAND

NAND

NOT

NOT

NAND

v

w

See how the same block is repeated?

Triode Vacuum Tubes


https://en.wikipedia.org/wiki/Vacuum_tube

Lee De Forest, 1906

https://en.wikipedia.org/wiki/Vacuum_tube

Triode Vacuum Tubes as Switches


Cathode

Plate

Gate

Negative Voltage

Electrons repelledby gate and don’t arrive at plate.Switch is off.

Positive Voltage

Electrons attractedby gate and pass to plate. Switch is on.



Lowvoltage

Switch is off

Highvoltage

Switch is on



0

Switch is off

1

Switch is on

Implementation of an Inverter


0

1

I

R V=I*R

1

0

I

R V=0

http://www.vintchip.com/mainframe/BENDIXG-15/BENDIXG15.html

Buffer inverter from Bendix G15 Computer

NOT

Implementation of NAND and NOR


NAND(X,Y )

NOR(X,Y )XX

YY

There is no “clean” 0 or “clean” 1

The Reality


0 1Forbidden regime

The circuitry has to be designed so a voltage in the forbidden regime occurs with very very very low probability (for example 10-30)

Division of labor• Circuit designers: guarantee 0 and 1• Digital designers: Design efficient digital circuits that can compute complex

functions

Distribution of voltage values

$ $ $ $ $ $

$ $ $ $ $ $ 0 1

This is one (of the many, many) reason(s) you need to learn probability theory and statistics (MATH-232)

Problems with Vacuum Tubes

• Bulky, big

• Can’t switch from a 0 to a 1 very fast

• Static current- For example, inverter needs to consume current to keep itself at “0”

- Power consumption can be high

• In general, high power consumption because of size

• Complex circuitry is very difficult to build with them


• Founded in 1925

- As a subsidiary of AT&T and Western Electric

• The name comes from Alexander Graham Bell, the inventor of the telephone

- Though the correct assignment of inventors has a long and complicated history….

• Bell Laboratories was mainly concerned with research on all aspects of long distance telephony

Bell Laboratories


Aerial view of Bell Labs in Murray Hillhttp://users.ece.gatech.edu/~juang/B%20JUANG%20Georgia%20Tech%20Pictures.html

Partial List of Bell Labs Innovations

• First Fax Machine (1920’s)

• Long distance television transmission (1920’s)

• Radio astronomy (1930’s)

• Relay computer (1930’s)

• Transistor (1947)

• Error-correcting codes (1940’s)

• Information Theory (foundation of modern communication - 1948)

• Signal processing fundamentas (Nyquist)

• Calculators (1940’s-50’s)

• Direct distance dialing (1950’s)

• First modem (1960)

• Communication satellite (1962)

• Laser (1964)

• Cosmic background radiation (1965)

• Computer animation (1960’s)

• UNIX (1969)

• C programming language (1972)

• Fiber optics systems

• HDTV

• TDMA/CDMA (for wireless multiple access systems)

• Wireless telephony (fundamentals by Bell Labs, first one by Motorola in 1973)

• C++ programming language (1986)

• And a LOT more…..


Transistor

• Bell Labs was interested in a telephone switching system that could handle calls made within the entire United States, and later, around the world- Switching systems based on vacuum tubes were bulky, and

consumed too much power

- Bell Labs wanted a solid-state equivalent that was smaller, faster, and more efficient

• They set out to “invent” the transistor- Remarkable achievement, since they didn’t know whether such a

thing existed at all

- Several years of work

- Final discovery was somewhat “accidental”


Invention of the Transistor (End of 1947)


Bardeen, Shockley, and Brettain, 1948Received the Nobel Prize in Physics in 1956

First Transistor

Bipolar Junction Transistor (npn)


NMOS Transistors


Low voltage

High voltage

You will learn this concept much more precisely in your “Electronics” classes (EE-202)

Implementation of the Negation Gate


01 0 1

Current “bleeds” to ground.Static power dissipation.

Transistors allow for much better implementation

PMOS and NMOS Transistors


High voltage

Low voltage

Low voltage

High voltage

NMOS transistors PMOS transistors

You will learn this concept much more precisely in your “Electronics” classes (EE-202)

Doesn’t have analog device with vacuum tubes.



1 0•

•

Closed

Open

1

•

•Open

Closed

0

Doesn’t bleed to ground!



NOTX ¬X

High voltage

Low voltage

Closed

Open

High voltage

Low voltage

Open

Closed

X ¬X1 00 1

Implementation of the NOR Gate


Low

LowHigh

LowLow

HighHigh

HighLow

Low

Low

High

NORX

Y

Closed Open

X Y ¬(X ^ Y )

1 1 0

1 0 0

0 1 0

0 0 1

¬(X _ Y )

Implementation of the NAND and the AND Gates


HHLLHLHL

LHHH

NAND

X

Y

¬(X ^ Y )

HHLLHLHL

HLLL

X

Y

AND (X ^ Y )

Note how an inverter is addedin order to produce AND from NAND

You will learn more about digital systems design in your course CS-173

• Discrete transistors were difficult to connect

- Too many wires

•The next big idea was the use of “integrated circuits

- Put all the electronic circuits onto one small plate of semiconductor material (typically silicon)

- The entire circuit can be made much smaller

• Today manufacturing uses the process of optical lithography

Integrated Circuits


http://www.just2good.co.uk/cpuSilicon.php

ARMv6 processor

• Billions of transistors on a single chip

• Reduction of size from generation to generation- Less power consumption

- More transistors ➡ Higher functionality of the

end device

- Lower cost

• We are using transistors with a pitch of 28nm in our devices today, moving towards 10nm and below in the future

The Result


Processor Transistor count Manufacturer Transistor pitch

ARM-1 25,000 ARM Holdings 3000 nm

Intel i-960 250,000 Intel 600 nm

Pentium Pro 5,500,000 Intel 500 nm

Pentium III 45,000,000 Intel 130 nm

Pentium 4 184,000,000 Intel 65 nm

Apple A7 1,000,000,000 Apple 28 nm

Apple A8 2,000,000,000 Apple 20 nm

Xeon Haswell 5,560,000,000 Intel 22 nm

Fiji (GPU) 8,900,000,000 AMD 28 nm

iPhone 6

iPhone 5S

PC’s

Gaming

Computers

• Today’s computing era is (at a technical level) dominated by - Shannon’s thesis (digital design)

- The transistor (Bell Labs)

- Integrated circuits (Texas Instruments, Intel)

- Manufacturing processes (Intel, IBM, TSMC, etc.)

• But this is only one side of the story…

- What can (and can’t) be done with computers?


Alan Turing


Alan Turing 1912 - 1954

What is an Algorithm?

• A recipe?

- Algorithm for cooking Spaghetti Bolognese, for example

- Not satisfactory, though➡ Not completely reproducible…

• According to Wikipedia “a self-contained step-by-step procedure to be performed”

- Still not satisfactory

- Not precise enough to let us argue about it

• Another definition (Wikipedia) “An effective method that can be expressed within a finite amount of space and time and in a well-defined formal language for calculating a function”

- Oh well…. 38Discrete Structures - 2015

Check out your “Algorithms” class (CS-250) next year!

Algorithm Concept

• Concept has existed for a long time

- The name comes from the Persian mathematician “Al-Khawrizmi” from his book “On the calculation with Hindu Numerals” (825 AD)

• However, formalization happened with an attempt to solve Hilbert’s “Entscheidungsprobleme” (decision problems) he posed in 1928

- Difficult to even say what these are at this stage of your studies

• A precise formulation and investigation of the problem requires the definition of “computability” and that of algorithms.

• This was done independently by Alonzo Church and Alan Turing.

- One invented the notion of the lambda-calculus

- One invented what is today called a “Turing Machine”39Discrete Structures - 2015

The “Paper”


Proceedings of the London Mathematical Society (Ser. 2, vol. 42, 1937)

• Consist of

- an infinite tape (memory) from which they can read and on which they can write

- A “control” which upon reading the content of a cell performs some operation, and possibly writes another content on the same cell, and moves its head to the left or to the write

• The “program” is essentially the control, i.e., the behavior of the machine upon reading the contents of cells

• An “algorithm” is generally accepted to be the same as a Turing machine

• How “universal” are algorithms?

Turing Machines


http://physics.kenyon.edu/coolphys/thrmcmp/newcomp.htm

You will learn this concept much more precisely in your “Theory of Computation” class (CS-251)

You need set theory and the concept of functions/relations for a precise definition.

http://physics.kenyon.edu/coolphys/thrmcmp/newcomp.htm

Halting Turing Machines

• A Turing Machine can “halt” on some input (=content of tape at start of computation)

• Or it can go into an “infinite loop” (does not halt)- For example, the Turing Machine implementing the code on the right

does not halt➡ Caveat: this is NOT a real piece of computer code

• Halting problem: Does there exist a Turing Machine (call it a “Universal Turing Machine”) which accepts as inputs ➡ A description of a Turing Machine M➡ An input x to the Turing Machine M

- And decides whether M halts on input x?


i=0while (i≥0)

i = i+1end while

Trivial Algorithm (does not work)

• The answer should be easy, no?- Run M on input x and wait until it halts.

• But…..- What if M never halts on x?

- How can we distinguish between ➡ M never halting on x, and ➡ M taking a very very very long time to finish its computation on x?

• In fact, we cannot!

• Turing proved that there is NO Universal Turing Machine- Proof uses the concept of countability and Cantor’s diagonal principle

• It gives a negative answer to the “Entscheidungsproblem” posed by Hilbert


Turing Machines

• Turing machines allow us to also argue about the “efficiency” of algorithms that always halt- How much time do they need on their inputs before halting?

- How much space to they need to finish their computation?

• This allows to sub-divide computational problems into “efficient” and “inefficient” ones- PTIME (or simply P) for those that use “little” time to compute

- PSPACE for those that use “little” space to compute

- And a LOT more

• This leads to one of the biggest achievements of Theoretical Computer Science in the 20th century (NP-complete problems) and to one of the most notorious open problems- Is P = NP? (1 million dollar prize problem)


You will learn this concept much more precisely in your “Theory of Computation” class (CS-251)

Turing Machines in Nature?

• We generally accept Turing Machines as the equivalent of algorithms or computing devices

• But it is not clear that computing going on in nature is covered by Turing Machines- Is our brain a Turing Machine?

- How about the Nature in general➡ E.g., quantum effects?

• If they are not, then there is a vast source of computational power that we can tap into- And solve problems that are generally considered very hard using

traditional Turing Machines

• This is, however, purely speculative.


What Happened to Shannon Afterwards?

• Shannon finished his PhD at MIT in 1940 on “An Algebra for Theoretical Genetics”

• Joined Bell Labs

- During the heydays of research on transistors, though he was not affiliated with it

• And.. Founded “Information Theory” in 1948

- The theory that underlies cellular communication, image recognition/compression, hard disks, flash drives, and practically every device in our digital life➡ Arguably his biggest achievement

• After Bell Labs, he joined the Faculty at MIT in 1956

• He died in 2001 (Alzheimer’s disease) at the age of 84


You will learn more about Information Theory in “Information Sciences” COM-101, and COM-302,

What Happened to Turing?

• He went to Princeton University in 1936 to obtain his PhD - His paper on Turing Machines was written there, while he was a

graduate student!

• He obtained his PhD in 1938 under Alonzo Church

• He joined GC&CS (predecessor of GCHQ) in 1938, a British code-breaking organization, and developed the “Bombe” an electromechanical machine to break Germany’s Enigma- Much of his work was classified up until 2012

- Visited Bell Labs in 1942 as part of a trip to the US to work with US Navy Cryptanalysts (secure voice communication)

• In 1948 he joined the Faculty of the University of Manchester

• He was convicted of indecent behavior in 1952 because of his sexual orientation and was given the choice of a lengthy prison sentence or a hormonal therapy. - He chose the latter.

• He committed suicide in 1954 at the age of 42.


Date post:	13-Aug-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Lecture 07 Analog-Digital Claude Shannon, 1916-2001...

Documents