Information, Entropy and Reversible Computation

© Copyright Fujitsu Telecommunications Europe Ltd 1996-2000. All rights reserved

Information, Entropy andInformation, Entropy andReversible ComputationReversible Computation

Information, Entropy andInformation, Entropy andReversible ComputationReversible Computation

Michael C. ParkerMichael C. Parker11, and Stuart D. Walker, and Stuart D. Walker22

1:Fujitsu Laboratories of Europe 1:Fujitsu Laboratories of Europe Columba House, Adastral Park, Ipswich IP5 3RE, U.K.Columba House, Adastral Park, Ipswich IP5 3RE, U.K. [email protected] .com , [email protected]@fle.fujitsu .com , [email protected]: University of Essex2: University of Essex Dept. of Electronics Systems Engineering,Dept. of Electronics Systems Engineering, Wivenhoe Park, Colchester, Essex, CO4 3SQ, U.K. Wivenhoe Park, Colchester, Essex, CO4 3SQ, U.K.


IntroductionIntroduction

• How ‘Physical’ is Information?• Does Information obey Physical Laws?• Can Information travel faster than speed of light?• Does Information require energy to process?• Is Reversible Computation possible?• What about Information and Entropy?• What about Quantum Information? (Ask me questions at end, if there’s time!)

• Mathematics of Information• Fourier Transforms• Complex Function Theory (Cauchy-Riemann)• Maxwell’s Equations

• Miscellaneous Physics (and Meta-Physics)• Diffraction & Dispersion• Noise, e.g. Amplified Spontaneous Emission (ASE)• Causality• Holograms

Any meaningful discussion of Information immediately opens a large Pandora’s Box of questions, and requires a startlingly wide knowledge of physics!


Historical Background to InformationHistorical Background to InformationlogS k W• Entropy Boltzmann (1872)

Boltzmann

logI • Information content Shannon (1948)

Shannon

• Information=negentropy Brillouin (1956)I S

Brillouin

1bit ln 2k• ‘Information is Physical’ Landauer (1962)

Landauer

• ‘Everything is Information’ Wheeler (1998)

Wheeler

© Copyright Fujitsu Telecommunications Europe Ltd 1996-2000. All rights reserved Wang et al., Nature, vol.406, p.277, 20th Jul’00

‘‘Fast’ Information Transfer ?Fast’ Information Transfer ?

Nature, vol.424, p.638, 7th August 2003


Information obeys Physical LawsInformation obeys Physical Laws

• Information/signals cannot propagate faster than c Einstein (1905)

• Wavefront travels at c through any medium Sommerfeld/Brillouin (1914) The signal is always slower than c

BrillouinSommerfeld


• Erasure of information requires energy, e.g. setting a computer register to zeroLandauer’s PrincipleLandauer’s Principle

1 1 1 10 0 0 01 10 1

0 0 0 00 0 0 00 00 0

E Input Energy Required

•Creation of information doesn’t require extra energy input, e.g. equivalent to diffusion, Brownian motion, is not dissipative, i.e. doesn’t ‘yield’ energy.•Implies ‘processing of information’ or calculation requires no intrinsic energy.

Information I = High

Information I = Low

-veI

1

0

1 1 1 10 0 0 01 10 10 0 0 00 0 0 00 00 0E1

0


Reversible & Irreversible ComputingReversible & Irreversible Computing• Erasure or Loss of information requires energy

• Conventional Logic, e.g. AND, XOR gates• tends to ‘lose’ information• dissipates energy• irreversible

AB

A B0

11

110

00

000

1

C

CANDAB

A B0

11

110

00

110

0

C

CXOR A

A01 0

1C

CNOT

i/p 2 bitso/p 1 bit

i/p 2 bitso/p 1 bit

i/p 1 bito/p 1 bit

IRREVERSIBLE IRREVERSIBLE REVERSIBLE


Reversible Computing - Toffoli GateReversible Computing - Toffoli Gate• Reversible Logic requires equal number of i/p and o/p gates• No information may be lost

ABC

TOFFOLI

A’B’C’

A’=AB’=BC’=C, unless A=B=1=> C’=C

A B0

11

110

00

000

0

C

000

1

C’ A B0

11

110

00

111

1

C

111

0

C’

AB

A B0

11

110

00

000

1

C

C’AND

i/p 3 bitso/p 3 bits

REVERSIBLE

C=0

BA

C01 0

1C’

A=1B=1 C’=CNOT

C

BA

i/p 3 bitso/p 3 bits

REVERSIBLE


Information is also defined by:• Discontinuities & Points of Non-Analyticity

• It is also distributed/contextual and localised, so has Holographic characteristics

Definition & Properties of InformationDefinition & Properties of InformationDiscrete (Shannon) Information lni i

i

I

i p x x

ln lnI p x x p x x p x p x dx dx

lndiffI p x p x dx

However the divergence is ‘constant’ for all space (x), but when considering transfer of information from A to B, we are interested in differences in information at A and B. Hence the differential (a.c.) information is given by:

ln lnI p x p x dx p x dx dx a.c. d.c.

Diverges to infinity as0dx

Differential (Continuous) Information


Analytic/Holographic FunctionsAnalytic/Holographic Functionsz x iy f z u z iv z

u v

x y

, , ,f x y u x y iv x y

u v

y x

A function is Analytic if it obeys

the Cauchy-Riemann equations

f z =exp i x+iy =exp ix-y

f z = c osx+isinx exp -y

u x,y =exp -y c osx v x,y =exp -y sinx

du=-exp -y sinx

dx dv

=-exp -y sinxdy

du=-exp -y c osx

dy dv

- =-exp -y c osxdx

f z =exp ize.g.


Properties of Analytic/Holographic FunctionsProperties of Analytic/Holographic Functions

• Complementary Characteristics • Wave-Particle Duality

• Nature of Holograms• Complete image is represented over entire space• Shattered ‘particles’ each carry a representation of the overall image

Denis GaborWave-likeDelocalised

Particle-likeLocalised


f z

z

0z

0

0

1

2

f zf z dz

i z z

1z

1

1

1

2

f zf z dz

i z z

Holographic Functions: Analytic ContinuationHolographic Functions: Analytic Continuation

• Holographic/Analytic functions have a Wave-Particle Duality

• It is a wave function • fully delocalised

• Also completely defined from any single point• all ‘information’ is contained at any single point, anywhere in space.• fully localised

Q: What is f (z1) ?Q: What is f (z0) ?

I know what f (z) is - I’m standing there!

2

2

1

2

f zf z dz

i z z

2z

3

3

1

2

f zf z dz

i z z

3z

0

1

2

f zdz

i z z


Summary of Analytic-Holographic FunctionsSummary of Analytic-Holographic Functions

z x iy f z u z iv z

u v

x y

, , ,f x y u x y iv x y

u v

y x

A function is Analytic if it obeys the Cauchy-Riemann equations

0

0

1

2

v zu z dz

z z

00

1

2

u zv z dz

z z

Integral Form of

Cauchy-Riemann Equations

0

0

1

2

v xu x P dx

x x

00

1

2

u xv x P dx

x x

Dispersion Relations (Causality) e.g. refractive index

Analytic Continuation 0

0

1

2

f zf z dz

i z z

Has ‘Holographic’ Properties


Causality & Fourier TransformsCausality & Fourier TransformsFourier Transform of a causal function (i.e. bounded in time) is an analytic function[1]

[1] J.S. Toll, “Causality and the dispersion relation: logical foundations”, Phys Rev, 104(6), p.1760-70, 1956

0f t t

0t

0t t

i tF f t e dt

i

, ,F u iv Complex frequency plane

real frequency axisimaginary frequency axis

2 f

[s-1][is-1]

f t

u v

v u

obeys the Cauchy-Riemann equations:

i

,F i F


FT of a bounded function (e.g. bounded in space) is also an analytic function:

2 X

i xRi

G X g x e dxR

Far-field Fresnel-Kirchhoff Diffraction Integral X X iY

‘Complex’ space plane

XiY

[m][im]

real space axisimaginary space axis , ,G X p X Y iq X Y

p q

X Y

p q

Y X

XiY

,G X iYobeys the Cauchy-Riemann equations: G X

Fourier Transform of a causal function (i.e. bounded in time) is an analytic function

R

g(x)x X

G(X)

Far-field (Fraunhofer) diffraction yields an analytic diffraction pattern

Young’s Slits

Interference Patternis Holographic


Maxwell’s Equations are an example of the Cauchy-Riemann equationsMaxwell’s EquationsMaxwell’s Equations

D dDH J

dt

dBE

dt 0B

3-D

dA dE

dx dy dE dA

dx dy

, , ,F x t E x t iA x t

The Electric and Magnetic fields form an analytic function in space-timeThe Electric and Magnetic fields form an analytic function in space-timeAlternative explanation for Wave-Particle Duality for lightAlternative explanation for Wave-Particle Duality for light

1/c /Z

A HZy ct

z x ict x iy

Speed of Light Impedance of Medium

Ohm’s LawTime is the imaginary axis (c.f. Space-time continuum)

d dEH

dx dt

dE dH

dx dt

0J

D EB H

d EdH

dx dt

d HdE

dx dt

1-DAssume no currents & charges(e.g. dielectric medium)


Analytic ContinuationAnalytic Continuation

“Leading Edge” of pulse contains completeinformation about the rest of the pulse shape

f t

t0t

0

0

1

2

f zf z dz

i z z

0 0 0z x ict 0t

Analytically-continued “superluminal” pulse

Q: Is information “transferred” when analytic continuation takes place?

t

z x ict

“Superluminal” pulse


0f z dz

Holomorphic (Holographic) Functions

Closed contour integral is zeroi.e. sum of Residues is zero

Analytic function which is entire across the z-planei.e. does not have any points of non-analyticity

xiy

2f z dz i R

Closed contour integral is equal to the sum of the Residues

Meromorphic FunctionsAnalytic function with discrete points of

non-analyticity, e.g. a pole in the z-plane

xiy

Contour IntegrationContour Integrationz x iy f z u z iv z

x yu v y xu v Cauchy-Riemann Equations

Analytic function which is entire across the z-planei.e. does not have any points of non-analyticity


2

ln

1

f xdx

x

Square integrability

All roots of the denominator polynomial must be in the upper half-plane e.g. Trig functions do not satisfy this - hence cannot be physical information-bearing signals

Hurwitz Polynomials

Paley-Wiener Theorem (also from Causality)

Some Criteria for Physical FunctionsSome Criteria for Physical Functions

Hence, function must be bounded both in space and in time, and tend to zero at infinity. e.g. Gaussian function does not obey Paley-Wiener, and it is not causal (bounded).

These theorems are well-known from filter theory.


How much information is in a Holographic Function?How much information is in a Holographic Function?

2logdiffI p x p x dx 2p x x 1p x dx

x r x is x

* *2logdiffI x x x x dx

Analytic solution to a wave equationr & s are purely real functions

2log 2diffI p x p x dx i R

2logdiffI G z U z G z U z dz

2 2log log 2diffI p x p x dx GU GU dz i R

, 0G U z 0 x

iy

z

2logp x x

2logG z U z G z U z

z x iy plane

, ,A x r x iy is x iy g x y ih x y G z

* , ,A x r x iy is x iy u x y iv x y U z

A

is the Analytic Continuation Operator, such that x x iy z x iy

Complex Conjugate (does not commute)

g & h , u & v are purely real functions, respectively obey C-R equations


2log 2diffI p x p x dx i R

Information = Sum of ResiduesInformation = Sum of Residues

Hence, a holomorphic function (entire across the z-plane) has no points of non-analyticityand so contains no differential information. It cannot be used for information transfer.

Likewise, a function which allows analytic continuation from x to x0 contains no informationbetween those points, so that no information is transferred between x and x0.

f t

0t t

“Superluminal” pulse

The “superluminal” pulse allowing analytic continuation from t to t0

transfers no information between these points, since R=0,so that zero information transfer takes place, let alone “superluminal”

information transfer.

Theoretically, “zero” information can be superluminally transferred!! (But that’s not saying very much!)

Information is contained in the points of non-analyticity, and discontinuities etc.


Information & EntropyInformation & EntropyInformation is contained in points of non-analyticity, points of discontinuity etc.Hence, from Sommerfeld/Brillouin information cannot travel faster than c through a medium

Information is inimical to adiabaticity, since “slow moving” and “smooth” conditions do not apply to points of non-analyticity (poles) or discontinuities.

Entropy must increase when information is transferred.

Due to the effects of diffraction (space), or dispersion (time) all information signals will tend to reduce in magnitude, or be absorbed when travelling through space. This leads to information loss (i.e. entropy increase, c.f. Brillouin) or reduction in SNR (also equivalent to entropy increase.)

Passive Media

This is in agreement with the impossibility of noiseless amplification:• Noise must always increase after amplification (3dB minimum optical Noise Figure)• “No-cloning Theorem” also states that perfect (noiseless) duplication of quantum states is impossible.

Active Media

Either way, information transfer is accompanied by an increase in entropy


Dynamic Formulation of Landauer’s PrincipleDynamic Formulation of Landauer’s Principle

• Erasure of information requires energy

Static Case (Conventional form of Landauer’s Principle)

• Entropy increases with erasure of information (Brillouin: S = -I )

Dynamic Case

• Transfer of information is associated with an increase in entropy

• Transfer of information must require energy

• Transfer of information from A to B is equivalent to:• Erasure of Information at A (Landauer’s Principle states this requires energy)• Re-creation of Information at B (Landauer’s Principle doesn’t require energy

for this)


Reversible Computation?Reversible Computation?

• If ‘shuffling’ information is over a finite space, information is being moved about.

Computation tends to ‘shuffle’ information around:

• Energy must therefore be dissipated, and entropy increase.

Bottom Line: Reversible computation is only true for an infinitely-small computer

• A physically-realisable computer must have a finite size.

• A smaller computer will be more energy efficient.


ConclusionsConclusions• Perfectly holographic analytic functions

• contain infinitely redundant information• zero differential information• can’t be used to transfer information

• Information is associated with points of non-analyticity and discontinuity • Inimical to adiabaticity• Transferring information from A to B requires a change in entropy

• This is in accordance with:• dispersion & diffraction (signals tend to degrade when moved in space)• impossibility of noiseless amplification (3dB minumum optical noise figure)• quantum no-cloning theorem

• Information cannot propagate faster than the speed of light in vacuum c• superluminal information transfer is impossible

• Physical computers are finite in size• Information is moved/shuffled around during a computation• Computation must dissipate energy, and cannot be reversible.


Date post:	02-Jan-2016
Category:	Documents
Upload:	cirocco-keevan
View:	44 times
Download:	2 times

Information, Entropy and Reversible Computation

Documents