JOURNAL OF THE OPTICAL SOCIETY OF AMERICA

Information in a Photon Beam vs Modulation-Level Spacing

ELLEN HISDAL

-Istlit iate of Physics, University of Oslo, Blindern, Oslo 3, Norway(Received 9 June 1970)

The amount of information transmitted via a photon beam that is modulated in s equal, discrete stepsfrom a given maximum expected number of pulses kmax and down to a given kinin is computed. The numbersof pulses refer to the counts of a photoemissive detector. The information i reaches a maximum in for acertain number s= sm for which the difference Sk of the expected number of detected pulses from successivelevels is about 2.7 times the rms fluctuation of the k_. level. The reliability (ratio of the information ob-tained in the presence of noise to the information that would be obtained without noise) of this maximuminformation i, is about 0.8. im is very nearly a function of the single parameter c= ( -kmin)/kji. Theseresults hold, provided that the rms fluctuation of the k., level is not too different from that of the kninlevel. For k, in= 0, maximum information is obtained with a somewhat smaller spacing 6k, and its reliabilityis about 0.61 for kznax>20.INDEX HEADING: Information theory.

The information i that can be carried by a photon beamis computed as the difference between two terms,

t=to-3,

where io is the information that the beam would havecarried if there had been no noise or fluctuations in thereceived signal. In Shannon's terminology, it is theentropy of the source. We wish to reserve the wordentropy for the thermodynamic entropy and havetherefore called io the a priori information.1 It isdetermined by the number s of different input symbolsor discrete modulation levels. If these have equala priori probabilities, as we assume in this paper, then

io = logs; (1)

j is the information loss owing to noise or fluctuations.Shannon calls it the equivocation.

Jones2 and Gordon3 have used the value 2 for s,assuming on-off modulation of the beam.

Suppose that the receiver is a photoemissive detector,and let n=kin be the maximum number of expectedpulses recorded by the detector with the given beam.Another plausible value for s is n+1, corresponding to0, 1, 2, .. ., it expected pulses. This value has previouslybeen used by Gordon (Ref. 3, p. 1907 in the limitingcase of n>>1) and by Hisdal.1

Actually, however, we have a priori no limitation onthe possible number of transmitted symbols. In thispaper, we assume that s may be any integer greaterthan 1.

Jones and Gordon have investigated the dependenceof i on the a priori probabilities of the input symbols.In this paper, we investigate the dependence of i on s,the number of input symbols or modulation levels.We shall show that, for a given beam, the information iis maximum for a certain finite s=s5 m Although ioincreases idenfinitely with s, the information loss jalso increases with s, so that i reaches a, maximum fora finite s.

It is assumed that the photon beam can be modulatedfrom a maximum-expected number of pulses km and

down to a minimum-expected number of pulses kmin,in equal steps of size

5k = (kMax-kmin)/(s -1), (2)

the expected number of pulses for the different inputsymbols being kmii kmin+5k, lemin+23k, ... krax. Theseneed not be integers. The actual measured number ofpulses mn is a fluctuating integer with a Poisson distribu-tion whose mean is the expected number of pulses forthe given symbol. This holds for a thermal beam oflow degeneracy or for a single-mode-laser beam.

The formulas for the computation of j are given inRef. 1. The only difference in the present case is that kvaries in steps of 5k instead of steps of 1; (n)-Qt)+1in Eqs. (3) and (4) of Ref. 1, which is the number ofinput symbols, must be replaced by s.

I. RESULTS FOR ki = 0

The results in the case of k =0in=° are shown in Fig. 1where the information i is plotted as a function of thenumber s of modulation levels for different values ofthe maximum-expected number of pulses is = kmax ofthe beam. For n< 5, maximum information im isobtained with sm=2, i.e., with two modulation levelsof 0 and n expected pulses, respectively. For greater i,i reaches a maximum for a given Sam that is more than 2and decreases again for greater s. For moderatelygreat and great is this decrease is, however, very slow.

In Fig. 2, we have plotted the maximum informationim (curve marked i), and other quantities that areobtained with the level number s=s,, that givesmaximum information, as functions of it. The curve(5k) rn/n4 shows that maximum information is obtainedwith a spacing 5k of the expected number of pulses,which is of the order of magnitude of no, i.e., of the rmsfluctuation of the number of pulses.

We call the quantity R=i/io the reliability.4 Itrepresents the actual average information obtained ina measurement, relative to the information that wewould have obtained in the absence of noise. The

328

VOLUME 61, NUMBER 3 MARCHI 1971

INFORMATION IN A PHOTON BEAM

reliability can assume values from 0 to 1. A reliabilityof 1 means that the input symbol can be identified withcomplete certainty from the output symbol. No codingis necessary to utilize the total amount of mutualinformation i. A reliability of 0 means completeuncertainty as to which input symbol corresponds to agiven output symbol. The reliability, or rather theunreliability 1 -R, is a measure of the complexity ofcoding (of the error-correcting type) which is necessaryin order to transmit the information with a negligibleerror percentage. When we perform a measurement of aphysical quantity (without transmitting information),the reliability is a measure of the uncertainty intro-duced by the noise.

For n> 20, the reliability RK of the maximuminformation i, is nearly independent of the maximum-expected number of pulses of the beam, being equal toabout 0.61 (curve marked R in Fig. 2).

We have previously 5 derived an approximate analy-tical formula for the information i in the case of single-pulse spacing, 3k =1, and large n. The information,according to this formula, is

i=log(n+1)-4 log(2 7rn). (3)

For large a, this formula gives a tolerable approximationnot only to i for single-pulse spacing, but also to themaximum information im. For at 100, at =200, itgives values that are 11% and 6% less than iimp respec-tively. The approximation improves with increasing n.

We conclude that for a given a = kiax, there is adefinite level number sm with which maximum informa-tion im is obtained. A somewhat greater level numberwill not do much harm as regards i, but it decreasesthe reliability.

II. RESULTS FOR k j,, 1 ;0

The case of kmi.0 can be applied to several differentphysical situations. In the method of modulation

Ci (bit) n =20026 ~ A

2.4-n =100

2.2 /

2.0-1.6 t gn =501.6

1.4 n =20

1.2

1.0 =1

0.6.

0.6£ =

0.4 n =20.2 n

0'2 5 10 15 20 s

20

18

l 16

14

12

- 10

6

4

2

1 2 3 4 5 7 10 20 30 40 50 70 100 200 n

FIG. 2. i, maximum information im in bits as a function of themaximum expected number of pulses n kmnx. Computed pointsare shown by circles. s (squares), level number sm that givesmaximum information. a (points), (ik)m/kinA, (Bk)m being thelevel spacing for s =s,,,. R (triangles), reliability for s=Sm.

described in Ref. 1, the beam is reflected from s sampleswith equally spaced reflectance values from 0 to 1.If the thermal radiant energy emitted from the samplescannot be neglected, we have the case of sample noise.6

krin) org in the notation of Refs. 1 and 6, is the expectednumber of pulses from a black sample (reflectancer=0); kmax is the expected number of pulses from awhite sample (r= 1). All samples are assumed to havethe same, known, temperature.

Another type of noise that gives rise to kmsn#0 ispath noise.6 This is due to thermal radiant energyemitted somewhere along the path of the modulatedbeam, e.g., by the atmosphere. Let X be the numberof pulses owing to the path-noise beam alone, and a

the number of pulses without path noise, assuming thatthe emitting materials have absolute temperature 0.To obtain the information formulas in the case of pathnoise, we must replace kinin by X and kma..-kmin by a.t64

The parameter c, defined by Eq. (4) below, is thereforeequal to nt/nl. We may call it the signal-to-noise ratioin the case of path noise, if we understand by noise therms fluctuation of the path noise.

Also, without sample or path noise, the case ofkmin#0 is of interest for the determination of thediscrimination ability between two pulse-number levelsor between two reflectance values.4

We make use of the two parameters

6= (kmac-kmin)/kmin (4)

y= (kmax-kmin)/kmin (5)

We found previously' that for single-pulse spacing,i depends mainly on c and only slightly on y. It turnsout that the same is true for i when s equals 2, especiallyfor higher c. The results for i (bit) =i/io=R in the caseof s=2 are shown in Fig. 3. The upper curve is fory=0.05, the lower for y=0.3. As kmin=j(c/y)2 , thenumbers of pulses for the two curves differ by a factorof 36 for the same c, while the information i varies bya few percent only, especially for higher c.

FIG. 1. Information i as a function of the number of levels s fordifferent maximum expected numbers of pulses n=kmi. Arrowsare placed at those integer s values for which i is a maximum.

March 19711 329

33 ELI-EN HISDAL

So

0.9

0.8 F

0.7

0.5

0.4

0.3

0.2

0.1

0

. i (bit)=R

L

0

FIG. 3. Informain the case of 2curve (triangles) f

Figure 3 alsowe must requirlevels with athis case is equ;c rwfrllrt

gives maximum information corresponds to a 3k thatis equal to about 2.7k.,,j. The data show that this resultholds for all c>3 and y<0.5.

Because wkŽ2.7kmoP, we have in the above range ofc and y an approximate formula for Sm

i/ = 1 + (kmax kmni)/2.7kmax4, (6)

and for the maximum information

/ -tm=Rm logsm. (7)

The curve marked R on Fig. 4 shows the value of/ Rmzsm/iom, tm being the a priori information withthe sample number sm' We see that also the reliabilityof the maximum information is approximately constantwith respect to c (as well as with respect to y) and equalto about 0.8.

The computed values do not always lie on smoothcurves. The reason for this is that the quantity s cannotbe varied continuously, because its physical meaningis the number of modulation levels; s must therefore

| | | ______ | __ be restricted to integer values equal to or greater than 2.2 3 4 5 6 7 8 C With only one modulation level, we get no information.

tion i or reliability 1? as a function of c [Eq. (4)] .nt, (8k)m/kmmj, and RP would show a more regularlevels1 s=2. Upper curve (circles) and lower variation if we could have determined i as the maxi-'or y=0.05 and y=0.3: respectively [Eq. (5)]. mum value of i with respect to the real variable s.

The corresponding value som of s would then, in general,

-shoas how great a difference kmR-krnin noLt be an integer. Wlle have drawn the smooth curves inre if we wish to discriminate between two Figs. 2 and 4 through those computed points whoseriven ave rage reliability i/i0i which in integer Sm value is nearest to the s.r value that would be11 totheinformationinbits.Thismethod obtained with a continuously varying s. The curves+bth nnP +haf iiCaR cinnhi +i]n11- cn.kr nn f therefore give the main tendency of the variation.

determine the discrimination step' because the questionwe pose is about the ability to distinguish between 2levels, not between km,,-,-kiin+1 levels. As an example,we see irom Fig. 3 that we must have cL3.4 if we re-quire an average reliability of 0.8. If we require anaverage reliability of 0.7, we must have ccs!2.9, theexact value of c depending on y.

For s=3 the constancy of i and i with respect toy is somewhat less marked; it deteriorates graduallyfor higher s.

Also, in the present case of kimiO, we find that, for agiven kin£B and kiniu or c and y, there is an s=s, withwhich i becomes a maximum in. This s is equal to 2 forc<4; it increases gradually for higher c. Again sm isindependent of y, and im nearly so.

The results for sm. im Rm, and )mlkm,2 are shownin Fig. 4. The values of sm are independent of y. Theother quantities are shown for y=0.2 . For y=0.3 theyare less by 1%/-4%; for y=0.l they are correspondinglygreater.

The quantity (6k) m/kx.os is approximately constant,not only with respect to y, but also with respect to c.This can be seen from the curve marked 8 in Fig. 4.From this curve, we find that the level spacing which

,i,R

22

1a

I'S

1.4

1,2

to

Oh

0.6

X 3 6 9 12 15 18 21 C

FIG. 4. Same quantities as in Fig. 2 in the case of kinb#0,as a function of c [Eq. (4)].

330 Vol. 61

a's5

INFORMATION IN A PHOTON BEAM

III. COMPARISON WITH RESULTSOF OTHER AUTHORS

Jones2 computes the information transmitted by aphoton beam with on-off signal modulation. Thiscorresponds to our case of two levels with expectedpulse numbers 0 and 1, respectively. He assumes onlytwo possible received symbols, no photons and one ormore photons, whereas we differentiate between allreceived numbers of pulses. We should therefore expectthat our values for the information i are slightly higherthan those of Jones. A comparison of Jones's Table I andour Fig. 2 shows that this is indeed the case. By inter-polation, we get from the C column of Jones's table aninformation of 0.416 bits for 1 expected incidentphoton (M= 1) whereas our value (Fig. 2) is 0.426 bits.Jones also treats the case of different a priori probabil-ities of the on and off signals.

Gordon (Ref. 3, p. 1904) computes the informationcapacity when a quantum counter is used as a detectorfor two limiting cases. The first case is for small numbersof photons. His treatment of this case is, in principle,the same as that of Jones. In the second case, he assumesthat the average number of received photons is verylarge. The possible transmitted symbols correspond to0,1,2... co expected received photons, and the possiblereceived symbols are 0,1,2,. .. .o photons. In both cases,he maximizes the information for a given average re-ceived power by varying the probabilities of the ex-pected received numbers of photons.

Gordon's final formula, in the case of very largenumbers of photons (the one following Eq. 12 of Ref. 3),is i=4 log(QA/27r)+0.304 bits, mh being the averagenumber of received photons. This formula is found forsingle-pulse spacing. For the same spacing, we havefound the approximate formula i=4 log(n/2 7r).1 Theexact values are 470 greater than this for n=2001;the relative difference decreases for greater v. Our nis the maximum expected number of pulses. Because weassume constant sample spacing, we get n = 2&.Substituting this into our approximate formula, weget i= 2 log(if/27r)+0.5 bits, the exact value beingslightly more. Thus, our result for i is slightly greaterthan Gordon's, although we do not vary the probabilityof the expected numbers of received photons, as Gordondoes, to maximize i. [In Gordon's terminology, weassign equal probabilities to all expected numbers ofphotons from 0 to 2?h and zero probability to all highernumbers of photons. Gordon finds that the informationis maximized for a distribution of the form (1/rn)Xexp(-ff/m) for the a priori probabilities.] When weuse the optimum level number Sm instead of single-pulsespacing, we get an even greater value for i accordingto Sec. I.

A reader of this paper has drawn the attention of theauthor to the interesting paper of Watanabe,' who alsoinvestigates the dependence of the information on thenumber of levels. Watanabe finds that the information

increases gradually with the number of levels until itreaches the value for infinitely small spacing. He doesnot find a maximum for a finite spacing. The differencebetween Watanabe's and our result may be due to oneor several of the following reasons.

(1) Watanabe assumes additive noise. We havenoise that depends on the signal, its rms deviationbeing equal to the square root of the expected numberof pulses. (2) Watanabe assumes a continuous distribu-tion of the measured quantity. We assume a discrete,Poisson distribution. (3) Watanabe's expressions arevalid only for a spacing that is small compared to therms deviatibn. (4) Watanabe approximates the informa-tion by an analytical formula. Our values are exact.Our own guess is that the difference between Watanabe'sand our result is due to reasons (3) and (4), but futurecomputations (valid also for large spacing) will haveto show whether the information reaches a maximumalso for the type of signal and noise treated by Watanabe.Qualitative considerations lead us to expect a maximumalso in this case.4

Shannon's well-known formula,8 i2= log(1+S/N)(S stands for the ms signal and N for the ms noise)for the maximum-attainable information cannot beapplied to our case. In addition to reasons (1) and (2)mentioned in connection with Watanabe's work, wehave the following differences between Shannon's andour treatment. Shannon's formula is valid for a signalwith a gaussian probability density with mean 0 andvariance S. Our signal is always positive with a constantprobability distribution inside its allowed range ofvariation kmin, kmax. Indeed it would be hard to saywhat we should substitute for the symbol S in Shannon'sformula, in our case. Shannon's formula is valid forinfinitely small spacing; he does not investigate thedependence of the information on the spacing.

Our Eq. (7) for the information in the case of kmin.5Ohas, however, some similarity with Shannon's formula.In the case of krnin=O, kmx,>>l, we have from Eq. (3),

i = logka,/ (27rkmax) t (8)

where ka,, kmaxi may be said to stand roughly forsignal and noise, respectively. As we saw in Sec. I,Eq. (8) gives a value that is less than the maximumattainable information, but the relative differencedecreases with increasing kmi.

IV. REMARKS

One way to modulate the beam is to let it be reflectedfrom s samples with equally spaced reflectance values.The results of this paper are applicable not only to theproblem of communication via a photon beam but alsoto the problem of the information obtained in a measure-ment,4 in this case a reflectance measurement. Wemention here only that the above result about thereflectance spacing that gives maximum information

331March 1971

£11W E N JI ISDAL

shows that the accepted practice of assuming an error ofobservation of the order of magnitude of the rmnsfluctuation of the measurement does indeed yield anamount of information near to the maximum obtainable,with an average reliability of about 80%.4 The relationbetween thermodynamic entropy and information isalso discussed in Ref. 4.

REFERENCES'E. Hisdal, J. Opt. Soc. Am. 59, 921 (1969).

2 C. Jones, J. Opt. Soc. Am. 52, 493 (1962).3 P. Gordon, Proc. IRE 50, 1898 (1962).4 E. Hisdal, Phys. Norvegica 4, No. 4 (1971).6 E. Hisdal, J. Opt. Soc. Am. 55, 1446 (1965).6 E. Hisdal, J. Opt. Soc. Am. 58, 977 (1968).7 S. Watanabe, IRE Trans. IT-3, 214 (1957).8 R. M. Fano, Transmnission of Information (MIT Press,

Cambridge, Mass., 1961), Ch. 5, p. 146.

vol. 61