JOURNAL OF THE OPTICAL SOCIETY OF AMERICA

Information in the time of arrival of a photon packet:capacity of PPM channels*

Israel Bar-DavidtFaculty of Electrical Engineering, Teclhnion-Israel Institute of Technology, Haifa, Israel

(Received 6 March 1972)

Expressions for the information rate and capacity of amplitude-modulated photon beams are available inliterature. Recent interest in position-modulated laser pulses has motivated the investigation of the follow-ing model: A random variable 1D, which can take on values in the interval (- T, T), is transmitted by center-ing a narrow, coherent, single-mode light pulse of duration 2D and constant irradiance at I = D. It is assumedthat no other than quantum fluctuations, of the otherwise stable source, disturb the transmission. The in-formation that the received photon packet carries about 1) is registered as the instants ItkI of emission ofphotoelectrons at the detector. If H denotes the entropy of D and Q, the expected number of photoelectronsis 2D, and I)<<T, then the mutual information between the D- and ltk l ensembles is, for large Q, approxi-mately I - In (21)/pQ). Here, p = exp (y -1) and y is Euler's constant. Under the peak-excursion constraintI DI T, this is maximized by H = ln2T, for uniformly distributed 0D, so that for large Q the capacityC = In (pQT/I)). The accurate expression, valid for all Q, involves the exponential integral El (Q). The valueof C is used to derive a lower bound on the mean-square error of any estimator of X, by the rate-distortionmethod. The bound, which decreases as Q2, is compared with the variance of maximum-a-posteriori-proba-bility estimators of delay which, in the case of differentiable pulses, decreases only as Q.Index Headings: Information theory; Laser; Detection.

The information rates attainable in discrete as well ascontinuous intensity-modulated optical communica-tion channels have been extensively studied.1' 2 Pulse-position modulation (PPM) systems have receivedattention mainly in their discrete version,3 i.e., withthe allowable locations of pulses restricted to a finiteset. The continuous-PPM model has been investigatedfor coherent electromagnetic pulses having a differenti-able envelope,' and the variance of the delay-measure-ment error for a maximum-likelihood estimator hasbeen approximately calculated. An approximate boundon the capacity of such channels has been derived 5 byuse of this calculated variance as the entropy power6 ofthe measurement error.

In this paper, the information content in the time ofarrival of a rectangular pulse of coherent radiation,detected by a photon-sensitive device, is derived fromfirst principles. From it, the capacity of a coherentoptical PPM channel, subject to peak-delay-deviationconstraint, is calculated. The analysis assumes thatthe only disturbance is due to the quantum fluctuationsin the electromagnetic radiation.

I. FORMULATION OF THE COMMUNICATIONPROBLEM

A message to be transmitted is represented by therandom variable D that can assume a continuum ofvalues in the interval 0 = (- T, T) with prior-probabilitydensity p)(:D). A pulse of radiation of duration 2D iscentered at D- iDo, where Do is the assumedly fixeddelay between the transmitting and receiving planes.Whereas a complete description of the received pulseis always given by the electromagnetic field, by refer-ring here to the pulse as a packet of photons, we under-stand that the only observables are the photons or,more accurately, the photoelectrons emitted by the

intercepting photosensitive surface. Let K be thenumber of emitted photoelectrons and let (tk), k=1,2, . . ., K, denote the set of instants of their emissions.Since Ik can assume values in the T=(T-D, T+D)-interval, (t4) is a point in the K-fold product space fTK.

The question under investigation is now: What is the in-formation available in the set of observables (tk) aboutthe message, here represented by SD?

In order to be able to use Shannon's quantitativeapproach6 and calculate the average mutual infor-mation I[{EK); 0], defined by

I[{[fK); 0]=E[lnp({tck) I D)/P((tk))]], (1)

where {TK) denotes the ensemble of TK spaces, as Kgoes from 0 to oo, and E denotes the expectationof a random variable, knowledge of the conditionalK-dimensional joint-probability density functionP(Itki)1), of tlk) for given 0D, is required. The un-conditional density p({tk}) is then given, for anyp5)(), by

(p{tk) = E[p((tkc) I i)] (2)

the expectation being taken with respect to 0, underthe given probability law p)(D). In Eq. (1) the ex-pected value is taken with respect to tik) and a), undercalculable P((tk),D)=P((tk) 1 JD)p5(D). The mathe-matically simplest and physically still meaningfulassumption is that the source of radiation is coherent,as for example that of a pulsed stable laser. For then 7' 8

the fluctuations of its intensity, as well as of the photo-electron count, obey Poisson statistics; for such a caseit has been shown4 that

K

PENtk I D]=exp(_Q.) 11N S(t), K > 1,k=1

-T<tl<t 2 < *.. <tK<T

=exp(-Q), K=0, (3)

166

VOLUME 63, NUMBER 2 FEBRUARY 1973

ARRIVAL TIMES OF PHOTON PACKET

where X(t) is the time-varying parameter of thePoisson process, in this case describing the expectednumber of observed photoelectrons per unit time forgiven delay O, and where Q= f y-A (t)dt is the expectedvalue of the total number of counts, i.e., that of K.In the case under consideration, if the transmittedpulse, centered at 1)-00 and of duration 2D<<2T, isof constant power over its duration and its energy isE, then the density of the expected number of observedphotoelectrons X5(t) conditioned on OD is given by

X5Q(t)=Q/2D=A, O-D<t<D+D= 0, otherwise (4)

where Q=a-aE, a is the attenuation in the propagationpath, and 77 is the quantum efficiency of the detector.Then Eq. (3) becomes

p ({ tk I O) = e-QA K,

= 0, otherwiseand

1TP({tk}))= JP({Ik} I D)P (5))d5D

-T

=exp(-Q),

(5)

K=O

fTH= e-QA K |pa(5)da, K>1,

J L

11< 12< ... < tK, tK-tl< 2D

p({tk))>O and if P(K) denotes the probability of Kcounts within X, we find, by integrating Eq. (5),P(0) = exp(-Q) and, for K> 1,

rT rt1+2D

P(K) =eQAKj dtip5(1i) D dt2 .-T I t

pt1+2D

Xy (2D+tl-IK)dtKlK-1

rT

= e-QA KJ dtps(t 1 ) (2D) K(K !)-'-T

=eXp(-Q)QK (K!)-', (8)

which indeed corresponds to a Poisson distribution forthe total number of counts and yields o0- P(K) = 1.

In the following calculations, the end effects willbe neglected, i.e., t1 +D and IK-D will replace TH andTL, respectively, wherever appropriate.

II. THE AVERAGE MUTUAL INFORMATION

A well-known decomposition of the mutual infor-mation Eq. (1) is into the difference between the un-conditional and conditional entropies, H(( YK} ) andH((TIK} I 0), respectively. We have

I({ 'K); O) = H({TK}) -H({ rK} I 0) (9)

=0, tK-11>2D (6)

where TH=minEtl+D, T] and TL=max[tK-D, -T].By the mean-value theorem of the integral, sinceps)(D) is non-negative,

JTH

TL

pa(D)d D = (TH - TL)P5)(t')

Consider first the conditional entropy, using Eq. (4):

H({( K} 6 0)

= -E~lnp({tk} I D) I

=Q exp(-Q)-E [ exp(-Q)AKJ dt1K=1 J - D

dt. .D

dt2 . . .t Ir-

dtK(-Q+K lnA)],r D

Xyt1

for some t' in the interval (TL, TH). For relatively smootha priori densities pn(O), as compared to the narrowpulse width 2D, to a very good approximation we havepn(ti') -p)(t 1 ). Also for very large intervals T, ascompared to pulse width D, the end effects can beneglected and TH can be taken as t+D, and TL astK -D. Thus Eq. (6) can be rewritten without significanterror as

(p{tk})=exp(-Q), k=O

=e QAK(2D+t1-tK)p5(t1),

K> 1, tl< * -* <tK, tK-ti< 2D

where the expectation is with respect to O). Integrationgives a value independent of OD, so that after a fewmanipulations, lengthy but straightforward,

H({TK} I0)

=Q exp(-Q)

++F, [(Q-KlInA) exp(-Q)A K(2D)K(K!)-']

=Q-Q lnA,

=0, tK-tl>2D. (7)

It is of interest to observe that the basic property ofp({tk)) being a probability density has been maintainedover the approximation leading to Eq. (7). Indeed

(10)

where, as before, A =Q/2D, Next, using Eq. (7), in

H({rK}) = -E~lnp((tk))], (10a)

we write, as before, the term for K = 0 first, that for

February 1973 167

.o -s

ISRAEL BAR-DAVID

dt2. . .

K= 1 next, and then the infinite sum for K> 2:

H(( Y1K}) =Q exp(-Q)

rT

-/ dtlQ exp(-Q)pj(1,)[lnp5(tl) -Q+lnQ]-T

XA J d tj+2D-EF e-QA K |dtspi)(11)

K-2 J-rtj+2D

X -1 dtK(2D+t,-tK)[{ -Q+K InA +lnpD(ti))tK-I

+ln(2D+tl-tK)]. (11)

The first integral yields Q exp(-Q)[H(0)+Q-lnQ],where

H(0) = -E~lnp(fD)] (12)

is the entropy of the random variable 0D. Thus, thefirst two terms on the right-hand side of Eq. (11) yield

Q exp(-Q)[1+Q-lnQ+H(0)]. (13)

Turning now to the integrals in the infinite sum, weobserve that the term in the curly bracket is indepen-dent of t2, ... , 1K, so that its integration down to tIis similar to that in Eq. (8). As for the second term,logarithmic in tK, we obtain the recursion formula

which, being larger than the conditional entropy Eq.(10) by exactly the average mutual information Eq.(9), yields the required result

I({ QK}; 0) =[H(0) -ln2D](1 -eQ)

+e-Q E QK(K!)- 4 g(K). (18)

The first term is the difference between the entropiesof 0D and that of a uniform distribution over an intervalequal to the duration of the pulse, multiplied by theprobability of at least one count. If T -oo and 0D is ofgiven variance, a)2, it is well known6 that H is boundedby

H(O) < 2 ln27reao-5 (variance limited), (19)

equality holding if the distribution of D is gaussian. Thesecond term in Eq. (18) is identified as the expectedvalue of g(K) with respect to the Poisson distribution[recall that g(O) =0], here denoted by E[g(K)]. Thus,

I({ TYK}; 0) = (1 -eQ)[H(0) -ln2D]+E[g(K)]. (20)

A straightforward and rather tight bound on E[g(K)]is obtained by making use of the convexity of g andapplying Jensen's inequality. It is shown in the Ap-pendix that

(21)Iyk(lny-g(k))(k !)-dy

where

Combining Eqs. (19)-(21), we obtain the simple andrather tight upper bound on the information in thepacket (dropping the argument of I)

g(k) = _ n-', k > 2n=2

=0, k=0, 1.

By inspection of the sum vs the integral of 1/x, wecan bound g(k) by

ln(k+1)-1<g(k)<lnk, k>1. (15)

The infinite sum therefore equals

00 T

-E e-QA K dtjpj)(t1)(2D)K (K!)-lK-2 J-T

X[lnpj(1l)+ln(2DA K) -Q-g(K)]00

=eQ (QK/K!)(H(0)+Q+g(K)K-2

-ln(2D) -K lnA}. (16)

Combining Eqs. (13) and (16), we get

H({ rK)) =Q-Q lnA +[H(0) -ln2D]( -e-Q)

Ivariance limited< (1-e Q)X lna Q(1-ee Q)-l(se/D) (7re/2)ij. (22)

(14) EEg(K)] can also be expressed (see Appendix) as

E[g(K)] = lnpQ+E,(Q)+ exp(-Q), (23)

where EI(Q) is the exponential integral, tabulated inAbramowitz and Stegun," p=exp(-y-1)_0.656, andy=0.577 21 . . . is Euler's constant. Furthermore theinequalities" involving E1 (Q) yield immediate upperand lower bounds on EEg(K)],

InpQ+e-Q(! ln(1+2Q-')+1)< EIEg(K)]< InpQ+ eQ(ln(l+Q-')+ 1). (24)

These bounds are extremely tight for large Q. ForQ<<1 the following bounds are tighter,

Q2 /4 -Q 3 /6< E[g(K)] < Q2 /4+ Q3 /18. (25)

The various bounds and the accurate graph of E[g(K)]are plotted in Fig. 1. We conclude that

E[g(K)] 5lnpQ, Q> 4, p = exp(T-1)-2 (26).0

+e-Q 7 QK(K!)-lg(K),1

(17) is an excellent working approximation for Q>4 (errorbeing less than 2%). For Q<4, better approximations

168 Vol. 63

EEg(K)] < (1 - e-Q) 1nEQ(1 - e-Q)-l].

=Y'+'(Iny -g(k+ 1))E(k+

ARRIVAL TIMES OF PHOTON PACKET

.751

0

0

'a

w

1.50

L25

1.00

0.75

0.50

0.25

a1 2 3 4 5 6

0

FIG. 1. The expectation of g(K) (full line), which is the increaseof capacity of a PPM channel above the logarithm of the ratio ofintrapulse interval to pulse duration, as a function of expectednumber of received photoelectrons Q. Other curves are upper andlower bounds. Lower dotted curve is lnpQ, p=exp(7y-1),-Y=Euler's constant, and approximates well E[g(K)] for Q>4.Upper dotted curve is fairly good bound for all Q. Numbers corre-spond to the respective equations in text.

areE[g(K)]_-(1.1Q-0.4)--Q14, 2 <Q<4

=-Q114, 0<Q<l. (27)

For analytical investigations, the bound in Eq. (21)might be used as an approximation over the entireQ axis; its error, however, increases asymptoticallyfrom 0 to 1 -y-y0. 4 2 3.

III. PULSE-POSITION MODULATION

In a practical communication system, the excursion4- T of D must of necessity be limited, in order to allowrepeated uses of the channel. It is then known6 that

H(0)< ln2T,

with equality obtained for uniform distribution of D.The capacity of the PPM channel is therefore, fromEq. (20),

C=rmax1({(Y'K); 0)=(1-erQ) lnM+E[g(K)], (28)

and depends only on the number M=T/D of pulsebins of duration 2D that can be accommodated withoutoverlapping in the prior interval (-T, T), and on thepulse energy Q. Restricting discussion to Q>4, we useEq. (26) and neglect exp(-Q) with respect to 1, toobtain

C-lnpMQ, Q>4, p=exp(-y-1). (29)

We conclude that pulse energy can compensate fordeficiency of ability to sharpen pulses, much as in thegaussian-noise case.9 A similar conclusion, for Q<4,can be derived using Eqs. (21) or (27).

An instructive interpretation of Eq. (26) is possibleif it is rewritten as

C=F e-QQK(K!)'1[lnM+g(K)],

which, except for the missing term (K = 0) is the expec-tation of lnM+g(K). This can be thought of as thechannel capacity when K is -the total number of counts,and is exactly by g(K) larger than the capacity of anM-symbol noiseless channel. If K =1 then g(K) = 0, asindeed we would expect, because a single count couldnot possibly do more than point out a bin. For largerK, however, the counts can already help in centeringthe pulse; channel capacity increases, however, onlyroughly logarithmically with K, as evidenced by Eq.(15). It is also satisfactory that the summation startswith K =1; after all, no information on location canbe conveyed when no counts occur.

IV. RATE-DISTORTION BOUNDON VARIANCE

Equation (20) can be used to derive a lower bound onthe mean-square-error 1 achievable by any algorithmthat will compute an estimate 6({tk)) of Oc from ob-served (1k). If Rj/fi) denotes the rate of the source withrespect to an assumedly quadratic distortion measure

( -~D)2 with mean A, then Shannon's theory6

requiresRI)(A)<C<H(0)-ln2D+E[g(K)], (30)

where 1-exp(-Q) has been increased to 1, assumingthat in any measurement situation H(0)> ln2D.Furthermore for arbitrary source distribution'

R5)(3) 2 H(0) -2 ln2ireo3, (31)

equality holding for gaussian D. Combining Eqs. (30)and (31), we obtain

-2 ln22ref< -ln2D+E[g(K)],yielding

A> (2/2re)D2 exp(-2E[g(K)]),

which can be simplified by using Eq. (26),

AdŽ (2/re)p- 2D2 Q-2-0.57D 2Q-A

(32)

(33)

This lower bound, holding for packets having rec-tangular envelopes, decreases as the second power of Q.

The maximum-likelihood estimator for the differ-entiable pulses analyzed in Ref. 4 yields an error thatdecreases only as the first power of Q. It should bekept in mind, however, that the model in the maximum-likelihood estimation approach, used in Ref. 4, assumeddifferentiable pulse envelopes, whereas the presentanalysis is restricted to nondifferentiable rectangularpulses. This difference is essential in the parallelproblem of estimating the delay of pulses in whitegaussian noise.'0 The mean-square error actuallyachievable with rectangular pulses is the subject offurther investigation.

V. CONCLUSION

The simplest possible model of a position-modulatedpulsed photon communication system has been ana-

169February 1973

ISRAEL BAR-DAVID

lyzed. A direct tradeoff between pulse width and energyhas been demonstrated. The basic assumption that noother than quantum fluctuations of the otherwise-stable coherent-radiation source disturb the transmis-sion of a rectangular pulse has simplified the mathe-matics sufficiently for the probabilistic averages to becarried out. It is hoped that models that includeadditive noise as well as multimode radiation from thesource will lend themselves in the future to someapproximate analysis.

ACKNOWLEDGMENT

The author is indebted to the anonymous reviewerwho, by pointing out the sequence of identities, Eq.(A2), and Ref. 11, greatly contributed to the simplifi-cation of the final results.

APPENDIX

The infinite sum in Eq. (18), denoted by E[g(K)],can be rewritten as

obtain

1 XQK 1 re-Q x, - E 9 - I xle-xdx

r=1 r K=r K! r=o r! Jo.0 1-e6-

8 j dx=E(Q)+lnQ+,y, (A2)

where -y=0.57 72 1 . . . is Euler's constant andE,(Q) = fQ'j e-xdx/x is the exponential integral." Thebounds in Eq. (25) are obtained by integrating, termby term, the expansion

rQEin(Q)= (1 -e-)dx/x

rQ(1-x/2+x'/6-*--)dx.

ThenQ -Q 2/4< Ein(Q) < Q -Q'/4+Q 3/18

and, since

(-Q+Q2/2-Q'/3)<exp(-Q)-1< (-Q+Q'/2),the bounds follow directly.

(1 -e-Q)E'[g(K)] REFERENCES

-(1-e-Q) E e-Q(le-Q)-lQK(K!)'1g(K), (Al)K-1

where E'[ .* ] denotes the expectation with respect tothe Poisson distribution, conditioned on KOO, i.e.,on at least one photoelectron. Using Eq. (15) and apply-ing Jensen's inequality,

E'[g(K)]< E'[lnK]< ln(E'[K]) = lnQ(1 -e-Q)-l,

which, together with Eq. (At), leads to Eq. (21). Toobtain Eq. (23), we observe that E[g(K)] can also bewritten as

co QK K 1-Z7 (-E~g(K)j=,e-Q E -(,-)K=1K! r~1r

Rearranging the terms in the double summation, we

* Paper presented at the 1972 IEEE International Symposiumon Information Theory, Asilomar, California. Work supportedby NASA grant Nos. NGR 33-013-048 and NGR 33-013-063.

t Formerly with the Department of Electrical Engineering, CityCollege of the City University of New York, New York, wherethis work was completed.

I R. Jodoin and L. Mandel, J. Opt. Soc. Am. 61, 191 (1971),which includes additional references.

2 E. Hisdal, J. Opt. Soc. Am. 61, 328 (1971), which includesadditional references.

I For a state-of-the-art exposition, see S. Karp, E. L. O'Neill,and R. M. Gagliardi, Proc. IEEE 58, 1611 (1970).

4I. Bar-David, IEEE Trans. IT-15, 31 (1969).5I. Bar-David, Proc. IEEE 59, 1612 (1971).6 C. E. Shannon, Bell System Tech. J. 27, 643 (1948).7 L. Mandel and E. Wolf, Rev. Mod. Phys. 37, 231 (1965).8 L. Mandel, Phys. Rev. 138, B753 (1965).9 C. W. Helstrom, Statistical Theory of Signal Detection (Per-

gamon, London, 1960), Ch. 3, Eq. (2.13).10 See, for example, A. S. Terentiyev, Radio Engr. Electron.

Phys. 13, 569 (1968).l" Handbook of Matheinatical Functions, edited by M. Abramo-

witz and I. A. Stegun, Natl. Bur. Std. (U.S.) Appl. Math. Ser.(U. S. Government Printing Office, Washington, D. C., 1964;Dover, New York, 1965), Ch. 5.

170 Vol. 63