Modulation-transfer-function analysis for sampled image systems Stephen K. Park, Robert Schowengerdt, and Mary-Anne Kaczynski Sampling generally causes the response of a digital imaging system to be locally shift-variant and not directly amenable to MTF analysis. However, this paper demonstrates that a meaningful system response can be calculated by averaging over an ensemble of point-source system inputs to yield an MTF which accounts for the combined effects of image formation, sampling, and image reconstruction. As an illustration, the MTF of the Landsat MSS system is analyzed to reveal an average effective IFOV which is significantly larger than the commonly accepted value, particularly in the along-track direction where undersampling contributes markedly to an MTF reduction and resultant increase in image blur. 1. Introduction It is widely recognized that the modulation transfer function (MTF) of an imaging system is of fundamental importance in both the initial specification and design of the system and in any subsequent detailed analysis of the images it produces. Because the MTF is the modulus of the system optical transfer function (OTF), which is, in turn, the Fourier transform of the system point-spread function (PSF), the MTF of an imaging system can be determined, at least in principle, by generating the modulus of the Fourier transform of the image of a point source. Implicit in all of this is the mathematical assumption that the system is both linear and shift-invariant, i.e., that the location (and strength) of the point source can be chosen arbitrarily. If the shift-invariant (isoplanatic) assumption is not valid, the image of a point source will depend on its location, and the system MTF will have a spurious location-depen- dent phase dependence. Unfortunately, the MTF approach to system per- formance analysis is not directly applicable to many contemporary sampled imaging systems (e.g., elec- trooptical line-scan and sensor-array devices) because sampling causes these systems to have a particular kind of local shift variance. That is, whenever these systems form the (sampled) image of a point source, the ap- pearance of the (reconstructed) image will depend on Robert Schowengerdt is with University of Arizona, Office of Arid Lands Studies, Tucson, Arizona 185721; the other authors are with NASA Langley Research Center, Hampton, Virginia 23665. Received 22 December 1983. the location of the point source relative to the sampling (i.e., pixel) grid. This sample-scene phase phenomenon is familiar to anyone who has tried to analyze system performance by observing the appearance of small high-contrast objects in a digital image. For such sys- tems the basic question then is how can the MTF ap- proach to system performance analysis be rigorously modified to account for the particular kind of shift variance that sampling introduces? This paper pro- vides an answer to that question. In this paper, the combined effects of imaging, sam- pling, and reconstruction are analyzed in terms of the spatial frequency response of a general sampled imaging system. The (linear) system model used is 2-D and sufficiently general so that the results of this MTF analysis apply to most line-scan and sensor-array imaging systems, particularly those used for remote sensing. Sampling effects are accounted for by using a point-source system input randomly located relative to the sampling grid and by averagingthe system output over all possible point-source locations. This random sample-scene phase approach has been used success- fully by the authors recently in related studies of the radiometric error in remote-sensing imagery introduced by the combined effects of sampling and reconstruc- tion. 1 ,2 A similar sample-scene phase approach to system analysis was used recently by Wittenstein et al. 3 However, their approach is distinctly different from ours in that it is nonrandom and is restricted to a class of systems for which sample-scene phase dependence is negligible. Our results, as well as those of Ref. 3, are based on the basic research of many others. 4 - 11 In particular, the paper by Peterson and Middleton 4 and the survey articles by Schade 5 and Legault 6 have been most influential. However, we believe that our sto- 2572 APPLIED OPTICS/ Vol. 23, No. 15 / 1 August 1984
Transcript
Modulation-transfer-function analysis for sampled imagesystems
Stephen K. Park, Robert Schowengerdt, and Mary-Anne Kaczynski
Sampling generally causes the response of a digital imaging system to be locally shift-variant and not directlyamenable to MTF analysis. However, this paper demonstrates that a meaningful system response can becalculated by averaging over an ensemble of point-source system inputs to yield an MTF which accounts forthe combined effects of image formation, sampling, and image reconstruction. As an illustration, the MTFof the Landsat MSS system is analyzed to reveal an average effective IFOV which is significantly larger thanthe commonly accepted value, particularly in the along-track direction where undersampling contributesmarkedly to an MTF reduction and resultant increase in image blur.
1. Introduction
It is widely recognized that the modulation transferfunction (MTF) of an imaging system is of fundamentalimportance in both the initial specification and designof the system and in any subsequent detailed analysisof the images it produces. Because the MTF is themodulus of the system optical transfer function (OTF),which is, in turn, the Fourier transform of the systempoint-spread function (PSF), the MTF of an imagingsystem can be determined, at least in principle, bygenerating the modulus of the Fourier transform of theimage of a point source. Implicit in all of this is themathematical assumption that the system is both linearand shift-invariant, i.e., that the location (and strength)of the point source can be chosen arbitrarily. If theshift-invariant (isoplanatic) assumption is not valid, theimage of a point source will depend on its location, andthe system MTF will have a spurious location-depen-dent phase dependence.
Unfortunately, the MTF approach to system per-formance analysis is not directly applicable to manycontemporary sampled imaging systems (e.g., elec-trooptical line-scan and sensor-array devices) becausesampling causes these systems to have a particular kindof local shift variance. That is, whenever these systemsform the (sampled) image of a point source, the ap-pearance of the (reconstructed) image will depend on
Robert Schowengerdt is with University of Arizona, Office of AridLands Studies, Tucson, Arizona 185721; the other authors are withNASA Langley Research Center, Hampton, Virginia 23665.
Received 22 December 1983.
the location of the point source relative to the sampling(i.e., pixel) grid. This sample-scene phase phenomenonis familiar to anyone who has tried to analyze systemperformance by observing the appearance of smallhigh-contrast objects in a digital image. For such sys-tems the basic question then is how can the MTF ap-proach to system performance analysis be rigorouslymodified to account for the particular kind of shiftvariance that sampling introduces? This paper pro-vides an answer to that question.
In this paper, the combined effects of imaging, sam-pling, and reconstruction are analyzed in terms of thespatial frequency response of a general sampled imagingsystem. The (linear) system model used is 2-D andsufficiently general so that the results of this MTFanalysis apply to most line-scan and sensor-arrayimaging systems, particularly those used for remotesensing. Sampling effects are accounted for by usinga point-source system input randomly located relativeto the sampling grid and by averaging the system outputover all possible point-source locations. This randomsample-scene phase approach has been used success-fully by the authors recently in related studies of theradiometric error in remote-sensing imagery introducedby the combined effects of sampling and reconstruc-tion.1,2
A similar sample-scene phase approach to systemanalysis was used recently by Wittenstein et al.
3
However, their approach is distinctly different fromours in that it is nonrandom and is restricted to a classof systems for which sample-scene phase dependenceis negligible. Our results, as well as those of Ref. 3, arebased on the basic research of many others.4-11 Inparticular, the paper by Peterson and Middleton4 andthe survey articles by Schade5 and Legault6 have beenmost influential. However, we believe that our sto-
chastic approach to MTF analysis is unique to thispaper and that our results are new.
The results of this paper are illustrated by performingan MTF analysis of the Landsat multispectral scannersystem (MSS).12 This example was chosen because theMSS is a familiar and important (line-scan) imagingsystem which has been in operation more than ten years,and yet a comprehensive system analysis of its fre-quency response, including the combined effects ofsampling and reconstruction, has never been published.We show that (1) in the along-track direction, the MSSresponse is seriously degraded by the undersamplinginherent to a system design in which consecutive in-stantaneous fields of view (IFOVs) are nonoverlapping;(2) in the along-scan direction, undersampling is not aproblem, instead the MSS response is dominated by thescanning aperture and electronic filter; (3) the MSSresponse is somewhat asymmetric with superior per-formance in the along-scan direction; and (4) the com-bined effects of the image-forming optics, scanningaperture, electronic filter, 2-D sampling, and bilinearimage reconstruction, produce an effective IFOV(EIFOV) which is 104 X 148 m.
II. Formulation
A general sampled imaging system is illustrated inFig. 1. All the quantities in this figure are referencedto a common orthogonal spatial coordinate system (xy)normalized so that the sampling interval in both di-rections is unity. Because of this normalizing conven-tion, when the system in Fig. 1 is analyzed in the Fourierdomain, the associated spatial frequencies (v) willhave units of cycles/sample interval, and the Nyquist(or folding) frequency will be 0.5.
The input scene (radiance distribution) is f(x,y) andthe PSF of the (presampling) imaging subsystem ish(xy). The imaging subsystem is assumed to be linearand shift invariant; thus its output image g(xy) is thespatial convolution of the input scene and the subsys-tem PSF. The PSF of the imaging subsystem mustaccount for all the system processing prior to sampling.Thus, in the analysis of an electrooptical scanner likethe Landsat MSS, for example, h (xy) accounts for thecombined effects of the image-forming optics, scanningaperture, and electronic filter. In general, the effect ofthe imaging subsystem is to low-pass filter the spatialfrequency components of the input scene, producing anoutput image which is a smoothed (i.e., blurred) copyof the scene.
Image sampling is the processing which occurs be-tween the continuous output of the imaging subsystemand the discrete input to the reconstruction subsystem.Image sampling is accomplished mathematically bymultiplying the output of the imaging subsystem by the2-D comb function
comb(x,y) = 6(x-my - n), (1)m n
where the notation indicates that the double sum is overall (positive, zero, and negative) integers m, n, and it isassumed that sampling occurs on an equally spaced
Fig. 2. (a) Illustration that the sampling process is not shift-in-variant. A (binary) image g(x,y) with the sampling grid and sampledimagegs(x,y) superimposed. The sampled image is b(xy). (b) The(binary) image g(x - u, y - v) shifted relative to the sampling grid;
the sampled image is not (x - u, y - v).
rectangular grid. Typically, the image is both sampledand quantized for digital transmission or storage priorto image reconstruction. In this paper the effects ofquantization are ignored.
Because of the continuous input/discrete outputnature of the sampling subsystem, it is not shift-in-variant. For example, Fig. 2 depicts a simple binary(i.e., two-level) input image and demonstrates thatcontinuous subpixel shifts in the position of the inputimage can produce significant discrete changes in theoutput sampled image. This shift-variant nature ofimage sampling has significant system implications; itmeans that the sampling subsystem does not have atransfer function, and so the transfer functions for eachof the three subsystems in Fig. 1 cannot simply be cas-caded as one would do in a traditional system MTFanalysis.
The final output of the system in Fig. 1 is a continu-ous reconstructed image gr (xy) which is formed by thespatial convolution of the discrete sampled imageg' (X,y) with the reconstruction (i.e., interpolation)function r(xy). The reconstruction function can bethought of as the (continuous) output response corre-sponding to a (discrete) reconstruction subsystem inputwhich is one at the origin of the sampling grid and zeroelsewhere. Because the reconstruction subsystem iscompletely characterized by its PSF, in an analysis of
Fig. 3. Illustration of the combined effects of imaging, sampling, and reconstruction. The targets in (a) (upper left) are identical as are theirimages in (b) (upper right). Image (c) (lower left) is a reconstruction of a sampled version of (b). Image (d) (lower right) is the difference
of (b) and (c); it illustrates the shift-variant image degradation associated with sampling and reconstruction.
a digital image rectification system, for example,'3r(x,y) must account for the combined effects of allpostsampling operations such as resampling and dis-play.
Figure 3 is a digital simulation of the system in Fig.1 which provides an illustration of the combined effectsof imaging, sampling, and reconstruction. In thissimulation the size of the targets, the effective size of theimaging subsystem PSF, and the spacing (i.e., pixel size)of the sampling grid are all equal. Figure 3(a) is a sceneconsisting of identical (square) targets randomly locatedon a uniformly black background, and Fig. 3(b) is theimage of this scene prior to sampling and reconstruction.Because the imaging subsystem is shift-invariant, eachobject in Fig. 3(b) is blurred identically independent ofits location. Figure 3(c) is the reconstructed imageformed by first sampling the image in Fig. 3(b) and thenreconstructing the sampled image using bilinear inter-polation. Figure 3(d) is the difference between Figs.3(b) and (c), i.e., Fig. 3(d) illustrates the additional
shift-variant image degradation introduced by thesampling and reconstruction subsystems. Two effectsare evident in Figs. 3(c) and (d): (1) the extent of theimage degradation introduced by sampling and recon-struction is very much a function of sample-scene phase;and (2) on the average, the effect of sampling and re-construction is to further blur (low-pass filter) theoutput of the imaging subsystem The intent of thispaper is to rigorously develop an approach to MTFanalysis which will account for the type of shift varianceillustrated in Fig. 3 and correctly characterize on theaverage the net loss of resolution produced by thecombined effects of imaging, sampling, and recon-struction.
The output of the system in Fig. 1 is given by
gr(x,y) = ilf(x,y)*h(xy)J comb(x,y)J*r(x,y), (2)
where * denotes spatial convolution. This equation isthe basis for all the analysis that follows, and, conse-quently, the results of this paper are applicable to the
MTF analysis of any sampled imaging system whoseperformance is characterized by Eq. (2). This equationis also consistent with the system model used by Wit-tenstein et al. ,3 except that their model only includeda 1-D sampling subsystem.
From Eq. (2), the system response to an impulsive(i.e., point-source) input scene 5(x - u,y - v) is thesystem PSF (SPSF) given by
SPSF(x,y;u,v) = [h(x - uy - v) comb(xy)]*r(x,y). (3)
As the notation indicates, the system is shift-variant,and the SPSF is not simply a function of the difference(x - uy - v). As in Refs. 1 and 2, the parameters u andv which locate the point-source input relative to thesampling grid are the sample-scene phase parameters.Because the sampling grid is periodic, it is sufficient torestrict these parameters to the range 0 < u < 1, 0 < v< 1, i.e., to restrict the point source to the sampling gridcell defined by the vertices (0,0), (0,1), (1,1), and (1,0).Furthermore, it is natural to assume that both u and vare uniformly distributed random variables so that thepoint source is equally likely to lie at any position withinthe grid cell. Equation (3) describes the reconstructedimage of the point source for each possible value of (u,v)and accounts for the combined effects of imaging,sampling, and reconstruction. This stochastic ap-proach simulates the process of imaging a scene whichis an ensemble of point sources randomly located in the(x,y) plane and pairwise separated by at least severalsampling cells to avoid response overlap. An averageover all the point-source reconstructions yields a sta-tistical estimate of the average system point-spreadfunction (ASPSF) which includes the combined effectsof imaging, sampling, and reconstruction.
From Eq. (3) and the assumption that the sample-scene phase parameters are uniformly distributed, theASPSF is given by
ASPSF(x,y) = fS JS SPSF(x,y;u,v)dudv. (4)
Since the system in Fig. 1 is linear, as is the averagingprocess in Eq. (4), and since the ensemble of impulsiveinputs are equally likely to lie anywhere within a sam-pling grid cell, it follows that
ASPSF(x,y) = {Frect (x - 2' Y -2) *h(xy)]
X comb(xy)l *r(x,y). (5)
where
rect 1)=(1 O<x<1, 0 < y <1 (6)rec (x 2, ~ - t0 elsewhere.
Equation (5) is fundamental in that it represents theaverage PSF of the entire system and, therefore, ac-counts for the combined effects of imaging, sampling,and reconstruction. Provided the PSF of the imagingsubsystem, h(x,y), and the PSF of the reconstructionsubsystem, r(x,y), are known in a common coordinatesystem normalized to the sampling grid, the averagesystem PSF can be generated as follows: (1) convolveh(x,y) with rect(xy) and center the result at (xy) =
r(x,y)
(a) (b)
ASPSF
(c)
Fig. 4. Average system PSF (c) corresponding to an idealized systemwith a square-imaging subsystem PSF (a) and a bilinear image re-construction PSF (b). All three PSFs are scaled relative to a common
sampling grid.
(0.5,0.5); (2) sample the result of step (1) at all integervalues of (xy); (3) reconstruct the discrete result of step(2) by convolving with r(x,y).
Figure 4 illustrates the ASPSF for an idealized systemin which the imaging subsystem PSF [Fig. 4(a)] is asquare aperture function (i.e., the detector aperturedominates the response of the imaging subsystem), andimage reconstruction [Fig. 4(b)] is bilinear interpo-lation. In the situation illustrated, the IFOV of theimaging subsystem is matched to the sampling grid sothat consecutive sampled IFOVs are just contiguous, acommon design for some line-scan systems and mostsensor array systems. As illustrated in this example,the ASPSF [Fig. 4(c)] is considerably broader than ei-ther the imaging subsystem's PSF (whose width is de-termined by the IFOV) or the reconstruction subsys-tem's PSF (whose width is determined by the samplinggrid). This broadening is caused not only by the inev-itable blurring introduced by the imaging and recon-struction subsystems but also by the combined effectsof sampling and sample-scene phase averaging.
Although considerable insight into system responsecan be gained from Eq. (5), this equation is more easilyanalyzed and understood in the frequency domain. InSec. III of this paper we derive the Fourier transformof Eq. (5), thereby using the concept of sample-scenephase averaging to provide an MTF analysis of thesystem in Fig. 1.
Ill. Fourier Analysis
We define the system optical transfer function(SOTF) as the Fourier transform of the system PSF[Eq. (3)], i.e.,
SOTF(A,v;u,v) = S ¢J SPSF(x,y;u,v)X exp[-2iri(,ux + vy)]dxdy, (7)
where the spatial frequencies ,,v have units of cycles/
I nput imaging I Reconstruction ResponseI_ Ima-ing o- Sampling ---- a
I I i j i(p. i)l
Fig. 5. General sampled image system is amenable to MTF analysisprovided the imaging and sampling subsystems are combined to forman imaging-sampling subsystem whose sample-scene phase-averaged
MTF is Ii(! t ,v)I.
sample interval. From the convolution theorem andthe transform properties of the comb functions it fol-lows that
SOTF(Mv;u,v) = P(,,v) L E exp -22ri[u(u - m)m n
+ (v - n)]&lh( - m,v -n),
M1F
(8)
where Ih and P are the Fourier transforms of h and r,respectively, i.e., i (u,v) is the OTF of the imaging sub-system and P(,,v) is the reconstruction filter.
Just as the average system PSF [Eq. (4)] is the sam-ple-scene phase-averaged system PSF, we define theaverage system OTF (ASOTF) as the sample-scenephase-averaged system OTF, i.e.,
ASOTF(,u,v) = ' Jo STF(ku,v;uv)dudv. (9)
Equation (9) can be reduced by combining Eqs. (8) and(9) and making use of the identity
0' exp(-2iruti)du = exp(-rti) sinc(Q), (10a)
where as usual
smrsinc(Q) - ii ~(10b)The result is the expression
ASOTF(p,v) = P(!t,v) E Lm n
X exp[-7r( - m + v - n)i sinc( - m)X sinc( - n) ( - m, v - n), (11)
which characterizes completely the sample-scenephase-averaged system OTF in terms of the imagingsubsystem OTF and reconstruction filter. It can alsobe verified directly that the Fourier transform of Eq. (5)is Eq. (11).
It is important to recognize that Eq. (11) can bewritten as
ASOTF(uv) = i(,uvW(,e,v) exp[-r(,u + v)i], (12a)
where
= (-l)-+n sinc( - m)m n
X sinc( - n) ( - m,v - n) (12b)
accounts for the combined effects of imaging, sampling,and sample-scene phase averaging. As Eq. (12) reveals,a frequency response analysis of the system in Fig. 1consists of cascading t (,uv) with the reconstruction filterP(u,v) to yield the average system OTF. The phaseterm exp[-7r( + )i] in Eq. (12a) is of no practical sig-
.5 1.0
\\\ \ tV)
\\1V- "(V (- 1) sic ( - 1) \\
-1.0'
Fig. 6. Comparison of the image subsystem OTF 4(v) and the av-erage MTF of the imaging-sampling subsystem It(v)l. Below theNyquist frequency, 0.5 cycles/sample interval, the combined effectsof imaging, sampling, and sample-scene phase averaging reduce theMTF from (v) l to ji(v) j. Above the Nyquist frequency, spectrum
replication causes a spurious MTF increase.
nificance; it is a consequence of the coordinate con-ventions used in this paper which yield an ASPSF [seeFig. 4(c)] centered at (xy) = (0.5,0.5) rather than at theorigin.1
In practice, the modulus of the system frequency re-sponse is the quantity of primary interest. Therefore,we define the sample-scene phase-averaged systemsMTF (ASMTF) as the modulus of the average systemOTF [Eq. (12a)], i.e.,
ASMTF(,4,v) = Ii(,u,v)jjP(,,v)j. (13)
Equation (13) is particularly significant because it re-veals that a sampled imaging system is amenable toMTF analysis provided the imaging and samplingsubsystems are combined into a single imaging-sam-pling subsystem whose sample-scene phase-averagedMTF is It (y,v) I as illustrated in Fig. 5. This MTF iscascaded with the MTF of the reconstruction subsystemto form the MTF of the entire system.
IV. Imaging-Sampling Subsystem MTFBecause the coordinate system in this paper is nor-
malized so that both sampling intervals have unit length(and so the Nyquist frequency is 0.5), the MTF of theimaging-sampling subsystem I i(yu,v) is determined bythe OTF of the imaging subsystem 1(4,v). In an un-normalized coordinate system, I t (yu,v) I would also de-pend explicitly on the size of both sampling intervals.Figure 6 illustrates (in 1-D) an imaging subsystem OTFand the corresponding average MTF of the imaging-
define the effective sampling passband as 0 < v < 0.5,where 0.5 is the Nyquist frequency. For the OTFs il-lustrated in Fig. 7, the effective passband of the imagingsubsystem 0 < v < vi and the effective passband of thesampling subsystem are matched when v, = 0.5. AsFigs. 7(b) and (c) illustrate, the average imaging-sam-pling subsystem MTF is relatively insensitive to changesin the optical subsystem OTF if v, > 0.5. In general,I i (,v) I must be evaluated using Eq. (2b); this is par-ticularly true when vP 0.5. However, it is instructiveto consider the two cases vc < 0.5 [Fig. 7(b)] and vc > 0.5[Fig. 7(c)], for in each case an approximation to Eq.(12b) can be derived which provides insight into theMTF of the imaging-sampling process.
It is intuitive that, if vP is significantly <0.5, the fre-quency response of the imaging subsystem will domi-nate the frequency response of the composite imag-ing-sampling subsystem, i.e.,
(14)
.5 0 .5v v
Fig. 7. Comparison of the optical subsystem OTF (a) with the as-sociated imaging-sampling subsystem average MTF [(b) and (c)].The dashed lines indicate in (b) the approximation lI(v)l (v)I forv = 0.2, 0.3 and in (c) the approximation I (v) I I cos(rv) . In (b)and (c) only the frequency band 0 S v 0.5 is illustrated; at higher
frequencies I (v) is periodic as indicated in Fig. 6.
sampling subsystem. As this figure illustrates, I tk(Mv) Iand h(,i,v)I are nearly identical only at very lowfrequencies; at higher frequencies they differ signifi-cantly because of three effects: (1) The infinite seriesin Eq. (12b) represents frequency spectrum replication,which is the inevitable consequence of periodic sam-pling.4 Because of this replication, I (pt,v) I is periodic,while I h (/,,v) I is not, resulting in a spurious undesirableMTF increase at frequencies beyond 0.5. (2) If, as il-lustrated, the effective cutoff for Ih (i,,v) I is typically notsharp, and some significant image subsystem responseis present at frequencies above 0.5, adjacent terms in Eq.(12b) will overlap. Since these adjacent terms haveopposite signs, cancellation will occur causing significantmodulation suppression, particularly at frequenciesclose to 0.5. This suppression is a mixed blessing:above the Nyquist frequency it is desirable; below theNyquist frequency it is not. (3) The sinc terms in Eq.(12b) are the result of the combined effects of samplingand sample-scene phase averaging; these terms are theprimary source of frequency suppression at interme-diate frequencies.
To what extent does It(u,v)J depend on h(,uv)?Figure 7 illustrates this dependence (in 1-D) for a familyof imaging subsystem OTFs [Fig. 7(a)] with effectivecutoff frequencies v, [i.e., h(v) = 0.5] between 0.2 and0.8. In the discussion that follows it is convenient to
for u < 0.5, v < 0.5. In this case it can be seen that Eq.(12b) reduces to Eq. (14), because at low frequencies thesideband terms [corresponding to (m,n) is (0,0)] can beignored and sinc(A) sinc(v) 1. The dashed lines inFig. 7(b) illustrate (in 1-D) the validity of Eq. (14) fortwo values of vc. If v, = 0.2, Eq. (14) is nearly exact, butif v, is much larger than 0.2, Eq. (14) is not valid exceptat very low frequencies. Of course, for any value of v,Eq. (14) is not valid at frequencies above 0.5 because ofthe periodicity of I (/Lv) I .
It is also intuitive that, if v, is significantly >0.5, thefrequency response of the composite imaging-samplingsubsystem will be dominated by what can be thoughtof as the average effective MTF of the sampling sub-system. This response can be determined from Eq.(12b) by first recognizing that an equivalent represen-tation for t(g,v) is
i(,u) = exp[7r(,u + v)il Z Z[rect(x -1/2,y -
112 )*h(X,y)]mn
X exp[-2r(mp + nv)i], (15)
where [rect(x - 1/2,y - 2)*h(xy)]mn denotes the valueof the convolution at x = m,y = n. The equivalence ofEqs. (12b) and (15) can be verified by taking the inverseFourier transform of each. If vP is significantly largerthan the Nyquist frequency 0.5, the size of the imagingsubsystem PSF will be so small that [rect(x - 1 /2,y -
1/2)*h(xY)]mn will be effectively nonzero only at (m,n)= (0,0), (0,1), (1,0), and (1,1). In addition, if h(x,y) issymmetric in x and y, these four nonzero values will beequal with a common value of 0.25. Thus Eq. (15) willreduce to
Fig. 8. (a) Comparison of the average imaging-sampling subsystemMTF Ii(v)I and the reconstruction system MTF I (v)I corresponding(in 2-D) to bilinear interpolation. (b) Resulting average system MTF,
which is the product of I(v)I and IP(v)I.
Figure 7(c) illustrates the accuracy of this approxima-tion; for vP = 0.8 the approximation is nearly exact.
Again, it should be emphasized that, in the situationwhen the effective passbands of the imaging and sam-pling subsystems are nearly matched, neither Eq. (14)nor Eq. (16b) is a particularly good approximation.Instead, Eq. (12b), or equivalently Eq. (15), should beused to accurately calculate the average frequency re-sponse of the imaging-sampling subsystem.
V. Reconstruction Subsystem MTF
It is well-known that ideal image reconstruction isaccomplished with a low-pass filter whose response isone within the sampling passband and zero elsewhere.However, the reconstruction functions commonly usedin practice are spatially limited, and at best their re-sponse is only a crude approximation to the ideal. Thatis, they do not have either a flat response at lowfrequencies or a sharp frequency cutoff, and they havea significant response at frequencies above 0.5. Thisis illustrated in Fig. 8 for a reconstruction filter whichcorresponds (in 1-D) to bilinear interpolation. Al-though there are other practical reconstruction filterswith somewhat better frequency characteristics,15 theuse of bilinear interpolation in this illustration is ap-propriate because it is a standard reconstruction tech-nique which enjoys widespread use in remote-sensingapplications where reconstructed images are routinelyresampled onto different coordinate grids.
ASMIF .5
0 .5 1.0v
Fig. 9. Comparison of the average system MTF for two recon-struction filters: linear and parametric cubic convolution (PCC) witha = -0.5. For comparison, the ASMTF associated with ideal re-construction [r(x) = sinc(x)] is also shown. Below the Nyquist fre-
quency (0.5), PCC is clearly superior to linear interpolation.
Figure 8(b) illustrates the average system MTF (in1-D) which is the product of I(Eu,v)i and IP(/,,v)I. Thenonideal response of the reconstruction filter leads totwo distinctly different effects: (1) within the samplingpassband the nonideal (z 1) response has the net effectof reducing system MTF and creating additional blurin the reconstructed image; (2) beyond the samplingpassband the nonideal (iz 0) response fails to suppressany spurious system response created by sampling andout-of-band imaging subsystem response. It is thislatter effect which produces aliasing in, the recon-structed image if the scene has significant energy atfrequencies outside the sampling passband.
In a recent article,15 the authors discussed a new al-gorithm, parametric cubic convolution (PCC), whichcan be implemented efficiently and yet contains a pa-rameter a which can be adjusted to tailor the MTF ofthe reconstruction filter. For a particular choice of thisparameter, a = -0.5, the MTF is nearly flat at lowfrequencies and effectively zero for frequencies beyond1.0. Figure 9 illustrates the ASMTF improvementwhich is possible if PCC (with a = -0.5) is used in placeof bilinear interpolation for image reconstruction. Thesuperior ASMTF achieved with PCC has been shownto produce reconstructed images with less radiometricerror than those produced with bilinear interpolationand standard bicubic convolution.2
VI. Example: Landsat MSS
The results of this paper are applicable to any line-scan or sensor-array imaging system provided it ischaracterized by Eq. (2). As an illustrative example,these results will be applied to the Landsat multispec-tral scanner system (MSS). This example was chosenbecause the MSS is a familiar and important line-scanimaging system whose multispectral images have beenused in a multitude of remote-sensing applications, andyet, to date, a comprehensive system analysis of theMSS frequency response has not been published.Fortunately, the paper by Slater16 and the four re-sponses that followed by Colvocoresses 1 7 Thompson, 18
Slater,19 and Friedmann20 provided all the information
Butterworth filter produces significant low-pass fil-tering and thus additional image blur, but only in thealong-scan direction. (2) Unlike the PSFs for theimage-forming optics and the scanning aperture, thePSF for the Butterworth filter does not have spatialsymmetry, and, as a result, its Fourier transform iscomplex; that is, the three-pole Butterworth filter hasan amplitude response and a phase response. At lowspatial frequencies the phase response is linear, but athigher spatial frequencies it is not, and, as a result, theButterworth filter introduces an undesirable non-neg-ligible scene-dependent lag in the along-scan directionas illustrated by Thompson. 1 8 (3) The analog-to-digitalconversion (sampling) in the along-scan direction re-quires some nonzero integration time and thus intro-duces some additional image blur. However, this time(80 nsec) is so small that the resultant image blur isnegligible, i.e., it corresponds to <0.5 m in the along-scan direction.1 6
In summary, the spatial-frequency response of theMSS imaging subsystem is given by
where ib is the (complex) three-pole Butterworthtransfer function defined by21
Fig. 10. Imaging subsystem MTF of the Landsat MSS. In thealong-track direction, the frequency response is significantly
I undersampled.
and data needed to prepare this example. As is com-mon in remote sensing, all the MSS system parametersare referenced to equivalent ground-projected dimen-sions at nadir. Thus, for example, 76 X 76 m is theground-projected area of the IFOV.
Scene radiance entering the MSS is first low-passfiltered by the imaging-forming optics whose PSF canbe approximated by a Gaussian with a 30-m blur cir-cle.1 6 The image-forming optics is followed by a scan-ning aperture (formed by optical fibers in the focalplane) which has the effect of convolving the output ofthe imaging imaging-forming optics with the IFOV.Thus the combined effect of the imaging-forming opticsand the scanning aperture is to low-pass filter (i.e., blur)the input scene equally in both the along-scan (x) andthe along-track (y) directions.
The scanning aperture has the effect of sampling inthe along-track direction with an 81.5-m sampling in-terval. It is a consequence of the MSS design that ad-jacent sampled IFOVs in the along-track directioncannot overlap and, in fact, they are not even contiguousbecause of the cladding that surrounds the optical fi-bers.16
As the scan mirror of the MSS oscillates, the analogoutput of the detectors is electronically processed witha low-pass three-pole Butterworth filter and thensampled at a rate corresponding to a ground-projected58-m sampling interval in the along-scan direction.The electronic signal processing in the along-scan di-rection has led to some confusion in the remote-sensingliterature, apparently for the following reasons: (1) The
Iib(@) =(1 - 2W2) - i(2 - W2)
1 + W6 (c = kbat), (17b)
and the constants are determined from those publishedby Friedmann 20 :
31.1 76.2 138 31.1 76.2
58 58 58 81.5 81.5
The first three terms in Eq. (17a) account for the spa-tial-frequency response of the imaging subsystem in thealong-scan direction, and the last two terms account forthe response in the along-track direction.
Figure 10 illustrates the MTF (I F (,u,v) I) of the MSSimaging subsystem in unnormalized spatial frequencies1u',v', where it' = A158 [cycles/m] and v' = v/81.5 [cy-cles/m]. For reference, the Nyquist frequencies (163)-i= 0.00613 cycles/m and (116)-i = 0.00862 cycles/m inthe along-track and along-scan directions, respectively,are also indicated. In the along-track direction [Fig.10(a)], the effective cutoff frequency of the imagingsubsystem (0.00765 cycles/m) is larger than the Nyquistfrequency corresponding to the case illustrated in Fig.7(c). In contrast, for the along-scan direction [Fig.10(b)], the effective cutoff frequency (0.0065 cycles/m)is smaller than the Nyquist frequency corresponding tothe case illustrated in Fig. 7(b).
Figure 11 illustrates the average imaging-samplingsubsystem MTF (It(,i,v)l) of the MSS. In the along-track direction [Fig. 11(a)], sampling produces a sub-stantial MTF reduction (the effective cutoff frequencyis 0.0041 cycles/m) because the sampling passband issignificantly smaller than the imaging subsystempassband. This mismatch of passbands is, of course,an inevitable consequence of the MSS design; in retro-spect, it could have been avoided by shaping the scan-
Fig. 11. Average imaging-sampling subsystem MTF of the LandsatMSS. In the along-track direction, undersampling has producedlarge MTF reductions; in the along-scan direction that is not the
case.
ning aperture in the along-track direction and bystaggering the optical fibers in the focal plane to yieldoverlapping consecutive IFOVs (see Refs. 22 and 23).
In the along-scan direction [Fig. 11(b)] samplingproduces very little MTF reduction (the effective cutofffrequency is 0.0058 cycles/m) because the samplingpassband is larger than the imaging subsystem pass-band. This more desirable relation between passbandsis the result of the low-pass electronic filter and theoverlapping of consecutive IFOVs along the line scan.
As Fig. 5 illustrates, to determine the MTF of theMSS system (including ground processing), it is nec-essary to cascade the MTFs in Fig. 11 with the MTF ofthe filter used for image reconstruction (i.e., resam-pling); for this example, we have chosen bilinear inter-polation. The 2-D reconstruction filter associated withbilinear image resampling is
P(liv) = sinc2(Ae) sinc2 (v). (18)
The result of cascading this filter (see Fig. 8) with theMTF in Fig. 11 is the average system MTF illustratedin Fig. 12. For comparison the ASMTF in both direc-tions is plotted on a common frequency axis; this com-parison illustrates the superior MTF of the MSS in thealong-scan direction.
Figure 12 reveals that the Landsat MSS systemproduces images which are somewhat more blurred inthe along-track direction. That is to be expected be-cause, even though the IFOV is square, there are 1.31samples per IFOV in the along-scan direction and just
ASMTF
0
Average system MTF, along-track
Average system MTF, along-scan
.005 .010Spatial freauency. cycles/m
.015
Fig. 12. Average system MTF of the Landsat MSS, including theeffects of bilinear image reconstruction. Frequency response is sig-
nificantly better in the along-scan direction.
Fig. 13. Simulated image of a target as it would be formed by asystem whose EIFOV is as indicated. The additional blur associatedwith the larger asymmetric EIFOV is much more evident than the
associated asymmetry.
0.93 samples per IFOV in the along-track direction.However, this asymmetric blurring is not readily ap-parent in visual examinations of Landsat images.17
Why? To answer this question, we need to relate theMTF to spatial dimensions which are a measure of theaverage system PSF size. Such a measure was proposedin 1973 by a NASA working group10 and later refinedby Slater"1; it is the effective IFOV (EIFOV). In thenotation of this paper,
(EIFOV)aiong-scan = I-- = 104 m,2A'c
(EIFOV)' =g-track 1 = 148 m,x~~along-track 2v~
(19a)
(19b)
where Ac and v are the effective system cutofffrequencies, i.e., where Muc and 'c are the spatialfrequencies (cycles/m) at which the ASMTF (Fig. 12)is reduced to 0.5. Thus the combined effects of imag-ing, sampling, and resampling produce on the averagea rectangular EIFOV which is 104 X 148 m. For mosttargets this rectangular EIFOV is not sufficientlyasymmetric to produce an evident asymmetric targetblur. We demonstrate this with Fig. 13 using a targetsuggested by Colvocoresses.17 The target is in the
Fig. 14. (Sample-scene) phase-averaged EIFOV of the Landsat
MSS: (a) based on only the scanning aperture; (b) based on the
scanning aperture, image-forming optics, and electronic filter; (c)
including also the effects of sampling; (d) the system EIFOV with
bilinear image reconstruction; (e) the system EIFOV using PCC(a = 0.5) in place of bilinear image reconstruction.
upper left, and the object in the upper right is the imageof this target as it would be formed by convolution withthe square EIFOV indicated. The two objects in thelower left and right are more blurred images of thistarget as they would be formed by convolving with thelarger rectangular 104- X 148-m EIFOV. Two orthog-onal orientations of this rectangular EIFOV are shownto illustrate, when compared with the object in theupper right, just how little asymmetry is present in MSSimages of this target. In addition, the actual appear-ance of this object in MSS images would be further de-graded by system noise, and, as a result, any asymme-tries would be even less evident than that illustrated.
It should be possible to demonstrate the asymmetricblurring of the MSS by imaging a point source, for ex-ample, a small highly reflective mirror, as was done inthe Shadow Mountain Eye Project.24 However, it ismisleading to consider just one or two point source im-ages because of the sample-scene phase phenomena il-lustrated in Fig. 3. If an ensemble of properly regis-tered point-source images were averaged, the resultwould be the ASPSF, and the magnitude of its Fouriertransform would be the ASMTF. This ASMTF shouldbe asymmetric as indicated in Fig. 12. Unfortunately,to the best of our knowledge, such an analysis has neverbeen published.
The results of this system MTF analysis for theLandsat MSS are summarized in Fig. 14 in terms ofground-projected EIFOVs using Eq. (19) and the vari-ous effective cutoff frequencies illustrated in Figs.10-12. In each case, in the discussion that follows, theEIFOV dimension in the along-scan direction will belisted first. However, it is first necessary to recognizethat, because Slater's definitional of the EIFOV is basedon a 50% MTF reduction level and because the solutionto sinc(v) = 0.5 is v 0.60 rather than v = 0.50, theEIFOV of the 76- X 76-m scanning aperture is just 63 X63 m. Thus, all the EIFOVs listed in the next para-
graph and illustrated in Fig. 14 can be meaningfullycompared in a relative sense, but some might argue thatall are too small by some common multiplicativefactor.
If it is assumed that the system response is just theimage subsystem response (Fig. 10), the (erroneous)conclusion would be that the EIFOV is 77 X 65 m.Comparing this EIFOV with the scanning aperture 63-X 63-m EIFOV, we see that the imaging-forming opticscause very little increase in EIFOV. However, theelectronic filter causes (on the average) an increase of-12 m but only in the line-scan direction. The EIFOVcorresponding to the average imaging-sampling sub-system response (Fig. 11) is 86 X 122 m. Thus the(average) effect of sampling is to increase the EIFOVby just 9 m in the along-scan direction. However, in thealong-track direction the EIFOV increase is 57 m, il-lustrating dramatically the consequences of under-sampling. Finally, since the system EIFOV is 104 X 148m, we see that bilinear image resampling yields an ad-ditional EIFOV increase of 18 m in the along-scan di-rection and 26 m in the along-track direction. Thisadditional EIFOV increase can be reduced by usingreconstruction filters with superior frequency response.For example, using PCC with a =-0.5 in place of bi-linear interpolation (see Fig. 9) yields a smaller systemEIFOV of 93 X 132 m. However, image resamplingwith PCC involves computations with a 4- X 4-pixelmask; for bilinear resampling the pixel mask is just 2 X2. Thus the superior ASMTF (and smaller systemEIFOV) associated with PCC image reconstruction isachieved at the cost of increased digital processing.
VII. Summary
This paper has presented a general approach to de-termining the frequency response of a sampled imagingsystem. The approach accounts for the local systemshift variance induced by sampling and the combinedeffects of image formation, sampling, and image re-construction by using an ensemble of point-sourcesystem inputs and averaging over all possible point-source locations. The resultant expression for theaverage system MTF has general applicability andprovides a numerical measure of system performancewhich is easily evaluated in either of two equivalentforms.
As an example, the average system MTF of theLandsat MSS was analyzed. The results of this analysisprovide for the first time a comprehensive examinationof all the interacting factors which contribute to thetotal MSS frequency response. The results weresummarized in terms of a system EIFOV which turnedout to be significantly larger than the commonly ac-cepted IFOV, particularly in the along-track directionwhere the effect of undersampling contributes markedlyto an MTF reduction and resultant increase in imageblur.
This paper is based on one presented at the SecondTopical Meeting on Coherent Laser Radar, Aspen,Colo., 1-4 Aug. 1983.
