1. INTRODUCTIONGround-based optical sensors have been developed withthe capability to produce quality images of satellites andother space objects that reveal the shape of their three-dimensional (3-D) structure.1–4 An obvious opportunityprovided by such imagery is the ability to estimate a sat-ellite’s orientation.5,6 It also opens up a capability forcharacterizing the state of a satellite and its subcompo-nents in terms of external features such as their relativeorientation, size, shape, and reflectance properties. Im-age data taken at multiple wavelengths may permit de-termination of surface temperatures and, to some degree,material types. The accuracy with which satellite prop-erties can be assessed is limited by many factors, whichcan be grouped into two categories: image degradationand nonoptimality of the estimation algorithm. The fun-damental accuracy limits imposed by image degradationfactors (i.e., assuming an optimal estimation algorithm)can be expressed by the Cramer–Rao lower bound(CRLB).7,8 This bound describes the minimum meansquare error (MMSE) with which an unbiased estimate ofa set of system properties can be produced, given thenoise properties of the measurements being made and therelation of the measurements to the unknown param-eters. It is specified in terms of the sensitivity of themeasurement to the target properties and the statistics ofthe noise processes involved. For future reference, themathematical expressions describing these relationshipswill be termed the Bayesian model. A convenient aspectof the CRLB metric is that it can be calculated directlyfrom the Bayesian model and does not require actualimplementation of an optimal estimation algorithm(which may be difficult or impractical).
The subject of this paper is to explore the use of CRLBsas an analysis methodology for design, upgrade, and op-eration of space target characterization systems. The fo-cus will be on object orientation. Determining satelliteorientation is of special interest (generally more so than
for ground targets) because orientation is closely associ-ated with a satellite’s health, status, and operation. Thiswork advances on previous work related to target orien-tation estimation in several ways. First, it demonstratesthe use of a CRLB approach to calculating orientation ac-curacy bounds. To the knowledge of the authors, all pre-vious work on Bayesian-based 3-D orientation accuracybounds has used the MMSE or similar metrics, instead ofCRLBs.9–14 An advantage of the MMSE is that it isachievable by the MMSE estimator, whereas except insimple cases the CRLB is achievable only asymptoticallyas signal-to-noise ratio or data set size approachesinfinity—in which case both bounds coincide.7,13,14 How-ever, calculating the MMSE and implementation of theMMSE estimator typically involves evaluation ofweighted averages of the probability density function(pdf) associated with the Bayesian model over the space ofpossible parameter values. The CRLB, however, is solelya function of the local derivatives of the PDF and thustends to require fewer computational operations for itsnumerical evaluation. In this sense, the CRLB may bemore convenient, especially for problems involving pa-rameter spaces of high dimensionality. It must be notedthat the use of the standard CRLB formulation for pa-rameter sets that do not form a simple flat Euclideanspace (for example, orientation Euler angles) lacks themathematical rigor of MMSE-based approaches.9,15 Thislimits its practical use to situations in which the accuracybound values are small relative to the parameter space—for example, situations with orientation accuracy boundsmuch less than 360°.
A second major contribution of this work is its explora-tion of the influence of uncertainties regarding the stateof the target/sensor system that are in addition to mea-surement noise. As used here, the term uncertainties in-dicates secondary model parameters, which are not of di-rect interest but, by virtue of being uncertain, impact theaccuracy with which the unknowns of principal interest
2003 Optical Society of America
818 J. Opt. Soc. Am. A/Vol. 20, No. 5 /May 2003 Gerwe et al.
can be estimated. These secondary parameters aresometimes referred to as nuisance parameters. Theprincipal parameters concentrated on in this paper arethe three Euler angles used to describe a satellite’s orien-tation. Some examples of secondary parameters in-cluded in the CRLB calculations performed in this paperare the relative orientation of the solar panels, the lengthof the body, surface reflectivities, and the width of the sen-sor’s point-spread function (PSF). Typically, the valuesof the secondary values are known within some tolerancebefore the measurement. The degree of certainty associ-ated with such a priori information may be characterizedin the Bayesian model. As will be demonstrated in Sec-tion 4, the inclusion of, and the degree of uncertainty as-sociated with, the secondary parameters can have a pro-found influence on the calculated CRLBs. This indicatesthat to obtain realistic accuracy limit predictions, it is im-portant for the Bayesian model to include all sources ofuncertainty that significantly influence estimationerror.14 Satellite orientation algorithms that include so-lar panel articulation as a free parameter have been dem-onstrated, and their accuracy characterized, by using aMonte Carlo approach.5,6 However, the influence of in-cluding satellite articulation on estimation accuracy wasnot explored, and the algorithms were not Bayesianbased.
It is also worth mentioning that the novel image-rendering technique developed for this work provides spe-cific advantages over typical ray-tracing-based methods.8
The technique provides a robust method for accurate cal-culation of partial derivatives of the expected image withrespect to the target/sensor parameters using finite differ-ences. These derivatives are needed in calculating theCRLB. They are also of importance in implementing es-timation algorithms such as least squares, maximumlikelihood (ML), maximum a posteriori, and MMSE withthe use of gradient-based search approaches.2,10,11
The ability to analyze accuracy limits is important atmany stages of system design, upgrade, and operation.Once a need for a new capability is recognized, the CRLBprovides a convenient way of determining the feasibilityof different sensor designs. It can also reveal how theunderlying physics may give one sensing modality a fun-damental advantage over another for estimating particu-lar target properties. An intuitive example would beverification that the depth of an object can be more accu-rately determined by 3-D range laser radar than by pas-sive imaging, given the same aperture size and illumina-tion levels. A more subtle problem would be to determinewhether satellite orientation is more accurately measuredby a shared-aperture system with a passive imager and3-D laser radar or by a system consisting of two geo-graphically separated passive imagers.
Once the sensor is built, CRLB analysis serves as auseful tool for optimizing sensor operation and eliminat-ing residual sources of error. If operational accuracy ap-proaches the CRLB, there will be little to be gained by in-vesting more effort in refining the estimation algorithm orin improving measurement calibration and technique.More likely, accuracy limits indicated by initial CRLB cal-culations will be well below the performance that a sys-tem is actually providing. The discrepancy can be sepa-rated into two components. The first component will bedue to nonoptimality of the algorithm. Once identified,this part offers a potentially inexpensive opportunity forimproving performance (algorithm development often be-ing less costly than hardware upgrades). The other por-tion of the discrepancy will be due to the fact that therewill always be uncertainties about the target/sensor sys-tem and residual sources of measurement noise that arenot included in the Bayesian model used for the CRLBcalculations. As indicated in Fig. 1, increasing the fidel-ity of the model to include more factors of uncertainty willimprove the relevance of the resulting bound. A trouble-
Fig. 1. Noise sources and uncertainties regarding the target/sensor system that are not included in the forward model on which theCRLB calculations are based can cause the resulting bound to be optimistically low.
Gerwe et al. Vol. 20, No. 5 /May 2003/J. Opt. Soc. Am. A 819
Fig. 2. Hypothetical example of how CRLB calculations might compare against performance accuracy of optimal estimators and opera-tional codes.
some aspect to this problem is deciding when all signifi-cant factors have been included.
One method that may assist in revealing missed factorsis to compare estimator performance as applied to realdata against performance against data simulated by theBayesian model. If the algorithm is optimal, the differ-ence in performance will be directly related to the sourcesof noise and system uncertainty that have not been mod-eled. Optimal estimators (ML, maximum a posteriori, orMMSE depending on definition of optimal) can often berealized by using the Bayesian model and an approachbased on a systematic or iterative search over the param-eter space.1–3,7,10–12 (If the associated numerical calcula-tions are computationally intensive and convergence isslow, operational requirements may motivate develop-ment of suboptimal noniterative estimators.) Hopefully,such a comparison gives insight into improvements thatcan be made to the Bayesian model or identifies unneces-sary noise that can be removed by fixing the system (e.g.,miscalibrations or noisy circuits).
Figure 2 presents a hypothetical example of how agree-ment between predicted and actual performance improvesas such discrepancies are resolved. In this example, dur-ing the system design phase, performance accuracy is pre-dicted through CRLB analysis and by applying aniterative-search-based ML estimator to simulated data.
The small discrepancy between the resulting performancevalues indicates that the CRLB is approached but not at-tained for this case. Uncertainty in solar panel articula-tions (secondary nuisance parameters) is found to signifi-cantly impact performance and should be included infuture analysis. When the system goes on line, both anoperational algorithm and the ML algorithm are appliedto the experimental data. Discrepancies in the resultsassist in discovering the importance of including sometarget dimensions as estimated parameters, leading toimprovements in the operational estimator and corre-sponding increases in the CRLB. Meanwhile, bugs in thesystem are discovered, improving agreement betweenpredicted and actual performance. Further CRLB analy-sis indicates that atmospheric-turbulence-induced PSFuncertainties are found to significantly impact perfor-mance, and the optimal estimator is modified to includethese factors as additional free parameters. A way of ad-dressing this issue while maintaining timing require-ments on the operational code is not apparent, and a re-sulting performance penalty must be suffered until abetter solution can be found. The discrepancy betweenperformance obtained by applying the optimal ML esti-mator against the real and simulated data, indicates thatthere are still factors in the system that, if better under-stood, might lead to a better estimator. Motivated by
820 J. Opt. Soc. Am. A/Vol. 20, No. 5 /May 2003 Gerwe et al.
this possibility, further work uncovers some nuances re-lated to calibration measurements used in the algorithm.Application of a combination of better calibration proce-dures and inclusion of related secondary parameters inthe estimators brings all the performance metrics withingood agreement. The discrepancy between operationaland potential performance is now quite small, and futureeffort at system improvement is deemed to be at a point ofdiminishing returns.
Section 2 will derive a set of CRLB formulas appropri-ate to systems that are dominated by an additive combi-nation of Gaussian and Poisson noise that is uncorrelatedbetween data elements. These formulas require deriva-tives of the average measurement with respect to all un-known target/sensor parameters. The 3-D geometry ofthe targets makes the relationship of the measurementand their derivatives with respect to the orientation vec-tor highly nonlinear. Our approach for numerically ap-proximating the derivatives is briefly described in Section3. Section 4 examines how the CRLB calculations are in-fluenced by secondary parameters used to describe uncer-tainties regarding the details of the target/sensor model.Initially, these additional parameters are treated as ifthey were completely uncertain. Section 5 builds onthese results by including the possibility of knowing theparameters to some precision before the measurement,i.e., a priori knowledge. Conclusions are drawn in Sec-tion 6.
2. CRAMER–RAO LOWER BOUNDCALCULATIONS FOR TARGETCHARACTERIZATIONIn this section, the CRLB will be derived for vector mea-surements that are dominated by a combination of photonshot noise (Poisson) associated with a sum of sources in-cluding target, background, ambient light sources, anddark current, and of sensor electronics noise (Gaussian).The details of the noise model are specified later in thissection. Presume that j 5 $j1 , j2 ,..., jk ,..., jN% speci-fies all relevant target/sensor system parameters in whichthere is a degree of uncertainty. The CRLB for estimat-ing each parameter jk from a measurement vector d isgiven by7
^@ jk 2 jk#2& > CRLB$jkuj% [ @FD21~j!#kk , (1)
@FD~j!#kl 5 2K ]2 ln p~duj!
]jk]j lL
5 K ] ln p~duj!
]jk
] ln p~duj!
]j lL . (2)
In these equations, j is the estimated value of j, FD(j) isthe Fisher information matrix representing the informa-tion content of the data, and the averages are performedover the noise statistics. The quantity p(duj) is the pdffor obtaining measurement d given values of j corre-sponding to a particular system state. This functionmust be specified such that it accurately models the sys-tem of interest.
Note that in the rigorous sense the classical Cramer–Rao bound formulation applies only to flat Euclideanspaces in which each element of the vector parameter jmay take on any real value. Alternative treatments havebeen developed for parameter spaces with curved geom-etries (such as orientation Euler angles in which, for ex-ample, u50° may be identical to u5360°).9,13–15 How-ever, as long as the actual bounds are small relative to therange of the space, the Cramer–Rao approach can be suc-cessfully used to produce reasonable results. For ex-ample, use of the CRLB is probably a valid approach forsystems in which the calculated bound on an orientationangle is typically less than 10°. (In this paper, all calcu-lated bounds are less than 1°.) However, when boundsexceed 10°, the results might be suspect, and, of course,bounds greater than 360° are completely meaningless—other than the conclusion that the measurement providesalmost no information regarding the target’s orientation!Related issues of rotational symmetries in which two ormore orientations appear similar may also be better ad-dressed by bounds such as that given in Refs. 9 and 13–15. In this paper, it is assumed that the combination ofsensor resolution and a priori knowledge of the target’sorientation state is such that such symmetries do not posean issue.
Since the ordering of the Euler rotations used to specifyan object’s pose with respect to a reference orientation isnot communicative, care must be taken in interpretingbounds given on an Euler angle orientation parameteriza-tion. That is, they must be interpreted in the context ofthe relation between the actual and reference orienta-tions. To make interpretation as simple as possible, wecalculate the CRLBs presented in this paper by using theactual orientation of the object as the reference orienta-tion. Given that the resulting bounds are small (in thiscase, less than 1°), differences between the ordering of ro-tation is negligible, and the bounds can be interpretedsimply as the accuracy in yaw, pitch, and roll about theactual orientation.
Typically, in addition to the principal parameters thatone wishes to estimate, there are additional uncertaintiesassociated with the target/sensor system. As discussedin Section 1, these secondary or nuisance parameters mayinfluence the accuracy with which the principal param-eters can be estimated. (An optimal algorithm wouldjointly estimate the values of both the principal and sec-ondary parameters. The CRLB corresponds to the bestaccuracy achievable by this approach. Realistic estima-tors would treat many of the secondary parameters as de-terministic inputs. Uncertainties in these inputs induceunwanted biases in the estimates that would on averagetypically degrade accuracy.) Continuing to use j to de-note the set of uncertainties considered in calculating theCRLB, we will denote the set of all other uncertainties byv (since, most generally, the target/sensor system is em-bedded in the universe, then, by definition, v is infinitelylarge). The values of parameters included in v that ap-pear in the Bayesian model equations are treated as be-ing known with absolute certainty; i.e., there is no errorassociated with these inputs. All other aspects of v areinherently not included in the CRLB calculations by vir-tue of the fact that their absence from the Bayesian model
Gerwe et al. Vol. 20, No. 5 /May 2003/J. Opt. Soc. Am. A 821
is an implicit statement of their complete lack of relationto the target/sensor system and associated measure-ments. To obtain realistic bounds, it is important for j toinclude all uncertainties that significantly influence esti-mation of the principal parameters. Results presented inSection 4 illustrate that choosing a parameter set j thatneglects important uncertainties can produce bounds thatare overly optimistic.
Realistic Cramer–Rao calculations also necessitateconsideration of the influence of any a priori informationthat might be employed. This is easily accomplishedthrough use of the joint distribution p(d, j)5 p(duj)p(j), where p(j) is the statistical prior on j.The influence of additional a priori information on thebound is captured by the second term in the modified in-equality
^~ jk 2 jk!2& > @$FD~j! 1 FP~j!%21#kk , (3)
where FP(j) quantifies the Fisher information associatedwith the prior p(j) and is given by
@FP~j!#kl 5 2K ]2 ln p~j!
]jk]j lL 5 K ] ln p~j!
]jk
] ln p~j!
]j lL .
(4)
When the a priori statistics of j are jointly Gaussian withcorrelation Lj and mean ^j&, i.e.,
p~j! 5 @~2p!N/2uLju1/2#21 exp@212 ~j 2 ^j&TLj
21~j 2 ^j&!#,(5)
the a priori information is simply described by the inverseof the correlation matrix of j, i.e., FP 5 Lj
21. The influ-ence of secondary parameters and a priori information ontarget parameter estimation accuracy bounds will be ex-plored in detail in Sections 4 and 5.
To develop the imaging CRLB further, we must nowspecify the noise statistics described by p(duj). We con-sider imaging systems that are dominated by a combina-tion of photon shot noise (Poisson) associated with a sumof sources including target, background, ambient lightsources, and dark current, and of sensor electronics noise(Gaussian). Noise corresponding to different measure-ment elements and different exposure times is assumedto be statistically independent. Note that noise associ-ated with digital quantization of the signal has been ne-glected. This approximation is reasonable as long as thequantization step size is significantly smaller than eitherthe Poisson or Gaussian noise components.
With the state of the target/sensor system specified asabove by j, the measured data dq( x) at sensor element(pixel) x 5 (x, y) of image q is given by
dq~ x ! 5 Poisson$ gq~ x, j!% 1 nsq~ x,j! . (6)
Here nsq( x,j) is a zero-mean Gaussian random variablewith standard deviation sq( x, j). Poisson$J% is a Pois-son random variable with mean J. The function gq( x,j)specifies the ensemble-average photoelectrons plus darkcurrent detected at sensor element x of image measure-ment q when the target/sensor system state is j. Thestandard deviation of the Gaussian noise component isalso allowed to be a function of pixel position x and mea-surement index q but henceforth will be treated as inde-
pendent of target/sensor state and thus will be written ass x,q . Note that the Poisson and Gaussian noise termsmay each include contributions from several noise pro-cesses. This noise model is appropriate for incoherentimage measurements using a wide class of sensors suchas CCDs,16,17 complementary metal-oxide semiconductor,and photomultiplier arrays along with many IR sensors.Accurate treatment of noise sources with pixel-to-pixelcorrelations such as laser speckle, turbulence-inducedscintillation, and backgrounds with random structure(such as clutter) would require further generalization ofthe imaging model presented here.
It is often reasonable to treat the described noise com-bination by introducing an artificial measurement bias toa Poisson distribution such that the resulting mean andvariance are the same as those described by Eq. (6)16,17:
dq~ x ! > Poisson$ gq~ x, j! 1 s x,q2 % 2 s x,q
2 . (7)
If s x,q2 is the level of Gaussian noise at pixel x, then the
pdf for a set of image measurements indexed by q and cor-responding to the approximation made in relation (7) is
p~d 1 s 2uj!
> )q
)xPFq
3exp@2~ gq~ xuj! 1 s x,q
2 !#@ gq~ xuj! 1 s x,q2 #dq~ x ! 1 sx,q
2
@dq~ x ! 1 s x,q2 #!
.
(8)
Here Fq denotes the set of focal-plane-array (FPA) ele-ments in the field of view (FOV). The function gq( xuj)specifies the mean FPA element values corresponding tothe system state j and thus embodies the relation of theensemble-average image to all target/sensor parameters.If we substitute relation (8) into Eq. (2) and employ thefact that for Poisson statistics the variance is equal to themean, it is straightforward to show that
@FD~j!#kl > (q
(xPFq
]gq~ xuj!
]jk
]gq~ xuj!
]j l
gq~ xuj! 1 s x,q2 . (9)
The factors s x,q2 and g( xuj) in the denominator of Eq.
(9) correspond to the Gaussian and Poisson noise compo-nents, respectively. When s x,q
2 5 0, i.e., the noise con-sists solely of the Poisson component, the s x,q
2 term disap-pears and relations (7)–(9) become exact. If the noiseconsists only of a Gaussian component, a similar formulato relation (9) can be derived:
@FD~j!#kl 5 (q
(xPFq
]gq~ xuj!
]jk
]gq~ xuj!
]j l
s x,q2 . (10)
When k 5 l, each term of the sums in relation (9) ex-presses the Fisher information gained from an individualFPA element x about a single state parameter jk . Fromthis, we see that the Fisher information for parameter jkis proportional to the sensitivity of a pixel measurementto variations in the state parameter under considerationand inversely proportional to the measurement noise.
822 J. Opt. Soc. Am. A/Vol. 20, No. 5 /May 2003 Gerwe et al.
Because of statistical independence of the noise at eachpixel, the total Fisher information is simply the sum overthe Fisher information obtained from all detector ele-ments. When k Þ l, each term in relation (9) expressesthe joint Fisher information for system parameters jk andj l contributed by each FPA element. As a result of theindependence of noise between images, the Fisher infor-mation available from the combined image set is simplythe addition of the Fisher matrices corresponding to eachimage individually. Because of this simple result, it iseasy to calculate the potential performance benefits thatcould be achieved by fusing data from multiple imagemeasurements and imaging modalities. It can be shownthat for each element of the estimated vector parameter,combining information from multiple independent mea-surements always reduces the CRLB below that obtainedfrom any single measurement.8 This is a direct result ofthe data processing theorem.18,19 Examples of this effectwere explored in a previous paper.8
Later sections of this paper will focus on sensors thatmeasure the intensity distribution of light reflected off atarget as a function of angle about the line of sight. Shot-and sensor-noise-dominated gray-scale images taken by aconventional passive electro-optic system are common ex-amples that follow this model. Relation (9) also appliesto many IR and multispectral imaging systems and maybe appropriate to other sensor types as well.
3. BAYESIAN MODELING OF THE TARGET/SENSOR SYSTEMSection 2 demonstrated that a general statistical modelfor many imaging modalities and the correspondingCRLB for estimating a set of unknown parameters couldbe described by Eq. (6) and relation (7), respectively.Lurking within these simple formulas is the potentiallycomplicated function g( xuj), which describes the meanmeasurement that would be made by sensor q when thestate of the target/sensor system is parameterized by thevector j. For a typical imaging system, g( xuj) must de-scribe the 3-D relations between the sensed image andthe target structure along with effects such as directionallighting, viewing perspectives, radiometry, spectral ef-fects, system PSF, properties of the detector elements(e.g., CCD FPA), and the bidirectional reflectivity distri-bution function (BRDF) describing the surface reflectanceproperties. Relation (7) also requires calculation of thederivative of the mean measurement at each FPA elementas a function of each state parameter of interest, i.e., jk .For all but the simplest shapes, the complex relationshipbetween a 3-D structure and its two-dimensional projec-tion onto an image plane prohibits closed-form expres-sions. To enable the treatment of realistic targets, wetake the common approach of approximating the deriva-tives by using finite differences. Achieving the necessaryaccuracy for these approximations is a nontrivial en-deavor. Difficulties associated with rendering techniquesthat use ray-tracing techniques and that model imageblurring by convolution on a discrete grid necessitated de-velopment of a novel target/sensor rendering model.
The steps used in this model are as follows8,20:
1. The target is described as a composition of geometricprimitives (rectangle, circle, triangle, sphere, cone, etc.),with a material type specified for each.
2. The primitives are tessellated such that the pro-jected sizes of the constituent facets are at least 2–4 timessmaller than both the width of a FPA element and thewidth of the optical system’s PSF. This ensures that,from the standpoint of the sensor, the tessellated model ofthe object is indistinguishable from its true form.
3. The unobscured area of each facet is computed withrespect to the sensor and each light source.
4. The total radiance reflected by each facet in the di-rection of the sensor is calculated by using the informa-tion produced in step 3 and the BRDF of the target.
5. For each facet, the radiant flux incident at each pixelon a fine grid is calculated directly from an analytic ex-pression describing the system’s PSF, which takes as itsargument the relative separation between the pixel cen-ter and the projection of the facet’s center. The grid usedin this step is chosen to adequately sample the structureof the PSF and to be finer than the FPA grid.
6. The radiant flux at each pixel is summed over all fac-ets.
7. The high-resolution flux map produced by steps 5and 6 is downsampled to the FPA grid.
The direct calculation made in step 5 avoids spatialquantization issues associated with intermediate projec-tions onto grids and as a result is able to provide smoothand continuous changes in the image with respect to verysmall (,10212°) changes in the target’s orientation.8
Use of a small step size for finite-difference approxima-tion to first-order derivatives helps to ensure that the con-tribution of higher-order terms is negligible. Examplesof a target model, and the resulting images, are shown inFigs. 3–6 in Section 4.
4. INFLUENCE OF SECONDARYUNCERTAINTIES ON THE CRAMER–RAOLOWER BOUNDAs stated above, uncertainties in secondary sensor/targetmodel parameters can influence the CRLB for the princi-pal parameters of interest. The goal of this section is todetermine the extent to which these uncertainties influ-
Fig. 3. 3-D plot of the centers of the facets used to tessellate atarget’s exterior. The number of points in the model shown herehas been reduced for purpose of illustration.
Gerwe et al. Vol. 20, No. 5 /May 2003/J. Opt. Soc. Am. A 823
ence the CRLB and which specific parameters have thegreatest effect on increasing the CRLB.
For this study, we will consider the three Euler orien-tation angles of the satellite—roll, pitch, and yaw(uR , uP , uY)—to be the principal parameters that we areinterested in estimating. These parameters will alwaysbe included in the set j considered in the CRLB calcula-tion. In theory, there are an infinite number of possiblesecondary parameters that could influence the accuracywith which the principal parameters can be estimated.We have generated a partial list of 50 secondary param-eters present in the Bayesian model that were thought tohave a possibility of significantly influencing the CRLB.The entire list or any subset could be selected for inclu-sion in j when calculating the CRLB. This list repre-sents uncertainties in the sensor, such as the PSF size,the relative pointing offset of the FPA, and the instanta-neous field of view (IFOV). It also includes uncertaintiesin the target’s characteristics such as size, articulation,and reflectivity of its subcomponents (solar panels, aper-ture door, main body, scuff plates, radar dishes, and an-tennas). The sensor parameters used in the simulationsare listed in Table 1. These parameters are representa-tive of processed imagery from large-aperture ground-based systems (such as sensors on the large-aperture tele-scopes at the Maui Space Surveillance Site or the StarfireOptical Range in New Mexico), but without deconvolutionartifacts.1–4,21 The satellite target considered is theHubble Space Telescope as imaged under terminator illu-mination conditions. Also note that the bounds beingcalculated are on the satellite’s orientation relative to theFPA. The mount may be a limiting factor on determiningthe satellite’s absolute orientation if the accuracy withwhich it can be pointed is less than the calculated bounds.
Throughout this section, we will compare how treatingdifferent combinations of secondary parameter values ascertain or uncertain (i.e., by including them in j or v) in-fluences the CRLB for estimating orientation. We exam-ined the following definitions of j for several different ori-entations of the target model:
(A) Inclusive: j includes all 53 (three Eulerorientation150 secondary) sensor/target parameters.
(B) Orientation1Sensor: j includes target orienta-tion and the sensor parameters (target shape and reflec-tivity parameters are in v).
Table 1. Imaging Conditions Used for This Study
Target range 900 kmAperture diameter 1.57 mFPA IFOV 2.5 mradFPA size 64364PSF width 0.414 mradExposure Time 4 msWavelength 0.65 mmBandwidth 0.2 mmAnalog-to-digital gain 1CCD read noise 40e2 rmsTypical signal 48.7 3 106 photoelectrons total,
37.9 3 103 per pixel on targetBRDF Lambertian
(C) Selective: j includes all target orientation andsensor parameters along with a choice of one parameterfrom the list associated with target shape/reflectivity.Such a case was examined for all possible choices of thislast parameter.
These combinations allowed us to see the influence ofthe secondary parameters both individually and as awhole. The CRLBs on orientation calculated when theset of uncertainties j equals the inclusive set (A) andwhen j includes orientation and sensor parameters onlye.g., set (B), are tabulated in Tables 2, 3, and 4 corre-sponding, respectively, to the viewing perspectives shownin Figs. 4, 5, and 6. Examining the bounds correspond-ing to the various selective uncertainty sets (C) allowedthe particular target model uncertainties with the great-est impact on the CRLBs to be determined. These re-sults and their interpretation are now discussed.
For the orientation depicted in Fig. 4, by examining theresults corresponding to the different choices of the pa-rameter set j listed above, we determined that uncertain-ties regarding the solar panels and aperture door have astrong influence on determining the satellite’s pitch, i.e.,CRLB(uP). These uncertainties include the vertical loca-tion and length of the solar panels, as well as the orien-tation of the solar panels about the azimuth and line-of-sight axes and the aperture door length, width, andarticulation.
Uncertainty in solar panel articulation does not haveas great an effect on CRLB(uP) as we had expected. Thisis because we are looking at the panels almost straighton, and so a small change in solar panel articulation anglewill have a relatively small effect on the projected area.Since knowing solar panel articulation is not particularlybeneficial in determining the Euler angles, treating it asunknown does not significantly increase the CRLB. Thishas been confirmed by a test in which the model was
Fig. 4. Image at orientation A (uR 5 0, uP 5 0, uY 5 0).
Table 2. Effects of Choice of UncertaintiesIncluded in j for the CRLB Calculation at
Orientation A
CRLB(°) (A) Inclusive
(B) Orientation plusSensor Parameters
uR 0.0316 0.0227uP 0.1762 0.0140uY 0.0063 0.0039
824 J. Opt. Soc. Am. A/Vol. 20, No. 5 /May 2003 Gerwe et al.
placed in the same orientation (0, 0, 0) and the solar pan-els were articulated almost 90°, so they appeared to be al-most edge on. In this case, solar panel articulation influ-enced CRLB(uP) significantly; not knowing it increasedthe CRLB by a factor of 4.
For orientation B shown in Fig. 5, secondary uncertain-ties have the greatest effect on estimating roll (uR). Spe-cific parameters that influence CRLB(uR) are vertical lo-cation, length, and orientation about the azimuth axis of
Fig. 5. Image at orientation B (uR 5 0, uP 5 90, uY 5 0).
Fig. 6. Image at orientation C (uR 5 45, uP 5 45, uY 5 0).
Table 3. Effects of Choice of UncertaintiesIncluded in j for the CRLB Calculations at
Orientation B
CRLB(°) (A) Inclusive
(B) Orientation plusSensor Parameters
uR 0.2526 0.0066uP 0.0456 0.0046uY 0.0351 0.0070
Table 4. Effects of Choice of UncertaintiesIncluded in j for the CRLB Calculations at
Orientation C
CRLB(°) (A) Inclusive
(B) Orientation plusSensor Parameters
uR 0.0316 0.0227uP 0.1762 0.0140uY 0.0063 0.0039
the solar panels. Uncertainty in solar panel articulationhas a significant effect on CRLB(uP) in this orientation.This is because we are looking at the solar panels almostedge on, and a small change in articulation angle appearsto move the panels a significant amount. When the ar-ticulation is known, the solar panels contribute signifi-cantly to estimating pitch.
At orientation C, the CRLB for all three Euler angles isstrongly influenced by uncertainties in the secondary pa-rameters. The biggest contributing factors are solarpanel horizontal location (affecting the CRLB for uR , uP ,and uY), solar panel width (affecting the CRLB for uR),and solar panel orientation about the azimuth axis (alsoaffecting the CRLB for uR).
When the solar panel horizontal location is unknown,the increase in CRLB(uP) is much greater than we wouldintuitively expect. Further investigation indicated thatthis influence is present only when we assume that an-other secondary parameter, the FPA IFOV is also un-known. This example demonstrates that the joint effectof a combination of target and sensor uncertainties maybe greater than the sum of their individual effects. Forexample, uncertainties about parameter A or B may havelittle effect individually, but their combination maygreatly increase the CRLB. An intuitive way of lookingat this is that since parameter A is correlated with B, andB is correlated with C, not knowing C magnifies the effectof uncertainties in B on estimation of A. This effect isconfirmed for this orientation by looking at the Fisher in-formation matrix. The portion of the (normalized) Fishermatrix corresponding to this example is given in Table 5.It is seen that the influence of panel location, denoted byxs , on the image measurement is not strongly correlatedwith uR (roll). However, it is strongly correlated withIFOV, which is in turn strongly correlated with uR (roll);therefore uncertainties in xs make it harder to estimateIFOV, which in turn makes it harder to estimate uR .
In Fig. 7, the root-mean-square sum of the accuracybounds of the three Euler angles is plotted as a function ofthe number of secondary parameters included. It is ap-parent that there is a general cumulative effect alongwith a few large jumps corresponding to secondary pa-rameters, to which estimation accuracy is particularlysensitive. Note that because of the cross-correlation ef-fects just described, the magnitude of the CRLB increasethat occurs when a new parameter is added is somewhatdependent on which parameters have already been in-cluded. In other words, a different ordering of the pa-rameters would result in different CRLB jumps associ-ated with each parameter, although the cumulative resultof the full set would be the same.
We must comment that the orientation accuracies ob-tained are rather small. Root-mean-square sums on theorder of a few tenths of a degree might seem plausiblegiven fairly detailed knowledge of the target’s structureand excellent sensor calibration. However, in manycases, accuracy bounds of less than a hundredth of a de-gree were associated with some of the individual Eulerangles—these are undoubtedly overly optimistic. A real-istic number of degrees of freedom associated with thetypical uncertainties in the target/sensor system is prob-ably far greater than the 53 included in this study. For
Gerwe et al. Vol. 20, No. 5 /May 2003/J. Opt. Soc. Am. A 825
example, treating the target surfaces as Lambertian is anoverly simplistic model of the complexity of the real ma-terial BRDFs. We expect that as the number of free pa-rameters included in the model is increased, the accuracycurve in Fig. 7 will continue to grow and that the boundson each individual Euler angle will become more realistic.
The examples above demonstrate the importance of in-cluding secondary parameters in the CRLB analysis. Tocalculate realistic bounds, one must model all uncertain-ties in the target/sensor model that significantly influenceestimation of the parameters of principal interest. A de-tailed look at the bounds corresponding to different com-binations of uncertainties allowed us to identify uncer-tainties that stood out as having the strongest influence.We also saw that the remaining parameters, which indi-vidually increased the CRLB by only a few tenths of a per-cent, could as a whole increase the CRLB by a factor of 2or more because of their cumulative effect. This may befurther enhanced by the fact that the effect of one uncer-tainty on the CRLB can be magnified if its influence onthe image measurement is correlated with the influenceof combinations of other uncertainties. The importanceof modeling an inclusive set of target/sensor parametersis clear.
Fig. 7. Increase in the calculated CRLB as additional nuisanceparameters are included in the analysis (i.e., added to j). Theparameters associated with particularly large jumps are calledout. The satellite orientation is as in Fig. 4.
Table 5. Portion of the (Normalized) FisherInformation Matrix That Indicates Why
Uncertainties in the Solar Panel Position (xs)Strongly Influence Estimation of the Satellite’s
Roll Euler Angle uR Even Though the Correlationbetween the Influence of These Parameters on the
Image Measurement Is Weak
uR xs IFOV
uR 1 0.0536 0.4561xs
a 0.0536 1 20.5260IFOV 0.4561 20.5260 1
a Uncertainties in xs are able to influence uR because of their mutualcorrelations with the IFOV.
5. A PRIORI KNOWLEDGEThe values of secondary parameters such as the size of in-dividual components of the satellite or the pixel size ofour sensor will typically not be completely unknown andcan be specified to some precision before the image mea-surement. Such knowledge is referred to as a priori in-formation. To determine the effect of a priori knowledgeon the CRLB, we calculated the bound both with andwithout inclusion of a priori knowledge of various modeland sensor parameters. For a single parameter, theCRLB is equal to
CRLB 51
~Fdata 1 Fa priori!1/2 ,
where Fdata corresponds to information available from thedata measurement and Fa priori corresponds to a priori in-formation.
In the case where the data measurement is much moreaccurate than the a priori knowledge, or vice versa, theCRLB is determined by the more accurate of the two, e.g.
CRLB '1
@max~Fdata ,Fa priori!#1/2 .
However, if the Fisher information matrices from thedata measurements and the a priori knowledge areroughly equal, then
CRLB '1
~2Fdata!1/2 '1
~2Fa priori!1/2 .
If the a priori knowledge is more accurate than theCRLB, the bound is reduced. For example, the CRLB in-dicated that the aft shroud length of the Hubble SpaceTelescope could be estimated from the image measure-ment with an accuracy of 0.2% without a priori knowl-edge. Assuming that the target parameter was known towithin 10% a priori had little effect on the CRLB becausethis knowledge is less accurate than the knowledge pro-vided by the measurement. At the other extreme is theCRLB for determining the location of the antennas, whichhad a bound of 71.0% with the use of only measurement-based knowledge. However, if the location was known toan accuracy of 10% a priori, the CRLB reduces by 9.9%, orroughly equal to the a priori knowledge.
Including a priori knowledge of the secondary param-eters in j had a significant effect on the CRLB for estimat-ing the three Euler orientation angles of the HubbleSpace Telescope. When we assumed a conservative a pri-ori knowledge of 20% accuracy for the reflective model pa-rameters and 10% accuracy for the remaining model pa-rameters (lengths and distances), the CRLB for the threeEuler angles was reduced by 5%–20% (depending on ori-entation). Improving this a priori accuracy by a factor of10 further reduced the CRLB by as much as 40% belowthe bound as calculated by using just the informationavailable from the measurement. As improved a prioriinformation about the secondary target parameters re-duces our uncertainty in their values, the influence ofthese uncertainties on our ability to estimate satellite ori-entation decreases and the bound improves.
826 J. Opt. Soc. Am. A/Vol. 20, No. 5 /May 2003 Gerwe et al.
6. CONCLUSIONFrom the illustrative examples given in Sections 4 and 5,it is clear that solely basing the CRLB on a model thattreats only the orientation vector as being uncertain willgenerate bounds that are falsely optimistic. Further-more, it is expected that further additions to the list of 50secondary parameters that this study included in the listof uncertainties j treated in the CRLB calculations wouldcontinue to drive the bound upward. The accuracy withwhich secondary parameters are known a priori wasshown to be a determining factor in whether a particularuncertainty would have a significant effect on the bound.In general, these effects were intuitive. However, the de-tails of the off-diagonal Fisher information entries proveduseful in understanding complex interactions between pa-rameters that were not directly correlated with eachother. In addition to giving lower bounds, this analysismethodology provides considerable insight into how vari-ous object features influence the estimation accuracy.Such insights are likely to be useful in the design of sen-sor systems, improving how the measurements are made,and in improving estimation algorithms.
Realistically, for complicated problems such as this, aniterative approach of model development, algorithm im-provement, and system refinement will be needed to havecomplete confidence in the calculated bounds. Despitethe fact that further research is warranted, the authorsbelieve that the bounds are starting to approach realisticnumbers and provide some immediate insight, especiallyin terms of how the details of a target’s 3-D geometry andthe viewing perspective influence estimation accuracy.Such an understanding can play a strong role in planningdata collection and system operation and in providinghints for algorithm improvement.
ACKNOWLEDGMENTSThe authors acknowledge the reviewers for commentsthat led to several improvements of this paper. First, itwas pointed out that there are some fundamental limita-tions in using the classical CRLB approach to addresssystems that are described by parameter sets that existon a curved space as opposed to a flat Euclidean space.Second, in pointing out Refs. 9–11 and 15, of which wewere previously unaware, the reviewers enabled the in-troduction of this paper to provide better context to itscontributions relative to the wider body of literature. Fi-nally, the comments led to a clearer description of thenoise model and the choice of secondary parameter setsused in the CRLB calculations.
REFERENCES AND NOTES1. D. R. Gerwe, D. J. Lee, and J. D. Barchers, ‘‘Supersampling
2. J. H. Seldin, M. F. Reiley, R. G. Paxman, B. E. Stribling, B.L. Ellerbroek, and D. C. Johnston, ‘‘Space-object identifica-tion using phase-diverse speckle,’’ in Image Reconstruction
and Restoration II, T. J. Schulg, ed., Proc. SPIE 3170, 2–15(1997).
3. T. Schulz, J. Miller, and B. Stribling, ‘‘Multiframe blind de-convolution with real data: imagery of the Hubble SpaceTelescope,’’ Opt. Express 1, 355–362 (1997).
4. J. B. West and D. Utley, ‘‘Radiance map improvement inAEOS LWIR Images,’’ in Prceedings of the 2001 AMOSTechnical Conference, P. Kervin, L. Bragg, and S. Ryan, eds.(Maui Economic Development Board, Kihei, Maui, HI,2001), pp. 542–550.
5. X. Du, S. Ahalt, and B. Stribling, ‘‘Three-dimensional vectorestimation for subcomponents of space object imagery,’’ Opt.Eng. 37, 798–807 (1998).
6. J. Zhao, S. Ahalt, and C. B. Stribling, ‘‘3-D orientation vec-tor estimation from satellite imagery,’’ in Signal Processing,Sensor Fusion, and Target Recognition V, I. Kadar and V.Libby, eds., Proc. SPIE 2755, 472–483 (1996).
7. S. Kay, Fundamentals of Statistical Signal Processing: Es-timation Theory (Prentice-Hall, Englewood Cliffs, N.J.,1993).
8. D. R. Gerwe, P. S. Idell, and J. Vaughn, ‘‘Cramer–Raobound analysis of target characterization accuracy limitsfor imaging,’’ in Dual Use Technologies for Space Surveil-lance and Assessment II, P. Idell, ed., Proc. SPIE 4490, 245–255 (2001).
9. U. Grenander, M. I. Miller, and A. Srivastava, ‘‘Hilbert–Schmidt lower bounds for estimators on matrix Lie groupsfor ATR,’’ IEEE Trans. Pattern Anal. Mach. Intell. 20, 790–802 (1998).
10. M. I. Miller, U. Grenander, J. A. O’Sullivan, and D. L. Sny-der, ‘‘Automatic target recognition organized via jump-diffusion algorithms,’’ IEEE Trans. Image Process. 6, 157–174 (1997).
11. A. D. Lanterman, M. I. Miller, and D. L. Snyder, ‘‘GeneralMetropolis–Hastings jump diffusions for automatic targetrecognition in infrared scenes,’’ Opt. Eng. 36, 1123–1137(1997).
12. J. Kostakis, M. Cooper, T. J. Green, Jr., M. I. Miller, J. A.O’Sullivan, J. H. Shapiro, and D. L. Snyder, ‘‘Multispectralsensor fusion for ground-based target orientation estima-tion: FLIR, LADAR, HRR,’’ in Automatic Target Recogni-tion IX, F. A. Sadjadi, ed., Proc. SPIE 3718, 14–24 (1999).
13. U. Grenander, A. Srivastava, and M. I. Miller, ‘‘Asymptoticperformance analysis on Bayesian target recognition,’’IEEE Trans. Inf. Theory 46, 1658–1665 (2000).
14. M. L. Cooper and M. Miller, ‘‘Information measures for ob-ject recognition accommodating signature variability,’’IEEE Trans. Inf. Theory 46, 1896–1907 (2000).
15. H. Hendriks, ‘‘A Cramer–Rao type lower bound for estima-tors with values in a manifold,’’ J. Multivar. Anal. 38, 245–261 (1991).
16. D. L. Snyder, A. M. Hammoud, and R. L. White, ‘‘Image re-covery from data acquired with a charge-coupled-devicecamera,’’ J. Opt. Soc. Am. A 10, 1014–1023 (1993).
17. D. L. Snyder, C. W. Helstrom, A. D. Lanterman, M. Faisal,and R. L. White, ‘‘Compensation for readout noise in CCDimages,’’ J. Opt. Soc. Am. A 12, 272–283 (1995).
18. R. E. Blahut, Principles and Practice of Information Theory(Addison-Wesley, Reading, Mass., 1987).
19. A. Hero and J. A. Fessler, ‘‘A recursive algorithm for com-puting Cramer–Rao-type bounds on estimator covariance,’’IEEE Trans. Inf. Theory 40, 1205–1210 (1994).
20. Note that although newer versions of the rendering codesupport computation of the fractional unobscured area ofeach facet by comparing the mutual overlap of all facets,the version used for this paper effectively rounded this frac-tion to 0 or 1. Also, in the version of the code used for thispaper, only Lambertian BRDFs were supported.
21. D. Tyler, S. Ford, B. Hunt, M. Roggemann, T. Schulz, K.Schulze, J. Seldin, D. Sheppard, B. Stribling, W. van Ka-mpen, and B. Welsh, ‘‘Comparison of image reconstructiontechniques using adaptive optics instrumentation,’’ inAdaptive Optical System Technologies, D. Bonaccini and R.K. Tyson, eds., Proc. SPIE 3353, 160–170 (1998).
