Automatic Target Recognition Automatic Target Recognition Demonstration for CST ReviewDemonstration for CST Review

Professor Joseph A. OProfessor Joseph A. O’’SullivanSullivanLee Lee Montgnino Montgnino

Center for Security Technologies, Washington Universityjao@ee.wustl.eduhttp://essrl.wustl.edu/~jao

Supported by: ONR, ARO, DARPAONR, ARO, DARPA

• Object Recognition and the Role of Templates• Our Methodology Based on Likelihoods• Comparative Results: MSTAR website• Open Problems:

– Fundamental Performance Bounds– Extensions to Optical Imagery

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

2

CollaboratorsCollaborators

Michael D. DeVoreNatalia A. SchmidLee MontagninoSushil AnandAndrew LiVikas Kedia

Donald L. SnyderDaniel R. FuhrmannMichael I. MillerJeffrey H. Shapiro

Faculty Students and Post-Docs

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

3

MotivationMotivation• Many reported approaches to ATR from SAR• Performance and database complexity are interrelated• We seek to provide a framework for comparison that:

- Allows direct comparison under identical conditions- Removes dependency on implementation details

2S12S1 T62T62 BTR 60BTR 60 D7D7 ZIL 131ZIL 131 ZSU 23/4ZSU 23/4

Publicly available SAR data from the MSTAR program

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

4

LEE THINKS: More MotivationLEE THINKS: More Motivation• Say we have MSTAR since it is readily available to all

•Experiments are performed in a controlled manner•Data is well understood, etc.

• Extensions into Optical imaging• Direct Link (?) to Airport Security

•The Scanners implemented in SEATAC• Other Modalities

Publicly available SAR data from the MSTAR program

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

5

Target Orientation EstimationTarget Orientation Estimation

Likelihood Model Approach:Likelihood Model Approach:ppRR||ΘΘ,,aa((rr||θθ,,aa) ) -- Conditional Data ModelConditional Data ModelppΘΘ,,aa((θθ,,aa) ) -- Prior on orientation (known or simply uniform)Prior on orientation (known or simply uniform)P(P(aa) ) -- Prior on target class (known or simply uniform)Prior on target class (known or simply uniform)

Target Target ClassifierClassifier ââ=T72=T72

Orientation Orientation EstimatorEstimator θθ=135=135

aa=T72=T72

Given a SAR image Given a SAR image rr, , determine a corresponding determine a corresponding target class target class ââ∈∈AA

Given a SAR image Given a SAR image rr and and a target class a target class ââ∈∈AA, , estimate target orientationestimate target orientation

Target Classification ProblemTarget Classification Problem

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

6

Training and Testing Problem:Training and Testing Problem:Function Estimation and ClassificationFunction Estimation and Classification

FunctionEstimation

L(r|a,θ) Inferenceââ=T72=T72

Scene and SensorPhysics

Training Data

Raw DataProcessing

Image

• Labeled training data: target type and pose• Log-likelihood parameterized by a function:

mean image, variance image, etc.

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

7

Function Estimation and ClassificationFunction Estimation and Classification

•• Functions are estimated fromFunctions are estimated from-- sample data sets onlysample data sets only-- physical model datasets only (PRISM, XPATCH, etc.)physical model datasets only (PRISM, XPATCH, etc.)-- combination of thesecombination of these

•• Training sets are finiteTraining sets are finite•• Computational and likelihood models have a finite Computational and likelihood models have a finite

number of parametersnumber of parameters•• Estimation errorEstimation error•• Approximation errorApproximation error•• Some regularization is neededSome regularization is needed

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

8

ApproachApproach• Select 240 combinations of implementation parameters• Execute algorithms at each parameterization• Scatter plot the performance-complexity pairs• Determine the best achievable performance at any complexity

BMP2 Variance Image at 6 Sizes

ZIL131 Variance Image at 6 Sizes

Six different image sizes from 128x128 to 48x48

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

9

System Parameters and ComplexitySystem Parameters and ComplexityApproximate Approximate αα((θθ,,aa) and ) and σσ22((θθ,,aa) as piecewise constant in ) as piecewise constant in θθImplementations parameterized by:Implementations parameterized by:

ww -- number of constant intervals in number of constant intervals in θθdd -- width of training intervals in width of training intervals in θθNN22 -- number of pixels in an imagenumber of pixels in an image

Database complexity ≡ log10(# floating point values / target type)

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

10

Performance and ComplexityPerformance and ComplexityForty combinations of angular resolution and training interval width.

Variance image of aT62 tank1 Window trained over 360°

Variance images of a T72 tank72 Windows trained over 10°

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

11

Problem StatementProblem Statement• Directly compare conditionally Gaussian, log-magnitude MSE,

and quarter power MSE ATR Algorithms- identical training and testing data- identical spatial and orientation windows

• Plot performance vs. complexity- probability of classification error- orientation estimation error- log-database size as complexity

• Use 10 class MSTAR SAR images

ApproachApproach• Select 240 combinations of implementation parameters• Execute algorithms at each parameterization• Scatter plot the performance-complexity pairs• Determine the best achievable performance at any complexity

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

12

Approaches: Conditionally Approaches: Conditionally GaussianGaussian

J. A. OJ. A. O’’Sullivan and S. Jacobs, IEEESullivan and S. Jacobs, IEEE--AES 2000AES 2000

Model each pixel as complex Model each pixel as complex GaussianGaussian plus uncorrelated noise:plus uncorrelated noise:

( ) ( )( )( )∏ +

−

Θ +=

i

NaKr

iA

i

i

eNaK

ap 0

2

,

0, ,

1, θ

θπθrR

ˆ a Bayes r( ) = argmaxa

maxk

ˆ p rθk ,a( )ˆ θ HS r,a( ) = argmax

θ k

ˆ p rθk ,a( )

GLRT Classification and MAP Estimation:GLRT Classification and MAP Estimation:

J. A. OJ. A. O’’Sullivan, M. D. Sullivan, M. D. DeVoreDeVore, V. , V. KediaKedia, and M. Miller, IEEE, and M. Miller, IEEE--AES to appear 2000AES to appear 2000

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

13

Approaches: LogApproaches: Log--MagnitudeMagnitudeMinimize distance between rdB = 20 log |r| and dB templates

d2 rdB,µLM( )= rdB − µLM2

Make decisions according to:

ˆ a LM r( )= argmina

mink

d2 rdB,µLM θk, a( )( )ˆ θ LM r a( )= argmin

θk

d2 rdB,µLM θk ,a( )( )

Alternatively, use a form of normalization:

d2 rdB − rdB,µLM θk,a( )− µLM θk ,a( )( )G. Owirka and L. Novak, SPIE 2230, 1994

L. Novak, et al., IEEE-AES, Jan. 1999

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

14

Approaches: Quarter PowerApproaches: Quarter Power

d2 rQP,µQP( )= rQP − µQP

2

Minimize distance between rQP = |r|1/2 and quarter power templates

Make decisions according to:

ˆ a QP r( ) = argmina

mink

d2 rQP,µQP θk, a( )( )ˆ θ QP r,a( ) = argmin

kd2 rQP,µQP θk,a( )( )

d2 rQP

rQP,

µQP θk ,a( )µQP θk ,a( )

⎛ ⎝ ⎜ ⎞

⎠

Or, normalized by vector magnitude:

S. W. Worrell, et al., SPIE 3070, 1997Discussions with M. Bryant of Wright Laboratory

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

15

PerformancePerformance--Complexity LegendComplexity Legend

Forty combinations of number of piecewise constant intervals and training window width

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

16

Conditionally Conditionally GaussianGaussian ResultsResults

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

17

Normalized Conditionally Normalized Conditionally Gaussian Gaussian ResultsResults

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

18

LogLog--Magnitude ResultsMagnitude Results

Recognition without normalization Arithmetic mean normalized

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

19

Quarter Power ResultsQuarter Power Results

Recognition without normalization Recognition with normalization

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

20

SideSide--byby--Side ResultsSide ResultsComparison in terms of:• Performance achievable at a given complexity• Complexity required to achieve a given performance

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

21

2S1 BMP 2 BRDM 2 BTR 60 BTR 70 D7 T62 T 72 ZIL131 ZSU 23 42S1 262 0 0 0 0 0 4 8 0 0 95.62%BMP 2 0 581 0 0 0 0 0 6 0 0 98.98%BRDM 2 5 3 227 1 0 14 3 5 4 1 86.31%BTR 60 1 0 0 193 0 0 0 0 0 1 98.97%BTR 70 4 5 0 0 184 0 0 3 0 0 93.88%D7 2 0 0 0 0 271 1 0 0 0 98.91%T 62 1 0 0 0 0 0 259 11 2 0 94.87%T 72 0 0 0 0 0 0 0 582 0 0 100%ZIL131 0 0 0 0 0 0 2 0 272 0 99.27%ZSU 23 4 0 0 0 0 0 2 0 1 0 271 98.91%

•• Probability of correctProbability of correctclassification: 97.2%classification: 97.2%

Target ClassificationTarget ClassificationResultsResults

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

22

• General problem in training/testing posed as estimation/classification

• Method of sieves (polynomial splines chosen)

• Comprehensive performance-complexity study for

- Ten class MSTAR problem

- Conditionally Gaussian model

- Log-magnitude MSE

- Quarter power MSE

• Provided a framework for direct comparison of alternatives and selection of implementation parameters

• Analysis ongoing

ConclusionsConclusions

J. A. O’Sullivan. CST Review, 01/13/2003ATR Demo

23

• Extensions into other domains….

•Optical, etc.

LEE THINKS: ExtensionsLEE THINKS: Extensions