Automated Target Recognition Using Mathematical Morphology

Prof. Robert HaralickIlknur IckeJosé Hanchi

Computer Science Dept.The Graduate Center of CUNY

Outline

• Gray Scale Morphology

• Converting Images to Datasets

• Decision Tree Classifier

• Results / Conclusions

Outline

• Gray Scale Morphology

• Converting Images to Datasets

• Decision Tree Classifier

• Results / Conclusions

Mathematical Morphology• Given an image II EN and a structuring element S S EN define the morphological operation of

Dilation

},,|{ sicSsIiEcSI N

},|{)( xicIiEcI Nx

and set translation as

xx SISI )()( Dilation is translation invariant

Mathematical Morphology

I

S

SI

Dilation

Mathematical Morphology• Define the morphological operation of

Erosion

},|{ IsxSsExSI N

Erosion is translation invariant

xx SISI )(

xx SISI )(

Mathematical Morphology

If a structuring element can be decomposed as

YXS

then

YXIYXISI )()(

YXIYXISI )()(

Basic Morphology Operators

Opening

Closing

SSISI )(

SSISI )(

SISSI )(

SISSI )(

Gray Scale Morphology

Dilation of f by k

)}()({max))(( zkzxfxkfFzx

Kz

Erosion of f by k

)}()({min))(( zkzxfxkfKz

Gray Scale Morphology

Opening of f by k

kkfkf )(

kkfkf )(

Closing of f by k

• We have used flat structuring elements

of size { 3,5,7,9,11,13,15,17,19,21 }

Structuring Elements Used

Hw = Horizontal

Vh =Vertical

Bwxh = Box

h = 5

w = 5

… an illustration

Dilation9HI

w = 9

Erosion9HI

w = 9

Opening9HI

w = 9

Closing9HI

w = 9

The van-Herk-Gil-Werman (HGW) Algorithm—dilationSTAGE 1

• Given the input signal stream and a flat structuring element of size = 3 x0 , x1, x2 , x3 , x4 , x5 , x6 , x7 , x8 , x9 , x10 , x11 , x12 , x13 ,…

9 4 5 14 21 18 12 7 8 3 …

• center segments located at x0 , x1, x2 , x3 , x4 , x5 x6 , x7 , x8 x9 , x10 , x11 … example:

… take the first segment and find the max (i.e. dilation)…

The van-Herk-Gil-Werman (HGW) Algorithm—dilation

STAGE 1.a

9 4 5 14 21

5

copy

9

max

R0

5

R1

5

R2

9 4

max

preprocess the prefixesx0 x4x2x1 x3

The van-Herk-Gil-Werman (HGW) Algorithm—dilation

STAGE 1.b

9 4 5 14 21

5

copy

21

max

14

max

preprocess the suffixes

S0

5

S1

14

S2

21

x0 x4x2x1 x3

The van-Herk-Gil-Werman (HGW) Algorithm—dilationSTAGE 2

9 4 5 14 21

R0

5

R1

5

R2

9

merging prefixes and suffixes

S0

5

S1

14

S2

21

max

99

max

1414

max

2121

p

p

p

pp 43)2()1(2

number of max operations per window:

x0 x4x2x1 x3

The van-Herk-Gil-Werman (HGW) Algorithm—dilation

Processing a given input signal for p=3 , segment size=5

x0 , x1, x2 , x3 , x4 , x5 , x6 , x7 , x8 , x9 , x10 , …

9 14 21 21 21 18 12 8 12 …

9 4 5 14 21 18 12 7 8 3 …12

Calculating Morphological Features in 2-D

The HGW algorithm works on 1-D inputTo apply it to 2-D images apply

– Horizontal Structuring Elements

process the image line by line

– Vertical Structuring Elements

transpose the image

process line by line

transpose again

– Box shaped Structuring Elements

horizontal first, then vertical

Efficiency of Flat Structuring Elements

Given the flat structuring elements H and V• Dilation• Erosion• Opening• Closing

Since and given w = h

• Dilation• Erosion• Opening• Closing

HI HI HI HI

VI VI VI VI

hwhw VHB

VHIBI )(VHIBI )(

VHVHIBBIBI )))((()(VHVHIBBIBI )))((()(

Ori

gina

l Im

age

Dilation with H5

ErosionWithH5

OpeningWithH5

Closing With H5

Using V5 Structuring Element

erosiondilation opening closing

Using B5x5 Structuring Element

dilation erosion opening closing

Outline

• Gray Scale Morphology

• Converting Images to Datasets

• Decision Tree Classifier

• Results / Conclusions

Using Morphological Operations As Features for a Pixel

ground truth image

II

(3 structural elements) x (10 sizes) x (4morphological operations) = 120 transformed images

…

5HI 5HI 5HI 5VI 5VI 5VI 55 BI 55 BI

…

55 BI55BI 5VI 5HI

Using Morphological Operations As Features for a Pixel

ground truth image

II

……

5HI 5HI 5HI 5HI 5VI 5VI 5VI 5VI 55 BI 55BI 55 BI55 BI

class label

ff1313 ff1414 ff1515 ff1616 ff1717 ff1818 ff1919 ff2020 ff2121 ff2222 ff2323 ff2424

{t,c}{t,c}={1,0}={1,0}

Given a pixel

Ground Truth Imageof mxnmxn pixelsII

(x(x1,f11,f1, x, x1,f21,f2, x, x1, f31, f3,... , x,... , x1, f1191, f119, x, x1,f1201,f120, , tt))

(x(x2,f12,f1, x, x2,f22,f2, x, x2, f32, f3,..., x,..., x2, f1192, f119, x, x2,f1202,f120, , tt))

(x(x3,f13,f1, x, x3,f23,f2, x, x3, f33, f3,..., x,..., x3, f1193, f119, x, x3,f1203,f120, , cc))

......

(x(xN-1,f1N-1,f1,x,xN-1,f2N-1,f2, x, xN-1, f3N-1, f3,..., x,..., xN-1, f119N-1, f119, x, xN-1,f120N-1,f120, , cc))

(x(xN,f1N,f1, x, xN,f2N,f2, x, xN, f3N, f3,..., x,..., xN, f119N, f119, x, xN,f120N,f120, , tt))

data setrepresentationof I I of sizeN = mxnN = mxn

DD

Morphological Features Data Set From an Image

5HI 5HI 5HI 5HI 5VI 5VI 5VI 5VI 55 BI 55BI 55 BI55 BI

Preparation Of Data Sets to Train and Test the Classifier

DD11

DD22

……

DDkk

DDk+1k+1

……DDKK

Createdatasets

separatevectors

II11

II22

……

IIkk

IIk+1k+1

……IIKK

ground truth images

targetdataset

clutterdataset

trainingdataset

testdataset

Outline

• Gray Scale Morphology

• Converting Images to Datasets

• Decision Tree Classifier

• Results / Conclusions

Creating a Decision Tree Classifier

classify classified classified datasetdataset

evaluate accuracyaccuracy

create decision

treedecision treedecision tree

trainingdataset

parametersparameters

testdataset

decision treedecision tree

testdataset

classified classified datasetdataset

Creating a Decision Tree Classifier

11

1

1

1

1 1

11

0 0

0

0

00

11

22

f1f1

f2f2DDtrainingtraining

f1f1 > 11

true

f2f2 > 22

class 0

true

00

00

0

00

033

44

f2f2 > 33

class 1

true

class 0f1f1 > 44

class 1

true

class 0

• threshold decision rule

• max.entropy = 0.001

• max. depth = 20

Outline

• Gray Scale Morphology

• Converting Images to Datasets

• Decision Tree Classifier

• Results / Conclusions

Decision Tree Classifier resultsfor test dataset derived from

images of resolution = 75mm

Clutter Target

Clutter 1,712,090 15,422

Target 1,419 54,503

train dataset size = 292,831 vectors

test dataset size = 1,783,434 vectors

true

cla

ss

assigned class

accuracy (% correct classification) = 99.046%

Decision Tree Classifier resultsfor images of resolution = 75mm

• 345 images of clutter-only

• 44 images with mostly target

accuracy # clutter images # target images

[0.91 , 0.92) 4 -[0.92 , 0.93) 3 -[0.93 , 0.94) 4 -[0.94 , 0.95) 12 -[0.95 , 0.96) 14 -[0.96 , 0.97) 18 -[0.97 , 0.98) 31 5[0.98 , 0.99) 93 21[0.99 , 1.00) 166 18

Total # images 345 44

Decision Tree Classifier resultsfor test dataset derived from

images of resolution = 200mm

Clutter Target

Clutter 231,495 1,702

Target 254 8,391

train dataset size = 64,127 vectors

test dataset size = 241,842 vectorstr

ue c

lass

assigned class

accuracy (% correct classification) = 99.19%

Decision Tree Classifier forimages with resolution = 200mm

• 689 images with mostly clutter

• 34 images with mostly target

accuracy # clutter images # target images

< 0.90 4 -[0.90 , 0.92) 1 -[0.92 , 0.94) 1 -[0.94 , 0.95) 2 1[0.95 , 0.96) 3 -[0.96 , 0.97) 8 1[0.97 , 0.98) 12 1[0.98 , 0.99) 76 10[0.99 , 1.00] 582 21

Total # images 689 34

<end/>