Composite harmonic filters for scale-,projection-, and shift-invariant pattern recognition

David Mendlovic, Zeev Zalevsky, Irena Kiryuschev, and Guy Lebreton

310 APPLIED OPTICS @

The Mellin radial harmonic filter and the logarithmic harmonic filter are useful for performing opticalscale- and projection-invariant pattern recognition, respectively. To our knowledge, on the basis of theharmonic-function method, no one has been able to obtain more than one invariant property 1in additionto the shift invariance2 when using the matched-filter approach. A new method of combining the scale-,the projection-, and the shift-invariance properties is proposed, based on two decomposition stages of theinput pattern. Computer simulations are presented as well as preliminary experimental results.

1. Introduction

A convenient approach for achieving pattern recogni-tion is the 4 2 f optical correlator, as suggested byVanderLugt.1 This setup uses matched filters andthus has the highest signal-to-noise ratio response forwhite noise. Its disadvantage is that it provides noinvariant properties 1such as rotation, scale, or projec-tion2 except shift invariance. Adding such invari-ance is done in a relatively efficient way with har-monic expansions. In this method the object isdecomposed into a certain orthogonal harmonics ex-pansion, and the filter is chosen as a single order ofthe expansion. This approach results in a high signal-to-noise ratio because the harmonic contains a signifi-cant amount of the object’s energy. The suggesteddecompositions have been as follows: circular har-monics for rotation invariance,2 radial harmonics1RH2 for scale invariance,3 and logarithmic harmonics1LH2 for projection invariance.4 Later, a unificationof these harmonic decompositions and a generaliza-tion for other distortion properties were done with thedeformation harmonic functions.5Scale and projection-invariant pattern recognition

was achieved by coordinate transformations. Thetransformation u 5 ln x, v 5 ln y 1Ref. 62 was imple-mented by optics and resulted in these invariances.The disadvantage of such a method is that the

D. Mendlovic, Z. Zalevsky, and I. Kiryuschev are with theFaculty of Engineering, Tel Aviv University, Tel Aviv 69978,Israel. G. Lebreton is with the Universite de Toulon et du Var, LaGarde cedex 83957, France.Received 25 February 1994; revised manuscript received 15

August 1994.0003-6935@95@020310-07$06.00@0.

r 1995 Optical Society of America.

Vol. 34, No. 2 @ 10 January 1995

coordinate transformation replaces the shift-invari-ance property with the other two invariance proper-ties 1scale and projection2. Other methods that per-mitted the achievement of more than one invariantproperty were the synthetic discriminant function7method, which works with a data bank, or the defor-mation harmonic filter5. The major disadvantage ofsuch methods is that they require heavy calculations.In this paper we propose an algorithm that guaran-

tees more than one invariant parameter in addition tothe shift invariance and is based on the harmonicexpansions. We demonstrate the case of projection-,scale-, and shift-invariance properties simultaneously.Briefly the proposed algorithm is as follows: first, wedecompose the object f 1x, y2 into RH and then selectfrom this decomposition only the single harmonic thatis optimal according to the energy consideration.8The selected RH is decomposed in a second stage intoLH. Once again we choose only one LH out of thedecomposition. The order of the chosen harmonics isoptimal according to the correlation peak energy.9In both stages the decomposition order is chosen in away that the correlation peak has maximal energyand yet is not too wide.Section 2 recalls theMellin radial harmonics decom-

position. In Section 3 the logarithmic harmonicsfilter and its properties are discussed. Section 4explains and proves the suggested algorithm. InSections 5 and 6, computer simulations and experi-mental results are shown.

2. Radial Harmonic Filter

An object function f 1r, u2 can be expanded into a set oforthogonal functions 5ri2pN216, called Mellin radial

harmonics, as follows3:

f 1r, u; x0, y02 5 oN52`

N5`

fN1u; x0, y02r i2pN21, 112

fN1u; x0, y02 51

L er0

R

f 1r, u; x0, y02r2i2pN21rdr; 122

r0 is the object’s minimal radius, andR is the maximalradius of the object 3 f 1r, u2 5 0 for r . R4. L is aninteger number that satisfies the relation r0 5 Re2L.N is the expansion order, and 1x0, y02 is the Cartesian-coordinate origin of the 1r, u2 polar coordinates. Inpractice the value of r0 should not be too big becausethen areas of the object would not affect the harmonicfM1u2. The value should also not be too small becausethen the filter fM1u2r i2pM21 would have too-high spatialfrequencies that are difficult for holographic encoding.Thus the chosen value of r0 is optimal according to thecorrelation-peak form and the encoding of the filter.In the following, for the sake of simplicity the

coordinates of the origin 1x0, y02will be used only whennecessary. By choosing a filter as a single harmoniconly, g1r, u2 5 fM1u2r i2pM21, we obtain the scale-invariance property.

3. Logarithmic Harmonic Filter

A similar expansion for projection-invariant patternrecognition is the set of orthogonal functions5 0x 0 i2pN21@26, called logarithmic harmonics 1LH2, asfollows:

f 1x, y2 51

2L oN52`

N5`

fN1 y2 0x 0 i2pN21@2, 132

fN1 y2 5 e2X

2x0

f 1x, y212x22i2pN21@2dx

1 ex0

X

f 1x, y2x2i2pN21@2dx, 142

where x0 is the object’s minimal x coordinate and themaximal x coordinate is X 3 f 1x, y2 5 0 for x . X 4. Fora normalized axis, X , 1 and the object exists within21 , x , 1. We define an integer number L thatsatisfies the relation x0 5 exp12L2. N is the expan-sion order. Again the value of x0 is chosen not toosmall 1in order to avoid high spatial frequencies2 andnot too big 1so that all the pixels of the object will affectthe harmonic2. A filter with a single harmonic only,g1x, y2 5 fM1 y2 0x 0 i2pM21@2, provides projection-invariantpattern recognition.

4. Radial and Logarithmic Harmonic FilterDecomposition

The suggested filter is based on two harmonic decom-positions in cascade. First, a radial harmonic filter

out of the object f 1r, u2 is prepared according to Eqs. 112and 122:

fr1r, u2 5 fM11u2r i2pM121 152

fM11u2 5 e

r0

R

f 1r, u2r2i2pMi21rdr, 162

whereM1 is the RH expansion order. Then, a singleharmonic out of this decomposition is decomposedinto LH expansion with Eqs. 132 and 142. We denotethe LH harmonic asHM1 y2xi2pM21@2, whereHM1 y2 is

HM1 y2 5 eexp12L2

1

fr1r, u2x2i2pM21@2dx. 172

M is the LH expansion order. M1 and M are deter-mined from the maximal energy and the minimalcorrelation peak.8,9 In Eq. 172 we assumed that theobject exists only in the plane x . 0. The analysis forx , 0 is exactly the same; with the theorem ofsuperposition, the proof would be valid for all x.Let us now prove the scale and the projection

invariance of the theory given above. The correla-tion of the input pattern f 1x, y2 and the filterH*M1 y20x 0 i2pM21@2 is defined by

C1x8, y825 e2`

` e2`

`

f 1x, y; x8, y82H*M1 y2 0x 02i2pM21@2dxdy,

182

where

f 1x, y; x8, y82 5 f 1x 1 x8, y 1 y82. 192

From Eqs. 182 and 192 it is easily seen that thecorrelation is invariant to lateral shift, and shifting ofthe input pattern causes only a shift of the correlationpattern:

f 31x 1 a2 1 x8, 1 y 1 b2 1 y824

5 f 3x 1 1x8 1 a2, y 1 1 y8 1 b24. 1102

To prove the scale-invariance property, we use ascaled input:

g1x, y; x8, y82 5 f 1ax, ay; x8, y82, 1112

where a is a scale factor. Substituting g1x, y; x8, y82into Eq. 182, one gets

C1a21x8, y82 5 e2`

` e2`

`

g1x, y; x8, y82H*M1 y20x 02i2pM21@2dxdy.

1122

Thus for the scale-invariance property the followingrelation should be proven:

C1x8, y82 5 AC 1a21x8, y82, 1132

where A is a complex constant.

10 January 1995 @ Vol. 34, No. 2 @ APPLIED OPTICS 311

To insert the logarithmic harmonic function, weshould use Eqs. 132 and 142. We shall examine onlythe plane x . 0, assuming that the object exists onlyin x . 0. The analysis for x , 0 is exactly the same,and with the theorem of superposition, the proofwould be valid for any x. Thus for x . 0, Eqs. 132 and142 become

f 1x, y2 51

2L oN

fN1 y2xi2pN21@2, 1142

fN1 y2 5 eexp12L2

1

f 1x, y2x2i2pN21@2dx. 1152

The scaled object is

g1x, y2 5 f 1ax, ay2 51

2L oNgN1 y2xi2pN21@2, 1162

where

gN1 y2 5 eexp12L2@a

1@a

f 1ax, ay2x2i2pN21@2dx. 1172

The integral limits were changed because, if f 1x, y2were not zero in the range exp12L2 , x , 1, f 1ax, ay2would not be zero in the range exp12L2 , ax , 1.The range for x is exp12L2@a , x , 1@a for this reason.When we change the integration variables to z 5 ax,

gN1 y2 5 11a22i2pN11@2 e

exp12L2

1

f 1z, ay2z2i2pN21@2dz. 1182

After another change of variables to y8 5 ay,

gN1y8a 25 11a22i2pN11@2 e

exp12L2

1

f 1z, y82z2i2pN21@2dz. 1192

By substitution of y8 5 y and z 5 x, Eq. 1182 becomes

gN1ya2 5 11a22i2pN11@2 e

exp12L2

1

f 1x, y2x2i2pN21@2dx, 1202

gN1 y2 5 11@a22i2pN11@2fN1ay2. 1212

The correlation of Eq. 12 is now

C 1a21x8, y82 51

2L oN

e2`

` eexp12L2

1

ai2pN21@2fN1ay; x8, y82

3 xi2pN21@2H*M1 y2x2i2pM21@2dxdy

51

2L oN

e2`

` eexp12L2

1

ai2pN23@2fN1 y; x8, y82

3 xi2pN21@2x2i2pM21@2H*M1ya2dxdy, 1222

312 APPLIED OPTICS @ Vol. 34, No. 2 @ 10 January 1995

where fN1 y; x8, y82 is the fN1 y2 according to Eq. 1152when the object f is f 1x, y; x8, y82 5 f 1x 1 x8, y 1 y82;i.e., f 1x, y2 is shifted by 12x8, 2y82.With the orthogonality property of the LH 1Ref. 42,

eexp12L2

1

xi2pN21@2x2i2pM21@2dx 5 5fi0 for N 5 M

0 otherwise,

1232

the correlation expression of Eq. 22 is

C 1a21x8, y82 5 A e2`

`

fM1 y; x8, y82H*M1ya2dy. 1242

For a 5 1,

C1x8, y82 5 e2`

`

fM1 y; x8, y82H*M1 y2dy; 1252

thus the scale invariance will occur only if the follow-ing important condition is valid:

H*M1 y2 5 BH*

M1 y@a2, 1262

where B is a complex factor.Now let us prove the projection invariance 1scale

changes along one dimension2 for the suggested corre-lator. We define b as the scale factor of the x axis.Thus

C 1b21x8, y82 5 e2`

` e2`

`

f 1bx, y; x8, y82H*M1 y2x2i2p21@2dxdy.

1272

Naturally the scale change of b may be done in the yaxis as well, and then the proof would be exactly thesame. f 1bx, y2 is decomposed according to the follow-ing equations:

f 1bx, y2 51

2L oN

f N1b21 y2xi2pN21@2 1282

with

fN1b21 y2 5 eexp12L2@b

1@b

f 1bx, y2x2i2pN21@2dx. 1292

Changing the variables to z 5 bx, one gets

fN1b21 y2 5 11b22i2pN11@2 e

exp12L2

1

f 1z, y2z2i2pN21@2dz

5 11b22i2pN11@2

fN1 y2. 1302

Thus

f 1bx, y2 51

2L oN11b2

2i2pN11@2

fN1 y2xi2pN21@2. 1312

According to the correlation expression 3Eq. 12724,

C 1b21x8, y82 51

2L e2`

`

oN

bi2pN21@2fN1 y2H*M1 y2

3 eexp12L2

1

xi2pN21@2x2i2pM21@2dxdy. 1322

By using the orthogonality property, from the entiresum, we are now left with only the harmonic N 5 M.Thus

C 1b21x8, y82 5 11b22i2pM1 1@2e

2`

`

fM1 y2H*M1 y2dy. 1332

Because in a similar way it is obvious thatC1x8, y82 5 e

2`

` fM1 y2H*M1 y2dy, we conclude thatC 1b21x8, y82 5 bi2pM21@2C1x8, y82, and thus projection in-variance is achieved.So far we have demonstrated a correlator that is

shift and projection invariant. Regarding the scale-invariant property, one should show that Eq. 1262 isfulfilled. According to Eq. 172,

HM1 y2 5 eexp12L2

1

fr1r, u2x2i2pM21@2dx

5 eexp12L2

1

fr1x, y2x2i2pM21@2dx

5 e0

`

fr1x, y2x22pM21@2dx, 1342

where fr1x, y2 is zero beyond exp12L2 , x , 1.According to Eq. 152, fr1r, u2 5 fM1

1u2r i2pM121, one gets

fr1r@b, u2 5 fr1x@b, y@b2 5 fM11u211@b2i2pM121r i2pM121

5 11@b2i2pM121fr1x, y2; 1352

thus

HM1 y@b2 5 e0

`

fr1x, y@b2x2i2pM21@2dx

5 e0

`

fr1x@b, y@b21x@b22i2pM21@2dx

b. 1362

By use of Eq. 1352,

HM1 y@b2 5 K e0

`

fr1x, y2x2i2pM21@2dx

5 K eexp12L2

1

fr1x, y2x2i2pM21@2dx

5 KHM1 y2, 1372

with

K 5 1b22i2p1M12M223@2. 1382

The shift-invariant property is valid as well, and thesuggested filter provides scale, projection, and shiftinvariance.

5. Computer Simulations

The filter design method was applied to the F-18airplane 1Fig. 12. Its frame size is 128 3 128. Afterseveral computer simulations 1according to the algo-rithms presented in Refs. 8 and 92, it was decided thatthe optimal results were achieved when the decompo-sition order of the first stage 1the RH decomposition2was chosen as M1 5 0.315 and L 5 2 and when theorder of the second decomposition stage 1the LHdecomposition2was chosen asM 5 0.315 and L 5 3.The designed filter demonstrated good autocorrela-

tion peaks when the input object was scaled by factorvalues between 0.7 and 1.4. For the projection-invariance property, again good correlations wereobtained for the same range of projection factors.Figure 2 shows the input pattern of a combination offour F-18 airplanes being shifted, scaled, and pro-jected by different factors. The upper-left airplanehas both the scale and the projection factors equal to1 1the original airplane2. The upper-right airplanehas a scale factor 1 1not scaled2 and a projection factorof 0.7. The lower-left airplane has a scale factor of0.7 and a projection factor of 1. The lower-rightairplane has a scale factor of 1.4 and a projection

Fig. 1. F-18 airplane object for computer-simulation tests 1outlineversion2.

10 January 1995 @ Vol. 34, No. 2 @ APPLIED OPTICS 313

factor of 0.7. Figure 3 shows the correlation peakthat are obtained at the ouput. Four bright correla-tion peaks are clearly seen at the output, and thus theinvariance to scale, projection, and shift is proven.For cross correlation tests, Fig. 4 shows a differentinput pattern. In Fig. 5 one can see that no correla-tion peak and no recognition are obtained when thisincompatible input pattern is used. Thus the selec-tivity property of the filter is seen. Correlation calcu-lations were performed with Matlab mathematicalsoftware.

6. Experimental Results

The experimental performances of the suggested fil-ter were tested. Using the Dolev PS Scitex machine,we generated a binary hologram. The hologram has64 3 64 pixels 1we assumed that the results obtainedwere similar to those obtained for computer simula-tions with 128 3 128 pixels2, and the hologram usedvertical encoding.10 The mask was plotted directlyon Kodak Image-Lite scanner film with a size of 8 3 8mm.A conventional 4-f optical processor was con-

structed, and the composite filter 1based on the object

Fig. 2. Four shifted, scaled, and projected F-18 airplane inputpatterns.

Fig. 3. Correlation at the output plane for the input of Fig. 2.

314 APPLIED OPTICS @ Vol. 34, No. 2 @ 10 January 1995

of Fig. 12 was placed at its Fourier plane. The filterwas encoded as a Fourier-plane computer-generatedhologram with the detour phase method. At theoutput plane a CCD camera, connected with a micro-computer, was placed in order to grab the correlatoroutputs. In Figs. 6–10 the input patterns were theF-18 airplane, scaled and projected simultaneously,by the parameters s and p, as indicated in the figurecaptions. We denote the scale factor as s and theprojection factor as p. The pictures show the inten-sity at the output correlation plane. Figure 6 showsthe correlation plane for the input airplane pattern,which was also used for the calibration of the system.In Figs. 6–10 we can clearly see the bright and the

narrow autocorrelation peaks at the negative-firstdiffraction order. At the first diffraction order theconvolution pattern is observed 1much weaker andwider2, and in the zero diffraction order we seeundesired information.Figure 11 is the output plane when the input

pattern was the Tornado airplane, and as predictedfrom the theory, we did not obtain a correlation peak.Instead of one strong peak as achieved before, one cannotice at least three wide peaks. This input Tornadopattern had s 5 1 and p 5 1. For other values of sand p the output contains fewer peaks.

Fig. 4. Different input object for computer-simulation tests 1out-line version2.

Fig. 5. Correlation at the output plane for the input of Fig. 4.

Fig. 6. Experimental results for the correlation with the F-18input, scaled and projected by the factor values of s 5 1 andp 5 1. The correlation peak is obtained at the negative-firstdiffraction order, on the left side of the picture2.

Fig. 7. Same as Fig. 6 but with the factor values of s 5 0.6 andp 5 0.75.

Fig. 8. Same as Fig. 6 but with the factor values of s 5 1 andp 5 1.25.

Fig. 9. Same as Fig. 6 but with the factor values of s 5 0.8 andp 5 0.75.

Fig. 10. Same as Fig. 6 but with the factor values of s 5 1 andp 5 0.75.

7. Conclusions

In this paper we have discussed a two-stage decompo-sition method. We have proved, both theoreticallyand experimentally, that the method obtains scale-and projection-invariant pattern recognition. Thismethod is based on the following: first, the inputobject is decomposed into the RH set, and then fromthis decomposition a single harmonic is selected.In the second stage, the selected RH is decomposedinto the LH set. The final filter is a single harmonicout of this decomposition. The order of the selectedharmonic in each decomposition stage is determinedto give themaximal energy and the narrowest correla-tion peak. Because this design method is based onthe matched-filter approach, it has a good signal-to-noise ratio.It is important to note that, by using this method,

we cannot achieve any arbitrary invariant properties.For instance, we cannot apply our two stages ofdecomposition in the following way: first, to de-compose the object’s function into RH 1in order toachieve scale invariance2 and then to decompose thechosen harmonic into circular harmonies 1in orderto achieve rotation invariance as well2 because thesetwo harmonics are orthogonal to each other 1i.e.,e02p fN1u2r i2pN21exp12iMu2duwill be equal to r i2pM21 onlyfor N 5 M and zero elsewhere2, and the composedfilter will be without any spatial information.Computer simulations were done and proved the

ability of the method to recognize scaled, projected,and shifted patterns, although other combinations ofinvariant parameters are possible. Experimental re-sults confirmed the theory and demonstrated goodand narrow correlation peaks.

References1. A. VanderLugt, ‘‘Signal detection by complex spatial filtering,’’

IEEE Trans. Inf. Theory IT-10, 139–145 119642.2. Y. N. Hsu and H. H. Arsenault, ‘‘Optical pattern recognition

using the circular harmonic expansion,’’ Apppl. Opt. 21, 4016–4019 119822.

3. D. Mendlovic, E. Marom, and N. Konforti, ‘‘Shift and scaleinvariant pattern recognition using Mellen radial harmonics,’’Opt. Commun. 67, 172–176 119882.

4. D. Mendlovic, N. Konforti, and E. Marom, ‘‘Shift and projec-

Fig. 11. Same as Fig. 6 but for the Tornado airplane input andwith the factor values of s 5 1 and p 5 1.

10 January 1995 @ Vol. 34, No. 2 @ APPLIED OPTICS 315

tion invariant pattern recognition using logarithmic harmon-ics,’’ Appl. Opt. 29, 4784–4789 119902.

5. E. Marom, D. Mendlovic, and N. Konforti, ‘‘Generalizedspatial deformation harmonic filter for distortion invariantpattern recognition,’’ Opt. Commun. 78, 416–424 119902.

6. D. Mendlovic, N. Konforti, and E. Marom, ‘‘Scale and projec-tion invariant pattern recognition,’’ Appl. Opt. 28, 4982–4986119892.

7. D. Casasent and W. T. Chang, ‘‘Correlation synthetic discrimi-nant functions,’’ Appl. Opt. 25, 2343–2350 119862.

8. A. Moya, E. Tajahuerce, J. Garcia, D. Mendlovic, and C.

316 APPLIED OPTICS @ Vol. 34, No. 2 @ 10 January 1995

Ferreira, ‘‘Method for determining the proper expansion cen-ter and order for Mellin radial harmonic filters,’’ Opt. Com-mun. 103, 39–45 119932.

9. D.Mendlovic, Z. Zalevsky, J. Garcia, andC. Ferreira, ‘‘Logarith-mic harmonics proper expansion center and order for efficientprojection invariant pattern recognition,’’ Opt. Commun. 107,292–299 119942.

10. A. W. Lohmann and D. P. Paris, ‘‘Binary Fraunhofer holo-grams, generated by computer,’’ Appl. Opt. 6, 1739–1748119672.