EXTRACTING NATURAL SCALES USING FOURIER DESCRIPTORS
PAUL ROSIN and SVETHA VENKATESH Cognitive Systems Group, School of Computing Science, Curtin University of Technology, Perth, 6001,
Western Australia
(Received 24 April 1992; in revised form 3 November 1992; received for publication 12 February 1993)
A b s t r a c t - Since contours often contain many different sized structures they need to be described at multiple scales. Rather than describe a contour at all scales it is more efficient to identify the most significant scales that best represent the structures present. These natural scales are determined by examining the Fourier descriptors of the contour and searching for significant bands. These correspond to bandwidths required to extract features at qualitatively significant resolution. Examples are given of using the determined scales in both low pass and high pass filters to eliminate fine and coarse detail.
Recognition of shape is a fundamental concern in com- puter vision. Most shape representations encode either the form of a curve "-3) or the surface t4) in a way that captures some intrinsic attribute. In the recognition of shape, as in other computer vision tasks, the visual events perceived and found meaningful vary in size and extent. 1~'6) Attempting to analyse an object at the incorrect scale leads to difficulties. At too fine a scale the features of interest may be swamped by irrelevant detail and noise. At the other extreme, too coarse a scale will distort or obliterate the feature. The aim of standard methods of segmentation is to represent the data at some appropriate level. However, the shape of many objects consists of several superimposed struc- tures of different sizes. For example, consider the profile of a face. At the largest scale it appears as an ovoid. Finer scales show facial features such as the noise, eye socket, and brow. At even finer detail, wrinkles, the eye brows and lashes, etc. become apparent. This implies that a representation at a single scale is inadequate.
There has been a great deal of work on multi-scale descriptions, t7-91 All these methods produce a large number of descriptors at different scales. However, traditional multi-scale methods have several defici- encies. As Witkin ~5) observed, "merely computing des- criptions at multiple scales does not solve the problem; if anything, it exacerbates it by increasing the volume of data". One problem is that the multiple descriptions contain much redundant information. There is little qualitative change between most adjacent scales. In addition, the important scales, i.e. those that best des- cribe the various features--are not explicitly identified and differentiated from all the other (redundant) scales. In this paper we propose a multi-scale representation which eliminates this redundancy by identifying the natural scales of a curve. Those are the scales which correspond to some qualitative good description of the c u r v e .
Our technique involves determining the Fourier descriptors of a curve. We propose that structures in the curve produce frequency components which pro- duce significant structures in the Fourier transform. The sub-parts of each important structure give rise to similar frequency components which cluster to form bands in the Fourier transform. Locating these bands allows the natural scales that best describe the signi- ficant features in the curve to be determined.
The layout of this paper is as follows: Section 2 describes the techniques used, Section 3 contains the experimental results, and the discussion follows in Sec- tion 4.
2. PROPOSED TECHNIQUES
Although several techniques have been proposed for the multi-scale representation of images and curves, little effort has been directed towards the extraction of only the significant scales that are required to ade- quately describe objects that are composed of several superimposed structures at different scales. One meth- od of extracting significant scales was proposed by Rosin. I1°) A given curve is first smoothed at multiple scales. At each scale the curve is assigned a signifi- cance value determined by the number of zero-crossings of curvature of the curve normalized by the smoothing parameter a. Counting zero-crossings is akin to meas- uring the wiggliness of the curve. Curves with minimal significance values with respect to curves at adjacent scales have a high rate of change in the numbers of the zeros of curvature. This corresponds to curves with locally high levels of structural information. Rather than indirectly analysing the structure of curves by their zero-crossings of curvature we propose in this paper to use the Fourier transform to extract the natu- ral scales. The advantages of the Fourier transform are that it provides direct access to frequencies, and it is well understood since a great deal of work has been spent analysing its properties.
1383
1384 P. ROSIN and S. VENKATESH
There are two main techniques for generating Fourier descriptors. In the first method I~ ~ the parametric rep- resentation of a curve Z( l )= (x(l),y(l)) is used, where I is the arc length and varies from 0 to L, and the curve has a period of L. If the angular direction of the curve at a point l is given as O(l), the cumulative angular function ~b(l) is defined as the total angular bend be- tween the starting point 0, and the point I. Then q~*(l) can be defined as
~*(i) = \ ~ / + t (1)
where t is in the range from 0 to 2n. Then ~,b*(t) can be expanded as
q~*(t) = A + ~ ancos(kt - a.) (2)
where a n and an are the Fourier descriptors for the curve.
An alternate method I~ 2~ of deriving the Fourier de- scriptors also considers the parametric representation of a curve Z(l) = ix(I), y(l)). A point along the boundary is considered to generate a complex function u(l) = x(l) + iy(l). Thus the Fourier descriptors can be defined as
1 i u( l) e-II2~/L)ntdl (3)
an = Loo o
and
u(l) = ~ an e it2~/Llnt. (4)
We use the latter parameterization because it is di- rectly based on the coordinate values of points on the curve rather than cumulative or absolute angular differences. 111'13) Angular differences are difficult to compute accurately/~a) Also, Gralund's parameteriz- ation requires no integration for reconstruction (which leads to increased computation), and closed curves re- main closed on reconstruction (in contrast to Zahn and Roskie's parameterization).
Each of the Fourier transform components can be considered to be a shape descriptor. Thus the Fourier descriptor of a circle has only two components, where the zeroth harmonic equals the coordinate at the centre,
bands (see Fig. 1) which we propose are formed by the clustering of similar frequency components produced by similar sub-parts of each important structure. Each band can be considered as a hill on the one-dimensional signal, bounded by two valleys and containing a peak. We propose that significant spectral bands or hills identify passbands of bandpass filters which when ap- plied to the curve will yield structures at significant scales. Since we require the upper end of the band as the cut-off frequency we locate them by the minima of significant valleys rather than the one sided ends of hills. These natural scales when ordered by significance may not produce curves that are ordered in scale. The problem, therefore, is to first devise reliable means of extracting the significant bands, and to then verify the premise that bandpass filters of these bandwidths when applied to the curve do indeed extract structures at qualitatively significant scales.
2.1. E.,ctracting extrema of the FFTmagnitude
Valleys are defined by two maxima and one minima in the magnitude of the discrete Fourier transform. However, locating valleys is complicated since the sig- nal contains many spurious maxima and minima. This is caused in part by signal noise. Also, real objects contain a variety of structures at different scales whose combination blurs and confuses the incidence of ex- trema. To minimize the problems arising from spurious extrema the magnitude of the Fourier transform is smoothed at multiple scales, and extrema are tracked over scale in a procedure similar to Witkin's scale- space analysis of zero-crossings. Extrema arising from noise are quickly annihilated with increasing smoothing whereas the most significant extrema persist much longer. The procedure starts by smoothing the magni- tude of the Fourier transform by Gaussians starting at a = 0.1 and continuing smoothing at octave separated scales until only one extremum is left. The extrema are determined in each smoothed transform, and are then linked to matching extrema in the adjacent higher scale. The linking rules are briefly described below:
For each extremum Ea~ in signal smoothed with Gaussian a 1 Compare with all extrema at adjacent scale a2 (where a2 = x/2a~) Find the extremum Ea2 with minimum distance d ...... t from Eal If Ea 2 is already linked to some extrema at al (with distance dp~evious) then
If d . . . . . . . • dprevious then Unlink Ea 2 from previous extrema at scale a t Link Ea 2 with Ea 2 and record link distance as deurren t.
and the first harmonic is real and equal to the radius. Gralund t~2~ and Persoon and Fu 13) were amongst the earlier authors who proposed the use of these descrip- tors for character recognition and machine part rec- ognition.
To determine natural scales the magnitude of the Fourier descriptors is analysed. This function contains
Each extremum at one scale is only linked to a single extremum at the lower adjacent scale. If multiple links were allowed the scale-space would show better the re- lationship between extrema at different scales. That is, several extrema merging together to form a single extre- mum shows that a single extrema at one scale is made up from several extrema at lower scales. Witkin's scale-
Extracting natural scales using Fourier descriptors 1385
Fig. 1. Magnitude of the Fourier transform vs. frequency.
space and interval tree provides this sort of information. However, for our purposes a set of unbranched linked extrema provides a simpler representation that can adequately capture the lifetime of individual extrema.
The Gaussian filter is used to smooth the signal since it has been shown to be the only filter that does not produce new zero-crossings in the derivatives of in- creasingly smoothed profiles, tlS) This is crucial because smoothing is being applied to eliminate fine detail in order to better reveal the coarse structure. The intro- duction of new structures at intermediate levels of
PR 26:9-E
| u
i U l l W g n I H m H I l l l ' l l - - - - " l U l l
I n n u i l l n - - l i l g . . . . I l i . . . . .
g f l l l l l l I N I l I l I l U I InIInUUlll R~ I INI I INNI IN I IIIIn
Fig. 2. Magnitude of the Fourier t ransform and tracked extrema.
1386 P. RO~IN and S. VENKATESH
V c
V a
|
b a
Fig. 3.
Fig. 4.
smoothing would invalidate this process of going from fine to coarse structures.
Part of the magnitude of the Fourier transform for a curve is shown with the set of smoothed magnitudes superimposed in Fig. 2. The tracked extrema using the above linking rules are shown below.
2.2. D e t e r m i n i n 9 s ign i f i can t va l leys
Having located the extrema over different scales the most significant valleys must be identified. This will enable us to extract the frequencies that define the passbands of only the most important scales. The scale- space analysis is necessary so that small valleys in the transform, resulting from image noise or discrete trans- form inaccuracies can be eliminated. At each scale extrema are used to generate valleys. Valleys are then linked over scales. The significance measure of valleys is quantified by combining two aspects: (1) the size of the valley at an individual scale, and (2) the lifetime of the valley through scale-space.
As in Rosin, (1°) we need to first quantify the signifi- cance of the valleys in the magnitude of the transform at each level of smoothing. We consider the area of a valley as an indication of its degree of significance. Several researchers have proposed methods to quantify the size of valleys (e.g. Ehrich and Foith (16)). The most
obvious method is to measure the area between two consecutive maxima. However, to better account for uneven valleys which have one large neighbouring maximum and one small maxima we compute the area of the valley V~ at the smoothing level tr in Fig. 3 as
V , = (va - vb)*(a -- b) (5)
Fig. 5(a). Original curve.
Fig. 5(b). Extracted curve at first significant scale.
Extracting natural scales using Fourier descriptors 1387
Fig. 5(c). Extracted curve at second significant scale. Fig. 5(d). Extracted curve at third significant scale.
Fig. 6(a). Extracted curve at first significant scale. Fig. 6(b). Extracted curve at second significant scale.
388 P. Rosin and S. VENKATESH
Fig. 6(c). Extracted curve at third significant scale.
Fig. 7(a). Original curve.
Fig. 7(b). Extracted curve at first significant scale.
Fig. 7(c). Extracted curve at second significant scale.
where va is the value of the smaller neighbouring max- imum.
Thus, in this measure valleys with large peaks and wide base-values will have higher values than both
Extracting natural scales using Fourier descriptors 1389
Fig. 7(d). Extracted curve at third significant scale.
valleys with large peaks and small base-values (such as caused by noise) or low peaks and low base-values (such as will exist in the high frequency region). A one sided valley measure is required to prevent small dips on the side of large slopes being incorrectly estimated as large (and significant). This is determined in Fig. 4, in which the left-hand side of the valley would have a very large area while the right-hand side is small. Com- bining both parts would result in a moderately large average, whereas taking the smaller part correctly assigns the valley a small value reflecting its insigni- ficance.
Since extrema were linked across scales, the valleys (which are defined in terms of extrema) are also linked. This allows valleys to be tracked across scale and their sizes to be calculated in spite of the obscuring noise and fine detail at low levels of smoothing. Two alter- native significance measures are defined:
L
• Significance S = ~ V~ (6) a = O . l
where L is the largest a for which V exists according to the linking rules specified in Section 2.1. This meas- ure incorporates the lifetime of the valley, assuming that the significance of a valley is proportional to its persistence in scale-space.
L
• Significance M = max V,. (7) a=O.1
This measure allows the size of the valley to be measured even if the valley is fragmented into smaller sub-valleys at fine scales.
3. E X T R A C T I N G N A T U R A L SCALES
3.1. Law pass filtering
We show the above techniques on three sets of closed contours (Figs 5-7). For each contour, the positions of the three most significant minima were extracted. Each of these minima identify the upper bandwidth of a low pass filter. Low pass filtering removes detail finer than the structures of interest at the natural scale. The struc- tures of interest are superimposed on the coarser struc- tures which remain. The contour is filtered with low pass filters of the extracted bandwidths and the resultant contours obtained are shown in Figs 5-7. In each case the resultant contours show how different levels of structure are extracted, and are shown in decreasing order of significance.
Table 1 gives the three most significant minima extracted by considering the two measures of signifi- cance that have been defined. Although the order varies slightly the minima that are identified as significant are essentially the same.
For the cartoon figure in Fig. 5(a), Fig. 5(b) shows an ovoid giving the general size and orientation. Figure 5(c) delineates the head, arms and legs while removing the fine detail. Figure 5(d) separates the figure into the body, head, and arms.
Figure 6(a) shows a structure composed of an un- dulating curve with superimposed undulations. Again the most significant scale (Fig. 6(a)) shows an ovoid, while Fig. 6(b) recovers the largest two undulations and Fig. 6(c) captures the four main undulations.
Similar results are demonstrated for the synthetic figure shown in Fig. 7(a). At the most significant scale the figure is represented by an ovoid (Fig. 7(b)), followed by the basic dumbell structure (Fig. 7(c)). The third significant scale describes the finer protrusions as well (Fig. 7(d)).
Different methods have been suggested for handling open curves. When dealing with open curves in the spatial domain, the curve is usually extended by dupli- cating and reflection about the perpendicular to the tangents at the ends of the curve. However, it is often difficult to obtain a reliable estimate of the tangent. Standard Fourier techniques are restricted to closed curves, but Weyland and Prasad have indicated meth- ods by which this can be extended to open curves, t~ a~ We use a simpler technique suggested by Persoon and Fu. TM When tracing along an open curve we double
Table 1
Significance Position of Figure measure natural scales
5 M 2,12,7 S 2,4,12
6 M 2,7,13 S 2,6,17
7 M 2,7,28 S 2,8,4
1390 P. ROSIN and S. VENKATESH
(a) (b) (c) (d)
(
Fig. 8. (a) Original curve; (b)-(e) extracted curve at increasing significant scales.
(e)
S
! 1 Fig. 9(a). High passed version of the contour superimposed Fig. 9(c). High passed version of the contour superimposed
on a circle is shown with the original curve of H. on a circle is shown with the original curve of H.
Fig. 9(b). High passed version of the contour superimposed Fig. 9(d). High passed version of the contour superimposed on a circle is shown with the original curve of H. on a circle is shown with the original curve of H.
back from the end to the start, thereby obta in ing a closed contour . An example of such an open curve is shown in Fig. 8(a), and the results of extract ing signifi- cant scales are shown in Figs 8(b)-(e). Since this is a
fractal curve, it conta ins a well defined set of s t ructures at dist inct scales. The results (Figs 8(c)-(e)) iodicate tha t three quali tat ively distinct descript ions have been extracted.
Extracting natural scales using Fourier descriptors 1391
( / Fig. 9(e). High passed version of the contour superimposed
on a circle is shown with the original curve of H. Fig. 10(a). Extracted curve at first significant scale.
3.2. High pass filtering
The last section demonstrated low pass filtering to remove fine detail. Alternatively, high pass filtering can be employed to eliminate coarse detail leaving only the structures at smaller scales. This is useful if structttres are to be analysed without being biased by the distor- tion and transformation caused by their parent struc- tures. However, in order to view the structures some low frequency components must be present. Here we superimpose the structures of interest on a circle by reintroducing the components at to = 0 and 1.
A set of examples are given in Figs 9(a)-(e), which show the letter H superimposed with different types of finer structures. The significant scales of each contour are determined as before. The natural scale for each curve corresponding to the H shape is identified by hand and is used as the cut-off point of a high pass filter. The results of high pass filtering and reintroducing the low frequency components show that in each case the finer structures have been successfully extracted and displayed about a circle. However, at certain lo- cations of the curve some distortion of the structures is evident.
4. DISCUSSION
There are both advantages and disadvantages of the technique we propose. The closest method we can compare this technique to is that of Rosin. I1°~ It re- quires the contour to be smoothed with many Gaussians of different widths. In contrast, a single Fourier trans- form provides all the multiple scale information re- quired. Smoothing is simpler since there is no shrinkage effect which has to be compensated for when smoothing in the spatial domain. Another advantage of this tech- nique is that since we can bandpass as well as low pass the contour once we identify the frequency bandwidths
f ~ ~ x
Fig. 10(b). Extracted curve at second significant scale.
of importance, we can eliminate the coarse structure and extract only the finer structures as described in the earlier section. Secondly, much early research ~11'12~ has demonstrated the usefulness of Fourier descriptors in the description of contours. As outlined in Persoon and Fu, (3) one of the specific advantages of the Fourier descriptors we have chosen as the basis of our technique is that all sequences {an} for which the series converges describes closed curves and both open and closed curves can be reconstructed simply and efficiently. One of the disadvantages however is that since u(I) is a complex function, a n is not equal to a_ n. We have tackled this problem by computing both an and a_ n as (a n + a_n)/2. One of the strengths of both this method and that proposed by Rosin is that neither requires user-set parameters.
1392 P. ROSIN and S. VENKATESH
Fig. 10(c). Extracted curve at third significant scale.
There are two main disadvantages of this method. First, the amplitude of the discrete Fourier transform is noisy, and methods had to be devised to overcome this. We believe that the multi-scale analysis of the Fourier transform adequately handles this problem and yields satisfactory results. Second, and more im- portantly, the harmonic components caused by sharp corners interfere with those that are caused by the finer structures in the contour. There is no way in which these harmonic components can be totally eliminated. However, in spite of this periodicity, the techniques we propose generally locate significant natural scale ade- quately. Arguably, some of these scales can be con- Sidered as redundant. For example, Figs 10(a)-(c) show the results of extracting the first three significant levels of a slightly noisy square. A circle is extracted at the highest level of significance, followed by contours of a square-like contour with increasing sharpening of the corners. Both the circle and the rounded square seem reasonable extractions, although some of the extracted figures are probably redundant.
A possible solution to the ringing artifacts produced by the Fourier transform is to use alternative trans- forms whose basis functions are better suited to de- composing the signal. For example, the Walsh trans- form "7~ uses rectangular like functions as a basis set. The Walsh transform is given by
I 3 / - 1 n - I
where bk(Z ) is the kth bit in the binary representation of z.
The use of the Walsh functions implies that sharp corners will not produce the ringing effect that results from using smooth basis functions such as sines. How- ever, al though such a technique is well suited to poly- gonal curves the Walsh transform of smooth curves is noisy. It can be seen that no single basis function is suitable for decomposing all signals. If prior knowledge about the nature of the curves is available then an appropriate transform can be chosen.
Acknowledgements--We thank Geoff West and Marsudi Kisworo for the initial FFT code and for helpful discussions.
REFERENCES
1. H. Freeman, Computer processing of line drawings im- ages, Comput. Surveys 6, 57-98 (1974).
2. T. Pavlidis, Structural Pattern Recognition. Springer, New York (1977).
3. E. Persoon and K.S. Fu, Shape discrimination using Fourier descriptors, IEEE Trans. Syst. Man Cybern. 126- 130 (1977).
4. S. A. Coon, Surface patches and B-spline curves, Computer Aided Geometric Design, R. E. Barhill and R. F. Riesenfeld, eds. Academic Press, New York (1974).
5. A. P. Witkin, Scale-space filterng, Proe. 7th IJCAI, pp. 1019-1022 (1983).
6. J.J. Koenderink, The structure of images, Biol. Cybern. 50, 363-370 (1984).
7. A. Rosenfeld and M. Thurston, Edge and curve detection for visual scenes, IEEE Trans. Comput. C20, 562-569 (1971).
8. D. Marr and T. Poggio, A computational theory of human stereo vision, Proc. R. Soc. Lond. B 204, 301-328 (1979).
9. P. Meer, E. S. Baugher and A. Rosenfeld, Extraction of trend lines and extrema from multiscale curves, Pattern Recognition 21, 217-226 (1988).
10. P. L. Rosin, Representing curves at their natural scales, Pattern Recognition (in press).
11. C.T. Zahn and R.Z. Roskies, Fourier descriptors for planar closed curves, IEEE Trans. Comput. C21,269-281 (1972).
12. G.H. Gralund, Fourier preprocessing for hand print character recognition, I E E E Trans. Comput. C21,195-201 (1972).
13. N. Weyland and R. Prasad, Criterion for characterisation of line drawings using generalised Fourier descriptors, Proc. 7 th Scandinavian Conf. on Image Analysis, Denmark, pp. 48-55 (1991).
14. J. R. Bennett and J. S. MacDonald, On the measurement of curvature in a quantised environment, IEEE Trans. Comput. C24, 803-820 (1975).
15. J. Babaud, A. P. Witkin and R. O. Duda, Uniqueness of the Gaussian kernel for scale-space filtering, IEEE Trans. Pattern Analysis Mach. Intell. 8, 26-33 (1986).
16. R. W. Ehrich and J. P. Foith, Topology and semantics of intensity arrays, Computer Vision Systems, E. M. Riseman and A. Hanson, eds, pp. 111-127. Academic Press, New York (1978).
17. R.C. Gonzalez and P. Wintz, Digital Image Processing. Addison-Wesley, Reading, Massachusetts (1979).
About the Author--PAUL ROSIN was born in Glasgow in 1963. He gained the B.Sc. degree in computer science and microprocessor systems in 1984 at Strathclyde University, Glasgow, and the Ph.D. degree in information engineering at City University, London, in 1988. He was a Research Fellow at City University, developing a prototype system for the Home O~ce to detect and classify intruders in image sequences.
Extracting natural scales using Fourier descriptors 1393
While at the Department of Neurology at Guy's Hospital, London he worked on the Alvey project "Model-based Interpretation of Radiological Images". He is currently at the School of Computing Science, Curtin University where his research interests include the representation and segmentation of curves, knowledge-based vision systems, and early image representation.
About the Author--SVETHA VENKATESH is currently a Senior Lecturer at the Curtin University of Technology in Perth, Western Australia. Her research interests include artificial intelligence, pattern recognition and computer vision. She has mainly worked in the areas of low level feature extraction, object recognition, spatial reasoning and planning. She is a Senior Member of the Institute of Electrical and Electronic Engineers, and is currently the President of the Australian Pattern Recognition Society.
