Fourier descriptor based technique for reconstructing 3D contours from stereo images

H.-T. Sheu M.-F. WU

Indexing terms: Contour based matching, Constrained correspondence, Fourier descriptors, Stereo imaging

Abstract: A Fourier descriptor based hierarchy for stereo correspondence has been suggested (Wu and Sheu, 1993). However, the offset in starting points and the correspondence between the points constrained on the contours associated with the contours in correspondence were unsolved. An iterative algorithm is proposed for the computa- tion of the 3D FDs, given two sets of 2D FDs associated with the correspondence contours. For precise reconstruction, both the starting point offset and the constrained point correspondence problems are also solved. Experiments with a cylindrical object and a bottle gourd show that such an iterative procedure is both fast in the 3D contour reconstruction and indispensable for precise representation if the sizes of the point sets associated with the two correspondence contours deviate significantly.

1 Introduction

Stereo correspondence has been an interesting and diff- cult issue in the stereo vision arena. Most researchers determine the correspondence between pairs of two feature points, or dominant points to solve their particu- lar stereo problems [l]. To do this, they have to find the relationships between pairs of points over the entire sets of feature points in the stereo images. Thus the problem is complicated and usually difficult to solve. In Reference 2, Wu and Sheu have proposed a two level hierarchy for image matching. They suggest first seeking for contour matching then for point correspondence. Basically, their idea stems from the fact that the primary goal in the traditional approach for feature point corre- spondence is for ‘object’ matching, and a direct contour based object matching would be advantageous. For some recognition purposes the first level contour correspon- dence is enough 121. However, for reconstruction appli- cations, the second level point correspondence problem needs also to be solved.

In this paper, the association between contours are assumed to be known and the problem is to determine the correspondence between pairs of points constrained on the correspondence contours. For contour matching, it is natural to use Fourier description as has been

C) IEE, 1995 Paper 1851K (E4), first received 27th September 1994 and in revised form 23rd January 1995 The authors are with the Department of Electrical Engineering, Na- tional Taiwan Institute of Technology, #43, Keelung Road Section 4, Taipei, Taiwan, Republic of China

I E E Proc.-Vis. Imuge Signal Process., Vol . 142, N o . 2, Apri l 1995

adopted in Reference 2. There, only 2D FDs are employed. For point correspondence, however, 3D FDs are needed since, when the 3D contour is mapped on to both stereo images, pairs of correspondence points can be determined easily by examining the matched 2D FDs that result in nearly identical parameters.

Object representation by FDs has been developed for more than three decades and researches that follow this line have been numerous [3-101. Basically, these researches only considered 2D contours. For 3D applica- tion, Arbter et al. have developed the affine invariant Fourier descriptors for both 2D and 3D contours for object recognition [ll], and Lin and Jungthirapanich have proposed a method for computing invariants for 3D objects described by 3D FDs which are obtained from range image [12]. In contrast to the rich set of researches on 2D FDs, direct extensions to 3D objects are limited since the determination of 3D FDs is not straightforward. The main difficulties are that the description of the pro- jection mechanism that maps a 3D object on to a 2D image and vice versa is yet to be found, and the relation- ship among the FDs of the 3D object and that of the corresponding projected 2D images is hard to determine. Nevertheless, such a problem is very important and must be investigated carefully for the evaluation of 3D FDs. Thus, in this research an iterative scheme is developed for determining 3D FDs directly from the 2D FDs associ- ated with the two correspondence contours, successively improving the offset of the starting point, and approx- imating the correspondence among the pairs of points on the matched contours.

2 Geometry and projection

2.1 Camera geometry and point projection Consider a lateral stereo imaging setup shown in Fig. 1 where xyz refer to the world co-ordinate system; ui ui wi is for the ith camera co-ordinate system in which the camera lens centre Oi is the origin, i = 1, 2; and uiwi is the co-ordinate system for the ith image plane, i = 1, 2. A 3D object with contour r is projected through the lens centres 0, and 0, along the straight lines L , and L , on to the image planes called the u’,w‘, and U; w; planes as r, and r,, respectively. Note that a point on the image plane i can be denoted as ( u i , - L i , wi) in the camera co-ordinate i , where is the focal length of the ith camera.

The relationships among the world co-ordinate system and the camera co-ordinate system are established. Assume that a general lateral-stereo camera model [13, 141 is employed and let ( x C i , y C i , zci) be the co-ordinate of the origin of the (u i , u i , wi) system in the world co- ordinate system; R i be the rotation matrix of the ith

95

camera co-ordinate system with respect to the world co- ordinate system. Then, the difference between the origins of the two camera co-ordinate systems is [x,, y , zCdlT =

U " JQ$$ t W 2

W1

camera 2 camera 1

U1

L1 L

L L , /

X J

Fig. 1 Object ond camera geometry

[x,, - x,, y,, - y,, zC1 - z , ~ ] ~ , where [ . I T represents the transpose of the vector [ '1. Further, a point in the camera co-ordinate system can be transformed into the world co-ordinate system by

where Pi = [xCi yei zCilT. Thus the relationship between the two camera co-ordinate systems is

where

The perspective projection of a 3D object onto the image plane is established by parametrising the projecting lines. Consider the point projection first. For a point P on the object and its projection on to Qi on the ith image plane through the projecting line Li, and since Li passes through the lens centre Qi , i = 1, 2, the co-ordinates of Q2 and 0, with respect to the first camera coordinate system are R,[u; U; w;]' + P, and P, , respectively, and the parametric equations of the projecting lines L , and L , are

L , : [ ::] Z S , [ "'I W1 w;

and

L,: u1 = P, + s ,R, U; [::I [:I ( 4 4

where s1 and s2 are scalar parameters associated with L , and L , , respectively. Note that, when s, = s2 = 0, the points calculated by eqns. 4a and 4b represent the lens

96

centres on both cameras; whereas for s1 = s, = 1, they reduce to the projected points on both image planes, all in terms of the first camera co-ordinate system. If the co- ordinate of the point P of the object is [U;, y;, w;]', in the first camera system, since U; = -A, and U; = -A,, extrapolation of L , and L , to the point P yields

where D , , E , and F, are the first, second, and third entries in the second row of R , , respectively. Note that s, and s2 are functions of the range information U:.

2.2 Contour projection using FDs Since shapes (e.g. silhouettes of objects) can be character- ised by their FDs efficiently, it is quite reasonable to use FDs involving object contours. In general, the approaches to describe a 2D closed curve in terms of FDs can be categorised into three types, namely, Zahn and Roskies' cumulative angle approach 181, Granlund's complex plane representation [SI, and Lin and Hwangs' direct scheme [lo]. Compared with the cumulative angle FD, the last two methods contain not only the informa- tion about the shape but also the location, orientation, size, and the starting point. In addition, since the complex plane FD is only suitable of dealing with plane contours, it is determined that the direct scheme is a better way for describing a 3D contour.

In this Section, the contour projection is developed using FDs. The advantage of FD based contour is that most of the information about the shape is contained in the first few (lower frequency) coefficients, and noise usually affects only the coefficients of the higher har- monics. Therefore, pattern recognition (shape classification) can be carried out by only examining the first few coefficients. Let the closed contour of a 3D object be expressed by three Fourier series as shown in eqn. 6

in the first camera co-ordinate system, where w = 271t/T, I is the parameter (or the point number) that varies as the contour is traversed, and T is the period (or the total number of points detected on the contour) of the func- tions that describe the contour.

As a point moves along the contour, the projecting lines also vary so that s, and s2 are also periodic func- tions of, t. Hence s,(t) and sz ( t ) can be expressed in the Fourier series as

1 s,(t) = - pU1(t) - - - - [c,, + f (ck cos kw + d, sin kw) A , A I k = 1

( 7 4

and m

s,(t) = So + (Sak cos kw + S,, sin kw) (7b) k = l

I E E Proc.-Vis. Image Signal Process., Vol. 142, No. 2, April 1995

Further, the 2D curves rl, r2 on the image planes and the corresponding 3 D contour r can be expressed as

and

Note that r represents the 3 D description of the contour of the object with unknown coefficients.

Substituting eqns. 6, 7a, and 8a into 4a yields the equation for the left image (i = 1) as

1 = [eo + f (ck cos kw + d, sin k o ) '-1 k = l

Similarly, combining eqns. 6, 76 and 86 into 46, for the right image (i = 2) results in

where

(1 1) \ ,

Note that eqns. 9 and 10 involve the multiplication of two periodic functions and the product must also be periodic. The relationships among the coefficients of the

Theorem 1 in the Appendix. Thus, WJn(k) f i . n + k

+ dn[bl'n+k] + d.+k[::]} (13Y) f i , . + k

Fourier series expansions are derived as shown in [ @.@)I =; { . p + k ] - ."+.[kj - d n [ : l : : : : ] + d n + k [ : l l ] } (13j)

= [ t] + 'O[ + j1 {'a.[ + ' b k [ ; : ] ] Note that, in eqns. 12a-13h, although 4 + 8k unknowns

5 + 10k equations are involved, they represent an infinite a,, CO, eo, a,, b k , c k , d k , e k ? f , , S O , sa,, and s b k , and

(12a) I E E Pruc.-Vis. Image Signal Process., Vol. 142, No. 2, April 1995 91

dimensional problem as k + m. Thus, instead of the analytic approach, an iterative algorithm is proposed for the computation of the unknowns.

3

3.1 Iterative algorithm An iterative algorithm is proposed for the computation of the 3D FDs. The algorithm first starts by ignoring the infinite summation parts in eqns. 12a-12A then improves the solution at each iteration by including the infinite summation terms one at a time, and finally terminates as the preset accuracy is achieved.

From eqns. 124 the initial solution [a: e: e$' is obtained as

Iterative computation of 3D Fourier descriptors

where ( . y denotes the jth iteration of (.). Adding the first row to the second row in eqn. 12b and combining the result with eqn. 14 yields

where

A = I1am + (aio + elo)clo + A,e,, (16) Further, let Vi,(k)-V;,(k) and, Vin(k)-V$m(k) represent the values involving SLk and Sik in eqns. 13a-l3d, and W;,(k)-W$,(k) and Wi,(k)-W$,(k) stand for those associated with e', and d$ in eqns. 13e-13h. Then, from eqns. 12c and 12d, the initial solution for the coefficients is

and

~ ] = [ ~ ~ 1 s : + [ ; ~ 1 s . + [ ~ ] (17b)

Adding the first row to the second in both eqns. 12e and 12fand substituting into eqns. 17a and 17b results in

-1 A s : k = - {(alk + elk)c: + [ A l a c k

+ (a10 + eIO)ctk f A l e t k l s : + A&(k)} (lEa)

Note that, for k = 1, Ai,,,(k), and Ai,,,(k) reduce to 0 since l/j om (1) = . . . = V$,,,(l) = W;,(I) = ... = W$,,,(I) = 0 in eqns. 13a, 136, 13e and 13j

Including the infinite summation terms in eqns. 12a- 12f, the constant terms at the jth iteration can be obtained as

(20)

Adding the first row to the second row in eqn. 12b and substituting into eqn. 20, it follows that

-1 A

SJ - - - { I I J + (al,, + e, , )K + 1,L + A;;'} (21)

where

1 " A&1 = 1 { [ I l a t k f (a10 + e l O ) c r k + A l e ~ k l s i ~ l

-

k = l

+ [nib, + (a10 + eio)& + 11 &]si;' + (sit + e 1 k ) d - l + (bik + f i k ) d i - ' ) (22)

Similarly, the coefficients of the kth harmonics at the jth iteration are

and [;I =

(234

Adding the first row to the second row in both eqns. 12e and 12f and putting into eqns. 23a and 23b, it follows that

- 1 '!k = { ( a l k + e l k ) c & f [ l i a t k + (a10 + elO)crk

+ I l e , k ] S i + Ai,(k) + Aj;'(k)} (244

where

A&'(k) = A,Vi;'(k) + (a,,, + e,,,)V!;'(k)

+ A,V:;'(k) + Wd;'(k) + W&'(k) (25a)

A',iL(k) = I , V i ; ' ( k ) + (alo + e,,,)Vi;'(k)

+ I ,V$; ' (k) + Wi;'(k) + W$;'(k) (25b)

Finally, after transforming back to the world co-ordinate, the 3D FDs of the contour of the object described by

IEE Proc.-Vis. Image Signal Process., Vol . 142, No. 2, April 1995

eqn. 8c can be determined as

and

3.2 Convergence test for the iterative algorithm The convergence test of the iterative algorithm for com- puting the 3D FDs is determined in this Section. First, the approximation errors in the jth iteration are defined as

[ ek] [a! ~ a i - ' ] ijJk = c i - .{-I

e, e: ~ e{-' bj b.'-1

[ aik "1 = [ f j di I f j - 1 di-11 (27)

k - k

for the contant, the cosine harmonic and the sine har- monic terms, respectively. Next, the following definition is proposed.

DeJnition I : The approximation errors in the jth iter- ation with respect to the U, U and w components are defined as

After taking the triangular inequality for the right-hand side of eqn. 28, the upper bounds for the approximation errors in the jth iteration with respect to the U, U and w components are

Taking 2-norm to the elements of the vector in eqn. 29, Definition 2 follows.

Definition 2: The total upper bound of the approx- imation errors in the jth iteration is defined as

6;"b = [(6$ + (Sj,")2 + (6 iw)q1'2 (30) Thus, the problem for testing the convergence of the approximation errors becomes that for testing the con- vergence of h{ub. It is worthwhile to note that, for all of the experiments in Section 5, the convergence of Siub is very quick.

4 Starting point adjustment and constrained point correspondence

4.1 Starting point adjustment The iterative computation of 3D FDs described above assumes that, for both images, the continuous Fourier

I E E Proc.-Vis. Image Signal Process., Vol. 142, N o . 2, April I995

series sets off simultaneously. In practice, since the two images involved are processed separately, the starting points do not necessarily match with each other. More- over, for digitised data, the size of the point set associated with rl may not be the same as that of r,, and the cor- respondence among pairs of points on rl and r, needs to be solved. In this Section, the relationship that allows automatic correction for the starting points on the two contours will be derived first, followed by the determina- tion of the correspondence among pairs of points on rl and r2 .

Consider the contours on images 1 and 2, or, rl and r2 as shown in Fig. 2. Assume that the starting points of

startinq point

v " 2 image plane2 '1 image plane 1

Fig. 2 Shgt in starting points

rl and r2 are at t' = 0 and t = 0, respectively, and let

t' = t - t , (3 1 ) Then, the set of Fourier series expansion for rl becomes

Note that the contant terms in the Fourier series are not changed by the offset in starting point. Substituting eqn. 31 into 32 and comparing the result with eqn. Sa yields

2knt 2knt T T

all, = aik cos 2 - b,, sin 2 ( 3 3 4

2knt 2knt + b l k cos 2 T T

b,, = aik sin

2knt 2knt, e l k = e ik cos 2 - f i t sin - T T

( 3 3 4 2knt, 2knt

flk=e',ksin-+f;kcos2 T T

Since the FDs are determined iteratively, it is convenient to determine the approximate t , in each iteration.

Considering eqns. 23a and 236 for k = 1, and substi- tuting eqns. 33a-33d into eqns. 24a, 246, 12e and 12f yields

[ 3 j = [ : j S i l + [ :::]Si + [ ~~~~]

Sj -'i[ T

(34a) vi; '( 1 ) [i] =[3:1...[i3s.[~~~~d VJbnyl) ( 3 4 4

2nt =- ( a ~ l + e ' , , ) c o s - - - ( b i l A

x c i + C4al l + (alo + elo)cll + 4et l lS&

+ A:; '( 1)) ( 3 4 4

99

+ w:, '(1) (344

and

(34f) + wj-1 bn ( l )

Then, from eqns. 25a, 25b, and 34a-34f and after a series of computations (see the Appendix), it follows that

where

Solving t, from eqn. 35a at the jth iteration yields

~ l ~ l , ~ ~ 1 2 ~ 1 0 - P,lP,O)

Note that by ignoring Q{il and Qi;', eqn. 36 reduces to

(37)

which represents the offset in starting point associated with the first iteration j = 0. That is, Q{il and Qi i ' that are comprised of infinite summation terms act as modi- fiers for t, in each iteration. Algorithm 1 summarises the iterative computation of 3D FDs in which the adjustment of the offset in starting point is involved.

100

4.1 . I Algorithm 1 Step 1 : Set the number of harmonics to he considered

as N and the minimum total upper bound of the approx- imation errors as 6,,, .

Step 2: Obtain the 2D FDs (a,,. e,,, a;,, b;,, e;,,

Step 3: Calculate the coefficients (a,,, ctO, e,, , a,&)

Step 4 : Calculate the initial values S: and (a:, c: , e:)

Step 5: Calculate the values (P,,, PI, , P,, , PZo) from

Step 6: Calculate the offset t, in starting point associ-

Step 7: Calculate the modified 2D FDs ( a rk , blk, e,,,

Step 8: Calculate the initial values (Sf,, Sf,) from eqns.

Step 9: Calculate the initial values (aE-f:) from eqns.

Step 10: If k < N , then goto Step 8 ; else proceed., Step 11: Calculate the jth iteration values Sl, and

Step 12: Calculate the values (Q{;', Q&') from eqn.

Step 13: Calculate the offset t, in starting point associ-

Step 14: Calculate the modified 2D FDs(a,,, b,,, e,,,

Step 15: Calculate the jth iteration values (Si,, Si,)

Step 16: Calculate the jth iteration values (ai-fi) from

Step 17: If k < N , then goto Step 15; else proceed. Step 18: Using eqn. 30 to calculate the total upper

bound of the approximation errors 6:ub in the jth iter- ation.

Step 19: If 6jub > 6,,, , then goto Step 11 ; else proceed.

f;3 and (azo, e20, a2k, b2,, e 2 k J2A. from eqn. 11.

from eqns. 15 and 14, respectively.

eqns. 35b and 35c.

ated with the first iteration j = 0 from eqn. 37.

f,,) from eqns. 33a-33d.

18a and 18b.

17a and 176.

(a&, e', , a,) from eqns. 21 and 20, respectively.

35d.

ated with the jth iteration from eqn. 36.

f ix ) from eqns. 33a-33d.

from eqns. 24a and 246.

eqns. 23a and 236.

Step 20: Calculate (XO > Y O > ZO, x d , Xbk > y c k , ydk 7 zeP >

zrr) from eqn. 26.

4.2 Constrained point correspondence The correspondence between pairs of points on the correspondence contours r, and r2 is treated in this section. From Fig. 1, if Q, and Q2 are the images of the 3D point P, they must satisfy the epipolar equation (epipolar constraint [2])

a(w;K +AIL)+ b(u;L - w;J)-c(u;K + A,J) = 0 (38)

where [a b cl' = R,[u; - A, w;lT. Thus projection of L , on to image plane 1 results in an epipolar line L; described by eqn. 39

w', = mlu; + k , (39)

where

C K - b L aK - bJ

CJ - aL aK - bJ m , =- and k , = - A,

Note that the epipolar equation is a necessary condition only. That is, any pair of points satisfying eqn. 38 does not necessarily imply correspondence between the points. For instance, in Fig. 1, both points Q, and Q; satisfy the epipolar equation with respect to Q2 but only Q, corres- ponds to Q2 , This is because Q, is another point on the contour intersected by L; on the epipolar plane. For general types of contours, the number of points satisfying

/,?E Pioc.-Vis. Image Signal Process., Vol. 142, N o . 2, April 1995

the epipolar equation can be more than two and it is necessary to determine the correct correspondence point in the set of points satisfying eqn. 38.

Another problem that restrains one from matching a pair of points is digitisation. In this case, the equality in eqn. 38 becomes an almost impossible goal to achieve. Thus, instead of testing points for the epipolar equation, a minimum distance test is adopted for finding the pos- sible correspondence points. Define the function

f(u;, w;, U;, w;) = a(w;K + 1,L) + b(u;L - w;J)

- c(u;K + i , J ) (40) and for (U;, w’,) E l-, and (U;, w;) E r2, let

be the correspondence measure for I-, with respect to r2. If, for some point (U;, w;) E r2, a point (U’,, w;) E rl minimises the function F , the point (U;, w;) is called a ‘possible correspondence point’ with respect to (U;, w;). Note that there can be many possible correspondence points on rl associated with a point on r2 and it is necessary to determine the true correspondence point from the set of possible correspondence points. Algo- rithm 2 summarises the approach for this constrained point correspondence problem.

4.2.1 Algorithm 2:

FDs based on the 2D FDs associated with rl and r2 3D contour r using the initial 3D FDs

call the resulting curve ria, as shown in Fig. 3.

S t e p I : Use Algorithm 1 to calculate the initial 3D

S t e p 2: Compute the co-ordinates of the points on the

S t e p 3: Project the 3D points on to image plane 1 and

Fig. 3 Comparison between ‘possible correspondence points’ and r,#

Step 4 : Represent rlo in terms of Fourier series with the same argument t as that of T2

S t e p 5 : Obtain the correspondence point 4,. with respect to q2 by taking the same parameter value t for both T2 and rla

S t e p 6 : Compare the points in the set of ‘possible cor- respondence points’, with 4,. and determine the one that is closest to 4,. as the true correspondence point, say q l l for the case in Fig. 3.

After finding the set of the true correspondence points, the precise reconstruction of 3D contour can be achieved using Algorithm 3.

4.2.2 Algorithm 3 S t e p I : Employ Algorithm 1 to compute the precise 3D FDs except that:

(i) Step 2 in Algorithm 1 is replaced by ‘Obtaining the 2D FDs from the set of the true correspondence points’

(ii) Steps 5-7 and 12-14 in Algorithm 1 are skipped

S t e p 2: Reconstruct the 3D contour from its precise 3D FDs.

5 Experimental results

To determine the efficiency of the reconstructed 3D con- tours from Algorithms 1, 2 and 3, experiments under natural lighting are conducted for both a cylindrical

I < object and a bottle gourd as shown in Figs. -4 and 5,

a b

C

Fig. 4 Cylindrical object a Left image b Right image c Contour for (1

d Contour for b

d

a b

C

Fig. 5 Bottle gourd a Left image b Right image c Contour for (I d Contour forb

d

IEE Pmc.-Vis. Image Signal Process., Vol. 142, No. 2, April I995 101

respectively. After finding the 3D FDs of object contour, the shape of axisymmetric objects can then be recon- structed using the rotationally symmetric property. In the experiments, two CCD cameras are arranged in a general stereo fashion and the parameters are calibrated using the vanishing line technique suggested in [15]. Image processing techniques including moment preserving thresholding [16], Sobel edge detection [17], fast parallel thinning algorithm 118, 191, and contour tracing are used to extract the contour of an object from the images.

In Fig. 4, the left and the right images of the cylin- drical object are shown in a and b, and their correspond- ing contours are shown in c and d, respectively. While the 2D FDs are computed after tracing the 2D data from both stereo contours in Figs. 4c and d, Algorithm 1 then commences to calculate the 3D FDs. In the iterative algorithm, the offset t , in the starting points is automati- cally calculated, and the correspondence among pairs of points constrained on both curves can be solved by Algo- rithm 2. After a series of computations, the 3D FDs are stored as the representation of the cylindrical object. Sub- sequently, the centre line of the 3D object is determined based on the axisymmetrical property of the object, and the surface of the object is reconstructed by rotating with respect to this line the contour generated by using these FDs. The 3D reconstruction for the cylindrical object with 3D FDs up to the first, third, and twentieth har- monics are shown in Figs. 6a-c, respectively; whereas the

U b

similar no matter whether the point correspondence is solved or not. The reason is that the sizes of the point sets and r2 are almost the same. The situation is dif-

C

Fig. 7 U Up to first harmonic b Up to third harmonic c U p to twentieth harmonic d Up to twentieth harmonic (without considering correspondence)

Reconstruction of bottle gourd

ferent for the bottle gourd in which the sizes of r l and r2 deviate by about 4%. Consequently, smoothing at the neck of the bottle can be observed in Fig. 7d. Note that, if

is selected as the value of ami,, only four iterations are needed for both objects. Also, to avoid significant occlusion from happening between the two images, the depth of the objects is assumed to be long enough relat- ive to the separation between the two lens centres. In the experiment the objects are located about 1 m away from the cameras placed approximately 12 cm apart.

The computation time needed for the calculation of 2D FDs up to the twentieth harmonic is 1.1 seconds on a PC486-33, whereas less than 0.2 seconds are required for the calculation of 3D FDs up to the same number of harmonics. Reconstruction of the object from its 3D FDs takes less than 2.2 seconds. Thus, although the entire procedure looks quite involved, it is fast. If appropriate hardware support is available, the computation time can be further reduced.

Fig. 6 a Up to first harmonic h Up to third harmonic c Up to twentieth harmonic d Up to twentieth harmonic (without considering correspondence)

Reconstruction of cylindrical object

reconstruction up to the twentieth harmonics yet without considering point correspondence is shown in Fig. 6d. For the bottle gourd, the left and the right images are shown in Figs. 5a and b, and their corresponding con- tours are shown in Figs. 5c and d, respectively. For Fig. 5, similar results for the situations as those in Figs. 6a-d are shown in Figs. 7a-d, respectively.

Comparing these two objects indicates some difference when the constrained point correspondence is not treated. In the former, the reconstructed diagrams look

102

6 Conclusions

A method has been proposed to compute 3D FDs directly from 2D FDs associated with two stereo images. The suggested iterative algorithm computes not only the offset in starting point but also the FDs associated with the 3D object. In addition to the automatic adjustment of the offset in starting points associated with the two correspondence contours, another interesting idea in this research is the ‘contrained point correspondence’ problem which, to the authors’ best knowledge, has not been raised previously. Such a problem is solved by means of a ‘correspondence measure’ function. With both the starting point displacement corrected and the con- strained point correspondence problem solved, Fourier

IEE Proc.-Vis. Image Signal Process., Vol. 142, No. 2, April 1995

descriptor based precise 3D object representation and reconstruction can be achieved. Experiments with a cylin- drical object and a bottle gourd support these arguments.

7 References

1 BARNARD, S.T., and FISCHLER, M.A.: ‘Computational stereo’, ACM Comp. Surveys, 1982,14, pp. 553-572

2 WU, M.F., and SHEU, H.T.: ‘Stereo correspondence by using Fourier descriptors and minimal spectrum disparity’. Proceeding of 1993 IPPR conference on computer vision, graphics and image pro- cessing, 1993, pp. 263-270

3 COSGRIFF, R.L.: ‘Identification of shapes’. Ohio State university research foundation report, Columbus, Rep. 820-1 1, ASTIA AD 254 792,1960

4 RICHARD, C.W. Jr., and HEMAMI, H.: ‘Identification of three- dimensional objects using Fourier descriptors of the boundary curve’, IEEE Trans., 1974, SMC-4, (4), pp. 371-378

5 WALLACE, T.P., and MITCHELL, O.R.: ‘Analysis of three- dimensional movement using Fourier descriptors’, IEEE Trans., 1980, PAMI-2, (6), pp. 583-588

6 WALLACE, T.P., and WINTZ, P.A.: ‘An eficient three-dimensional aircraft recognition algorithm using normalized Fourier descriptors’, Comp. Graphics Image Proc., 1980,13, pp. 99-126

7 KUHL, F.P., and GIARDINA, C.R.: ‘Elliptic Fourier features of a closed contour’, Comp. Graphics Image Proc., 1982,18, pp. 236-258

8 ZAHN, C.T., and ROSKIES, R.Z.: ‘Fourier descriptors for plane closed curve’, IEEE Trans., 1972, C-21, (3), pp. 269-281

9 GRANLUND, G.H.: ‘Fourier preprocessing for hand print charac- ter recognition’, IEEE Trans., 1972, C-21, pp. 195-201

10 LIN, C.S., and HWANG, C.L.: ‘New forms of shape invariants from elliptic Fourier descriptors’, Part. Recog., 1987, 20, (9, pp. 535-545

I I ARBTER, K., SNYDER, W., BURKHARDT, H., and HIR- ZINGER, G.: ‘Application of afine invariant Fourier descriptors to recognition of 3D objects’, IEEE Trans., 1990, PAMI-12, (7),

12 LIN, C.S., and JUNGTHIRAPANICHI, C.: ‘Invariants of three- dimensional contours’, Putt. Recog., 1990, U , @), pp. 833-842

13 SHEU, H.T., and WU, M.F.: ‘The development and application of a general lateral stereo camera model‘, J. Chinese Inst. Eng., 1994, 17, (3), pp. 317-334

14 ALVERTOS, N., BRZAKOVIC, D., and GONZALE, R.C.: ‘Camera geometries for image matching in 3D machine vision’, IEEE Trans., 1989, PAMI-11, (9), pp. 897-915

15 WANG, L.L., and TSAI, W.H.: ‘Computing camera parameters using vanishing line information from a rectangular parallelipiped. Proceedings of 1989 IPPR conference on computer vision, graphics, and image processing, 1989, pp. 11 1-1 19

16 TSAI, W.H.: ‘Moment-preserving thresholding: a new approach‘, IEEE Compuf. Vis . Graph. Image Proc., 1985,29, pp. 377-393

17 JAIN, A.K.: ‘Fundamentals of digtal image processing’ (Prentice- Hall International Editions, 1989)

18 ZHANG, T.Y., and SUEN. C.Y.: ‘A fast pallel algorithm for thin- ning digital patterns’, Cummun. ACM, 1984,27, pp. 236-239

19 LU, L.E., and WANG, P.S.P.: ‘A comment on “a fast parallel algo- rithm for thinning digital patterns”’, Commun. A C M , 1986, 29, pp. 239-242

pp. 640-647

8 Appendix

8.1 Theorem 1 Consider three periodic functionsfl(t),f2(t), and F ( t ) with period T and the Fourier series expansions

2knt k = l T

+ b,, sin ’””> (42)

2knt k = l T

aZk cos + b,, sin ””) (43)

and

2knt F(t) = a, + f (ak cos + bk sin __

k = 1 T

. -

and

Proof: Multiplying the Fourier series expansions of fl(t) and f,(t), collecting the coefficients for the same harmo- nics, and comparing with those of F(t ) , eqns. (45)-(47) are obtained. QED

8.2 Proof of equations 35a-35e

In this Appendix, eqns. 35a-3% are proved via a series of computation. Rearranging eqn. 34e yields

+ WiL1(1) = 0 (48)

Substituting eqn. 34a into eqn. 48

y ~ j , + (Alatl + al0c,,)Sj, + L1vi;’(I) + aloYb;’(l)

where y = i ,a , , + a,, cIo. Further, the quantity A defined in eqn. 16 can be rearranged as

A = a + y (50) where a = i.,eto + e , ,c , , . Substituting eqns. 50 and 25a into 34c followed by substituting eqn. 34c into eqn. 49 results in

IEE Proc.-Vis. Image Signal Process., Vol. 142, No. 2, April 1995 103

and where

1 PI, = -Eatl + Pctl + ye,, (55) P = (yelo -@a,,) = e l0a , , - aloeto (53) l and

Q{G1 = - a [ i l V i ; ' ( l ) + Wi;'(l)] + fil.lVjn-l(l) and substituting eqns. 52 and 53 into eqn. 51, it follows that

+ y[%'V:;l(l) + w:;1(1)] (56)

T Thus the first row in eqn. 35a is identified. The second row can also be proved via similar computation if eqns. 256, 34b, 34d and 34fare considered. = AIS& P , , + Q{il (54)

104 IEE Proc.-Vis. Image Signal Process., Vol. 142, N o . 2, April 1995