Multirate predictor control scheme for visual servo control

T.P.Sim, G.S.Hong and K.B.Lim

Abstract: An attractive control scheme for a position-based dynamic “look and move” eye-in- hand visual servo system is proposed. The multirate predictor control scheme is used to overcome the inherent delay in the vision system and the multirate nature of the dynamic look-and-move visual servo control scheme. The control scheme takes the form of a simple predictor placed in the major feedback of the standard Smith predictor control scheme. This in turn eliminates the delay time between the dynamics of the observed target and the control, in a similar fashion to the feedfonvard Smith predictor. Hence the proposed control scheme is also known as the modified Smith predictor (MSP) control scheme. A detailed analysis of the proposed control scheme is presented for visual servo control. Representative computer simulation studies are presented to verify the effectiveness of the proposed multirate predictor control scheme for dynamic visual tracking.

1 Introduction

The dynamic “look and move” approach investigated in this research work is a dual-loop control scheme where the local controller presents us with an abstraction of the robot as a position-controlled device. Hence the visual servo controller must be capable of generating a series of smooth manipulator joint-angle set points. To accomplish this, the vision latency and target-tracking problems need to be addressed. A multirate Smith-like predictor control scheme is employed to solve these problems. In earlier work on a Smith control scheme for gaze control by Brown [I], simulation studies have shown that the prediction results of the Smith predictor (SP) control scheme have three effects: delays are overcome, dynamic interactions of delays on the control loop are overcome, and performance is improved. Moreover, Sharkey [2] has successfully employed the SP control scheme on an experimental gaze control with the Yorick head. Using accurate knowledge of actuator and vision processing delays, the target position and velocity predictor are used to counter the delay and to command the position-controlled actuators. They suggested that this scheme naturally emulates the Smith regulator. These works have shown the effectiveness of the Smith predictor control scheme in improving the perfor- mance of the visual servoing system. However, both works did not include implementation of such a control scheme that allows a robot to actively interact with the motion of the desired target such as visual servo tracking by a robot. In this paper, the modified Smith predictor (MSP) control scheme is employed in a dual-loop multirate time-delayed

0 IEE, 2002 IEE Pmceedmgs online no 20020238 DO( 10 1049hp-cta 20020238 Paper first received 25th June 2001 and in revised form 9th January 2002 The authors are with the Department of Mechanical Engineering, National University of Singapore, 10 Kent Ridge Crescent, Singapore 119260

IEE Proc -Control Theory Appl Vol 149 No 2 Mdrch 2002

visual servo control system. The multirate predictor control scheme to address the multirate and time-delayed nature of the visual servo control system is described by Huang [ 3 ] , Hong and Owens [4, 51.

In the dual-loop controller, the inner loop is the low- level robot controller that enables us to view the AdeptOne manipulator as a position-based device, assumed to be dynamically decoupled. The outer loop is the visual servo control loop that generates a series of smooth trajectory commands for target fixation. Both of these loops operate at different sampling rates. The low-level robotic control loop operates at 500Hz, and the visual control loop operates at the camera capture rate of 50 Hz. Apart from that, the vision system also imparts a 20ms sensor delay to the visual servo control loop. This is largely due to the computation time required for image processing and feature extraction. Previously, the most common method to deal with delays in the closed-loop system is to detune the primary gains to increase damping, thus making the system more robust in the presence of delays. However, the resultant response of the system will be lowered, leading to a sluggish overall performance. Also, as noted by Nemani [6], to enhance the speed and perfor- mance of the visual servo controller it becomes necessary to address the dynamic interactions between the faster servo inner loop and the slower visual outer loop. Hence, the control scheme should be designed to take into account adequately the dual-loop multirate effects. The proposed multirate predictor control structure achieves this by sampling the output of a modelled plant at a fast servo rate. The predicted response is fed back into the controller K via the inner visual servo loop to provide control information between measurements. This predicted response is also sampled at a slower rate and deliberately delayed for comparison with the actual plant’s response. The signal is then fed back through a fast shift operator to convert the slower-rate response into a fast counterpart for physical comparison with the demand signal. Hence the time-delay and multirate issues of a position-based visual servo control form the basis of this paper.

117

In the control formulation the error signal is taken as the difference between the desired and estimated pose vectors. The objective of the visual tracking control loop is to keep the target centred in the camera's field of view. The target motion is viewed essentially as a deviation from the desired position, which is not directly measurable. The eye-in-hand configuration enables only the measurement of the differ- ence between the target and robot's end-mounted camera position. The multirate predictor control scheme or the modified Smith predictor (MSP) control scheme is proposed to overcome the inherent delay in the visual tracking control loop. In this respect it is similar to standard Smith predictor (SP) control scheme. The SP control scheme is widely used in the chemical and process industries, especially where there are significant known processing time delays, and where the plant can be easily modelled. The principle is to take the delay outside the feedback loop, thereby allowing the controller to be tuned on the basis of an equivalent system without any delays. However, the effectiveness of the SP control scheme is influenced by its sensitivity to modelling error and its poor disturbance rejection capability [3]. Since good distur- bance rejection capability is necessary for good target- tracking characteristics, this justifies the use of a modified Smith predictor (MSP) control scheme. It has an added predictive operator in the outer visual loop. The predictive operator maps the slow sampling-rate signal to the fast sampling rate, as in the inner loop.

2 Theoretical development

The main aim of this Section is to make a systematic theoretical study of the proposed control scheme. It is of interest to develop conditions for good target tracking and stability of the proposed multirate predictor control scheme. The theoretical analysis follows the work of Hong and Owens [4, 51. For simplicity of presentation and without the loss of generality we concentrate on the

I case of a single-inputhngle-output (SISO) system. The proposed multirate predictor control scheme is as shown in Fig. 1. The visual servo system is regarded as a series connection of a robot manipulator with a visual capturing system. The robot manipulator is represented by a convo- lution subplant G with unirate sampling interval T,. The visual capturing system is represented by the sampling operator S which selects every Nth output from G to generate the measured output samples, and a delay opera- tor D represents the delay time involved in image proces- sing. The MSP control scheme consists of an inner loop which uses a subplant model GA to predict the fast sampling-rate system output to be fed back to the fast- rate controller K. The outer loop involves the feedback of the errors between the measured actual output yN and the predicted measured output y N A . Because this error is at a

?d

I: I WN F I

Fig. 1 Multirate predictor control scheme

slow sampling rate it is passed through a predictive operator F before the closing of the outer loop. The motion of the target can be considered as the disturbance input d. The disturbance response should ideally be zero for perfect tracking. Hence the control scheme should exhibit good disturbance rejection characteristics.

To enable a systematic analysis of the multirate control scheme to be carried out an infinite series sequence {xg, XI , x2,. . . , x,, . . .} of vector space loo is defined. This space is a Banach space with norm given by

llxllcr = sup I-c, I for x E loo (1) n?O

In this definition G is assumed to be a loo stable convolu- tion operator G: lco + loo. By defining the input to the sample operator to be Cfo, j r I , f2, . . .} with sampling time T,, the S operator can be defined as the mapping of &,f i ,

f2 , . . .} + cfg , f iN,f2N, . . .} of loo onto itself. S: em + loo,,v is a linear bounded operator. The induced norm of the operator S is

IlSll, = sup ~~ l l s f " m , ~ = 1 for N 2 1 (2) f € @ ( S ) IIJII,

f f 0

The delay operator D delays the sequence V 0 , f i N , f 2 N , . . . , fkN,. . .} by a unit-time step of T,=NT,. Hence D maps

& I ) N , . . .} of loo,N onto itself. D IS a bounded linear operator on loo,N+lco,N with the induced norm of operator D

C f O , f i N , f 2 N , . . . ) fkN,...}' CfTIN> f o , f i N , . , . )

f N i 6 0

Associated with the 'sample operator' and the 'delay operator' is the 'predictive operator' F: loo, + too. Given that w ~ = {wo,, w ~ A , , w ~ ~ , . . . ,wkN,. . .}, then the mapping of v = FwN is defined as

v(k + j ) = z'wN(k) (4) where z' is defined as a pure fast-forward shift operator that allows mapping of loo,N+ loo. Hence zJ has unity gain. Then the induced norm of F is given by

(5) W N # 0

Under this idealised condition F can be regarded as a right inverse of DS in the sense that

(6) FLS = 1

Let GA be a loo stable convolution system chosen to approximate the subplant dynamics G. K is a digital controller designed to produce the required stability and performance characteristics for a unity, negative-output feedback configuration of GA . The closed-loop equation of the multirate predictor control scheme takes the form of

u = K { r - F[DS{GU .f d - GAU}] - GAu}

= (1 + KG,)- 'K~ -- (1 + ~ ~ ~ 1 - l

x KFDS(G - GA)u - (1 + KGA)-'KFDSd (7)

2. I To meet the requirement of good target-tracking character- istics, the multirate predict" control scheme must exhibit good disturbance rejection characteristics. It is desirable to

Conditions for goad target tracking

IEE Proc.-Control Theory Appl.. Vol. 149, No. 2, March 2002 118

obtain conditions that can fulfill these requirements in its application for visual servoing. From Fig. 1, the following gives the mathematical derivation to arrive at the multirate predictor control scheme:

y , = DS(Gu + d ) (8) Hence the predictive output with a sampling time of T, is written as j j = FyN. From (8) and by realising the predictive output signal J we have

j = FyN = GAu + FbN - DSGAU] (9) The predictive output equation clearly gives the scheme of the multirate predictor control. The multirate predictor for the visual servo control enables good disturbance rejection, i.e. good target tracking, and also allows the use of higher controller gain in the visual servo control loop thus improving the overall performance of visual servoing. The main benefit comes with the ability of the visual servo control scheme to be controlled like a delay-free system by an ideal digital controller K. Rearranging (7)

K (1 + KGA) + KFDS(G - GA)

U =

d (10) KFDS

-

( 1 + KGA) + KFDS(G - GA)

Substituting (10) into (8), let A = G - GA, then

DSGAK yN = (1 + KGA) + KFDSA

- DsGAKFDs d + DSd (1 1) (1 + KG,) + KFDSA

Assuming no modelling mismatch, hence A = 0, we have from (9)

(12) FDSGA K FDSGA KFDS + FDsd

' = ( l +KGA)'- (1 +KGA)

Hence the system error is written as

e = r - y r (1 - FDS)KGA - -

( 1 +KG,)+ (1 +KGA)

d (13) FDSKGA ( 1 - FDS)

- FDs d - (1 +KGA) (1 + KGA)

where e E e,, d E loo and Y E e,. The convergence of e in loo space can be stated in a normal way as

This is the norm convergence in lm space, where e* E loo is called the limit of e. For good target-tracking character- istics

e* = 0 ( 1 5 )

Hence

Taking the induced norm of (1 3) and applying the triangle inequality, we have

FDS FDSKGA(1 - FDS) +- l/(l +K(;,ld1l + 11 (1 +KGA)

(17)

IEE Proc.-Control Theor?, Appl.. Vol. 149, No. 2, March 2002

Expr. 17 allows each individual term to be evaluated to satisfy the condition of (1 6).

Let K be a high-gain controller such that IKGI > A where A denotes a high gain. Assume an idealised situa- tion where A + 00. Then the operator 111 + KGA has an attenuating effect such that

The high-gain assumption also implies that

Consider

From (6) we have

Lastly, consider

FDSKGA ( 1 - FDS) 11 ( l + K G A )

From (6) and (1 9)

(23) FDSKGA(1 - FDS) d = O

Hence condition (16) for good tracking can be achieved. With regard to disturbance rejection, we assume the case for r = 0, in which (12) can be re-written as

11 + K G A ) ll w

with the ideal predictive operator F and the attenuating condition in (1 8) it is clear that the effect of j j due to Y is attenuated and hence disturbance rejection is achieved. This leads us to the following conditions to guarantee good target-tracking characteristics of the multirate predic- tor visual servo controller

C1: The controller gain IRI must be sufficiently high gain to make lKGl > A . C2: Ideally GA = G, i.e. the plant modelling error, A = 0. C3: FDS=DSF= 1 implies that the predictive filter F should be as close as possible, and act as a fast-forward shift operator.

2.2 Stability in the presence of model mismatch This Section analyses the robustness of the multirate predictor visual servo controller [4]. Here it is desirable to study the effect of model mismatch G - GA on the stability of the control scheme. In the previous Section we assumed thaf AZO; this is the ideal situation. In fact modelling will, in general, contain errors due to uncertain- ties of system dynamics, unknown time variation or the deliberate choice of a simple model to reduce control computational requirements. Hence, when the ideal situa- tion is complicated by GA # G, the error loop is activated.

119

This affects both the stability and performance of the predictor control scheme. (7) can be written compactly as

u = K*r - K*FDSd - K*FDS(G - GA)u (25) where K* = (1 + KGA)-'K. From (25), K* consists of K which is the controller operator designed to stabilise GA . Hence it can be deduced that U A =K*r and K*FDSd is bounded. Hence the mapping of r-+ u is BIBO stable in the sense of loo provided that the induced operator n o m of II( 1 + KGA)-'KFDSII, is a contraction mapping. Thus (25) can be compactly restated in the form of

u = U B - K*FDS(G - GA)u = W(U) (26)

where uB = uA - K*FDSd and mapping of UB-+ u has been shown to be BIBO stable in the sense of loo. Note that (26) takes the form of a fixed-point equation u = W(u) where W: lm -+ lm. By successive substitution (26) takes the form, for any p 1

uk = V B - (K*FDS(G - GA))pu(-l)p+l (27) with vB E loo. By the definition of the contraction mapping theorem, the stability of the predictive control scheme is guaranteed if for some p 2 1

Ap E II[K*FDS(G - GA)]pll, < 1 (28)

Noting that

[K*FDS(G - GA)]' =

K*FDS((G - G,)K*FDS)P-'(G - G ~ ) (29)

this yields

AP 5 IlK*ll, . IIFII, . 1 1 ~ 1 1 , . IISII,

' ll(G - GA)K*FDSII$-$-' ' - GA>ll, (30)

From norm evaluation of (2), (3) , and ( 5 ) we obtain llFlloo= 1, llD1loo= 1, IISlloo= 1. Hence (30) can be simpli- fied to

Ap 5 IIK*lloo ' ll(G - GA)ll, ' l l ( G - GA)K*IIk-l (31)

and thus Ap < 1 for some p 2 1 if

ll(G - GA)K*II, < 1 (32)

Therefore we have arrived at the main result of this . Section. Y stabilises G in the time-delayed multirate

predictive control scheme of Fig. 1 if

C4: both G and GA are stable, and C5: ll(G - GA)K*II, < 1, where K* =(1 +KGA)-' K. This implies that to maintain system stability it is necessary to have a small modelling error.

Note that C1 and C5 are contradictory conditions. C1 indicates the requirement of high controller gains whereas C5 provides limit for gain setting with model mismatch. This is an inherent problem with the MSP control scheme. C5 provides a metric for the designer to choose an appro- priate A subjected to the physical conditions.

3 Realisation

To realise the multirate predictor control scheme we need to satisfy all the conditions from Cl to C5. To approximate the requirements of C1, a PID controller is used. It is desirable to have high PID controller gains to maintain good target-tracking characteristics. The plant G is the AdeptOne robotics system encompassing the mechanical AdeptOne robotic arm and its master controller. Hence G is

120

a stable system because the low-level controller controls the arm in such a manner that it can be viewed as a position-based device. To realise the multirate predictor control scheme it is necessary. to model G. The modelled G is represented by GA in Section 2. Modelling such a decoupled system is not difficult. However, encoder values from the low-level control loop are readily acces- sible. Therefore they are taken directly as the required model feedback values. This eliminates the need to model the robotic system in practice. Thus, to realise the visual servo control scheme it is necessary to establish a high- speed communication link between the visual servo control loop and that of the local robotic controller. This approx- imation is taken to be adequate to satisfy conditions C2, and C5. Note that since the robotic system G is a stable system, the approximation ais taken here also implies that the modelled system GA is also a stable system. Hence condition C4 is satisfied. Lastly, we realise and satisfy condition C3 with a first-order fixed-gain gh filter [ 7 ] coupled with a trajectory planner. While the gh filter predicts the vision-delayed pose parameters 20 ms time- step ahead, the trajectory planner linearly samples this predicted value at the controller sampling time of 2ms. This provides a complementcay command set to the 500 Hz robot local controller.

From Fig. 1, good visual tracking is accomplished by rejecting the disturbance signal d. However, we are unable to formulate the required hlomogeneous robotic transfor- mation of the proposed multirate predictor control scheme within a consistent framework. To make such a formulation possible it is necessary to iincorporate the object's motion in the reference input. The revised multirate predictor control architecture is given in Fig. 2. Now the vision system is required to perform measurement of the object's pose with respect to the base frame. In fact the eye-in-hand configuration only enables the measurement of relative pose of the camera with respect to the camera frame. Nonetheless, by using this measurement and combining with other known transformations, it is possible to satisfy the given requirement. Hence we have

(33)

With reference to Fig. 3, BlrCref is the desired pose of the camera frame with respect to the base frame, ""fT, is the desired reference homo enous transformation matrix for visual fixation, and oTgb is the delayed pose estimate obtained from the eye-in-hand vision measurement from the DeMenthon-Horaud pose estimation algorithm [ 1 11. In Fig. 2, g(.) denotes a function that transforms the 4 x 4 homogenous transformation matrix T to the corresponding six pose parameters. k denotes the vision time-step of Tv=20ms, and n denote:; the controller time-step of Tc = 2 ms.

By defining the camera reference frame { Cref}, it is now possible to visualise visual tracking as the need to move the camera frame {C} from its present location to the desired reference location. The camera reference frame is fixed with respect to the object's co-ordinate frame, and during visual servoing this is the required fixation between the arm and the moving object. At each delayed time-step the reference co-ordinate frame is given by

(34)

From Fig. 2 the pose parameters B&-l can be defined as

IEE Pro,.-Control Theory Appl., El. 149, No. 2, MaPch 2002

, .~ ................. ~ . . ~ ............ --..-----.----.--- ................

I I I

Fig. 2 Predictor control scheme architecture

From (34) and (35)

In maintaining consistency with the ro osed predictor

one unit time-step ahead using a first-order gh filter. The gh filter is defined by the following prediction update equa- tions:

control scheme, the pose parameters B P &-, are predicted

BWZ,k-I = "wf~j~k-2 + T,. 'wZ7I,k-2

+ diag(g>(B<k-l - Bw$-l,k-2> (38)

where 'wf,k-, and BWf,k-l denote the predicted velocity and position at time-step k based on the current measure- ment at delayed time-step k - 1. The g and h fixed gains are determined by the Benedict-Bordner method [7]. To realise the control scheme, the position and orientation parameters are needed at the sampling instance of kT,+ mT,, m = 1, 2, 3 , . . . , T,/T,. This is accomplished by assuming constant velocity over the sampling period [k, k - 11 based on the following trajectory planner equation:

%(kT,+mT,) = diag(8)(Bwz.k-1 - 'wz-l,k-2)m + B ~ ~ , k - l

(39)

where Bn(kTV + mT,) is the estimated predicted pose para- meter at the sampling instance of kT, + mT, and 8 is the trajectory planner gains. Normally 8 = T,/Tc { 1, 1 , . . .} . (38) enables data to be continuously fed to the robot controller, providing a complementary command set at the robot's controller rate of 500Hz. Hence the required error signal can be formulated. Noting that nT,- kT, + m T, , then

e(n) = Bn(n) - q(BT;) (40)

where BTZ is the homogeneous transformation matrix of the camera co-ordinate frame at every time-step n. This error signal is then fed into a digital PID controller. To avoid the integral windup effect at the boundary of the robot workspace for visual servoing, a tracking antiwindup

digital PID controller is used. It can be compactly written as

where K,, T D I , TI, are the digital PID controller gains at ith pose direction, T, is the discrete controller sampling time,

92

Fig. 3 Pose vector relationship

IEE Proc.-Control Theory Appl., Vol. 149, No. 2, March 2002 121

u is the control signal and u, is the saturation bound. TT is the tracking time constant. In general the setting of high controller gains as required by condition C1 compromises the robustness of the system to perform its task. The PID controller is implemented to enable a certain degree of freedom to improve both the transient and steady-state tracking performance. The PID parameters will be deter- mined through the normal heuristic tuning procedure using the root mean square of the tracking errors as the perfor- mance index.

4 Simulation results

This Section gives representative computer simulation results pertaining to the use of multirate% predictor control scheme for visual servo control. Specifically we study the target-tracking capabilities in using the predictor control scheme compared with a normal single visual feedback control loop. It is a norm in the dynamic look-and-move approach to employ a single visual feedback system [8, 91 where optimised predicted visual values are fedback for closed-loop control. Ideally, if the predictions are exact the time-delayed condition of the visual servo system is resolved. With this comparison it is the objective of this computer simulation to verify the overall improvement of introducing the proposed control scheme for visual track- ing. We carried out the simulation studies by modelling the first two links of the AdeptOne robot. The low-level controller is taken to be a computed torque controller. The vision system operates at SO Hz while the robot servo rate is S00Hz. A fixed time delay of 20ms is assumed. This presents us with a multirate time-delayed visual servoing system capable of xy planar tracking.

4.1 Tracking performance This Section compares the tracking performance of the proposed predictor control scheme with the single visual feedback control scheme. To have a valid comparison of the two visual servo control schemes, both retain similar elemental algorithms: the DeMenthon-Horaud pose esti- mation algorithm for 3D pose-parameter recovery from successive 2D image data, the gh prediction filter and trajectory planner, and the digital PID controller. The only difference is the manner in which the available information is utilised. The control architecture is shown in Fig. 4.

In this simulation study the movement of the object is simulated to follow a square motion path (200" per side) as in [9]. The speed is maintained at SOmm/s

Fig. 4

550, . . . . . . . / " " " ' I

3001 1 250' " ' " " ' J -

t l : F ] & O

-5 0 2 4 6 8 1 0 1 2 1 4 1 6 1 8

time, s

E E c (u A

0 2 4 6 8 1012141618 time, s

Fig. 5 predictor control scheme

object motion - arm motion Lower traces: tracking error

Peformance of simulated path-following of square traversal with

. . . . . . . . 550 , 1

500

E 450 EG 400

m* 350 300

250 , A , , , , , , JU 5

E

time, s time, s

Fig. 6 single-feedback control scheme

~ arm motion Lower traces: tracking error

Performance of simulated pathTfollowing of square traversal with

object motion

throughout the square transversal with gaussian image noise corruption of N (0, 0 = 1 pixel). The results of this simulation for the predictor control scheme and single feedback visual control scheme are presented in Figs. 5 and 6, respectively.

In the situation in which the object is travelling on a straight path with near constant velocity, denoted by the portion AB in the given plots, the target-tracking perfor- mance of the predictor conmol scheme is clearly superior. The RMS tracking error for the complete square path is [ 1.43, 1.181 mm in the x, y-direction for the MSP control scheme, and [1.65, 1.651 mm in the x, y-direction for the

, .................................................................. AdeptOne module

i

L

Single feedback visual servo control architecture

camera 07ri

122 IEE Proc.-Control Theory Appl., Vol. 149, No. 2, March 2002

single feedback visual control scheme. The RMS tracking error for the MSP control scheme is smaller because the predictor control scheme affords the setting of higher controller gains. This enables tracking of objects travelling at higher velocity. It differs from the normal visual feed- back system whereby the setting of high controller gains results in instability. This success verifies the validity of the approximation taken in realising the multirate predictor control scheme. The setting of higher controller gains also improves the sluggishness of the overall control system due to vision delay. The ability to respond faster also makes it react more strongly to changes in the object’s velocity. This results in the hike in tracking error as shown in Fig. 5. The problem is caused by the imperfection in implementing the fast shift operator F: A pure fast shift operator to realise the mapping given by (4) will not be possible in practice without violating the causality constraint. In practice, this hike in tracking error can be overcome if a sufficient camera field of view is available to accommodate the arm’s overshoot.

4.2 Varying object speed We also investigate the target-tracking performance under varying object speed. We assumed that there was no hardware limitation and the object’s feature point would always be in the camera’s field of view. This is a necessary assumption because the displacement per unit time-steps increases with the object’s speed. The object is simulated to move from an initial point with gradient of -1.667 in xy- plane for a duration of 4.0s along a straight path with s eed vI varying from 0.05 to 0.25m/s. Note that v1 =v, +v;, hence relative to x, motion in y will be faster. The results are presented in Fig. 7. RMS tracking errors are taken as the quantitative measurement of track- ing performance [9, lo]. In general, the tracking perfor- mance deteriorates with the object’s speed. This simply implies that it is getting harder to follow the object as it moves faster. It also explains the higher tracking lag in y relative to x because the object is travelling faster in this direction. In addition, Fig. 7 clearly indicates the improve- ment in tracking performance by the use of the predictor control scheme.

4 2

4.3 Implementation issues The proposed control scheme has been successfully imple- mented in an industrial AdeptOne system. From this experimental work, and the computer simulation presented

7

6

E 5 E c

2 4

3

2

1 50 100 150 200 250 !

0.5

b’

100 150 200 250

object velocity, mm/s

Fig. 7 -@- predictor control -A- single visual feedback

IEE Proc.-Control Theov Appl., Vol. 149, No. 2, March 2002

RMS of tracking deviation for continuous linear motion

in earlier Sections, it is vital to observe the following implementation considerations:

0 The camera’s field of view should be adequately sized to accommodate changes in the object’s motion. However, this implies that a longer computational time is required for image processing in the visual capturing system. This may result in a longer time delay (Tv) unless a more powerful processor is used for image processing.

The use of a PID controller necessitates good pose- -estimation results. The PID controller may be simple but it does not compensate for noisy vision measurements that are inherent in the actual vision system [12].

The experimental results confirm the observation given by the simulation studies [ 13 1. Even though the demonstrated system lacks the computational power of today’s compu- ters, it has shown the ability to perform visual tracking. The simple and pragmatic approach taken in this research is the main feature of such a predictor control scheme compared with other proposed visual servoing systems.

5 Conclusions

We have introduced an elegant and effective control scheme for a position-based dynamic look-and-move visual servoing system. Inherent in a dual-loop visual servo control system, the dominant dynamics were found to be the time delay of the vision system and the multirate nature of the control system. These two effects are unavoidable in any vision-based system due to limitations in the vision processing. To resolve this, a multirate predictor control scheme is employed. The predictor control scheme has good disturbance-rejection capabilities. Since the object motion is considered a disturbance input, this resulted in good target tracking because the predictor control scheme takes the inherent vision delay outside the feedback loop, thereby allowing the controller to be tuned like a delay-free system. Hence application of the multirate predictor control scheme affords better setting of control parameters. This avoids the sluggishness faced by the conventional single-feedback visual servo control scheme. Analytical and representative computer simulation results demonstrated the improvements in using the proposed control scheme. It contributes significant improvement in the tracking capability of the visual servo- ing system.

6

1

2

3

4

5

6

7 8

9

References

BROWN, C.: ‘Gaze control with interactions and delays’, IEEE Trans.,

SHARKEY, P.M., and MURRAY, D.W.: ‘Coping with delays for real- time gaze control’, SPIE Sensor Fusion VI, 1993, 2059, pp. 292-304 HUANG, H.-P., CHEN, C.-L., CHAO, Y.-C., and CHEN, P.-L.: ‘A modified Smith predictor with an approximate inverse of dead time’, AlChE 1, 1990,36, (7), pp. 1025-1031 HONG, G.S., and OWENS, D.H.: ‘Stability ofpredictor control schemes for multirate sampling’. Proceedings of the IEE conference ‘Control 85’, University of Cambridge, 1985, pp. 605-608 OWENS, D.H., and HONG, G.S.: ‘Frequency domain robust stability conditions for multirate predictor control schemes’. Proceedings of 25th conference on Decision and control, Athens, Greece, December 1986,

NEMANI, M., TSAO, T.-C., and HUTCHINSON, S.: ‘Multirate analysis and design of visual-feedback digital visual servo-control system’, Trans. ASME J1 Dyn. Syst. Meas. Control, 1994, 116, pp. 45-55 BROOKNER, E.: ‘Tracking and Kalman filter made easy’ (Wiley, 1998) WILSON, W.J., HULLS, C.C.W., and BELLS, G.S.: ‘Relative end- effector control using Cartesian position-based visual servoing’, IEEE Trans. Robot. Autom., 1996, 12, (9, pp. 651-670 WESTMORE, D.B., and WILSON, W.J.: ‘Direct dynamic control of a robot using an end-point mounted camera and Kalman filter position estimation’. Proceedings of IEEE international conference on Robotics and automation, Sacramento, CA, April 1991, pp. 2376-2384

1990, SMC-20, (I), pp. 518-527

pp. 774-775

123

10 C O E , P.I.: ‘Visual control of robots-high-performance visual servo- ‘Visual tracking of a moving target by a camera mounted on a robot: a combination of control and vision’, IEEE Truns. Robot. Autom., 1993,9,

13 SIM, T.P., HONG, G.S., and LIPA, K.B.: ‘A multirate predictor control scheme for visual servo control’, NUS technical report ME-CO-01-05. National University of Singapore, Singapore, 2001

ing’ (Wiley, New York, 1996)

‘Object pose: the link between weak perspective, paraperspective, and full perspective’, Int. 1 Comput. vis., 1997, 22, (2), pp. 173-189

12 PAPANIKOLOPOULOS, N.P., KHOSLA, P.K., and KANADE, T.:

11 HORAUD, R., DORNAIKA, E, LAMIROY, B., and CHRISTY, S.: (l), pp. 14-28

124 IEE Proc-Control Theory Appl., 61 . 149, No. 2, March 2002