104 IEEE JOURNAL OF ROBOTICS AND AUTOMATION. VOL. 4. NO. I , FEBRUARY 198R

Robot Control Using Adaptive Transformations

ARNOLD J . LOBBEZOO, MEMBER, IEEE, P. M. BRUIJN, M. S . DAVIES, MEMBER, IEEE, WILLIAM G. DUNFORD. MEMBER, IEEE, PETER D.

LAWRENCE, MEMBER, IEEE, AND H. R. VAN NAUTA LEMKE

Abstract-A control strategy is presented for robots that do not have accurately known mechanical structures or have inaccuracies caused by bending, slip, or backlash. In the system, the manipulator’s endpoint is monitored in a servo loop, so that inaccuracies in the structure can be compensated. An adaptive transformation from task to robot-oriented coordinates has been used on-line, without prior modeling or calculation. The developed strategy was simulated for a two degree of freedom robot. Results were compared with those obtained using the inverse Jacobian as part of the control system.

INTRODUCTION

In most applications a manipulator’s motion is defined in task- oriented coordinates, whereas control is joint-oriented. This implies that the inverse kinematics problem, which gives the relationship between joint and task coordinates, has to be solved [lo], [12]. Paul [ lo] and Duffy [2] give extensive analyses of such coordinate transformations. In addition to problems in the derivation and programming, these transformations involve large numbers of transcendental functions, multiplications, and additions, which re- quire much computing effort. The inverse kinematics problem is said to be one of the most difficult in robotics [ lo] .

Currently available robot control systems, using predetermined coordinate transformations, are not effective with manipulators that contain flexible links since such links introduce inaccuracies due to bending. Other sources of inaccuracy can occur in the joints between the arm segments (for example, slip and backlash).

Problems also exist even when the Cartesian endpoint position is measured as in [ 141. A detected error in the endpoint position has to be transformed into new setpoints for the joint servo controllers. This is done by a prederived coordinate transformation which depends on the joint angles of the manipulator. Such a control system is not able to adapt itself perfectly to a distortion in a manipulator’s structure.

Attempts have been made to include bending in the modeling of manipulators by using virtual joints to represent distributed flexibil- ity, but such models are too complex for use in real-time control of industrial robots [ 121.

Some manipulator control systems can adapt to system changes (e.g., load or moments of inertia), but such adaptation is limited to changes that influence the joint dynamics [4], [6]. Koivo [6] suggested that the coordinate transformation could be included in an adaptive controller for manipulators and reported on this work in [7]. Asakawa [ I ] describes a robot control system with optical position feedback. This system detects the location of the endpoint of the manipulator relative to its task. The desired direction of motion for each joint is determined, and each joint in turn is activated at constant speed until a stopping criterion signals that the minimum relative

Manuscript received July 9 , 1986; revised December 2 , 1986. This work was supported by the Natural Sciences and Engineering Research Council of Canada under Grants A4924, A4148, and A1703.

A. J. Lobbezoo was with the Department of Electrical Engineering, University of British Columbia, Vancouver, BC, Canada. He is now with American Motors (Canada) Ltd., Bramalea, Ont., Canada L6T 4Y6.

M. S . Davies, W. G. Dunford, and P. D. Lawrence are with the Department of Electrical Engineering, University of British Columbia, Vancouver, BC, Canada V6T 1W5.

P. M. Bruijn and H. R. van Naute Lemke are with the Electrical Engineering Control Group, Delft University of Technology, Delft, The Netherlands.

IEEE Log Number 8718080.

?\ I I

I I I I I I I I I I I I I I

\ \ \ \ \

Fig. 1. Two-joint two-link manipulator (/, = /* = 1 m, m, = 3 kg, m2 = 5 kg).

distance, obtainable with that joint, has been reached. The resulting trajectory thus resembles an iterative process.

In the work reported here, a trajectory control strategy with visual real-time endpoint feedback was investigated. The endpoint position error was transformed into joint position errors which were used as inputs to joint servo controllers. The coordinate transformation was determined on-line and adapted continuously during endpoint motion. A nonadaptive form of control using the inverse Jacobian was described by Whitney [13]. In contrast, the present work separates the kinematics from the control, carrying out adaptation only of the kinematic transformations. The control law remains fixed.

The method proposed here allows for endpoint servoing of existing joint-controlled robots by using the adaptive inverse Jacobian along with a vision system. The number of joints and a set of initial conditions are the only apriori knowledge required for determination of the adaptive inverse Jacobian. The accuracy of control using the adaptive inverse Jacobian was compared with that produced by control with an exact predetermined inverse Jacobian. It is important to realize that the purpose of the proposed method is not solely to improve endpoint positioning. The proposed method has two other advantages. First, the inverse Jacobian, for which an analytic solution is not always easily available, can be estimated. Second, the physical dimensions of the robot arc not required, eliminating any need for kinematic parameters or concern about tolerances. Since accurate position sensing devices are becoming available, a control system using endpoint position measurement is becoming technically feasible P I , [9l.

MECHANICAL SYSTEM

The two degree of freedom manipulator of Fig. 1 was modeled to simulate the control strategy (see also [ IO]) . In Fig. I , I, = 1 m and 12

= 1 m are the lengths of the two links. All mass is assumed to be concentrated at the joint, with the mass at the second joint given by ml = 3 kg, and the mass at the endpoint given by m2 = 5 kg. Two joint coordinates are defined as 4I and 42 (in the rest position: 4, = 42 = 0.0). TI is the actuator torque at the first joint; T* is the actuator torque which acts on the second link.

For an ideally stiff system, the Cartesian endpoint position is given by

0882-4967/88/02OO-0104$01 .OO 0 1988 IEEE

IEEE JOURNAL OF ROBOTICS AND AUTOMATION, VOL. 4. NO. I , FEBRUARY 1988

Parameter

Identification

Manipulator Estimated Joint

Inverse Jacobian -

Q T Y Transformation Controller Dynamic Model

-7

-4+ I

105

L c

Pixel Y noise

The Lagrangian L is defined as the difference between the potential energy P and kinetic energy K of the system:

A second-order Runge-Kutta algorithm with variable step size was used for digital simulations of the model.

CONTROL Joint Control

Fig. 2 gives a model of the complete control system. The joint control systems used the estimated angular position errors as input, and velocity feedback was used to provide system damping (PD- control).

The central two blocks of Fig. 2 contain both joint control and dynamic model. The joint torque T was calculated by

The robot dynamic equations (2) were then integrated to give the resulting angular velocities and positions 4. The final block of Fig. 2 transforms the joint angle to Cartesian positions.

The following values were used as the controller parameters:

Joint 1 Joint 2

Kml = 50 K,I = 140 K,2 = 100 K,I = 15 Kv2 = 12.

Kn,2 = 30

The frequency of sampling for the control system was set at 200 Hz, which was substantially higher than the fastest eigenvalue of the system.

The control parameters were chosen to minimize the following function for the step responses at a number of operating points:

] (3) N

c= e:(kT)= 2 [e ; , (kT)+e: , (kT) .

This is the integrated squared endpoint error taken at each time sample kT; NT is the response time, and e, is the endpoint position error.

Endpoint Control To compare the proposed adaptive transformation with the

conventional inverse Jacobian, a configuration similar to that shown in Fig. 2 was used, with the adaptive coordinate transformation algorithm replaced by a calculated inverse Jacobian matrix. The system thus still contains an endpoint position feedback loop and joint velocity feedback loop. Endpoint position errors were transformed into joint position errors by the inverse Jacobian of the manipulator. For

k = I k = I

ay1 ay, J=[Z w1 4 using ( l ) , and inverting,

(4)

106 IEEE JOURNAL OF ROBOTICS AND AUTOMATION, VOL. 4, NO. I , FEBRUARY 1988

ADAPTIVE TRANSFORMATION In the adaptive transformation procedure used here, the inverse

Jacobian was estimated and not preca!culated as just described. A least squares method was used to estimate the entries of (5).

To develop the adaptive transformation algorithm, define the following:

@ = (41, 42)

Y = ( y l , y2)

joint coordinates

Cartesian endpoint coordinates.

At time nT the values of @(nT) and Y ( n T ) can be measured. With A@(nT) = @(nT) - @[(n - l ) f l andAY(nT) = Y ( n T ) - Y [ ( n - l ) r l >

A@(nT) =DB(nT) . A Y ( n T )

where DB(nT) gives the relationship between A@ and A Y at time ( n T ) . As the sampling interval is reduced (T -+ 0), the distance between successive observations tends to zero (AY -+ 0) and DB becomes the inverse Jacobian matrix:

1 ay2 J

The inverse Jacobian was tracked by using least squares estimation P I .

The signals y,(i = 1 , 2) and 4,( j = 1, 2) were measured, and the changes A Y and A@ were calculated. The error e, was calculated for each joint angle using

e,=A@,-db,, . A ~ , - d b , ~ . Ay2, j = 1 , 2 (7)

where

L

is the estimated inverse Jacobian. The cost criterion 5 to be minimized in estimating DB was

N t I

v , ~ + ~ = x ~ + ~ - ~ . efk, J = 1, 2 (8) k = I

with e,k meaning the angular coordinate error at time sample k. NT is the current time instant. The definition of vj provides for a “forgetting” factor h that reduces the weight assigned to older data. The following recursive algorithm for on-line calculation of db,, can be derived [ 111. Note that the scalar parameters in each row of the matrix DB were estimated independently:

[ “‘1 = [ + P N + ~ A Y N + I ~ , N + , , j = 1 , 2. (9) dbJ2 N + I db]2 N

The algorithm requires initial estimates of the parameters, and of the error covariance matrix P. P is initially set as diagonal with large positive diagonal entries (P(1, 1) = P(2 , 2) = I O s in the example discussed here) when the initial parameter estimates are uncertain. Lower values of initial P would be indicated if good initial estimates of the entries of PB were at hand. The identification results give the new values for the elements of the adaptive inverse Jacobian. The

-0 4

-0 b

-0 8

-1

-1.2

-I 4

-1 6 -02 0 0.2 0 4 O b 0 8 I 2

Fig. 3 . Full speed trajectory using J-I for control. No pixel noise. Coordinate axes labeled in meters.

structure of the control system with the adaptive inverse Jacobian is shown in Fig. 2.

The joint angle increment A@ output to the joint servos in Fig. 2 is determined by the present Cartesian position error A Y and the most recent estimate of the inverse Jacobian DB(nT) :

A @ = DB(nT)A Y . (10)

The parameter estimation was carried out in two ways. The direct estimation of the matrix DB as just described, and an inverse estimation, using the same least squares algorithm, of the parameters of the inverse matrix DB-’. After the inverse matrix is estimated. then a numerical matrix inversion provides the transformation parameters DB for use in (IO). Use of the inverse estimation procedure is appropriate in the presence of pixel noise arising in a vision system used to measure the manipulator endpoint position. When no noise is present, the estimation is providing a local linear approximation to the nonlinear coefficients, and there is no a priori reason to choose forward or inverse estimation. However, since additive pixel noise disturbs the Cartesian measurement data Ay, it is more reasonable to estimate the parameters of the inverse matrix. The least squares algorithm is optimal for additive zero-mean white Gaussian disturbances in the dependent variable.

RESULTS The same tests were carried out using both the analytic inverse

Jacobian (5) and the adaptive methods. The test trajectory was chosen to be the perimeter of a circular quadrant centered at (0, -0.5) enclosed within radial vectors terminating at points (0, - 1.5) and (1.0, -0.5). At normal speed, the circular arc was traversed in d 2 = 1.57 s and the radial paths each in I s , giving a total time of 3.57 s to complete the path.

As a benchmark, the initial test run used the analytic inverse Jacobian (5). Errors between the target and actual trajectories in this case (Fig. 3) are due only to the control system. Notice the gravitational loading on the horizontal portion of the trajectory as the quadrant is traversed in a counterclockwise direction.

In Fig. 4 the effect of using the estimation of DB is shown. Note that the target trajectory is followed with little degradation in the radial portions; however, the estimation errors lead to noticeable deterioration in the circular arc. In this case, the estimation algorithm used a forgetting factor, h = 0.85. To compare estimation accuracy for different situations, the following error criterion was used:

IEEE JOURNAL OF ROBOTICS AND AUTOMATION. VOL. 4. NO. I. FEBRUARY 1988 107

-0.2 0 0.2 0.4 0 .b 0.8 I I1

Fig. 4. Full speed trajectory using h = 0.85 estimation of DE. No pixel noise. Coordinate axes labeled in meters.

-0 4- -

-0.6-

-0.8 -

I -

-1.4

- I 6 I I 1

-0 .2 0 0.2 0.4 0.6 0.8 1 1.1

Fig. 6. Full speed trajectory using h = 0.95 estimation of DB I . No pixel noise. Coordinate axes labeled in meters.

-0.2 0 0.2 0 ' 4 0:6 0 .8 I 1.2

Fig. 5 . Half-apeed trajectory using h = 0.95 estimation of DB. No pixel noise. Coordinate axes labeled in meters.

The following table shows the average values of E as calculated for the cases illustrated in Figs. 4-6.

Error E

Fig 4 0 17

Fig 6 0 78 Fig 5 0 21

The trajectory-following may be improved using two approaches. Adjusting the value of A downward implies that fewer points are used in the least squares estimation and an improved local linear fit to the inverse Jacobian can be obtained, however. it is important to realize that the results described so far are with no pixel noise present. Low values of h are typically susceptible to observation noise since the estimation is averaged over fewer points. Increasing this factor to h = I gives traditional least squares estimation. with all observations equally weighted. In this case, a poor result is to be expected since the algorithm is no longer discarding old data points as the target moves on to regions in which a different set of transformation parameters applies.

In the second approach. reducing the speed at which the trajectory is followed can also be used to improve tracking. In Fig. 5 , the trajectory is followed at half speed. At a lower speed, the optimum

-0.1 0 0.1 0:4 0.6 0.8 I 1.1

Fig. 7. Full-speed trajectory using h = 0.95 e.\tirnation of DB. 1024 x 1024 pixels. Coordinate axes laheled in meters.

value of h had to be increased to 0.95 since the number of data points (assuming constant sampling rate) for approximately the same region of linearization is increased. As can be seen. the value off? is close to that for the parameters of Fig. 3 .

In Fig. 6 the inverse ( D E ' ) estimation procedure is illustrated. Notice that the performance is less accurate than provided by direct estimation of the matrix DB under similar conditions and that the error in the Jacobian estimation is considerably increased. In particular. the circular arc portion is poorly followed.

It is the effect of pixel noise, introduced in Figs. 7 and 8. that clearly distinguishes the two estimation algorithms. Fig. 7 shows the breakdown of the forward ( D B ) estimation procedure in the presence of pixel noise. The vision system is simulated by dividing the entire operating region (4 m x 4 m) of the manipulator into a rectangular array of 1024 x 1024 pixels and representing each endpoint position by the center of the pixel in which the endpoint lies. The estimation scheme is unable to provide adequate kinematic parameters, and it was found that this effect was not improved significantly by adjusting the value of h or the endpoint speed.

In contrast. the performance of the inverse estimation procedure. as shown in Fig. 8. is quite tolerable with pixel noise present. Although the errors remain significant. it is clear that in applications where noise is present. this estimation procedure is to be preferred. Nor should this be unexpected since the noise is predominantly an error in the Cartesian measurements rather than the angle measure- ments.

108

-0 .4

-0.6

-0.1

- I

-1.1

-1.4

-1.6 2 0 0 2 0 4 O b 0 8 I

Fig. 8. Full-speed trajectory using h = 0.95 estimation of DE-I . 1024 x 1024 pixels. Coordinate axes labeled in meters.

IEEE JOURNAL O F ROBOTICS AND AUTOMATION, VOL. 4, NO. I , FEBRUARY 1988

DISCUSSION Although the implementation of the adaptive transformation

consumes somewhat more arithmetic operations (3 1 multiply opera- tions and 24 adds with no trigonometric function evaluation) than the inverse Jacobian (ten multiplies, five adds. five trigonometric functions), it has at least two advantages. The inverse Jacobian does not need to be derived. Furthermore, the direct estimation of inverse kinematics bypasses all need for accurate knowledge of the physical dimensions of the arm. Changes from one arm to another due to tolerances or resulting from the deformation of stressed flexible links will automatically be incorporated into the kinematic estimation.

It was found that the accuracy of the adaptive system depended critically on the value chosen for X (the forgetting factor) on the speed with which the target trajectory was traversed, and on the level of observation noise. The forgetting factor must be sufficiently less than unity to ensure that the set of observations used in estimating the inverse Jacobian was located within a region sufficiently small to allow linear approximation. At a fixed sampling rate, a low target velocity implies a large number of samples within a given region and a higher value of A.

Lower values of the forgetting factor lead to unsatisfactory parameter estimation when pixel noise is present. Thus in any situation, the correct value of X will involve a tradeoff between the speed of operation and endpoint sensor accuracy.

To obtain endpoint position data, a vision system or other endpoint sensor would be required. The results have indicated that a resolution of 1024 X 1024 is sufficient if the manipulator is to be observed over its whole range of motion. Note that the quantization is over a 4 m x 4 m grid representing the working region of the robot. In some applications only a restricted workspace may be necessary.

In situations where accurate endpoint measurements are available. the best linear estimates of the kinematic transformations are obtained using the forward estimation procedure. Although the inverse estimator is less satisfactory in approximating the nonlinear parame- ters when no noise is acting, its superior performance with pixel noise present is likely to make this the more practical technique.

The example used here to demonstrate the method involves only two degrees of freedom; however, the extension to the three- dimensional case involves no new concepts. If the normal six degrees of freedom are partitioned into two sets of three, the first set locating the endpoint in three-space and the second set defining wrist orientation, then two 3 x 3 Jacobians are involved. The nine entries of each Jacobian are identified using observations of incremental changes in the workspace coordinates as a result of incremental changes in the three joint angle coordinates.

It should also be noted that there is a rich literature of least squares algorithms designed to introduce numerical stability in awkward cases. The work described here could routinely be extended to incorporate methods such as UDU factorization.

REFERENCES K. Asakawa, F. Tabata, and H. Komoriya, "A robot with optical position feedback," in Proc. IEEE Ind. Appl. Soc., Oct. 1982, pp. 1276-128 1. J . Duffy, Analysis of Mechanisms and Robot Manipulators. New York: Wiley. 1980. P. Eyckhoff, System Idenfificarion. B. Frankovic and S. Petras, "An adaptive robotic system as part of a process control system." in Proc. IFAC Control Problems and Devices, Budapest, Hungary. 1980, pp. 281-290. "Photosensitive devices S 1200 and S 1300 data sheet," Hammamatsu TV Co. Ltd.. Japan. A. Koivo and T. H. Guo. "Adaptive linear controller for robotic manipulators," IEEE Trans. Aurornar. Contr., vol. AC-28. p. 162. Feb. 1983. A. J . Koivo, "Self-tuning manipulator control in Cartesian base coordinate system." Trans. ASME J . Dynamic Syst. Meas. Conrr., vol. 107, pp. 316-323, 1985. C. F. Lin. "Advanced controller design for robot arms.'' IEEE Trans. Auromat. Conrr., vol. AC-29. p. 350. Apr. 1984. Y. Nahamura and H. Hanafusa. "A new optical proximity sensor for three dimensional autonomous trajectory control of robot manipula- tors," in Proc. Int. Cony. Advanced Robotics, Robotics Society of Japan, Sept. 1983. pp. 179-186. R . P. Paul. Robot Manipularors: Mathematics, Programming, and Conrrol. H. B. Verbruggen and H. W. Klessrr. "Estimation methods for static parameters," Delft Univ. Technol.. The Netherlands. 1979. M. Vukobratovic and V Potkonjak. "Dynamics of manipulator robots, theory and applications." i n Scientific Fundamentals of Robotics I. New York: Springer-Verlag. 1982. D. E. Whitneq . "The mathematics of coordinated control of prosthetic arm5 and manipulators." Tram. ASME J . Dynamic Sysf. Meas., Conrr., pp. 303-309. Dec. 1972. Y. Yamada. F. Matsuda. N. Tsuchida, and M Ueda. "Control of an industrial arm to grasp moving objects with the aid of a two- dimensional range sensor." in Proc. Inr . C'onf. Advanced Robotics, Robotics Soc. of Japan. Sept. 1983. pp. 359-366.

New York: Wiley. 1974.

Cambridge. MA: MIT Prcss. 1981.

An Agile Stereo Camera System for Flexible Image Acquisition

ERIC KROTKOV, FILIP FUMA. AND JOHN SUMMERS

Abstract-An agile stereo camera system designed for flexible image acquisition under a wide variety of viewing conditions and scenes is presented. A host processor sends commands to three microprocessors controlling, through 11 servomotors, the position and optical parameters of the cameras, and the scene illumination.

Manuscript received December 8, 1986; revised June 30, 1987. This work was supported in part by NSFIDCR, the US Air Force, DARPAIONR, the US Army, NSF-CER. DEC Corporation. IBM Corporation, and LORD Corpora- tion. This communication was presented in part at the International Confer- ence on Pattern Recognition, Paris, France, October 1986.

E. Krotkov is with the Robotics and Artificial Intelligence Group. L.A.A.S. du C.N.R.S., 7, Avenue du Colonel Roche, 31077 Toulouse Cedex, France.

F. Fuma and .I. Summers are with the GRASP Laboratory, Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA 19104-6389.

IEEE Log Number 87 18063.

08824967/88/02OO-0108$01 .OO 0 1988 IEEE