Biol. Cybern. 92, 229–240 (2005)DOI 10.1007/s00422-005-0542-6© Springer-Verlag 2005

Threading neural feedforward into a mechanical spring:How biology exploits physics in limb control

Karl T. Kalveram1, Thomas Schinauer1, Steffen Beirle2, Stefanie Richter3, Petra Jansen-Osmann1

1Department of Cybernetical Psychology and Psychobiology, University of Duesseldorf, Universitatsstr.1,40225 Dusseldorf, Germany2 Institute of Environmental Physics, Im Neuenheimer Feld 229, 69120 Heidelberg, Germany3 Institute of Aerospace Medicine, German Aerospace Centre DLR e.V., Linder Hohe, 51147 Cologne, Germany

Received: 3 December 2003 / Accepted : 14 December 2004 / Published online: 14 March 2005

Abstract. A solution is proposed of the hitherto unsolvedproblem as to how neural feedforward through inversemodelling and negative feedback realised by a mechani-cal spring can be combined to achieve a highly effectivecontrol of limb movement. The revised spring approachthat we suggest does not require forward modelling andproduces simulated data which are as close as possibleto experimental human data. Control models based onperipheral sensing with forward modelling, which arefavoured in the current literature, fail to create such data.Our approach suggests that current views on motor con-trol and learning should be revisited.

1 Introduction

Recent reviews (Kawato 1999; Sabes 2000) reveal thatthe currently favoured concept of movement control typi-cally includes negative feedback control based on periph-eral sensing, a forward model which bridges the delay inthe feedback lines, feedforward control through an in-verse model which gets its state input also via peripheralsensing and forward modelling, and neural merging offeedback and feedforward signals. The concept abandonsthe assumption that the stretch reflex (Sherrington 1906;Marsden et al. 1972; Vallbo 1974; Nichols and Houk 1976)plays an essential role in movement control, but it pre-serves the fundamental idea of negative feedback controltraced back to explicit peripheral measurements and fur-ther adds the notion of adaptive inverse control (Widrowand Walach 1996; Neilson et al. 1997). The concept alsoincludes the acquisition and tuning of the inverse model byan algorithm going through the forward model (Wolpertet al. 1995). Even a candidate for the anatomical loca-tion of both internal models is currently in discussion:the cerebellum (Wolpert et al. 1998; Spoelstra et al. 2000;Timmann et al. 2000). Thus the concept, as sketched inFig. 1, suggests an integrative approach to motor control,

Correspondence to: K. T. Kalveram(e-mail: kalveram@uni-duesseldorf.de)

expressed in terms of cybernetics, uses predicted measure-ments based on explicit peripheral sensing, and is widelyaccepted (Kalveram 1992; Stroeve 1997; Kalveram 1998;Wolpert and Kawato 1998; Bhushan and Shadmehr 1999;Kawato 1999; Desmurget and Grafton 2000; Flanaganet al. 2003), though not completely uncontroversial (Os-try and Feldman 2003).

Nevertheless, the peripheral sensing approach outlinedabove neglects to mention that the mechanical proper-ties of the limb-muscle system itself can also establisha negative feedback control loop which, however, doesnot demand explicit peripheral measurements (Hill 1938;Asatryan and Feldman 1965; Bizzi et al. 1976; Polit andBizzi 1979; Sternad 2002; Sainburg et al. 2003). In this‘equilibrium point hypothesis’, the loop gain is providedby the mechanical joint stiffness, and the setpoint by thejoint’s mechanical equilibrium position (this is where allmuscularly induced forces acting on the limb add up tozero, and where the limb comes to rest if persistent externalforces are absent). Assuming that this suffices for move-ment control, a goal-directed movement could be per-formed by simply turning the intended position of thelimb into the equilibrium position to be attained next. Thetask dynamics approach (Saltzman and Kelso 1987) or thespinal force field hypothesis (Mussa-Ivaldi and Bizzi 2000)extends this equilibrium point hypothesis (Feldman 1966)to limbs with several joints and to goals located on planesor even in space. Again, the basic idea is that the nervoussystem specifies muscular forces which superimpose ontoa force field having an equilibrium point at the goal posi-tion such that the limb’s position is automatically drivento – or, loosely speaking, attracted to – this location, inde-pendent of the starting position. At first glance, neitheran inverse nor a forward model seems necessary, so thatthe equilibrium point hypothesis appears to be the pointof departure in modelling motor behaviour (Bizzi et al.1992).

However, if gravitational or other persistent externalforces are acting upon the limb, equilibrium positionand actual resting position deviate considerably. Assum-ing for instance an angular setpoint of 90◦ to the direc-tion of gravity, a human forearm (mass = 1 kg placed at

230

Fig. 1. The peripheral sensing approach, expressed as a schematicdiagram of the internal models used in controlling a multijoint arm.The desired motion of the arm X* is fed into an inverse model of thearm, which acts as a feedforward controller, producing a commandsignal Uff . There is also evidence of late influence from a feedbackcontrol signal, Ufb. The two commands are combined, through sim-ple addition perhaps, to yield the final control signal U. The feedbackpathway is illustrated in grey to reflect the fact that it plays a sub-ordinate role as a result of feedback. (NB: The word ‘subordinate’

indicates that the peripherally sensed signals cannot be directly usedfor motor control, only via forward modelling). The true state of thearm, X, is estimated with a combination of the visual and proprio-ceptive feedback. The state estimate, X′, serves as input to the inversemodel and is also compared to the desired state to yield an estimate ofthe current motor error (E′). The latter signal is used both in feedbackcontrol and to drive adaptation of the inverse model (dashed arrow).(See Sabes (2000, p. 742). Figure 1 modified from Sabes (2000))

a distance of 0.25 m from the elbow, stiffness around theelbow = 10 N/rad as we found it to be the average value)will decline from this setpoint by about 15◦. This severeinaccuracy could be removed by an inverse model, as inthe peripheral sensing approach. However, this causes theproblem that the feedforward signal provided by the in-verse model is given in neural terms, whereas the feedbacksignal processed by the spring to establish negative feed-back control is given in mechanical terms, that is to say,by Hooke’s law. How these incompatible signals could bemerged to attain a uniform control signal remains an openquestion.

The revised spring model, which we offer in Fig. 2, solvesthis problem. In this model, we complete the equilibriumpoint hypothesis by an inverse model and a neuromechan-ical interface capable of merging the feedback and feed-forward signals mechanically, yet we leave out a forwardmodel.

The single-lined arrows in Fig. 2 represent time-dependent continuous variables which describe movementexecution, whereas double-lined arrows stand for sam-pled data being constant during a movement and refer tomovement planning. The part called ‘execution’, therefore,corresponds to the processes depicted in Fig. 1. In eachcase, planning of a reaching movement starts by choos-ing a desired end position, also known as the movementgoal. Next, a trajectory has to be selected which servesas a desired pathway between the start point and the goal.Therefore, in our case it is necessary to have a device whichchanges the angular distance to be passed over into anappropriate and temporally continuous sequence of de-sired angular positions ϕd. In technical terms, a deviceaccomplishing such a streaming function is called a paral-

lel-to-serial converter. We ascribe this streaming functionto a pattern generator. In biology, central pattern gener-ating is viewed as a basic principle for the organisationof rhythmic behaviour, regardless of whether one con-siders swimming in fish (Holst 1939), walking in mam-mals (Forssberg et al. 1980), or mastication of food in thelobster’s gastric mill region (Miller and Selverston 1985).Through efferent pathways from other regions of the brainsuch a pattern-generating neural system can be switchedon and off; in the latter case it may also be after exactlyone period. At the behavioural level, patterns emitted by acentral pattern generator manifest as ‘automatised’ move-ments distinguishable in terms of intensity, period length,and/or velocity. Nevertheless, from the perspective of thebehaviourally oriented biologist these movements appearas if they were of ‘constant form’ (Eibl-Eibesfeld 1987).Here, we define the pattern generator as a functional unitengaged in planning. It obtains the distance intended to bepassed over as an input value and, when triggered, emitsthe complete desired kinematics (=ϕd and its first and sec-ond derivative) leading the limb to the goal (Kalveram1991). However, we do not insist on a particular ana-tomical location of that unit. In movement execution, theinverse model then receives the stream of the desired kine-matics produced by that pattern generator as input andputs out a stream of torques Qff to be fed forward.

The neuromechanical interface in the middle of the exe-cution stage in Fig. 2 is the pivot of our concept. Its task isto thread the neurally coded feedforward torque Qff intothe mechanical spring and to make the desired angularposition ϕd the reference value of the mechanically realisednegative feedback loop. Figure 2 depicts how the neurome-chanical interface solves this task: at the neural side, first

231

Fig. 2. Revised spring model. In contrast to the peripheral sensingapproach sketched in Fig. 1, here the negative feedback loop is rea-lised through a mechanical spring which operates with vanishing timelag and, of course, without noise in the feedback line. X is replaced bythe joint angle ϕ, and U by the joint torque Q, while the feedback con-troller is functionally given by a scalar gain factor K representing thejoint stiffness. The neuromechanical interface suggests how mergingof feedforward (Qff ) and feedback (Qfb) can be realised mechanicallysuch that the desired position ϕd – and not the equilibrium positionϕ0 – becomes the reference (setpoint) for negative feedback control.The grey-coloured right-hand part represents the arm’s spring prop-

erty. K ′ and ϕ′0 are transformed into the mechanical values K and

ϕ0 by neuromuscular interfaces which are indicated by the two smallgrey boxes (for details see (3)). The model is completed with a plan-ning section containing a pattern generator which transforms thedifference between the planned angular end position and the actualangular start position into a desired temporal kinematic pattern (=trajectory ϕd and its first and second derivative). These trajectoriesprovide the command input (X∗ in Fig. 1) and state inputs into theinverse arm model. The arrangement shown in Fig. 2 allows one toomit entirely a forward model (see text for more information)

the equilibrium position ϕ′0 to be transferred to the arm is

determined by ϕ′0 =Qff/K

′ +ϕd, where K ′ denotes a freelyselected value for the arm’s stiffness. K ′ and ϕ′

0 are thentransformed into the respective mechanical values K andϕ0 by neuromuscular interfaces, as indicated by the twosmall grey boxes (the transformations are not explicitlyshown in Fig. 2, but see (3)). According to Hooke’s law, themuscularly generated torque then becomesQ=K(ϕ0 −ϕ).

Taking ϕ0 for ϕ′0 and K for K ′ yields ϕ0 =Qff/K +ϕd,

so Q = K(Qff/K + ϕd − ϕ) = Qff + K(ϕd − ϕ). The termK(ϕd − ϕ) describes the output – here called Qfb – of anegative feedback controller which uses ϕd as the setpoint.Therefore, though ϕ0 remains the equilibrium positionof the arm, it is not taken as the reference for negativefeedback control as is assumed in the equilibrium pointhypothesis.

Signal processing by the neuromechanical interfacealso explains the experimental findings that mechanicalstiffness and mechanical angular equilibrium position canvary independently of each other (Latash 1992) and evenallows one to consider the stream of equilibrium posi-tions as something like a ‘virtual trajectory’ (Hogan 1985),though this trajectory is not centrally determined as orig-inally proposed by Hogan but computed at the very endof the neural processing.

In Fig. 2, negative feedback control operates withouta delay in the feedback line. Additionally, the state inputof the inverse model is generated by the pattern genera-tor – and not, as suggested in Fig. 1, fed back from the

periphery via a forward model. Both facts make a forwardmodel dispensable. Therefore, our revised spring modelconforms neither to the equilibrium point hypothesis andits derivatives nor to the control model based on periph-eral sensing with forward modelling.

The purpose of this paper is to check which data – thosecreated by the peripheral sensing approach or those by therevised spring model – are closer to experimental humanmovement data.

2 Methods

The rationale of our method is to administer disturbingtorque impulses to the forearm of real subjects and to com-pare the kinematic records with simulated data. For thesesimulations, first the parameters of each subject’s forearmwere estimated and fed into the arm model being con-trolled according to the control model of interest. Thensimulated impulses were applied. The peripheral sensingconcept sketched in Fig. 1 is comprised of several differentaspects which are covered by three control models calledmodel 1a, model 1b, and model 1c. The revised springapproach of Fig. 2 is addressed by control model 2. Thissection includes a description of the experiment, the for-mulas, and procedures used to get the necessary param-eters for the simulations, while the next section concernsthe evaluation of the four control models 1a, 1b, 1c, and2 with respect to the human controller.

232

Fig. 3. Experimental setup. Subjects sat in an adjustable chair fac-ing a head-centred concave screen 1.5 m in front of them. Their rightforearms were inserted into an orthosis which was attached to a leverfixed on the axis of a torque motor. The axis of the motor was locatedunderneath the elbow joint. The size of the orthosis was adjustedaccording to each subject’s arm anthropometrics to ensure a secureand tight fit. That allowed flexion-extension movements of the fore-arm, but only in the horizontal plane. Throughout each block oftrials, the motor exerted a constant torque QC (Table 1). The angleof the forearm was measured with reference to the upper arm. Thedirection straight ahead and perpendicular to the upper arm wasdefined as zero. The joint angle was then transferred to a head-cen-tred flashlamp which projected the feedback marker on the screen.When viewing the feedback marker, therefore, the visual gaze anglewas congruent with the joint angle. During movement the feedbackmarker was darkened. The target marker could be placed either on theleft side (at +0.35 rad = +20◦) or the right side (at −0.35 rad = −20◦)of the screen. While subjects made a reaching movement towards thetarget marker, a perturbing torque impulse – short enough to preventadaptation – could be administered

2.1 The experiment

2.1.1 Experimental setup. Nine right-handed subjects(four female, five male) participated in the study. Agesranged between 19 years and 50 years (m= 32 years). Allparticipants gave written informed consent prior to takingpart in the study. The experimental procedures were ap-proved by the local ethics committee. See Fig. 3 for detailsof the setup. Control software used to drive the torque mo-tor (Mattke MC27P with amplifier MRL150/40) and themarker lights was based on the MATLAB technical com-puting language, Simulink, and Real-Time Workshop (allby The Mathworks, Natick, MA, USA). Angular position,velocity and acceleration were measured by a potentiom-eter, a tachometer and an accelerometer, which were fixedon the motor shaft and/or on the lever. The data weresampled at 500 Hz and digitised with a 12-bit analogue-to-digital converter (Meilhaus ME300). Digital data weresaved on hard disk after completion of a block.

2.1.2 Experimental procedure. The experiment startedwith three training blocks meant to familiarise the sub-jects with the hardware and the procedure, followed byfive experimental blocks (Table 1). One block included 42trials, each of which lasted 3 s. Between the blocks a pauseof about 60 s was inserted. The complete experiment took

about 30 min. At the beginning of a trial, the target jumpedalternatively either from the left (+0.35 rad = +20◦) to theright (−0.35 rad =−20◦) or vice versa. The laboratorywas darkened during the experiment. Subjects were in-structed to perform a goal-directed forearm movement– either by flexion or extension – to the requested tar-get position (Fig. 3). Reacting as quickly as possible wasnot emphasised, but the participants were asked to moveaccurately and at a quick pace once they had started mov-ing. They were also requested to stay relaxed and to bringthe ongoing movement to an end if they experienced aperturbation. An online operating motion detector sig-nalled movement if the absolute velocity exceeded 2◦/s.The feedback arrow was darkened 0.1 s after the detectionof movement onset and brightened again 0.1 s after detec-tion of a standstill, the related position of which we calledthe ‘first stop position’. The reason for this blacking outwas to prevent the subject from receiving visual feedbackduring the transport phase of the movement, and thus toprevent early corrective movements. However, after thesubject had stopped and visual feedback had reappeared,opportunity to make a corrective movement was given.

Because the forearm moved horizontally, gravity hadno effect on the movements. In order to mimic gravity, themotor could exert a constant torque QC on the forearm(Table 1). This dislocated the forearm’s angular restingposition from the equilibrium position (see (1) and thedescription of the variables).

In each block, a small number of trials (4 in trainingblocks 2 and 3, 12 in the experimental blocks) were singledout in which the motor generated a disturbing torque im-pulse with an amplitude of 5 N m and a duration of 50 msin addition to the constant torque, either in or against theactual movement direction. The torque impulse was trig-gered when the actual position crossed the ‘10◦ criterion’,in other words deviated 10◦ from the start position. Onaverage, this happened about 250 ms after onset of move-ment. It should be mentioned that the mean target errorof the movements at first stop position was independentfrom grade and direction of the constant torque as well asfrom application of a disturbing impulse.

2.1.3 Data processing and normalising. Kinematic datawere filtered offline by a fourth-order recursive Butter-worth low-pass filter with a cut-off frequency of 10 Hz.Due to considerable variability, all curves were first alignedat movement onset as determined by the motion detec-tor mentioned above and then normalised with respect totime and movement amplitude dA (= position at first stopminus start position). The normalised positional trajec-tory began at the zero position, crossed the 10◦ criterionpoint 250 ms after movement onset, and attained +1 or−1 at the time point of first stop, while normalised veloc-ity and acceleration curves had values which producedthis positional trajectory precisely. When visual feedbackreappeared after the first stop, subjects often performed acorrective movement such that the normalised positionaltrajectory drew away from +1 or −1. We denoted all nor-malised kinematics of actual disturbed movements again

233

Table 1. Application of constant torques in different experimental blocks

Training blocks Experimental blocks

Block number 1 2 3 1 2 3 4 5

Constant torque QC [N m] 0 +0.5 −0.5 +0.75 +1.5 0 −0.75 −1.5

by ϕ and its derivatives and those of undisturbed move-ments by ϕu and its derivatives.

2.1.4 Determination of spring parameters. For each dis-turbed movement, the spring parameters (inertia J , coeffi-cient of viscous damping B, and stiffness K; see (1) below)were determined (see (4)–(7) below) using the normalisedkinematics of a movement. The mean values attained bythe subjects were: K = 5.2 to 12.1 [N m/rad], B = 0.32 to0.80 [N m s/rad], J = 0.067 to 0.21 [kg m2] (sum of armand lever; lever alone: 0.045 kg m2). These values lie withinthe range reported by others as well (Bennett et al. 1992;Gomi and Kawato 1997). The stretch/squeeze factors fused for normalisation ranged from 0.89 to 1.4. Meansand standard deviations of J , B, K, and f per subject aregiven in Table 2. The measurement procedure of J , B, Kis described in detail in the following subsection.

2.2 Formulas and procedures

The dynamics of an arm with one degree of freedom aregoverned by the differential equation

J · ϕ︸︷︷︸

QJ

=K · (ϕ0 −ϕ)︸ ︷︷ ︸

Q

−B · ϕ︸︷︷︸

QB

+G · sin (ϕ −ϕg)︸ ︷︷ ︸

QG

−QX , (1)

where ϕ0, ϕ, and ϕ denote respectively time-dependentactual angular position, velocity, and acceleration; J isthe moment of inertia related to the centre of rotation; Kis the coefficient of joint stiffness; B is the coefficient of vis-cous damping; G is defined as mag, where m denotes themass, a the distance between the centres of mass and rota-tion, and g the gravitational constant; QJ , Q, QB , QG, andQX are torques generated respectively by inertial, elastic,viscous, and external forces which include gravitational(QG) and disturbing (QX) forces; ϕg is the angle betweenthe gravitational field and the body reference of the limb(here, ϕg = 0 is assumed); ϕ0 is the angular ‘equilibriumposition’ the joint angle ϕ will attain after a while if theexternal torques QG and QX vanish. In contrast, the jointangle finally attained with non-vanishing external torquesis called the angular ‘resting position’.

Note: In the experiment the forearm moved horizon-tally, so gravity had no effect on the movements. In order tomimic gravity, the motor exerted a constant torque calledQC on the forearm (Table 1), which persistently dislocatedthe forearm’s angular resting position from the equilib-rium position. This feature can simply be taken into ac-count by replacing in (1) the whole term QG with QC .

B and K are allowed to vary with time and/or angu-lar position. Active movements can be executed by shift-ing the equilibrium position ϕ0 away from its momentaryvalue, which generates a torque Q driving the arm towardsthe (new) equilibrium position.

Equation (1) is arranged to represent the arm’s forwarddynamics after division by J . In simulations, (1) can besolved online given that the parameters J , K, B, G andthe variables ϕ0, QX are known. The kinematic state is thenyielded by integrating the angular acceleration twice in amanner similar to that shown in Fig. 5 by an analogue rep-resentation. A forward model of the arm is obtained like-wise if the physical parameters and variables are replacedby their neural representations. An inverse arm model (see(2)) can be deduced from (1) if the actual kinematics are re-placed by the desired kinematics (denoted by the subscript‘d’). The desired command variable (here acceleration) issupplied by the pattern generator. The desired kinematicstate variables can be taken either from the pattern genera-tor or from peripheral sensing via the forward model. Theneural counterparts of the physical parameters J , B, andG are marked by superscripts ‘ ′ ’. They must be acquiredby learning, but this is not an issue of the present paper.Rearrangement of terms then yields the inverse model:Qff =K ′ · (ϕ′

0 −ϕd)=J ′ · ϕd +B ′ · ϕd−G′ · sin(ϕd −ϕ′

g)︸ ︷︷ ︸

Q′G

.(2)

To take account of the present experiment, again the wholeterm Q′

G must be replaced with Q′C (see (1) and the note

below the description of the variables).A simple neuromuscular interface similar to the alpha

model (Bizzi et al. 1992) is given by

joint stiffness : K =k(n1 +n2) ,

angular equilibrium position : ϕ0 = r(n1 −n2)/K ,

coefficient of viscous damping (proposed) : B =bK ,(3)

where n1, n2 reflect the numbers of motor units recruited inthe respective muscles of the antagonistic pair and k, r, andb denote appropriate positive constants. Viscous dampingwas originally not included in the alpha model but couldbe concluded from Hill’s model as being roughly propor-tional to K. More detailed muscle models which also takenon-linearities and differences in the muscular tissues intoaccount do exist (Winters 1995; Seyfarth et al. 2000), butthis simple model suffices in showing the basic idea.

The arm’s spring parameters valid in a single disturbedmovement were obtained using (1) as follows:

J · ϕ +B · ϕ +K · (ϕ −ϕ0)=QC +QX , (4)J · ϕu +B · ϕu +K · (ϕu −ϕ0)=QC , (5)

234

Table 2. Intra-individual means and standard deviations of stiffness K, viscosity B, inertia J , and stretch/squeeze factor f . Dimensions ofK, B and J are N m/rad, N m s/rad and N m s2/rad

Subject 1 2 3 4 5 6 7 8 9 Total

K Mean 12.0291 6.1858 6.9352 8.7832 9.9779 5.1901 5.1568 5.4440 8.2237 7.5473STD 1.7714 0.6056 0.2680 0.8570 0.5972 0.8644 0.3541 0.4428 0.5108 0.6968

B Mean 0.4834 0.3203 0.5080 0.4353 0.4523 0.4111 0.3828 0.3392 0.8018 0.4594STD 0.0468 0.0464 0.0181 0.0320 0.0583 0.0499 0.0497 0.0338 0.0842 0.0466

J Mean 0.1225 0.0717 0.0775 0.0925 0.1078 0.0670 0.0974 0.0864 0.2115 0.1027STD 0.0040 0.0047 0.0123 0.0176 0.0050 0.0065 0.0087 0.0060 0.0190 0.0093

f Mean 1.0837 0.9147 0.9703 1.0278 0.9693 0.8936 1.0770 0.9875 1.4090 1.0370STD 0.0468 0.0220 0.0554 0.0698 0.0168 0.0393 0.0272 0.0321 0.0361 0.0384

where (4) holds for a perturbed movement and (5) holdsfor the course which the arm would have taken if it hadbeen controlled identically, but without experiencing theperturbation. ϕ, ϕ, and ϕ denote the actually recordedkinematics of a movement (see experimental setup). QC

denotes the constant torque applied in the experiment,which mimicked the gravitationally induced torque QG

introduced in (1). ϕu and its derivatives describe the fic-tive angular kinematics of the arm if it were unperturbed.We estimated ϕu and its derivatives by averaging, sepa-rately with respect to extension and flexion, the norma-lised unperturbed movements in a block. Presuming thatthe perturbation leaves the spring parameters unchanged,(5) can be subtracted from (4):

J · (ϕ − ϕu)+B · (ϕ − ϕu)+K · (ϕ −ϕu)=QX . (6)

When turning to the normalised trajectories and applyingmatrix notation, (6) can then be written as

{ϕ − ϕu ϕ − ϕu ϕ −ϕu} •{

J∗B∗K∗

}

={

Q∗X

}

. (7)

The values between the left braces in (7) have to beinterpreted as an m × 3 matrix, whereby each of the mrow vectors represents one sample of differences betweenthe normalised measurements of a disturbed movementand the related assessed undisturbed movement. The left-hand three-element column vector contains the unknownswhich are the values for the coefficients of inertia,damping, and stiffness. The m-element column vectoron the right-hand side of (7) represents the normalisedtorque impulse. Due to normalisation (f is the tempo-ral stretch/squeeze factor and dA the movement ampli-tude; see data processing and normalising), these variablestransform to

J ∗=J ·f 2, B∗=B ·f, K∗=K, Q∗X =QX/(f ·dA) .

(8)

The sample matrix selected in this investigation contained300 samples and spanned the normalised interval fromt =200 ms (movement start) to t = 800 ms (whereaboutsthe real undisturbed prototype movement would havecome to an end). Equation (7) is an overdetermined systemof linear equations solvable for J ∗, B∗, and K∗ by the leastmean squares (LMS) method if the rank of the sample ma-trix is three. This was fulfilled in all cases. The procedure

should allow the assessment of the arm’s spring parame-ters to be valid in a single disturbed movement if they areapproximately constant during the time of measurement.If they are not constant, the procedure will acquire tem-poral means of the spring parameters. We took the LMSmethod from MATLAB.

Whether all these considerations indeed prove to be truewill be evident if the computed three parameters suffice toreproduce the related actual movement in the simulation.

2.3 Preparing the control models for numerical simulation

For the purpose of numerical simulation, the peripheralsensing approach must be formulated more precisely thanin Fig. 1 and must also be tailored to the dynamics of aone-jointed arm. To make the approach comparable tomodel 2, it is also necessary to add a pattern generatorand to take over the denotations of model 2. All this isdone in Fig. 4.

As mentioned above, the peripheral sensing conceptgiven in Fig. 1 (resp. Fig. 4) can be simulated in differentways, depending on, for instance, how the forward modelis realised or where the kinematic state input of the inversemodel is coming from. Therefore, to highlight the effectsof those different features, the concept shall be coveredby three models denoted by 1a, 1b, and 1c. Model 2 thenreflects the revised spring model outlined in Fig. 2. Spe-cifically:

Model 1a interprets the peripheral measurementschema depicted in Fig. 1 (resp. Fig. 4) to the largest possi-ble extent. The added pattern generator produces the de-sired angular kinematics, the desired acceleration of whichis used as the command input into the inverse model andthe desired position as the reference for negative feedbackcontrol. The time lag caused by delayed proprioceptivesensing is set to 50 ms (25 ms for the afferent and efferentneural running times, plus at least 25 ms for the neuro-muscular-mechanical transfer). A Smith predictor (Smith1957; Miall et al. 1993) as outlined in Fig. 5 serves to sus-tain the forward model, an arrangement which is supposedto create a nearly perfect prediction of the arm’s output ifthe movement is undisturbed. Predicted angular positionis then used for negative feedback control, while predictedangular position and velocity supply the inverse model’sstate input.

235

Fig. 4. The peripheral sensing approach as described in Fig. 1, butcompleted by a pattern generator and tailored to the dynamics of aone-jointed arm. The feedback controller is given by the gain K of thenegative feedback loop. Learning circuitry has been omitted becauselearning is not an issue addressed by the paper. To make the peripheralsensing approach (which comprises models 1a, 1b, and 1c) compa-

rable to model 2 with respect to the denotation, the command signalX∗ in Fig. 1 has been replaced with ϕd, the actual arm output X withϕ, ϕ, ϕ, the predicted (estimated) arm output X′ with ϕp, ϕp, ϕp, andthe command signals Uff , Ufb,U with Qff , Qfb, Q. The state feedbackis given by ϕp, ϕp. See text for additional information

Fig. 5. Forward model of arm dynamics. The figure shows in ana-logue representation that the forward model operates through onlinesolving of (1) by integration. In contrast to pure forward dynam-ics, two loops are added which include the temporal delay unitsdt. This enables the output of the integrators to readapt to the ac-tual kinematic state of the arm in case of disturbance. For that, theloop delays must be equal to the delays of the state measurements(Kalveram 1998). In the presented simulations, the loop gain g is

limited to 5. This value achieves optimal accuracy without destabil-ising control. The forward model is embedded into negative feedbackcontrol, which is additionally sustained by a Smith prediction loop(Miall et al. 1993). However, the latter can also be omitted at theexpense of accuracy. Notice that variables, parameters, and compu-tational units used in the model are neural representations of thevariables, parameters, and arithmetic functions physically defined in(1) and must be acquired by learning

236

Model 1b is quite similar to model 1a, except for twofeatures:

(1) The supply of the inverse model’s kinematic state in-put is taken, not from the periphery, but from thepattern generator. Thus, state feedback is replacedby state feedforward. But negative feedback is not al-tered and continues to be based on predicted angularposition.

(2) In the forward model the Smith predictor (Fig. 5) isdisconnected.

Model 1c is identical to model 1b in all features exceptthe forward model: the Smith predictor is reconnected.

Model 2 refers to the revised spring concept of Fig. 2:negative feedback control is based on the mechanicalspring property of the muscle-limb system, feedback andfeedforward torques are merged mechanically using theneuromechanical interface, and the state input of the in-verse model is provided by the pattern generator. A for-ward model of the arm dynamics is not necessary.

2.4 Movement simulation

Prior to simulation, the parameters of the arm (J , B,and K) were measured by the algorithm described in(4)–(8) and then inserted into the arm dynamics (1) andits inverse model (2). To create simulated disturbed move-ments, the arm dynamics were then computed accordingto (1) and the inverse model according to (2). The averagedundisturbed kinematics (ϕu,ϕu, ϕu) per subject, block, andmovement direction (see (5)) were taken as the respectivedesired kinematics thought to be emitted by the patterngenerator according to Fig. 2 (resp. Fig. 4). This is jus-tified because the negative feedback signal Qfb vanishesif the inverse model is correct, so that desired and actualpositions should coincide under undisturbed conditions.The constant torque mimicking gravity and the impulsetorque were applied to the simulated arm dynamics asin the experiment. In simulated movements, final brakingwas triggered as follows according to a prior finding (Kon-czak et al. 1999): stiffness K and the coefficient of viscousdamping B applied in the arm model underwent a rampincrease between 0.8 s and 0.9 s of movement time withrising rates of 25 K/s (resp. 50 B/s) such that they reachedthe levels 3.5 K and 6 B at 0.9 s. The neurally representedcoefficient of viscous damping in the inverse model (2),however, retained the originally inserted value B ′.

3 Results

Figure 6 demonstrates the effects of the disturbing torqueimpulses on the positional trajectories of some ongoingreal and simulated movements. The trajectories generatedby control model 2, which reflects the spring approach,obviously track the real data the closest, followed by con-trol model 1c, which is based on a forward model sustainedby the Smith prediction. Omitting the Smith prediction incontrol model 1b considerably increases the deviations.

Model 1a, though mirroring the ‘state of the art’ in theliterature, performs the worst.

In order to achieve a quantitative analysis, the nor-malised movement time was partitioned into five seg-ments, namely 200–450 ms, 450–800 ms, 800–1000 ms,1000–1200 ms, and 1200–1400 ms. The first segment cov-ered the undisturbed transport part of movement, the sec-ond the disturbed part. The last three segments enclosedthe ‘tail’ of the movement in which the limb came to rest.Not included in the analysis was the span between 0 andthe beginning of movement at 200 ms.

To get quantitative assessments regarding the goodnessof fit of the four control models under concern, we de-fined – with respect to a temporal segment – an ‘index ofmodel fit’ (IMF) for each control model. An IMF valuewas computed as follows: first, the mean absolute differ-ence between each real and the corresponding simulatedtrajectory was determined. This yielded 60 values per sub-ject (5 blocks times 12 disturbed movements per block).The average of these values was then taken as a subject’sIMF and the average of all subjects as the (overall) IMF– in the respective temporal segment. The lower such anIMF value is, the closer the fit. But an IMF value alsoincludes the random variation which occurs even if move-ments are executed under the same conditions. We quan-tified this random variation in a manner similar to theIMF outlined above and named it the ‘index of movementvariability’ (IMV): first, the mean absolute difference be-tween each real disturbed trajectory and the averaged realdisturbed trajectories accomplished under the same con-ditions (same movement direction, torque pulse direction,block) was computed. The average of these values over allconditions was then taken as a subject’s IMV value and theaverage among the subjects as the (overall) IMV – referringto the considered temporal segment. We regarded such anindex of movement variability as the baseline value, belowwhich an index of model fit can hardly fall, even if thedata produced by the model approximate the human dataas closely as possible.

Means and standard deviations of the segmental indi-ces of model fit and the corresponding indices of move-ment variability are shown in Fig. 7. Here, the main fea-tures already visible in Fig. 6 appear even more strikingly:in segment 2 (called pulse), the most critical temporal seg-ment in which the torque impulse is applied, as well as insegment 3 (called postpulse), the model 2 fit is the closestand is barely distinguishable from the movement variabil-ity which we consider as the baseline. Notice that in theprepulse segment all models exhibit a very good fit andalso perform equally well.

To statistically underpin these features, we decided tocompute five one-way analyses of variance (ANOVA), onefor each temporal segment. In each ANOVA, the indepen-dent variable was formally composed of five levels, and therelated enlisted values of the dependent variable includedthe indices of the fit of models 1a, 1b, 1c, and 2 and theindex of movement variability (see definitions above). Forall five temporal segments (numbered here from 1 to 5),the respective ANOVAs (SPSS) achieved significant effects(F -values: F1 = 28.8, F2 = 155.2, F3 = 129.4, F4 = 244.2,

237

Fig. 6. Normalised kinematics of four selected experimental move-ments (thick grey curves) and the related simulated trajectories refer-ring to control models 1a, 1b, 1c, and 2 (thin black curves). The ticson the x-axes indicate the points of time used to define the segmentsfor the computation of the segmental model errors. The circle at 0.2 smarks the movement onset, the cross at 0.45 s the impulse onset. Ticsand labels of the y-axes refer to normalised angular position. The

thin black rectangle marks the normalised disturbing torque impulse,and the thin grey curve the ‘desired’ positional trajectory which theactual movement would have tracked if it had been left undisturbed.The instances are taken from disturbed trials 4, 5, 6, 7 of experimen-tal block 2 of subject 6; however, the other subjects produced quitesimilar trajectories

F5 =294.7, with dfb=5−1=4 and dfw =45−5=40 for eachanalysis, and p1 to p5 <0.0001). Post hoc, at each segmentthe model 2 fit was compared to the movement variabil-ity and to the model 1c, 1b, and 1a fits. Notice that inthe following the subscripts 0, 1a, 1b, 1c attached to therespective p values refer to these comparisons, wherebythe subscript 0 belongs to the comparison with the move-ment variability. So we performed four (Bonferroni cor-rected) significance tests per segment. At segment 1, onlythe movement variability was found to differ significantlyfrom the model 2 fit (p0 <0.0001, p1a =0.499, p1b =0.649,p1c = 0.649). At segment 2 – which is the most criticalcomparison – the difference between the model 2 fit andthe movement variability is not significant (p0 = 0.111),whereas the model 2 fit significantly differs from the fitsof the other models (p1a to p1c <0.0001). The same holdsfor segment 3 (p0 =0.446, p1a <0.0001, p1b <0.0001, p1c <0.029). At segment 4, the model 2 fit becomes indiscerniblefrom movement variability and from the fits of models 1band 1c, while only the difference from the model 1a fitremains significant (p0 =0.137, p1a <0.0001, p1b =0.065,p1c = 0.366). The same holds for segment 5 (p0 = 0.602,p1a <0.0001, p1b =0.486, p1c =0.804).

4 Discussion

The quantitative analysis provides a clear-cut picture ofthe experimental results: prior to the disturbing impulse,the simulated data generated by control models 1a, 1b,1c, and 2 track the human data equally well and indis-criminately. Under the influence of the impulse, however,the behaviour of the models diverges. Related to the nat-ural movement variability, only model 2, which includesthe spring response and lacks a forward model, is as closeas possible to the human data. Models 1a, 1b, and 1c,which are based on delayed peripheral sensing and for-ward modelling, exhibit significantly greater deviationsfrom human data than model 2. Model 1a, which is pres-ently the favoured control model, performs the poorest ofall tested models. Here, the simulated movement does noteven converge to the low prepulse fit.

The revised spring model, though lacking a forwardmodel, operates much more effectively and precisely com-pared to models based on delayed peripheral sensing andforward modelling. Therefore, considered from an ecolog-ical perspective (Turvey et al. 1999), the manner of motorcontrol that we suggest may serve as an example showingthat the organism can achieve simple and effective controlstrategies by exploiting the physical properties of its envi-

238

Fig. 7. Means and standard deviations (SD) of indices of model fitand movement variability referring to the five temporal segments.Total lengths of the error bars refer to 2 SD. Models 1a, 1b, and 1crefer to the three different modifications of the peripheral measure-ment approach outlined in Fig. 1 (resp. Fig. 4). Model 2 refers tothe revised spring approach outlined in Fig. 2. An index of model fitreflects the mean absolute deviation of the real disturbed positional

trajectories from the trajectories simulated through the respectivemodel. An index of movement variability represents the natural ran-dom variation of the real disturbed positional trajectories. For eachtemporal segment, therefore, the index of movement variability canbe considered as a baseline value for the indices of model fit. See textfor more information

ronment or own body. Here, the organism exploits phys-ics through organising the limb into a mechanical spring,which provides an optimal negative feedback loop, andapplies then a neuromechanical interface to control thespring via feedforward through an inverse model. Merg-ing of feedforward and feedback through the proposedneuromechanical interface also makes the tuning of thespring’s stiffness and equilibrium position independent ofeach other. Therefore, if external perturbations are to beexpected, or if the inverse model turns out to be imprecise,stiffness K can be given high values without destabilisingcontrol. So the movement error can be kept low withoutmaking it necessary to recompute movement planning, orto alter the inverse model of the arm.

Our results question a main feature in current modelsof behavioural control: the forward model. Such a modelis unnecessary, at least in low-level motor control. So weare even contradicting several of our own past papers inwhich we focused on forward modelling as an integrativepart of motor control (Kalveram 1998; Timmann et al.2000). But as indicated in Fig. 1 and its legend, the learn-ing of the inverse model may still demand to be takenthrough forward modelling (Jordan 1988; Kawato 1990;Miall and Wolpert 1996; Wolpert et al. 1998). For that,two algorithms are currently being discussed: Kawato’sfeedback error learning (Kawato 1990) and Jordan’s for-ward and inverse modelling (Jordan 1988). However, both

methods obviously lack biological plausibility (Kalveramand Schinauer 2002). A recent alternative suggestion isthe ‘learner-operator model of motor control’ (Kalveram2004) which applies auto-imitation (Kalveram 1981, 1992,2000) to the part of the learner while exerting controlthrough the operator part. This algorithm appears bio-logically more plausible than those just mentioned butdispenses with a forward model. Thus, forward model-ling turns out to be unnecessary for both exertion andacquisition of control.

Acknowledgements. This article is dedicated in memoriam toGustav A. Lienert, who edited the ‘Draht-Biege-Probe’ (Lienert1961), a wire-bending test measuring complex sensorimotorskills in a genially simple manner. We thank Nicole Pledgerfor language assistance. This work was supported by GrantsKa 417/18 and Ka 417/24 from Deutsche Forschungsgemeins-chaft (DFG).

References

Asatryan DG, Feldman AG (1965) Biophysics of complex sys-tems and mathematical models: functional tuning of nervoussystem with control of movement or maintenance of a steadyposture: I. Mechanographic analysis of the work of the jointon execution of a postural task. Biofizika 10:837–846

239

Bennett DJ, Hollerbach JM, Xu Y, Hunter IW (1992) Time-varying stiffness of human elbow joint during cyclic voluntarymovement. Exp Brain Res 88:433–442

Bhushan N, Shadmehr R (1999) Computational nature of hu-man adaptive control during learning of reaching movementsin force fields. Biol Cybern 81:39–60

Bizzi E, Hogan N, Mussa-Ivaldi FA, Giszter S (1992) The equi-librium-point framework: a point of departure. Behav BrainSci 15:808–844

Bizzi E, Polit A, Morasso P (1976) Mechanisms underlyingachievement of final head position. J Neurophysiol 39:435–444

Desmurget M, Grafton S (2000) Forward modelling allows feed-back control for fast reaching movements. Trends Cogn Sci4:423–431

Eibl-Eibesfeld I (1987) Grundriss der vergleichenden Verhal-tensforschung. Piper, Munchen

Feldman AG (1966) Functional tuning of the nervous systemduring control of movement or maintenance of a steady pos-ture: III. Mechanographic analysis of the execution by manof the simplest motor tasks. Biofizika 11:667–675

Flanagan JR, Vetter P, Johansson RS, Wolpert DM (2003) Pre-diction precedes control in motor learning. Curr Biol 13:146–150

Forssberg H, Grillner S, Halbertsma J (1980) The locomotionof the low spinal cat: I. Coordination within a hindlimb. ActaPhysiol Scand 108:269–281

Gomi H, Kawato M (1997) Human arm stiffness and equilib-rium-point trajectory during multi-joint movement. Biol Cy-bern 76:163–171

Hill AV (1938) The heat of shortening and the dynamic constantsof muscles. Proc R Soc 126B:136–195

Hogan N (1985) The mechanics of posture and movement. BiolCybern 52:315–331

Holst Ev (1939) Die relative Koordination als Phanomen undals Methode zentralnervoser Funktionsanalyse. Ergebnisseder Physiologie 42:228–306

Jordan L (1988) Supervised learning and systems with excessdegrees of freedom. COINS Technical Report 88/27:1–41

Kalveram KT (1981) Erwerb sensumotorischer Koordinationenunter storenden Um-welteinflussen: Ein Beitrag zum Problemdes Erlernens von Werkzeuggebrauch (English title: Acquisi-tion of sensorimotor co-ordinations under environmental dis-turbances. A contribution to the problem of learning to use atool). In: Tent L (ed) Erkennen, Wollen, Handeln. Festschriftfur Heinrich Duker. Hogrefe, Gottingen, pp 336–348

Kalveram KT (1991) Pattern generating and reflex-like pro-cesses controlling aiming movements in the presence of iner-tia, damping and gravity. A theoretical note. Biol Cybern64:416–419

Kalveram KT (1992) A neural network model rapidly learninggains and gating of reflexes necessary to adapt to an arm’sdynamics. Biol Cybern 68:183–191

Kalveram KT (1998) Wie das Individuum mit seiner Umwelt in-teragiert. Psychologische, biologische und kybernetische Bet-rachtungen uber die Funktion von Verhalten. Pabst, Lenge-rich

Kalveram KT (2000) Sensorimotor sequential learning by a neu-ral network based on redefined Hebbian Learning. In: Niklas-son L (ed) Artificial neural networks in medicine and biology.Springer, Berlin Heidelberg New York, pp 271–276

Kalveram KT (2004) The inverse problem in cognitive, percep-tual and proprioceptive control of sensorimotor behaviour:

towards a biologically plausible model of the control of aim-ing movements. Int J Sport Exercise Psychol 2:255–273

Kalveram KT, Schinauer T (2002) The problem of adaptive con-trol in a living system or how to acquire an inverse modelwithout external help. In: Verleysen M (ed) ESANN’2002 –European symposium on artificial neural networks. Evere:DeFacto, Bruges, Belgium, pp 89–94

Kawato M (1990) Feedback-error-learning neural network forsupervised learning. In: Eckmiller R (ed) Advanced neuralcomputers. North-Holland, Amsterdam, pp 365–372

Kawato M (1999) Internal models for motor control and trajec-tory planning. Curr Opin Neurobiol 9:718–727

Konczak J, Brommann K, Kalveram KT (1999) Identificationof time-varying stiffness, damping, and equilibrium positionin human forearm movements. Motor Control 3:394–413

Latash ML (1992) Independent control of joint stiffness in theframework of the equilibrium-point hypothesis. Biol Cybern67:377–384

Lienert GA (1961) Die Drahtbiegeprobe als standardisierterTest. Handanweisung fur die Durchfuhrung und Auswertung.Hogrefe, Gottingen

Marsden CD, Merton PA, Morton HB (1972) Servo action inhuman voluntary movement. Nature 238:140–143

Miall RC, Weir DJ, Wolpert DM, Stein JF (1993) Is the cerebel-lum a Smith Predictor? J Motor Behav 25:203–216

Miall RC, Wolpert DM (1996) Forward models for physiologicalmotor control. Neural Netw 9:1265–1279

Miller JP, Selverston AI (1985) Neural mechanisms for the pro-duction of the lobster pyloric motor pattern. In: SelverstonAI (ed) Model neural networks and behavior. Plenum, NewYork

Mussa-Ivaldi FA, Bizzi E (2000) Motor learning throughthe combination of primitives. Phil Trans R Soc Lond B355:1755–1769

Neilson PD, Neilson MD, O’Dwyer NJ (1997) Adaptive modeltheory: central processing in acquisition of skill. In: Forss-berg H (ed) Neurophysiology and neuropsychology of motordevelopment. McKeith, London

Nichols TR, Houk JC (1976) The improvement in linearity andthe regulation of stiffness that results from the actions of thestretch reflex. J Neurophysiol 39:119–142

Ostry DJ, Feldman AG (2003) A critical evaluation of the forcecontrol hypothesis in motor control. Exp Brain Res 153:275–288

Polit A, Bizzi E (1979) Characteristics of motor programs under-lying arm movements in monkeys. J Neurophysiol 42:183–194

Sabes PN (2000) The planning and control of reaching move-ments. Curr Opin Neurobiol 10:740–746

Sainburg RL, Lateiner JE, Latash ML, Bagesteiro LB (2003)Effects of altering initial position on movement direction andextent. J Neurophysiol 89:401–415

Saltzman E, Kelso JAS (1987) Skilled actions: a task-dynamicapproach. Psychol Rev 94:84–106

Seyfarth A, Blickhan R, Van Leeuwen JL (2000) Optimum take-off techniques and muscle design for long jump. J Exp Biol203:741–750

Sherrington CS (1906) The integrative action of the nervous sys-tem. Yale University Press, New Haven, CT

Smith O (1957) Closer control of loops with dead time. ChemEng Prog 53:217–219

Spoelstra J, Schweighofer N, Arbib MA (2000) Cerebellar learn-ing of accurate predictive control for fast-reaching move-ments. Biol Cybern 82:321–333

240

Sternad D (2002) Wacholder and Altenburger 1927: Founda-tional experiments for current hypotheses on equilibriumpoint control in voluntary movements. Motor Control 6:299–318

Stroeve S (1997) A learning feedback and feedforward neuro-muscular control model for two degrees of freedom humanarm movements. Hum Mov Sci 16:621–651

Timmann D, Richter S, Bestmann S, Kalveram KT, Konczak J(2000) Predictive control of muscle responses to arm pertur-bations in cerebellar patients. J Neurol Neurosurg Psychiatry69:345–352

Turvey MT, Shockley K, Carello C (1999) Affordance, properfunction, and the physical basis of perceived heaviness. Cog-nition 17:B17–B26

Vallbo AB (1974) Human muscle spindle discharge during iso-metric voluntary contractions. Amplitude relations betweenspindle frequency and torque. Acta Physiol Scand 90:319–336

Widrow B, Walach E (1996) Adaptive inverse control. Prentice-Hall, London

Winters JM (1995) An improved muscle-reflex actuator for usein large-scale neuromusculoskeletal models. Ann Biomed Eng23:359–374

Wolpert DM, Ghahramani Z, Jordan MI (1995) An internalmodel for sensorimotor integration. Science 269:1880–1882

Wolpert DM, Kawato M (1998) Multiple paired forward andinverse models for motor control. Neural Netw 11:1317–1329

Wolpert DM, Miall RC, Kawato M (1998) Internal models inthe cerebellum. Trends Cogn Sci 2:338–347