IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 40, 1995:413-440 1

Control of Systems Without Driftvia Generic Loops

Eduardo D. Sontag

Abstract— This paper proposes a simple numerical tech-nique for the steering of arbitrary analytic systems with nodrift. It is based on the generation of “nonsingular loops”which allow linearized controllability along suitable trajeto-ries. Once such loops are available, it is possible to employstandard Newton or steepest descent methods, as classicallydone in numerical control. The theoretical justification ofthe approach relies on recent results establishing the gener-icity of nonsingular controls, as well as a simple convergencelemma.

Keywords—steering of nonholonomic systems, nonsingularcontrols, mechanical systems, nonlinear control, nonlinearfeedback

I. Introduction

This paper deals with the problem of numerically findingcontrols that achieve a desired state transfer. That is, forany given initial and target states ξ0 and ξF in IRn, onewishes to find a time T > 0 and a control u defined on theinterval [0, T ], so that u steers ξ0 to ξF , for the system

x = f(x, u) . (1)

More precisely, the question of approximate controllability(for any ε > 0, find a control that brings the state to withinε distance of ξF ) will be considered.

A number of preliminary results will be developed forgeneral analytic systems of the type (1), but the controlla-bility application is restricted to the case of systems withoutdrift :

x = G(x)u , (2)

i.e., the right-hand side f(x, u) is linear in u. For suchsystems it is relatively straightforward to decide control-lability, but the design of explicit control strategies hasattracted considerable attention lately.

Problems of steering systems without drift are in partmotivated by the study of nonholonomic mechanical sys-tems. Many sophisticated control strategies have been pro-posed, based on a nontrivial analysis of the structure ofthe Lie algebra of vector fields generated by the columnsof G; see for instance [1], [11], [12], and [9]. The approachpresented in this paper is of an entirely different nature.It represents a simple-minded algorithm, in the style ofclassical numerical approaches, and it requires no symboliccomputation to implement. In fact, a short piece of codein any numerical package such as MATLAB is all that isneeded in order to obtain solutions. Obviously, as with any

E. Sontag is with SYCON - Rutgers Center for Systems andControl, Department of Mathematics, Rutgers University, NewBrunswick, NJ 08903. e-mail: sontag@control.rutgers.edu.

This research was supported in part by US Air Force Grant 91-0346.

general procedure, it can be expected to be extremely inef-ficient, and to result in poor performance, when comparedwith techniques that use nontrivial information about thesystem being controlled. Perhaps it will be useful mainly inconjunction with other techniques, allowing gross controlactions that help bring the system into regions of the statespace where the asumptions required for the more refinedtechniques hold.

Mathematically, the main contribution of this paper isin the formulation of the “generic loop” approach and thejustification of the algorithm. The latter relies on a newresult proving the existence of such loops with good con-trollability properties. This approach was motivated to agreat extent by related work on time varying feedback laws;see especially [5] and [13]. The last section of the papermakes some remarks regarding connections with that work.The technique described in this paper was presented at theMarch 1992 Princeton Conference on Information Sciencesand Systems, the February 1993 IMA Robotics ControlWorkshop, and the 1993 IEEE Conference on Decision andControl. Independently, Sussmann ([21]) proposed a nu-merical approach based on homotopy-continuation ideas;such an approach may be expected to be numerically moreuseful, but it requires strong assumptions on the system inorder to apply. Also related is the work by Brockett ([3]),who proposed a method which relies on randomization andsystem inversion.

A. Classical Iterative Techniques

It is assumed from now on that in (1) the states x(t)evolve in IRn. (Systems on manifolds can also be consid-ered, but doing so only complicates notations and adds inthis case little insight.) Controls u(t) take values in IRm,and are measurable and essentially bounded as a functionof time. Further, f is continuously differentiable (later re-sults will impose analyticity). Given a state ξ0 ∈ IRn anda control

u : [0, T ]→ IRm

so that the solution x : [0, T ]→ IRn of the equation (1) withthis control and the initial condition x(0) = ξ0 is defined onthe entire interval [0, T ] —that is, u is admissible for x,—the state x(t) at time t ∈ [0, T ] is denoted by φ(t, ξ0, u).As discussed above, the objective, for any given initial andtarget states ξ0 and ξF in IRn, is to find a time T > 0 anda control u defined on the interval [0, T ], so that u steersξ0 to ξF , that is, so that φ(T, ξ0, u) = ξF , at least in anapproximate sense. After a change of coordinates, one mayassume without loss of generality that ξF = 0.

Classical numerical techniques for this problem are based

2 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 40, 1995:413-440

on variations of steepest descent; see for instance [4], or [7]for a recent reference. The basic idea is to start with aguess of a control, say u : [0, T ] → IRm, and to improveiteratively on this initial guess. More precisely, let x =φ(·, ξ0, u). If the obtained final state x(T ) is already zero,or is sufficiently near zero, the problem has been solved.Otherwise, we look for a perturbation ∆u so that the newcontrol

u+ ∆u

brings us closer to our goal of steering ξ0 to the origin. Thevarious techniques differ on the choice of the perturbation;in particular, two possibilities are discussed next, later tobe analized.

The first is basically Newton’s method, and proceeds asfollows. Denote, for any fixed initial state ξ0,

α(u) := φ(T, ξ0, u)

thought of as a partially defined map from Lm∞(0, T ) intoIRn. This is a continuously differentiable map (see e.g. [18],Theorem 1), so expanding to first order there results

α(u+ v) = α(u) + α∗[u](v) + o(v)

for any other control v so that α(u + v) is defined, wherewe use “∗” as a subscript to denote differentials. If we cannow pick v so that

α∗[u](v) = −α(u) (3)

then for small enough h > 0 real,

α(u+ hv) = (1− h)α(u) + o(h) (4)

will be smaller than the state α(u) reached with the initialguess control u. In other words, the choice of perturbationis ∆u := hv, 0 < h� 1.

It remains to solve equation (3) for v. The operator

L : v 7→ α∗[u](v) (5)

is the one corresponding to the solution of the variationalequation

z = A(t)z +B(t)v z(0) = 0, (6)

where A(t) := ∂f∂x (x(t), u(t)) and B(t) := ∂f

∂u (x(t), u(t)) foreach t, that is,

Lv =∫ T

0

Φ(T, s)B(s)v(s) ds ,

where Φ denotes the fundamental solution associated toX = A(t)X.

The operator L maps Lm∞(0, T ) into IRn, and it is ontowhen (6) is a controllable linear system on the interval[0, T ], that is, when u is a control nonsingular for ξ0 relativeto the system (1). In other words, ontoness of L = α∗[u] isequivalent to first-order controllability of the original non-linear system along the trajectory corresponding to the ini-tial state ξ0 and the control u. The main point of this paper

will lie in showing that it is not difficult to generate usefulnonsingular controls for systems with no drift.

Assuming nonsingularity, there exist then many solu-tions to (3). Because of its use in (4) where a small vis desirable, and in any case because it is the most naturalchoice, it is reasonable to pick the least squares solution,that is the unique solution of minimum norm,

v := −L#α(u) (7)

where L# denotes the pseudoinverse operator (see e.g. [18],Section 3.5, for details; we are using the canonical innerproduct on IRn, and L2 norm in Lm∞(0, T ), and inducednorms for elements and operators).

The technique sketched above is well-known in numericalcontrol. For instance, the derivation in pages 222-223 of [4],when applied to solving the optimal control problem havingthe trivial cost criterion J(u) = 0 and subject to the finalstate constraints x = ψ(x) = 0, results in formula (7), andis derived in the same manner as here.

Alternatively, instead of solving (3) for v via (7), onemight use the steepest descent choice

v := −L∗α(u) (8)

where L∗ is the adjoint of L. Formula (8) also results fromthe above derivation in [4], now when applied using thequadratic cost J(u) = ‖α(u)‖2 but relaxing the terminalconstraints (ψ ≡ 0). In place of (4), now one has

α(u+ hv) = (I − hLL∗)α(u) + o(h), (9)

where I is the identity operator. If again L is onto, that is,if the control u is nonsingular for ξ0, then the symmetricoperator LL∗ is positive definite, so 0 < h � 1 will give acontraction as earlier. An advantage in using L∗ instead ofL# is that no matrix inversion is required in this case.

It is also possible to combine these techniques with linesearches over the scalar parameter h or, even more effi-ciently in practice, with conjugate gradient approches (seefor instance [10]). Line search corresponds to leaving vfixed and optimizing on the step size h, only recomputinga variation v when no further improvement on h can befound. (The control applied at this stage is then the onefor the “best” stepsize, not the intermediate ones calculatedduring the search.)

Of course, in general there are many reasons for whichthe above classical techniques may fail to be useful in agiven application: the initial guess u may be singular forξ0, the iteration may fail to converge, and so forth. Themain point of this paper is to show that, for a suitableclass of systems, a procedure along the above lines canbe guaranteed to work. The systems with which we willdeal here are often called “systems without drift” and arethose expressed as in Equation (2). A result given belowshows that for such systems (assuming analytic G) ratherarbitrary controls provide the desired nonsingularity, andcan hence be used as the basis of the approach sketchedabove.

SONTAG: CONTROL OF SYSTEMS WITHOUT DRIFT VIA GENERIC LOOPS 3

The next section establishes the basic iterative procedureand proves a convergence result assuming that nonsingularcontrols exist. After that, we state the existence theoremfor nonsingular controls in the analytic case (a proof isgiven in an Appendix), and explain the application to sys-tems without drift. Several remarks are also provided inthe last section, and relationships to time-varying feedbackdesign are briefly discussed.

II. Justification of the Iterative Method

We now prove the convergence of the algorithm consist-ing of repeatedly applying a control to obtain a nonsin-gular trajectory, and at each step perturbing this controlby means of a linear technique. As a preliminary step, weestablish a few results in somewhat more generality; theseare fairly obvious remarks about iterative methods, but wehave not found them in the literature in the form neededhere.

Lemma II.1: Let B be a compact subset of IRn, and letH > 0. Assume given

F : B × [0,H]→ IRn

and a continuous matrix function

D : B → IRn×n

so that D(x) is symmetric and positive definite for each x.Assume further that the function

g(x, h) := F (x, h) + hD(x)x− x

is o(h) uniformly on x, that is, for each ε > 0 there is aδ > 0 so that

h < δ ⇒ ‖g(x, h)‖ < εh for all x ∈ B . (10)

Then the following conclusion holds, for some constant λ >0: For each ε > 0 there is some δ > 0 so that, for eachh ∈ (0, δ) and each x ∈ B,

‖F (x, h)‖ < max{

(1− λh)‖x‖, ε}. (11)

Proof: Note that since D(x) is continuous on x, itssingular values also depend continuously on x (see e.g. [18],Corollary A.4.4). Let 2λ > 0 be a lower bound and let λ bean upper bound for the eigenvalues of D(x). Pick a k > 2so that kλ > 2λ.

Now fix any ε > 0. There is then some 0 < δ < 1/λ suchthat, for each 0 < h < δ,

‖g(x, h)‖ < λεh

k<

ε

k(12)

for all x ∈ B and all the eigenvalues of hD(x) are in theinterval (0, 1).

Pick any h ∈ (0, δ) and any x ∈ B. As the eigenvalues ofthe symmetric matrix I−hD(x) are all again in (0, 1), thismatrix must be positive definite and so its norm equals itslargest eigenvalue; thus:

‖I − hD(x)‖ ≤ 1− 2λh .

Therefore, for ‖x‖ > ε/2 it holds that:

‖F (x, h)‖ ≤ ‖(I − hD(x))x‖+ ‖g(x, h)‖≤ (1− 2λh)‖x‖+ λεh/k

=(

1− 2λh+λεh

k‖x‖

)‖x‖

< (1− λh)‖x‖ ,

which implies the desired conclusion. If instead ‖x‖ < ε/2,then

‖F (x, h)‖ ≤ ‖I−hD(x)‖‖x‖+‖g(x, h)‖ < ε/2+ε/k < ε ,

so the conclusion holds in that case as well.Observe that continuity of D(x) is only used in guar-

anteeing that the singular values are bounded above andaway from zero.

Lemma II.2: Let B be a closed ball in IRn, centered atthe origin, and let H > 0. Assume given a map

F : B × [0,H]→ IRn ,

with F (x, 0) = x for all x, so that F is continuously differ-entiable with respect to h ∈ [0,H], with ∂F

∂h continuous on(x, h), and

∂F

∂h(x, 0) = −D(x)x ,

where D : B → IRn×n is a continuous matrix functionsatisfying that D(x) is symmetric positive definite for eachx. Denote Fh := F (·, h). Then the following propertyholds: For each ε > 0, there is some δ > 0 so that, for each0 < h < δ there is some positive integer N = N(h) so that

‖FNh (B)‖ < ε ,

where FNh denotes the Nth iterate of Fh.Proof: In order to apply Lemma II.1, we only need to

check that in the expansion

F (x, h) = x− hD(x)x+ g(x, h)

the last term is o(h) uniformly on x. But (Lagrange for-mula):

g(x, h) = F (x, h)−F (x, 0)−∂F∂h

(x, 0)h =∫ 1

0

G(x, h, t)h dt

where

G(x, h, t) :=∂F

∂h(x, th) − ∂F

∂h(x, 0)

and ∂F∂h (x, h) is continuous by hypothesis. On the compact

set B × [0,H], this function is uniformly continuous; inparticular it is so at the points of the form (x, 0). Thus foreach ε > 0 there is some δ > 0 so that whenever h < δ then‖G(x, h, t)‖ < ε for all x ∈ B and all t ∈ [0, 1]. Thereforealso ‖g(x, h)‖ < εh holds, and Lemma II.1 can indeed beapplied.

4 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 40, 1995:413-440

As B is a ball, the iterates remain in B. So, for each land each x ∈ B,

‖F lh(x)‖ < max{

(1− λh)l‖x‖, ε}.

This gives the desired result.For each ξ ∈ IRn and each control u ∈ Lm∞(0, T ) admissi-

ble for ξ, we let Lξ,u be the linear operator Lm∞(0, T )→ IRn

defined as in (5), that is, the reachability map for the time-varying linear system (6) that results along the ensuingtrajectory. Introducing the matrix functions

A = A(x, u) =∂f

∂x(x, u) and B = B(x, u) =

∂f

∂u(x, u) ,

we may consider the following new system (the “prolonga-tion” of the original one):

x = f(x, u) (13)z = A(x, u)z +B(x, u)v (14)

seen as a system of dimension 2n and control (u, v) of di-mension 2m. Observe that Lξ,u(v) is the value of the z-coordinate of the solution that results at time T when ap-plying controls u, v and starting at the initial state (ξ, 0).If we add the equation

Q = AQ+QA+BB∗ (15)

(superscript ∗ indicates transpose) to the prolonged sys-tem, the solution with the above controls and initial state(ξ, 0, 0) has

Q(t) =∫ t

0

Φ(t, s)B(s)B∗(s)Φ(t, s)∗ ds

so that (see e.g. [18], Section 3.5) ontoness of Lξ,u is equiv-alent to the Grammian W = Q(T ) being positive definite.Note that, by continuous dependence on initial conditionsand controls, W depends continuously on ξ, u. Similararguments show that other objects associated to the lin-earization also depend continuously on ξ, u, and any stateq: application to q of the adjoint, L∗ξ,uq, which is the sameas the function B(t)∗Φ(T, t)∗q, and of the pseudoinverse,L#ξ,uq = L∗W−1q.Fix now a control u and a closed ball B ⊆ IRn so that u is

admissible for all ξ ∈ B, and denote Lξ,u just as Lξ. (Thisis the zero-initial-state reachability map of the linearizedsystem when applying u and starting at the state ξ; thusfor each ξ, Lξ is a map from controls into states of thelinearized system.) In the next result, the map Nξ playsthe role of a one-sided “approximate inverse” of Lξ (foreach state ξ, Nξ is a map from states into controls).

Corollary II.3: Assume that the control u is so that

φ(T, ξ, u) = ξ for all ξ ∈ B .

Assume given, for each ξ ∈ B, a map Nξ : IRn → Lm∞(0, T )so that Nξ(ξ) depends continuously on ξ and so that theoperator

D(ξ) := LξNξ

is linear, and in the standard basis is symmetric positivedefinite and depends continuously on ξ. Pick an H > 0 sothat u−hNξ(ξ) is admissible for each ξ ∈ B and h ∈ [0,H],and let

F (ξ, h) := φ(T, ξ, u− hNξ(ξ)) .Then, for each ε > 0, there is some δ > 0 so that, for each0 < h < δ there is some positive integer N = N(h) so that

‖FNh (B)‖ < ε ,

where Fh := F (·, h).Proof: Observe that, since ∂φ(T,ξ,u)

∂u

∣∣∣u=u

is the sameas Lξ, we have that, in general,

∂φ (T, ξ, u− hNξ(q))∂h

∣∣∣h=0

= −D(ξ)q ,

so in particular ∂F∂h (ξ, 0) = −D(ξ)ξ, as needed in order to

apply Lemma II.2. Note that ∂F∂h (ξ, h) is continuous, as it

equals−Lξ,u−hNξ(ξ)Nξ(ξ)

and each of L and N are continuous on all arguments.Note that an H > 0 as needed in the statement always

exists, by continuity of solutions on initial conditions andcontrols.

III. Application to Systems with no Drift

The application to systems without drift, those that areas in Equation (2), is as follows. As discussed in the nextsubsection, rescaling if necessary, we may assume that thesystem is complete. In order to apply the numerical tech-niques just developed, one needs to find a control u whichleads to nonsingular loops:• u is nonsingular for every state x in a given ball B, and• φ(T, x, u) = x for all such x.

It is shown later that for analytic systems that have thestrong accessibility property, controls which are generic –in a sense to be made precise– are nonsingular for all states.(For analytic systems without drift, Chow’s Theorem statesthat the strong accessibility property is equivalent to com-plete controllability.) Starting from such a control ω, de-fined on an interval [0, T/2], one may now consider thecontrol u on [0, T ] which equals ω on [0, T/2] and is thenfollowed by the antisymmetric extension:

u(t) = −ω(T − t) , t ∈ (T/2, T ] . (16)

This u is as needed: nonsingularity is due to the fact thatif the restriction of a control to an initial subinterval isnonsingular for the initial state, the whole control is, andthe loop property is an easy consequence of the special form(2) in which the control appears linearly.

In practice, one might try using a randomization tech-nique in order to obtain ω, and from there u. More directly,one might use instead a finite Fourier series with randomcoefficients:

u(t) =l∑

k=1

ak sin kt , (17)

SONTAG: CONTROL OF SYSTEMS WITHOUT DRIFT VIA GENERIC LOOPS 5

which automatically satisfies the antisymmetry property(16) on the time interval [0, 2π]. There is no theoreticalguarantee that such a series will provide nonsingularity,but in any case, experimentally, one may always proceedassuming that indeed all properties hold. (It has beenpointed out by a referee that the results in [5] imply that,on any fixed compact, such finite Fourier series will pro-vide nonsingularity, at least if the coefficients are pickedsmall enough, and as long as the total number of terms l islarger than a certain integer l(n,m, r) computable from n,m, and the number r of Lie brackets sufficient to providethe accessibility property on the given compact. This is atopic worthy of further detailed research.)

The first application is with Nx = L#x , the pseudoin-

verse discussed earlier. Here D(x) = I is certainly positivedefinite and continuous on x.

The second application is with Nx = L∗x, the adjointoperator, in which case D(x) = W = Q(T ), as obtainedfor the composite system (13)-(15), and as remarked earlierthis is also continuous on x (and positive definite for eachx, by nonsingularity).

We may summarize the procedure as follows. The objec-tive is to transfer ξ0 to a neighborhood of ξF .

Step 1. Find an u that generates nonsingular loops, inthe above sense. Let ξ := ξ0.

Step 2. Calculate the effect of applying u, starting at ξ,and compute the linearization along the correspond-ing trajectory, using this in turn in order to obtainthe variation that allows modifying u by hNξ(ξ), asdescribed earlier.

Step 3. The original control u is not applied to the sys-tem (from state ξ), but the perturbed one is. Applythis new control to the system and compute the finalstate ξ′ that results.

Step 4. If ξ′ is not close enough to ξF , let ξ := ξ′, andgo to Step 2.

There is then guaranteed convergence in finite time toany arbitrary neighborhood of the origin, for small enoughstepsize. One may also combine this approach with linesearches, or even conjugate gradient algorithms, as dis-cussed earlier.

Such techniques are classical in nonlinear control; see forinstance [4], [10]. What appears to be new is the observa-tion that, for analytic systems without drift, generic loopsprovide nonsingularity. The techniques are also related tothe material in [16], which relied on control based on pole-shifting along nonsingular trajectories.

A. Rescaling: Obstacles and Completeness

For systems with no drift, a simple rescaling of the equa-tions may be an extremely powerful tool that allows (a)dealing with workspace obstacles and (b) the reduction tosystems that are complete (no explosion times). The basicidea, which is very straightforward and rather well-known,is as follows.

Assume that β : IRn → IR is any smooth mapping, andconsider the new system without drift

x = β(x)G(x)u . (18)

Suppose that one has found a control u, defined on an in-terval [0, T ], so that the state ξ0 is transferred into thestate ξF using this control, for the system (18). Let x(·)be the corresponding trajectory. Then, the new controlv(t) := β(x(t))u(t), when applied to the original system(2), also produces the desired transfer. In other words,solving a controllability problem for (18) provides imme-diately a solution to the corresponding problem for theoriginal system. (If one is interested in feedback design,as opposed to open-loop control as in this paper, the samesituation holds: a feedback law u = k(x) for (18) can bere-interpreted as a feedback law u = β(x)k(x) for (2).)

If β never vanishes, the controllability properties of theoriginal and the transformed systems are the same. Thisis clear from the above argument. Alternatively, one maysee this from the fact that, for any two vector fields g1, g2

and any two smooth scalar functions β1, β2,

[β1g1, β2g2] = β1β2[g1, g2] + β1g1(β2)g2 − β2g2(β1)g1 .

This implies inductively that the Lie algebra generated bythe columns of β(x)G(x) is included in the C∞-modulegenerated by the Lie algebra corresponding to the columnsof G(x), so the accessibility rank condition for the formerimplies the same for the latter (and viceversa, by reversingthe roles of β(x)G(x) and G(x)).

This construction is of interest in two ways. First ofall, one is often interested in control of systems in such amanner that trajectories avoid a certain subset Q of thestate-space (which may correspond to “obstacles” in theworkspace of a robot, for instance). If β vanishes exactlyon Q, then control design on the complement of Q can bedone for the new system (18), and controls can then bereinterpreted in terms of the original system, a discussedabove. Since β vanishes on Q, no trajectories starting out-side Q ever pass through Q (uniqueness of solutions). Ofcourse, in planning motions in the presence of obstacles,the control variations should be chosen so as to move instate space directions which do not lead to collisions. Onepossible approach is to first design a polyhedral path tobe tracked, and then to apply the numerical technique ex-plained in order to closely follow this path.

Reparameterization also helps in dealing with possibleexplosion times in the original system, a fact that had beenpreviously observed in [9], page 2542. In this case, onemight use an β(x) so that β(x)G(x) has all entries bounded;for instance, β(x) could be the chosen as (1+

∑i,j g

2ij(x))−1.

This means that the new system has no finite escape times,for any bounded control.

B. Some Implementation Questions

Next are derived explicit formulas for the use of the abovetechnique, in the case of systems without drift and whensteepest descent variations are used. As just discussed, onemay assume that the system is complete.

Assume that u(t), t ∈ [0, T ] satisfies the antisymmetrycondition

u(T − t) = −u(t) . (19)

6 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 40, 1995:413-440

If x(·) satisfies x = G(x)u then z(t) := x(T − t) satisfiesthe same equation; thus from the equality z(T/2) = x(T/2)and uniqueness of solutions it follows that z = x. In otherwords,

x(T − t) = x(t) (20)

for t ∈ [0, T ]. To distinguish the objects which dependexplicitely on time from those that depend on the currentvalues of states and controls, use the notation

A(x, u) :=m∑i=1

∂gi∂x

(x)ui

where gi is the ith column of G, ui is the ith entry of thevector u ∈ IRm, and the partial with respect to x indicatesJacobian. Note that A can be calcuted once and for all asa function of the variables x, u, before any numerical com-putations take place. For each u, and the trajectory x(·)corresponding to this control and initial state ξ0, denote

A(t) := A(x(t), u(t)) , B(t) := G(x(t)) .

Note that if (19), and hence also (20), hold then

A(T − t) = −A(t) , B(T − t) = B(t) (21)

hold as well. Consider next Ψ(t) := Φ(T, t), where Φ is thefundamental solution as before, corresponding to a given uand x(·) as above. Thus, Ψ satisfies the matrix differentialequation

Ψ(t) = −Ψ(t)A(t) , Ψ(T ) = I .

Consider the function Ψ(t) := Ψ(T−t). If u satisfies the an-tisymmetry condition, then Ψ satisfies the same differentialequation as Ψ, from which the equality Ψ(T/2) = Ψ(T/2)implies Ψ = Ψ. Hence also

Ψ(T − t) = Ψ(t) (22)

and so Ψ(0) = Ψ(T ) = I. The perturbed control to be ap-plied is u+hv = u−hL∗α(u) where α(u) = x(T ) = x(0) =ξ0 if u satisfies the antisymmetry condition. The adjointoperator is (L∗ξ0)(t) = B(t)∗Ψ(t)∗ξ0. Summarizing, thecontrol to be applied, which for small h should result in astate closer to the origin than ξ0, is

u(t)− hG(x(t))∗Ψ(t)∗ξ0 t ∈ [0, T ]

where

x(t) = G(x(t))u(t) , x(0) = ξ0

Ψ(t) = −A(x(t), u(t)) Ψ(t) , Ψ(0) = I .

The equations for the system evolution are as follows (thestate variable is now denoted by z in order to avoid confu-sion with the reference trajectory x):

z(t) = G(z(t)) [u(t)− hG(x(t))∗Ψ(t)∗ξ0]

for t ∈ [0, T ], with initial condition z(0) = ξ0. In a line-search implementation, one would first compute z(T ) for

various choices of h; the control is only applied once thatan optimal h has been found. Then the procedure can berepeated, using z(T ) as the new initial state ξ0.

Remark. Regarding the number of steps that areneeded in order to converge to an ε-neighborhood of thedesired target state, an estimate is as follows. For a fixedball around the origin, and sufficient smoothness, one cansee that h = O(ε) provides the inequality in (10), as re-quired for (12). Thus, the number of iterations N needed,using such a stepsize, is obtained from (11):

(1− cε)N < ε

where c is a constant. Taking logarithms and using log(1−x) = x+ o(x) there results the rough estimate

N = O

(1ε

log(

1ε

)).

Remark. The method introduced in this paper couldalso potentially be used in an adaptive control context,when the precise plant model is not known. In that case,it is of course not possible to find the necessary gradient orNewton corrections of the nonsingular control u. However,with an arbitrary choice of v, as long as this is not mappedby the differential into a direction orthogonal to that tothe target, either small h < 0 or h > 0 will provide animprovement. We leave this idea as a suggestion for furtherwork.

IV. Universal Inputs

In this Section, the systems considered will be of the type(1) where x(t) ∈ X , u(t) ∈ U , and:• X ⊆ IRn is open and connected, for some n ≥ 1;• U ⊆ IRm is open and connected, for some m ≥ 1;• f : X × U → IRn is real-analytic.A control is a measurable essentially bounded map ω :

[0, T ]→ U ; it is said to be smooth (respectively, analytic) ifit is infinitely differentiable (respectively, real-analytic) asa function of t ∈ [0, T ]. As before, we denote by φ(t, x, ω)the solution of (1) at time t with initial condition x andcontrol ω. This is defined for all small t = t(x, ω) > 0;when we write φ(·, x, ω), we mean the solution as definedon the largest interval [0, τ) of existence.

Recall that the system (1) is said to be strongly accessibleif for each x ∈ X there is some T > 0 so that

intRT (x) 6= ∅ ,

where as usual RT (x) denotes the reachable set from xin time exactly T . Equivalently, the system must satisfythe strong accessibility rank condition: dimL0(x) = n forall x, where L0 is the ideal generated by all the vectorfields of the type {f(·, u) − f(·, v), u, v ∈ U} in the Liealgebra L generated by all the vector fields of the type{f(·, u), u ∈ U}; see [22]. For systems affine in controls:

x = f(x) +m∑i=1

uigi(x) (23)

SONTAG: CONTROL OF SYSTEMS WITHOUT DRIFT VIA GENERIC LOOPS 7

the algebra L0 is the Lie algebra generated by all vectorfields adkf (gi), k ≥ 0, i = 1, . . . ,m.

Given a state x, a control ω defined on [0, T ], and apositive T0 ≤ T so that ξ(t) = φ(t, x, ω) is defined forall t ∈ [0, T0], we may consider the linearization along thetrajectory (ξ, ω):

z(t) = A(t)z(t) +B(t)u(t) (24)

where A(t) := ∂f∂x (ξ(t), ω(t)) and B(t) := ∂f

∂u (ξ(t), ω(t)) foreach t. A control ω will be said to be nonsingular for x ifthe linear time-varying system (24) is controllable on theinterval [0, T0], for some T0 > 0. When u is analytic, thisproperty is independent of the particular T0 chosen, and itis equivalent to a Kalman-like rank condition (see e.g. [18],Corollary 3.5.17). Nonsingularity is equivalent to a Frechetderivative of φ(T0, x, ·) having full rank at ω.

If ω is nonsingular for x ∈ X , and T0 is as above, thenRT0(x) has a nonempty interior. This is a trivial conse-quence of the Implicit Function Theorem (see for instance[18], Theorem 6). Thus, if for each state x there is somecontrol which is nonsingular for x, then (1) is strongly ac-cessible. The converse of this fact is also true, that is, if asystem is strongly accessible then for each state x there issome control which is nonsingular for x. This converse factwas proved in [17] (the result in that reference is stated un-der a controllability assumption, which is not needed in theproof of this particular fact; in any case, we review belowthe proof). The main purpose here is to point out that ωcan be chosen independently of the particular x, and more-over, a generic ω has this property. We now give a precisestatement of these facts.

A control ω : [0, T ] → U will be said to be a universalnonsingular control for the system (1) if it is nonsingularfor every x ∈ X .

Theorem 1: If (1) is strongly accessible, there is an ana-lytic universal nonsingular control.

Let C∞([0, T ],U) denote the set of smooth controlsω : [0, T ] → U , endowed with the C∞ topology (uni-form convergence of all derivatives). A generic subset ofC∞([0, T ],U) is one that contains a countable intersectionof open dense sets.

Theorem 2: If (1) is strongly accessible, the set ofsmooth universal nonsingular controls is generic inC∞([0, T ],U), for any T > 0.

A proof of this fact was originally given [19]. A proofis also given in an Appendix, in order to make this paperself-contained. The proof is heavily based on the universalinput theorem for observability. (The theorem for observ-ability is due to Sussmann, but the result had been suc-cessively refined in the papers [8], [14], [20]; see also [23]for a different proof as well as a generalization involvinginputs that are universal even over the class of all possibleanalytic systems. There is also closely related recent workof Coron ([6]) on generalizations of these theorems.)

V. Remarks

It is worth mentioning certain relations between the re-sults in this paper and recent work on time-varying feed-

back laws for systems without drift, especially the resultsin [5] and [13].

In [5], Coron proves, for controllable smooth systemswith no drift, that there is a smooth feedback law u =k(t, x), periodic on t and with k(t, 0) ≡ 0, such that theclosed-loop system x = G(x)k(t, x) is uniformly globallyasymptotically stable. The critical step in his proof is toobtain a smooth family of controls {ux(·), x ∈ IRn}, whereeach ux is defined for all t ∈ IR, so that the following prop-erties are satisfied:

1. ux(t+ 1) = ux(t) ∀x, t,2. ux(1− t) = −ux(t) ∀x, t,3. ux(t) is C∞ jointly on (x, t),4. for each x 6= 0, ux is nonsingular for x,5. u0 ≡ 0, and6. φ(t, x, ux) is defined for all t ≥ 0.

Observe that the second and last properties imply thatφ(1, x, ux) = x for all x. Thus, applying the control ux withinitial state x results in a periodic motion, φ(t+1, x, ux) =φ(t, x, ux). These properties are used in deriving stabilizingfeedbacks in [5].

It is possible to obtain a family of controls as above —at least in the analytic case— using Theorem 1. A sketchfollows. First note that one may take the system to becomplete, as discussed in Section III, so the last propertywill be satisfied for any choice of ux.

Assume that ω is a control which is analytic and univer-sal nonsingular, defined on the interval [0, 1]. As the systembeing considered in this case has no drift, it follows that foreach nonzero constant c the control cω(ct), defined on theinterval [0, 1/c], is again universal nonsingular. (Indeed, ifξ0 as any initial state and x(t) = φ(t, ξ0, ω) then x(ct) isthe trajectory corresponding to this new control, and thelinearization along this trajectory is controllable, because,with the notations in [18], Corollary 3.5.17 and using super-script c to denote the dependence on c, A(c)(t) = cA(ct)and B

(c)i (t) = ciB(ct) for i = 0, 1, 2, . . ..) Assume that

c < 1, so that cω(ct) is defined on [0, 1]. Since the systemand the control are both analytic, the restriction of cω(ct)to the interval [0, 1/6] is again universal and nonsingular.Observe that, by definition of analytic function on a closedinterval, this means that cω(ct) is in fact defined on somelarger interval of the form (−ε, 1), for some ε > 0. Letβ : IRn → IR be a smooth function which is positive forx 6= 0, vanishes at the origin, and is bounded by 1.

Consider now, for each x 6= 0, the control ux(t) which isdefined on the interval [0, 1/2] as follows. On the subinter-val [1/6, 1/3], this equals

β(x)ω(β(x)(t− 1/6)) .

Extend ux smoothly to [0, 1/6] in such a manner that allderivatives vanish at 0. Similarly, extend in the other di-rection, to [0, 1/2], so that all derivatives also vanish at1/2. Note that ux is still a universal nonsingular control,because its restriction to the subinterval [1/6, 1/3] is. Also,these extensions can be done in such a manner that ux de-pends smoothly on x and is bounded by a constant multipleof β(x). Finally, it is trivial to extend by antisymmetry to

8 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 40, 1995:413-440

[0, 1] and then periodically to all t ∈ IR, so that all thedesired properties hold.

VI. An Illustration

We now illustrate our technique with the simplest possi-ble example of a system with no drift which is controllablebut for which no possible smooth stabilizer exists. Thisexample is due to Brockett ([2]) and appears in most text-books in some variant or another (see e.g. [18], Example4.8.14); it is closely related, under a coordinate change,to the “unicycle” or “knife edge” example. The system inquestion has dimension 3 and two controls; the equationsare as follows:

x = u

y = v

z = xv

(we write x, y, z for the coordinates of the state and u, vfor the input coordinates, in order to avoid subscripts). Ashort program was written in order to simulate the behav-ior of the gradient descent algorithm based on the ideasdescribed in this paper.

As suggested earlier, periodic controls on intervals [0, 2π]symmetric about π are natural. In this case, in particular,the input u defined by u(t) ≡ 0, v(t) = sin(t) on thisinterval is already a universal nonsingular control (as shownnext), so we use u. Nonsingularity is shown as follows.Given any initial state ξ = (x0, y0, z0), the trajectory thatresults is

x(t) = x0

y(t) = y0 − cos tz(t) = z0 + x0t .

Along this trajectory, the linearized system has matrices

A(t) =

0 0 00 0 0

sin t 0 0

and B(t) =

1 00 10 x0

.

Let B0 := B and B1 := AB0 − B′0 (= AB since B isconstant). Since

(B0B1) =

1 0 0 00 1 0 00 x0 sin t 0

has rank 3 generically, this shows that the linearized systemis controllable (see e.g. [18], Corollary 3.5.17).

We simulated the gradient descent algorithm, using asimple line search consisting of optimizing the choice ofthe stepsize h after computing the gradient, and simulat-ing 25 steps of the procedure for several different initialconditions.

Table 1 provides two initial conditions (namely, ξ1 =(20, 10,−10) and ξ2 = (50, 10,−20)) and the stepsizes (first4 decimal digits) that resulted for each of them (the zero

entry is rounded-off). Choice of stepsize is critical for per-formance; a one-dimensional optimization over h must beperformed to obtain the best h at each step. The plotsin Figures 1-2 show the respective trajectories (note thatfor this simple example, it is also possible to compute theend points in closed form). Observe the oscillations: they variable, in particular, is driven by a sinusoidal nominalcontrol subject to a nonlinear correction. Oscillations canbe expected in general techniques dealing with nonholo-nomic control problems, since in one way or another, Liebrackets of vector fields are being approximated (the ma-noeuvres necessary in order to take an automobile out of atight parking space are a classical illustration of this fact).

ξ10.0004

0.2041

0.0044

0.0374

0.0089

0.0460

0.0250

0.0559

0.1458

0.0702

0.1415

0.0980

0.1657

0.1021

0.1639

0.1034

0.1630

0.1041

0.1627

0.1044

0.1625

0.1046

0.1624

0.1046

0.1623

ξ20.0000

0.0324

0.0001

0.1925

0.0021

0.0502

0.0050

0.0582

0.0155

0.0643

0.0756

0.0689

0.1447

0.0699

0.1591

0.1034

0.1654

0.1034

0.1654

0.1034

0.1654

0.1034

0.1654

0.1034

0.1654

Table: Columns provide stepsize schedules for each example

Newton’s method leads to even better results in this case(using the above nonsingular control). Indeed, since thefirst two equations are linear in controls, and the systemover all is quadratic in a suitable sense, Newton’s methodresults in exact convergence to zero in just two passes.We prove this fact next. With the above control u, thepseudoinverse of the reachability map is as follows (letting(x0, y0, z0) be the coordinates of the initial state):

L# = (1/π)

(1/2 + cos(t) −x0 cos(t) cos(t)

0 1/2 0

),

so the net control applied is −h

π

(x0

2+ x0 cos(t)− x0 cos(t)y0 + cos(t)z0

)sin(t)− hy0

2π

.

SONTAG: CONTROL OF SYSTEMS WITHOUT DRIFT VIA GENERIC LOOPS 9

0 20 40 60 80 100 120 140 160-15

-10

-5

0

5

10

15

20

25

30

35

Fig. 1. Simulation starting from ξ1

0 20 40 60 80 100 120 140 160-40

-20

0

20

40

60

80

100

Fig. 2. Simulation starting from ξ2

A Newton step is obtained by solving the correspond-ing differential equations with step size h; this gives thenew states: xh = x0 − x0h, yh = y0 − hy0, and zh =(h/2)(−2x0y0 + hx0y0 − 2z0) + z0. The stepsize h = 1gives zero values for the first two coordinates after one step,while the last coordinate becomes, under this choice of h,−x0y0/2. But any state of the form (0, 0, z) gets mappedto the origin in one step under the same iteration. In sum-mary, all states are mapped in two iterations to the origin.Figures 3-4 plot solutions using Newton’s method, for thesame initial conditions as those used to illustrate gradi-ent descent; note that convergence is achieved in two steps(total time 4π), but very large oscillations take place.

0 2 4 6 8 10 12 14-200

-150

-100

-50

0

50

100

Fig. 3. Newton, starting from ξ1

0 2 4 6 8 10 12 14-400

-300

-200

-100

0

100

200

300

Fig. 4. Newton, starting from ξ2

Appendix

I. Appendix: Proof of Nonsingularity Result

We first recall the fact, mentioned above, that for each xthere is a control nonsingular for x. This can be proved asfollows. Pick x, and assume that the system (1) is stronglyaccessible. Let y be in the interior of RT (x), for someT > 0. It follows from [15], Lemma 2.2 and Proposition2.3, that there exists some real number δ > 0 and somepositive integer k so that y is in the interior of the imageof

F : Uk → X , (u1, . . . , uk) 7→ exp (δfu1) . . . exp (δfuk)(x) ,

where we are using the notation exp (δfu)(z) = φ(δ, z, ω)for the control ω ≡ u on [0, δ]. This map F is smooth, soby Sard’s Theorem it must have full-rank Jacobian at some

10 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 40, 1995:413-440

point (u01, . . . , u

0k). This implies that the piecewise-constant

control ω, defined on [0, kδ] and equal to the values u0i

on consecutive intervals of length δ, is nonsingular for thegiven state x, as desired.

We next need what is basically a restatement of the mainresults in [20]:

Proposition A.1: Consider the (analytic) system (1) andassume that h : X → IR is a real-analytic function. Let Gbe the set of states x so that, for some control ω = ω(x),h(φ(·, x, ω)) is not identically zero. Then, there exists ananalytic control ω∗ so that, for every x ∈ G, h(φ(·, x, ω∗))is not identically zero; moreover, for each T > 0, the set ofsmooth such controls is generic in C∞([0, T ],U).

Proof: We consider the extended system (with statespace X × IR):

x = f(x, u)z = 0y = zh(x) ,

which is an analytic system with outputs. Consider twostates of the form (x, 0) and (x, 1), with x ∈ X . A controlω distinguishes these states if and only if h(φ(·, x, ω)) is notidentically zero.

Let ω∗ be a control for the extended system which isuniversal with respect to observability. There are analyticsuch controls, and the desired genericity holds, by Theo-rems 2.1 and 2.2 in [20]. Now pick any x in the set G.Then (x, 0) and (x, 1) are distinguishable, and hence ω∗

distinguishes among them. This means that h(φ(·, x, ω∗))is not identically zero, as desired.

We now prove Theorems 1 and 2. Let (1) be given, andtake the composite system consisting of (13) and (15) withoutput h(x,Q) = detQ, This is seen as a system with statespace X ×IRn×n. For an initial state of the form z = (x, 0),and a control ω, the solution φ of the larger system at timet, if defined, is so that

h(φ(t, z, ω)) = det(∫ t

0

Φ(t, s)B(s)B∗(s)Φ(t, s)∗ ds)

(where Φ denotes the fundamental solution of the lin-earized equation), so ω is nonsingular for x precisely whenh(φ(t, (x, I, 0), ω)) is not identically zero.

By the remarks made earlier, strong accessibility guar-antees that every state of the form (x, I, 0) is in the set Gdefined in Proposition A.1 (for the enlarged system); thusour Theorems follow from the Proposition.

References

[1] Bloch A.M., N.H. McClamroch, and M. Reyhanoglu, “Controland Stabilization of Nonholonomic Caplygin Dynamic Systems,”Proc. IEEE Conf. Decision and Control, Brighton, UK, Dec.1991, IEEE Publications, 1991, pp. 1127-1132.

[2] Brockett, R.W., “Asymptotic stability and feedback stabiliza-tion,” in Differential Geometric Control Theory (R.W.Brockett,R.S.Millman, and H.J.Sussmann, eds.), Birkhauser, Boston,1983, pp. 181-191.

[3] Brockett, R.W., “Pattern generation and feedback control ofnonholonomic systems,” paper prepared for Workshop “Mechan-ics, Nonholonomy, and Control” at IEEE Conf. Decision andControl, San Antonio, Dec. 1993 .

[4] Bryson, A.E., and Y-C. Ho, Applied Optimal Control , Wiley,NY, 1969.

[5] Coron, J-M., “Global asymptotic stabilization for controllablesystems without drift,” Math of Control, Signals, and Systems5(1992): 295-312.

[6] Coron, J-M., “Linearized control systems and applications tosmooth stabilization,” SIAM J. Control and Opt., 32(1994):358-386.

[7] Divelbiss, A.W., and J.T. Wen, “A global approach to nonholo-nomic motion planning,” Proc. IEEE Conf. Decision and Con-trol, Tucson, Dec. 1992, IEEE Publications, 1992, pp. 1597-1602.

[8] Grasselli, O.M., and A. Isidori, “An existence theorem forobservers of bilinear systems,” IEEE Trans. Autom. Control26(1981): 1299-1301.

[9] Gurvits, L., “Averaging approach to nonholonomic motion plan-ning,” in Proc. of the 1992 IEEE Int. Conf. Robotics and Au-tomation, Nice, France, May 1992, pp. 2541-2546, IEEE Publi-cations.

[10] Hasdorff, L., Gradient optimization and nonlinear control , Wi-ley, New York, 1976.

[11] Lafferriere G., “A General Strategy for Computing Steering Con-trols of Systems Without Drift,” Proc. IEEE Conf. Decision andControl, Brighton, UK, Dec. 1991, IEEE Publications, 1991, pp.1115-1120.

[12] Murray R.M., and S.S. Sastry, “Steering Nonholonomic Systemsin Chained Form,” Proc. IEEE Conf. Decision and Control,Brighton, UK, Dec. 1991, IEEE Publications, 1991, pp. 1121-1126.

[13] Pomet, J-B., “Explicit design of time-varying stabilizing controllaws for a class of controllable systems without drift,” Systemsand Control Letters 18(1992): 147-158.

[14] Sontag, E.D., “On the observability of polynomial systems,”SIAM J.Control and Opt., 17(1979): 139-151.

[15] Sontag, E.D., “On the preservation of certain controllabil-ity properties under sampling,” in Developpement et Utiliza-tion d’Outiles et Modeles Mathematiques pour l’Automatique,l’Analyse de Systemes et le Traitement du Signal, (I.D.Landau,ed.), pp.623-637, Editions CNRS, Paris, 1983.

[16] Sontag, E.D., “Controllability and linearized regulation,” IEEETrans. Automatic Control 32(1987): 877-888.

[17] Sontag, E.D., “Finite dimensional open-loop control generatorsfor nonlinear systems,” Int. J. Control 47 (1988): 537-556.

[18] Sontag, E.D., Mathematical Control Theory: Deterministic Fi-nite Dimensional Systems, Springer, New York, 1990.

[19] Sontag, E.D., “Universal nonsingular controls,” Systems andControl Letters 19(1992): 221-224. Errata in 20(1993): 77.

[20] Sussmann, H.J., “Single-input observability of continuous-timesystems,” Math. Systems Theory 12(1979): 371-393.

[21] Sussmann, H.J., “A continuation method for nonholonomic pathfinding problems,” in Proc. IEEE Conf. Decision and Control,San Antonio, Dec. 1993 , IEEE Publications, 1993, pp. 2718-2723.

[22] Sussmann, H.J., and V. Jurdjevic, “Controllability of nonlinearsystems,” J. Diff. Eqs. 12(1972): 95-116.

[23] Wang, Y., and E.D. Sontag, “Orders of input/output differen-tial equations and state space dimensions,” SIAM J. Controland Opt., to appear. (Preliminary version in “Orders of I/Oequations and uniformly universal inputs,” in Proc. IEEE Conf.Decision and Control, Orlando, Dec. 1994, IEEE Publications,1994, pp. 1270-1275.)