CHAPTER 1

Introduction to Nonlinear Dynamical Systems

Dynamical systems are mathematical systems characterized by a state that evolves over time under the actionof a group of transition operators. Formally, let X and U denote linear spaces that are called the state spaceand input space, respectively. Let G be a strongly ordered group of time indices over which the system’s stateswill evolve. Since G is strongly ordered, there is an order relation, , such that s t, t s or s = t for anys, t 2 G. Moreover for any s, t 2 G we can define the interval [s, t] as the set of indices ⌧ 2 G such thats ⌧ and ⌧ t when s t or s � ⌧ and ⌧ � t if t s. Let L(G, X) and L(G, U) denote the linear spaceof signals mapping a time index t 2 G onto a system state x(t) 2 X or system input u(t) 2 U , respectively.A dynamical system may be denoted by the tuple, S = (X, G,U, �), where � : G ⇥ X ⇥ L(G, U) ! X is acontinuous map whose values, �(t; x

0

, u) 2 X , represent the state of the system at time t 2 G assuming aninitial state of x

0

2 X under the action of an input signal u 2 L(G, U). The map � characterizes the mannerin which the system’s state transitions over time and so is sometimes called a transition operator. If the setof time indices, G, is the set of integers, Z, then the system is said to be discrete-time. If G equals the set ofreals, R, then the system is continuous-time and � is often referred to as the flow of the dynamical system.The flow operator is assumed to satisfy the following property for any s, t 2 G,

�(s + t; x0

, u) = �(s; �(t; x0

, u[s,s+t]), u[0,s])

�(0; x0

, u) = x0

(1)

where u 2 L(G, U) and u[s,t] is the restriction of u to the interval [s, t] 2 G. The preceding equation

formalizes what we sometimes call the group action of the transition operator. When the initial condition x0

and input signal u are assumed to be known, then it is customary to denote the state trajectory as a functionx 2 L(G, X) that takes values x(t) = �(t; x

0

, u) for all t 2 G.

These lectures focus on dynamical systems that are nonlinear in the sense that they do not satisfy the principleof superposition. Consider a dynamical system, S = (X, G,U, �), where X and U are linear spaces definedover the same field. Let u

1

, u2

2 L(G, U) be any two different input signals to the dynamical system S. Thissystem is said to be linear if

�(t; x, ↵u1

+ �u2

) = ↵�(t; x, u1

) + ��(t; x, u2

)(2)

for all t 2 G and any ↵, � in the field used to define the state and input spaces. The system is said to benonlinear if it does not satisfy the principle of superposition.

Linearity is extremely useful in the design and regulation of engineering systems. In particular, consider asignal u 2 L(G, U) that is formed from the linear combination of a set of basis signal, {u

1

, u2

, . . . , un}. If

1

2 1. INTRODUCTION TO NONLINEAR DYNAMICAL SYSTEMS

the system is linear then the response of this system to the input u can be determined by analyzing how thesystem responds to the basis signals and then taking the linear combination of these responses. The key, ofcourse, is to find a basis for which the response is easy to evaluate. Because we are focusing on nonlinearsystems, however, we will not be able to directly use linearity to help us understand the system’s behavior.

These lectures confine their attention to nonlinear dynamical systems that are causal. In particular, considera signal u 2 L(G, U) and define the truncation of that signal to a time index T 2 G as

uT (t) =

(

u(t) if T 0

0 otherwise(3)

In a similar way we can define the truncation of signals in L(G, X). The dynamical system S = (X,G, U, �)

is said to be causal if and only if for any T 2 G

�(t; x, uT ) = �(t; x, u) for t < T(4)

This equation asserts that the state of the system prior to time T under input u is equal to the system stateprior to T under the truncated input uT . In other words, the past behavior of the state prior to T is unaffectedby inputs after time T . Essentially asserting that the ”future” has no impact on the ”past”.

Let us consider a causal nonlinear dynamical system S = (X, G, U, �) with the zero input 0 2 L(G, U). Itwill be convenient to define a “zero-input” map �t : X ! X where t 2 G as

�t(x0

) = �(t; x0

, 0)

From equation (1) it should be apparent that the zero-input map �t satisfies the group relations

�s+t = �s�t, �

0

= I

where I is the identity map on X . Note that if the input u 2 L(G, U) is given, then the same notation is usedfor the map �t : X ! X that takes values �t(x0

) = �(t; x0

, u).

For a dynamical system with zero (or any fixed) input u 2 L(G, U), one can define a binary relation ⇠ on X

in which x ⇠ y if and only if there exists t 2 G such that �t(x) = y. This relation is clearly an equivalencerelation due to the group action of the transition operator. This means that one can use ⇠ to partition the statespace X into equivalence classes of ⇠. These equivalence classes are called orbits of the dynamical system.

Linear dynamical systems exhibit a very limited set of qualitative behaviors; convergence or divergence froma unique fixed point or periodic orbits about a fixed point. Because of the principle of superposition it ispossible to obtain a complete theory for linear dynamical systems that renders the analysis and regulation ofsuch systems highly tractable. The same cannot be said for causal nonlinear dynamical systems. Nonlinearsystems exhibit a wider range of qualitative behaviors than those found in linear systems. These nonlinearbehaviors include convergence (divergence) to a fixed point, multi-stationarity (i.e. multiple fixed points),limit cycles, periodic orbits, homoclinic and heteroclinic orbits, chaotic attractors, and bursting. The analysisand control of such behaviors requires new tools that don’t rely as heavily on the principle of superpositionand the purpose of these lectures is to provide an overview of these tools.

1. DISCRETE-TIME SYSTEMS BASED ON THE LOGISTIC MAP 3

This chapter does a quick tour through some of the complexities encountered in the analysis and controlof nonlinear dynamical systems. These examples will serve to motivate the selection of the topics in thefollowing lectures. The remainder of this chapter is organized as follows. Section 1 uses the logistic mapto illustrate how complex behaviors arise in discrete-time systems. Section 2 uses the Duffing oscillatorto illustrate how complex behaviors arise in continuous-time systems. The chapter then turns to examinesome specific applications involving the control of continuous-time nonlinear dynamical systems. Section3 examines the regulation problem in nonlinear control where feedback is used to linearize the input-outputmap, section 4 examines issues that arise in model reference adaptive control of linear systems, and section5 examines reachability problems in nonlinear dynamical systems. Section 6 concludes with remarks thatoutline how the future chapters will unfold.

1. Discrete-time Systems based on the Logistic Map

A nonlinear dynamical system, S = (X,G, U, �), will be said to be discrete-time if the index set, G, is theset of integers, Z. Discrete-time systems can exhibit a wide range of behaviors that include convergence to afixed point, limit cycles, and chaos. As an example, let us consider a discrete-time system defined on the realline X = [0, 1] ⇢ R and whose whose zero-input map, �

1

: R ! R, takes values

�

1

(x) = ax(1 � x)(5)

for x 2 [0, 1] = X with a being a positive real constant. Note that �n = �

n1

for any non-negative integern so we will drop the ”1” subscript in the above equation (5). The map � in equation (5) is often called thelogistic map.

One may characterize the state trajectories, x : Z ! [0, 1], generated by this system as a sequence {x[k]}1k=0

that satisfies the following recursive equation

x[k + 1] = ax[k](1 � x[k]) = �(x[k]), x[0] = x0

(6)

This equation is often used to model how a population of biological organisms changes with each generation.The number of organisms in the kth generation is denoted as x[k] 2 [0, 1]. This particular model assumes thepopulation lives in a box of fixed size and fixed food supply [May73]. The amount of food is modeled by theconstant a > 0. The transition map, � : R ! R, has a graph that forms a parabola with a single maximumat x = 1/2 and a magnitude of a/4. It therefore follows that for a < 4, this function maps the interval [0, 1]

back into itself and so we confine our attention to that set of a.

The choice of logistic function in equation (5) may be justified as follows. When the population is small,one expects x[k + 1] > x[k], or rather one expects the population to be increasing due to abundant foodand living space. As the population grows, however, it eventually finds there is insufficient food and spaceto continue its original rate of growth. This overcrowding scenario causes the population to decline so thatx[k + 1] < x[k] when the population is large. The logistic function in equation (5) has the property that�(x) > x for small x and �(x) < x for large x.

4 1. INTRODUCTION TO NONLINEAR DYNAMICAL SYSTEMS

One may generate the state trajectory, x : Z ! R, for a given initial population x0

in a graphical manner.Let a = 2 (to be concrete) and graph the function, y = �(x), as shown by the blue line on the left side ofFig. 1. Graph the line y = x (red line) in this figure also. First locate the point, (x[0], �(x[0])), on the planeand draw a horizontal line from that point to where it intersects the diagonal red line, y = x. Mark this pointwith a small circle. The x-coordinate of this point represents x[1] and one can use this to locate the point(x[1], �(x[1])) on the plane. This represents the next point of the state trajectory and one can then followthe procedure used in finding x[1] to identify the next point, x[2], of the state trajectory. One continues thisprocedure to generate a sequence of states, {x[k]}1

k=0

, which represents the system’s state trajectory. Theright hand side of Fig. 1 plots this sequence {x[k]} as a function of the time index k. For this choice of a,the population increases from an initial value of x[0] = 0.2 and asymptotically converges to a final value oflimk!1 x[k] ⇡ 0.5.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1x(k)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

f(x(k

))

cobweb plot

a=2

2 4 6 8 10 12 14 16 18 20generation (k)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

popu

latio

n

population state trajectory

FIGURE 1. Cobweb Graph and State Trajectory for Logistic System with a = 2 and x0

= 0.2

One can computationally examine what happens if the food supply, a, is increased without expanding thelogistic map’s positive support (i.e. the living space). The following scenarios will be found

• Extinction: (0 a < 1) - In this case the graph of �(x) never crosses the 45 degree line formed byy = x. In this case x[k] decreases with each generation and asymptotically approaches 0. In other words,for this range of a the population goes extinct due to a lack of food.

• Stable Population: (1 a < 3) - In this case the 45 degree line for y = x intersects the graph y = �(x) attwo points x⇤

u ⌘ 0 and x⇤s ⌘ 1 � 1

a . One can readily verify that the point x⇤s = 1 � 1

a is an asymptoticallystable equilibrium in the sense that x[k] ! x⇤

s as k ! 1. This case, therefore, corresponds to a scenarioin which the population achieves a “stable” fixed level.

• Limit Cycling Population: (3 < a 1 +

p6 = 3.449) - In this case the population goes through a “boom

and bust” cycle or what will be later referred to as a limit cycle. Fig. 2 shows the cobweb plot and statetrajectory for the case when a = 3.4. In this figure the population bounces between two different sizes oneach generation.

• Road to Chaos: (3.449 < a 3.570) - For a just greater than 3.449 = 1 +

p6, the period 2 limit

cycle from the preceding scenario becomes unstable and is replaced by a stable period 4 limit cycle. As

2. CONTINUOUS-TIME DYNAMICAL SYSTEM BASED ON A NONLINEAR OSCILLATOR 5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1x(k)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

f(x(k

))

cobweb plot

h=3.4

2 4 6 8 10 12 14 16 18 20generation (k)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

popu

latio

n

population state trajectory

FIGURE 2. Cobweb Graph and State Trajectory for Logistic System with h = 3.4 and x0

= 0.2

one continues increasing a, the limit cycle period doubles over and over again. A limit cycle of period 8

appears when a = 3.544, period 16 at a = 3.564 and so on. This doubling continues until a > 3.570

at which point the population’s state trajectory becomes chaotic as shown in Fig. 3. The term ”chaos”formally means that the future states vary in a discontinuous manner with the initial condition x

0

. Thechief consequence of this phenomenon is that the state trajectory appears to be ”randomly” switchingbetween different states.

This section showed that even very simple nonlinear discrete-time systems can exhibit a wide range of be-haviors. The next section shows that such complexity is also found in continuous-time nonlinear systems.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1x(k)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

f(x(k

))

cobweb plot

h=3.9

100 200 300 400 500 600 700 800 900 1000generation (k)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

popu

latio

n

population state trajectory

FIGURE 3. Cobweb Graph and State Trajectory for Logistic System with h = 3.9 and x0

= 0.2

2. Continuous-time Dynamical System based on a Nonlinear Oscillator

A nonlinear dynamical system, S = (X, G, U, �), is said to be continuous-time if the index set G is theset of reals, R. Continuous-time systems can also exhibit a range of behaviors that include convergence tofixed points, limit cycles, and chaos. As an example, let us consider a forced oscillator known as the Duffing

6 1. INTRODUCTION TO NONLINEAR DYNAMICAL SYSTEMS

oscillator [KB11]. In this case the system’s state trajectory, x : R ! R, satisfies a second order differentialequation of the form

x + �x � !2x + ✏x3

= � cos(⌦t)(7)

where !, �, ✏, and � are all positive real constants.

The Duffing equation (7) may be viewed as a forced oscillator whose spring providesa restoring force that is a cubic function of the system state. When ✏ < 0 thenthe spring is a ”hardening” spring and when ✏ > 0, then this spring is ”soft”. Theequation may be used to model a periodically forced steel beam in which x denotesthe deflection of the beam’s endpoint and the ”spring” refers to the compliance of thebeam.

Numerically integrating equation (7) forward in time shows a periodic limit cycle when � = 0.1, ✏ = 0.25,! = 1, � = 0.5 and ⌦ = 2. Fig. 4 plots the state trajectory in the right pane. The middle pane shows thecurve traced out by the trajectory in the phase space (x(t), x(t)). Since the curve in the phase space is acircle, we clearly see the periodic nature of the trajectory. The last pane in Fig. 4 shows the Poincare sectionof the trajectory.

80 100 120 140 160 180-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5time series

1.8 1.9 2 2.1 2.2

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

phase space

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Poincaré section (GAM=0.5)

FIGURE 4. State trajectory (left), phase plane trajectory (middle), and Poincare section(right) for forced Duffing oscillator (7) with � = 0.1, ✏ = 0.25, ! = 1, � = 0.5, and⌦ = 2.

FIGURE 5. Poincare Map

The Poincare section provides a convenient way of viewing the behaviorof periodic state trajectories. This method fixes a cross section, ⌃, of thephase space and then determine a map, F : ⌃ ! ⌃, that maps a statex 2 ⌃ onto the state trajectory’s first return, F (x), to this cross section.In particular, consider the state trajectory, x : R ! Rn, for an ordinarydifferential equation,

x(t) = f(x(t)), x(0) = x0

and suppose this system has a periodic solution with fundamental periodT . Let x

0

2 Rn be any point through which the periodic solution passes

3. REGULATION OF CONTINUOUS-TIME DYNAMICAL SYSTEMS 7

and let ⌃ be an n�1 dimensional hyperplane that is transverse to the vector field at x0

. In other words, thereis a normal vector, n(x

0

), to ⌃ at x0

such that hf(x0

), n(x0

)i 6= 0. We refer to ⌃ as a cross-section to thevector field. Since x(t) is periodic it will return to a point F (x

0

) 2 ⌃ in finite time as shown in Fig. 5. Themap F which characterizes the first return of points in ⌃ back to ⌃ is what we call the Poincare map.

One can graph the points of the orbit that intersect ⌃ to obtain a ”picture” of the set of return points. Thispicture is called the Poincare section. For the Poincare section in Fig. 4, there is only a single return point.But as we change the magnitude of the forcing function, we obtain more complex types of Poincare sections.In particular the Poincare map obtained when � = 1.5 shows a much more complex structure in Fig. 6. Thissection appears to consist of various strata that fold onto each other in a complex non-deterministic manner.In this case, the system’s orbit is aperiodic and is, in fact, chaotic. The Poincare section in Fig. 6 represents across section of the so-called chaotic or strange attractor for this system.

80 100 120 140 160 180

-2

-1

0

1

2

3time series

-3 -2 -1 0 1 2 3

-3

-2

-1

0

1

2

3phase space

-3 -2 -1 0 1 2

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

Poincaré section (GAM=1.5)

FIGURE 6. State trajectory (left), phase plane trajectory (middle), and Poincare section(right) for forced Duffing oscillator (7) with � = 0.1, ✏ = 0.25, ! = 1, � = 1.5, and⌦ = 2.

The examples presented in the last two sections show that both discrete-time and continuous-time nonlinearsystems are capable of generating complex qualitative behaviors that are not present in linear systems. Ob-viously, a major goal of these lectures is to develop those tools that allow one to characterize and ultimatelymanage these behaviors. For the rest of these lectures we focus on continuous-time dynamical systems whosestate trajectories satisfy systems of nonlinear ordinary differential equations. In the following sections we turnto some interesting cases in the “control” of such continuous-time systems in which the “nonlinear” natureof the system plays a major role.

3. Regulation of Continuous-time Dynamical Systems

A major problem in the control of nonlinear dynamical systems is the regulation problem. This sectionexamines the regulation problem for continuous-time dynamical systems that can be characterized by or-dinary differential equations. For a dynamical system called the plant, the regulation problem is to find afeedback controller that regulates the closed-loop system’s state about a desired operating point. A formal

8 1. INTRODUCTION TO NONLINEAR DYNAMICAL SYSTEMS

statement of the problem considers an input-output system whose state trajectory, x : R ! Rn, system outputy : R ! Rm, and control input u : R ! Rp satisfy the following set of equations,

x(t) = f(x, u)

y = h(x, u)

u = k(x)

We assume that the functions f : Rn ⇥ Rp ! Rn and h : Rn⇥ ! Rp ! Rm are known. Given a desiredoutput signal, yd : R ! Rm, the objective is to find the state feedback controller k : Rn ! Rp such that theoutput signal y(t) ! yd(t) as t ! 1.

There are alternative formulations of this regulation problem, but the above statement focuses on so-calledtracking problems in which there is direct access to the system’s state vector, x(t). We’ll find it convenient toaddress this problem by introducing a new signal, z : R ! Rm, that represents the so-called tracking error.This signal takes values

z(t) = y(t) � yd(t)

for all t 2 R. We then formulate the differential equations for this tracking error and look for a controller, k,such that the origin of the ”tracking error system” is an asymptotically stable equilibrium point for the closedloop system.

To make this problem more concrete, let us consider a specific example of a tracking problem. The “plant”is a two-wheeled robotic vehicle shown in Fig. 7. Let F denote the force applied by both wheels alongthe body’s x-axis and let T denote the torque developed by these wheels about the vehicle’s center of mass

which is located at point (x, y) in the plane. We introduce the control vector u(t) =

"

F (t)

T (t)

#

. The state

variables of this system are the plant’s center of mass, x and y, the angle of the body with respect to an inertialreference, ✓, the velocity of the vehicle in the direction of the body’s x-axis, vx, and the angular rate, !, ofthat body angle. With these conventions, the system to be regulated and its equations of motion are shown inthe middle pane of Fig. 7.

x

y

FIGURE 7. Two-wheeled Robot - (left) vehicle geometry - (middle) equations of motion -(right) picture of system

3. REGULATION OF CONTINUOUS-TIME DYNAMICAL SYSTEMS 9

We will examine two methods for determining the regulating controller, both of which involve linearizing theoriginal system equations. The first linearization approach is based on the Hartman-Großman or linearizationtheorem [Har02]. Consider a system x = f(x, u) where u is fixed and let � denote the system’s flow. A pointx⇤ 2 Rn is called an equilibrium point if f(x⇤, u) = 0 for the given u 2 Rm. This equilibrium point for theunforced system (i.e. u = 0) is said to be hyperbolic if the eigenvalues of the Jacobian matrix ,

h

@f@x (x⇤, 0)

i

,evaluated at x⇤ has no eigenvalues with zero real parts. The Hartman-Großman theorem establishes that theflow � in a suitably small neighborhood of a hyperbolic equilibrium is topologically equivalent to the flowsof the linearized system that satisfies

x =

@f

@x(x⇤, 0)

�

(x � x⇤) +

@f

@u(x⇤, 0)

�

u(8)

= A(x � x⇤) + Bu

Topological equivalence means there is a smooth invertible state-space transformation between the flows ofthe nonlinear system and its linearization that preserves the direction of time. Since A and B are real-valuedmatrices, this is a system that is commonly studied in linear systems theory [AM06]. This suggests thatif one were to design a state feedback controller matrix, K, such that the control signal u = K(x � x⇤

)

asymptotically stabilizes the linearized system about this equilibrium point, then we should achieve adequateregulation of the nonlinear system.

The first step in developing such a state feedback control is to find the linearization in equation (47) for ourtwo-wheeled cart. We start by introducing the new tracking variables z

1

= x�xd, z2

= y � yd, z3

= ✓ � ✓d,z4

= vx � vd, and z5

= ! where (xd(t), yd(t)) is the trajectory we want our vehicle to track in the plane,✓d(t) is the direction of the desired trajectory’s velocity vector, and vd(t) is the magnitude of that desiredvelocity vector. With this change of variables our system equations in Fig. 7 become

2

6

6

6

6

6

6

6

4

z1

z2

z3

z4

z5

3

7

7

7

7

7

7

7

5

=

2

6

6

6

6

6

6

6

4

(z4

+ vd) cos(z3

+ ✓d) � xd

(z4

+ vd) sin(z3

+ ✓d) � yd

z5

� ˙✓d

�vd

0

3

7

7

7

7

7

7

7

5

+

2

6

6

6

6

6

6

6

4

0 0

0 0

0 0

1 0

0 1

3

7

7

7

7

7

7

7

5

"

u1

u2

#

= F (x) + G(x)u

Computing the Jacobian matrix for F and Gu yields the following linearized system equation2

6

6

6

6

6

6

6

4

z1

z2

z3

z4

z5

3

7

7

7

7

7

7

7

5

=

2

6

6

6

6

6

6

6

4

0 0 �vd sin(✓d) cos(✓d) 0

0 0 vd cos(✓d) sin(✓d) 0

0 0 0 0 1

0 0 0 0 0

0 0 0 0 0

3

7

7

7

7

7

7

7

5

2

6

6

6

6

6

6

6

4

z1

z2

z3

z4

z5

3

7

7

7

7

7

7

7

5

+

2

6

6

6

6

6

6

6

4

0 0

0 0

0 0

1 0

0 1

3

7

7

7

7

7

7

7

5

"

u1

u2

#

= Az + Bu

We can then use a number of methods that design stabilizing controllers for this system. In particular, we’llcompute the linear quadratic regulator (LQR) [AM06] that find the state gains, K, such that the controller

10 1. INTRODUCTION TO NONLINEAR DYNAMICAL SYSTEMS

u = Kz minimizes the cost functional

J [u] =

Z 1

0

(zT z + uTu)d⌧

This controller was simulated in the following MATLAB script (Fig. 8) with the desired trajectory generated(xd, yd) being defined by the following equations

xd(t) = 50 sin

✓

2⇡t

50

◆

, xd(0) = 0

yd(t) = 50 cos

✓

4⇡t

50

◆

, yd(0) = 0

The LQR control was recomputed at each time instant using the desired reference trajectory states. Theresulting vehicle trajectory for a vehicle initially at rest at position (x

0

, y0

) = (50, 0) is shown on the righthand side of Fig. 8. We indeed obtain tracking of the desired reference trajectory, though the vehicle’s initialtransient shows some significant oscillation while it is picking up speed.

-100 0 100 200 300 400 500 600 700 800x

-250

-200

-150

-100

-50

0

50

100

150

200

250

y

desired trajvehicle traj

direction of time

initial positionfinal position

direction of timevehicle tracking

FIGURE 8. (left) MATLAB script - (right) trajectories for linearized control with(x

0

, y0

) = (50, 0)

An important limitation of the preceding linearization approach is that the topological equivalence is onlylocal (i.e. in a neighborhood of the equilibrium point). This suggests that if we were to start the vehiclefurther away from the desired reference trajectory then our control strategy might fail. This indeed is thecase for our system. In particular, if we change the initial condition to (x

0

, y0

) = (55, 0), then we obtain thesystem trajectory shown in Fig 9. In this case, we see the vehicle simply spins around close to its startingposition while it is trying to gather sufficient speed to catch up to the desired state trajectory. In this case thevehicle was never able to track the desired reference trajectory.

3. REGULATION OF CONTINUOUS-TIME DYNAMICAL SYSTEMS 11

-400 -200 0 200 400 600 800x

-200

-150

-100

-50

0

50

100

150

200

y

desired trajvehicle traj

direction oftime

vehicle is lost

FIGURE 9. Trajectories for linearized con-trol with (x

0

, y0

) = (55, 0)

The “local” nature of our control is an important limitation ofthe linearization approach used above. One way of overcom-ing this limitation is to adopt a more sophisticated feedbacklinearization [Isi95] method that uses feedback to force thesystem to appear as a linear dynamical system. This feedbacklinearization is a state transformation that transforms the orig-inal system states onto a state vector consisting of the desiredtracking outputs and their derivatives. The advantage of this ap-proach is that if that state transformation is “global”, then thecontrols we develop for this “feedback” linearized system arealso global and can thereby ensure asymptotic tracking of thereference trajectory for any initial vehicle condition. We nowdescribe this feedback linearization approach and demonstratethat it indeed is able to overcome the limitations of the original linearization method.

In the feedback linearization approach we will find it convenient to introduce a change of control variables inwhich

u1

(t) = F (t) =

Z t

0

v1

(s)ds

u2

(t) = T (t) = v2

(t)

This transformation may be seen as backstepping the original control u1

, through an integrator and using theresulting input, v

1

, to that integrator as our new control variable. The original control, u1

= F , is then treatedas another system state, thereby extending the state vector of the original system. With this change of controlvariable we obtain the following state equations for our cart,

d

dt

2

6

6

6

6

6

6

6

6

6

4

x

y

✓

vx

!

F

3

7

7

7

7

7

7

7

7

7

5

=

2

6

6

6

6

6

6

6

6

6

4

vx cos ✓

vx sin ✓

!

F

0

0

3

7

7

7

7

7

7

7

7

7

5

+

2

6

6

6

6

6

6

6

6

6

4

0 0

0 0

0 0

0 0

0 1

1 0

3

7

7

7

7

7

7

7

7

7

5

"

v1

v2

#

We now introduce a state transformation which is obtained by taking the derivatives of the tracking error.This means that the first three states are

z1

= x � xd, z2

= x � xd, z3

= x � xd

and the second three states are obtained from the derivatives of the y component,

z4

= y � yd, z5

= y � yd, z6

= y � yd

12 1. INTRODUCTION TO NONLINEAR DYNAMICAL SYSTEMS

The differential equations for these components are then readily computed as

z1

= vx cos ✓ � xd = z2

z2

= F cos ✓ � vx! sin ✓ � xd = z3

z3

= v1

cos ✓ � vxv2

sin ✓ � (2F! sin ✓ + vx!2

cos ✓) � ...xd

z4

= vx sin ✓ � yd = z5

z5

= F sin ✓ + vx! cos ✓ � yd = z6

z6

= v1

sin ✓ + vxv2

cos ✓ + (2F! cos ✓ � vx!2

sin ✓) � ...y d

These equations have the form of two chains of integrators driven by the inputs into states z3

and z6

. Thismeans we can rewrite the above differential equations in the following form,2

6

6

6

6

6

6

6

6

6

4

z1

z2

z3

z4

z5

z6

3

7

7

7

7

7

7

7

7

7

5

=

2

6

6

6

6

6

6

6

6

6

4

0 1 0 0 0 0

0 0 1 0 0 0

0 0 0 0 0 0

0 0 0 0 1 0

0 0 0 0 0 1

0 0 0 0 0 0

3

7

7

7

7

7

7

7

7

7

5

2

6

6

6

6

6

6

6

6

6

4

z1

z2

z3

z4

z5

z6

3

7

7

7

7

7

7

7

7

7

5

+

2

6

6

6

6

6

6

6

6

6

4

0 0

0 0

1 0

0 0

0 0

0 1

3

7

7

7

7

7

7

7

7

7

5

"

�(2F! sin ✓ + vx!2

cos ✓) � ...xd

2F! cos ✓ � vx!2

sin ✓ � ...y d

#

+

"

cos ✓ �vx sin ✓

sin ✓ vx cos ✓

#"

v1

v2

#

z = Az + E(↵ + ⇢v)

where A is the linear matrix representing the chain of integrators, ↵ and ⇢ are matrices whose componentsare functions of the original system states, F , !, ✓, and vx. Note that if we select the control v to have theform

v = ⇢�1

�↵ +

" ...xd...y d

#

+ Kz

!

(9)

Then the resulting state equation is given by

z = (A + EK)z(10)

where

ET=

"

0 0 1 0 0 0

0 0 0 0 0 1

#

K =

"

k11

k12

k13

k14

k15

k16

k21

k22

k23

k24

k25

k26

#

The important thing to note here is that equation (10) is a linear differential equation and so if we can selectK so that A + EK is a Hurwitz matrix, then we would have globally stabilized our vehicular system usingthe control in equation (9).

4. BURSTING IN ADAPTIVE CONTROL SYSTEMS 13

We can indeed check this out using a MATLAB simulation. The script and results are shown in Fig. 10 fora vehicle at rest with initial positions (x

0

, y0

) = (50, 0) and (x0

, y0

) = (250, 250). These simulation resultsshow convergence to the desired trajectory in which the tracking error appears to be a monotone decreasingfunction of time. The gains were not chosen, in this case to be optimal, they were simply chosen to place allof the system’s closed poles at (�1, 0). It would have been relatively easy to obtain better performance bysimply increasing these gain values. By showing the response from both initial conditions, we demonstratethat the feedback linearized control is indeed “global” in a manner that is far superior to the “local” linearcontroller.

0 100 200 300 400 500 600 700 800x

-200

-150

-100

-50

0

50

100

150

200

250

300y

desired trajx0=(50,0)x0=(250,250)

direction oftime

direction oftime

direction of timeperfect tracking

initial positionfinal position

FIGURE 10. Feedback Linearized Controller (left) script - (right) trajectories

This section examined methods for solving the regulation problem through a linearization of the nonlinearsystem. It was shown that linearizations based on a Taylor series expansion about the system’s equilibriumpoint are inherently “local” in nature which limits the region over which regulation can be assured. A lin-earization approach which uses nonlinear feedback state transformations was shown to provide a “global”way of solving the regulation problem. This approach represents an important technique in nonlinear controlbut it has a number of limitations that will need to be studied more carefully. In particular, it is unclear at thispoint which types of systems this approach can be applied to. More important, however, is the fact that thisapproach essentially “cancels” the nonlinear dynamics of the original plant and substitutes them with stablelinear dynamic. Such cancellations can be very sensitive to modeling uncertainty and this issue will need tobe studied in the future.

4. Bursting in Adaptive Control Systems

Adaptive control systems [NA89, SB89, IS96] are systems that adapt the controller’s gains in response tohow well the controlled system is actually behaving. Even when this approach is used to adapt a linear

14 1. INTRODUCTION TO NONLINEAR DYNAMICAL SYSTEMS

dynamical system, the overall system will be highly nonlinear since the control signal is a product of the gainand the system state; both of which in turn are time-varying. This nonlinearity makes can make it difficultto ensure the adaption rule is well behaved (i.e. stable) and it sometimes leads to unexpected behaviorssuch as bursting [And85]; a phenomenon that occurs when an adaptive system suddenly exhibits a burst ofoscillatory behavior before returning back to normal. Bursting is a qualitative behavior found in biologicalsystems [Rin87] and turbulent fluid flow [KRSR67]. The objective of this section is to illustrate bursting ina simple adaptive control problem and show how the nonlinearities in that system give rise to the burstingphenomena.

As a simple example, let us consider a linear plant whose state x : R ! R satisfies

x = ax + bu(11)

in which a and b > 0 are unknown system constants but in which we assume we can measure the systemstate x(t). We’ll assume, without loss of generality that b > 0, though its exact magnitude is unknown. Theobjective is to select a control signal u : R ! R so that x tracks a reference model whose state xm : R ! Rsatisfies

xm = �amxm + bmr(12)

where am > 0, bm > 0, r(t), and xm(t) are all known. With these conventions, we assume that the controlinput, u, is generated by the following equation

u(t) = k1

x(t) + k2

r(t)(13)

where k1

and k2

are “control gains”. The objective is to select these gains so that the model tracking error,e = x � xm, asymptotically goes to zero as time goes to infinity.

Let us first note that the tracking error satisfies the differential equation

e = x � xm

= ax + b(k1

x + k2

r) + amx � bmr

Note that if we select k⇤1

= �a+am

b and k⇤2

=

bm

b , then

e = ax + b(k⇤1

x + k⇤2

r) + amx � bmr

= ax + b

✓

�a + am

bx +

bmb

r

◆

+ amx � bmr(14)

= �am(x � xm) = �ame

Since am > 0, this would imply that for k1

= k⇤1

and k2

= k⇤2

that e(t) asymptoticaly goes to 0 as t ! 1.

This particular choice for the control gains, however, requires that one know what a and b are ahead of time.Since these system parameters are unknown, our actual control law uses a k

1

and k2

that are not equal to the

4. BURSTING IN ADAPTIVE CONTROL SYSTEMS 15

”optimal” values. For this non-optimal choice, the modeling error, e, satisfies

e = ax + b(k1

x + k2

r) + amx � bmr

= ax + b((k1

� k⇤1

)x + (k2

� k⇤2

)r) + bk⇤1

x + bk⇤2

r + am � bmr

We can then use equation (14) to reduce the above equation to

e = �ame + b(k1

� k⇤1

)x + b(k2

� k⇤2

)r(15)

which more clearly shows how deviations in k1

and k2

from their optimal values impact the model trackingerror.

In model reference adaptive control (MRAC), one seeks an adaptation rule that adjusts the controller param-eters k

1

and k2

to ensure model tracking. This adaptation rule may be expressed as a differential equation

˙k1

= �1

(k1

, x, r, e), ˙k2

= �2

(k2

, x, r, e)(16)

where �1

and �2

are the functions we need to determine. The adaptation rule in equation (16) takes as inputthe current plant output, x, the reference input r and tracking error e to adjust the gains k

1

and k2

. To seewhat a plausible form for these adaptation functions should be, let us consider a cost function

V (e, k1

, k2

) =

1

2

e2

+

b

2�(k

1

� k⇤1

)

2

+

b

2�(k

2

� k⇤2

)

2

where � > 0 is a constant. Note that V � 0 and let us consider the time derivative of this cost, ˙V , as e, k1

,and k

2

follow the error dynamics in equation (15) under the adaptation rule (16). In particular, if one canshow that ˙V 0, then since V is bounded below by zero, then V (t) must be a monotone decreasing functionof time that converges to a limit point.

So let us compute ˙V

˙V =

@V

@ee +

@V

@k1

˙k1

+

@V

@k2

˙k2

= ee +

b

�(k

1

� k⇤1

)

˙k1

+

b

�(k

2

� k⇤2

)

˙k2

(17)

Inserting equation (15) into (17) yields,

˙V = �ame2

+ b(k1

� k⇤1

)(xe +

1

�˙k1

) + b(k2

� k⇤2

)(xr +

1

�˙k2

)

Note that if we select

˙k1

= �1

(k1

, x, e, r) = ��xe˙k2

= �2

(k2

, x, e, r) = ��xr(18)

then we can ensure ˙V < 0 for all e 6= 0 and this would suggest V (t) is a decreasing function of time.

The adaptation rule in equation (18) is sometimes called the MIT rule [Kal58] and it represents one of theearliest adaptive control laws proposed for aerodynamic systems in the 1960’s. Various forms of the MIT

16 1. INTRODUCTION TO NONLINEAR DYNAMICAL SYSTEMS

rule have been used, but one well known variation takes the form

˙k1

= �� x✓+x2 e

˙k2

= ��xr(19)

This is sometimes called the normalized MIT rule since the size of the k1

update is normalized by the size ofthe current system state.

Let us now look at simulations of the normalized MIT rule and see what happens. In this simulation, we letthe plant and reference model be

x = 1.8x + 2u

xm = �3x + 3r

Let us first simulate this with � = 1 and ✓ = 0.1 using the normalized MIT rule from equation (19). Theresults from this simulation are shown in Fig. 11. The top plot shows the state trajectory for x and xm

assuming an input r(t) that switches back and forth between 1 and �1 for t < 100 and then settles to a finalvalue of r(t) = 1 for t > 100. As this plot shows, we quickly have e(t) ! 0 as time goes to infinity. Thelower plot shows k

1

and k2

and the optimal gains (k⇤1

and k⇤2

) for this example. Note that while k1

and k2

approach these optimal values, they do not actually attain those values asymptotically.

0 50 100 150 200 250 300 350 400 450 500time

-3

-2

-1

0

1

2

3x, plantxm, model

0 50 100 150 200 250 300 350 400 450 500time

-3

-2

-1

0

1

2

3k1k2k1stark2star

FIGURE 11. Simulation of Normalized MIT rule for x = 1.8x + 2u and xm = �3x + 3r

with no noise in the measured plant state.

An important thing to note about the simulation in Fig. 11 is that there is no modeling error in the linear plantand there is no measurement noise. Any “real-life” control system must be able to demonstrate some degree

4. BURSTING IN ADAPTIVE CONTROL SYSTEMS 17

of robustness to both modeling error and measurement noise. To see how our proposed adaptive controlworks under these conditions, let us assume that the plant state, x, satisfies the ODE,

x = 1.8x + 0.1x2

+ 2u

y = x + 0.1⌫

where ⌫ is a zero mean white noise process with unit variance. We now use the measured output y, ratherthan the true state in our feedback controller and adaptation rules,

e = y � xm

u = k1

y + k2

r

˙k1

= ��ey

0.1 + y2

˙k2

= ��er

0 50 100 150 200 250 300 350 400 450 500time

-4

-2

0

2

4

6

8x, plantxm, model

0 50 100 150 200 250 300 350 400 450 500time

-8

-6

-4

-2

0

2

4k1k2k1stark2star

burstingphenomenon

FIGURE 12. Simulation example showing bursting when x = 1.8x + 0.1x2

+ 2u andxm = �3x + 3r using output y = x + 0.1⌫ in the control loop where ⌫ is a zero mean unitvariance white noise process.

The simulation results are now shown in Fig. 12. As before the top plot shows the trajectory of the systemstate x and the model reference xm over time. We again see convergence of x to xm but now between time200 and 225 we see a burst occur in which the system becomes highly oscillatory and then recovers trackingagain. This burstiness, in fact, occurs in a sporadic manner with additional bursts occurring over and overagain after the end of simulation time t = 500. The lower plot gives an indication of what is happening. In thiscase, we see that after convergence k

1

and k2

are exhibiting a linear drift and that the bursting phenomenon

18 1. INTRODUCTION TO NONLINEAR DYNAMICAL SYSTEMS

occurs when these gains change sign. We can therefore see that the burst occurs due to a loss of stabilityabout the system’s equilibrium point, this results in an increase in the model tracking error, which allows thesystem to ”recover” from the burst as indicated in the simulation run.

Bursting is undesirable in the context of adaptive control. But there are numerous other systems where thebursting phenomenon plays an important functional role. This occurs in the dynamics of cell membraneswhere the ”burst” corresponds to an impulse representing the transmission of information down a nervecell’s axon. In this case, bursting plays a critical role in how such transmissions are generated. As a con-crete example, let us consider the following dynamical system which represents a simplified version of theFitzhugh-Nagumo model for nerve conduction [Rin87].

v = v � v3

3

� w + (y + I)

w = �(v + a � bw)

The variable v represents the cell membrane’s potential, w represents a ”gating” variable, and I representsthe stimulating current (assumed to be constant). a, b, and � are constant parameters. The preceding modelwas taken from [Rin87]. For a suitable constant I , the membrane potential exhibits a sustained oscillation asshown in the top plot of Fig. 13.

Bursting occurs when a small modulation in the driving currrent I + y is introduced. The modulation term,y, has a “slow” dynamic relative to the membrane potential’s time constants. Following [Rin87], one formof this dynamic is

1

✏y = �v � c � dy

where c and d are positive constants and ✏ is chosen to be very small (0.001). Under these conditions thevariation in the membrane potential exhibit bursting as seen in the bottom plot of Fig. 13.

The burst seen in Fig. 13 is triggered when the current perturbation, y, exceeds a given threshold. Thiscurrent variation is plotted in Fig. 14. What we see is that when that perturbation level gets close to beingpositive, then a burst is triggered. In particular, the mechanism driving this burst is identical to what we sawin the adaptive control example. Namely, when the ”slow” variable” crosses a threshold then it essentiallydestabilizes the ”fast” system. For the cell membrane example that ”fast variable was the membrane potentialand the ”slow” variable was the perturbed current y. In the adaptive control example, the ”fast variable”was the model tracking error e and the ”slow variables” were the gains, k

1

and k2

, that were being adjusted.The destabilizing of either system triggers large variations in the fast variable that, in turn, cause the ”slowvariables” to moderate themselves in a manner that automatically shuts off the burst. This type of behaviorcan only occur in a nonlinear dynamical system since it is the nonlinear interaction between the fast and slowvariables that allows them to regulate each other.

This section examined bursting in nonlinear dynamical systems. The bursting phenomenon occurs becausea state variable that changes in a “slow” manner triggers a change in the asymptotic behavior of the fastdynamical subsystem. Such ”slow” changes may also be viewed as variations in the system’s parameters. So,

4. BURSTING IN ADAPTIVE CONTROL SYSTEMS 19

0 50 100 150 200 250 300 350 400 450 500-4

-3

-2

-1

0

1

2

3Voltage Potential with Constant Current, I

0 50 100 150 200 250 300 350 400 450 500-4

-3

-2

-1

0

1

2

3Voltage Potential with Slow variation in I

bursting

FIGURE 13. Membrane Potential v, (top) constant applied current I - (bottom) modulatedcurrent I + y

0 50 100 150 200 250 300 350 400 450 500-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05variation in current I

burst burst burst burst burst burst burst

FIGURE 14. Current Variation triggers burst when it exceeds threshold

for example, in the adaptive control system, we saw that the closed loop dynamics of the system could bewritten as

x = (a + bk1

)x + b2

r

where a + bk1

determines whether or not x(t) is decreasing to zero or increasing without bound dependingon the sign of a + bk

1

. Essentially what happens here is that the variation in the slow variable triggersan abrupt ”transition” in the system’s qualitative behavior. For the preceding example, these shifts werecoupled back to the dynamics of the original system, but they could also have been generated exogeneously.In either case we can study the behavior of such systems as regime shifts in which “slow” variations insystem parameters trigger an abrupt shift in the system’s overall operating regime. Being able to predictand ultimately manage such regime shifts is an important open research problem in the study of complex

20 1. INTRODUCTION TO NONLINEAR DYNAMICAL SYSTEMS

nonlinear dynamical systems. The analysis tools to be introduced in these lectures will be useful in the futuresolution of such problems.

5. Reachability in Nonlinear Control Systems

Reachability and its dual concept of observability are useful properties of dynamical systems. In particular,consider a linear time-invariant system whose state satisfies

x(t) = Ax(t) + Bu(t)(20)

with initial condition x(0) = 0. We’re interested in finding a control input, u that transitions the system statefrom x(0) = 0 to the state x(T ) = x

1

where T > 0. The state x1

is said to be reachable if there exists afinite time T > 0 and a control input u : [0, T ] ! Rm such that

x1

=

Z T

0

eA(T�⌧)Bu(⌧)d⌧

The system is said to be reachable if every state in Rn is reachable from the origin. The necessary andsufficient condition for the reachability of the linear system in equation (20) is routine material for mostcourses in linear systems theory and linear feedback control. This condition is that

rank

h

A AB A2B · · · An�1Bi

= n(21)

There is nothing in the preceding definition that prevents it from being applicable to nonlinear dynamicalsystems. In particular, one usually examines reachability with respect to nonlinear systems that satisfy thefollowing state equations

x(t) = A(x) + B(x)u

where A(x) is an n⇥n matrix of functions, aij : Rn ! Rn, in which A(0) = 0 and B(x) is an n⇥m matrixof functions, bij : Rn ! Rm. Since we know the flows about a hyperbolic equilibrium of the nonlinearsystem are topologically equivalent to flows about the linearized system’s equilibirum, this suggests that onemight be able to use the reachability condition in equation (21) to determine whether or not the nonlinearsystem is reachable. This is, however, not as simple as it seems on the surface. A simple example can be usedto illustrate how nonlinearity complicates this question. In this example, we will see that while the linearizedsystem is unreachable, the nonlinear one is locally reachable.

Let us consider the rigid body dynamics for a satellite controlled by two gas jets [KLL83]. It will be con-venient to define an inertially fixed reference frame defined a dextral set of 3 vectors, {e

1

, e2

, e3

}, and areference frame attached to the satellite’s body defined by another dextral set of vectors {r

1

, r2

, r3

}. Wedefine the direction cosine matrix of the vehicle as a 3 by 3 matrix R such that Rei = ri for i = 1, 2, 3.This matrix may be obtained through the sequence of rotations {✓

1

, ✓2

, ✓3

} about the body frame {r1

, r2

, r3

}assuming that the body was originally aligned with the inertial frame {e

1

, e2

, e3

}. These angles are called

5. REACHABILITY IN NONLINEAR CONTROL SYSTEMS 21

Euler angles. For i = 1, 2, 3 let ci denote cos(✓i) and si denote sin(✓i) then we obtain the following explicitexpression for the direction cosine matrix,

R =

2

6

6

4

c2

c3

�c2

s3

s2

s1

s2

c3

+ s3

c1

�s1

s2

s3

+ c3

c1

�s1

c2

�c1

s2

c3

+ s3

s1

c1

s2

s3

+ c3

s1

c1

c2

3

7

7

5

Note that the order of the rotations can be changed. The preceding example assumed that we sequencedthrough each body axis sequentially. We could also have defined these angles through a different sequenceof rotations. Fig. 15 shows how these rotations would have occurred if we had used the sequence {✓

3

, ✓1

, ✓2

}rather than the 1 � 2 � 3 sequence.

FIGURE 15. 3-1-2 Rotation of Euler Angles

The body’s velocity vector, ! is defined about the 3 body axes, {r1

, r2

, r3

}. The direction cosines and theangular velocities satisfy the following set of ordinary differential equations

J ! = S(!)J! +

mX

i=1

biui(22)

˙R = S(!)R

where J is the inertia matrix, {bi}mi=1

, are vectors characterizing how the gas jets, ui, for i = 1, 2, 3 impactthe body’s angular acceleration, !, and

S(!) =

2

6

6

4

0 !3

�!2

�!3

0 !1

!2

�!1

0

3

7

7

5

We are more interested in how the three Euler angles change over time since these are tied to the inertialframes. So rather than characterizing the attitude’s time rate of change through its direction cosine matrix(i.e., ˙R = S(!)R), we determine the time derivatives for the Euler angles ✓i, for i = 1, 2, 3. For the 1�2�3

22 1. INTRODUCTION TO NONLINEAR DYNAMICAL SYSTEMS

sequence of body rotations it can be shown that [BI91]

˙✓ =

2

6

6

4

c2

0 s2

s2

s1

/c1

1 �c2

s1

/c1

�s2

/c1

0 c2

/c1

3

7

7

5

!(23)

Equations (22,23) will therefore be used in our example.

For this example, we will see whether or not the origin is reachable from an initial state where ✓1

= �0.3,✓2

= ✓3

= 0, and !1

= !2

= !3

= 0 using only two gas jets aligned with the vehicle’s pitch and yaw axes.We’ll assume that the inertia matrix is J = diag(0.1, 1.0, 1.0). So the vehicle is starting out at rest alignedwith the reference frame but with a roll angle of set of �0.3 radians. Since the two gas jets are aligned to thepitch and yaw axis then in equation (22) we know m = 2, b

1

= [0, 1, 0]

T and b2

= [0, 0, 1]

T .

Let us first look at the linearization of this system about the set point ! = 0 and ✓ = 0. We let the linearizedsystem’s state vector be z

1

= !1

, z2

= !2

, z3

= !3

, z4

= ✓1

, z5

= ✓2

, and z6

= ✓3

. It can be readily shownthat for this fixed point the linearized system’s state equations are

2

6

6

6

6

6

6

6

6

6

4

z1

z2

z3

z4

z5

z6

3

7

7

7

7

7

7

7

7

7

5

=

2

6

6

6

6

6

6

6

6

6

4

0 0 0 0 0 0

0 0 0 0 0 0

0 0 0 0 0 0

1 0 0 0 0 0

0 1 0 0 0 0

0 0 1 0 0 0

3

7

7

7

7

7

7

7

7

7

5

2

6

6

6

6

6

6

6

6

6

4

z1

z2

z3

z4

z5

z6

3

7

7

7

7

7

7

7

7

7

5

+

2

6

6

6

6

6

6

6

6

6

4

0 0

1 0

0 1

0 0

0 0

0 0

3

7

7

7

7

7

7

7

7

7

5

"

u1

u2

#

= Az + Bu

If we use the reachability condition in equation (21) then we see that

rank

h

A AB A2 · · · A5Bi

= 4 < 6

which means the linearized system is not reachable. In particular, this means that we cannot find a controlthat moves the roll angle, ✓

1

, to zero. For the linearization this should be obvious since we see there is nocontrol action that impacts z

1

.

The same, however, may not be said of the original nonlinear system. In particular, let us consider thefollowing control sequence

u =

"

p�

t�0.10.4

�� p�

t�1.80.4

�� p�

t�3.10.4

�

+ p�

t�5.250.4

�

p�

t�0.10.4

�� p�

t�1.80.4

�� p�

t�6.10.4

�

+ p�

t�7.7250.4

�

#

where p(t) = 1 for 0 < t < 1 and is zero elsewhere. A plot of these control jet actions are shown in Fig. 16

We simulated this system using the control jet firing sequence given above. The simulation results are shownin Fig. 17. What we see is that when we accelerate both the pitch and yaw rates at the same time, there isan induced roll rate that will null out the initial roll angle. We can then use the pitch and yaw jets to thenreturn the other angles to zero. What we see in this example, therefore, is that even though the origin isnot reachable from the initial attitude using the linearized system, it is reachable using the original nonlineardynamics. This is because the nonlinear coupling between the different axes allows us to induce a roll rate

6. CONCLUDING REMARKS 23

0 5 10 15-1

-0.5

0

0.5

1u1

0 5 10 15-1

-0.5

0

0.5

1

Pitch Control Jet

Yaw Control Jet

FIGURE 16. Gas jet commands used to null the roll angle with only pitch and yaw control

without actually having a control jet on the roll axis. This particular feature of nonlinear systems can be veryuseful and represents one important way in which nonlinear dynamics differs from what we would expect toaccomplish using linearized dynamics.

0 5 10 15-50

0

50

0 5 10 15-50

0

50

0 5 10 15-50

0

50

roll axis

pitch axis

yaw axis

FIGURE 17. Zeroing out the initial roll angle using only the pitch and yaw control jets

6. Concluding Remarks

This chapter introduced dynamical systems as a mathematical system consisting of a state space and a groupof transitions operators characterizing how a system state changes over time. We saw that this rather abstractdefinition can be used to characterize both continuous-time and discrete-time systems. When these flowsare nonlinear, they can exhibit a rich set of qualitative behaviors; much richer than the behaviors generatedby linear dynamical systems. The diverse behaviors in nonlinear dynamical systems certainly complicates

24 1. INTRODUCTION TO NONLINEAR DYNAMICAL SYSTEMS

our ability to manage and regulate their behavior; but as we saw in the examples it also provides certainopportunities as well.

This chapter presented three examples illustrating how the control of nonlinear systems differs from thatof linear systems. We saw that an important tool for designing such controllers often involve some formof “linearization” of the nonlinear plant. While traditional linearizations based on Taylor series expansionsare only “local” in nature; we also saw that it was possible to develop “global” linearizations as well. Thedistinction between local and global behaviors will be an important issue in our subsequent studies.

We also saw that because nonlinear systems exhibit a larger set of qualitative behaviors, they can sometimesexhibit “shifts” between these behaviors. In the adaptive control example, this shift manifested itself througha bursting phenomenon and was highly undesirable. In other cases, such shifts represent an important partof the system’s desired normal operation. The problem we face here is that such shifts are inherently non-equilibrium behaviors so that the “local” viewpoint inherent in traditional regulation problems is no longerappropriate. We will find it necessary to develop methods for characterizing when such shifts occur and forultimately managing their occurrence.

There are two fundamental problems in feedback control; regulation and reachability. Regulation is essen-tially a stability problem associated with the plant’s long-term behavior. Reachability, by its very definition,is concerned with the finite time behavior of the system. While the characterization of reachability andreachable sets is well understood for linear systems, we saw that nonlinear systems exhibit some surprisingreachability properties; that suggest they may have greater flexibility than our linearized versions of a system.Developing the formal mathematical framework to explain this flexibility will also couple back to our earlierdiscussion of feedback linearization.

The remaining lectures in this book are intended to provide a comprehensive introduction to the control andmanagement of nonlinear dynamical systems that are usually modeled as systems of ordinary differentialequations. The lectures may be divided into three distinct modules. The first module (chapters 2-4) reviewsadvanced material in the theory of ordinary differential equations. Of principle concern to us in that inves-tigation will be the existence and characterization of invariant manifolds about a system’s equilibrium point.These manifolds provide the justification for often using linear controllers in regulating nonlinear systems.The second module (chapter 5-7) examines a number of stability concepts; starting with Lyapunov stabilityfor unforced systems. We then turn to stability concepts for input-output systems (input-to-state stability, Lp

stability, and passivity) and examine some interesting facts regarding the stability of interconnected (feedbackand cascades) systems. We then examine notions of structural stability which is related to the notion of sys-tem bifurcations. The last module (chapter 8-11) investigates specific controller synthesis methods. We firstexamine constructive design methods based on Control Lyapunov functions and introduce the method knownas backstepping. We then examine an older feedback linearization approach to nonlinear control. We thenexamine the use of passivity-based control. The final chapter examines a novel approach ”equation-free” tononlinear control that uses observations of the open loop system behavior to develop hybrid control schemesfor managing complex non-equilibrium processes.