Input to State Stabilizing Control Lyapunov Functions forRobust Bipedal Robotic Locomotion

Shishir Kolathaya, Jacob Reher, Ayonga Hereid and Aaron D. Ames

Abstract— This paper analyzes the input to state stabilityproperties of controllers which stabilize hybrid periodic orbits.Systems that are input to state stable tend to be robust tomodeling and sensing uncertainties. The main contribution ofthis paper is in the construction of control Lyapunov functionsthat do not just stabilize, but also input to state stabilize a givenhybrid system. Bipedal robotic walking, which can be naturallymodeled as a hybrid system, is analyzed under this class ofcontrollers. Specifically, we will select a class of controllersvia rapidly exponentially stabilizing control Lyapunov functionsthat stabilize bipedal robotic walking; typically modeled ashybrid periodic orbits. We will show with simulation resultsthat given the control Lyapunov functions and the associatedset of stabilizing controllers, there exist input to state stabilizingcontrol Lyapunov functions and the associated set of controllersthat input to state stabilize the given periodic orbit.

I. INTRODUCTION

Model based controllers are highly sensitive to imper-fections in real world implementations. This mismatch isespecially relevant in the field of bipedal robotics and canaffect controllers adversely through imperfect sensing, inac-curate parameter estimation, input saturation and unmodeleddisturbances, to name a few. The notion of input to statestability (ISS) [13] captures this uncertainty such that thedeviation from the desired output is a function of the devia-tion from the stabilizing control input. Practical difficulties inthe realization of nonlinear feedback controllers on roboticsystems place heavy constraints on the ability to increasecontrol gains, and thus to improve convergence rates andtracking errors.

Control Lyapunov functions (CLF), popularized by Art-stein and Sontag [12] during the 1980’s, enable the use ofdynamic programming approaches to obtain optimal controlinputs in real-time controllers [4], [3]. The translation ofthis approach to hybrid systems, especially bipedal roboticsystems with underactuation and discrete jumps (impacts),brings with itself a larger challenge. For complex systemssuch as these, investigating input to state stability (ISS),i.e., studying output perturbations for all kinds of inputperturbations seems like an unavoidable task. Indeed, input tostate stability of hybrid systems has been studied extensivelyin literature. Some of the problems addressed are findinga common Lyapunov function [17], [5] and stability under

This work is supported by the National Science Foundation through grantNRI-1526519

Shishir Kolathaya, Jacob Reher, and Aaron D. Ames are with the Schoolof Mechanical & Civil Engineering, California Institute of Technology,Pasadena, CA, USA {sny,jreher,ames}@caltech.edu

Ayonga Hereid is with the School of Electrical & Computer Engineering,University of Michigan, Ann-Arbor, MI, USA ayonga@umich.edu

R0

x

y

z

Rb

qlhp

qlhy

qlhrqrhp

qrhy

qrhr

qwy

qwpqwr

qlkpqrkp

qrapqlapqlar

qrar

qlsqrs

Fig. 1: DURUS robot designed by SRI International incollaboration with AMBER Lab, Pasadena, and DynamicRobotics Laboratory, Corvallis.

fast switching [10]. There are also interesting properties ofseveral hybrid systems with discrete events that can stabilizecontinuous unstable dynamics [15].

Our quest for desirable stabilization properties under un-certainty lends itself naturally to CLFs. The focal point ofthis paper is to study and analyze the ISS properties of CLFsfor stabilizing nonlinear hybrid systems with affine controlinputs; specifically applying to bipedal robots. Through theuse of constructions given by Sontag [11], we show that it isindeed possible to find a subset of input to state stabilizingcontrollers from a given set of stabilizing controllers that areobtained from CLFs. The core advantage is the increase inthe number of choices from just one to infinitely many, anecessity for optimal control approaches. We will considerstable walking gaits, i.e., stable hybrid periodic orbits andobtain the ISS properties of these orbits under an ISSbased controller. Comparisons are also made between twospecific controllers in simulation: feedback linearization andits ISS equivalent, for the humanoid robot DURUS (Fig. 1).Robustness to pushing, uncertain terrain height, and modelperturbations are also shown in the analysis.

The paper is structured as follows: Section II containsa brief preliminary on input to state stability. Section IIIdefines input to state stabilizing control Lyapunov functions

(ISS-CLFs) and describes the construction process for theset of input to state stabilizing controllers via these ISS-CLFs. This section also introduces rapidly exponentiallystabilizing control Lyapunov functions (RES-CLFs) and thecorresponding rapidly exponential input to state stabilizingcontrol Lyapunov functions (Re-ISS-CLFs) that are importantin the context of hybrid periodic orbits. Section IV willintroduce the hybrid robot model of DURUS walking, andthe corresponding control methodologies used to realizewalking. Section V will introduce the main result and SectionVI will show the simulation results and comparisons betweena standard stabilizing controller and its ISS equivalent.

II. PRELIMINARIES ON INPUT TO STATE STABILITY

This section will introduce basic definitions and resultsrelated to input to state stability (ISS); for a detailed surveyon ISS, see [13]. We consider an affine control system

x = f (x)+g(x)u, (1)

with x taking values in Euclidean space Rn, the input u∈Rm,for some positive integers n,m. The mapping f : Rn → Rn

and g : Rn→Rn×m are both Lipschitz and f (0) = 0. We usea feedback control law

u = k(x), k(0) = 0, (2)

that makes the closed loop system

x = f (x)+g(x)k(x), (3)

globally asymptotically stable (GAS). We say that a con-troller, k(x), is stabilizing if it makes the closed loop system(3) GAS. Mathematically, the notion of input/output stabilityarises from the need to find a feedback control law (2) withthe property that the new control system

x = f (x)+g(x)k(x)+g(x)d, (4)

be input to state stable where d is called the disturbanceinput, which belongs to Lm

∞, i.e., ‖d‖∞ := supt≥0{|d(t)|}.Here, | . | is the Euclidean norm. It is a well known factthat the feedback control law k(x) which achieves states-pace stabilization does not necessarily produce input/outputstabilization [13]. It is specifically the classes of systemssatisfying this property that are of interest to us.

We will define ISS for the dynamics of the form (4).We will be utilizing comparison functions K , K L andK∞, detailed definitions of which can be found in [13]. Itis important to note that the input considered for ISS is thedisturbance d. Therefore, all ISS and related definitions arew.r.t. d. Let x(t,d) be the time solution of (4).

Definition 1: The system (4) is input to state stable (ISS-able) if there exists β ∈K L , and ι ∈K∞ such that

|x(t,x0,d)| ≤ β (|x0|, t)+ ι(‖d‖∞), ∀x0,d,∀t ≥ 0.

Definition 2: The system (4) is exponential input to statestable (e-ISSable) if there exists β ∈K L , ι ∈K∞ and apositive constant λ > 0 such that

|x(t,x0,d)| ≤ β (|x0|, t)e−λ t + ι(‖d‖∞), ∀x0,d,∀t ≥ 0.

Input to state stable Lyapunov functions. A direct conse-quence of using ISS concepts is the construction of input tostate stable Lyapunov functions (ISS-Lyapunov functions).

Definition 3: A smooth function V : Rn→R≥0 is an ISS-Lyapunov function for (4) if there exist functions α , α , α ,ι ∈K∞ such that ∀x,d

α(|x|)≤V (x)≤ α(|x|)V (x,d)≤−α(|x|)+ ι(‖d‖∞). (5)

We also have the exponential estimate:

V (x,d)≤−cV (x)+ ι(‖d‖∞), (6)

which is then called the e-ISS-Lyapunov function. It wasshown in [13] that a system of the form (4) is ISSable iffit admits a smooth ISS-Lyapunov function. Therefore, forsystems that are of interest to us (bipedal robots), we willestablish ISS via the construction of ISS-Lyapunov functions.

We say that (1) is smoothly stabilizable, if there is asmooth map k :Rn→Rm with k(0) = 0 such that (3) is GAS.(1) is smoothly input to state stabilizable (ISSabilizable) ifthere is a k so that (4) is ISSable. Accordingly, we say thatthe controller k is an input to state stabilizing (ISSabilizing)controller of (1). This can be generalized to define the set ofISSabilizing controllers (i.e., not just one k) via CLFs.

III. INPUT TO STATE STABILIZING CONTROL LYAPUNOVFUNCTIONS

The goal of this section is to derive the set of controllersfrom CLFs that ISSabilize (1). CLFs are obtained for thecontrol input u, and the ISS conditions are satisfied forthe disturbance input d. These CLFs are then called inputto state stabilizing control Lyapunov functions (ISS-CLFs)[8]. Towards the end of this section, we will derive theset of ISSabilizing controllers from rapidly exponentiallystabilizing control Lyapunov functions (RES-CLFs) that areimportant leading into the next section (for hybrid systems).

Input to state stabilizing control Lyapunov functions. Wedefine here a subclass of CLFs that render (1) input to statestable. See [8] for the original definition.

Definition 4: A continuously differentiable function V :Rn→ R≥0 is an input to state stabilizing control Lyapunovfunction (ISS-CLF), if there exists a set of controls U⊂Rm,and α, α,α, ι ∈K∞ such that ∀x,d

α(|x|)≤V (x)≤ α(|x|)infu∈U

[L fV (x)+LgV (x)(u+d)]≤−α(|x|)+ ι(‖d‖∞),

where L f ,Lg are Lie derivatives. Motivated by constructionsdeveloped by Sontag, specifically [11, equations (23) and(32)], we can construct ISS-CLFs in the following manner.Consider the following controller which ISSabilizes (1):

u = k(x)− 1ε

LgV (x)T , (7)

for some ε > 0. Based on this controller, we have thefollowing Lemma which defines a new CLF that input tostate stabilizes the system (1).

Lemma 1: The continuously differentiable function V :Rn→ R≥0 defined for α, α,α ∈K∞ as

α(|x|)≤V (x)≤ α(|x|) (8)

infu∈U

[L fV (x)+LgV (x)u+α(|x|)+ 1ε

LgV (x)LgV (x)T ]≤ 0,

is an ISS-CLF ∀ ε > 0.Proof: After substituting (8) in derivative of V

V (x,u,d) = L fV (x)+LgV (x)u+LgV (x)d (9)

≤−α(|x|)− 1ε

LgV (x)LgV (x)T +LgV (x)d.

Since LgV (x) ∈ R1×m, LgV LgV T = |LgV |2 ≥ 0. We have thefollowing inequality after adding and subtracting ε

‖d‖2∞4

V (x,d)≤−α(|x|)−(

1√ε|LgV (x)|−

√ε‖d‖∞

2

)2+ ε‖d‖2

∞

4

≤−α(|x|)+ ε‖d‖2

∞

4, (10)

which is of the form (5). It can be observed that an excellentway to reduce the effect of ‖d‖∞ is by decreasing ε .

Rapidly exponentially stabilizing control Lyapunov func-tions. With the goal of obtaining stronger convergence rates(especially used for hybrid systems like bipedal robots), arapidly exponentially stabilizing control Lyapunov function(RES-CLF) is constructed that stabilizes the output dynamicsat a rapidly exponential rate (see [3] for more details) througha user defined ε > 0.

Definition 5: The family of continuously differentiablefunctions Vε :Rn→R≥0 is a rapidly exponentially stabilizingcontrol Lyapunov function (RES-CLF) if there exists a set ofcontrols U ⊂ Rm, and positive constants c1,c2,c3 > 0 suchthat for all 0 < ε < 1, x,

c1‖x‖2 ≤Vε(x)≤c2

ε2 ‖x‖2 (11)

infu∈U

[L fVε(x)+LgVε(x)u+c3

εVε(x)]≤ 0.

Therefore, we can define a class of controllers Kε :

Kε(x) = {u ∈ U : L fVε(x)+LgVε(x)u+γ

εVε(x)≤ 0}, (12)

which yields the set of control values that satisfies the desiredconvergence rate.

If a RES-CLF also satisfies the conditions for ISS, thenwe have rapidly exponential input to state stabilizing controlLyapunov functions (Re-ISS-CLF). We therefore have thefollowing Lemma which provides Re-ISS-CLFs.

Lemma 2: The continuously differentiable function Vε :Rn→ R≥0 defined for c1,c2,c3 > 0 as

c1‖x‖2 ≤Vε (x)≤c2

ε2 ‖x‖2 (13)

infu∈U

[L f Vε (x)+LgVε (x)u+c3

εVε (x)+

1ε

LgVε (x)LgVε (x)T ]≤ 0,

is an Re-ISS-CLF ∀ 0 < ε < 1, ε > 0.

Fig. 2: Hybrid system model for the walking robot DURUS.

Proof of this is similar to (10). We therefore have the set

Kε,ε (x) = {u ∈ U : L f Vε (x)+LgVε (x)u+c3

εVε (x) (14)

+1ε

LgVε (x)LgVε (x)T ≤ 0}.

It can be verified that Kε,ε ⊆Kε (the set obtained from (14)is a subset of (12)). To summarize, for systems of the type(1), we showed here that we can create a set of ISSabilizingcontrollers via the RES-CLF. The purpose of Re-ISS-CLFswill be more clear in Section IV.

IV. HYBRID SYSTEMS

In this section, we will discuss the hybrid control systemmodel of DURUS walking. DURUS is an underactuated 23-DOF bipedal robot designed by SRI International (see Fig. 1)with 15 actuators and 2 springs.

Model. The walking model has two continuous events,double support (ds) and single support (ss), and two discreteevents, lift-off (ds → ss) and foot-strike (ss → ds), thatalternate between each other. We, therefore, have a directedgraph, Γ = (V,E), with the set of vertices, V = {ds,ss},representing the continuous events and the set of edges,E = {(ds,ss),(ss,ds)} ⊂ V× V, representing the discreteevents. A pictorial representation of these individual eventsand the switch between them are shown in Fig. 2. Given theconfiguration q ∈ Q ⊂ Rn of the robot, where n = 23, wehave the following continuous dynamics for each phase v:

D(q)q+H(q, q) = Bvuv +J Tv Fv

Jvq+Jvq = 0. (15)

Description of the notations can be found in [6]. Specifically,uv ∈ Rmv is the control input, with mds = 9, mss = 15. Notethat we can also represent the dynamics (15) in terms of thestate x := (q, q): x = fv(x)+gv(x)uv.

Hybrid control system. The hybrid control system modelof DURUS is a tuple H C = (Γ,U,D,S,∆,FG), withthe directed graph Γ = (V,E), the set of inputs U ={Uds,Uss}, the set of domains D = {Dds,Dss}, the setof guards S = {Sds,Sss}, the set of switching func-tions ∆ = {∆(ds,ss),∆(ss,ds)}, and the set of fields FG ={( fds,gds),( fss,gss)}. Note that Uv⊂Rmv , Dv⊂ TQ×Uv, forv ∈ V. Denote the projection of the domain and guard setsto the states (only) as Sv|x,Dv|x respectively. More detailson the hybrid system model of DURUS, and, in general, ofwalking robots can be found in [6], [16].

Trajectory tracking control. In the problem of trajectorytracking we drive k1,v relative degree one outputs

y1,v(q, q) = ya1,v(q, q)− yd

1,v(αv), (16)

and k2,v relative degree two outputs

y2,v(q) = ya2,v(q)− yd

2,v(q,αv), (17)

to zero, with v denoting the domain, α denoting the param-eters of the desired trajectory. The total dimension of theoutputs k1,v + k2,v = kv is typically equal to the number ofinputs mv. Since we are interested in seeking a set of controllaws, we will use CLF based controllers. Specifically, we willuse the CLF based controllers derived from IO linearization.

uIO = M−1v

(−[

L fvy1,vL2

fvy2,v

]+µv

), Mv =

[Lgvy1,v

LgvL fvy2,v

], (18)

where µv ∈Uv⊂Rmv denotes the auxiliary input applied afterIO linearization. Let ηv := (y1,v,y2,v, y2,v). Applying (18) forthe dynamics of (16), (17) results in the following:

ηv =

0 0 00 0 1k2,v

0 0 0

︸ ︷︷ ︸

Fv

ηv +

1k1,v 00 00 1k2,v

︸ ︷︷ ︸

Gv

µv. (19)

1k1,v ,1k2,v are identity matrices of appropriate sizes. The RES-CLF is thus obtained as Vε,v := ηT

v Pε,vηv, where Pε,v is thesolution to the CARE [3, equation (47)].

Re-ISS-CLF. Similar to (4), we are interested in the behaviorof DURUS when the controller of the form (18) is appliedwith a disturbance: uIO +d, resulting in

ηv = Fvηv +Gvµv +GvMvd. (20)

For the robot model considered (15), the matrix of terms Mvshown above is bounded (also invertible (18)). Hence, bydenoting dv := Mvd, we will construct control laws for theauxiliary input µv, and then study ISS properties for inputsof the type µv+dv. We have the set of stabilizing controllersobtained from the RES-CLF Vε,v

Kε,v(ηv) = {µv ∈ Uv : ω0,v(ηv)+ω1,v(ηv)µv ≤ 0}, (21)

and the set of ISSabilizing controllers from Vε,v

Kε,ε,v(ηv) = {µv ∈ Uv : ω0,v(ηv)+ω1,v(ηv)µv (22)

+1ε

ω1,v(ηv)ω1,v(ηv)T ≤ 0},

where

ω0,v(ηv) = LFvVε,v(ηv)+γv

εVε,v(ηv), with 0 < γv < 1

LFvVε,v(ηv) = ηTv (F

Tv Pε,v +Pε,vFv)ηv

ω1,v(ηv) = LGvVε,v(ηv) = 2ηTv Pε,vGv. (23)

Note that the set (22) would seem like an overcompensationfor the already linearized transverse dynamics (19). Sincewe are dealing with hybrid systems undergoing nonlinearimpacts, we still need the set (22) with the objective of

minimizing the disturbance effects via ε (like in (10)).Similar to (10), we have the following derivative of Vε,v:

Vε,v = LFvVε,v +LGvVε,vµv +LGvVε,vdv

≤−γv

εVε,v +

ε‖dv‖2∞

4, if µv(ηv) ∈Kε,ε,v(ηv). (24)

It can be verified from (24) that ε helps minimize the effectsof ‖dv‖∞ without the need to modify the rate ε .

Partial zero dynamics. As previously mentioned, DURUS isan underactuated robot consisting of two types of dynamics

ηv = Fvηv +Gvµv, zv = Ψv(ηv,zv), (25)

called the transverse and passive dynamics respectively. Inaddition, with the convergence of relative degree two outputsη2,v := (y2,v, y2,v)→ 0, we have partial zero dynamics [2]

y1,v = µ1v, zv = Ψv(y1,v,0,zv), (26)

where µ1v is the first element in µv, and the arguments inΨv are separated into three types of coordinates: (ηv,zv) =(y1,v,η2,v,zv). Accordingly, we can define the diffeomor-phism Φv : Dv|x→ R2n that maps from x to (ηv,zv):

Φv(x) =

Φ1,v(x)Φ2,v(x)Φ3,v(x)

=

y1,v(q, q)y2,v(q)

y2,v(q, q)zv(q, q)

(27)

ΦPZv (x) =

[Φ1,v(x)Φ3,v(x)

], Φ

η2v (x) = Φ2,v(x). (28)

Partial hybrid zero dynamics. Given the partial zero dy-namics (PZD) for each phase {ds,ss}, we can also realizepartial hybrid zero dynamics (PHZD) if the following hybridinvariance conditions are satisfied

∆(ds,ss)(PZds∩Sds|x)⊂ PZss, ∆(ss,ds)(PZss∩Sss|x)⊂ PZds,

where PZv = {x ∈ Dv|x|η2,v(x) = 0}, v ∈ V, is called thepartial zero dynamics surface. We also define switchingfunctions, ∆v (not ∆(ss,ds) or ∆(ds,ss)), for the transformedstatespace (from x to (η ,z)). For example, ∆ds(ηds,zds) :=Φss(∆(ds,ss)(Φ

−1ds (ηds,zds))), which can in turn be split into

two components ∆η2ds , ∆PZ

ds corresponding to the coordinatesη2,v and (y1,v,zv) respectively. With this new notation, we canreduce the hybrid invariance conditions to the following:

∆η2ds (y1,ds,0,zds) = 0, ∆

η2ss (y1,ss,0,zss) = 0. (29)

It was shown in [3] that if the PHZD has an exponentiallystable periodic orbit, then a Lipschitz continuous feedbackcontrol law from the set (21) yields an exponentially stableperiodic orbit in the full order dynamics. We will extend thisresult for inputs of the type (22) along with disturbances1.

1It is important to note that the hybrid invariance conditions shown aboveneed not necessarily be satisfied for the actual system (due to the gapbetween the assumed and the actual model). If ∆v is the assumed impactmodel, the gap can be viewed as: ∆v = ∆v +di, where di is the new impactbased disturbance input. This type of characterization was shown in [7],wherein the uncertainty was a function of the model parameters of thebipedal robot. We will ignore this gap here and assume that the disturbancesare solely due to the control inputs µv.

Periodic orbits and Poincare maps. Substitution of acontrol law kε,ε,v(ηv) ∈ Kε,ε,v(ηv) (from (22)) results inclosed loop dynamics of (25). Denote its flow as ϕt,v. Forthe resulting hybrid dynamics, we have a periodic orbit, if,for some (η∗ds,z

∗ds) ∈ Φds(Sds|x), (η∗ss,z

∗ss) ∈ Φss(Sss|x), and

some T ∗ds,T∗

ss > 0,

(η∗ss,z∗ss) = ϕT ∗ss,ss ◦∆ds(η

∗ds,z

∗ds)

(η∗ds,z∗ds) = ϕT ∗ds,ds ◦∆ss(η

∗ss,z∗ss). (30)

For v = ds, we have the set of points

Ods = {ϕt,ds(∆ss(η∗ss,z∗ss)) ∈Φds(Dds|x)|0≤ t < T ∗ds}. (31)

We can similarly obtain Oss. Hence, we can define theperiodic orbit to be the pair O = {Ods,Oss}, which hasthe period T ∗ = T ∗ds + T ∗ss. Similar formulations follow fordefining a periodic orbit in the PHZD as the pair OPZ ={OPZ

ds ,OPZss }, where the elements are defined via the reduced

order flow ϕPZt,v . Note that, T ∗ds,T

∗ss (similarly, Tϑds ,Tϑss for

PHZD) are called the times to impact (time to reach theguard) for the corresponding flows in the domain. This canbe generalized further to define time to impact functionsfor states starting from the neighborhood of the orbit. Forexample, for (ηss,zss) ∈ B∗ := Br(η

∗ss,z∗ss)∩Φss(Sss|x)

Tds(ηss,zss) = min{t ≥ 0|ϕt,ds ◦∆ss(ηss,zss) ∈ B∗}. (32)

Denote T := Tds+Tss (similarly, Tϑ := Tϑds +Tϑss for PHZD).Given ϕt,ds,ϕt,ss, and Tds,Tss, we can define the Poincare mapfor the initial state (ηss,zss) ∈ B∗ to be

P(ηss,zss) = ϕTss,ss ◦∆ds ◦ϕTds,ds ◦∆ss(ηss,zss). (33)

The Poincare maps are mapped to and from the guard ofthe final domain subscript ss. The Poincare map P can alsobe split into two components Pη2 ,PPZ corresponding to thecoordinates η2,v and (y1,v,zv) respectively.

Stability of periodic orbits. Stability of periodic orbits canbe defined via Poincare maps [9]. Hence, if the Poincare mapis applied i times on the initial condition (ηss,zss), then wehave the final state as Pi(η∗ss,z

∗ss). We say that the periodic

orbit O is exponentially stable if there is ξp ∈ (0,1), Np > 0such that for any initial condition (ηss,zss)∈B∗, the resultingdiscrete system satisfies

|Pi(ηss,zss)− (η∗ss,z∗ss)| ≤ Npξ

ip|(ηss,zss)− (η∗ss,z

∗ss)|.

Stability of OPZ can also be similarly defined. We willdiscuss e-ISS of O next.

V. ISS OF HYBRID PERIODIC ORBITS

The goal of this section is to establish e-ISS of O forinputs of the form: µv(ηv, t)= kε,ε,v(ηv)+dv(t). We will startwith the definition of e-ISS for O (defined via Poincare maps[14]). Without loss of generality, we will drop the domainsubscript notation for the initial states (η ,z), guard S|x,surface Z, and also assume at (η∗ss,z

∗ss) = (0,0). Given that

the disturbance dv is applied in addition to the control lawkε,ε,v, the resulting flows ϕt,v,ϕ

PZt,v , time to impact functions

Tv and the Poincare map P are now dependent on dv.

Definition 6: The periodic orbit O is e-ISSable (exponen-tial input to state stable) if there is ξp ∈ (0,1), Np > 0 andιp ∈K∞ such that for any initial condition (η ,z) ∈ B∗, theresulting discrete time system satisfies

|Pi(η ,z)| ≤ Npξip|(η ,z)|+ ιp(‖d‖V). (34)

e-ISS of OPZ is also similarly defined. Note that the dis-turbance input ‖d‖V is nothing but the maximum of theinput disturbances in each domain: ‖d‖V = maxv∈V ‖dv‖∞.Given Definition 6, we can now state the main theorem thatestablishes e-ISS of O .

Theorem 1: If OPZ is e-ISSable, then there exist suffi-ciently small enough ε, ε > 0 such that for all initial con-ditions (η ,z) ∈ B∗, and for all Lipschitz continuous controllaws kε,ε,v(ηv)∈Kε,ε,v(ηv) (22), the full order periodic orbitO is e-ISSable.

We will provide a sketch of the proof of Theorem 1. Beforeproving the theorem, we will first establish some propertiesof OPZ. Denote ζ := (y1,z). e-ISS of OPZ implies that fordv = 0 there exists r > 0 such that the restricted Poincaremap ϑ : Bζ → Bζ , with Bζ := Br(0,0)∩ΦPZ(PZ ∩ S|x) isexponentially stable i.e., |ζ (i)| ≤ Nξ i|ζ (0)| for some N >0,0 < ξ < 1. Therefore, there exists a Lyapunov functionVϑ , and positive constants b1,b2,b3,b4 such that

b1|ζ |2 ≤Vϑ (ζ )≤ b2|ζ |2Vϑ (ϑ(ζ ))−Vϑ (ζ )≤−b3|ζ |2

|Vϑ (ζ )−Vϑ (ζ′)| ≤ b4|ζ −ζ ′|.(|ζ |+ |ζ ′|).

(35)

We have the following Lemma (required for Theorem 1):

Lemma 3: Let OPZ be e-ISSable. Given the Lipschitz con-tinuous control law kε,ε,v(ηv) ∈Kε,ε,v(ηv) (22) that rendersthe transverse dynamics (20) e-ISSable in the continuousdynamics, then there exist constants A1,A2,D1,D2 > 0, andι ∈K∞ such that for all (η ,z) = (y1,η2,z) ∈ B∗

|T (η ,z)−Tϑ (y1,z)| ≤ A1|η2|+D1‖d‖V (36)|PPZ(η ,z)−ϑ(y1,z)| ≤ A2|η2|+D2‖d‖V. (37)

Proof of Lemma 3 will be omitted due to space constraints,but a similar proof can be found in [7, Lemma 7], whereinthe disturbance input was modeled as a function of parameteruncertainty. We will now prove Theorem 1.

Proof: [Proof of Theorem 1] We start by picking asuitable value of ε , as shown by [3, Theorem 2] that yieldsexponential convergence under a zero disturbance. In order toestablish e-ISS of O , it is sufficient to show that the Poincaremap P is e-ISS [14]. Hence, the goal now is to obtain anISS-Lyapunov function of the form (6) for the Poincare map.

For the Re-ISS-CLF Vε (domain subscript ss is sup-pressed), denote its reduced Lyapunov function (of only η2coordinates) and restriction to the switching surface by Vε,η2 .It can be verified that the matrix Pε can be separated intotwo block matrices, with the latter being the matrix used toobtain the Lyapunov function Vε,η2 . We define the followingcandidate Lyapunov function for some σ > 0:

VP(η ,z) =Vϑ (ζ )+σVε,η2(η2), (38)

defined on B∗. Given the initial state (η ,z), We have thevalue Vε,η2 after the first return on the Poincare section as

Vε,η2(Pη2(η ,z))≤c2,ss

ε2 |η2,ss(Tss)|2, (39)

where c2,ssε2 = λmax(Pε) is the maximum eigenvalue of Pε(=

Pε,ss) (domain ss is suppressed). Further substitution yieldsthe following inequality for some constants A3,A4,D′η > 0:

Vε,η2(Pη2(η ,z))≤ A3|η2|2 +A4|η2|‖d‖V+D′η‖d‖2V.

See [7, eqn. (87)] for a similar derivation. Therefore, we have

Vε,η2(Pη2(η ,z))−Vε,η2(η2)

≤ A3|η2|2 +A4|η2|‖d‖V+D′η‖d‖2V− c1|η2|2,

where c1 = λmin(Pε). By using (37), we have:

|PPZ(η ,z)| = |PPZ(η ,z)−ϑ(ζ )+ϑ(ζ )−ϑ(0)|≤ A2|η2|+D2‖d‖V+Lϑ |ζ |, (40)

where Lϑ is the Lipschitz constant of ϑ(ζ ). From (35)

Vϑ (PPZ(η ,z))−Vϑ (ϑ(ζ ))≤ b4(A2|η2|+D2‖d‖V) (41)(A2|η2|+D2‖d‖V+(Lϑ +Nξ )|ζ |).

Rest of the proof is similar to [7, equations (91) to (96)],where the final bounds on the states (η ,z) are obtained (fora small enough ε) that ensure e-ISS of O .

Remarks on e-ISS of OPZ. In Theorem 1, it was assumedthat the reduced periodic orbit OPZ is e-ISSable. This mayseem like a strong assumption, but, for the dynamics of theform (26), the disturbance input affects the outputs y1,v in anadditive manner. Therefore, it is straightforward to establishe-ISS of OPZ, even if we start with the assumption that OPZ

is exponentially stable under no disturbances.

VI. RESULTS

For verification of the improved stabilizing results pre-sented above, we simulate a bipedal robot (DURUS) undervarious disturbances and observe improvements of the stabil-ity of the gait. The generalized coordinates of the robot aredescribed in Fig. 1 (also see [6]) and the continuous dynam-ics of the bipedal robot is given by (15). The nominal walkinggait considered in this simulation study has two phases:single support, and double support, as shown in Fig. 2. Astable reference walking gait is obtained and verified via anoffline optimization algorithm [6]. Therefore, based on [3,Theorem 2], there is a small enough ε (observed to be ≤ 0.2)that makes the hybrid periodic orbit exponentially stable. Itis important to note that the torque requirements increasewith the decrease in ε .

The main objective of performing a perturbation analysisis to test the stability of the walking gait under uncertaintiesthat are as realistic as possible. Therefore, we set torquelimits of 250Nm for each joint and apply a modeling error of10% to the mass-inertial properties of the robot. Specificallythe modeling error was enforced on the mass, center of massand inertial properties of each link. It was assumed that

MaximumController IO Gain (ε) Allowable Push (N)

0.2 380IO 0.1 420

0.05 395ISS 0.2 380

(ε = 0.1) 0.1 4350.05 410

ISS 0.2 435(ε = 0.01) 0.1 435

0.05 405

TABLE I: Comparison of maximum recoverable push forcesin lateral direction [1]. The ISS based controller can handlegreater pushes. Also reducing ε leads to instability due tothe constraints on model uncertainty and torque limits.

0 1 2 3 4

0

10

20

30

40

50

60

70

0 1 2 3 40 1 2 3 40 1 2 3 40 1 2 3 4

Fig. 3: Comparisons of the Lyapunov function for variousvalues of ε for push recovery. The push force was 350N.The convergence is quicker for decreasing ε . The jumps aredue to discrete events (impacts).

other properties such as links lengths and spring constantsare accurate. The nominal stabilizing controller chosen forsimulation is IO linearization (18) with the auxiliary input

µ(ηv) =

[− 1

εy1,v

− 2ε

y2,v− 1ε2 y2,v

],

and the ISSabilizing controller chosen is (as given by (7))

uISS = uIO−1ε

LgV Tε ,

for the Lyapunov function obtained via IO linearization (18).Two test cases were considered: lateral push force to the

hip for a duration of 0.1s at the beginning of the singlesupport domain, and stepping onto an unknown groundheight. Table I shows the comparison for the push forcerecovery between uIO and uISS for different values of ε, ε . Itcan be observed that with uISS the robot can handle greaterpush forces. With lower ε , the stability of the robot is affected(due to 10% model error and torque saturations) resultingin poorer performance for ε = 0.05. On the other hand,Fig. 3 shows that the convergence improves as ε is lowered.Fig. 4 shows the Lyapunov function comparisons for thepush recovery. Fig. 5 and Fig. 6 show the comparisons forunknown step over different heights. Fig. 7 shows tiles ofpush recovery (top) and stepping over (bottom) for an ISSabi-lizing controller. A video link demonstrating the simulationsperformed on the robot is given in [1].

1 1.5 2 2.5 3 3.5 41 1.5 2 2.5 3 3.5 40

20

40

60

80

100

120

Fig. 4: Push recovery comparison via the Lyapunov functionsfor IO (a) and ISS (b) based controllers. ε = ε = 0.1. Thedeviations are lower for the ISS-CLF based controller.

0 1 2 3 40

20

40

60

80

100

-5

0

5

10

15

20

0 1 2 3 4

Fig. 5: Step over comparison via Lyapunov functions for IO(a) and ISS (b) based controllers. ε = ε = 0.1. The jumps inthe Lyapunov function is lower for ISS-CLF based controller.

0.82 0.84 0.86 0.88 0.9 0.92 0.94 0.96-0.4

-0.2

0

0.2

0.4

0.88 0.9 0.92 0.94-0.3

-0.2

-0.1

0

0.1

0.2

0.3

Fig. 6: Walking over 5cm step height. Phase portraits forvertical z position of the torso base are shown here. (a) showscomparison between IO (blue) and ISS (green) and (b) showsthe responses for different values of ε .

VII. CONCLUSIONS

In this work, it was shown how to obtain a class of inputto state stabilizing controllers for hybrid systems, given theset of stabilizing controllers. It was shown in the specificcase of the bipedal robot DURUS. We obtained the class ofinput to state stabilizing controllers (22) that adds robustnessto the given hybrid periodic orbit O . The simulation resultsdemonstrated that the auxiliary gain ε can be used to restrictthe ultimate bound of the outputs without compromising onthe convergence rate γ

εprovided by the RES-CLF (21). The

methodology shown can be used to realize robust quadraticprograms in real time with the end result being input to statestable walking on DURUS.

REFERENCES

[1] Robustness analysis of DURUS in simulation:.https://youtu.be/g5HlzXRhcSQ.

[2] A. D. Ames. Human-inspired control of bipedal walking robots.Automatic Control, IEEE Transactions on, 59(5):1115–1130, 2014.

[3] A. D. Ames, K. Galloway, K. Sreenath, and J. W. Grizzle. Rapidlyexponentially stabilizing control Lyapunov functions and hybrid zerodynamics. Automatic Control, IEEE Transactions on, 59(4):876–891,4 2014.

Fig. 7: The top tiles show push (350N) recovery and thebottom tiles show stepping onto an unknown disturbance foran ISSabilizing controller. The IO controller failed for ε =0.2 and a height of 5cm (shown in video [1]).

[4] R. Bellman. On the theory of dynamic programming. Proceedings ofthe National Academy of Sciences, 38(8):716–719, 1952.

[5] R. A. DeCarlo, M. S. Branicky, S. Pettersson, and B. Lennartson.Perspectives and results on the stability and stabilizability of hybridsystems. Proceedings of the IEEE, 88(7):1069–1082, 2000.

[6] A. Hereid, E. A. Cousineau, C. M. Hubicki, and A. D. Ames.3d dynamic walking with underactuated humanoid robots: A directcollocation framework for optimizing hybrid zero dynamics. In 2016IEEE International Conference on Robotics and Automation (ICRA),pages 1447–1454, 5 2016.

[7] S. Kolathaya and A. D. Ames. Parameter to state stability of controlLyapunov functions for hybrid system models of robots. NonlinearAnalysis: Hybrid Systems, 25:174 – 191, 2017.

[8] D. Liberzon. Iss and integral-iss disturbance attenuation with boundedcontrols. In Decision and Control, 1999. Proceedings of the 38th IEEEConference on, volume 3, pages 2501–2506 vol.3, 1999.

[9] B. Morris and J. W. Grizzle. A restricted Poincare map for determiningexponentially stable periodic orbits in systems with impulse effects:Application to bipedal robots. In IEEE Conf. on Decision and Control,Seville, Spain, 2005.

[10] R. Shorten, F. Wirth, O. Mason, K. Wulff, and C. King. Stabilitycriteria for switched and hybrid systems. SIAM review, 49(4):545–592, 2007.

[11] E. D. Sontag. Smooth stabilization implies coprime factorization.IEEE Transactions on Automatic Control, 34(4):435–443, 4 1989.

[12] E. D. Sontag. A ’universal’ construction of artstein’s theorem onnonlinear stabilization. Systems & control letters, 13(2):117–123,1989.

[13] E. D. Sontag. Input to State Stability: Basic Concepts and Results,pages 163–220. Springer Berlin Heidelberg, 2008.

[14] S. Veer, Rakesh, and I. Poulakakis. Poincare analysis of hybridperiodic orbits of systems with impulse effects under external inputs.arXiv preprint arXiv:1712.03291, 2017.

[15] Y. Wang and Z. Zuo. On quadratic stabilizability of linear switchedsystems with polytopic uncertainties. In 2005 IEEE InternationalConference on Systems, Man and Cybernetics, volume 2, pages 1640–1644 Vol. 2, 10 2005.

[16] E. R. Westervelt, J. W. Grizzle, C. Chevallereau, J. H. Choi, andB. Morris. Feedback Control of Dynamic Bipedal Robot Locomotion.Automation and Control Engineering. CRC Press, 2007.

[17] M. A. Wicks, P. Peleties, and R. A. DeCarlo. Construction of piecewiselyapunov functions for stabilizing switched systems. In Decisionand Control, 1994., Proceedings of the 33rd IEEE Conference on,volume 4, pages 3492–3497 vol.4, 12 1994.