Discrete Event Models + Temporal Logic = Supervisory Controller:Automatic Synthesis of Locomotion ControllersMarco Antoniotti� Bud MishraRobotics LaboratoryCourant Institute of Mathematical SciencesNew York University719 BroadwayNew York, NY, 10003, U.S.A.marcoxa@cs.nyu.edu mishra@nyu.eduAbstractIn this paper, we address the problem of the syn-thesis of controller programs for a variety of roboticsand manufacturing tasks. The problem we choose fortest and illustrative purposes is the standard \Walk-ing Machine Problem," a representative instance of areal hybrid problem with both logical/discrete and con-tinuous properties and strong mutual in uence with-out any reasonable separation. We aim to produce a\compiler technology" for this class of problems in amanner analogous to the development of the so-called\Silicon Compilers" for the VLSI technology. To copewith the di�culties inherent to the problem, we resortto a novel approach that combines many key ideas froma variety of disciplines: namely, Discrete Event Su-pervisory Systems [14] Petri Nets approaches [8], [10]and Temporal Logic [5].1 Introduction and Our GoalsThis paper describes a \Controller Synthesis Sys-tem" and its application in the development of awalking machine. Our (admittedly ambitious) goal isto build a comprehensive controller synthesis systembased on the Ramadge and Wonham's DES theory[14] and a form of temporal logic [5] widely used inthe �eld of veri�cation.Our synthesizer accepts a model of the legs (bothcontinuous and discrete) and a set of goals (express-ible in temporal logic) and automatically synthesizes acontroller that controls the legs. The controlled walk-�This research has been partially supported by the NationalScience Foundation under grant number CCR-9202900.

ing machine exhibits behaviors that are guaranteednot to violate any of the desired goals. This class ofbehaviors of the legs are called \gaits" and are ex-pected to depend on the leg model and desired goals.We also graphically simulate the gaits to gather in-sights about the formulation and hope to provide thedesigner feedback on how to make design changes.However, the problem is not as straightforward as itmay seem at the �rst glance. We need to address theproblem resulting from inadequate formulations of theclose interplay between the discrete and the \underly-ing" continuous levels. In the walking machine case,the reciprocal in uences between these levels mustbe taken into account in order to produce a reason-able integrated controller. The examples appearing inthe DES literature (mostly related to manufacturing)seem to be tractable otherwise. We believe that, inthe case of walking machine, the di�culties in spec-ifying the desired behavior arise from the fact thatthe system is inherently tightly coupled . In contrast,most examples seen in the DES literature appear tobe loosely coupled .Other works in DES theory [1] and [17] posemany interesting problems, especially with respect tothe inherent complexity of the manageability of thesystems1. For some related ideas, we refer the readersto [2] and [7].This paper is organized as follows. In the next sec-tion, we describe the model of the walking machine weare using. We discuss its DES components at lengthand its kinematics/dynamics brie y. Finally, we de-scribe how our work �ts in with other ongoing researchat our Laboratory. The next two sections are devoted1The synthesis algorithms used run against a state space ex-plosion, which is mostly unavoidable.

{0}

X

Y

X

Y

X

Y

X

X Y

Y

0

0

θ

θ

θ

1 1

1

{1}{2}

2

2

2

3

3

3

{3}

4

{4}

4

Tip

Ground

Base

Figure 1: Leg with frame assignments. Note thatfBg = f0g and fTg = f4g, following standard ter-minology.to an explanation of the interplay between the tem-poral logic used, a Model Checker for it and how itsimpli�es the use of DES theory for the walking ma-chine system. We conclude the paper by pointing outsome problems we encountered and worthy of furtherexploration.2 Walking Machine ModelThe Walking Machine Model we are building is afour legged system based on Microcontroller and Di-rect Drive Technology. We model the system as com-prising of a discrete and of a continuous layer. Oursystem follows standard modeling techniques proposedin [6], [12] and [15].2.1 Leg ModelWe use a standard three link leg model which weanalyzed as a planar manipulator. Figure 2.1 showsthe geometry of the leg and the assignment of the stan-dard coordinate frames. The Denavit-Hartenberg pa-rameters for the Leg are shown in table 1.At this point, it is fairly straightforward to derivethe kinematic and dynamic equations for the leg inorder to build position and force controllers for thejoints. Yet, our main interest remains to be in explor-

i = link# �i ai di �i1 0 0 0 �12 0 l1 0 �23 0 l2 0 �34 0 l3 0 0Table 1: The Denavit-Hartenberg Link Parameters as-signment.Start

Drive

Slipped

Unload

Recover

Load

eu

el

er

es ed

slip

eslFigure 2: Model of the discrete transitions of a singleleg with an uncontrollable event (slip).ing the interactions between the \low level" continu-ous control and the \discrete" synchronization schemeof the whole walking machine.2.2 Discrete Event ModelMost studies on walking machines use a Finite StateMachine (FSM) approach to the problem of \highlevel" synchronization ([6], [12]). Using DES Theorywe are able to take into account possible undesirablebehaviors. The �nite state model we use for a singleleg is depicted in �g. 2. The six states Start, Unload,Recover, Load, Drive and Slipping correspond to di�er-ent \movements" of the leg. E.g. in the Recover statethe leg is moving \forward" without contacts with theground; in the Slipping state, the leg has just lost thestance on the ground and is conceivably not support-ing the hip anymore. In the spirit of DES theory, theevents es, eu, er, el, ed and esl2 are all controllable, slipis instead uncontrollable.The walking machine is modeled by slightly dif-ferent equations in each state (with the possible ex-ception of the Slipping state, for which we assume nomodel). The role of the \high level" Discrete Con-troller is to choose the appropriate set of control lawsfor each discrete state. Such Discrete Controller issynthesized using the Supervisor Synthesis schemes ofDES3 [13].We build the actual FSM for the DES \plant" lan-guage L (in DES terminology) by taking the shu�e2The pre�x e- is intended to mean \end of".3We assume familiarity with the standard DES terminology.Refer otherwise to [14] for a survey of the original DES Theory.

Figure 3: Prototypes of the mini actuator links for thelegs of the Walking Machine built by Richard Wallaceand Fred hansen.product of the four distinct FSM's, one for each leg.This yields a FSM with 1296 states and 5184 transi-tions. How this model will be used to synthesize theactual DES Supervisor will be deferred to the nextsections.2.3 System Software and SimulationOur controller compiler was rapidly prototyped inCommon Lisp. Such a choice had many advantagesover a more traditional one, given the exibility of theLisp environment. Moreover, it does not hinder theactual production (through a \post processing" pro-cess) of low level Assembly, C or Ada modules for someof the architectures currently used in our laboratory(Motorola MC68332tm boards and VxWorkstm).Under the supervision of R. Wallace, our roboticslaboratory has developed an inexpensive yet powerfultechnology of mini actuators [16]. In collaborationwith this group, we have been designing direct drivewalking machines and constructing these out of mini-actuators. Fig. 3 shows prototype leg joints.3 Controller SynthesisWe modi�ed the standard DES Theory approach tothe Supervisor Synthesis Problem by using a Temporal

Logic formalism for the speci�cation and veri�cationof properties of the desired language K.3.1 CT L Speci�cations and SupervisorySynthesisThe Temporal Logics we use are the standardBranching Time Logics of the CT L family [5]. Weare not the �rst to explore the use of Temporal Logicin the DES realm (see [9]), but our approach (and theLogic used) is di�erent.CT L is a Logic whose semantics is de�ned in termsof aKripke Structure which can easily be reinterpretedin terms of FSM's.The well formed formulas of CT L are listed in ta-ble 2 along with their semantics in terms of the un-derlying FSM. This logic has been extensively usedin the �eld of Veri�cation of VLSI circuits [3, 11]. Itsusefulness comes from the existence of a linear timeModel Checking algorithm that works by recursivelylabeling the underlying �nite state machine. Whencoupled with hierarchical and State Space Compres-sion techniques, such algorithm becomes usable in awide range of cases [4].We specify the desired discrete behavior K by mark-ing o� \undesirable states". We do so by using a modi-�ed version of the Model Checking Algorithm. A CT Lformula that speci�es this for a train of legs (left orright) is the followingAG( :state([Drive1;Drive2]) ^:state([Recover1;Recover2]) ^:state([Slipping1; Slipping2])):The meaning of this formula is simply the statementof a state avoidance problem. We do not want thesystem to be in a state where both legs of a train areboth recovering, or driving, or (worse) slipping.Another property that we would like to enforceis what physiologists and zoologists call rear-to-frontwaves in animal gaits [6]. This is a constraint on thesequencing of events in the leg system. Specifying thissequencing constraint for a train of legs is rather easyusing CT L.AG:�state([Drive1;Recover2])) :EX:(state([Unload1;Recover2]))�:The meaning of this formula is that whenever the rearleg (number 2) is recovering , the front leg (number 1)cannot start unloading.The Supervisory Synthesis theory does guaranteethat a Supervisor exists and that it is \minimally re-strictive" [13]. Yet no guarantee is given that the su-pervisor will maintain all the properties that we may

Syntax Semantics DescriptionBase Formul�p p 2 A(s) A propositionf1 _ f2 f1 2 A(s) or f2 2 A(s) A disjunctionf1 ^ f2 f1 2 A(s) and f2 2 A(s) A conjunction:f f 62 A(s) A negationf1 ) f2 f1 62 A(s) or f2 2 A(s) An implicationTemporal Formul�EX(f) f 2 A(s0) and s0 is a successor state of s f will be true in some next stateAX(f) f 2 A(s0) for every s0 successor of s f will be true in all the next statesE[f1U f2] If s0; s1; : : : ; sn is a sequence of states and at each of them f1 2A(si) for i < n and f2 2 A(sn) There is a sequence of states where f1holds until f2 will.A[f1U f2] For any sequence of states s0; s1; : : : ; sn at each of them f1 2A(si) for i < n and f2 2 A(sn) There is a sequence of states where f1holds until f2 will.EF(f) There is a sequence of states where f will eventually hold (thisis actually an abbreviation for E[TrueU f ]) This formula represents a potentialevent.AF(f) For any sequence of states f will eventually hold (this is actuallyan abbreviation for A[TrueU f ]) This formula represents a necessaryevent.EG(f) There is a sequence of states, f will always hold (this is actuallyan abbreviation for :AF(:f)) The formula f will always hold on somepath.AG(f) For all sequences of states, f will always hold (this is actually anabbreviation for :EF(:f)) This formula states a global and invari-ant property of the system.Table 2: Syntax and informal Semantics for CT L. Note that A is an assignment of propositions and formul� toeach state. A proposition p (or, recursively, a formula f) is True, or holds in a state s when p 2 A(s).specify. We use the Modi�ed Model Checker again todebug the Synthesized Supervisor. This was actuallyour original motivation for the use of the CT L.As an example, our �rst attempts at the Supervi-sory Synthesis for a train of legs, kept removing thestates [drive1; recover2] and [recover1; drive2]. We wereable to discover this fact only by means of graphicalsimulation. The Model Checker turned out to be anexcellent tool for the debugging, signi�cantly reducingthe turnaround time. Moreover we were able to provefancier properties for the controlled system. As an ex-ample, we could check some liveness conditions suchas AG(state([drive1; ?])) AF(state([?; drive2])));and the fact that the supervised system werestill able to reach the states [drive1; recover2] and[recover1; drive2].3.2 Continuous Control ConstraintsThe \desired behavior" of the Walking Machinesystem is obviously not completely speci�ed by theconstraints we posed on the discrete level. The transi-tions between states are ruled by measurements takenfrom sensors. We used only position information in or-der to allow the transition from one state to the otherof the discrete control. This is su�cient to get nicesimulations and already poses interesting problems forthe control synthesis procedure.The geometric model that we use for our Walk-ing Machine is depicted in �g. 3.2. By following the

{B}

Leg2Leg1

-

p

p

p

p

r

f r

f

Figure 4: Simpli�ed Geometric Model of the WalkingMachine. fBg is a coordinate frame set in the body.All measurements are taken with respect to it.standard methodology, we obtain a discrete supervisorthat allows forEF:state([load1; load2])to be true for one train of legs4. In this state, the su-pervisor has to choose which transition to make nextto either state [load1; drive2] or [drive1; load2]. Sinceboth transitions are controllable and not forbidden bythe supervisor, the system might end up \taking a steplonger than the leg" by cycling one too many times4With respect to the \start" state. Actually we can proveEG(state([Load1; Load2])) EX(EF(state([Load1;Load2])))):

through the [load1; load2] state 5. In Petri Net termi-nology, this is called a con ict and it really representsa situation where \extra information" is needed (orassumed) in the system.We solved this problem by studying some algo-rithms that will allow us to identify these \con ictstates" in order to reduce the actual behavior of thesystem6 to a \geometrically acceptable" one. In thistask we are doing something similar to [10].When we consider a train of two legs, the transitioner1 for the front leg (leg 1) in state [Recover1;Drive2]causes the di�erence in the position of the feet�(pfeet) =j pf � pr j to change in the following way�(pfeet)[Load1;Drive2] =�(pfeet)[Recover1;Drive2] + 12step;if we assume the rear leg moved a \very small" dis-tance,�(pfeet)[Load1;Drive2] =�(pfeet)[Recover1;Drive2] + 2 stepif we assume the both legs moved (almost) the fullstep distance.We can repeat this reasoning for all the other states.This \interval" computation for the transitions canbe reconstructed from the description of the state inwhich it is taking e�ect, hence we can set up a simplegraph traversal which will mark the states where agiven constraint could (but not necessarily would) beviolated. In our case the simple constraint we wouldlike to maintain is�(pfeet) � `;where ` is derived from the mechanics of the WalkingMachine.The graph traversal simply maintains for each nodetraversed a possible maximum and minimum value for�(pfeet) while following only the controllable transi-tions enabled by the Supervisor. Whenever there aretwo or more such transitions outgoing a state s andone of the reachable states (or the state itself) pos-sibly violates the constraint, then s is marked as a\choice point". Eventually, we will be able to equipthe runtime of the system with appropriate tests thatwill avoid the controllable transitions that in speci�coccasions (usually after a few tours around a cycle inthe state space) would violate the constraint.5This argument applies also when we consider two legs inalternation { left and right { and virtual legs.6Note that we will be giving up some of the properties of thelanguage found by the approximation algorithm. I.e. we will beimposing further restrictions on the supremal language.

3.3 ExampleIn order to give a avor of the current usage of oursystem, we give some excerpts of a session where weconsider the behavior of one train of legs (i.e. a front(1) and a rear (2) leg).The state machine representing the behavior of oneleg is represented as follows7:(define-state-machine leg2:states (s2 r2 l2 d2 u2 sl2):start s2:alphabet (es2 er2 el2 ed2 eu2 esl2 slip2):uncontrollable (slip2):delta ((s2 es2 u2) (r2 er2 l2)(l2 el2 d2) (d2 ed2 u2)(u2 eu2 r2)(d2 slip2 u2) (sl2 esl2 u2)):final-states (s2 r2 l2 d2 u2 sl2))In order to specify the machine representing the inter-leaving of the discrete events we write(define-state-machine legs :op (shuffle leg1 leg2))which states that legs is the shuffle of the two ma-chines at hand (leg1 and leg2). To remove the un-desirable states we run the Model Checker, which, asa side e�ect, marks the states that do not satisfy theformula.CMUCL 4> (model-check legs'(AG (and (not (state (d1 d2)))(not (state (r1 r2)))(not (state (sl1 sl2))))))NILThe NIL result tells us that the unregulated shu�edoes not satisfy the property.The resulting language K is not controllable, hencewe need to build an approximation for it. In this casethe approximation algorithm terminates after two it-erations. The results are as follows:CMUCL 7> (omega-op K legs uncontrollable-events);; Debugging deleted...>> OMEGA(0): removable states = ((D1 SL2) (SL1 D2))---------------;; Debugging deleted...>> OMEGA(1): removable states = NIL#<Representation for the approximation to K>CMUCL 8>4 Conclusions and Open ProblemsWe have presented an application of DES theoryto a standard problem in robotics: the Walking Ma-chine. Our goal was to build an easy-to-use \supervi-sor compiler" system for a wide range of robotics and7The notation and the tricks used are standard CommonLisp. leg2 represents the state machine for the rear leg; s2 rep-resents the relative start state and so on. define-state-machineis a simple macro that extends the language.

1α

α2

α3

β1

β2

β2

β1

1s

2s

3s

4s

β1 p1

AG ( )Φ p1

*

β1 p1

AG ( )

p2Φ

*Φ p1

*AG ( )p

2β2

p2Φ *

AG ( )p2

β2Figure 5: A case where a na��ve CT L Supervisor Syn-thesis approach fails. In state s2 we have that AG(p1)is true for language ��1 , while in state s4 we haveAG(p2) true for ��2 .manufacturing systems. There are still many openproblems which we expect to face before actually pro-ducing a viable software environment capable of aidingthe practitioner in the production of code for PLC'sor microcontroller programs.One open problem that we are investigating con-cerns the direct synthesis of the supervisory map froma set of CT L speci�cations; i.e. without specifying Kas a language. This brought up some interesting ques-tions about the satis�ability of CT L formul� underthe control action of the supervisor: see �g. 4 for anexample. Suppose we want a supervisor that achievesa language satisfying the formulaAX(AG(p1)) _AX(AG(p2))under the assignment A(s2) = p1 and A(s4) = p2.Then the maximal controllable sublanguage K is notunique and hence not well-de�ned. The problem canbe traced to non-monotonicity of CT L modal oper-ators. We are investigating a solution based on arestriction on the logic called CT L� [11], which cir-cumvents this problem. The resulting algorithm alsoreduces the complexity of the synthesis by one orderof magnitude.Acknowledgments We thank Mohsen Jafari of Rutgers,Fred Hansen and Richard Wallace of NYU Robotics Lab fortheir help and suggestions.References[1] S. Balemi, G. J. Ho�mann, P. Gyugyi, H. Wong-Toi, andG. F. Franklin. Supervisory Control of a Rapid ThermalMultiprocessor. IEEE Transactions on Automatic Control,38(7):1040{1059, jul 1993.[2] A. Benveniste, M. Le Borgne, and P. Le Guernic. Hy-brid Systems: the SIGNAL approach. In R. L. Grossman,A. Nerode, A. P. Ravn, and H. Rischel, editors, Hybrid Sys-tems, volume 736 of Lecture Notes in Computer Science,pages 230{254. Springer-Verlag, 1993.

[3] M. Browne, E. M. Clarke, D. Dill, and B. Mishra. Au-tomatic veri�cation of sequential circuits using temporallogic. IEEE Transactions on Computers, c-35(12):1035{1044, 1986.[4] J. R. Burch, E. M. Clarke, K. L. McMillan, D. L. Dill, andL. J. Hwang. Symbolic Model Checking: 1020 States andBeyond. 5th LICS, pages 428{439, 1990.[5] E. M. Clarke, E. A. Emerson, and A. P. Sistla. Auto-matic Veri�cation of Finite-State Concurrent Systems Us-ing Temporal Logic Speci�cations. ACM Transactions onProgramming Languages and Systems, 8(2):244{263, 1986.[6] M. D. Donner. Real-Time Control of Walking, volume 7of Progress in Computer Science. Birkh�auser, 1986.[7] R. L. Grossman, A. Nerode, A. P. Ravn, and H. Rischel,editors. Hybrid Systems, volume 736 of Lecture Notes inComputer Science. Springer-Verlag, 1993.[8] L. E. Holloway and B. H. Krogh. Synthesis of FeedbackControl Logic for a Class of Controlled Petri Nets. IEEETransactions on Automatic Control, 35(5):514{523, may1990.[9] J.-Y. Lin and D. Ionescu. Analysiss and Synthesis Pro-cedured of Discrete Event Systems in a Temporal LogicFramework. In International Symposium on IntelligentControl. IEEE, 1992.[10] B. J. McCarragher and H. Asada. A Discrete Event Ap-proach to the Control of Robotic Assembly Tasks. InIEEE International Conference on Robotics and Automa-tion, pages 331{336. IEEE, 1993.[11] B. Mishra and E. M. Clarke. Hierarchical Veri�cation ofAsynchronous Circuits Using Temporal Logic. TheoreticalComputer Science, 38:269{291, 1985.[12] M. H. Raibert. Legged Robots That Balance. MIT Press,1986.[13] P. J. Ramadge and W. M. Wonham. On the SupremalControllable Sublanguage of a Given Language. SIAM J.Control and Optimization, 25(3):637{659, may 1987.[14] P. J. G. Ramadge and W. M. Wonham. The Control of Dis-crete Events Systems. Proceedings of the IEEE, 77(1):81{98, 1989.[15] S. Song and K. J. Waldron. Machines that Walk: TheAdaptive Suspension Vehicle. MIT Press, 1989.[16] R. S. Wallace. Miniature Direct Drive Rotary Actuators.Robotics and Autonomous Systems, 11:129{133, 1993.[17] R. A. Williams, B. Benhabib, and K. C. Smith. A HybridSupervisory Control System for Flexible ManufacturingWorkcells. In IEEE International Conference on Roboticsand Automation, pages 2551{2556. IEEE, 1994.