Lecture XIII: MLSC - Dr. Sethu Vijayakumar 1
Lecture XIIIDynamical Systems as Movement Policies
Contents:• Differential Equation• Force Fields, Velocity Fields• Dynamical systems for Trajectory Plans
• Generating plans dynamically • Fitting (or modifying) plans• Imitation based learning
Thanks to my collaborator Auke Ijspeert (EPFL) for many of the contents on the slides for this lecture.
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 2
Movement policies as Dynamical Systems
• Represent complex movements in globally stableattractor landscapes of nonlinear autonomous differential equations
• Choose kinematic representation for easy re-use in different workspace location
• Ensure easy temporal and spatial scaling (topological equivalence)
• Use local learning to modify the attractors according to demonstration of teacher and self-learning
Discreet & Rhythmic Movement Primitives
( , , )des desf goalτ =x x x&
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 3
Discreet & Rhythmic Movement superposition
Open loop with oscillators Closed loop control in horizontal plane
Open (vertical) + Closed (horizontal) loop control
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 4
What is a Differential Equation?
• Differential equation: an equation that describes how state variables evolve over time, for instance:
)( ycy −= α&
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 5
Some Definitions
• Ordinary differential equation: differential equation thatinvolves only ordinary derivatives (as opposed to partial derivatives)
• Autonomous equation: differential equation that does not(explicitely) depend on time
• Linear differential equation: differential equation in which thestate variables only appear in linear combinations
• Nonlinear differential equation: differential equation in whichsome state variables appear in nonlinear combinations (e.g. products, cosine,…)
• Fixed point: point at which all derivatives are zero (can be an attractor, a repeller, or a saddle point, cf later)
• Limit cycle: periodic isolated closed trajectory (can only occur in nonlinear systems)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 6
Interesting Regimes of Differential Equations
From Strogatz 1994
Limit cycles ChaosAttractors
Attractor
saddles
Unstable node
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 7
First Order Linear Systems
• First order linear system:
• How to solve this equation, for a given y(t=0), c, and α ?
• Two methods: analytical solution or numerical integration
• Analytical solution:
• Numerical integration: Euler method, Runge-Kutta,…
)( ycy −= α&
ctcyty +−−= )exp()()( 0 α
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 8
First Order Linear Systems
)( ycy −= α&
ctcyty +−−= )exp()()( 0 α
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 9
Second Order Linear Systems
yxyxcy
=−−=
&
& ))((βα
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 10
Second Order Linear Systems
yxyxcy
=−−=
&
& ))((βα
88
==
βα
18
==
βα
28
==
βα
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 11
Second Order Non-linear System
xyxyxy
coscos2coscos2
−−=−−=
&
&
Attractor
saddles
Repeller
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 12
Learning a movement by demonstration
Movement recognition
Movement execution
(Inv. Dyn.)
Trajectory formation
system
Joint angles
Task of the trajectory formation system: • To encode demonstrated trajectories with high accuracy, • To be able to modulate the learned trajectory when:
• Perceptual variables are varied (e.g. timing, amplitude)• Perturbations occur
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 13
Encoding a trajectory
Traditionally, the problem of replaying a trajectory has been decomposedinto two different issues:• One of encoding the trajectory, and• One of modifying the trajectory, for instance, in case the movement is
perturbed, or when it requires to be modulated.
Our approach: combine both abilities in a nonlinear dynamical system
Aim: to encode the trajectory in a nonlinear dynamical system with welldefined attractor landscape
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 14
Nonlinear Dynamical Systems Approach
y
dy/dt
Single point attractor
Discrete movements
y
dy/dt
Limit cycle attractor
Rhythmic movements
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 15
Two types of movement recordings
« Kinesthetic » demonstrationSensuit
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 16
Pointing Demonstration
position velocity
Demo forOne DOF
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 17
Shaping Attractor Landscapes
Goal: g
( )( )( )( )?
z zz g y zy f z
α β= − −
= +
&
&
( )( )z zz g y zy z
α β= − −=
&
&
Can one create more complex dynamics by non-linearly modifying the dynamic system:
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 18
Discreet Control Policy
Goal: g
Amplitude and phase system
Output System
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 19
Shaping attractor landscapes
( )( )z zz g y zy z
α β= − −=
&
&
y
dy/d
tg = goal Can one create more
complex dynamics by non-linearly modifying the above dynamic system:
( )( )( )( )?
z zz g y zy f z
α β= − −
= +
&
&
g = goal
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 20
Shaping Attractor Landscapes
A globally stable learnable nonlinear point attractor:
( )( )( )( )
( )( )
( )
( )
1
1
2 0
0
,where
,
1exp and 2
z z
y
v v
xk
i ii
k
ii
i i i
z g y zy f x v z
v g x vx v
w b vf x v
w
x xw d x c xg x
α βα
α βα
=
=
= − −
= +
= − −=
=
−⎛ ⎞= − − =⎜ ⎟ −⎝ ⎠
∑
∑
&
&
&
&
Trajectory PlanDynamics
Canonical Dynamics
Local LinearModel Approx.
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 21
Learning the Attractor
Given a demonstrated trajectory y(t)demo and a goal g
Extract movement durationAdjust time constants of canonical dynamics to movement durationUse LWL to learn supervised problem
( )target ,demo
y
yy z f x vα
= − =&
&
[ Stefan Schaal, Sethu Vijayakumar et al, Proc. of Intl. Symp. Rob. Res.(ISRR) (2001) ] Also extended to rhythmic primitives :
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 22
Trajectory following & Generalization
Backhand Demonstration Backhand Reproduction
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 23
Modulation of Goal: Anchor
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 24
Drumming: Modulating Frequency
Drumming: Kinesthetic Demo
Imitation and Modulation