Lecture 13: RLSC - Prof. Sethu Vijayakumar 1
Lecture 13: Dynamical Systems based Representation
Contents: • Differential Equation
• Force Fields, Velocity Fields
• Dynamical systems for Trajectory Plans
• Generating plans dynamically
• Fitting (or modifying) plans
• Imitation based learning
Thanks to my collaborator Auke Ijspeert (EPFL) for many of the contents on the slides for this lecture.
Lecture 13: RLSC - Prof. Sethu Vijayakumar 2
Movement policies as Dynamical Systems
• Represent complex movements in globally stable attractor landscapes of nonlinear autonomous differential equations
• Choose kinematic representation for easy re-use in different workspace location
• Ensure easy temporal and spatial scaling (topological equivalence)
• Use local learning to modify the attractors according to demonstration of teacher and self-learning
Discreet & Rhythmic Movement Primitives
),,( goalf desdes
Lecture 13: RLSC - Prof. Sethu Vijayakumar 3
Discreet & Rhythmic Movement superposition
Open loop with oscillators Closed loop control in horizontal plane
Open (vertical) + Closed (horizontal) loop control
Lecture 13: RLSC - Prof. Sethu Vijayakumar 4
• Differential equation: an equation that describes how state variables evolve over time, for instance:
)( ycy
What is a Differential Equation?
Lecture 13: RLSC - Prof. Sethu Vijayakumar 5
• Ordinary differential equation: differential equation that involves only ordinary derivatives (as opposed to partial derivatives)
• Autonomous equation: differential equation that does not
(explicitly) depend on time
• Linear differential equation: differential equation in which the state variables only appear in linear combinations
• Nonlinear differential equation: differential equation in which
some state variables appear in nonlinear combinations (e.g. products, cosine,…)
• Fixed point: point at which all derivatives are zero (can be an attractor, a repeller, or a saddle point, cf later)
• Limit cycle: periodic isolated closed trajectory (can only occur in nonlinear systems)
Some Definitions
Lecture 13: RLSC - Prof. Sethu Vijayakumar 6
Limit cycles Chaos Attractors
Attractor
saddles
Unstable node
From Strogatz 1994
Interesting Regimes of Differential Eqs.
Lecture 13: RLSC - Prof. Sethu Vijayakumar 7
• First order linear system:
• How to solve this equation, for a given y(t=0), c, and ?
• Two methods: analytical solution or numerical integration
• Analytical solution:
• Numerical integration: Euler method, Runge-Kutta,…
)( ycy
ctcyty )exp()()( 0
First Order Linear Systems
Lecture 13: RLSC - Prof. Sethu Vijayakumar 11
yxx
xyy
coscos2
coscos2
Attractor
saddles
Repeller
Second Order Non-linear System
Lecture 13: RLSC - Prof. Sethu Vijayakumar 12
Task of the trajectory formation system: • To encode demonstrated trajectories with high accuracy, • To be able to modulate the learned trajectory when:
• Perceptual variables are varied (e.g. timing, amplitude) • Perturbations occur
Movement recognition
Movement execution (Inv. Dyn.)
Trajectory formation
system
Joint angles
Learning a movement by demonstration
Lecture 13: RLSC - Prof. Sethu Vijayakumar 13
Traditionally, the problem of replaying a trajectory has been decomposed into two different issues: • One of encoding the trajectory, and • One of modifying the trajectory, for instance, in case the
movement is perturbed, or when it requires to be modulated.
Our approach: combine both abilities in a nonlinear
dynamical system Aim: to encode the trajectory in a nonlinear dynamical system
with well defined attractor landscape
Encoding a trajectory
Lecture 13: RLSC - Prof. Sethu Vijayakumar 14
y
dy/dt
Single point attractor
Discrete movements
y
dy/dt
Limit cycle attractor
Rhythmic movements
Nonlinear Dynamical Systems Approach
Lecture 13: RLSC - Prof. Sethu Vijayakumar 15
Sensuit
Two types of movement recordings
« Kinesthetic » demonstration
Lecture 13: RLSC - Prof. Sethu Vijayakumar 16
position velocity
Demo for
One DOF
Pointing Demonstration
Lecture 13: RLSC - Prof. Sethu Vijayakumar 17
Shaping Attractor Landscapes
Goal: g
?
z zz g y z
y f z
z zz g y z
y z
Can one create more
complex dynamics by non-
linearly modifying the
dynamic system:
Lecture 13: RLSC - Prof. Sethu Vijayakumar 18
Goal: g
Amplitude and phase system
Discreet Control Policy
Output System
Lecture 13: RLSC - Prof. Sethu Vijayakumar 19
Shaping Attractor Landscapes
1
1
2 0
0
,
where
,
1exp and
2
z z
y
v v
x
k
i i
i
k
i
i
i i i
z g y z
y f x v z
v g x v
x v
w b v
f x v
w
x xw d x c x
g x
A globally stable learnable nonlinear point attractor:
Local Linear
Model Approx.
Canonical Dynamics
Trajectory Plan Dynamics
Lecture 13: RLSC - Prof. Sethu Vijayakumar 20
Learning the Attractor
Given a demonstrated trajectory y(t)demo and a goal g
Extract movement duration
Adjust time constants of canonical dynamics to movement duration
Use LWL to learn supervised problem
target ,demo
y
yy z f x v
[ Stefan Schaal, Sethu Vijayakumar et al, Proc. of Intl. Symp. Rob. Res.(ISRR) (2001) ]
Also extended to rhythmic primitives :
Lecture 13: RLSC - Prof. Sethu Vijayakumar 21
Trajectory following & Generalization
Backhand Demonstration Backhand Reproduction
Lecture 13: RLSC - Prof. Sethu Vijayakumar 22
Basic oscillator:
Output signal:
1
)( 0
rrr
N
i i
N
i
T
ii
m
Iwzy
zyyz
1
1
))((
],,[ vx
TvxI ],[
Rhythmic Control Policies
Lecture 13: RLSC - Prof. Sethu Vijayakumar 24
Drumming: Modulating Frequency
Drumming: Kinesthetic Demo
Imitation and Modulation
Biped Locomotion using DMPs
Approach
5-link biped robot
- phase resetting and frequency adaptation - phase regulation between coupled oscillators
[Nakanishi, Morimoto, Endo, Cheng, Schaal, and Kawato, 2004]
•Movement primitives as a central pattern generator (CPG) • Synchronization between CPG and robot
Height 40cm Mass hip 2.0kg knee 0.64kg foot 0.06kg
Lecture 13: RLSC - Prof. Sethu Vijayakumar 25
Control Architecture
Rhythmic Movement primitives
Phase resetting
periodic pattern generation learning from demonstrated trajectories
reset the phase of the oscillator at heel contact
Dynamical primitives CPG
oscil lator
output with local models
Robot and Environment
(index: i=1~# of oscillators)
phase resetting frequency
update
[Nakanishi, Morimoto, Endo, Cheng, Schaal, and Kawato, 2004]
Frequency adaptation
Phase regulation among oscillators coupling among oscillators
Lecture 13: RLSC - Prof. Sethu Vijayakumar 26