Bilateral Teleoperation of Multiple Cooperative Robots over

Delayed Communication Network: Theory

Dongjun LeeMark W. Spong

d-lee@control.csl.uiuc.edu, mspong@uiuc.edu

Research partially supported by the Office of Naval Research (N00014-02-1-0011 and N00014-05-1-0186), the National Science Foundation (IIS 02-33314 and CCR 02-09202), and the College of

Engineering at the University of Illinois.

Outline

1. Motivations

2. Problem Formulation

3. Passive Decomposition of Slave Robots

4. Control Design

5. Conclusions

Part II: Simulation and Semi-Experiment

MotivationsApplications:

1. Space Structure Construction/Maintenance

- Hubble telescopes, International Space Station,…

2. Remote Construction/Maintenance of Civil Structures

- Bridge, Highway, Tall buildings,…3. Operations in Hazardous Environments

- Nuclear plants, Deep water, …

Bilateral Teleoperation

- Human’s intelligent intervention

in uncertain environmentsMulti-Robot Cooperation

- Mechanical strength and dexterity

- Robustness and safety

Bilateral Teleoperation of Multiple Cooperative

Robots

Challenges and Requirements

1. Abstraction - human is able to operate only small DOF

simultaneously2. Secure grasping - no dropping of the grasped object

3. Haptic feedback - crucial for manipulation tasks

4. Interaction safety and stability - stably coupled with humans, objects, and

environments

Outline

1. Motivations

2. Problem Formulation

3. Passive Decomposition of Slave Robots

4. Control Design

5. Conclusions

Dynamics of Master and Multiple Slave Robots

Dynamics of

a single master

(m-DOF) Dynamics of multiple

slave robots(n1+n2+…+nN-DOF)

n-DOF product system(n=n1+n2+…+nN-dimensional)

Stack-up

inertia Coriolis control humanforce

velocity

Grasping Shape Function: Holonomic Constraints

Grasping shape control objective desired (constant)

grasping shape

1 2 41 2 3

2 3

( , , )E

q qq q q q

q q

q1

q2

q3m-dim.

level sets

- m-dim. holonomic constraints on the config. space of slave robots (m < n)

- assumed to address the internal formation shape for cooperative grasping

- smooth and full-rank Jacobian (i.e. smooth submersion) - overall group motion evolving on m-dim. level sets

(submanifold)

master’s DOF

Communication and Control (C&C) Structure

- C&C delay between the master and the slaves

- Centralized C&C module for multiple slaves - negligible delays among the slaves - workspaces of slaves are close to each other (e.g.

cooperative grasping)

Semi-Autonomous Teleoperation Architecture

Observation: - secure grasping is of foremost importance for safety - the system cannot be completely free from time-delay, i.e. system performance would be compromised in

some aspects

Semi-autonomous teleoperation: 1. local grasping control - secure grasping immune to communication-delay - autonomous control would be enough due to

simplicity of cooperative grasping control objective

2. delayed bilateral teleoperation - communication-delay effect confined in bilateral

teleoperation - sluggish response could be taken care of by

intelligent humans - delayed teleoperation is relatively well-studied areas

Energetic Passivity for Safe/Stable Interaction

- passive with total master/slave mechanical power as supply rate

- stable interaction with any E-passive humans[Hogan]/objects/environments

Energetic

passivity

total slave-ports mechanical power

master-portmechanical power

Outline

1. Motivations

2. Problem Formulation

3. Passive Decomposition of Slave Robots

4. Control Design

5. Conclusions

Passive Decomposition of Multiple Slaves Robots

The Passive Decomposition [Lee&Li, CDC03] decouples the locked and shape systems from each other while enforcing passivity- Can achieve tight/secure grasping regardless of overall

group behavior - Ensure secure grasping and interaction stability simultaneously

internal group coordination (cooperative grasping)

Shape System

behavior of overall group(and grasped object)

Locked System

Coupling:dropping object!!!

Orthogonal Decomposition w.r.t. Inertia Metric

Locked system velocity vL : parallel w.r.t. the level sets of qE: (behavior of grasped object and

total group)

Shape system velocity vE : orthogonal complement w.r.t.

inertia matrix (cooperative grasping)

locked systemvelocity vL

shape systemvelocity vE

Grasping shape function

Tangent space decomposition

basis of kernel of

qE

basis of orthogonal

space

Passive Decomposition of Slave Group Dynamics

- Shape system ((n-m)-DOF) explicitly represents cooperative grasping shape qE(q)

- Locked (m-DOF) system describes overall group behavior

- Locked and shape dynamics are similar to usual mechanical systems:

- ML(q), ME(q) : symmetric and positive-definite

- ML(q)-2CL(q,q), ME(q)-2CE(q,q) : skew-symmetric

- Coupling is energetically conservative: Passive Decoupling - CLE(q,q) =-CEL

T(q,q) -> vLTCLE(q,q)qE +

qETCEL

T(q,q)vL=0

- Power and kinetic energy are also decomposed

Original Slave Dynamics

PassiveDecomposition

DecomposedDynamics

Energetic Structure of Decomposed Dynamics

- We can decouple the shape system (cooperative grasping) and the locked system (overall group) from each other while enforcing passivity

- Desired cooperative grasping and overall group behavior can be achieved

simultaneously while enforcing interaction stability

Original System Decomposed System

passivedecoupling

Outline

1. Motivations

2. Problem Formulation

3. Passive Decomposition of Slave Robots

4. Control Design

5. Conclusions

Semi-Autonomous ControlDecompos

edDynamics

Scattering-based teleoperation

control for decoupled locked system

Local grasping control

control for decoupled shape system

Passive decoupling

Total Slave

Control

- Adjusting qEd, and PD-gains, fixtureless grasping can be achieved

for flexible object- Although dynamics is decoupled, other effects (e.g. inertia of

object) can still perturb the shape system via the internal force FE: feedforward

cancellation is necessary

Grasping Dynamics (Decoupled Shape

System)internal force

PD/FF-based Control

estimate ofinternal force

desired grasping shape

Local Grasping Control

Scattering-Based Teleoperation of Locked System

control human/combined external forces

Dynamics of Master Robot

and Slave Locked System(both are m-

DOF)

Shape system(locally

controlled)

Locked System

(decoupled)

By operating the master robot of manageably small DOF, human operators

can tele-control the behavior of the grasped object over the delayed

master-slave communication channel while perceiving combined external

forces acting on the grasped object and slaves

Symmetric Scattering-Based Teleoperation: - scattering communication (to passify comm. delays) and

impedance (PI) controls - asymptotic position-error convergence proof with Z=Kv (i.e. matching condition [Stramigioli et al, TRA03]) : so far, only boundedness of position-error has been

established. - force reflection in static manipulation (negligible

acceleration/velocity)

Impedance Control

(PI-Control)

line impedance(user-specific)

Scattering Variables (Power

Decomposition)

reflected (from comm.)

incident (to comm.)

Scattering-Based Symmetric Teleoperation

Conclusions

We propose a control framework for bilateral teleoperation of multiple cooperative robots over delayed master-slave comm. channel:

- passive decomposition: the decoupled shape (cooperative grasping)

and locked (behavior of the grasped object) systems - local grasping control for the shape system: high

precision cooperative grasping regardless of human

command/comm. delays - scattering-based bilateral teleoperation of the locked

system: human can tele-control behavior of the cooperatively

grasped object by operating a small-DOF of the master robot,

while perceiving combined force on the slaves and the

grasped object over the delayed communication channel - enforce energetic passivity: interaction safety and

stability are enhanced

Part II will present simulation and semi-experiment results.