Control with external sensors - Intranet DEIBhome.deib.polimi.it/rocco/cir/Control with...

Control of industrial robots

Control with external sensors

Prof. Paolo Rocco ([email protected])Politecnico di MilanoDipartimento di Elettronica, Informazione e Bioinge gneria

Introduction� So far we have assumed that the robot is controlled in a free environment

� However the robot has to interact with the environment either to manipulate it, or to avoid to collide with it, or to interact with other devices or humans

� In order to effectively interact with the environment, the robot must be endowed with external (exteroceptive) sensors, which make the robot controller aware of the situation

� Common sensors used to measure such interaction are force sensors and vision sensors

� In the following we will study with some detail the control systems based on force measurements. We will also give a short outline on vision based control systems

Control of industrial robots – Control with external sensors − P. Rocco [2]

Interaction with the environmentA robot commonly interacts with the working environment, manipulating objects and performing operations on surfaces (think of grinding, deburring, part assembly) .

In modern robotics, situations are common where the robot physically interacts with the operator (think of telemanipulation or rehabilitation).


Passive control of compliance

� The RCC is used in assembly tasks (peg-in-a-hole)� It is typically placed between the robot’s wrist and the gripper� The RCC lets the gripper assembly move in the plane perpendicular to the peg’s

axis. This allows the peg to rotate� With the RCC, the forces generated by any misalignment can be compensated in

a passive way

RCC: Remote Center of Compliance

A first way to address the interaction with the environment is to endow the manipulator with devices that facilitate the execution of the task in a passive way:


Active control of the interaction

During more general interaction tasks, the use of a purely positional control strategy (the same adopted in free motion) may lead to problems due to positioning errors and uncertainties in task planning, related to a incomplete knowledge of the environment.

Examples where an active control of the interaction is desirable include:� mechanical machining (deburring, surface

finishing, polishing, assembly,...)� tele-manipulation� cooperation of multi-manipulator systems� dexterous robot hands� physical human-robot interaction

For an active control of interaction it is necessary to use devices for force (and moment) measurement .

Experiments in force/position control with a rigid surface at POLIMI.


Force sensors

Force/torque sensors are devices that return the measurements of three components of force and three components of moment with respect to a local frame.


Internal behaviour

Sensors are based on strain gauges, i.e. devices that can measure a strain. The strain induces a variation in the resistance of a circuit.

Strain gauges are suitably mounted, in such a way as to return information from which three components of force and moment can be computed.


� Impedance/admittance control� The goal is to assign a prescribed dynamic relation between interaction forces and position

errors� We want that the manipulator, in case of unplanned interactions, behaves like a generalized

mass-spring-damper system

� Hybrid position and force control� We separate directions which are only position controlled from directions which are only

force/torque controlled� In the directions constrained by the environment we want that a force/torque with a specified

value be established

TaxonomyForce based interaction control strategies can be divided into two categories:

Furthermore we will talk about:

� Explicit control� The force (or the impedance) is directly assigned, acting on the control variables (joint

torques)

� Implicit control� The force (or the impedance) is indirectly assigned, acting on the set-points of the position

control loops


Mechanical impedance� The flow of power among two physical systems can always be defined as the

product of two conjugate quantities: a (generalized) effort and a (generalized) flow

� In the electrical framework, the flow is the current, while the effort is the voltage. In the mechanical framework, instead, the flow is the (linear or angular) velocity, the effort is the force (or the moment)

� The mechanical impedance is then defined as dynamical relation that is established between force and velocity (or displacement) for a mechanical system. The admittance is just the reciprocal of the impedance

� The impedance control aims at making the manipulator, position controlled and in interaction with the environment, assume a desired mechanical impedance, like a generalized mass-spring-damper system


Mechanical impedanceAn example of impedance/admittance control for physical human-robot interaction:


A 1 d.o.f. systemIn order to understand how an impedance controller operates, let us first consider the simple example of a body with mass M interacting with the environment, which exerts a force f:

The system can be modeled with the block diagram:

The model of the interaction with the environment is not specified

x+

f

u

M1

s1

s1x x

...

M

x

u f


A 1 d.o.f. systemLet us consider an inverse dynamics motion control law:

Myu =

with:

xKxKxy PDd~~ ++= &&&

The closed loop dynamics are obtained:

fxKxDxM dd =++ ~~~ &&&

where:

xxx d −=~

where:

PdDd MKKMKD == ,

The equation establishes a mechanical impedance relation between the force acting on the system and the positional error. However the inertial component of the impedance (M) cannot be arbitrarily assigned.


A 1 d.o.f. systemSuppose now that the force f is measured and let:

This corresponds to the block diagram:

fMyu +=

with:

( )fxKxDxMM

y ddddd

−++= ~~1 &&&

x+

f

u

s

1x x...

+

Kd−

Dd

+

++ +

− y

s

1

M

1

Md

1M

+

+xd

xd

xd

.

..

The closed loop dynamics are:

fxKxDxM ddd =++ ~~~ &&& It is an impedance relation where all the terms can be arbitrarily assigned.


Implicit impedance control (admittance control)

When a positional control is already applied to the mechanical system, it might be convenient to close the impedance control outside the position control. Consider again the case of single d.o.f. system, position controlled:

The system can be modeled with the block diagram:

+

f

uG(s)

xR(s)

+xd

( )2

1

MssG =

M

x

u fR

xd


Consider the following block diagram, where the force applied by the environment is measured and filtered through a system with transfer function Gd(s), to give a variation of the reference xd:

Elaborating the block diagram:

( )( ) ( ) ( )( ) ( ) ( )

( ) ( )sGsRsGsR

fsGxfsGsR

sGx dd +

−++

=11

+

f

uG(s)

xR(s)

++xd

Gd(s)impedance controller


Also called admittance controller(it accepts a force and delivers a displacement)


In a band of frequencies well inside the bandwidth of the position control system, we can assume the force disturbance well rejected and the position reference correctly followed. Thus:

( )fsGxx dd −≈ ( ) xsGf d~1−=

If we impose:

( )ddd

dKsDsM

sG++

=2

1

we assign to the manipulator the desired impedance behaviour.

The advantage of this scheme is that the position control is kept unchanged: the impedance control modifies the set-points of the position control, which is usually the only openness allowed by an industrial motion control system.

fxKxDxM ddd =++ ~~~ &&&



Statics of the manipulatorBefore applying the concepts of impedance control to a complete manipulator, we need to include the effect of a force applied at the end-effector in the model.

Suppose then that a force is applied at the end-effector.What are the joint torques that keep the system in equilibrium?

?

? ?

�


Systems of forces

As a matter of fact a system of forces mayact at the end-effector.

How can this system be reduced? From classical mechanics…

� One force applied in a point (the resultant)� One torque

It follows that we will always be able to represent a system of forces with two vectors:

� A force vector applied in a point O (the resultant)� A moment vector (the moment of the original system of forces with respect to

the fulcrum O)


Statics of the manipulatorLet us then consider a manipulator subject to a set of forces at the end effector, having a resultant f and moment, w.r.t. the origin of the end effector frame, µ. We then define vector:

qδ=δ τTW τ

The virtual work of the forces/torques at the joints is:

( ) ( ) ( ) qqJhqqJqqJfpf δ=δ+δ=+δ=δ TO

TP

TTTh dtW µωµ

The virtual work of the forces at the end-effector is:

Let us call τ the vector of torques (or forces) applied to the joints.

=

µf

h

where δq is the elementary displacement of the joint coordinates.

where δp is the virtual displacement of the origin of the end-effector frame, consistent with the differential kinematics of the robot, and ω is the angular velocity of the end effector frame.


Statics of the manipulator

qδ=δ τTW τ

Based on the principle of virtual work the manipulator is in static equilibrium if and only if:

( ) ( ) ( ) ,qqJhqqJqqJf δ=δ+δ=δ TO

TP

ThW µ

hWW δ=δ τ ( )hqJT=τ

( ) ( )( )

=

qJ

qJqJ

O

P

position Jacobian

orientation Jacobian

Notice that the same matrix, the Jacobian matrix, which defines the differential kinematics of the manipulator, is also involved in the statics of the system (kineto-static duality).


Inverse dynamics controlThe dynamic model in interaction with the environment is then given by the equation:

( ) ( ) ( ) ( )hqJqgqqqCqqB T−=++ τ&&&& ,

where h is the vector of the contact forces (and moments) applied by the end effector on the environment.

Let us consider now the adoption of an inverse dynamics control law:

( ) ( ) ( )qgqqqCyqB ++= &&,τ

Substituting we obtain:

( ) ( )hqJqByq T1−−=&&

nonlinear coupling term


Inverse dynamics controlAssume now for y the expression introduced in the inverse dynamic control in the operational space:

By substituting the expression for y we obtain the equation of the dynamics in closed loop:

where x = xd − x, while:

( ) ( ) ( ) ( )qJqBqJqB 1−−= AT

AA

( ) ( )( )qqqJxKxKxqJy &&&&&& ,~~1APDdA −++= −

( ) AADP hqBxKxKx 1~~~ −=++ &&&

is the inertia matrix in the operational space and:

( ) ,hxTh TAA = ( ) AA JxTJ =

~

We have obtained an impedance relation which is coupled and only partially assignable: a force applied along a Cartesian direction yields motion in other directions as well.

ATA

T hJhJ =

generalized forces performing work on x

.


Impedance control with force measurement

If force measurements are available, we can set:

( ) ( ) ( ) ( )hqJqgqqqCyqB T+++= &&,τ

and:

( ) ( )( )AAddddddA hqqqJMxKxDxMMqJy −−++= −− &&&&&& ,~~11

We obtain:

Addd hxKxDxM =++ ~~~ &&&

i.e. a completely decoupled system.

where Md, Dd, Kd are diagonal, positive definite matrices.


Impedance controlThe equation:

Addd hxKxDxM =++ ~~~ &&&

defines a mechanical impedance relation between the forces/moments defined at the right hand side of the equations and the position/orientation errors in the operational space.This impedance has the form of a mechanical system with mass matrix Md, damping Dd, and stiffness Kd.

The dynamic behaviour along each direction of the operational space can thus be specified: a force or a moment along a certain Cartesian direction yields a motion in such direction, which is related to the force and moment by a fully controllable dynamics.

But what do we exactly mean for impedance in the operational space?


Translational impedanceThe first three degrees of freedom in the operational space are the position coordinate of the end-effector.In this case the definition of the impedance is rather intuitive:

fpKpDpM =∆+∆+∆ ppp&&&

pd: desired position

pe: actual position

ed ppp −=∆

The impedance relation is:

We thus duplicate on each degree of freedom what we have already seen for the single mass.


Rotational impedanceThe remaining three degrees of freedom in the operational space consist in a minimal representation (Euler angles) of the end-effector orientation.In this case the definition of the impedance is less intuitive:

( )µφφφφ eΤTKDM =∆+∆+∆ φφφ

&&&

φd: desired orientation

φe: actual orientation

ed φφφ −=∆

The impedance relation is:

Due to the presence of matrix T, this relation is subject to representation singularities. Better definition of the rotational impedance have been proposed in the literature.


Natural constraints and artificial constraints

An alternative way to perform an interaction task is to assign reference values of the forces and of the positions, consistent with the geometry of the environment.Some directions can be in fact subjected to constraints on the position, other ones to constraints on the forces that can be applied.In general, it is not possible to impose both the force, and the position, along the same direction.

We make the following distinction:

Natural constraints: they are imposed by the environment along each degree of freedom of the task; they depend on the geometry of the task.

Artificial constraints: constraints imposed by the control system, related to the task execution strategy.

For task description we refer to a constraint frame xc, yc, zc.


Sliding on a flat surface

cz

cz

cy

cy

cx

cx

cy

cy

cx

cx

cz

cz

pf

pf

fp

ωµ

µω

µω

&

&

&

sConstraint

Artificial

sConstraint

Natural

This picture is taken from the textbook:

B. Siciliano, L. Sciavicco, L. Villani, G. Oriolo: Robotics: Modelling, Planning and Control, 3rd Ed. Springer, 2009


Insertion of a peg in a hole

cz

cz

cz

cz

cy

cy

cx

cx

cy

cy

cx

cx

pf

fp

fp

ωµ

µω

µω

&

&

&

sConstraint

Artificial

sConstraint

Natural

Usually we set pzc ≠ 0 and all the

other artificial constraints to zero.

.




Rotation of a crank

cz

cz

cy

cy

cy

cy

cx

cx

cz

cz

cx

cx

pf

fp

fp

ωµ

µω

µω

&

&

&

sConstraint

Artificial

sConstraint

Natural

Normally artificial constraints on forces and moments are assumed to be zero.

Idle knob

Oriented like the crank




Selection matrix

We assume that the constraints can always be expressed along mutually orthogonal directions of frame C. Thus on each direction either a natural constraint on position, or a natural constraint on force, exists.

In order to specify the constraints we use a selection matrix Σ diagonal and with null or unitary elements:

Natural constraints

( ) cn

ccn

c hhIvv =−= ΣΣ

Artificial constraints

( ) ca

cca

c hhvvI ==− ΣΣ

The selection matrix extracts directions where a natural constraint in velocity applies and thus an artificial constraint in force can be enforced

=

01

11

00

000000000000000000000000000000

Σ


Hybrid force/position control

Hybrid force/position control is a control strategy that uses the artificial constraints to specify the goals of the control system, in terms of both force and position.

The 6 directions of the operational space are divided into position/velocity controlled directions and force/moment controlled directions:

MOTION

CONTROL

+

INTERACTION

CONTROL

hAdc

+Rc NONLINEAR

COMPENSATIONS

MANIPULATOR

AND ENVIRONMENT

RcΣ

RcI−Σ

Σ

I−Σvd

c

yFc

yvc

yc y u hA

v

hAc

vc


Sources of inconsistency

Hybrid position/force control is based on a nominal model of the interaction. Inconsistency may however occur in the measurements, due e.g. to:

1. friction at the contact (a force is detected in a nominally free direction)

2. compliance in the robot structure and/or at the contact (a displacement is detected in a direction which is nominally constrained in motion)

3. uncertainty in the environment geometry at the contact

The first two sources of inconsistency are automatically filtered out through the selection matrices. The third source of inconsistency can be mitigated by real time estimation process.


A 1 d.o.f. systemIn order to have a better idea on how the control system operates along the force controlled directions, let’s consider the simple example of a body of mass M in interaction with the environment:

Block diagram:

M

x

uf

+fu

G(s)x

K− fu

Gf(s)

( )KMs

KsGf +

=2

We assume a purely elastic model of the environment.


A 1 d.o.f. system

The system presents a unitary gain and resonant, high frequency (for a stiff contact) dynamics. Notice that f = u for K → ∞.

The best way to control this system is through an integral controller:

fuGf(s)

uGf(s)

fRf(s)

+fd ( )s

KsR if

f =

This way the bandwidth of the force control is directly assigned with the integrator, leaving outside the bandwidth the high frequency dynamics of the interaction with the environment.

( )KMs

KsGf +

=2


Implicit force controlAs with impedance control, when a position control is already active, it might be convenient to close the force control around the position control loop.Consider for simplicity only the case of a, position controlled, single degree of freedom:

M

x

ufR

xd

The system can be modelled through the following block diagram:

+fu

G(s)x

R(s)+xd K


If a PID position controller is used:

Under the assumption of an infinitely stiff contact (K → ∞) we obtain x=0 and f ≅ u:

fuR(s)

xd

≅1

( )s

KsKsKsR IPD ++

=2

we can adopt for the force control the scheme:

fuR(s)

xd

≅1Rf(s)+fd C(s)

with:

( ) ( )s

KKsR

KsKsKsC if

pffIPD

+=++

= ,1

2

The advantage is that the position control is left unchanged.

Implicit force control


The position control loop opens!

Industrial application of force control

ABB RobotWare Machining Force Control (machining of blades):


Visual measurementsArtificial vision devices are useful sensors for robotics because they mimic the human sense of sight and allow to gather information from the environment without contact.Nowadays several robotic controllers integrate vision systems.

Alternatively, visual measurements can be used in a real time feedback loop in order to improve position control of the end effector: this is the concept of visual servoing.

The typical use of vision in industrial robotics is to detect an object in the robot’s scene, whose position (and orientation) is then used for online path planning in order to drive the robot to the identified object.

Online re-planning of the path can also be performed when the vision system detects some unexpected change in the path the robot is supposed to follow (for example a corner in a contouring task).


An example of path re -planning Following a corner during a contouring task:

View from an external camera View from the onboard camera


The hardware

camera

video dataframe grabber

industrial PC

zoom lens


The camera

The camera is a device that can measure the intensity of light, concentrated by a lens on a plane, the image plane, which contains a matrix of pixels (CCD: Charge Coupled Device). Pixels then transform the light energy in electrical energy.

The camera performs a 2D projection of the scene. This projection entails a loss of depth information: each point in the image plane corresponds to a ray in the 3D space.

In order to determine the 3D coordinates of a point corresponding to a 2D point in the plane additional information is needed:

� multiple views with a single camera� multiple cameras� knowledge of characteristic relations between relevant points of the framed

objects


Perspective projection

A point with coordinates (X, Y, Z) in the camera frame is projected into the following point in the image plane, expressed in pixels:

λα−=

=

Y

X

Zy

xξ

λ: focal lengthα: scale factor in pixel/m

Other methods exist to represent the projection on the 2D plane (scaled orthographic projection, affine projection)


Image features

In artificial vision we denote with image feature any characteristics that can be extracted from an image (e.g. an edge or a corner).

We then define a parameter of an image feature a quantity, expressed by a real numeric value, which can be computed from one or more image features.Parameters of an image feature can be gathered in a vector: ξ = [ξ1, ξ2, … ξk].

Examples of parameters of image features:

� point coordinates� length and orientation of a line connecting two points� centroids and higher order moments� parameters of an ellipse


Calibration

� Internal calibration:

� Determination of the intrinsic parameters of the camera (like the focal length λ) as well as of some additional distortion parameters due to lens imperfections and misalignments in the optical system

� External calibration:

� Determination of the extrinsic parameters of the camera like the position and the orientation of the camera with respect to a reference frame

The camera has to be calibrated before usage in a robotic vision system:


Camera configuration

The first decision to be made when setting up a vision-based control system is where to place the camera.The camera can be:� mounted in a fixed location in the workspace (eye-to-hand configuration) so

that it can observe the manipulator and any objects to be manipulated� attached to the robot above the wrist (eye-in-hand configuration)

Eye-In-HandEye-To-Hand


Eye-to-hand configuration

Eye-To-Hand

� Advantages� the field of view does not change as

the manipulator moves� the geometric relationship between

the camera and the workspace is fixed and can be calibrated offline

� Disadvantages� as the manipulator moves through

the workspace it can occlude the camera’s field of view


Eye-in-hand configuration

Eye-In-Hand

� Advantages� the camera can observe the motion

of the end effector at a fixed resolution and without occlusion as the manipulator moves through the workspace

� Disadvantages� the geometric relationship between

the camera and the workspace changes as the manipulator moves

� the field of view can change drmatically for even small motion of the manipulator


Control architectures: classification

Vision control systems can be classified based on various criteria. A first classification is based on the following question:

Is the control structure hierarchical, with the vision system providing set-points as input to robot’s joint-level controller, or does the visual controller directly compute the joint-level inputs?

In the first case: dynamic look and moveIn the second case: direct visual servoing

Advantages of the dynamic look and move approach:� the reduced sampling rate of the visual signal does not compromise the

overall performance of the position control system� in several industrial robot controllers it is only allowed to operate at the

position setpoints level� the robot can be seen as an ideal positioner in the Cartesian space, thus

simplifying the design of the vision control system


A second classification is based on the following question:

Is the error signal defined in 3D (task space) coordinates or directly in terms of image features?

In the first case : position based controlIn the second case : image based control

Position based control:� vision data are used to build a partial 3D representation of the world� pose estimation algorithms are computationally intensive (a real-time

implementation is required) and sensitive to errors in camera calibration

Image based control:� uses the image data directly to control the robot motion� an error function is defined in terms of quantities that can be directly measured

in an image, and a control law is constructed that maps this error directly to robot motion

Control architectures: classification


Typical control architectures

Cartesian

control lawJoint

controllers

Actuators

Camera

Image feature

extraction

Pose

estimationVideo

dc x

xc

+−

Control law in

the image

feature space

Joint

controllers

Actuators

Camera

Image feature

extractionVideo

+−

Dynamic position-based look-and-move

Dynamic image-based look-and-move


Cartesian

control law

Actuators

Camera

Image feature

extraction

Pose

estimationVideo

dc x

xc

+−

Control law in

the image

feature spaceActuators

Camera

Image feature

extractionVideo

+−

Position-based visual servoing

Image-based visual servoing

Typical control architectures


Date post:	14-Feb-2020
Category:	Documents
Upload:	others
View:	8 times
Download:	0 times

Control with external sensors - Intranet DEIBhome.deib.polimi.it/rocco/cir/Control with...

Documents