8/8/2019 Mae 345 Lecture 23

1/12

Task Planning andMulti-Agent Systems

Robert StengelRobotics and Intelligent Systems, MAE 345,

Princeton University, 2009

Copyright 2009 by Robert Stengel. All rights reserved. For educational use only.

http://www.princeton.edu/~stengel/MAE345.html

Decision making

Task decomposition, communities, and connectivity

Cooperation, collaboration, competition, and conflict

Multi-agent architectures

Control Functions ofan Intelligent Agent

Conscious Thought

- Awareness- Focus

- Reflection

- Rehearsal- DeclarativeProcessing of Knowledge or Beliefs

Unconscious Thought

- Subconscious Thought

> ProceduralProcessing> Communication

> Learned Skills

> Subliminal Knowledge Acquisition- Preconscious Thought

> Pre-attentive DeclarativeProcessing

> Subject Selection for Conscious Thought> Concept Development

> Information Pathway to Memory

> Intuition

Intelligent AgentCharacterized byDeclarative, Procedural,and Reflexive Actions

ReflexiveBehavior- Instantaneous Response to Stimuli

- Elementary, Forceful Actions

- Stabilizing Influence

- Simple Goals

Task Planning Goals Accomplish an objective

Make a decision

Gather information

Build something

Analyze something

Destroy something

Determine and follow a path

Minimize time or cost

Take the shortest path

Avoid obstacles or hazards

Work toward a common goal

Integrate behavior with higherobjectives

Do not impede other agents

Central Pacific and Union Pacific Railroads

meet in Promontory, Utah, 1869

8/8/2019 Mae 345 Lecture 23

2/12

More Task Planning Goals

Provide leadership forother agents

Issue commands Receive and decode

information

Provide assistance toother agents

Coordinate actions

Respond to requests

Defeat opposing agents Compete and win

Common Threads inTask Accomplishment

Optimize a cost function

Satisfy or avoid constraints

Exhibit desirable behavior Tradeoff individual and team goals

Use resources effectively and efficiently

Negotiate

Cooperate with team members

Overcome adversity and ambiguity

TaskPlanning

Situation awareness

Decomposition and identification of communities

Development of strategy and tactics

PhaseProcess Outcome

Objective Tactical(short-term)

SituationAssessment

SituationAwareness

Strategic(long-term)

Comprehension Understanding

Boyd!s OODA Loop

for Combat Operations

Derived from air-combatmaneuvering strategy

General application to learning

processes other than military

8/8/2019 Mae 345 Lecture 23

3/12

Endsley, 1995

Elements ofSituation Awareness

Perception

Comprehension

Projection

Important Dichotomiesin Planning

Strength, Weakness, Opportunity, and

Threat (SWOT) Analysis

Knok-Knoks and Unk-Unks

Simultaneous Locationand Mapping (SLAM) Build or update a local map

within an unknown environment Stochastic map, defined by mean

and covariance

SLAM Algorithm = State estimationwith extended Kalman filter

Landmark and terrain tracking

Durrant- Whyte et al

Multi-Agent Control Example Based on

Linear-Quadratic-Gaussian

(LQG) Optimal Control

E(J) = E

! x(tf )"# $% + L x(t),u(t)[ ]to

tf

& dt

=1

2x

T(tf )S fx(tf )+ x

T(t)Qx(t) + u

T(t)Ru(t)"# $%

to

tf

& dt'()

*)

+,)

-)

'

(

))

*

)))

+

,

))

-

)))

Quadratic cost function

Linear dynamic model

!x(t) = Fx(t) +Gu(t) + Lw(t)

8/8/2019 Mae 345 Lecture 23

4/12

Stochastic Optimal Control Law

u(t) = !R!1G

TS(t)x(t) = !C(t) x(t)

S(t) = !FTS(t)!S(t)F + S(t)GR

!1G

TS(t)!Q, S(tf) = Sf

Solution of Euler-Lagrange equationsleads to optimal feedback control law

Where S(t)is the solution to a matrixRiccati equation

(Estimator not shown)

A Federated Optimization Problem

x(t) = Fx(t)+Gu(t) =FA

FB

A

FAB

FB

"#

%&xA

xB

!

"#

$

%&+

GA

GB

A

GAB

GB

"#

%&uA

uB

!

"#

$

%&

Dynamic models for two agents, A and B, are coupled to

each other and expressed as a single system

E(J) = E1

2xT(t)Qx(t)+ u

T(t)Ru(t)!" #$

to

tf

% dt&'(

)(

*+(

,(

= E1

2xAT

xBT!

"#$

QA QBA

QAB

QB

!

"

--

#

$

.

.

xA

xB

!

"--

#

$..+ uA

TuB

T!"

#$

RA RBA

RAB

RB

!

"

--

#

$

.

.

uA

uB

!

"--

#

$..

!

"

--

#

$

.

.to

tf

% dt&

'(

)(

*

+(

,(

u(t) = !Cx(t) =uA

uB

"

#

$$

%

&

''= !

CA

CB

A

CA

BC

B

"

#

$$

%

&

''

xA

xB

"

#

$$

%

&

''

Cost function minimizes performance-control tradeoff

Optimal feedback control laws are coupled to each other

A Distributed Optimization Problem

x(t)= Fx

(t)+Gu

(t)=

FA

0

0 FB

"# %&

xA

xB

"# %&+

GA

0

0 GB

"# %&

uA

uB

"# %&

Each sub-system can be optimized separately

Each control depends only on separate sub-state

E(J) = E1

2xT(t)Qx(t) + uT(t)Ru(t)!" #$

to

tf

% dt&'(

)(

*+(

,(

= E1

2xAT

xBT!

"#$

QA

0

0 QB

!

"--

#

$..

xA

xB

!

"--

#

$..+ uA

T uBT!

"#$

RA

0

0 RB

!

"--

#

$..

uA

uB

!

"--

#

$..

!

"

--

#

$

.

.to

tf

% dt&'(

)(

*+(

,(

u(t) = !R

A0

0 RB

"

#

$$

%

&

''

!1

GTSx(t) = !Cx(t) =

uA

uB

"

#

$$

%

&

''= !

CA

0

0 CB

"

#

$$

%

&

''

xA

xB

"

#

$$

%

&

''

Coupling between actions of two agents, A and B, is negligible

Pursuit-Evasion:A Competitive

Optimization Problem

Linear model with two competitors, Pand E

x(t) = Fx(t)+Gu(t) =x

P

xE

!

"#

$

%&=

FP

0

0 FE

!

"#

$

%&x

P

xE

!

"#

$

%&+

GP

0

0 GE

!

"#

$

%&uP

uE

!

"#

$

%&

Pursuer!s goal: minimize final miss distance

Evader!s goal: maximize final miss distance

Example of a differential game, Isaacs (1965), Bryson & Ho (1969)

8/8/2019 Mae 345 Lecture 23

5/12

Pursuit-Evasion:A Competitive

Optimization Problem

Quadratic minimax (saddle-point) cost function

Optimal control laws for pursuer and evader

E(J) = E1

2xT(t

f)S(t

f)x(t

f)!" #$ +

1

2xT(t)Qx(t) + u

T(t)Ru(t)!" #$

to

tf

% dt&'()(

*+(

,(

= E

1

2xPT(t

f) xE

T(tf)!

"-#$.

SP SPE

SEP SE

!

"

--

#

$

.

.f

xP (tf )

xE(tf)

!

"

--

#

$

.

.

+1

2x

P

T(t) x

E

T(t)!

"#$

QP QPE

QEP QE

!

"--

#

$..

xP (t)

xE(t)

!

"--

#

$..+ u

P

T(t) u

E

T(t)!

"#$

RP 0

0 /RE

!

"--

#

$..

uP(t)

uE(t)

!

"--

#

$..

!

"

--

#

$

.

.to

tf

% dt

&

'

(((

)

(((

*

+

(((

,

(((

u(t) =u

P(t)

uE(t)

!

"

##

$

%

&&= '

CP(t) C

PE(t)

CEP

(t) CE(t)

!

"

##

$

%

&&

xP(t)

xE(t)

!

"

##

$

%

&&

Strategy/Tactics Developmentand Deployment

Development of long- and short-termactions/activities for implementation and

operation Sequence of procedures to be executed

fixed or adaptive

Exposition of approach Rules of engagement

Concept of Operations (CONOPS)

Spectrum of flexibility Rigid sequence Learning systems

Think Expert System

Program Management:Gantt Chart

Project schedule

Task breakdown and dependency Start, interim, and finish elements

Time elapsed, time to go

Program Evaluation and ReviewTechnique (PERT) Chart

Milestones

Path descriptors

Activities, precursors, and successors

Timing and coordination

Identification of critical path

Optimization and constraint

8/8/2019 Mae 345 Lecture 23

6/12

Path Planning

Trajectory decompositionand segmentation

Environment idealizationand nominal path

Waypoints

Path primitives (line, circle, etc.)

Timing and coordination

Obstacle detection and avoidance

Feasibility and regulation

Optimization and constraint

Task Decomposition: Path Planning

Tessellation (tiling) of a2-D decision space

Given set of points, e.g.,obstacles, destinations,or centroids of multiplepoints

2-D Voronoi diagram Polygons with sides

equidistant to two nearestpoints (black dots)

Distance= Euclidean norm other metrics can be used

transformations

Voronoi diagram

Dual Graph is theDelaunay Triangulation

Edges (black) connect all triplets of points lying on

circumferences of empty circles, i.e., containing noother points

Voronoi segment boundaries (red) are perpendicularto each edge

Minimum spanning treeis a subset of theDelaunay graph

Paths with farthestdistances from obstacles

Voronoi Diagrams inPath Planning

Obstacle avoidance

Nearest neighbor identification, e.g., closesthospital

8/8/2019 Mae 345 Lecture 23

7/12

Voronoi Diagrams inData Processing

Computer graphics textures (2-D and 3-D meshes)

Voronoi diagram with

fuzzy boundaries

http://www.data-compression.com/vqanim.shtml

Vector quantization in data compression

Density characterization (3-D mesh)

2004 DARPAGrand

ChallengeRoute

Specification

Waypoint # Latitude Longitude

Lateral

Boundary

Offset

Speed Limit

(mph)

Drop-Dead

Time,

hours minutes seconds1 34.7607233 -117.0107599 90 10 #### #### ####2 34.7607515 -117.0107413 15 10 #### #### ####3 34.761033 -117.01056 12 10 #### #### ####4 34.7611515 -117.0105663 12 10 #### #### ####5 34.7612679 -117.0106085 10 10 #### #### ####

732 34.9313317 -116.7157473 13 15 13 30 0

2582 35.6168352 -115.3820928 10 5 #### #### ####2583 35.6164206 - 115.382102 10 5 #### #### ####2584 35.6162642 -115.3821041 10 5 #### #### ####2585 35.6162367 -115.3821032 6 2 #### #### ####2586 35.6162012 -115.3821023 12 0 #### #### ####

Task Decomposition:Community Identification

Connectivity ofindividuals

Individualsassemble incommunities orclusters

Complexnetworks

Randomnetworks

Small-worldnetworks

Scale-freenetworks

Degrees ofseparation

Fully connectedRandom

Clustered small worldSmall world ring lattice

Community Communication

Strogatz, 2001

Scale-Free Networks Frequency and cumulative distributions of cluster sizes inversely

proportional to !x

Strogatz, 2001

Scale-Free

8/8/2019 Mae 345 Lecture 23

8/12

Community Examples Associations

Governments Agencies

Laboratories Managers

Scientists

Military organizations Army

Corps Division

Brigade

Regiment Battalion

Company

Platoon

Squad Soldier

Special Operations

Terrorist organizations

Families

Classmates

Neighbors

Social Networks

Facebook LinkedIn

Media Networks

Corporations

Employees

Customers

Sports Leagues Teams

Managers

Players

Trainers

Airlines

Cities

Multi-Agent Systems

Specialized vs. general-purpose agents

Organizational models

Cooperators

Leader/follower (hierarchical)

Equal members Collaborators

Air, ground, and sea traffic

Customers

Competitors

Individual game players

Sports teams

Political/military organizations

Negotiators

Politicians

Employer/employee representatives

Multi-Agent Systems

Cooperation and collaboration should

lead to win-win (non-zero-sum)solutions

Competition should lead to win-lose(zero-sum) solutions

Negotiation should lead to win-win butmay lead to win-lose solutions

Team Work

Cooperativemaneuvering

Collaborativemaneuvering

8/8/2019 Mae 345 Lecture 23

9/12

Typical Characteristics of Multi-Agent Architectures

Federated (centralized)problem solving

Doctrinaire Coupled

Synchronous

Fragile

Complex

Strategic

Information-rich

Unified

Integrated

Top-down

Distributed problemsolving

Autonomous Independent

Asynchronous

Robust

Simple

Tactical

Parsimonious

Idiosyncratic

Modular

Bottom-up

Hierarchical Tree orHub-and-Spoke Network?

What is the Nature, Quality, andSignificance of Connections?

CommunicationCollaboration

Coordination

Negotiation

Competition

Conflict

Connections May ConnoteDifferent Relationships

Communication Collaboration

Coordination

Negotiation

Competition

Conflict

8/8/2019 Mae 345 Lecture 23

10/12

A CooperativeMulti-Agent System

Air Traffic Management: ACollaborative Multi-Agent System

http://www.natca.org/flight-explorer/united-states.aspx

Collaboration Coordination

8/8/2019 Mae 345 Lecture 23

11/12

Competition Principled Negotiation:Getting to Yes(Fisher, Ury, 1981)

Separate the agents from the problemFocus on interests, not positions

Invent options for mutual gain

Insist on using objective criteria

Principled Negotiation:Getting Past No(Ury, 1991)

Prepare by identifying barriers to cooperation, options, standards,and your Best Alternative to a Negotiated Agreement (BATNA)

Understand your goals, limits, and acceptable outcomes Buy time to think

Know your hot buttons, deflect attacks

Acknowledge opposing arguments

Agree when you can without conceding

Express your views without provoking

I statements, not you statements

Negotiate the rules of the game

Reframe the negotiation

Build a golden bridge that allows opponent to retreat gracefully

Engage third-party mediation or arbitration

Aim for mutual satisfaction, not victory

Forge a lasting agreement

Conventional Conflict

8/8/2019 Mae 345 Lecture 23

12/12

Terrorist Attack

http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/941/863

The 9/11 TerroristNetwork

Control Functions ofan Intelligent Agent

ConclusionMAE 345 Course Learning Objectives

! Dynamics and control of robotic devices.! Cognitive and biological paradigms for system design.! Estimate the behavior of dynamic systems.! Apply of decision-making concepts, including neural networks, expertsystems, and genetic algorithms.! Components of systems for decision-making and control, such assensors, actuators, and computers.! Systems-engineering approach to the analysis, design, and testing ofrobotic devices.! Computational problem-solving, through thorough knowledge,application, and development of analytical software.! Historical context within which robotics and intelligent systems haveevolved.! Global and ethical impact of robotics and intelligent systems in thecontext of contemporary society.! Oral and written presentation.