V-Lab … An Intelligent Virtual Laboratory for Autonomous Agents – Towards Simulating System of...

transcript

V-Lab … An Intelligent Virtual Laboratory for Autonomous

Agents – Towards Simulating System of Systems

MoMo JamshidiJamshidi,, Ph.D., DEgr., Dr. H.C. Ph.D., DEgr., Dr. H.C.F-IEEE, F-ASME, F-AAAS, F-NYAS, F-HAE, F-TWASF-IEEE, F-ASME, F-AAAS, F-NYAS, F-HAE, F-TWAS

Regents Professor, Electrical and Computer Engr. Department and

Director, Autonomous Control Engineering (ACE) CenterUniversity of New Mexico, Albuquerque

Advisor, JPL (1992-93), NASA Headquarters (1996-2003)Sr. Research Advisor, US AF Research Lab. (1984-90,2001-present)

Consultant, US DOE Office of Renewable Energy (2001-2003)

http://ace.unm.edu www.vlab.unm.edu pursue.unm.edu, moj@wacong.org

OUTLINE

• Introduction to V-Lab ®

• Discrete-EVent simulation Specification (DEVS)

• Intelligent DEVS … I-DEVS

• Applications to Multi-agent Systems

• System of Systems Engineering

• Simulation and Experiment Movies

V-Lab® is …

… a NASA Research Announcement (NRA),Cross-Enterprise Grant with NASA Ames Research Center and a joint effort between UNM ACE Center, JPL, in cooperation with the Arizona Integrate Modeling and Simulation Center (U Arizona and Arizona State U) Dr. Edward Tunstel, Jr. of JPL is one of the 3 Co-PIs on the grant.

V-Lab®’s potential role for NASARobotic Colonies – System of Systems Paradigm

MERMER

Simulation Design Qualities

Layered Architecture Breaks up the entire simulation into layers,

each with a distinct purpose and function Object Modularity

Functional components of the individual layers should be broken up into distinct objects to allow for:

• Maintainability – Changing a single object without modifying entire simulation.

• Reuse – Distinct objects can be reused in different simulations

• Object Hierarchy – Provide a structure for the composition and communication of objects

Our Solution – V-Lab®

V-Lab® V-Lab® Environment Layers

Hardware Networking • Networking foundation for inter-

machine communication Middleware

• Software environment for inter-process communication (CORBA, HLA, Sockets, etc.)

I-DEVS• Library of tools for

computational intelligence in DEVS formalism

V-Lab®• Organizational structure for

DEVS objects. • Management objects to control

time and message flow in multi-agent systems

PHYSICAL NETWORK

MIDDLEWARE

I-DEVS

V-Lab®

Provides C++ and Java classes for object design

Hierarchical structure of objects

Two Types of Objects Coupled (A-BC, B-C)

• Contains other objects

Atomic (A, B, C)• Basic functionality

to be used by coupled objects

Hierarchal tree

A B CB-C

Coupling Relation

DEVS execution

Create Models Determine imminent models

Imminent models send messages to output ports

Imminent models call Internal Transition function

Other models receive external messages Other models call External Transition

function Repeat until no models can be imminent

IIDEVSDEVS …… IIntelligent DEVSntelligent DEVS

DEVSDEVS

Coupled ModelsCoupled Models

AtomicModels

Discrete

Events

Discrete

Events

ComputationalIntelligence

Fuzzy-DEVS

Object: Implement various fuzzy functions and operations within a DEVS Java environment.Example – fuzzy inferencing

A DEVS model for a typical fuzzy rule:

IF x is A OR y is B THEN z is C

Fuzzy-DEVS, Cont’d.

Example of using fuzzy-DEVS to control an inverted pendulum

a) Membership functions for fuzzy-DEVS b) The fuzzy-DEVS controller

POSNEG ZERO

0 5 10 15 20 25 30 35-0.5

Time(sec)

c) The close-loop system d) The output of the system ()

GA’s Basic Cycle

EvaluationRecombination

New Population

Selection

Old Population

0 11 0 00 1 1 001 0

A binary chromosome

CrossoverMutation

GA-DEVS, Cont’d.

GA-DEVS implementation inside V-Lab®

GA-DEVS, Cont’d. 2

A simulation window of GA-DEVS

Stochastic Learning Automata

What is SLA? SLA is a sequential machine. Given a finite number of actions that can

be performed in a random environment, when a specific action is taken place the environment provides a random response which is either favorable or unfavorable.

The objective in the design of the automaton is to determine how the choice of the action at any stage should be guided by past actions and responses.

Stochastic Learning Automata – Reinforcement

Learning

UpdateAction

Probabilities

G(.) F(w(n),s(n))

StochasticLearningAutomaton

SLA-DEVS

Schematic of SLA inside the DEVS Environment

Stochastic Learning Automaton – A statisticalLearning approach to learning …

Simulation, DEVS-SLA … 2 Agents

(d)(c)

Simulation, Cont’d … 2 Agents

(e) (f)

(g) (h)

Neural Networks Multi-Layer Perceptron

Input Layer

Hidden LayersOutput Layer

Output 1

Output 2

The interest in neural networks comes from the networks’ ability to mimic human brain as well as its ability to learn and respond.

NN-DEVS

A perceptron structure inside DEVS

Neural Networks – A neural-based approach to learning … BPNN

NN-DEVS

Number of training data errors versus epochs(X, training epochs; Y, number of training data errors)

Neural Networks – Logical connectives “AND”, “OR”, and “XOR” are implemented in IDEVS

(b) OR(a) AND

(c) XOR

Uses the I-DEVS structure for creating objects

Organizes objects into 6 different categories SimEnv/SimMan Terrain Models Agent Models Physics Models Control Models Dynamic Models

Provides objects for the time and message flow of simulation

SimEnv

Terrain

Agents

Physics

Control

SimMan Dynamic

Hierarchical Tree

SimMan and SimEnv SimEnv

Highest leveled coupled model

Instatiates all other models

Houses all other models Couples all other

models together SimMan

Message liaison providing indirection between all other models

Controls flow of time for the simulation

Dynamic

SimEnv

SimMan

Agent Control Physics

Terrain

in/out in/out in/out

in/out in/out

Coupling Relation

High-Level Models Physics

Model physical phenomena Possibly a differential equation solver and set of

differential equations

Terrain Information about the layout of the environment

Dynamic Analyze agent’s actuator state to make

environmental changes, e.g. agent’s position and velocity.

High-Level Models

Control Algorithms that control the actuators of an

agent model Agent

Represent the agents in the simulation Contain Sensor models

• Sensor models contain all information about external world agent is aware of

Contain Actuator models• Actuators contain state information that dynamic

models use

V-Lab® CycleSimulation execution is iteration through a 5 phase cycle

Phase 1 Check for termination conditions

Phase 2 Update Agent Sensors

Phase 3 Control Algorithms update Agent Actuators

Phase 4 Wait for all actuators to change state

Phase 5 Dynamic models update agents and environment

A V-Lab® THEME EXAMPLE SIMULATION

SimEnv

Terrain Model

SimManSimMan

PlotModel

PlotModelRover

Dynamics

Sensors

Controller

Block Diagram of Theme Example

A THEME EXAMPLE SIMULATION – Rover Dynamics

,max||rrvv

cosxv& w&cosxv&sinyv&w&

dx/dt=vcosdy/dt=vcsin

d/dt=w

A THEME EXAMPLE SIMULATION – Fuzzy Controller

c) Forward velocityd) Angular velocity. d) Angular velocity.

a) Distance to objectiveb) Angle error b) Angle error

Rule1: IF distance is ‘Near’, THEN forward velocity is ‘Slow’.Rule2: IF distance is ‘Far’, THEN forward velocity is ‘Speedy’.Rule3: IF angle is ‘Right’, THEN angular velocity is‘TurnRight’; forward velocity is ‘Slow’. etc.

Pushing task:Pushing task:

Pushing region

GMMMMMMMMMMMMMM

BMMMMMMMMMMMMMM

rwMMMMMMMMMMMMMM

lwMMMMMMMMMMMMMM

fr<0fr>0

fl=0fl>0

Cooperative pushing task:Cooperative pushing task:

GMMMMMMMMMMMMMM

BMMMMMMMMMMMMMM

rwMMMMMMMMMMMMMM

lwMMMMMMMMMMMMMM

Pushing regions

Cooperative Pushing TaskCooperative Pushing Task

“in”?command

“out”!goal_reached

“in”?object_reached

“in”?object_lost

“out”?object_touched

“in”?everybody_ready

“in”?outside_pushing_region

“out”?object_reached_the_goal

“in” “out”

Simulation ResultsSimulation Results

GA-DEVS Controller (1)

Goal (xg, yg)

(x, y,θ)

sin)(2

cos)(2

Goal Goal Goal

First simulation sample task using the GA control system, T=142 steps

Second simulation sample task using the GA control system, T=236 steps

Goal Goal Goal Goal

1 2 3 4

5 6 7 8

GA-DEVS Controller (6) Trajectory of the robot in a task

Trajectory of the robot for first task Trajectory of the robot for second task

THEME EXAMPLE SIMULATION - Sensors

Infra-red sensors. … Each sensor has a unique ID,X_s, Y_s and theta_s, where X_s is the X-position, Y_s is the Y-position and theta_s is the angle of each sensor relative to the rover in X-Y plane.

GPS sensors … An atomic model simply simulates a position sensor, giving out the current position of the rover

Compass Sensors … Respective atomic models simulate the sensing by converting the angle of the rover from radians to degrees.

A THEME EXAMPLE SIMULATION – Distributed 2

Parts of a distributed simulation with GenDEVS and Sockets

Commercial ROVERS

Two Pioneer II Mobile rovers are used to implement IDEVS-JAVA(V-Lab®).

Movies will show demos at the end.

Experiment ResultsExperiment Results (DEVS-Fuzzy)(DEVS-Fuzzy)

Experiment Results (GA-DEVS)

Application: Pioneer 2 mobile robot Two experiments of path planning

Goal searching task w/o obstacles. Robot (10, 9, -π/2) (4, 4, -π/2)returns to the home point (10, 9, -π/2) after traveling to the goal point.

Robot task of goal seeking (Home -> Goal -> Home)

System of Systems - Introduction

Changing Aerospace and Defense Industry

Emphasis on “large-scale systems integration” Customers seeking solutions to

problems, not asking for specific vehicles Mix of multiple systems capable of

independent operation but interact with each other

Emerging System of Systems Context

EMERGING CONTEXT: SYSTEM OF SYSTEMS

Meeting a need or set of needs with a mix of independently operating systems New and existing aircraft,

spacecraft, ground equipment, other independent systems

System of Systems Examples Coast Guard Deepwater

Program FAA Air Traffic

Management Army Future Combat

Systems Robotic Colonies

System of Systems

SoS: A metasystem consisting of multiple autonomous embedded complex systems that can be diverse in:

Technology Technology Context Context Operation Operation Geography Geography Conceptual frameConceptual frame

An airplane is not SoS, an airport is a SoS. A robot is not a SoS, but a robotic colony is a SoS Significant challenges:

Determining the appropriate mix of independent systems The operation of a SoS occurs in an uncertain environment Interoperability

Keating, et al., 2003

System of Systems Definitions SoS: No universally accepted definition

1. Operational & Management 1. Operational & Management independence+Geographical Dist. + independence+Geographical Dist. + Emerging Behavior+Evolionary Dev. Emerging Behavior+Evolionary Dev. (ML, Space)(ML, Space)

2. Integration+Inter-Operability.2. Integration+Inter-Operability.+Optmiz. to enhance battlefield +Optmiz. to enhance battlefield scenarios (ML)scenarios (ML)

3. 3. Large scale + distributed Systems Large scale + distributed Systems Leading to more complex systemsLeading to more complex systems (Private Enterprize)(Private Enterprize)

4. Within the context of warfighting 4. Within the context of warfighting systems – Inter Op.+Com’d. systems – Inter Op.+Com’d. Synergy+Cont.+ Comp.+ Comm. Synergy+Cont.+ Comp.+ Comm. +Info. (C4I) & Intel. (ML)+Info. (C4I) & Intel. (ML)

DISTINCTION BETWEEN SYSTEM ENGINEERING AND SoSE

SoSE represents a necessary extension and evolution of traditional system engineering.

Greatly expanded SoS requirements for tiered levels of discipline and rigor.

Centralized control structure vs. de-centralized control structure

A typical individual system (well defined end state, fixed budget, well defined schedule, technical baselines,homogeneous)

A typical System of Systems (not well defined end state, periodic budget variations, heterogeneous)

Nature of SoSE EngineeringNature of SoSE Engineering

Existing Complex SystemsExisting Complex SystemsExclusive, Autonomous, Local Exclusive, Autonomous, Local TransformationTransformation

System of Systems

Integrated, Aligned, and Transforming

System of SystemsSystem of SystemsInterconnected, Integrated Mission, Interconnected, Integrated Mission, Global, Emergent StructureGlobal, Emergent Structure

System of Systems Engineering

The design, deployment, operation, and transformation of metasystems that must function as an integrated complex system to produce desirable results.

Keating, et. al 2003 Jamshidi, 2005

System of SystemsPROBLEM THEMES

1. Fragmented Perspectives1. Fragmented Perspectives

2. Lack of Rigorous Development2. Lack of Rigorous Development

3. Lack of Theoretical Grounding 3. Lack of Theoretical Grounding 4. IT Dominance4. IT Dominance

5. Limitations of trad. SE single 5. Limitations of trad. SE single system focus system focus

6. Whole Systems Analysis6. Whole Systems Analysis

Modeling IssuesModeling of Systems of Systems?

Traditional LSS Modeling

LSSLSS

SoSE Modeling Difficulty

SS-1 SS-1 SS-1

SS-2SS-2

EXAMPLES

Air Traffic Control Personal Air Vehicles Future Combat Systems Internet Intelligent Transport Systems US Coast Guard Integrated

Deepwater System, etc., etc.

US COAST GUARD INTEGRATED DEEPWATER SYSTEM

The United States Coast Guard Protect the public, the

environment, and U.S. economic and security interests in any maritime region

International waters and America's coasts, ports, and inland waterways.

Missions Maritime Security Maritime Safety Maritime Mobility National Defense Protection of Natural Resources

US COAST GUARD INTEGRATED DEEPWATER SYSTEM

An integrated approach to upgrading existing assets while transitioning to newer, more capable platforms with improved systems for command, control, communications, computers, intelligence, surveillance, and reconnaissance and innovative logistics support.

Ensure compatibility and interoperability of deepwater asstes, while providing high levels of operational effectiveness.

Conclusions V-Lab® is still at the beginning of its development Three sub-groups are working:

Software: Responsible for the design and implementation of V-LAB® (from Network to V-LAB®)

Hardware: Implementation of its tasks, features on industrial or commercial rovers

Theory: Develop new theories to bridge the gap between Soft Computing (FL, NN, GA, GP, SLA, etc.) and DEVS environment.

Future: Leverage V-Lab ® success and 35 years of research on LSS for SoSE applications is among future directions.

Cover , Modeling and Simulation Magazine, 2003

Publications – to date

1. J. Burge, A. El-Osery, M. Jamshidi and M. Fathi “V-LAB : A Virtual Laboratory for Distributed Robotic Modeling and Simulation,” Proc. IEEE Int. Conference on Systems, Man and Cybernetics, Tucson, AZ, October, 2001.

2. A. El-Osery, J. Burge, M. Jamshidi and M. Fathi “Stochastic Learning

Automaton for Learning Control of Robotic Agents, “Proc. IEEE Int. Conference on Systems, Man and Cybernetics, Tucson, AZ, October, 2001.

3. A. El-Osery, J. Burge, A. Saha, M. Jamshidi, M. Fathi and M. Akbarzadeh-T., “V-Lab – A Distributed Simulation and Modeling Environment for Robotic Agents – Control Through Stochastic Learning Automata,” to appear in IEEE Transactions on Systems, Man and Cybernetics , Vol. 32, No. 6, pp. 795-804, December, 2002.

Publications – to date 24. A. El-Osery and M. Jamshidi, “Implementation of an SLA-Based Controller in a Virtual Laboratory,” Proc. IEEE Int. Conference on Systems, Man and Cybernetics, Hammamet, Tunisia, October, 2002

5. M. Jamshidi, S. Sheikh-Bahaei, J. Kitzinger, P. Sridhar, S. Xia, Y. Wang, J. Liu, E. Tunstel, Jr , M. Akbarzadeh, A. El-Osery, M. Fathi, X. Hu, and B. P. Zeigler, “A Distributed Intelligent Discrete-Event Environment for Autonomous Agents Simulation,” Chapter 11, Applied Simulation, Kluwer Publishers, Amsterdam, the Netherlands, 2003.

6. M. JAMSHIDI, S. Sheikh-Bahaei, J. Kitzinger, P. Sridhar, S. Beatty, S. Xia, Y. Wang, T. Song, U. Dole, J. Liu, E. Tunstel, Jr., M. Akbarzadeh, P. Lino, A. El-Osery, M. fathi, X. Hu, and B. P. Zeigler,

“V-Lab© - A Distributed Intelligent Discrete-Event Environment for Autonomous Agents Simulation,” Intelligent Automation and Soft Computing - Autosoft Journal, Vol. 9, No. 3, pp. 181-214 2003.

6. M. Jamshidi, S. Sheikh-Bahaei, J. Kitzinger, P. Sridhar, S. Xia, Y. Wang, J. Liu, E. Tunstel, Jr , M. Akbarzadeh, A. El-Osery, T. Song, M. Fathi,“A Distributed Intelligent Discrete-Event Environment for Autonomous Agents Simulation,” Chapter 11, Modeling and Simulation Magazine, (Cover article), M&S International, Snn Diego, CA, 2003.

7. P. Sridhar and M. Jamshidi, “A Framework for Multi-agent Discrete Event Simulation: V-Lab®,” Proc. IEEE SMC, the Hague, the Netherlands, Oct 10-12, 2004.

Publications – to date 3

8. S. Sheikh-Bahaei, M. Jamshidi,” Discrete Event Fuzzy Logic Control with Application to Sensor-Discrete Event Fuzzy Logic Control with Application to Sensor-Based Intelligent Mobile Robot,” Based Intelligent Mobile Robot,” Proc. WAC 2004, Seville, Spain, June 28-July 1, 2004.

9. P. Sridhar and M. Jamshidi, “A Multi-agent Discrete Event Simulator: V-Lab®,” Proc. WAC 2004. Seville, Spain, July 28-July 1, 2004.

10. S. Sheikh-Bahaei, M. Jamshidi, and P. Lino, “An Intelligent Discrete Event Approach to Modeling, Simulation and Control of Autonomous Agents,” Intelligent Automation and Soft Computing - Autosoft Journal, Vol. 10, No. 4, pp. 337-348, 2004.

THANK YOU!

V-Lab … An Intelligent Virtual Laboratory for Autonomous Agents – Towards Simulating System of...

Documents