V-Lab … An Intelligent Virtual Laboratory for Autonomous
Agents – Towards Simulating System of Systems
MoMo JamshidiJamshidi,, Ph.D., DEgr., Dr. H.C. Ph.D., DEgr., Dr. H.C.F-IEEE, F-ASME, F-AAAS, F-NYAS, F-HAE, F-TWASF-IEEE, F-ASME, F-AAAS, F-NYAS, F-HAE, F-TWAS
Regents Professor, Electrical and Computer Engr. Department and
Director, Autonomous Control Engineering (ACE) CenterUniversity of New Mexico, Albuquerque
Advisor, JPL (1992-93), NASA Headquarters (1996-2003)Sr. Research Advisor, US AF Research Lab. (1984-90,2001-present)
Consultant, US DOE Office of Renewable Energy (2001-2003)
http://ace.unm.edu www.vlab.unm.edu pursue.unm.edu, [email protected]
OUTLINE
• Introduction to V-Lab ®
• Discrete-EVent simulation Specification (DEVS)
• Intelligent DEVS … I-DEVS
• Applications to Multi-agent Systems
• System of Systems Engineering
• Simulation and Experiment Movies
V-Lab® is …
… a NASA Research Announcement (NRA),Cross-Enterprise Grant with NASA Ames Research Center and a joint effort between UNM ACE Center, JPL, in cooperation with the Arizona Integrate Modeling and Simulation Center (U Arizona and Arizona State U) Dr. Edward Tunstel, Jr. of JPL is one of the 3 Co-PIs on the grant.
V-Lab®’s potential role for NASARobotic Colonies – System of Systems Paradigm
MER
MERMER
MER
Simulation Design Qualities
Layered Architecture Breaks up the entire simulation into layers,
each with a distinct purpose and function Object Modularity
Functional components of the individual layers should be broken up into distinct objects to allow for:
• Maintainability – Changing a single object without modifying entire simulation.
• Reuse – Distinct objects can be reused in different simulations
• Object Hierarchy – Provide a structure for the composition and communication of objects
Our Solution – V-Lab®
V-Lab® V-Lab® Environment Layers
Hardware Networking • Networking foundation for inter-
machine communication Middleware
• Software environment for inter-process communication (CORBA, HLA, Sockets, etc.)
I-DEVS• Library of tools for
computational intelligence in DEVS formalism
V-Lab®• Organizational structure for
DEVS objects. • Management objects to control
time and message flow in multi-agent systems
PHYSICAL NETWORK
MIDDLEWARE
I-DEVS
V-Lab®
DEVS
Provides C++ and Java classes for object design
Hierarchical structure of objects
Two Types of Objects Coupled (A-BC, B-C)
• Contains other objects
Atomic (A, B, C)• Basic functionality
to be used by coupled objects
A-BC
A B-C
B C
Hierarchal tree
A-BC
A B CB-C
Coupling Relation
DEVS execution
Create Models Determine imminent models
Imminent models send messages to output ports
Imminent models call Internal Transition function
Other models receive external messages Other models call External Transition
function Repeat until no models can be imminent
IIDEVSDEVS …… IIntelligent DEVSntelligent DEVS
DEVSDEVS
Coupled ModelsCoupled Models
AtomicModels
AtomicModels
Discrete
Events
Discrete
Events
ComputationalIntelligence
Fuzzy-DEVS
Object: Implement various fuzzy functions and operations within a DEVS Java environment.Example – fuzzy inferencing
X Yμ
X Yμ
OR
X Yμ
x
y
z
A
B
C
AMF
CMF
A DEVS model for a typical fuzzy rule:
IF x is A OR y is B THEN z is C
Fuzzy-DEVS, Cont’d.
Example of using fuzzy-DEVS to control an inverted pendulum
a) Membership functions for fuzzy-DEVS b) The fuzzy-DEVS controller
POSNEG ZERO
0 5 10 15 20 25 30 35-0.5
0
0.5
1
Time(sec)
Theta
c) The close-loop system d) The output of the system ()
GA’s Basic Cycle
EvaluationRecombination
New Population
Selection
Old Population
0 11 0 00 1 1 001 0
A binary chromosome
CrossoverMutation
GA-DEVS, Cont’d.
GA-DEVS implementation inside V-Lab®
GA-DEVS, Cont’d. 2
A simulation window of GA-DEVS
Stochastic Learning Automata
What is SLA? SLA is a sequential machine. Given a finite number of actions that can
be performed in a random environment, when a specific action is taken place the environment provides a random response which is either favorable or unfavorable.
The objective in the design of the automaton is to determine how the choice of the action at any stage should be guided by past actions and responses.
Stochastic Learning Automata – Reinforcement
Learning
UpdateAction
Probabilities
G(.) F(w(n),s(n))
s(n)
StochasticLearningAutomaton
y(n)
SLA-DEVS
Schematic of SLA inside the DEVS Environment
Stochastic Learning Automaton – A statisticalLearning approach to learning …
Simulation, DEVS-SLA … 2 Agents
(a)
(d)(c)
(b)
Simulation, Cont’d … 2 Agents
(e) (f)
(g) (h)
Neural Networks Multi-Layer Perceptron
Input Layer
Hidden LayersOutput Layer
Input
Output 1
Output 2
The interest in neural networks comes from the networks’ ability to mimic human brain as well as its ability to learn and respond.
NN-DEVS
A perceptron structure inside DEVS
Neural Networks – A neural-based approach to learning … BPNN
NN-DEVS
Number of training data errors versus epochs(X, training epochs; Y, number of training data errors)
Neural Networks – Logical connectives “AND”, “OR”, and “XOR” are implemented in IDEVS
(b) OR(a) AND
(c) XOR
V-Lab
Uses the I-DEVS structure for creating objects
Organizes objects into 6 different categories SimEnv/SimMan Terrain Models Agent Models Physics Models Control Models Dynamic Models
Provides objects for the time and message flow of simulation
SimEnv
Terrain
Agents
Physics
Control
SimMan Dynamic
Hierarchical Tree
SimMan and SimEnv SimEnv
Highest leveled coupled model
Instatiates all other models
Houses all other models Couples all other
models together SimMan
Message liaison providing indirection between all other models
Controls flow of time for the simulation
Dynamic
SimEnv
SimMan
Agent Control Physics
Terrain
outin
inout
in/out in/out in/out
in/out in/out
Coupling Relation
High-Level Models Physics
Model physical phenomena Possibly a differential equation solver and set of
differential equations
Terrain Information about the layout of the environment
Dynamic Analyze agent’s actuator state to make
environmental changes, e.g. agent’s position and velocity.
High-Level Models
Control Algorithms that control the actuators of an
agent model Agent
Represent the agents in the simulation Contain Sensor models
• Sensor models contain all information about external world agent is aware of
Contain Actuator models• Actuators contain state information that dynamic
models use
V-Lab® CycleSimulation execution is iteration through a 5 phase cycle
Phase 1 Check for termination conditions
Phase 2 Update Agent Sensors
Phase 3 Control Algorithms update Agent Actuators
Phase 4 Wait for all actuators to change state
Phase 5 Dynamic models update agents and environment
A V-Lab® THEME EXAMPLE SIMULATION
SimEnv
Terrain Model
Terrain Model
SimManSimMan
PlotModel
PlotModelRover
Dynamics
Sensors
Controller
Block Diagram of Theme Example
A THEME EXAMPLE SIMULATION – Rover Dynamics
1cos
21
sin21
r l
r l
r l
x v v
y v v
v vl
,max||rrvv
x
y
lv
rv
cosxv& w&cosxv&sinyv&w&
dx/dt=vcosdy/dt=vcsin
d/dt=w
A THEME EXAMPLE SIMULATION – Fuzzy Controller
c) Forward velocityd) Angular velocity. d) Angular velocity.
a) Distance to objectiveb) Angle error b) Angle error
Rule1: IF distance is ‘Near’, THEN forward velocity is ‘Slow’.Rule2: IF distance is ‘Far’, THEN forward velocity is ‘Speedy’.Rule3: IF angle is ‘Right’, THEN angular velocity is‘TurnRight’; forward velocity is ‘Slow’. etc.
Pushing task:Pushing task:
Pushing region
rA
rB
GMMMMMMMMMMMMMM
BMMMMMMMMMMMMMM
rwMMMMMMMMMMMMMM
lwMMMMMMMMMMMMMM
fr=0
fr<0fr>0
fl<0
fl=0fl>0
A
Cooperative pushing task:Cooperative pushing task:
rB
GMMMMMMMMMMMMMM
BMMMMMMMMMMMMMM
rwMMMMMMMMMMMMMM
lwMMMMMMMMMMMMMM
Pushing regions
Cooperative Pushing TaskCooperative Pushing Task
“in”?command
“out”!goal_reached
“in”?object_reached
“in”?object_lost
“out”?object_touched
“in”?everybody_ready
“in”?outside_pushing_region
“out”?object_reached_the_goal
“in” “out”
Simulation ResultsSimulation Results
GA-DEVS Controller (1)
x
Goal (xg, yg)
l
(x, y,θ)
y
ir1
ir3
ir2
θg
θ
)(1
sin)(2
1
cos)(2
1
lr
lr
lr
vvl
vvy
vvx
1
0
0
0
sin
cos
wvy
x
vl
vr
vw
GA-DEVS Controller (4)
Goal Goal Goal
Goal Goal Goal
1 2 3
4 5 6
First simulation sample task using the GA control system, T=142 steps
GA-DEVS Controller (5)
Second simulation sample task using the GA control system, T=236 steps
Goal Goal Goal Goal
Goal Goal Goal Goal
1 2 3 4
5 6 7 8
GA-DEVS Controller (6) Trajectory of the robot in a task
X
YY
X
Trajectory of the robot for first task Trajectory of the robot for second task
THEME EXAMPLE SIMULATION - Sensors
Infra-red sensors. … Each sensor has a unique ID,X_s, Y_s and theta_s, where X_s is the X-position, Y_s is the Y-position and theta_s is the angle of each sensor relative to the rover in X-Y plane.
GPS sensors … An atomic model simply simulates a position sensor, giving out the current position of the rover
Compass Sensors … Respective atomic models simulate the sensing by converting the angle of the rover from radians to degrees.
A THEME EXAMPLE SIMULATION – Distributed 2
Parts of a distributed simulation with GenDEVS and Sockets
Commercial ROVERS
Two Pioneer II Mobile rovers are used to implement IDEVS-JAVA(V-Lab®).
Movies will show demos at the end.
Experiment ResultsExperiment Results (DEVS-Fuzzy)(DEVS-Fuzzy)
Experiment Results (GA-DEVS)
Application: Pioneer 2 mobile robot Two experiments of path planning
Goal searching task w/o obstacles. Robot (10, 9, -π/2) (4, 4, -π/2)returns to the home point (10, 9, -π/2) after traveling to the goal point.
Robot task of goal seeking (Home -> Goal -> Home)
System of Systems - Introduction
Changing Aerospace and Defense Industry
Emphasis on “large-scale systems integration” Customers seeking solutions to
problems, not asking for specific vehicles Mix of multiple systems capable of
independent operation but interact with each other
Emerging System of Systems Context
EMERGING CONTEXT: SYSTEM OF SYSTEMS
Meeting a need or set of needs with a mix of independently operating systems New and existing aircraft,
spacecraft, ground equipment, other independent systems
System of Systems Examples Coast Guard Deepwater
Program FAA Air Traffic
Management Army Future Combat
Systems Robotic Colonies
System of Systems
SoS: A metasystem consisting of multiple autonomous embedded complex systems that can be diverse in:
Technology Technology Context Context Operation Operation Geography Geography Conceptual frameConceptual frame
An airplane is not SoS, an airport is a SoS. A robot is not a SoS, but a robotic colony is a SoS Significant challenges:
Determining the appropriate mix of independent systems The operation of a SoS occurs in an uncertain environment Interoperability
Keating, et al., 2003
System of Systems Definitions SoS: No universally accepted definition
1. Operational & Management 1. Operational & Management independence+Geographical Dist. + independence+Geographical Dist. + Emerging Behavior+Evolionary Dev. Emerging Behavior+Evolionary Dev. (ML, Space)(ML, Space)
2. Integration+Inter-Operability.2. Integration+Inter-Operability.+Optmiz. to enhance battlefield +Optmiz. to enhance battlefield scenarios (ML)scenarios (ML)
3. 3. Large scale + distributed Systems Large scale + distributed Systems Leading to more complex systemsLeading to more complex systems (Private Enterprize)(Private Enterprize)
4. Within the context of warfighting 4. Within the context of warfighting systems – Inter Op.+Com’d. systems – Inter Op.+Com’d. Synergy+Cont.+ Comp.+ Comm. Synergy+Cont.+ Comp.+ Comm. +Info. (C4I) & Intel. (ML)+Info. (C4I) & Intel. (ML)
Keating, et al., 2003
DISTINCTION BETWEEN SYSTEM ENGINEERING AND SoSE
SoSE represents a necessary extension and evolution of traditional system engineering.
Greatly expanded SoS requirements for tiered levels of discipline and rigor.
Centralized control structure vs. de-centralized control structure
A typical individual system (well defined end state, fixed budget, well defined schedule, technical baselines,homogeneous)
A typical System of Systems (not well defined end state, periodic budget variations, heterogeneous)
Nature of SoSE EngineeringNature of SoSE Engineering
Existing Complex SystemsExisting Complex SystemsExclusive, Autonomous, Local Exclusive, Autonomous, Local TransformationTransformation
Keating, et al., 2003
System of Systems
Integrated, Aligned, and Transforming
System of SystemsSystem of SystemsInterconnected, Integrated Mission, Interconnected, Integrated Mission, Global, Emergent StructureGlobal, Emergent Structure
Keating, et al., 2003
System of Systems Engineering
The design, deployment, operation, and transformation of metasystems that must function as an integrated complex system to produce desirable results.
Keating, et. al 2003 Jamshidi, 2005
System of SystemsPROBLEM THEMES
1. Fragmented Perspectives1. Fragmented Perspectives
2. Lack of Rigorous Development2. Lack of Rigorous Development
3. Lack of Theoretical Grounding 3. Lack of Theoretical Grounding 4. IT Dominance4. IT Dominance
5. Limitations of trad. SE single 5. Limitations of trad. SE single system focus system focus
6. Whole Systems Analysis6. Whole Systems Analysis
Keating, et al., 2003
Modeling IssuesModeling of Systems of Systems?
LSS
ss ss
…
Traditional LSS Modeling
LSS
LSS
LSSLSS
LSSLSS
TOP
BOT.
BOT.
TOP
SoSE Modeling Difficulty
SS-1 SS-1 SS-1
SS-2SS-2
SS-2
SoSE
LSS
EXAMPLES
Air Traffic Control Personal Air Vehicles Future Combat Systems Internet Intelligent Transport Systems US Coast Guard Integrated
Deepwater System, etc., etc.
US COAST GUARD INTEGRATED DEEPWATER SYSTEM
The United States Coast Guard Protect the public, the
environment, and U.S. economic and security interests in any maritime region
International waters and America's coasts, ports, and inland waterways.
Missions Maritime Security Maritime Safety Maritime Mobility National Defense Protection of Natural Resources
US COAST GUARD INTEGRATED DEEPWATER SYSTEM
An integrated approach to upgrading existing assets while transitioning to newer, more capable platforms with improved systems for command, control, communications, computers, intelligence, surveillance, and reconnaissance and innovative logistics support.
Ensure compatibility and interoperability of deepwater asstes, while providing high levels of operational effectiveness.
Conclusions V-Lab® is still at the beginning of its development Three sub-groups are working:
Software: Responsible for the design and implementation of V-LAB® (from Network to V-LAB®)
Hardware: Implementation of its tasks, features on industrial or commercial rovers
Theory: Develop new theories to bridge the gap between Soft Computing (FL, NN, GA, GP, SLA, etc.) and DEVS environment.
Future: Leverage V-Lab ® success and 35 years of research on LSS for SoSE applications is among future directions.
Cover , Modeling and Simulation Magazine, 2003
Publications – to date
1. J. Burge, A. El-Osery, M. Jamshidi and M. Fathi “V-LAB : A Virtual Laboratory for Distributed Robotic Modeling and Simulation,” Proc. IEEE Int. Conference on Systems, Man and Cybernetics, Tucson, AZ, October, 2001.
2. A. El-Osery, J. Burge, M. Jamshidi and M. Fathi “Stochastic Learning
Automaton for Learning Control of Robotic Agents, “Proc. IEEE Int. Conference on Systems, Man and Cybernetics, Tucson, AZ, October, 2001.
3. A. El-Osery, J. Burge, A. Saha, M. Jamshidi, M. Fathi and M. Akbarzadeh-T., “V-Lab – A Distributed Simulation and Modeling Environment for Robotic Agents – Control Through Stochastic Learning Automata,” to appear in IEEE Transactions on Systems, Man and Cybernetics , Vol. 32, No. 6, pp. 795-804, December, 2002.
Publications – to date 24. A. El-Osery and M. Jamshidi, “Implementation of an SLA-Based Controller in a Virtual Laboratory,” Proc. IEEE Int. Conference on Systems, Man and Cybernetics, Hammamet, Tunisia, October, 2002
5. M. Jamshidi, S. Sheikh-Bahaei, J. Kitzinger, P. Sridhar, S. Xia, Y. Wang, J. Liu, E. Tunstel, Jr , M. Akbarzadeh, A. El-Osery, M. Fathi, X. Hu, and B. P. Zeigler, “A Distributed Intelligent Discrete-Event Environment for Autonomous Agents Simulation,” Chapter 11, Applied Simulation, Kluwer Publishers, Amsterdam, the Netherlands, 2003.
6. M. JAMSHIDI, S. Sheikh-Bahaei, J. Kitzinger, P. Sridhar, S. Beatty, S. Xia, Y. Wang, T. Song, U. Dole, J. Liu, E. Tunstel, Jr., M. Akbarzadeh, P. Lino, A. El-Osery, M. fathi, X. Hu, and B. P. Zeigler,
“V-Lab© - A Distributed Intelligent Discrete-Event Environment for Autonomous Agents Simulation,” Intelligent Automation and Soft Computing - Autosoft Journal, Vol. 9, No. 3, pp. 181-214 2003.
6. M. Jamshidi, S. Sheikh-Bahaei, J. Kitzinger, P. Sridhar, S. Xia, Y. Wang, J. Liu, E. Tunstel, Jr , M. Akbarzadeh, A. El-Osery, T. Song, M. Fathi,“A Distributed Intelligent Discrete-Event Environment for Autonomous Agents Simulation,” Chapter 11, Modeling and Simulation Magazine, (Cover article), M&S International, Snn Diego, CA, 2003.
7. P. Sridhar and M. Jamshidi, “A Framework for Multi-agent Discrete Event Simulation: V-Lab®,” Proc. IEEE SMC, the Hague, the Netherlands, Oct 10-12, 2004.
Publications – to date 3
8. S. Sheikh-Bahaei, M. Jamshidi,” Discrete Event Fuzzy Logic Control with Application to Sensor-Discrete Event Fuzzy Logic Control with Application to Sensor-Based Intelligent Mobile Robot,” Based Intelligent Mobile Robot,” Proc. WAC 2004, Seville, Spain, June 28-July 1, 2004.
9. P. Sridhar and M. Jamshidi, “A Multi-agent Discrete Event Simulator: V-Lab®,” Proc. WAC 2004. Seville, Spain, July 28-July 1, 2004.
10. S. Sheikh-Bahaei, M. Jamshidi, and P. Lino, “An Intelligent Discrete Event Approach to Modeling, Simulation and Control of Autonomous Agents,” Intelligent Automation and Soft Computing - Autosoft Journal, Vol. 10, No. 4, pp. 337-348, 2004.
THANK YOU!