Modeling Teamwork and Command and Control in Multi-Agent Systems
Thomas R. IoergerDepartment of Computer Science
Texas A&M University
TaskableAgents Architecture• similar to process networks, or HTN’s• has a customized knowledge representation
language (TRL) for encoding knowledge about tasks and methods (doctrine, mission)
• agents run as independent processes• each may have multiple parallel activities• agents represent staff positions (S2, S3...)• communicate with each other for teamwork• interact with humans (via forms: info/cmds)• interact with OTB for scenario simulation
Simulation
DIS
Agents
OneSAFTestbed
KBPDUs Translated To Facts (speed, location, unit type, etc.)
Cache
Periodic Updates From Simulation
High-Level Architecture of DBST
text, forms, map
actionsinform, request, direct, approve,
respond
RFS,CFF
mouse
PDUsOTB Agents
PucksterInterfac
e
BDE Interfac
e
PDUs
TaskableAgents Architecture
• Written in Java• TRL Knowledge Representation Language
- For Capturing Procedural Knowledge (Tasks & Methods)
• APTE Method Selection-Algorithm- responsible for building, maintaining, and
repairing task-decomposition trees• Inference Engine JARE
- Java Automated Reasoning Engine- Knowledge Base with Facts and Horn Clauses- back-chaining (like Prolog)- Updating World With Facts
TaskableAgents
TaskableAgents
sensingmessage
s
JARE KB: facts &Horn-
clauses
OTB(simulation)
operators
results
assert, query,retract
messages
APTEAlgorith
m
TRL TaskDecomposition
Hierarchy
ProcessNets
Other Agent
s
TRL KB:tasks &methods
Task Representation Language (TRL)
• Provides descriptors for: goals, tasks, methods, and operators• Tasks: “what to do”
– Can associate alternative methods, with priorities or preference conditions
– Can have termination conditions• Methods: “how to do it”
– Can define preference conditions for alternatives– Process Net
- Procedural language for specifying how to do things- While loops, if conditionals, sequential, parallel constructs- Can invoke sub-tasks or operators- Semantics based on Dynamic Logic
• Operators: lowest-level actions that can be directly executed in the simulation environment, e.g. move unit, send message, fire on enemy– Each descriptor is a schema with arguments and variables– Conditions are evaluated as queries to JARE
Example TRL Knowledge
(:Task Monitor (?unit) (:Term-cond (destroyed ?unit)) (:Method (Track-with-UAV ?unit) (:Pref-cond (not (weather cloudy)))) (:Method (Follow-with-scouts ?unit) (:Pref-cond (ground-cover dense)))) (:Method Track-with-UAV (?unit) (:Pre-cond (have-assets UAV)) (:Process (:seq (:if(:cond(not(launched UAV)))(launch UAV)) (:let((x y)(loc ?unit ?x ?y))(fly UAV ?x ?y)) (circle UAV ?x ?y))))
• useful for describing multiple ways of accomplishing tasks
• may encode preference conditions• APTE algorithm will automatically try another if one
method fails• examples:
– use of UAV vs. ATK helicopters vs. scouts for recon– suppression of direct fire with Arty/CAS– use of FASCAM to slow or re-direct advancing enemy– maintaining security:
• flank guard, patrols• neighboring units• use of terrain features• electronic surveillance
Alternative Methods
Task-Decomposition Hierarchy
T1 M1
T2 T5
T3 T4
M7 M12 M92 M60
T15 T18 T40 T45 T40 C T45
T2
level 1 level 2 level 3 level 4 level 5
Tx =Task Mx = MethodC = Condition
TOC Staff - Agent Decomposition
CDR
FSO
S3
S2
CompaniesScouts
Control indirect fire,
Artillery, Close Air,
ATK Helicopter Maintain enemy situation,Detect/evaluate threats,
Evaluate PIRs
Maintain friendly situation,Maneuver sub-units
Maneuver,React to enemy/orders,
Move along assigned route
Move to OP,Track enemy
Move/hold, Make commands/decisions,
RFI to Brigade
S2 Agent and Interactions
DP approva
l
Move to OP
Threat level,PIRs
RFI/RFSSALT/ INTSUM
intel intel
Enemy info
S2
BDE/DIV Sensors/ Recon
BDES2
Scout
CDR
CCIR S3
spot reports
Vignette 2 – Decision Point 1 [Shift Main Effort]
Variation A:Enemy major threat is on main route (AA) as in route (planned AA).
CompanySize forces
3-66=1-22 3-66=1-22 4ID X X
1CD
4ID X X
1CD CoCTmB CoC
TmATmB
TmA
AA3 AA4 AA5a
PL
IBIB
PL
PL PL
AA3 AA4
AA5a
AA5
AA6c
AA5
AA5c
3 234 2341 2351 1-2353-2341-234
CompanySize forces
Main Effort(ME)
Main Effort(ME)
Shift ME from Tm B to Co C?
DP1
• 2 Companies of 3-234 heading along AA3• 3-234 lead Bn of 234 Regt.• Situation unclear on AA5
INTEL
N Y
Variation B:Enemy major threat changes to secondary approach
• 2 Companies of 234 heading along AA3 & AA4• 3-234 lead Bn of 234 Regt intent is unclear.• Lead Bn (1-235) of 235 Regt on AA5a
INTEL
Shift ME from Tm B to Co C?
DP1
N Y
CompanySize forces
ME Switch
Vignette 3 – Decision Point 2 [Commit TF Reserve]
Variation A:Heavy enemy threat across across entire sector.
CompanySize forces
3-66=1-22 3-66=1-22 4ID X X
1CD
4ID X X
1CD CoCTmB CoC TmB
TmA
AA3 AA4 AA5a
PL
IBIB
PL
PL PLY
AA3 AA4
AA5a
AA5
AA6c
AA5
AA5c
3 234 23412351 1-2353-234
1-234
CompanySize forces
Main Effort(ME)
Main Effort(ME)
Commit the TF Reserve platoon?
DP2
• Company units of 3 different Bns on all 3 AAs • Estimate enemy will reach PL Y at same time• 238 Regt lead units not committed
INTEL
N Y
Variation B:Major enemy movement on one avenue (AA).
• 2 Companies of 234 heading down AA3• Uniform pressure on AA’s 4 & 5ar.• Calculations indicate Tm A unit can move to PL Y prior to lead of enemy unit.
INTEL
DP2
Y
Res…
Go to DP 3
Res…
Commit the TF Reserve platoon?
TmA(-)
Blocking PositionsTAI
CompanySize forces
(+)
N
Modeling Teamwork• TOC is more than just collection of staff members
• helping/backup behavior, information fusion, resource sharing, and joint decision-making
• how to model collaborative behavior?
• CAST (extension to TaskableAgents)
• adds features to TRL language for encoding team structure and process (MALLET)
• adds algorithms for coordination and communication within teams
• semantics based on joint intention theory and mutual awareness (beliefs)
Key Concept: Shared Mental Models
• Various components– static: structure of the team, communication policies...
– goals and plans
– dynamic: current situation, others’ workloads/status
• Team knowledge needed by agent team members:– roles, responsibilities, capabilities, team plans
– need to know who should act and when
– need to reason about each other
– need to know when to communicate for synchronization, coordination, disambiguation, infomation sharing, etc.
The CAST Agent Architecture• MALLET - team knowledge repres. language
– team structure (roles, capabilities, responsibilities)– team process (plans, policies)
• CAST kernel (interpreter)– convert to Petri nets (track progress, select actions)– use back-chaining theorem-prover for inference– dynamic role selection - make choices in context
• DIARG - information exchange algorithm– proactive: offer new info to those who need it
• primary references: (Yen et al., IJCAI, 2001), (Yin et al., Autonomous Agents Conf., 2000)
MALLET Multi-Agent Logical Language for Encoding Teamwork
• extension of TRL• basic definitions
– (team search-and-rescue (bill ted))– (role pilot) (role spotter)– (plays-role bill pilot)– (capable spotter use-IR-binoculars)
• conditions: (<predicate>*) with variables prefixed by ‘?’ – e.g. ((forward-scout ?unit) (location ?unit ?x ?y))
• team operators:(team-oper lift-heavy-object (?obj)
(pre-cond (at ?obj) (num-agents >= 2))(share-type AND)))
• share types: AND=together, OR=any, XOR=only 1 (excl.)
• team plans: can select certain agents or roles to do steps (like SharedPlans of Kraus and Grosz)(team-plan indirect-fire (?target) (select-role (scout ?s) (in-visibility-range ?s ?target)) (process (do S3 (verify-no-friendly-units-in-area ?target)) (while (not (destroyed ?target)) (do FSO (enter-CFF ?target)) (do ?s (perform-BDA ?target)) (if (not (hit ?target)) (do ?s (report-accuracy-of-aim FSO)) (do FSO (adjust-coordinates ?target))))))
• “compile” these into TRL using methods of Biggers and Ioerger (2001)
• other scouts can take over as backup in case of failure of ?s• responsibilities (such as monitoring, reporting); semantics
similar to joint intentions (Johnson and Ioerger, 2001)
CAST Kernel• compile team plans into Petri nets (expand sub-tasks)• cycle: sense/decide/act loop
1. update beliefs about environment in self’s KB2. check for any incoming messages from other agents3. find active steps in plan (transitions with tokens in all input places)4. if self is uniquely resp., consider executing oper.5. if oper is XOR and resp. is ambiguous, offer6. if oper is AND, broadcast READY and wait for others7. randomly choose among remaining actions and execute8. inform others of completed steps
• Dynamic Role Selection (DRS)– check role definitions, must satisfy any constraints, capable?– communicate when ambiguity exists– sync. for AND operators; select for XOR operators– could also allow individuals to vote/negotiate
DIARGDynamic Inter-Agent Rule Generator
• Info. sharing is a key to flexible teamwork
• Want to capture information flow in team, including proactive distribution of information
• Want to restrict to only the most relevant cases
• Ideal criterion:(Bel A I) ^ (Bel A (Bel B I)) ^ (Bel A (Goal B G)
^ [(Bel B I) � (Done B G)]
^ [(Bel B I) � (Done B G)]
(Goal A (Inform B I))
where is the temporal operator for ‘always’�
DIARG, continued• Explanation - A should send message I to B iff:
– A believes I is true– A believes B does not believe I (or believes it is false)– I is relevant to one of B’s goals
• i.e. pre-cond of current action that B is resp. for in team plan, • and that action would not succeed without knowing the info.
• Reasoning about observability– agents can sometimes infer that other team members
already believe certain information– e.g. based on common observability in environment– use this to filter out superfluous messages– recent work: (Rozich and Ioerger, submitted)
Command and Control• Need for tactical decision-making
– more flexibility in unplanned situation– commander agent
• How to represent of “tactics”?– battlefield geometry, relative force strength,
combined arms theory– terrain, effects on mobility– discovery of enemy intent
Naturalistic Decision-Making• NDM (Klein) - a cognitive model of human
decision-making in complex environments
• based heavily on situation assessment (SA)– 3 stages (Endsley):
• acquisition of factual information
• comprehension (abstraction, relevance, goal impact)
• projection (prediction of consequences)
• NDM is “satisficing”– take first adequate match; don’t extensively
evaluate and compare alternatives; respond
Evidence for NDM in TOCs• verbal protocol analyses
– characterize types of utterances and interactions
• representative studies– Serfaty, Entin, et al. (NDM, 1997)
• expertise as independent variable
– Pascual and Henderson (NDM, 1997)• reliance on recall from experience
– Schmitt and Klein (CCRTS, 1999)• recognitional processes in MDMP/COA
– Endsley (ARL report)• information flow in infantry platoon/urban combat
• lots of other similar C2 environments...– CIC/AAW, AWACS, fire fighting, ATC
Recognition-Primed Decision Making (RPD)
• a model of NDM• characterized by “feature-matching”• look for enough cues to trigger recognition• features could be weighted• for each situation, there is a typical response
(doctrinal, or learned from experience)• role of “mental simulation”?
RPD Flow Chart• Len Adelman and Denny Leedom
– integrate RPD within battalion TOC staff
situationclear?
status quoacceptable?
generatenew options
reduceuncertainty
currentmission plan
monitorprogress
modify
No
No Yes
Yes
generateresponse,mental sim,compat. test,modify plan...
Methods to Deal With Uncertainty
• often features are unknown, can’t evaluate
• options include: 1. suppress uncertainty 2. make default assumptions 3. confirmation bias (expectations) 4. take “probing” actions 5. forestalling until situation is more clear
Implementation of RPDin TaskableAgents
• task DetectSituation runs in parallel with other routines...
• loop until match enough features for one situation
• while some features are unknown, try various methods to findout (e.g. UAV, scouts, radar, JSTARS, Bde Int, feint...)
• drive information collection to discriminate situations
• trigger plans for response maneuvers once situation is determined (takes priority)
• go back to original mission once threat is handled
• all of this can be implemented as tasks and methods in TRL
Characterization of Situations• Situations must have lists of typical features
• Characterize by:– location of nearby enemy, combat strength– in regions relative to local axis (frame of ref.)– distance, speed, direction– intent? (e.g. attacking, bypassing, objective)– cover, uncertainty– effect of terrain on mobility/reachability
• roads, mountains, streams, bridges, marshes, forests
• also minefields, targeted areas of interest...
Types of Situations• Defensive
– getting: flanked, ambushed, enveloped, etc.• response: shift, request for support, withdraw...
– over-whelming enemy maneuver/main effort• response: impede, divert, CAS, inform Bde...
– forked maneuvers (intent?); bypass attempt
• Offensive (opportunities worth recognizing)– exploit gaps, isolate enemy units, envelopment,
use fixing force + flank attack, canalize enemy, bypass
Practical Issues• Managing priorities
– when to abandon mission plan & react tactically?– return to mission plan once done?
• Dependence of response on goals– ROE, aggression/initiative vs. defense/security– should SA involve impact on goals? (= threat?)
• Need to have “critics” to revise responses– avoid enemy TAI’s, minefields, contact...– stay near adjacent friendly units, air defense...
Conclusion
• TaskableAgents architecture
• CAST extensions for teamwork
• TOC staff agent model
• modeling command and control (on-going)
• HLA interoperability (on-going)