K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling

Oct 10 2002 1

Behavior Recognition and Behavior Recognition and Opponent Modeling inOpponent Modeling in

Autonomous Multi-Robot SystemsAutonomous Multi-Robot Systems

Keith J. O’HaraCollege of Computing

Georgia Institute of Technologykjohara@cc.gatech.edu

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling

Oct 10 2002 2

IntroductionIntroduction

• Recognizing and modeling behavior from low-level action thru high-level strategy.– Single agent primitive action– A sequence of single agent

actions– Group behavior

• To understand opponents• To understand teammates

– No Communication– Communication troublesome or

dangerous– Speak different “languages”

• Operate based on a different behavior vocabulary

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling

Oct 10 2002 3

OutlineOutline

• 2 Approaches

– Intille and Bobick (MIT)• Application of bayesian belief

networks for American football play recognition.

– Han and Veloso (CMU)• Behavior Hidden Markov Models for

robot soccer behavior recognition.

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling

Oct 10 2002 4

Important ThemesImportant Themes

• Single/Multi agent• Recognition of agents and primitive actions• Agent subgoals, goals, intentions • Group subgoals, goals, intentions• Online recognition• Uncertainty in Perception• Uncertainty/Flexibility of Plan

• Use of probabilistic techniques to deal with uncertainty.• Completely described action and observation spaces.

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling

Oct 10 2002 5

““Recognizing Multi-Agent Recognizing Multi-Agent Action from Visual Evidence”Action from Visual Evidence”

• Recognition of American football plays from real games.– Assumes we have labeled participants with rough

position and orientation estimates.

• Properties of the domain:– ComplexComplex: partially ordered causal events– Multi-agentMulti-agent: parallel event streams– Uncertain: Uncertainty in Uncertain: Uncertainty in both data and model

• Other domainsOther domains– Sports, military, traffic, roboticsSports, military, traffic, robotics

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling

Oct 10 2002 6

MethodMethod

• Method inspired by model-based object recognition techniques.

• Database of plays (temporal structure descriptions) described by temporal and logical relationships of events.

• Construct “visual network” to detect individual goals (primitive actions) from visual evidence.

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling

Oct 10 2002 7

Temporal Structure DescriptionsTemporal Structure Descriptions

• Individual Goal• Action Components• Object Assignment• Temporal Constraints

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling

Oct 10 2002 8

Visual NetworksVisual Networks

• Construct belief network (visual network) based upon visual evidence.

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling

Oct 10 2002 9

Multi-Agent BeliefMulti-Agent Belief Network Network

• Multi-Agent Networks normally contain at least 50 belief nodes and 40 evidence nodes

• Conditional and prior probabilities are determined automatically

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling

Oct 10 2002 10

ResultsResults• System of 29 tracked plays, 10 temporal play descriptions• 21/25 were recognized correctly• False positives are a problems. (plays that aren’t defined)• Recognized single-agent behavior and multi-agent plays.• Handled fuzzy temporal relationships (around, before).• Not evaluated online.• Assumes tracking/labeling/localization problem is solved. (Manually done in this work.)• Must know entire domain of observations (player states), and all possible plans (play

book).

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling

Oct 10 2002 11

““Automated Robot Behavior Automated Robot Behavior Recognition”Recognition”

• Robot Soccer– Adaptable Strategy– Narrative Agents – Coaches

• Formalism– Agent R is the observed robot– Agent O is the observing robot– R acts according to a known set of behaviors h(i)– O has a model of the set of the possible behaviors.– O must decide which h(i), R is performing.

• Must be online algorithm.• One observed robot and one observed ball.

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling

Oct 10 2002 12

• Use Hidden Markov Models (HMMs) to recognize behaviors– Motivated by success of HMMs in other “recognition”

tasks. (e.g. speech, gesture)

• A Behavioral HMM() for each behavior– Set of States

• Initial, intermediate, accept, reject

– Observations Space• Absolute/Relative Position, Dynamic (velocity)

– State Transition Matrix– Observation Probabilities– Initial State Distribution

• P(this state | observations, )

Method(1)Method(1)

s1 s2 s3

s4

O1 O2, O3 O3

O1

Go-To-Ball

O1O2O3

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling

Oct 10 2002 13

• The BHMM()– Set of States

– Observations Space

– State Transition Matrix

– Observation Probabilities

– Initial State Distribution

Method(2)Method(2)

s1 s2 s3

s4

O1 O2, O3 O3

O1

Go-To-Ball

O1O2O3

NjisSsSPa

aA

itjtji

ji

,1),|(

}{

1

}{ ioM

NisSoPob

obB

iti

ki

1),|()(

)}({

}{ i

}{}}{{}{ int rejectacceptermidiateinitial ssssN

)()()1(

)(

)(

),,...,,|(

1

21

tijij

ji

i i

i

tit

obatt

t

t

oooOsSP

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling

Oct 10 2002 14

ResultsResults

• Online algorithm• Applied to robotics domain (simulation/real-robots)• Implemented everyone’s favorite behaviors

– Go-To-Ball, Go-Behind-Ball, Intercept-Ball, Goalie-Align-Ball

• Not much quantitative evidence.• Only single agent case. • Assume each behavior to be a sequence of state traversals.• BHMM and behavior initial states must match up, or use a

timeout/restart mechanism.– Mentioned by Intille and Bobick as a problem with treating

temporal constraints as first-order markovian.

K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling

Oct 10 2002 15

ConclusionsConclusions

• New and hard problem.• Use of probabilistic techniques to deal with

uncertainty in perception and the plan.• Completely described action and observation

spaces.