Imbuing Human-Robot Teams with Intention

RecognitionDr. Gita Sukthankargitars@eecs.ucf.edu

Students: Ken Laviers, Bennie Lewis

Funder: DARPA CSSP

Intelligent Agents Lab

SoftwareAgents

Robots(Mechanical Agents)

Humans(Biological Agents)

Intention(Plan,

Activity, Goal)Recognition

Research Problems

Improving plan, activity, and intention recognition Fast Sufficiently accurate Acquiring training data Making it sample-efficient

Determining when to act autonomously Transfer-of-control

Identifying what to do This can be the hardest problem!

Not annoying the human users! Mutual predictability Human must be able to infer the intentions of the agents/robots

Improved game and simulation AI Intelligent training and tutoring systems Human-robot interfaces Elder-care home monitoring systems

Domains

Example: Adaptive Opponents

Exploit adversarial models to improve team decision-making

Create tree of team spatio-temporal traces

Combine output from multiple classifiers to reliably recognize plays

Modify policy of key players to improve play of entire team

Adapt in real-time to the strategy employed by the human player

Online Play Recognition

Learning to Adapt Divide and conquer the

problem into several learning modules Play recognizer Successor state estimator Reward estimator

Individual modules are inaccurate but combine to learn an effective play adaptation.

Use Monte Carlo search to rapidly evaluate large number of play adaptations

Play Recognizer

?

Results

8

Adaptive agents improves the yardage gained in a play and reduce the number of interceptions

over the standard game AI.

Human-Agent-Robot Teams

Robots need humans to use their past experience and common-sense knowledge: To process ambiguous sensor data To solve complicated planning problems (e.g., figuring out the

grasp points on objects) Humans need help with repetitive tasks:

Monitoring multiple information streams (video or audio) Toggling between multiple robots

Agents can facilitate HRI by: Monitoring the humans to identify operator distraction Remembering and propagating commands intelligently across

teams of robots

User Interface

(view from an overhead camera)

User Interface

(gamepad control is popular with our student test subjects)

RSARSim

(video)

Learn Models of User Distraction

Learn model of user distraction by inserting artificial visual distractions into simulation

Identify which of the three robots the user is paying attention to

Features based on robot motion trajectories

Use EM to fit parameters to HMM model

Perform transfer-of control-when distraction levels go over a certain level

Agent

Learn Models of User Distraction

Identification of user distraction level more accurate than models that don’t remember past state

Two state classification accuracy shows our decision threshold (control vs. no-control)

Statistically significant improvements (p<.05) on time required to find the total number of victims in urban rescue scenario

5 class 2 classHMM 37% 88%

SVM 28& 67%

Bayes 28% 67%

DT 28% 67%

Multi-Robot Manipulation Sensors on robot are insufficient

for good grasp planning Toggling rapidly between robots is

complicated for users Idea: leverage commands given

by user to one robot to propagate (and translate) for second robot)

User study evaluating command paradigm: Follow Me: 2nd robot joins the 1st

robot Mirror Me: 2nd robot copies the 1st

robot Scenario involves moving piles of

objects to a goal location, some of which require two robots to move

Human-Agent-Robot Teams

User study on 20 users had very promising results Introducing these two new primitives results in

reductions in both time required to complete the task and in reducing the number of object drops in most of the scenarios

Favorable responses on the post-test questionnaire

Current work: Incorporating a learning by demonstration mode to

allow users to learn the primitives rather than having them preprogrammed by the developer

Conclusions

SoftwareAgents

Robots(Mechanical Agents)

Humans(Biological Agents)

Agents are well-positioned to serve as an enabler of mutual predictability through a combination of intention recognition

and communication monitoring.

Multi-Robot Manipulation