Faculty: Manuela Veloso, Anthony Stentz, Alex Rudnicky Brett Browning, M. Bernardine Dias Students:...

transcript

Faculty: Manuela Veloso, Anthony Stentz, Alex Rudnicky

Brett Browning, M. Bernardine Dias

Students: Thomas Harris, Brenna Argall, Gil Jones

Satanjeev Banerjee

Dynamic Human-Robot Teams Engaged in Complex Adversarial Tasks Using Language-

Based Communication

Carnegie Mellon School of Computer ScienceBoeing Human-Robot Teams Project

2Carnegie Mellon School of Computer Science Boeing Human-Robot Teams Project

Project Goals

Treasure hunt with two or more teams of humans and robots competing to locate target objects as they explore an unknown space

Research Goals/Questions: Specify team member’s roles and capabilities to

perform tasks Rapidly form ad-hoc heterogeneous teams of robots

and humans Robots and humans executing synchronized action

as a team, while communicating via speech Improve team performance with experience

Alignment with Boeing’s Objectives

Our project develops component technologies relevant to:

FCS – Force multiplication for human-robot teams NASA Code T – Emphasis on robots assisting and augmenting

humans in complex tasks Space activities (Other than Code T)

Automating launch sites Automating escape systems Automating maintenance inspections/repair Ice inspections

Aircraft maintenance Automating maintenance inspections/repair Ice inspections

Automated baggage handlers

In collaboration with Phillip Koons, Boeing

STP: Overview

“Skills” for low-level control, “Tactics” for single robot, “Plays” for teamwork

APPLICABLE offenseDONE aborted !offense

ROLE 1 pass 3 mark best_opponentROLE 2 blockROLE 3 pos_for_pass R B 1000 0 receive_pass shoot AROLE 4 defend_lane

STP: Implementation

Play selection from playbook Dynamic role assignment Active tactic determines

transition Execution monitoring Reward, and adaptation

Play 1Play 2

Play 3Role 1 Search Retrieve

Selection

ExecutionMonitoring,Adaptation

Robot 1TacticRobot 1

TacticRobot NTactic

Relevant Research Interests

1. Methods for pickup teams

2. Extend play-based coordination to distributed execution

3. Effective human-robot teamwork with pickup teams

TraderBots: Overview

Robots are organized as an economy

Team mission is to maximize production and minimize costs

Robots exchange virtual money for tasks to maximize individual profit

System is designed to align local and global profit maximization

TraderBots: Implementation

Operator

OpTrader

Robot 1

Robot 2

Robot 3

XTask 1 Task 2 Task 3

Task 4

BidsRoboTrader

Auction

Announce and clear auction

OpTraderGreedy Auction

Announce and clear auction

Relevant Research Interests

1. Human-multi-robot interaction

2. Role assignment for highly heterogeneous teams

3. Improving robustness and adaptivity

Speech: Overview

Communication in mobile environments Natural spoken language-

based interaction Supporting high semantic

content Communication in multi-

participant domains Human(s) and robots(s)

interacting on the same channel

Speech: Implementation

Sphinx ASR engine Speaker-independent recognition Speaker-adaptive capability

Phoenix semantic parser Concept extraction

RavenClaw dialogue engine Task-based representations Full mixed-initiative dialogue

Festival/Theta synthesis engine Rosetta generation engine

Portable/Wearable platforms

ROBOT (A)RavenClaw

ROBOT (B)RavenClaw

Comms Channel

Sphinx

Phoenix

Rosetta

Relevant Research Interests1. Developing conversational

algorithms for human-robot polylogue

2. Communicating navigational information between humans and robots in unstructured environments

3. Cooperative grounding of objects, locations, and tasks in novel environments

Current Status Wireless network-based (UDP) communication

protocol for human-robot interaction designed and implemented

Closed loop integration between speech-based command system, a Segway robot, and a Pioneer robot accomplished

2-D kinematic simulation tool for testing interface to Pioneer robots

Video demonstration of speech-based command of a Segway robot and a Pioneer robot

Plan for “year 1 scenario” and relevant technology development and integration to accomplish this scenario

Treasure Hunt Scenario

First Year Objectives

Basic Scenario 2 Segways, 2 Pioneers, and 1 human Discover and return to base as much treasure as

possible within a 20 minute period Treasure will be identified by landmarks currently used

by the Segway soccer team

Mapper (M) Seeker (S) Handler (H) Deliverer (D) Leader (L)

Humans X X X

Segways X X X

Pioneers X X X

Gators X X X

AIBOs X

Detailed Scenario Goal

To combine TraderBots for negotiation and role assignment with plays for synchronization

Low-level software remains independent Challenges

How do we decide which Play to adopt? How do we handle sub-teams? How do we generate “leaders”?

Step 1: Form Sub-Teams

Lets form a sub-team

Fixed sub-teams for May 2005

Step 2: Command to Search

Team A, search area 1

1. Build map

Step 3: Explore and Search

1. Follow 1. Go to search area 2. Build map

Step 4: Search

1. Follow and search for treasure

1. Execute search pattern2. Build map

Step 5: Found!

We found it!

We are at <x,y>

Step 6: Recovery

Human aided loading

Step 7: Return

1. Follow2. Unload

1. Return home2. Unload

Human aided unloading

Post Year 1 Goals Failure detection and recovery Outdoor environments Adversaries Larger more diverse teams Adaptation and learning

Faculty: Manuela Veloso, Anthony Stentz, Alex Rudnicky Brett Browning, M. Bernardine Dias Students:...

Documents