Date post: | 19-Dec-2015 |
Category: |
Documents |
View: | 214 times |
Download: | 0 times |
Heterogeneous Multi-Robot Dialogues for Search Tasks
Thomas K Harris, Satanjeev (Bano) Banerjee Alexander Rudnicky
AAAI Spring Symposium 2005: Dialogical Robots
AAAI Spring ’05 Symposium: Dialogical Robots 2
Communication among autonomous robots and humans
→ Embodied (not necessarily robotic, or even
anthropomorphic) agents will become ubiquitous
→ NL dialogue will be a useful modality
→ Agent’s dialogue and task models will remain tightly coupled, and independent from other agents
We will be talking to multiple agents without a (completely) shared dialog system
AAAI Spring ’05 Symposium: Dialogical Robots 3
Gedankenexperiment
• The problem of many robots: sweeper, furniture mover, baby monitor.
• Task: clean up house; don’t wake baby up.
• Who do you talk to?• What do you say?• Who talks back?
AAAI Spring ’05 Symposium: Dialogical Robots 4
Issues for polylogical systems
• What’s a reasonable architecture?– A single system controlling multiple robots– A single DM directing autonomous robots– Shared understanding/generation– Shared microphone and speakers– Robots self-contained: Nothing shared
• How to communicate with multiple agents?– Directions to the entire team? (“talk into the air”)– Mediation through a designated team leader? (foreman)– Instructions to each team member? (direct management)– To a proxy agent within/without the team? (personal assistant)
AAAI Spring ’05 Symposium: Dialogical Robots 5
What do we have today?
• A platform that supports multiple (robot) participants– Sphinx II ASR; Festival TTS– RavenClaw dialogue manager– Galaxy-II message-passing architecture– Several “back-end” interfaces
• A basic corridor movement domain– Commands to move robots along in vectors and
towards named locations– Mechanisms for describing robot status and location
to human interlocutor
AAAI Spring ’05 Symposium: Dialogical Robots 6
AGENT2
AGENT1
HUMAN INTERFACE
Current Architecture
ASR
Multi-voice TTS
DM1
DM2
Robot1
Robot2
Interpret1
Interpret2
AAAI Spring ’05 Symposium: Dialogical Robots 7
AGENT2
AGENT1HUMAN INTERFACE
Implementation
Sphinx II Phoenix Helios
Galaxy
RavenclawBashful
RavenclawClyde
Clyde
Bashful
Robot Communication System
RosettaSwift
AAAI Spring ’05 Symposium: Dialogical Robots 8
Human-Robot Treasure Hunt
• One Human
• Two robots– Segway– Pioneer
• Treasures “hidden” in large space
• TASK: Retrieve all treasures
AAAI Spring ’05 Symposium: Dialogical Robots 10
Issues in H-R communication
1. How do people decompose the task into sub-tasks?
2. What language do people use to get the tasks performed by the robots?
3. Given a human command, what is the expected robot behavior?
Explore using Wizard of Oz experiments
AAAI Spring ’05 Symposium: Dialogical Robots 11
WOZ design
Communication takes place by normal speech, walkie-talkie, and through the webcam.
•1 experimenter, 2 robot actors, 1 participant.•Experimenter places treasure, simulates robot treasure sensors.•1st robot actor is blind, but carries a webcam for the participant’s consumptions.•2nd robot actor can only move by crawling.
AAAI Spring ’05 Symposium: Dialogical Robots 12
Annotation and analysis
Data transcribed and annotated
Utterances classified into functional categories
394 utterances
20 utterance categories
8 major categories
Carnegie Mellon MockBrow annotation tool
AAAI Spring ’05 Symposium: Dialogical Robots 13
Utterance/Task Breakdown
Controlling team behaviors Grounding
Positive/negative feedback Informing robot of it’s state or
the world Explanations of commands Orientation Grounding
Navigation Simple Navigation commands Spatial Referential Navigation Object Referential Navigation
Manipulation Manipulating the environment Manipulating treasure
Coverage Manipulating the webcam view Object coverage commands Generic coverage
Asking about the robot’s abilities Filler Real-time command
modifications
AAAI Spring ’05 Symposium: Dialogical Robots 14
Summary / Future Work
• Architecture for human-robot teams
• WoZ study of language requirements
• Different WoZ scenarios
• Implement language for robot team
• “Field” testing
AAAI Spring ’05 Symposium: Dialogical Robots 15
Controlling team behaviors
“you guys get together” “T- you go first and B- follow”
AAAI Spring ’05 Symposium: Dialogical Robots 16
Grounding
Positive/negative feedback “ok that’s better”
Informing robot of state “so that’s up” “I don’t see anything there”
Explanations of commands “so I can see which direction is up”
Orientation Grounding “What you’re facing now with the camera – is that the
vehicle that you just circumnavigated” “I can tell you’re going in the wrong direction, stop”
AAAI Spring ’05 Symposium: Dialogical Robots 17
Navigation
Simple Navigation commands “so um T- turn to you left” “T- I want you to turn right 90 degrees” “can you go in that general direction” “can you proceed in that direction”
Spatial Referential Navigation “go to that open area” “continue around the periphery of that open area” “back out of that alley” “proceed in that direction until you find an opening to turn left”
Object Referential Navigation “go over by T-” “can you go on the other side of that vehicle” “go over by the posters”
AAAI Spring ’05 Symposium: Dialogical Robots 18
Manipulation
Manipulating the environment “T- why don’t you move the trash can”
Manipulating treasure “T- bring the coin to me”
Manipulating the webcam view “ok B- look to your left” “B- can you look around with the camera a
little”
AAAI Spring ’05 Symposium: Dialogical Robots 19
Coverage
Object coverage commands “ok so examine the shelf” “do you see something on that shelf in front of
B-” “can you look over by that table over there”
Generic Coverage “do you see anything that looks interesting”
AAAI Spring ’05 Symposium: Dialogical Robots 20
Asking about the robot’s abilities
“is that possible”