Human Robot Interaction in Guardians

Amir M. Naghsh Sheffield Hallam University, UKJeremi Gancet Space Applications Service N.V., BelgiumAndry Tanoto Heinz Nixdorf Institute, Paderborn, GermanyJacques Penders Sheffield Hallam University, UKChris R. Roast Sheffield Hallam University, UKMichel Ilzkovitz Space Applications Service N.V., Belgium

RISE 2008 Worksop, Benicassimwww.guardians-project.eu

Outline

1. Introduction and context: Guardians environment2. On-the-field HRI3. Remote HRI4. Communication issues5. HRI metrics6. Conclusion and future work

www.guardians-project.eu

1. Guardians’ Environment / Fire-fighters operations

• Environmental Conditions– No Visibility– Reduced sense of hearing– Restricted sense of touch

1. Guardians’ Environment / Fire-fighters operations

• Fire-Fighters Operations– Fire-fighters safety– Teams of 2 (3,4)– Progress along a wall when it is possible– Frequently debrief the Entry Control Officer

(ECO)– Span of control (Overall structure) ADD DIAGRAM ???

2. HSI Requirements• The swarm notifies the humans of possible hazards (e.g.

obstacles, high temperature, chemicals)

• The swarm stays within a relatively close range but also maintains its distance to the fire fighters to allow them freedom of action

• The swarm indicates unambiguously the direction to the scene of incident or backwards to the exit point.

• The interfaces indicating hazards should be most noticeable for the fire fighters.

2. Swarm Human Interactions

• Visual– Visual presentation

using LED / LCD screens– Providing Navigational

information / Hazards

• Audio– Dialogue based

interactions / Speech recognitions

– Not recommended

2. Swarm Human Interactions

• Tactile– Using sense of touch for

presenting information– Hazards – Obstacles /

Navigation information

Vibrators

2. Human Swarm Interactions• Passive

– Autonomous – form a bubble surrounding fire fighters – maintain a min/max distance

• Tangible– Big buttons – high level tasks

• Movement-based– Gesture recognition

• Audio / Visual– Not recommended

VPL Research, 1987 MIT, 2007

3. Remote HRI: the base stationBase station’s purpose

– Monitoring and control of the robots and humans on the field– Offline and online support to mission planning, coordination,

scheduling and execution– Data recording and replay with support to debriefing

Users roles– Different types of users, with different roles

1. Guardians’ appliance coordinator2. Operator(s)3. Sensor data specialist(s)4. Stakeholders in the commanding chain5. Tier information channels (media...)

www.guardians-project.eu

3. Remote HRI: the base station

Base station components

www.guardians-project.eu

MTE and MPSEM

Base Station

CoreMDRDHMI Clients

Base Station

SDPUsers Interfaces to

VF robots, HCM, and their sensors

3. Remote HRI: the base station

Base station components

www.guardians-project.eu

MTE and MPSEM

Base Station

CoreMDRDHMI Clients

Base Station

SDP

=> HMI Servers=> Base station administration / supervision

3. Remote HRI: the base station

Base station components

www.guardians-project.eu

MTE and MPSEM

Base Station

CoreMDRDHMI Clients

Base Station

SDP

=> Mission edition support tools=> Planning and scheduling=> Execution monitoring

3. Remote HRI: the base station

Base station components

www.guardians-project.eu

MTE and MPSEM

Base Station

CoreMDRDHMI Clients

Base Station

SDP

=> Mission data recording / replay=> Mission data dispatching

3. Remote HRI: the base station

Base station components

www.guardians-project.eu

MTE and MPSEM

Base Station

CoreMDRDHMI Clients

Base Station

SDP

Sensor Data Processing=> Ground based (cpu-consuming) data processing=> Data fusion and map building=> Robot localization

3. Remote HRI: interfaces

www.guardians-project.eu

Nominal baseline Convenience / backup Restricted to well-scoped purpose

• Advanced visualization, extension of “ecological display” approach

• Joystick

• Touchscreen

• HHI: sound-based

• Mouse

• Keyboard

• Immersive VR

• HRI: sound-based

Foreseen interfaces & approach to remote HRI:

•ADD ILLUSTRATIONS HERE ???

4. Communication Issue

R

H

Incident Area

Base Station

R

R

R

Short range Communication

Middle range Communication

Issues:• Interferences due to warehouse

infrastructures• No communication infrastructure is

available at the incident• Communication protocol• Communication bandwidth

4. Wireless CommunicationThree wireless communication standards:• WLAN (802.11)

– High datarate, high power consumption, point to point connection– For high-rate data transfer: video, audio

• Bluetooth– Middle datarate, low power consumption, point to multi-point connection– For continuous, middle-rate data transfer: audio, sensors

• ZigBee– Low datarate, low power consumption, fast network formation time, high

number of nodes– For time-critical low-rate data transfer: critical data (commands, sensor)

Which, When, and Where?

5. Validation & EvaluationWhat to measure?• Situational awareness

– To understand the extreme condition of the environment and robots• Cognitive/Work load

– To ensure load below humans’ capability• Team performance

– The outcome of teamwork between robots and humans

How to measure?• Interaction debugger [Kooijmans & Al, 2006]

– to collect, match, visualize, and analyze• SAGAT

– to measure situational awareness• Wizard of Oz

www.guardians-project.eu

6. Conclusion and future work• HSI is a novel research area• We presented:

– issues on human-robot swarm interaction for search and rescue mission– System architecture to support human-robot swarm interaction

• We proposed– human-robot swarm interface– Metrics for validation and evaluation

• Future work to provide conceptual design of human-robot swarm interaction

• Interface prototyping and testing is ongoing with participation of end-users

www.guardians-project.eu

Thank you for your attention!Thank you for your attention!