DARPA ITO/MARS Project UpdateVanderbilt University

A Software Architecture and Tools for Autonomous Robots

that Learn on MissionK. Kawamura, M. Wilkes, R. A. Peters II,

D. Gaines, N. Sarkar

Vanderbilt UniversityCenter for Intelligent Systems

http://shogun.vuse.vanderbilt.edu/CIS/IRL/23 May 2000

Vanderbilt MARS Team• Kaz Kawamura, Professor of Electrical & Computer Engineering.

MARS responsibility - PI, Integration

• Dan Gaines, Asst. Professor of Computer Science. MARS responsibility - Mission planning, learning

• Alan Peters, Assoc. Professor of Electrical Engineering. MARS responsibility - DataBase Associative Memory, Sensory EgoSphere

• Nilanjan Sarkar, Asst. Professor of Mechanical Engineering. MARS responsibility - Multi-Robot Control

• Mitch Wilkes, Assoc. Professor of Electrical Engineering. MARS responsibility - System Status Evaluation

• Jim Baumann, Nichols ResearchMARS responsibility - Technical Consultant

Sponsoring AgencyArmy Strategic Defense Command

Project Goal

Develop software control system for autonomous mobile robots that:

• accepts mission-level commands

• learns from experience to use / acquire behaviors

• can be trained with intuitive interface

• shares learned knowledge with other robots

Project Approach

SelfAgent

A

A

A A

AA

Atomic Agents

Sensory EgoSphere

DataBase Associative

Memory

SESManager

DBAMManager

CommanderAgent

Schedule

YEAR ONE 1 2 3 4 5 6 7 8 9 10 11 12

Requirement Analysis/Concept Development

IMA (A/C) Deployment for HelpMate

IMA (A/C) Deployment for ATRV-Jr

Robust System Status Analysis

Mission Planning / Learning Framework

Develop Egosphere and DBAM

Demo Scenario – Simple HR interaction – Mission Planning for ATRV-Jr

Project Organization

Mission Planning / Learning

Overview of ERA

• Based on Georgia Tech’s MissionLab, added:

– terrain conditions

– noise to robot actions

Overview of Mission Planner

Results 1: Finds Efficient Path

Results 2: Different Robot

Results 3: Transfer Knowledge

Results 4: Different Robot

The ISAC Humanoid

• Arms: two 6 DOF, pneumatic McKibben artificial muscles.

• Hands: anthropomorphic, hybrid pneumatic / electric.

• Head: independent pan, tilt, and verge.

• Sensors: vision, audition, joint position, touch, proximity, motion.

• Computation: PDP on Win NT 4.0 under IMA

Humanoid: Role in the MARS Project

• Learning Sensory Motor Coordination• Robot Attention• Object Recognition and Analysis• Human Robot Interaction• Robot Learning from People• Design of Control Programs and Data Structures

• Intelligent Machine Architecture (IMA)

• Sensory EgoSphere (SES)

• Attentional Networks (AN)

• Database Associative Memory (DBAM)

• System Status Evaluation (SSE)

• Intelligent Machine Architecture (IMA)

• Sensory EgoSphere (SES)

• Attentional Networks (AN)

• Database Associative Memory (DBAM)

• System Status Evaluation (SSE)

Year 1: Development of Key Technologies

System Architecture: High Level Agent Perspective

High level IMA agents are virtual — dynamic collections of atomic (low level) agents whose configuration depends on the context.

humanagent

selfagent

peeragent

peeragent

environmentagent

objectagent

objectagent

IMA Agents and Data Structuresfor Human - RobotInteraction

IMA Agents and Data Structuresfor Human - RobotInteraction

High level IMA agents: Human Agent & Self Agent

Low level IMA agents: Atomic Agents

Data Structures: Sensory EgoSphere (SES)

DataBase Associative Memory (DBAM)

High level IMA agents: Human Agent & Self Agent

Low level IMA agents: Atomic Agents

Data Structures: Sensory EgoSphere (SES)

DataBase Associative Memory (DBAM)

SelfAgent

HumanAgent

A

A

A A

AA

Atomic Agents

Sensory EgoSphere

DataBase AssociativeMemory

SESManager

DBAMManager

Sensory EgoSphere (SES)

A Short-Term Memory

• Albus: a dense instantaneous map of the environment

• Our approach: a sparse spatio-temporally indexed Short-Term Memory (STM)

• Structure: a variable density geodesic dome

• Nodes: links to data structures and files

• Indexed by azimuth, elevation, and time

• Searchable by location and content

• Nodes have numerical activation levels

• Albus: a dense instantaneous map of the environment

• Our approach: a sparse spatio-temporally indexed Short-Term Memory (STM)

• Structure: a variable density geodesic dome

• Nodes: links to data structures and files

• Indexed by azimuth, elevation, and time

• Searchable by location and content

• Nodes have numerical activation levels

Searching the Sensory EgoSphere

• Geodesic Dome

• Variable Density

• Location Search

• Content Search

• Geodesic Dome

• Variable Density

• Location Search

• Content Search

starting node

active node

goal nodebreadth first location search

Sensory EgoSphere

Current Status:

• SES Data structure complete

• Links to other data structures and files

• Searchable by location or content

• Time stamping

Current Status:

• SES Data structure complete

• Links to other data structures and files

• Searchable by location or content

• Time stamping

To do year 1:

• Motion transformations

• Attentional network

To do year 1:

• Motion transformations

• Attentional network

To do year 2:

• Coupling to DBAM

• Coupling to task context and affect

To do year 2:

• Coupling to DBAM

• Coupling to task context and affect

Visual Attention: FeatureGate

• A model of human visual attention, proposed by Kyle R. Cave.

• Activates locations in the visual field using salience and discrimination of features.

• A pyramid structure where info is gated to the next (smaller) level as a function of local activations.

Update Delay Histogram (Arm Agent)

0

50

100

150

200

1 9 17 25 33 41 49 57 65 73 81 89 97

Delay (10ms)

Fre

qu

en

cy

Update Delay Histogram (Arm Agent)

0

50

100

150

1 9 17 25 33 41 49 57 65 73 81 89 97

Delay (10ms)

Fre

qu

en

cy

Update Delay Histogram (Hand Agent)

0

500

1000

1500

1 10 19 28 37 46 55 64 73 82 91 100

Delay (10ms)

Fre

qu

en

cy

System Status EvaluationSystem Status Evaluation

• Uses inter agent timing

• Timing patterns are modeled.

• Deviations from normal bias affect

• Affect threshold triggers exception processing

• Visual Servoing: error vs. time

• Arm Agent: error vs. time, proximity to unstable points

• Camera Head Agent: 3D gaze point vs. time

• Tracking Agent: target location vs. time

• Vector Signals / Motion Links: log when data is updated

MeasurementsMeasurements

SelfAgent

HumanAgent

A

A

A A

AA

Atomic Agents

Sensory EgoSphere

DataBase AssociativeMemory

SESManager

DBAMManager

Database Associative MemoryA Long Term Memory (Proposed )

Database Associative MemoryA Long Term Memory (Proposed )

• Records: Competency Modules (CM) for a Spreading Activation Network (SAN).

• Associations: similarity between pre- and post conditions of CMs.

• Activation: CMs modulated by SES, Task Context, and Affect.

• Learning: motor state changes and the sensory events that consistently precede or follow them register on the SES to form CMs.

• Associations: formed during offline processing.

S1

S2

S3

Sn

max activation

activation link

competency module

...

Maximally activated SES node supplies task context and affect biasing.

Maximally activated SES node supplies task context and affect biasing.

Spreading Activation in the DBAMAction Selection:

activation pathways

• Sensing: constant, concurrent w/ attention

• SES: attentionally cued Short-Term Memory (STM) with sensitization and habituation

• DBAM: long-term memory (LTM) coupled to SES

• Learning: continual, via Hebbian and RL, of Sensory Motor Coordination (SMC)

• Association: offline in DBAM

• Affect: from LTM, sensory input, and system status; modulates action selection

System Architecture: Sensory Motor Control Perspective

Conclusion: Key Technologies

• IMA — the Intelligent Machine Architecture• SES — Sensory EgoSphere (for humanoid robot)• SSE — System Status Evaluation (via agent timeout)• Affect (scalar version controlled by SSE)

Now Implemented

To Implement:

• Sensory EgoSphere (for mobile robots)• Database Associative Memory• Learning SMC — Sensory Motor Coordination• Affect (vector version controlled by SSE, SES, DBAM)

Video Taped Demo: ISAC

• Human - Robot Interaction• System Status Evaluation• Sensory EgoSphere