Human Directability of Agents

Karen Myers, David Morley

{myers, morley}@ai.sri.com

AI CenterSRI International

True Confessions

Why am I here?1. Directing Agents: learning by being told

2. Critical need for learning technology to develop real-world agent applications

*** I am not a Machine Learning Person ***

3/20/2004K.L. Myers SRI International

AAVs

Smart CockpitSmart Home/Office Robot Teams

Networks

SpacecraftSoftBots

Agents Everywhere!

Current Practice

Objective: mixed-initiative directability of agents by a human supervisor

Delegation without loss of control

Fully AutonomousAgent makes all decisions

Ex: mobile robots

TeleoperationHuman makes all decisionsEx: internet agents, UCAVs

Acts according to human preferences Little knowledge modeling neededX Human bears cognitive load

Little human influence XMust encode all expertise X

Low human cognitive load

Interaction Spectrum

Supervised Autonomy

Scope of applicability Agents capable of fully autonomous operation Want agents to be mostly autonomous Human influence would improve performance Humans want to customize agent operations

Approach Dynamic guidance for management of agents

Strategy Preference Adjustable Autonomy

Disaster Relief Intel Management

TRAC

Supervisor

controlled uncontrolled

CoordinatorAgent

Truck Agents Heli AgentsCommsAgent

MAPLESIM

BDI Agent Model (a la PRS)

Executor

Plan Library Tasks

IntentionsBeliefs

User

World

Strategy Preference

Strategy: how to make decisions

Assumption: agents have library of parameterized plans

Approach: guidance defines policies on plan selection, parameter instantiation

Example

Only use helicopters for survey tasks in sectors more than 200 miles from base.

Adjustable Autonomy

Autonomy: degree to which agent makes its own decisions

Assumption: agents capable of full autonomyApproach: guidance restricts space of agent

decisions

Permission Requirements gating conditions on actions

Obtain permission before abandoning survey tasks with Priority>3

Consultation Requirements deferred choice

Consult me when selecting locations for evacuation sites.

Guidance Foundations

1. Language for expressing guidance Belief-Desire-Intention (BDI) Model of Agency FOL Domain Metatheory

2. Formal Semantics Guidance-compatible execution

3. Enforcement Methods Operationalization within BDI interpreter loop

Domain Metatheory

Base-level Agent Theory Individuals Relations modeling the world, internal agent state Tasks Plans

Domain Metatheory Captures high-level, distinguishing attributes of plans,

tasks Features, Roles

Example Domain Metatheory

Feature - distinguishing attribute of a plan/task Plans for Task: MOVE(Obj1 Place1 Place2)

Move-by-Land-Opr: LAND Move-by-Sea-Opr: SEA Move-by-Air-Opr: AIR

Role - capacity in which a variable is used Origin: Place.1, Destination: Place.2

Key Idea: abstraction over individual plans, tasks

Guidance Components

Use domain metatheory to define abstract classes of plans, goals, and agent state Activity specification Desire specification Agent context

Activity Specification

Abstract characterization of a class of activities Defined in terms of:

Features required/prohibited Constraints on role values

Example: Abandon a survey task

Features: Abandon

Roles: Current-Task

Role Constraints: (= (TASK-TYPE Current-Task) SURVEY)

Desire Specification

Abstract characterization of a class of desires Defined/used similarly to Activity Specification

Agent Context

Describes an operational state of agent

BDI Construct Agent Context Equivalent

Beliefs conditions that must be believed true

Desires desire specifications for tasks

Intentions activity specification for intended plans

Example: Performing a communication plan for a Survey task within 10 miles of the Base

Beliefs: (< (Distance (Current-Position) Base) 10) Desires: Features: Survey

Intentions: Features: Communication

Permission Requirement

Definition <agent-context, activity-specification>

Semantics when in the context, permission is required to adopt plans that match the activity specification

Ex: Seek permission to abandon survey tasks with priority > 5

Agent Context:

Intentions: Feature: SURVEY-TASK

Activity-Spec:

Features: ABANDON

Roles: Current-Task

Role Constraints: (> (Task-Priority Current-Task) 5)

Consultation Requirement

Definition <agent-context, role>Semantics when in the context, consult the

supervisor when there are options for the designated role

Ex: When responding to medical emergencies, consult when selecting MedEvac facilities.

Agent Context:

Intention:

Features: Medical-Emergency, Response

Role: MedEvac-Facility

Strategy Preference

Definition <agent-context, activity-specification>

Semantics when in the context, plans matching activity specification should be preferred

Ex: Respond to rescue emergencies involving more than 10 people when the severity exceeds the current task priority.

Agent Context:Features: Emergency, ResponseRoles: Current-Task, Severity, NumberRole Constraints: (AND (> Number 10) (> Severity (TASK-PRIORITY Current-Task)))

Activity Specification:Features: ADOPTRoles: New-TaskConstraints: (= (TASK-PRIORITY New-Task) ESEVERITY.1)

GuidanceInterface

Tools

Guidance Enforcement

P5

P1 P3

P2P4

Good

Bad

Filter-based Semantics

Simple Semantics: guidance as filters on applicable plans

Enforcement:• Simple extension to BDI executor• Modify plan selection step to incorporate

– Filtering of plans with respect to guidance constraints– User consultation

Guidance Conflicts (1)A. Plan Selection: guidance yields contradictory

suggestions– Execute Plan P / Don’t execute Plan P

P5

P1 P3

P2P4

Good

Bad

P5

P1

P3

P2

P4

Ranking

Filter-based Semantics Prioritized Semantics

Solution– Rank applicable plans according to guidance satisfaction– Select higher-ranked plan(s) when there is a conflict

Guidance Conflicts (2)B. Situated Conflict: prior activities block guidance

application– Guidance would recommend a response to an emergency

but required resources are unavailable

P5

P1 P3

P2P4

Good

Bad

P5

P1

P3

P2

P4

Ranking

P6

P7

P8

Filter-based Semantics Prioritized Expansion Semantics

Solution– Expand the set of candidate plans proactively

Resolution Plans: Delay current task to obtain required resource

Related Work

Deontic logics Obligation, permission, authority modalities Mostly formal rather than practical

Policy-based systems management Incorporating deontic concepts for runtime definition of

behaviors Sets authority parameters for components

Adjustable Autonomy Electric-Elves: MDP based approach for consultation

Summary

Technical Contributions: Language, semantics, enforcement techniques for

agent guidance Form of ‘learning by being told’ --- limited to control

rather than core knowledge Benefits:

Combines capabilities of humans and agents Adapts to dynamic user preferences Reduced knowledge modeling effort

Status: TRAC implementation on top of PRS; reimplementation

in SPARK

CALO: Cognitive Assistant the Learns and Organizes

Develop an intelligent personal assistant for a high-level knowledge worker

Large project encompassing ~20 different research organizations in the US; led by SRI

“Integrated Learning” as a key theme

EPCA Reasoning & Action TFC

t

CALO Task Manager

NoticePlan

Anticipate

Now

t

Interact

Timeline

IntrospectTask Task

ManagerManager

Capabilities: Perform tasks on behalf of the user (reactively, proactively) Manage user commitments (time, workload) Keep the user informed Coordinate interactions with other CALOs

Act

The Need for Integrated Learning

Capabilities User customization Extending/modifying procedural knowledge Performance improvement

Setting Learning unobtrusively Learning from small number of cases (for some things) Mixed-initiative setting

Learning in the Task Manager (Current)

1. Learning by Being Told Human Guidance for Agents (Myers, Morley) Interactive Acquisition/Modification of Procedures (Blythe)

2. Preference Learning for Email Management (Gervasio) folder and priority prediction

3. Preference Learning for Calendar Management (Gervasio) Schedule evaluation functions

4. Reinforcement Learning for Reminder Customization (Pollack)

5. Query Relaxation via online data mining (Muslea) mine small subset of solution space for rules that relate domain

attributes; use the rules to relax query constraints

Learning Procedural Knowledge

1. Programming by demonstration Calendar Manager: how to arrange meetings of different types Observe sequence of actions from meeting initiation to actual

meeting

2. Failure-driven learning procedure adaptation (automated, mixed-initiative)

Adapt/extend predefined core of procedures to handle a broader set of tasks, improve robustness

User & Agent explore high-dimensional traces of failed tasks