1

(Embodied) Interface Agents(Embodied) Interface Agents

John StaskoJohn Stasko

Spring 2007Spring 2007

This material has been developed by Georgia Tech HCI faculty, and continues to evolve. Contributors include Gregory Abowd, Al Badre, Jim Foley, Elizabeth Mynatt, Jeff Pierce, Colin Potts, Chris Shaw, John Stasko, and Bruce Walker. Permission is granted to use with acknowledgement for non-profit purposes. Last revision: January 2007.

26750-Spr ‘07

AgendaAgenda

•• UI AgentsUI Agents–– IssuesIssues

–– ExamplesExamples

2

36750-Spr ‘07

Yet To Come…?Yet To Come…?

46750-Spr ‘07

Interaction ParadigmsInteraction Paradigms

•• Direct ManipulationDirect Manipulation–– User initiates actions and carries them out User initiates actions and carries them out

directlydirectly

•• Indirect ManagementIndirect Management–– Cooperative process where human and Cooperative process where human and

computer both initiate actionscomputer both initiate actions

3

56750-Spr ‘07

Agency & Norman’s GulfsAgency & Norman’s Gulfs

•• Direct ManipulationDirect Manipulation–– Narrow the gulfNarrow the gulf

–– User initiates actions and User initiates actions and carries them out directlycarries them out directly

•• Indirect ManagementIndirect Management–– Bridge the gulf with Bridge the gulf with

intermediaryintermediary

–– Cooperative process where Cooperative process where human and computer both human and computer both initiate actionsinitiate actions

User System

User System

User System

Agent

66750-Spr ‘07

Autonomous AgentAutonomous Agent

•• Personal assistant who collaborates with Personal assistant who collaborates with user to accomplish tasksuser to accomplish tasks–– Level of autonomy can varyLevel of autonomy can vary

–– Takes directionsTakes directions

–– Takes initiativeTakes initiative

–– May learn user’s preferencesMay learn user’s preferences

–– Human appearance?Human appearance?

4

76750-Spr ‘07

TerminologyTerminology

•• IVA IVA –– Intelligent Virtual Agent (Assistant)Intelligent Virtual Agent (Assistant)

•• ECA ECA –– Embodied Conversational AgentEmbodied Conversational Agent

•• ChatterbotChatterbot, , ChatbotChatbot

86750-Spr ‘07

ExamplesExamples

5

96750-Spr ‘07

AckAck! It’s the Paper Clip! It’s the Paper Clip

106750-Spr ‘07

ChallengesChallenges

•• Two challenges existTwo challenges exist–– CompetenceCompetence -- Does the agent have the Does the agent have the

requisite knowledge to truly assist the user?requisite knowledge to truly assist the user?

–– TrustTrust -- Does the user feel comfortable Does the user feel comfortable delegating task to agent?delegating task to agent?

6

116750-Spr ‘07

More help… More help…

126750-Spr ‘07

PossibilitiesPossibilities

•• What could agents do for us?What could agents do for us?

7

136750-Spr ‘07

Agency ApproachesAgency Approaches

•• 1. Application is semi1. Application is semi--autonomous agentautonomous agent–– User programs rules a priori for how agent User programs rules a priori for how agent

should performshould perform

•• 2. Knowledge2. Knowledge--basedbased–– Give the agent interface domain knowledge Give the agent interface domain knowledge

and user knowledgeand user knowledge

•• 3. Learning approach3. Learning approach–– Give agent minimal domain knowledge, then Give agent minimal domain knowledge, then

have it watch user and learn behaviors have it watch user and learn behaviors

146750-Spr ‘07

Learning ApproachLearning Approach

•• Like a personal assistant who gets better Like a personal assistant who gets better and betterand better

•• Learns byLearns by–– 1. Looking over shoulder, watching actions1. Looking over shoulder, watching actions

–– 2. Direct and indirect feedback2. Direct and indirect feedback

–– 3. Hypothetical examples3. Hypothetical examples

–– 4. Asking other agents for advice4. Asking other agents for advice

8

156750-Spr ‘07

ExamplesExamples

•• Email agentEmail agent–– Prioritize, delete, sort, …Prioritize, delete, sort, …

–– Looks at fields to make decisions (How weighted?)Looks at fields to make decisions (How weighted?)

–– Has “tellHas “tell--me” and “dome” and “do--it” thresholds for individual it” thresholds for individual actionsactions

–– Has facial expressions to communicate stateHas facial expressions to communicate state

•• Meeting schedulerMeeting scheduler–– Very personalized behaviorsVery personalized behaviors

166750-Spr ‘07

ExamplesExamples

•• News filteringNews filtering–– Watches what you read, then does filteringWatches what you read, then does filtering

–– Uses keywordsUses keywords

–– Needs deeper natural language helpNeeds deeper natural language help

9

176750-Spr ‘07

ExamplesExamples

•• Entertainment agentEntertainment agent–– Agent memorizes user’s preferencesAgent memorizes user’s preferences

–– Goes out and talks to other agents and looks Goes out and talks to other agents and looks for correlationsfor correlations

–– Makes recommendationsMakes recommendations

186750-Spr ‘07

VideosVideos

•• Vision: Apple’s Vision: Apple’s Knowledge NavigatorKnowledge Navigator–– Early ’90’sEarly ’90’s

•• Reality: MIT’s REAReality: MIT’s REA–– CHI ‘99CHI ‘99

10

196750-Spr ‘07

IssuesIssues

•• Should agents be made humanShould agents be made human--like?like?

•• If so, should they have personalities?If so, should they have personalities?

•• How can we guarantee privacy if agent How can we guarantee privacy if agent collaboration occurs?collaboration occurs?

•• Should someone be held responsible for Should someone be held responsible for what their agent does?what their agent does?

206750-Spr ‘07

ProjectProject

•• P3 feedbackP3 feedback

•• Will have reports for you to look atWill have reports for you to look at

•• Demo sign upDemo sign up

11

Would you like to participate in an Augmented Reality Research Study?

Will involve approximately 90 minutes of your time at $5/half-hour

If you are interested in participating in this study, please contact Cindy Robertson at

leistner@cc.gatech.edu.

• An investigation of ways of giving instructions using 3D graphics • Experience a see-

through head-worn display that blends 3D graphics with the real world

226750-Spr ‘07

InfoVisInfoVis HWHW

•• Pile on deskPile on desk

12

236750-Spr ‘07

UpcomingUpcoming

•• CSCWCSCW

•• Ubiquitous ComputingUbiquitous Computing

•• Project presentationsProject presentations