CALO Learning Overview
AIC Machine Learning Discussion Group
26 October 2004
with material shamelessly pilfered from previous presentations by:
• Tom Dietterich/Leslie Kaelbling (Transfer Learning)• Colin Evans (Task Setup)• Lynn Voss (Task Discussion)• David Martin (Task Fulfillment)
The Learning Picture (Terminology)
learningalgorithm
learneddevice
naïve Bayesmaximum entropyC4.5k-meanslearning by being told…
Bayesian networkHMMdecision treeinformation extractorprocedureclusterer…
predicted categoriesranked listsfacts/relationssocial networksclusters…
(test) instancecurrent statedocument corpusprior knowledge…
labeled training setannotated corpusexecution traces…
algorithminput
deviceinput
deviceoutput
meta-learningalgorithm
ML Today: “Engineered” Learning
data set
learningalgorithm
learneddevice
algorithminput
deviceinput
deviceoutput
human engineers features,invents algorithms, runs experimentsto find the best performance on(static) data sets
The Vision: Learning in the Wild
learningalgorithm
learneddevice
algorithminput
deviceinput
deviceoutput
system decides when to learn, what to learn, and how to learn, and adapts itself through interaction with the environment
ENVIRONMENT
The Vision Behind the Vision: Robust, Enduring Systems
CALO learns toperform Task A
CALOimmediately performs
Task B better
CALO learns toperform Task B
faster
transfer learning
Learned on Task BLearned on Task A
Example 1: Transfer of Learned Facts
Task A: Meeting Planning
Who should attend budget meeting for Project X?
Task B: Purchasing
Who can approve purchases on Project X?
Financial officers should attend budget meetings
Stephen Q. is financial officer for Project X
Financial officers can approve purchases
Stephen Q. should attend budget meeting
Stephen Q. can approve purchases
Transfer Learning, Tom Dietterich & Leslie Kaelbling
Example 2:Transfer of Learned Subprocedures
Task A: Purchasing Computers Task B: Purchasing Books
Tradeoff Specs, Price, Availability
ComputerMeetsSpecs
AvailabilityShipping Cost
Tradeoff Specs, Price, Availability
BookMeetsSpecs
AvailabilityShipping Cost
Computer Specs:• CPU speed • Memory size • Disk size
Book Specs:• Title • Author • Binding
Availability:• Discontinued • Back ordered • Delivery date
Availability:• Out of print • Back ordered • Delivery date
Transfer Learning, Tom Dietterich & Leslie Kaelbling
Example 3:Transfer of Learned Ontology
Task A: Tenure review in university Task B: Command and control in Air Force
Leader
Leader Leader
Leader LeaderLeader Leader
Member
Member
Member
Member
Member
Member
Member
Member
Member
Member
Member
Member
Organization is a hierarchy of groups
Each group has a team leader and team members
The members of all groups except the lowest are the team leaders of subgroups
Organization is a hierarchy of groups
Each group has a team leader and team members
The members of all groups except the lowest are the team leaders of subgroups
Tenure dossier flows up hierarchy Orders flow down hierarchy
Note: Domain facts and procedures do NOT transfer:
Leader
Leader Leader
Leader LeaderLeader Leader
Member
Member
Member
Member
Member
Member
Member
Member
Member
Member
Member
Member
Transfer Learning, Tom Dietterich & Leslie Kaelbling
Example 4:Transfer of Learned Feature Relevance
Task A: Routing Complaints Task B: Meeting Scheduling
Job title determines job responsibilities
Carpenter: framing, installing cabinetsDrywaller: taping, sealing, texturingPainter: masking, paintingContractor: scheduling, project planning
Job title determines job responsibilities
“Chief Evangelist” might be able to substitute for “Evangelist” in meeting
These inferences can be made without even knowing what “sealing” or
“Evangelist” mean
Transfer Learning, Tom Dietterich & Leslie Kaelbling
CALO Organization
• Technology Focus Centers (TFCs)• Reasoning & Action (RA)• Cyber Awareness (CA)• Physical Awareness (PA)• Multi-Modal Dialogue (MMD)• Learning (L)
• Scenarios (Year 1)• meeting scheduling• meeting understanding• laptop purchase
Functional Columns (Year 2)
• Task Setup• recognize implications of starting a new task
• information harvesting, scheduling setup, dossier preparation
• Task Discussion• integrate results of interaction between humans
and CALOs into task management• meeting understanding
• Task Fulfillment• support user in performing tasks
• scheduling, procurement
Task Setup: Information Harvesting
Task Setup, Colin Evans
Learning in Task Setup:Information Harvesting
Component Algorithm Input Device Output
SemEx desktop contents
extraction rules
facts about and relations between documents, email, etc.
DEX home pages
information extractor (CRF)
information about people, including contact information, areas of expertise, and social groups
Activity Clusterer
bi-directional clustering
emailbox activity (task) groupings of email
Task Setup: Scheduling Setup
Task Setup, Colin Evans
Learning in Task Setup:Scheduling Setup
Component Algorithm Input Device Output
SpeechAct email message
email type classifier
email type
Activity Classifier
email message
activity classifier activity of message
emailME email message
information extractor
meeting constraints
Task Manager advice policies for determining meeting constraints (e.g., start time constraints, participants)
Task Setup: Dossier Preparation
Task Setup, Colin Evans
Learning in Task Setup:Dossier Preparation
Component Algorithm Input Device Output
TaskTracer desktop operations
logging relations between tasks (activities) and desktop operations
Naive Bayes
desktop operation
task (activity) classifier
Activity Classifier
email message
activity (task) classifier
activity (task) of message
Task Discussion: Meeting Room
CAMEOCAMEO
Frame
Frame
Fram
e
User w/ headset
User w
/ he
adse
t
User w/ headset
SMART Board
Stereo Camera
Frame includes:•Stereo Camera•(IR - Blue Eyes Camera)•Array Microphones•All attached around a user’s laptop
Task Discussion, Lynn Voss
Task Discussion: Architecture
Meeting Recorder Architecture
Meeting Playback System
Meeting Record / MOKB
Body Tracker3D-Gesture
Face TrackerFace Recog.
Activity Recog.
Speakerlocalization
Affect Recog.
Head, eye,gaze tracker
Object Recog.Video & Array Microphone
Classifiers DialogueManager
Suite
FSDB
MS ProjectAgent
Task Setup
Purchase Request
Tracking DataIntegrator
Meeting DossierAgendaParticipant List
Handwriting2D Gesture
Charter
Digital InkRecognizers
NTP
OAAFacilitator
Raw Data Capture
Audio Server
InstrumentedText Notes &Power Point
CAMEOPanoramic
MPEG encoder
Whiteboard’sStereo
CameraFrame
SMARTBoard
Digital Ink
CloseTalkingSpeech
TasksMilestones
OOV Words
OOVAgentSuite
MS ProjectFile
End Pointer
Transcription
Prosody
TopicSupporting Docs
Meeting Room IRIS Data Store
Meeting Room
IRIS Data Store
UserFeedbackLoop
Multi-parser
MSBITS
Offline Analysis Suite
Agenda Topics
Phases Action Items
Roles Rough Sum.
Tracking DataIntegrator &Audio Server
MeetingBrowser
Task Discussion, Lynn Voss
Learning in Task Discussion:Project Plan Capture
Component Algorithm Input Device Output
Agent Sphinx
multi-modal learning
speech, gestures, writing
words (complete model)
instruction chart types
2D gesture recognizer
2D gesture recognizer
(written) symbols
speech recognizer
speech recognizer speaker ID
handwriting recognizer
handwriting recognizer
words
Learning in Task Discussion:Physical Awareness
Component Algorithm Input Device Output
CAMEO face recognizer person ID
coarse activity recognizer
sitting, standing, walking, …
BodyTracker articulated tracker participant movement
3D gesture recognizer
pointing, orientation
object recognizer objects
affect recognizer affect
Frame speaker localizer speaker location
speaker detection speaker ID
high-level activity recognizer
Learning in Task Discussion:Meeting Awareness
Component Algorithm Input Device Output
Meeting Analysis Suite
topic tracker topic shifts
agenda tracker agenda segments
action item identifier
action items
decision identifier decisions
meeting phase segmenter
meeting phases
role tracker participant role
acoustic model speech transcript
Task Discussion:Meeting Record Content
• Raw Streams• raw audio, raw video, whiteboard strokes, text notes, PPT
presentations
• Low Level Events• out-of-vocabulary words• participant locations with torso and body positions• participant activities (coarse)• who spoke to whom• recognized affects• recognized words & symbols on the whiteboard• word transcripts• new participants• new chart types• new 2D & 3D gestures
Task Discussion:Meeting Record Content
• High Level Events• project plan (task names, durations, milestones)
• participants, including entrance/exit
• when each agenda item was discussed
• topics/subtopics and relevance to agenda
• action items, including responsible parties, deadlines
• decisions and proposers; alternative proposals and reasons for/against
• participant roles (participator, observer, presenter)
• meeting phases (introductions, discussions, briefings, presentations)
Task Fulfillment: Scheduling
FormulateScheduling
Request(Task Setup)
Gather Information
Prepare Schedule Candidates
Get User Selectionsand/or Confirmations
Update Calendars,Send Notifications
Send Reminders
RelaxScheduling
Request
Task Fulfillment, David Martin
Learning in Task Fulfillment:Scheduling
Component Algorithm Input Device Output
PLIANT SVM schedule ranker value (cost)
Task Manager
advice policies for scheduling, relaxation, reminder
AutoMinder reinforcement learning
reminder strategy
procedural learner
revision procedure
memory-based learner
case-based learning
scheduling procedure
Task Fulfillment: Purchasing
Select Type of Item
LearnVendors
Choose Vendors &Define Requirements
Get Quotes
Get User Selectionsand/or Confirmation
Refine Purchase ProcedureWrap
VendorsAdd
Vendors
Execute Purchase Procedure
Select Type of Item
RelaxQuery
Task Fulfillment, David Martin
Learning in Task Fulfillment:Purchasing
Component Algorithm Input Device Output
KnowIt product vendor sites
product ontologies
Fetch learning by being told
Web site Web site wrapper product information
Prometheus Mediator
model of new source
LOQR C4.5 unsatisfiable query
decision rules relaxed query
Tailor learning by being told
revised procedure
Task Manager
advice policies for procurement
The CALO Test
Main Claim: CALO performs well and, through learning, performs even better.
• The Test• AP-style exam• Administered regularly throughout the year• Must show general improvement overall.• Only learning in the wild counts.
The CALO Test
CALO 2.0
CALO 2.1
CALO 2.2
CALO 2.3
CALO 3.0
Tes
t S
core
improvement due to learning
improvement due to engineering
totalimprovement
due toengineeringand learning
Situated Learning
CALO is a cognitive assistant.
• Task Manager (the heartbeat of CALO)• controls what CALO does• situation assessment• workflow management
• Knowledge Machine/Query-Update Manager• what CALO knows• CALO ontology
Situated Learning
CALO is deployed in the office environment.
• IRIS• suite of integrated desktop applications • ontology-driven architecture• provides instrumentation and automation facilities
Learning Issues
CALO is not (yet) a robust, enduring system.
• much in-the-wild learning is not truly online• concept drift/shift is not addressed• disparate sources are not coordinated• new tasks require human engineering• ontology changes require lobotomies• learning is component-specific