Post on 03-Aug-2015
transcript
FORR A Cognitive Architecture for Expertise
Susan L. Epstein
The Graduate Center and Hunter College of The City University of New York
Executive summary
• FORR (FOr the Right Reasons) is an architecture • FORR-based systems develop expertise • FORR-based systems learn quickly from problem solving experience • FORR-based systems are built from
§ World knowledge (descriptives) § Good reasons for making decisions (Advisors)
• FORR-based systems can restructure their decision process • FORR confirmed cognitively plausible on human subjects
2 Background • FORR • Applications
People, agents and expertise
• People are our best model of intelligent agents § Some human approaches work well on really hard problems § Their methods are robust to imperfect data § They pursue multiple goals
• If an agent is to collaborate with people, it is necessary to understand human decision processes
• A cognitively plausible agent simulates significant human characteristics • Expert does things faster and better than the rest of us [D’Andrade 1990]
3 Background • FORR • Applications
Characteristics of human experts • They work in a domain (set of related problem classes) • They satisfice = make good enough decisions • They entertain multiple decision-making heuristics [Ratterman
& Epstein 1995] • They access multiple representations • They do situation-based reasoning [Klein & Calderwood 1991]
• Human experts are made, not born
4
Learning is the hallmark of human intelligence
Background • FORR • Applications
Agent architecture • Postulates general principles • System shell for diverse domains
• Requirements for cognitive plausibility § Display reasonable behavior
• Make obvious decisions • Avoid obvious errors • Solve easy problems quickly
§ Balance accuracy and speed § Be robust to error § Tolerate and reason with inconsistent, incomplete, noisy data § Learn
5 Background • FORR • Applications
Do forever Sense the world Select an action Execute that action
Fundamental issues for a learning architecture
• What is there to learn? • From whom to learn? • When to learn? • How to learn? • How to use learned knowledge to make decisions? • How to manage reality and noise?
6 Background • FORR • Applications
Cornerstones of FORR’s pragmatic approach
• Expertise is learned, that is, it develops with experience • Easy questions should have fast (reactive) answers • Satisfice = make good enough decisions in a simplified model of a complex
world (and recover if need be) • Exploit synergy inherent in multiplicity
§ Multiple domain-dependent representations § Multiple domain-dependent heuristics for decision making § Multiple learning methods
• Maintain flexibility § Decouple data, learning methods, and decision methods § Restructure its own decision-making process
• Transparency: explain decisions
FORR's building blocks are descriptives and Advisors
7 Background • FORR • Applications
Multiple representations
• Descriptive = a shared data object § Value provided on demand § Defined with functions that determine how and when to update it § Value may be learned
• Although a descriptive has a single representation, many descriptives can represent the same world state
• Examples: X-O-blank empty/occupied lines on the board
8 Background • FORR • Applications
Multiple ways to use knowledge
• Operationalization = how to use a data object • Although a descriptive has a single representation, it can be
operationalized in many ways Ways to reason about the empty/occupied squares
Calculate possible actions Predict opponent's move
Ways to reason about the lines Report a result Finish a winning line Block your opponent’s winning line Create a fork Plan a win on a specific line
9 Background • FORR • Applications
An Advisor operationalizes descriptives • Implements a class-independent, action-selection rationale • Limitedly-rational (resource-limited) procedure • Input: state of the world + descriptives + possible actions • Output: comments whose strengths express intensity of support or
opposition to individual actions (or sets of actions) • Domain-specific
< Advisor, action, strength>
Advisor
current state possible actions relevant descriptives
10 Background • FORR • Applications
Often, Advisors disagree
11
O X X O Panic
(prevent immediate loss)
Worried (prevent long-range loss)
Victory (win!)
And rely on learned descriptives • Good openings • Endgame play • Strategies
…
Background • FORR • Applications
More about Advisors
• Advisors have different properties § Some are always right § Some need more time to decide § Some would like to make a sequence of decisions, not just one
• Comments are opinions from the perspective of the Advisor's rationale § On a single action
do x x is better than y don’t do z do x or y x is a 10, y is an 8, but z is a –3
§ On an (unordered or fully or partially ordered) set of actions do x and y do p and then q do p and then do q and r
12 Background • FORR • Applications
FORR (FOr the Right Reasons) • Premise: synergy among domain-specific rationales solves problems • Descriptives isolate representation from reasoning • Advisor hierarchy
§ Tier 1: correct, quick, pre-sequenced § Tier 2: reactive plan rationales § Tier 3: voting among heuristics based on their comment strengths and learned weights
13
<AdvisorA, action2, 10> <AdvisorA, action4, 8> <AdvisorA, action7, 6> <AdvisorB, action2, 7> <AdvisorB, action3, 9> <AdvisorC, action1, 9> <AdvisorC, action2, 7> <AdvisorC, action3, 9> <AdvisorC, action7, 9> …
Voting
For Advisor i and action j
argmaxj
diwicij∑
Background • FORR • Applications
The FORR decision cycle
14
take action yes
Tier 1: Reaction from perfect knowledge
Victory T-11 T-1n …
Decision? no
Background • FORR • Applications
state actions descriptives
The FORR decision cycle
15
take action yes
Tier 1: Reaction from perfect knowledge
Victory T-11 T-1n …
Decision?
begin plan yes
Tier 2: Plans triggered by situation recognition
no
T-21 T-22 T-2m …
Decision?
Background • FORR • Applications
state actions descriptives
The FORR decision cycle
16
take action yes
Tier 1: Reaction from perfect knowledge
Victory T-11 T-1n …
Decision?
begin plan yes no
T-32 T-31 T-3k … … Tier 3: Heuristic reactions
Voting take action
Tier 2: Plans triggered by situation recognition
no
T-21 T-22 T-2m …
Decision?
Background • FORR • Applications
state actions descriptives
How to develop a problem solver
17
• Specialize FORR with domain knowledge § Problem classes § Advisors § Descriptives with learning methods
• To solve a class of problems robustly, FORR learns § Descriptives’ values § Rationales’ relative utility § New Advisors § How to reorganize tier 3
Domain knowledge
FORR
FORR-based problem solver
Learned problem solver
Problem class
Experience
WARNING: problem solving often provides noisy data
Background • FORR • Applications
FORR-based single agents
18
• Hoyle learned to play 19 two-person, perfect-information, finite-board games as well or better than human / machine expert [Epstein, 2001]
• Ariadne learned to navigate efficiently in grid worlds, despite perceptual limitations and no map [Epstein, 1995]
• ACE learned to solve constraint satisfaction problems and rediscovered the Brélaz heuristic [ Epstein & Freuder, 2005]
• SemaFORR: controls an autonomous search-and-rescue robot [Epstein, Schneider, Ozgelen, Munoz, Costantino, Sklar & Parsons, 2012]
Background • FORR • Applications
Lessons learned
19
• Reactive plans work well • Elimination of inaccurate heuristics produces substantial speedup • Lazy descriptive computation also provides speedup • Self-awareness supports transparency • Advisor weights may have problem-stage context • Weight learning has subtle pitfalls (example extraction) • Autonomous restructuring must balance accuracy against risk • Sometimes it is more efficient not to reason at all
Background • FORR • Applications
FORR-based collaborating agents
20
• Co-FORR: 5 collaborating agents for 2D park design [Epstein, 1998] • FORRSooth: learned to conduct a spoken dialogue with a library patron
who orders books [Epstein, Passonneau, Gordon, & Ligorio, 2012] • SemaFORR: controls autonomous search-and-rescue robot team [Epstein,
Aroor, Evanusa, Sklar & Parsons, 2015]
Each new domain poses new challenges whose solution strengthens FORR
Background • FORR • Applications
FORR-based results
21
• PhD theses § Shih on learning multiple behavior sequences, 2000 § Lock on learning multiple plans from behavior sequences, 2003 § Petrovic on weight learning for multiple Advisors, 2008 § Ligorio on learning to select attributes, 2011 § Li on representation and exploitation of multiple complex relationships, 2011 § Yun on parallelization of multiple solvers, 2013 § Osisek on application of multiple relationships in recommendation (in progress) § Aroor on reactive planning for multiple robots (in progress)
• Applications to bioinformatics (with Dr. Lei Xie) § Protein-protein interaction networks § Virtual drug screening
Background • FORR • Applications
Take home message
To develop expertise
FORR learns to harness the synergy of
multiplicity in representation and reasoning
22 Background • FORR • Applications
Acknowledgements
We gratefully acknowledge the support of The National Science Foundation CUNY’s High Performance Computing Center
Continued thanks to my collaborators
Gene Freuder Rebecca Passonneau Rick Wallace Lei Xie Elizabeth Sklar Simon Parsons
and a host of undergraduate and graduate students with whom I continue to learn
23
Selected references
• Epstein, S. L. 2001. Learning to Play Expertly: A Tutorial on Hoyle. In Machine Learning in Game Playing
• Epstein, S. L. 1998. Pragmatic Navigation: Reactivity, Heuristics, and Search. Artificial Intelligence, 100 (1-2): 275-322.
• Epstein, S. L., E. C. Freuder and M. Wallace 2005. Learning to Support Constraint Programmers. Computational Intelligence 21(4): 337-371.
• Epstein, S. L., R. J. Passonneau, T. Ligorio and J. Gordon. 2012. Data Mining to Support Human-Machine Dialogue for Autonomous Agents. In Proceedings of Agents and Data Mining Interaction (ADMI2011).
• Epstein, S.L., Aroor, A., Evanusa, M., Sklar, E.I., Simon, S. 2015. Navigation with Learned Spatial Affordances. In Proceedings of CogSci 2015.
http://www.cs.hunter.cuny.edu/~epstein/
24