Metacognition for Effective Deliberation in Artificial Agents

Darsana Josyula18 November 2011

Bowie State University

Artificial Agents

Deliberation

– Process by which agents plan the tasks to perform in order to accomplish current goals

Action

– Process by which agents perform each task in the plan

Deliberation Time

Static Environments

– Deadlines Dynamic Environments

– Change in Environment

– Preconditions not in effect

– Plans not effective

Approaches to Managing Deliberation Time

Hybrid architectures with deliberative, reactive and action selection components

– Gerhard Lakemeyer (Golog for Robotic Soccer)

– Deliberation time decided by the action selection component

Anytime algorithms (estimates the efficiency of a solution as a factor of algorithm run time)

– Nikos Vlassis

– Deliberation time decided by the process that invokes the anytime algorithm

Approaches to Deliberation Time Management

Metacognition

– Awareness of one's own thoughts and the factors that influence one's thinking

– Based on estimates of allowable deliberation and action times

– Monitoring deliberation and action

– Making adjustments to deliberation and action processes

Metacognition in Deliberation and Action

Metacognition

Deliberation Action

Factors that influence Metacognition

Goals Emotions Resource Constraints Plans Performance Optimization Influence of other Agents

Goals

Set of goals to be achieved Choosing relevant goals

Type of goals

– Mandatory versus Desirable (Needs versus Wants)

– Desires versus Intentions (BDI agents)

– Desires -> Intentions -> Actions

– Maintenance versus Achievement

– Conflicting goals Source of goals

Progress towards goals

Metacognitive Monitoring and Control of Goal Processing

Marking the type of goals

Setting priorities for goals

Maintaining Expectations on progress towards goal

Metacognitive Monitoring and Control of Goal Processing

Expectation failures

– Possible anomalies to be evaluated

– Create new goals to deal with expectation failures

– NAG cycle

– MCL

– Indications, Failures and Resonses

Emotions

Animals that exhibit more emotional behavior tend to be better suited for survival

D. Keltner, “Darwin’s touch: Survival of the kindest,” Psychology Today, February 11 2009.

Emotion has been shown to alter the cognitive process in human beings allowing for different responses to the same problem based on different emotional states.

L. Berkowitz, Causes and Consequences of Feelings, Cambridge University Press, 2000.

Emotions - Russel's Circumplex model of Positive and Negative Affect

Affective states organized in a circular structure in a 2D plane

Emotions arise from cognitive interpretations of neural sensations that are the product of two independent neurophysiological systems

These neurophysiological systems correspond to the pleasure axis and activation axis in the circumplex model

Emotions - Russel's Circumplex model of Positive and Negative Affect

The circumplex model of affects is consistent with findings in cognitive neuroscience, neuroimaging, and developmental studies of affects.

The circumplex model has been used to study the development of affective disorders as well as the genetic and cognitive underpinnings of affective processing within the central nervous system.

Emotions – Pleasure Axis Transitions

Represent the agent’s feelings about its own performance

Correspond to the number of expectation violations that occur

When no expectation violation occurs, the agent is pleased with its performance and hence moves its state to the right on the pleasure axis

When expectation violations occur, the system is frustrated by its inability to quell the violations and hence moves to the left on the pleasure axis

The intensity of the expectation violation (the difference between the observed value and the expected value) decides how far to the left the system moves with respect to its current emotional state

Emotions - Activation Axis Transitions

The activation axis transitions are based on the observations of the system and represent the system’s feeling of stress

As the number of observables that the system has to deal with increases, the system becomes more stressed and hence its emotional state moves upward in the activation axis

As the number of observables decrease, the system can relax and hence its emotional state moves downward in the activation axis

Metacognitive Monitoring and Control of Emotions

Monitoring number of observables Maintaining Expectations on number of observables

Monitoring number of expectation violations

Metacognitive Monitoring and Control of Emotions

In the MCL model, failure nodes activate a set of possible responses and instantiates the highest utility response that corresponds to the type of failure.

Which action is deemed to be the best is a learned metric that could be altered for different emotional states; for instance, when stressed the best action may simply be the quickest, when relaxed the best action may be the slowest.

Plans/Actions to Perform

How a goal is achieved Is a plan to achieve the goal known ?

If a plan is unknown, agent has to create a plan

Are the pros and cons of the plans known or unknown? Which plan is better?

Success rate Costs Resource Usage

Conflicting plans

Metacognitive Monitoring and Control of Plans

Monitoring the success rate of plans adopted Maintaining expectations on rate of success for plans adopted

Monitoring the actual costs for adopted plans and resource usage Maintaining plan cost expectations

Monitoring the resource usage of adopted plans Maintaining resource usage expectations

Monitoring Contradictions Active Logic

Resource Constraints

Time Constraints Deadlines

Resource Constraints Minimize resource usage

Metacognitive Monitoring and Control of Resource Constraints

Monitoring passage of Time Maintaining expectations on time requirements

Active Logic

Monitoring usage of Resources Minimize resource usage

Influence of Other Agents

Competitive versus Cooperative

Cooperative agent changing to competitive or vice versa

Metacognitive Monitoring and Control of Influence of other Agents

Maintaining Expectations on the influence of other agents

Performance Optimization

May adversely influence resource constraints

Very important in competitive settings, but may be important in single agent settings as well

Examples

Get better in achieving a goal Win against an opponent Collect more rewards even at the expense of

spending more resources

Metacognitive Monitoring and Control of Performance Optimization

Monitoring Performance Metrics

Maintaining Expectations of Performance Metrics

Metacognition – Vedantic Underpinnings

Goals

– (Kāma – desire / goal to be achieved) Emotions

– (Krōdha – anger / emotional state) Resource Constraints

– (Lōbha – greed / miserliness) Plans

– (Mōha – delusion / Not seeing pros and cons) Performance Optimization

– (Mada – pride / vanity) Influence of Other Agents

– (Mātsarya – competitiveness)

Conclusion

Monitoring Expectations is the key?