Embedding Artificial Intelligencein Agent-Based Models
Guillem Frances1 Xavier Rubio-Campillo2 Carla Lancelotti1
1Universitat Pompeu Fabra 2Barcelona Supercomputing Centre
SAA, San Francisco - April 16 2015
Introduction Motivation
IntroductionAgent-Based Modeling in the Social Sciences
• Agent-Based Modeling (ABM) is becoming a popular paradigm in thesocial sciences — used more and more to gain insight into theemergent properties of complex systems involving heterogeneousagents.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 2 of 15
Introduction Motivation
IntroductionAgent Behavior in ABM Simulations
• ABM agents in social sciences and archaeology are usually endowedwith a simplistic, reactive behavior — and for good reasons:
• They are simple to understand.• They are simple to implement.• They are able to produce rich behaviors nonetheless.
• However, in between a purely reactive agent and a hyperrational,deliberative agent, there is a continuum of behavioral choices withvarying degrees of cognitive (strategic) complexity.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 3 of 15
Introduction Motivation
IntroductionAgent Behavior in ABM Simulations
• ABM agents in social sciences and archaeology are usually endowedwith a simplistic, reactive behavior — and for good reasons:• They are simple to understand.• They are simple to implement.• They are able to produce rich behaviors nonetheless.
• However, in between a purely reactive agent and a hyperrational,deliberative agent, there is a continuum of behavioral choices withvarying degrees of cognitive (strategic) complexity.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 3 of 15
Introduction Motivation
IntroductionAgent Behavior in ABM Simulations
• ABM agents in social sciences and archaeology are usually endowedwith a simplistic, reactive behavior — and for good reasons:• They are simple to understand.• They are simple to implement.• They are able to produce rich behaviors nonetheless.
• However, in between a purely reactive agent and a hyperrational,deliberative agent, there is a continuum of behavioral choices withvarying degrees of cognitive (strategic) complexity.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 3 of 15
Introduction Motivation
IntroductionAgent Behavior in ABM Simulations
• It is important to acknowledge that the degree of cognitive complexitywe endow our agents with has a strong impact on the emergentproperties (e.g. carrying capacity) of the simulated system.
• In consequence, it is essential to
• Conceptualize agent behavior as a relevant parameter of the simulation.• Derive methodological guidelines to select the degree of strategic
complexity that is adequate for the purpose of our simulation.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 4 of 15
Introduction Motivation
IntroductionAgent Behavior in ABM Simulations
• It is important to acknowledge that the degree of cognitive complexitywe endow our agents with has a strong impact on the emergentproperties (e.g. carrying capacity) of the simulated system.
• In consequence, it is essential to
• Conceptualize agent behavior as a relevant parameter of the simulation.• Derive methodological guidelines to select the degree of strategic
complexity that is adequate for the purpose of our simulation.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 4 of 15
Introduction Motivation
AI and Agent-Based ModelsWhat can AI offer to ABM Simulations?
• The issue of agency and intelligent behavior has been a coreconcern of Artificial Intelligence and several of its sub-disciplines(automated planning, multi-agent systems) since the dawn of thediscipline in the late 1950s.
• Planning focuses on selecting which action to perform next given asuitable specification of the goals of the agent and the state of theworld.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 5 of 15
Introduction Motivation
AI and Agent-Based ModelsWhat can AI offer to ABM Simulations?
• The issue of agency and intelligent behavior has been a coreconcern of Artificial Intelligence and several of its sub-disciplines(automated planning, multi-agent systems) since the dawn of thediscipline in the late 1950s.
• Planning focuses on selecting which action to perform next given asuitable specification of the goals of the agent and the state of theworld.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 5 of 15
Introduction Motivation
AI and Agent-Based ModelsWhat can AI offer to ABM Simulations?
• Certain properties of agent behavior which would be desirable from anArtificial Intelligence point of view:
• A principled and general approach.
• Use crisp and clear decision-making models whenever possible.• Adaptive Behavior ←→ Generalized Planning• Prevent excessive overfitting to the particular simulation models.
• Cognitive plausibility: prefer a model with empirical supportwhenever possible.
• Especially when agents are supposed to model human beings.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 6 of 15
Introduction Motivation
AI and Agent-Based ModelsWhat can AI offer to ABM Simulations?
• Certain properties of agent behavior which would be desirable from anArtificial Intelligence point of view:
• A principled and general approach.
• Use crisp and clear decision-making models whenever possible.• Adaptive Behavior ←→ Generalized Planning• Prevent excessive overfitting to the particular simulation models.
• Cognitive plausibility: prefer a model with empirical supportwhenever possible.
• Especially when agents are supposed to model human beings.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 6 of 15
Introduction Motivation
AI and Agent-Based ModelsWhat can AI offer to ABM Simulations?
• Certain properties of agent behavior which would be desirable from anArtificial Intelligence point of view:
• A principled and general approach.
• Use crisp and clear decision-making models whenever possible.
• Adaptive Behavior ←→ Generalized Planning• Prevent excessive overfitting to the particular simulation models.
• Cognitive plausibility: prefer a model with empirical supportwhenever possible.
• Especially when agents are supposed to model human beings.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 6 of 15
Introduction Motivation
AI and Agent-Based ModelsWhat can AI offer to ABM Simulations?
• Certain properties of agent behavior which would be desirable from anArtificial Intelligence point of view:
• A principled and general approach.
• Use crisp and clear decision-making models whenever possible.• Adaptive Behavior ←→ Generalized Planning
• Prevent excessive overfitting to the particular simulation models.
• Cognitive plausibility: prefer a model with empirical supportwhenever possible.
• Especially when agents are supposed to model human beings.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 6 of 15
Introduction Motivation
AI and Agent-Based ModelsWhat can AI offer to ABM Simulations?
• Certain properties of agent behavior which would be desirable from anArtificial Intelligence point of view:
• A principled and general approach.
• Use crisp and clear decision-making models whenever possible.• Adaptive Behavior ←→ Generalized Planning• Prevent excessive overfitting to the particular simulation models.
• Cognitive plausibility: prefer a model with empirical supportwhenever possible.
• Especially when agents are supposed to model human beings.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 6 of 15
Introduction Motivation
AI and Agent-Based ModelsWhat can AI offer to ABM Simulations?
• Certain properties of agent behavior which would be desirable from anArtificial Intelligence point of view:
• A principled and general approach.
• Use crisp and clear decision-making models whenever possible.• Adaptive Behavior ←→ Generalized Planning• Prevent excessive overfitting to the particular simulation models.
• Cognitive plausibility: prefer a model with empirical supportwhenever possible.
• Especially when agents are supposed to model human beings.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 6 of 15
Introduction Motivation
AI and Agent-Based ModelsWhat can AI offer to ABM Simulations?
• Certain properties of agent behavior which would be desirable from anArtificial Intelligence point of view:
• A principled and general approach.
• Use crisp and clear decision-making models whenever possible.• Adaptive Behavior ←→ Generalized Planning• Prevent excessive overfitting to the particular simulation models.
• Cognitive plausibility: prefer a model with empirical supportwhenever possible.
• Especially when agents are supposed to model human beings.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 6 of 15
Empirical Evaluation
Empirical Evaluation
• We want to empirically explore the feasibility and effectiveness ofcognitively more sophisticated agents that use Artificial Intelligencetechniques to guide their decisions.
• We have designed and run a generic simulation model and comparedthe performance of different types of agent behaviors on it.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 7 of 15
Empirical Evaluation
Empirical Evaluation
• We want to empirically explore the feasibility and effectiveness ofcognitively more sophisticated agents that use Artificial Intelligencetechniques to guide their decisions.
• We have designed and run a generic simulation model and comparedthe performance of different types of agent behaviors on it.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 7 of 15
Empirical Evaluation
Empirical EvaluationThe Simulation Environment
We use a Sugarscape-like model where
• Agents move and forage with stochastic results in a 50 × 50 grid.
• Different types of resource distribution and resource scarcity aretaken into account.
• Agents consume resources, and reproduce / die based on resourceaccumulation thresholds.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 8 of 15
Empirical Evaluation
Empirical EvaluationThe Simulation Environment
We use a Sugarscape-like model where
• Agents move and forage with stochastic results in a 50 × 50 grid.
• Different types of resource distribution and resource scarcity aretaken into account.
• Agents consume resources, and reproduce / die based on resourceaccumulation thresholds.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 8 of 15
Empirical Evaluation
Empirical EvaluationThe Simulation Environment
We use a Sugarscape-like model where
• Agents move and forage with stochastic results in a 50 × 50 grid.
• Different types of resource distribution and resource scarcity aretaken into account.
• Agents consume resources, and reproduce / die based on resourceaccumulation thresholds.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 8 of 15
Empirical Evaluation
Empirical EvaluationAgent Behavior
• Our AI-based agent is based on a well-known decision-making modelin Artificial Intelligence: Markov Decision Processes (MDP).
• At every simulation step, the agent casts the surrounding world as anMDP and chooses an action that approximately optimizes a certainutility function proportional to the amount of resources held by theagent.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 9 of 15
Empirical Evaluation
Empirical EvaluationAgent Behavior
• Our AI-based agent is based on a well-known decision-making modelin Artificial Intelligence: Markov Decision Processes (MDP).
• At every simulation step, the agent casts the surrounding world as anMDP and chooses an action that approximately optimizes a certainutility function proportional to the amount of resources held by theagent.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 9 of 15
Empirical Evaluation Setup
Empirical EvaluationExperimental Setup
We run a number of simulations where:
• We measure the carrying capacity of the system on different typesof agents.
• We compare the performance of our agent to three baselines:
1 Random agent: chooses actions at random.2 Greedy agent: moves to the neighbour cell with most resources.3 Lazy agent: only moves if the current cell does not satisfy her needs.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 10 of 15
Empirical Evaluation Varying Resource Scarcity
Empirical Evaluation: Varying Resource Scarcity
0 200 400 600 800 1000
Time
0
50
100
150
200
250
300
350
Popu
latio
n
λ=3
randomlazy
greedymdp
0 200 400 600 800 1000
Time
0
10
20
30
40
50
Popu
latio
n
λ=4
randomlazy
greedymdp
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 11 of 15
Empirical Evaluation Varying Resource Distribution
Empirical Evaluation: Varying Resource Distribution
0 200 400 600 800 1000
Time
0
50
100
150
200
250
300
350
Popu
latio
n
autocorrelation=1
randomlazygreedymdp
0 200 400 600 800 1000
Time
0
50
100
150
200
250
300
350
Popu
latio
n
autocorrelation=25
randomlazygreedymdp
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 12 of 15
Empirical Evaluation Varying Resource Distribution
Empirical EvaluationKey Points
Key points:
• AI-based agents perform much differently and consistently betterregardless of abundance or distribution of resources, increasing theirpopulation faster and reaching a higher carrying capacity.
• Even among the simple baseline agents there are very significantdifferences of performance.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 13 of 15
Empirical Evaluation Varying Resource Distribution
Empirical EvaluationKey Points
Key points:
• AI-based agents perform much differently and consistently betterregardless of abundance or distribution of resources, increasing theirpopulation faster and reaching a higher carrying capacity.
• Even among the simple baseline agents there are very significantdifferences of performance.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 13 of 15
Conclusions
Take-Home Messages
• There is a wide range of agent decision-making strategies beyondsimple, reactive agents.
• These decision-making strategies have a crucial impact on theusual summary metrics of simulations such as the carrying capacityof a system.
• Either we conceptualize agent behavior as a relevant parameter of thesimulation or we develop some sound methodological principles tounderstand which type of agent behavior is right for which simuation.
• Artificial Intelligence planning techniques offer a promising avenuetowards principled and cognitively richer ABM agents.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 14 of 15
Conclusions
Take-Home Messages
• There is a wide range of agent decision-making strategies beyondsimple, reactive agents.
• These decision-making strategies have a crucial impact on theusual summary metrics of simulations such as the carrying capacityof a system.
• Either we conceptualize agent behavior as a relevant parameter of thesimulation or we develop some sound methodological principles tounderstand which type of agent behavior is right for which simuation.
• Artificial Intelligence planning techniques offer a promising avenuetowards principled and cognitively richer ABM agents.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 14 of 15
Conclusions
Take-Home Messages
• There is a wide range of agent decision-making strategies beyondsimple, reactive agents.
• These decision-making strategies have a crucial impact on theusual summary metrics of simulations such as the carrying capacityof a system.
• Either we conceptualize agent behavior as a relevant parameter of thesimulation or we develop some sound methodological principles tounderstand which type of agent behavior is right for which simuation.
• Artificial Intelligence planning techniques offer a promising avenuetowards principled and cognitively richer ABM agents.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 14 of 15
Conclusions
Take-Home Messages
• There is a wide range of agent decision-making strategies beyondsimple, reactive agents.
• These decision-making strategies have a crucial impact on theusual summary metrics of simulations such as the carrying capacityof a system.
• Either we conceptualize agent behavior as a relevant parameter of thesimulation or we develop some sound methodological principles tounderstand which type of agent behavior is right for which simuation.
• Artificial Intelligence planning techniques offer a promising avenuetowards principled and cognitively richer ABM agents.
Guillem Frances et al. Embedding Artificial Intelligence in Agent-Based Models 14 of 15
Embedding Artificial Intelligencein Agent-Based Models
Guillem Frances1 Xavier Rubio-Campillo2 Carla Lancelotti1
1Universitat Pompeu Fabra 2Barcelona Supercomputing Centre
SAA, San Francisco - April 16 2015
