Evolving Agents in a Hostile Environment

Alex J. Berry

Overview

Motivation Background The Approach Map Generation Evolutionary Algorithm Experiments

Training First Responders

VEnOM Labs is developing a suite to train First Responders

Is the training effective? How can we make the training more

effective?

Goal

To develop a system to allow for friendly and hostile AI agents in the training environment.

To develop a system to create better agents for training First Responders.

Simulation of Adaptive Agents in a Hostile Environment[HW95]

Thomas Haynes Simple Agents Mines and Energy Experiments

Single Agent, Static and Random Environment Multiple Agent, Static and Random Environment

The Complicator

Aliases: Dr. T, Dr. Tauritz

Input: 2+2=4 Output:

0,45

%7

2)84)%42118798452((9303

162

The Approach

Randomly Generated Grid Environment Three Types of Agents:

First Responders Terrorists Victims

Genetic Programming to Evolve the Agents

Randomly Generated Maps

Any Dimension Percentage walls Bit Array to Hold the

Data

Demo

What’s an Agent to do?

Victims Move Randomly Remember Things Forget Things Survive

Terrorists Kill Victims Kill First Responders Lay Traps Disguise Themselves Not Get Caught

First Responders Help Victims Find and Disarm Traps Survive Catch Terrorists

Evolutionary Algorithm

Two Agents to Evolve First Responder Terrorist

Two Competing Evolving Populations Genetic Programming for the Evolutionary

Implementation

What An Individual Looks Like

Terminals Current Grid Location (C) Surrounding Grid Locations

(S) Rand (R)

Non-Terminals If-Then-Else

Threat And, Or, Not Victim, First Responder,

Terrorist, Trap Valid Move

Actions Save Kill Move Place Trap Remove Trap Not (Action) to invert an

Action

Sample Individuals

Move

Genetic Programming Evaluation

First Responder Victims Helped Terrorists Caught Traps Found Traps Removed Survival Amount of the Map

explored

Terrorist Kills using Traps Kills on Contact Effective Disguises Amount of Time

Survived

Experiments

Static Environment Evolution Random Environment Evolution Varying Ratios of First Responders, Victims, and

Terrorists Evolving one Population at a Time

Summary

Looking at agents operating in a hostile environment. First Responders, Terrorists, and Victims

Evolving first responders and terrorist using genetic programming techniques.

Future Work and Questions

Other Evolutionary Approaches LCS GP-LCS Hybrid

Integration into a 3D environment Playable Human Mode

Representations of Real Buildings Test effectiveness of adding this to Affective Intensity

Experiment Integration of other types of Traps and sensors to detect

those traps