Emotional Agents in Game PlayingMSc Dissertation submitted
By Mehul Shukla
For MSc in Games Programming
University of Hull 16th September 2011
Project Supervisor: Dr Darryl N. Davis
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
This report is dedicated to my parents Mr Y K Shukla and Mrs Veena Shukla. Without their love and support this degree would not have been possible.
1
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
AcknowledgementsFirst and foremost, I would like to thank my girlfriend Karishma for all her love and support and for being the constant nudge that brought me here and kept me going till the end.
I would also like to thank my supervisor, Dr Darryl N. Davis, for all his help and invaluable knowledge provided during the dissertation. I am also thankful for the opportunity to pursue this topic and for all his patience which I didn’t refrain from testing.
Special thanks goes to Roger Pujol Serra for guiding me and helping me through a lot of my programming limitations throughout the year. His persistent and harsh guidance was much appreciated as it provided me with the base to complete this dissertation.
Last but not least a big thanks to all my friends Aritra, Nuno, Mario and Roger for sharing and drowning all the pains and hardships of the degree, in many a round of pints at the pub.
2
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Table of ContentsTable of Contents......................................................................................3
Table of Figures........................................................................................6
Abstract.....................................................................................................7
Chapter 1 Introduction............................................................................8
Chapter 2 Project Overview..................................................................112.1 Initial Brief...........................................................................................................11
2.2 Context..................................................................................................................11
2.3 Aims and objectives.............................................................................................12
2.4 Project Plan..........................................................................................................12
Chapter 3 Project Background.............................................................133.1 Early Theories of Emotion..................................................................................13
3. 2 Affect and Affective Computation....................................................................14
3.3 Requirement for an Emotion Architecture.......................................................15
3.4 Emotion Redefined: The Emotion Model..........................................................16
Chapter 4 Theoretical Design................................................................184.1 Game Story and Agents......................................................................................18
4.2 Emotion Actor Scenario......................................................................................19
4.3 Agent Architecture..............................................................................................22
Chapter 5 AI Test Bed Design...............................................................265.1 Testbed Theory....................................................................................................26
5.2 Testbed Design Constraints................................................................................27
5.2.1 Choice of a planning or reactive system:.........................................................................27
5.2.2 Complexity of the world..................................................................................................27
5.2.3 Sensing and Effecting......................................................................................................27
5.2.4 Partial Satisfaction of Goal States...................................................................................28
5.2.5 Multiple Agent Coordination and Communication.........................................................28
5.2.6 A Clean Interface.............................................................................................................28
5.2.7 Time Module...................................................................................................................29
3
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
5.2.8 Experimentation and Testing...........................................................................................29
Chapter 6 Software Design....................................................................306.1 Software tools and SFML...................................................................................30
6.2 C++ Architecture and Implementation.............................................................31
6.2.1 Core Game Engine...........................................................................................................32
6.2.2 Map..................................................................................................................................32
6.2.3 Class LevelLoader...........................................................................................................33
6.2.4 Class ImageManager.......................................................................................................34
6.2.5 Class Agent......................................................................................................................34
6.2.6 Class Pathfinder...............................................................................................................34
6.2.7 Class Engine....................................................................................................................36
6.3 Agent Hierarchy..................................................................................................38
6.3.1 Class Agent......................................................................................................................39
6.3.2 Class Agent0....................................................................................................................40
6.3.3 Class Civilian, Robber and Police...................................................................................40
Chapter 7 Gameplay and Design..........................................................427.1 Detailed Storyline................................................................................................42
7.2 Control Factors and rules for the Testbed........................................................43
7.3 Finite State Machine Design...............................................................................45
7.4 Agent Behaviour and Design..............................................................................45
7.4.1 Base Agent.......................................................................................................................45
7.4.2 Agent0..............................................................................................................................46
7.4.3 Police...............................................................................................................................46
7.4.4 Civilian............................................................................................................................49
7.4.5 Robber..............................................................................................................................52
Chapter 8 Emotion Architecture..........................................................568.1 Civilian emotion model.......................................................................................58
8.2 Robber emotion model........................................................................................58
8.3 Police Emotion model..........................................................................................59
Chapter 9 Conclusion.............................................................................609.1 Critical Evaluation..............................................................................................60
7.2 Future Work........................................................................................................60
7.3 Conclusions...........................................................................................................61
4
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
References...............................................................................................62
Bibliography................................................Error! Bookmark not defined.
Appendix A Original Specification......................................................65
Appendix B Updated Specification......................................................66
Appendix C Requirements Analysis Table.........................................67
Appendix D Character Icons................................................................68
Appendix E Project Plan......................................................................69
Appendix F Considerations..................................................................72Legal Considerations.................................................................................................72
Ethical Considerations..............................................................................................72
Social Considerations................................................................................................72
Professional Considerations......................................................................................72
5
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Table of FiguresFigure 1 Example Game world.................................................................................................18Figure 2. Fear............................................................................................................................19Figure 3. Sadness......................................................................................................................20Figure 4. Disgust.......................................................................................................................20Figure 5. Anger.........................................................................................................................21Figure 6. Happiness..................................................................................................................22Figure 7. Reflex agent architecture...........................................................................................23Figure 8. Reactive agent architecture.......................................................................................23Figure 9. Reactive and deliberate level architecture.................................................................23Figure 10. Reactive behaviour architecture..............................................................................24Figure 11. Goal based reactive behaviour................................................................................24Figure 12. Emotion based reactive behaviour..........................................................................24Figure 13. Emotion and goal based reactive agent architecture...............................................25Figure 14. Example Benchmarks..............................................................................................26Figure 15. Core Game Engine Class Diagram..........................................................................32Figure 16. Example Map File...................................................................................................33 Figure 17. A* Search Algorithm.............................................................................................35Figure 18. Agent Inheritance Hierarchy...................................................................................38Figure 19 Game/Test bed Window...........................................................................................43Figure 20 Police FSM...............................................................................................................48Figure 21 Civilian FSM............................................................................................................51Figure 22 Robber FSM.............................................................................................................55
6
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
AbstractThe focus of this research was in the area of Artificial Intelligence in Computer Games. The
main goal was to investigate how emotions can be modelled in character agents in computer
games. The study was based on the effect of emotions as an arbitration mechanism for a
reactive goal based architecture. The research approach involved simulation of real world
situations in the form of a C++ program. The program consisted of three types of agents
namely police, civilians and robbers. The program puts these agents in different situations
where they can interact with each other i.e. the game includes multi-agent interaction. An
example situation consists of a rescue operation carried out by a group of policemen where
insurgents have broken into a bank and taken some civilians hostage. . Such scenarios
provide a good investigative platform for research as they commonly occur in real life and
are also frequently modelled in computer games. An emotion model has been used to select
the emotions for the project. These are fear, anger, disgust, sadness and happiness. These
emotions are applied on top of the reactive agent architecture. The comparison made between
the emotive and non-emotive agents, trying to accomplish the same goals, has been used to
analyse the effect of emotion in video games. Other essential features include the judicious
use of Finite State Machines as part of the architecture and an efficient approach towards AI
test-bed design.
7
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Chapter 1 Introduction
This study focuses on the research and implementation of emotions in computer games. The
focus is on how the overall nature of a game can be changed with the addition of human like
emotions in character agents. The comparison and contrast of the resulting emotional
response, along with the default behaviour of an agent can then be further investigated for the
possibility of improving realism in computer games.
Many of the theories and techniques applied in computer games are derived from the field of
Artificial Intelligence (AI). The common goal of majority of these techniques is to model the
most realistic human or human-like behaviour possible in artificial entities or agents (Nilsson
1998). Such behaviour is complex, uncertain and is expected to occur immediately or in real-
time (Agre and Chapman 1987). A game agent can be defined as a computer controlled
human or non-human character or entity. In other words, any object in the game that is not
controlled by the player (Shihab 2009). This form of computer controlled behaviour is what
gives the perception of ‘intelligence’ in agents. Intelligence can be generalised in terms of
“perception, learning, communication, and acting in complex environments” (Nilsson 1998).
This allows us to understand the rationality of the term ‘Artificial Intelligence’ and allows us
to coin this term once again through our own observation of what an agent does when
programmed to behave in a certain way i.e. a rational response.
It is then we realise where the term ‘Artificial’ comes from and for this reason the term
artefacts applies aptly to agents in computer games. Computer games are geared towards
providing a predictable form of intelligence because a game is planned to start and end in a
specific manner. However, with the evolution of the computer games industry, many gamers
now require a more immersive and unconventional gameplay experience.
In order to achieve said experience, the AI has to move away from a rational behaviour
towards a more empirical methodology (Russell and Norvig 2003). Emotional response is one
such area of research. An emotion can be defined as something that occurs spontaneously. It
is a term that is synonymous with human behaviour. This has to be taken into account when
designing suitable software involving such behaviour. If the application and translation of
emotional behaviour in a video game is done accurately, it will by default exhibit
characteristics of unpredictability and differ from the conventional progression of computer
8
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
games from past years. Emotions and their resulting behaviours are complex and have been
deeply studied by theorists. Some of these theories also have biased opinions. This research
chose not to delve on such theories or postulate any new ideas in order to form a
revolutionary theoretical base. In contrast, the idea of the study was to embrace and conform
to existing suitable ideas from the area of human psychology. These ideas were then matched
with computational techniques in order to design an application to demonstrate emotional
behaviour in a computer game.
Adapting emotions in games is a recent idea and is the main reason for the choice of this
topic. As mentioned previously, not only do emotions aim to replicate human behaviour, they
do so with a high level of unpredictability. This does not mean that this will be beneficial; it
can also be detrimental to a game scenario. The extent of application of emotions in computer
games has to be in accordance with the requirements of a game scenario. For this reason, such
applications have to be tackled in an empirical manner. Only through accurate observation
and experimentation, suitable game AI can be designed that offers not only unique gameplay
but also holds entertainment value for the player.
Some people may argue that this is what learning agents provide in a game. They adapt to the
gameplay and offer a different experience each time. Emotional agents differ in the manner
that they exhibit behaviour that allows the player to connect very easily on a physiological
level. The emotions and actions displayed are familiar to the player in some past scenario.
The study involved implementation of human emotions in the form of a software test bed.
The testbed consists of a mini game involving 3 types of agents namely Civilians, Robbers
and Police. The game puts these agents in a shared game world where they can interact
amongst each other and with the environment. The behaviour of the agents is according to a
game storyline. The storyline puts these agents in a real-time bank robbery scenario.
Systematic and hierarchical Agent design was one of the biggest challenges of this
dissertation. A goal based reactive architecture was used for agent design. The project was
implemented using C++ and SFML1. The agent decisions were implemented using Finite
state Machines (FSM). Detailed explanation of the FSM implementation has been provided in
the report but an attempt will not be made to explain the theory behind finite state machines
as it was a tool that was required for the project. Briefly,
“A finite state machine is a device, or a model of a device, which has a finite number of states
it can be in at any given time and can operate on input to either make transitions from one
1 Simple and Fast Media Library www.sfml-dev.org
9
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
state to another or to cause an output or action to take place. A finite state machine can only
be in one state at any moment in time.”(Buckland2005).
Further work involved the selection of a suitable mechanism that would employ emotions as
an arbitrator for the selection of actions.
The report has been written in conjunction with the implementation of the project. It will
therefore be noticeable that each subsequent section in the report is dependent on the previous
one and the report progression is in a chronological format.
Other emphasis has been placed on good AI testbed design and good software practices for AI
programming. The programming techniques have been evaluated against suitable testbed
requirements to ensure that the end product was of the highest standard and could be extended
further in the future.
10
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Chapter 2 Project Overview
2.1 Initial Brief
The initial brief of the project was open to negotiation as many different approaches can be
followed. The original project specification can be seen in Appendix A.
This specification was updated and can be found in Appendix B .
2.2 Context
The main goal of the research is to study a method of Artificial Intelligence that can make
games more interesting and immersive. The idea is to involve human emotions in character
agents in computer games. There have been many games in the past where emotions have
been used. Some examples are Black and White and the Fable series from Lionhead Studios.
These games present emotion in such a way that the outcome of a decision taken by the player
causes an emotion or affect in the character agents which depends on their current emotive or
non-emotive state. Another example of emotion can be seen in football games. Most of the
emotion in football games is displayed out of the game by the players. This is triggered by the
competitive gameplay which football games provide and shows that emotions also play a big
part when it comes to competition or rivalry. Another popular genre of games is first person
shooters. In these games emotions are more likely triggered in players through story or
aesthetic elements and visual feedback of critical events in the game. This leads to heightened
emotions in the player which then affects the overall outcome of the player’s decision and
game events. These are all examples where emotions are displayed and provide evidence of
the outcome of emotions in terms of game playing. It is with this in mind that this project has
been chosen and aims to investigate how an ever changing gameplay experience can be
achieved through human like emotions using multi-agent coordination.
11
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
2.3 Aims and objectives
For the purpose of clarity the aims have been listed in a chronological order
i. Research on emotion and affective computation
ii. Research and selection of one model of emotion for application
iii. Scenario and story design of computer game test-bed
iv. Selection of characters and events in storyline
v. Graphical interface design of software test-bed in C++
vi. Design of Finite state machines including input, change and output functions for characters
vii. Design of a character agent termed level0 agent with wandering movement attributes using
finite state machines in C++
viii. Use the level0 agent as the base agent and derive all required characters from this agent
ix. Implement required AI features to enable inter-agent coordination i.e.
functionality/playability of the game.
x. Extend the level 1 agent with the emotion model.
xi. Experimentation and testing/comparison of emotive and non emotive agents and their actions
xii. Further scope of the project by extending the chosen emotion model.
2.4 Project Plan
The project plan can be seen in Appendix E
12
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Chapter 3 Project BackgroundThis chapter provides an insight into the theoretical framework of the study. It discusses the
need for an emotion model that has been implemented in the design and provides a detailed
overview of the essential background research for this project. As human emotion is a topic
that is more concurrent with the field of psychology, the implementation in video games
requires a broader understanding of the topic. It is for this reason an emotion model has to be
adapted into the design.
The emotion model provides an in depth view of the main theme of this study, emotion and
affect, thereby solidifying the view on affective computation. It shows through example
scenarios how and which kind of emotions can be elicited in character agents.
These ideas are extended further and embedded in a storyline that consists of events where
emotions are induced an affect is experienced. A C++ based programme is used to provide a
visual outcome of the interactions between emotive agents.
Me: needs a bit of editing a slightly broader introduction to the chapter more emphasis on affective computation.
3.1 Early Theories of Emotion
The oxford dictionary defines emotion as “a strong feeling deriving from one's circumstances,
mood, or relationships with others”. This provides us with a generic definition that many
people can relate to but likewise there are many that disagree.
The earliest theories are related to the theories of evolution. Charles Darwin proposed that
emotions evolved in order for humans to adapt. It was a process that was part of Natural
Selection (Darwin 1872). This was his reason behind all humans all around the world sharing
the same emotional response (Gaulin and Donald 2003).
William James, a pioneer psychologist and physiologist, defined emotion as “the perception
of bodily changes as they occur is defined as emotion” (James 1884). This was supported by
Carl Lange and collectively became known as the “James-Lange Theory”. In contrast to this
theory the “Cannon-Bard” Theory says that bodily changes are caused by emotion
completely opposing James and Lange (Britannica Encyclopaedia 2011). A third theory that
13
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
attempted to mediate between these two conflicting theories was the “two factor theory of
emotion”. This theory stated that emotions were caused by both “physical arousal” but was
(were?) not the sole cause of emotion (Schater and Singer 1962).
All these theories provided conflicting ideas as to the origin of human emotion when grouped
together. At this stage, none of these theories could be applied in any way in the
implementation phase of the project. However, these theories provided a benchmark of ideas
to be compared with theorists in the later years. It is the theories of this newer era that would
fulfil the requirement for a suitable model of emotion for the project. Prior to this discovery,
some additional gaps have to be filled in for a better understanding.
3. 2 Affect and Affective Computation
In order to understand affect and its relevance to the study, it was necessary to delve further
into different branches of psychology. Before understanding affect, it is necessary to define
another term relevant to the thought process. ‘Cognition’ refers to a state of mental activity.
Cognition refers to the minds abilities which include memory, perception, thought processes,
speech and problem solving (Feist and Rosenberg 2009). Affect can now be defined as the
feeling of emotion that can be differentiated from all aspects of cognition (Huitt 2003). Since
this is a very abstract definition, affect can also be said to represent a reaction to external or
internal stimuli, manifesting itself in the form of feelings and or emotions. Therefore, affect
can be thought to occur in the time period between a stimulus and the reaction to a stimulus.
This knowledge was sufficient to move on to the computational domain. The transition of a
theoretical model of emotion into a software design depends on previous work done in this
area. As the topic of emotion is a much specialised topic, it falls under a niche topic of AI
known as Affective Computation.
It has been a challenge for people to understand the meaning of affect due to the abstract
nature of the term. For this reason, affective computation was not used in computing for a
very long time and has undergone development in fairly recent years. The origins of affective
computing date back to as recently as 1995 where it has been defined as “computing that
relates to, arises from, or deliberately influences emotions” (Picard 1995). Picard has been
part of MIT University’s research group on affective computation, which is one of the leading
authorities in this field. At this point this was enough understanding of the problem domain in
order to progress to a computational model. Any proposed computational model would have
to conform to the rule that emotions were being influenced deliberately. However, the
reaction of the agents will have following the emotional response will be very intriguing. The
reactions may or may not follow trends of human unpredictability and complexity but will
14
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
however follow a very intelligent and unique response. It is these outcomes and responses that
will allow the evaluation of what new ideas can make a computer game more interesting.
3.3 Requirement for an Emotion Architecture
This section takes the definitions of emotion and affect and puts them into a more practical
perspective. The task at hand involves creating a game that consists of core entities in a
world, which involves real time interaction in human based scenarios. In order to do so the
architecture has to consist of a core driving mechanism. In this case the game will consist
consists of goal based reactive agents. A reactive agent can be defined as an agent which upon
receiving an input, will process it and return or carry out a suitable output. The agent will
continue to follow these protocols until it has achieved its assigned goal or it ceases to exist.
The input for the reactive agent will come from interaction with other agents and external
stimulus from the game world. Once this has been enforced, it can be said that the closed
system now consists of a game world and human-like agents carrying out assigned tasks.
In spite of this, the agents are only human-like in the nature of tasks that will be performed
and not in the way that these tasks should be performed. For the agent to be human, it will
have to display characteristic of a similar grade of unpredictability and complexity. In
retrospect to the agent being goal based and reactive, it still follows a sequence of carefully
selected inputs. In other words the agent ‘does what it is told to do’.
On the other hand, in a multi-agent environment, unpredictability is factored in
automatically. The probability of a specific output being executed reduces after interaction
with another agent. The precedence of execution of the output of one reactive agent may
suppress the input or even the output of another agent. For e.g. if an agent is killed in its input
processing stage, then it will cease to exist even before its output can be executed.
In this way the interactions of multiple agents provide an admissible level of unpredictability
and complexity in a game world.
In many games, this architecture will suffice. When coupled with all the other aesthetic
aspects of a video game, such goal based reactive agents provide a reasonable entertainment
experience. The only problem is that this experience is short lived because it is completely
exhaustible. What this means is, there will be a time when the game will behave in a way that
has been seen many times before. This is when a game becomes “boring”. It ceases to provide
15
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
any kind of challenge to the player. It has now become completely exhaustible . The player
will now abandon the game and move onto the next one.
A popular method of making a game more interesting is a learning agent. A learning agent is
an agent that will constantly improve itself in different situations in order to find a beneficial
outcome (Russell and Norvig 2003). The detailed theory behind a learning agent is out of the
scope of this project. The learning agent offers a comparison against an emotive agent. The
learning agent differs in the fact that it will continue to improve itself until it ceases to exist.
In this manner a learning agent will be different every time. The disadvantage of the learning
agent is that if it gets too proficient at its task, the game will run the risk of abandonment by
the player. The whole idea of a computer game is to offer the players an achievable challenge
to make them think that they have won, while it can so easily be said that the AI has been
tweaked to lose. Thus a learning agent cannot be forever learning in a commercial computer
game. It will have to be limited at some point to ensure the success of the game.
It is imperative at this point that a new class of AI agents is required that which are not only
unpredictable but engaging at the same time. This is where the emotive agents differ from
learning agents. The first and foremost quality of an emotional agent is that it is akin to
humans because it is “emotional”. It exhibits one of the biggest qualities in human beings. We
have already established that a reactive agent can process tasks. Adding emotion in this
equation will add a new stage in the input processing and output of a reactive agent. It is this
new stage which will allow for a more realistic
Need something more here but for now a mental block on this topic,. Will review later needs a sort of link to the next topic
and engaging gameplay.
3.4 Emotion Redefined: The Emotion Model
With the basic theory in place, this section focuses on the research of emotion by theorists in
the late 20th century following the incomplete ending of section 2.1.
The emotions that will be modelled in the project are based on the definition of Oatley and
Jenkins(1996) :
“A state usually caused by an event of importance to the subject. It typically includes
(a) a conscious mental state with a recognizable quality of feeling and directed towards
16
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
some object
(b) a bodily perturbation of some kind
(c) recognisable expressions of the face, tone of voice, and gesture
(d) a readiness for certain kinds of action”.
Other theorists and psychologists have postulated similar theories. These include Frijda
(1986),
Power and Dalgeish (1997). Their agreement on the definitions of the 5 basic emotions will
be used to model the agent behaviour in this project.
These are quoted as follows:
Fear defined as the physical or social threat to self, or a valued role or goal
Anger defined as the blocking or frustrations of a role or goal through the perceived actions
of another agent
Disgust defined as the elimination or distancing from person, object, or idea repulsive to self
and valued roles and goals
Sadness defined as the loss or failure (actual or possible) of a valued role or goal
Happiness defined as the successful move towards or completion of a valued role or goal
17
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Chapter 4 Theoretical Design
4.1 Game Story and Agents
Figure 1 shows an example of the game world. The game world consists of a top down view
of a bank. The world is tile based and each tile represents a movement node. The physical
properties of the world include obstacles that are represented by different pictorial forms. In
addition to this, the game world consists of entities and agents as can be seen in the
screenshot below. A detailed description of individual agents has been provided in the
following sections.
Figure 1 Example Game world
The game story consists of 3 types of agents namely, policemen, civilians and bank robbers.
The scenario involves a bank in which some robbers have broken into. The robbers will
attempt to get to the vault holding the money and the police will try and stop them from doing
so. It is the job of the soldiers to evacuate the civilians and arrest the robbers.
18
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
In simple terms each agent has its own designated goal. A more specific breakdown of the
agents’ tasks has been shown in chapter 7, which discusses the overall gameplay. These goals
eventually lead to default actions of each agent. Adding emotions in the game will lead to a
change inaction selection by the agents in accordance with the current emotion actor scenario.
Some examples of these scenarios are shown in the next section.
4.2 Emotion Actor Scenario
A further understanding of how the agents in the game will act out emotions can be shown
with the help of some examples. The following examples outline the 5 basic emotions in a
game scenario. The agents in question are part of 3 different sets. The police(green squares),
bank robbers (red triangles) and the civilians (blue circles). The following figures show how
these emotions can be elicited. These scenarios are only a few of the limitless possibilities
that can be modelled. At this point they are purely to detail understanding of the 5 basic
emotions and how they can be applied in the game.
Figure 2. Fear
Figure 2 A civilian get attacked and the first emotion they feel is fear.
19
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Figure 3. Sadness
Figure 3 A team member of the Police dies, or was killed by an enemy. The rest of the team
feel sadness.
Figure 4. Disgust
Figure 4 One of the members of the police is a traitor and in the middle of the game defects to
the other side. The other team members are now in disgust.
20
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Figure 5. Anger
Figure 5 Robbers are trying to take civilians hostage. This is their goal. The police are
preventing the robbers from completing their goal. The robbers feel angered.
21
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Figure 6. Happiness
Figure 6 A Robber has taken a civilian hostage. The tactical team’s goal is to rescue the
civilian. They complete their goal and are now happy
4.3 Agent Architecture
The design of the agents, their behaviour and action was to be done incrementally starting
from the lowest level. The overall architecture can be explained better with the help of
diagrams. These diagrams will be modelled sequentially to design an overall evolved
emotional agent.
Figure 7 shows the most basic level of agent design. This level comprises of an agent’s
sensory level or the means by which an agent experiences external stimuli. The action based
on this stimulus will be totally reflexive, akin to the most natural of responses by humans.
There are virtually no goals present in the architecture at this time. The agent can therefore
22
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
only offer an involuntary reaction. Example predator coming towards prey, prey screams in
fear.
Figure 7. Reflex agent architecture
The next step in this design is to give the agents some thought. This level is known as the
reactive level because it allows the agents to perform an action or a reaction. This is slower
than a reflex and where it differs is that the reaction is voluntary. This takes into account the
reflex level. This is represented in Figure 8. Example prey now runs in fear. This gives the
agent some reactive behaviour.
Figure 8. Reactive agent architecture
In order to make the agent more intelligent, the agent requires decision making capabilities. In
order to accomplish this agent needs some goals. The actions and decisions of the agents will
now be based on their goals i.e. the agents are completely goal driven. This is shown in
Figure 9. For example, a robber trying to take a civilian hostage; it will not deviate from this
goal until achieved.
Figure 9. Reactive and deliberate level architecture
Figure 9 can be simplified by combing the reactive and deliberate level into one level called
the Reactive behaviour level. Figure 10 now defines how an agent behaves in an environment.
23
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Figure 10. Reactive behaviour architecture
This will be the basis for a Class 0 agent, the most simplified agent in the architecture.
This architecture is now extended by adding more goals to its existing reactive behaviour.
These goals will differ across the 3 classes of agents i.e. the soldiers, robbers and the
civilians. This is shown in figure 11
Figure 11. Goal based reactive behaviour
The next agent architecture is the core of the project. This is the point where emotion will be
added to the reactive behaviour. As shown in Figure 12, the agents will now be driven by a
particular emotion. There will be 5 possibilities of this architecture, each modelled with one
of the 5 basic emotions. Not all agents may require each of the 5 emotions. The civilians for
example will be modelled with a subset of the 5 emotions. This is termer the Class1 Agent
Figure 12. Emotion based reactive behaviour
24
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Class 1a agent will be the combination of both the class 0 & class 1 agent. The behaviour
will be interdependent with the current goals and resulting emotions of the agent. Both the
goals and emotions will control how the agent behaves. This is shown in figure 13
Figure 13. Emotion and goal based reactive agent architecture
If time permits this architecture will be extended further, to incorporate the secondary goal of
agents with multiple emotions and goals i.e. agents with a personality. This will be the most
complete architecture that can be used in a commercial game because agents will then be able
to behave differently depending on how a player has progressed through the game and what
kind of agent behaviour his/her actions have resulted in.
25
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Chapter 5 AI Test Bed Design This chapter involves the implementation of the AI testbed in C++.
5.1 Testbed Theory
It was mentioned earlier that the project requires an empirical approach towards
experimentation. There are many theories that have been mentioned that need to be
investigated through careful and controlled testing. From here onwards the game will be
referred to as the Emotion AI testbed.
Before we can understand what testbed is, the term Benchmark needs to be introduced. In
computing, a benchmark is usually used to compare performance against. A specification is
the minimum requirements a product must complete in order to be acceptable. A benchmark
differs in the respect that it allows the scientist to deduce of the behaviour of the product
(Hanks 1993). In the case of this project the minimum benchmark requirements refer to a
series of tasks that must be accomplished. Figure 14 shows a table in which some of the
benchmarks have been listed for each of the 3 characters.
Agent Benchmarks
Civilian Ability to Move in the world environment
Ability to stay isolated in idle tasks
Soldier Ability to Move in the world environment
Ability to differentiate between civilian and robber
Robber Ability to Move in the world environment
Ability to target civilian or soldier
Figure 14. Example Benchmarks
26
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Usually a benchmark is a measure of performance. In the case of this project, a benchmark is
a reflection of the basic mandatory tasks that a controlled AI test bed should be able to
perform. These tasks are invaluable as they form the basis of the implementation of the AI
test bed.
Taking the context of this project into account, an AI test bed can be defined as a platform for
evaluation and experimentation of proposed emotion theories and the relevant computational
model.
5.2 Testbed Design Constraints
Test bed design is one of the most critical aspects of agent design. There are some
characteristics that needed to be taken into consideration for the AI test bed. These are issues
that are prevalent in testbed design across different AI research applications and need to be
addressed for any significant progress. The following characteristics define an ideal test bed
(Hanks 1993).
5.2.1 Choice of a planning or reactive system:
A planning system requires that the input state of a system is known beforehand. The task of
the agent is to achieve the state through a series of pre-defined actions. This however was not
the requirement of the project. The whole idea behind the project was to design unpredictable
behaviour in the agents. A reactive system was employed which consisted of a set of pre-
defined actions and the selection of these actions was dependent on the current state of the
agents. In other words the choice of the actions was in reaction to the current state .i.e. a
reactive system.
5.2.2 Complexity of the world
In order to model realistic human behaviour the world properties also have to be taken into
account. Ina computer game, the world defines the domain which an agent will interact with.
In order for this interaction to be realistic the physical properties of the world also had to be
modelled in the test bed. Depending on the application some characteristics may be irrelevant.
In this case the only characteristic modelled was the ability of the walls and objects to act as
physical obstacles. This was the sole interaction of the agents in the game world.
5.2.3 Sensing and Effecting
Once the world properties have been incorporated, the test bed requires an agent to interact
with its surroundings and other agents in the environment. This requires the agent to be
perceptive in its environment. This ability is termed as sensing. The actions followed from
27
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
sensing and the interpretation of external or internal stimuli is known as effecting. In order to
preserve the causality of an AI application, the agents need to be aware of all events around
them. Appropriate sense modules need to be included in the test bed to provide this
functionality. The sense modules then need to be linked with the action execution mechanism
of the test bed. The action selection can be done using many of the preferred AI design
schemes. This project used finite state machines as the technique of choice. This linked with
the sense module allows the agent to interact with its surroundings.
5.2.4 Partial Satisfaction of Goal States
For an agent to be believable, it has to display some imperfections which are akin to human
behaviour. It is for this reason that the sensing and effecting cannot always be accurate. An
agent is usually required to move from one state to another or towards a required goal. An
exogenous event is likely to interrupt this transition from one goal state to another. These are
imperfections in the sensing and in turn the effecting because as is evident in human
behaviour, not all goals are not achieved all the time. In certain scenarios, this interruption
may even be beneficial.. For example in a reactive system sometimes it may be beneficial for
the prey to interrupt a goal transition to flee if a predator suddenly approaches. An ideal test
bed should be able to provide support for partial fulfilment of goal states.
5.2.5 Multiple Agent Coordination and Communication
The selection of the bank robbery scenario implied first and foremost that the test bed would
consist of multiple agents of different types. In due course it was inevitable in the
implementation that the agents were required to interact with each other. This was essential
for the success of the game and the progression of the story line. There were certain factors in
the game that required one agent to interact with another. For example, if a robber wanted to
take a civilian hostage, it had to instruct the civilian to follow it. To support the sense modules
in a test-bed, support for multiple agent interaction is not only advantageous but essential. It
is advantageous because it exposes the unknown behaviours that are presented when agents
interact with each other. It is essential because even though individual agents are predictable,
multiple interactions are not and need to be controlled.
5.2.6 A Clean Interface
A big aspect of software design is software maintenance. With this in mind, the test bed was
designed in a modular fashion. The bottom up approach consisted of adding different sections
and modules as the implementation progressed. This meant that the sensing and effecting
modules were designed in distinct and sequential stages. Also all the data was managed
28
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
elsewhere in the software and passed onto the agents as was required. The clean designed
offered a clear understanding of the data flow in the test bed, the evaluation of the data by the
agents, and the actions in reaction to the data. The chapter on software implementation
discusses this in more detail.
5.2.7 Time Module
The occurrence of events and actions in a testbed must be measured through the passage of
time. All inputs processing and output has to be invoked at specific time intervals for a
feasible operation of the program. For the simulation to be acceptable each agent requires a
global time measurement strategy that it can use in its action selection. The design scheme for
this project consisted of Finite-State-Machines. At each instant of time, an agent was
considered to be in one of many finite states applicable to its design. The timing strategy
chosen was a turn based clock. Each turn represented a unit of time, the culmination of
which, led to the next or final state. In addition to this another clock was implemented that
saved the system time for the entire simulation. This was also used by various other parts of
the software, such as the graphics modules. The entire passage of simulation has to be
available as it is a vital tool used in experimentation and testing.
5.2.8 Experimentation and Testing
The method of experimentation depends on the technique of AI. An ideal testbed should be
able to gather data from the simulation and present it for analysis at the end. Again the criteria
for this data can vary. For example, when testing two different path finding techniques, a
measure of time and the number of movement nodes selected offers a brief evaluation of the
performance of each technique. This can be stored in a text format over multiple simulations.
On a larger scale, a survival game can store success and failure rates of a protagonist over
time or multiple simulations. These results will reflect how the AI affects the behaviour in a
testbed.
For rigorous evaluation, a testbed should be flexible enough to alter different behaviours and
observe the results. This is of utmost importance because it allows the designer to compare
and contrast plausible scenarios in order to deduce the most proficient one. The overall
quality in terms of experimentation and testing of a testbed depends on the number of
available attributes that can be altered as required.
29
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Chapter 6 Software Design The chapter covers design methodologies used for the software. It covers the design tools
used and focuses on the overall implementation in C++. Initial topics cover the tools used
followed by a class diagram of the testbed. The roles of individual classes are also explained.
The idea is to give a clear understanding of how the gameplay elements function in terms of
programming.
6.1 Software tools and SFML
The software has been designed using Microsoft Visual Studio 2010 using the C++
programming language. There are no basic hardware requirements as the testbed is not
graphically intensive. The graphics have been implemented using SFML.
The Simple and Fast Media Layer (SFML) is an object oriented API (Application
programming Interface) that has been written in C++. It provides high and low level access to
features such as graphics, input, audio and sound among others. It has been designed by
Laurent Gomila and is an open source free to use API.
It was used to fulfil the requirement of a quick OpenGL interface with C++ in visual studio.
This was a better alternative than glut or any other OpenGL library because it provided a
clearer and easier to manager interface with many other media components including
graphics. The commands and level of abstraction provided by SFML made graphics
programming easy.
The version of SFML used for this project was version 1.6. All the required files have been
included with the source code. The key things to note that for using SFML the libraries need
to be compiled before they are included in a project or this will result in linker errors. The
libraries also need to be specified in the linker settings in visual studio. The exact procedure
of setting up visual studio with SFML can be found on the website. http://www.sfml-dev.org/
Please note that the website mentions the libraries for use with Microsoft Visual studio 2008.
In order to use the libraries with visual studio 2010, these steps need to be followed.
i. Download the required version of SFML from the website
ii. If the version contains a visual studio solution open it with visual studio 2010
30
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
iii. Compile the solution in both release and debug mode to build the static(.lib) and dynamic
libraries(dll) for visual studio 2010.
iv. Use these newly generated library files for visual studio 2010.
If the suitable include and lib folders have been included in the project and the required lib
files have been specified for the linker in visual studio, the project will compile.
6.2 C++ Architecture and Implementation
It was initially intended to use a pre built game and modify it into a desired test bed. The idea
was to use a game called Raven (Buckland 2008). This idea was abandoned and the complete
software design was done from the beginning. The idea behind this was to personalise all
aspects of the dissertation. Another reason was to implement all the basic requirements of a
good testbed that have been mentioned in section 5.2. This resulted in a small 2D tile game
engine. The key features of the engine have been highlighted in this chapter.
The complete software architecture has been explained with the help of diagrams. The
diagram shows the agent hierarchy as well as the other main and utility classes. A brief
description of each class is specified after each diagram.
31
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
6.2.1 Core Game Engine
Figure 15. Core Game Engine Class Diagram
Figure 15 shows the Engine Architecture in its most basic form. The helper and utility classes
have been removed to give a clearer understanding of the software architecture.
6.2.2 Map
Before the explanation of all the classes it will be useful to explain how some of the data is
stored. The project contains a text file which contains the game map. The map contains an
array of numbers which represents the map grid. The rows and columns are specified at the
beginning of the file. The numbers in the map vary from 0 to 5. This can be extended as
required up to 9. The 0s on the map represent walk able grid nodes and the 1s on the map
were used to represent obstacles. The numbers 2, 3, 4 and 5 have been used to represent the
start/spawn positions of the agents in the game. One of the tasks of the policemen was to
patrol from room to room. This has been achieved by representing room doors as number 8s
in the map. The police then patrol between these specified doors. This design was one of the
32
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
main reasons for creating an all new test bed. The advantage of this design is that it is
easily extendable and adaptable. It is also not testbed specific. This approach can be used in
many other tile based games. As will be seen later, it is perfect for path finding algorithms as
the numbers can be used to represent different terrain along with obstacles. Another
advantage is that the size and complexity of the map can be increased or decreased by
just editing the map file. The software remains unaffected by this and will continue to
function as normal assuming all the data required has been provided in the map.
The only disadvantage of this design is that one cell can only represent one unit of
information. An improvement on this design will be to use the bits of a number in each cell.
An example map can be seen in figure 16.
Figure 16. Example Map File
6.2.3 Class LevelLoader
LevelLoader acts as a parser for the map data. It specifies storage containers in the form of
STL2 vectors for storing the data from the map file. Separate vectors were created for
individual data types. These can be seen declared in the header file. The most important thing
to understand is that the LevelLoader provides the entire 2 dimensional x and y coordinates
for every piece of information in the game. These coordinates are essential for the movement
2 Standard Template Library
33
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
logic including path finding. The coordinates are also used for the graphics in the Engine
class. The Load() method parses the file data. GetMapGrid() gets the number of cells in the
map along with the number of rows and columns. GetSpawnPoints() gets the position of all
the spawn points for every agent along with the door positions.
6.2.4 Class ImageManager
The biggest problem in 2D games is image management. When an image is loaded from a
file, it is stored as an array of pixels in memory. This image is now a resource used by the
program (Gomila 2007-2011). The problem with this is that the image itself is an expensive
resource. It is slow to load every time and large in size. This is where the term sprite comes
in. The sprite is a copy or a visual representation of the image that points to the original image
in memory. With sprites it is possible to have multiple sprites of the same image. It is also
faster to load and smaller in size. This was one of the main reasons for using SFML as it has
built in support for image to sprite conversion.
The problem faced in the testbed was loading and allocating these images in a clear and
concise manner. This task was abstracted to the ImageManager class. All the images were
stored in an STL vector for dynamic allocation. The methods AddImage() and GetImage()
were responsible for adding and retrieving images from memory respectively. In this way the
ImageManager is responsible for the Memory management of the images. All the user has to
do is define a path to the image and specify a sprite tag to use with the image.
6.2.5 Class Agent
This class is the parent or base class for all the agents in the system. All the other agents
inherit from this class. It interacts with the Engine class to gather all the required data which
was subsequently used by the derived or children classes. Section 6.2.2 details the
implementation of the agent hierarchy.
6.2.6 Class Pathfinder
There were various elements in the game that required specific character movements. For
example, movement of the robbers to the bank vault as well as chasing and evading agents.
Two approaches were chosen at this point. The first was to implement some logic for the
different types of movement. The second was to use a search space algorithm for path finding.
The latter approach was thought to be beneficial because it could fulfil all the requirements. A
path finding algorithm could be used in many different ways without editing the actual
algorithm. It could also be used in combination with a Finite State Machine (FSM) design for
reactive agents. The FSM design has been discussed in the next chapter.
34
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
The algorithm used for this project was the A* (A-Star) Search Algorithm. Even though the
implementation of this was not an easy task, it has not been fully explained in the report. This
was a tool that was required and its explanation is not in the scope of this project. A clearer
explanation of how the search algorithm works can be understood from the source code in the
Pathfinder class. A brief explanation has been given with the help of figure 17.
In the AI test bed, the A* algorithm is applied on the game map, which consists of a grid of
squares. A* works by finding the shortest distance over the map grid from a start node to a
target node. The nodes are the x and y coordinates in 2 dimensional space on the map. The
advantage of A* is that it is guaranteed to always find the optimum path.
Figure 17. A* Search Algorithm
In figure 16, consider Yellow circle at the start node. We can estimate how many cells it will
take to Enemy B, which is our end node represented by one of the blue circles. This
estimation of how much we think will cost to get to the goal is known as the heuristic or h.
The next movement for the Yellow circle is downward at a cost of 1 cell. This is known as
our cost or g. The best cell/node for movement is therefore determined by f which is a sum of
g+h. In addition we can predefine wall avoidance by identifying cells with 0 movement.
This allows for built in collision detection if the pathfinder is used. The addition of this
technique gave the programmer two kinds of wall avoidance mechanisms in the testbed.
35
Enemy A Enemy B
Worst Path
Optimum Path
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
The first one was through the map file and the second one was through the pathfinder. Either
of these could be used. The idea behind this was to make the testbed more flexible for
future projects and work.
One of the biggest disadvantages of A* is that it is computationally more expensive. The
reason A* was chosen was because of the familiarity with its implementation in C++. An
alternate approach would be to use the Best First Heuristic Search. It is a more simplified and
computationally less expensive than A*. It differs to A* in such a way that instead of
calculating a global path, it checks the next available nodes and expands them further to find
the best path (Davis 2011).
In a game where low performance is a limiting factor, Best First is a better option because it
offers a similar outcome to A* but at a lower computation overhead.
The map file provided agents with a copy of the world indicating obstacles. A second copy
was provided to the pathfinder class which was used by the A* implementation code.
6.2.7 Class Engine
This section focuses on the relation between the previously mentioned classes and the Engine
class in order to provide a stronger understanding of how the complete software works
together. The Engine aptly named, is the biggest and most important class of the entire engine
or testbed. It is responsible for all the inputs, processing and execution in the system. Its
functionality is better explained with the help of the methods used.
Init(): This method is used to initialize the engine. The initialization includes setting up a
rendering window in conjunction with SFML. Init() was also used to create a new object of
the LevelLoader class, required to load the level from the map file. The final method called in
Init() was the LoadContent() method.
LoadContent(): The beginning of this method includes data for the loading of sprite images
and passing them onto the ImageManager class. In the future if sprite animation is required, it
would be more beneficial to include these commands in the respective agent class.
The LevelLoader object created in the init() method is used here to specify which text file to
load for the map data. Note: If multiple levels are required different text files can be used and
a selection can be made between them.
The next content information is the map grid. This and other map information is stored in
respective STL vectors. At this point the engine has all the data from the map file in memory.
The map file is no longer required because continuously reading the map file is extremely
36
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
slow. In fact is the slowest process because it is read from the hard drive. This is where the
significance of data management can be seen. Since the engine has all the data now it can
be passed to whichever class requires it.
For example, a copy of the map is provided to both the pathfinder and the Agent class.
The final task of the LoadContent() method is to create all the agents in the game with the
help of the spawn positions gathered from the map file in one of the earlier steps.
ProcessInput(): The first method in the sequence of the 3 methods which are included in the
Main game loop. This method is used to detect all system events and all user input events. An
example is the movement of agents through keyboard or joypad buttons.
Update(): The second method in the main game loop. This is the global update for all the
agents in the game. It carries out all the individual updates of the agents in conjunction with
the system clock. One thing to highlight here is the distinct sensing and effecting
mechanisms in this method. The sense() method invoked by all the agents is the continuous
sensing for exogenous events and interactions. The call of the update () method calls each of
the individual updates of the agents. As mentioned in chapter 5, this is a good example of a
clean design interface.
RenderFrame(): The final method in the game loop. The graphics of the game are drawn in
this method. Initially, this method starts with the spawn positions of the agents. This position
changes at runtime in accordance with the game logic. This position is passed to the display
command which renders the graphics on screen.
MainLoop(): Calls ProcessInput() , Update() and RenderFrame() Sequentially per unit
execution time.
To finalise this section, it can be seen that the game Engine incorporates a clear flow of data.
It can be clearly seen that the data comes from the file is stored in the engine and forwarded to
respective classes. The design of the engine is also modular. Each new class or section was
added in the course of the implementation as per the requirement. The engine also takes into
account the requirements for a good testbed as stated in Chapter 5. All the characteristics
mentioned were implemented successfully. A lot of time was spent in the design of the
software. This proved to be a very good decision because the expansion of the game was
fairly quick afterwards because of the clean design.
As can be seen this engine is not project specific. Many other 2D based AI projects can be
implemented using the same architecture. There are also enough tools present to implement
37
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
simple 2D games with this architecture. This was what was meant by personalising the
architecture to suit the requirements.
Other classes that have not been mentioned were helper or utility classes and can be seen in
the source code.
6.3 Agent Hierarchy
Figure 15 was incomplete because it did not show the inheritance hierarchy of the Agent
class. This can be seen in figure 18.
Figure 18. Agent Inheritance Hierarchy
Class Agent is the parent class and all other classes have been derived from Agent0 which in
turn is a derived class of Agent. As per the previous section each class has been explained
with the help of its software implementation.
6.3.1 Class Agent
The first thing to note is that the class Agent does not represent an AI agent as per the
definition in the Introduction. To recap this definition, a game agent can be defined as a
38
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
computer controlled human or non-human character or entity. The Agent class can be seen as
more of as an entity rather than an agent. This class represents an agent that is human
controlled i.e. via the keyboard.
The reason for implementing this class was to distinguish between an entity and an agent. The
Agent has been represented by a small car in the game. Considering a remote controlled car, it
complies with given commands of movement. If however, it reaches a wall it cannot move in
that direction because the world does not permit it. The same analogy has been applied to the
Agent. It can be controlled by arrow keys and can move in 4 directions; up, left, down and
right. If however it reaches a wall it cannot move in that direction. This is to show that a
realistic knowledge of the world has been implemented in the testbed. The Agent has this
knowledge of the world but it does not have a thought process to control where to go in the
world. It does what it is told to without a mind of its own. This is the reason that it is an entity
rather than an agent.
In terms of implementation, the Agent class requests a copy of the map grid from the Engine
class. This agent also has default enumerated data types which are DEFAULT, AGENT0,
CIVILIAN, POLICE and ROBBER. DEFAULT refers to the entity or Agent class. This is to
distinguish the name from the Agent0 class. These types are used to represent the type of
agents during interaction with each other. Note: Another characteristic from chapter 5.
Agent also includes methods for movement in 4 directions. In addition to this it also includes
methods to check if a certain direction is free or blocked. This check is only done when the
agent is instructed to move in a certain direction. If there is an obstacle and it is instructed to
move, it will still try to move irrespective of the fact that it knows there is an obstacle there. It
cannot move because it has realistic knowledge of the world.
An important aspect of the design of this class is the 2D coordinates. Since this class was
derived further it was essential to separate the two kinds of coordinates in each agent or
entity. The first x and y coordinates of the agent refer to the logical nodes of the agent. These
logical nodes refer to the map and are values of x and y from the map grid. For example, (5,
5) is a position from the map grid. The second type of coordinates refers to the x and y
coordinates in terms of pixel size. These are used for the rendering of the graphics on screen
in the rendering window. These coordinates are the logical coordinates multiplied by the pixel
size of the window. For example, (5x1024, 5x768) are the render positions of the agents in the
game. These two data types have been separated clearly in the design such that the agent
only deals with its logical coordinates and the game engine deals with the conversion of
the logical coordinates to pixel coordinates. Note: Another example of a clean interface
39
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
The agent also includes virtual methods called sense() and update(). These are the sensory and
effect modules. In this class they are empty but have been overridden in the derived classes as
explained further.
6.3.2 Class Agent0
Agent0 represents the default behaviour of each character agent in the game and derives all
characteristics from the parent Agent class.
Agent0 has been represented in the game by a skeleton sprite. The sense and update methods
of Agent0 were overridden to give the agent some thought. Agent0 exhibits random
movement behaviour across the grid. What is actually happening is that at each node the
agent checks which of the 4 directions are free to move. It then stores all the available
directions and randomly selects between them to move onto the next node. This differs from
the parent class Agent because the Agent0 does not move if there is an obstacle whereas the
parent Agent realises there is an obstacle after the move command has been given by the user.
This shows that Agent0 is now able to sense and effect. It is able to first sense if a
direction is free, make a choice out of all the free directions and then effectively move in
one direction. This is one of the key aspects of agent design. This is a clear
representation of sensing, processing and action. At this point the hierarchy can be
extended further because Agent0 is the first unit of Artificial Intelligence in the testbed. From
here onwards this agent could have been extended to any required behaviour. This is another
example of the ability of the testbed to build many different projects using this architecture.
The drawback here is that Agent0 only interacts with the game world. In this respect the test
bed is said to be incomplete because multi-agent interaction is absent. The next level in the
hierarchy accommodates this requirement.
6.3.3 Class Civilian, Robber and Police
These three classes represent the more sophisticated agents in the game. Each class contains
its own unique methods for sensing and effecting. These methods are overrides of the Agent0
class. If in any instance these methods failed to activate, the default behaviour will be
that of the Agent0 class.
The sense modules of the these classes are continuously active i.e. they are always sensing. If
an agent senses another agent, it is able to distinguish which kind of agent by the enumerated
TYPE that was specified earlier in the inheritance hierarchy. Once an agent has sensed
something it activates specified Boolean flags. These flags are the control variables for the
effect methods.
40
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
The effect methods consist of Finite State Machines (FSM) for the Civilian, Robber and
Police classes. The FSMs have been explained in detail in the next chapter. The transition of
the states in the FSM is dependent on the Boolean flags which are set in the sense method . As
can be seen from this the software has created a link between sensing processing and
effecting for a Multi-Agent Environment . Other characteristics that have been
implemented include the ability of agents to send instructions of messages to each other. For
example, a robber tells a civilian that it is a hostage and the civilian is instructed to follow the
robber.
This chapter can be concluded on the grounds that at this stage in the dissertation all
functional tools of the testbed had been implemented. The testbed was complete but was
missing the gameplay logic. The missing piece of the testbed was the FSM design of the
civilian, robber and police along with the Emotion Architecture. The next chapter focuses on
the gameplay design and includes detailed information about the inner workings of each
agent. This understanding is necessary as it forms the basis for the implementation of the
emotion architecture.
41
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Chapter 7 Gameplay and DesignChapter 6 focused on the low level design of the testbed. This chapter contains information
about the behavioural design of the game and the agents. It explains the rules of the game and
provides a clear understanding of the scope and limitations of the agents. In addition to this it
focuses on the biggest aspect of the testbed, the FSM design of the character agents. The FSM
design is closely linked to the gameplay of the system as this is the area where most of the
game logic was used.
7.1 Detailed Storyline
A brief description of the story was given in chapter 4. There are various details that were
omitted and are highlighted in this section. To put the gameplay into perspective, the game
window consists of a top down view of the bank. At any given time the interactions of all the
agents and their movement can be viewed by the user. Other things included in the scene
included various furniture and devices found in a bank. These are pictorial representations of
obstacles in the game. An easily distinguishable vault has also been shown in the scene.
These are labelled in figure 19.
This view represents the action scenario of the game. Action scenario represents the current
time in which an event has occurred i.e. bank robbery. There are three agents involved in the
scenario. The first are robbers which have invaded the bank and want to steal money from the
vault. Second are the civilians which are present in the bank. Finally, the task of the
policemen is to warn civilians and arrest the bank robbers. This completes the game scenario.
Once the simulation occurs, these actions are executed and continue until all the agents reach
their respective final goals. These are explained in more detail in the upcoming FSM design
section.
It is useful to note that there is a broader overview of the game scenario that had to be taken
into consideration. This is explained as follows.
It was mentioned that a bank robbery had occurred but not what event initiated the robbery.
One such event would be the triggering of the alarm in the bank. Another would be robbers
posing as civilians which suddenly show their presence.
42
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Another situation to consider is the presence of the police. In a broader perspective, the police
would have to be alerted by the alarm or someone in the bank.
Finally the presence of civilians can be explained by dividing the civilians into further
categories such as employees and customers.
These were some of the many possible precursors of in scenario that needed to be taken into
consideration while designing the test bed. For this reason they have been mentioned in this
section but have not been shown in the implementation. This was to place emphasis on the
functionality of the agents in the test bed and not on the origin of actions and precursors.
However, in terms of research, these are important factors that need to be taken into account
on order to present a realistic and robust game scenario.
Figure 19 Game/Test bed Window
7.2 Control Factors and rules for the Testbed
The test bed was designed to act as a controlled and closed simulation environment. For the
agents to comply with the scenario of a bank robbery, certain rules and limitations had to be
43
VAULT
POLICE CIVILIAN
ROBBER
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
imposed. These can be seen as guidelines that form a controlled environment for specific test
bed experimentation.
A different scenario or test bed will have its own set of rules. As defined in chapter 6, the base
architecture of the software can remain the same. It is the scenario and the rules imposed in
the scenario that distinguishes one implementation from another. The following set of rules
highlight this project and its limitations.
i. The scope of each agent is limited in the area of the world that can be seen on screen. It is a
bounded region and the agent cannot exist outside it. The map can be extended and changed
as required but should continue to impose a boundary around it to prevent agents from leaving
the scope of the game.
ii. The start positions of all the agents need to be specified in the map file beforehand. The
agents will always start from this position.
iii. When designing the map, it needs to be insured that critical notes are not blocked by walls on
all sides. This is to insure that the required path is available to the Pathfinder. Critical nodes
should not require diagonal movement because the path finder can only move in 90 degree
directions.
iv. An agent can be interrupted during state transitions but once it has achieved the final state in
its cycle, the game is over for that agent. The final state was decided as part of the FSM
design. At this point the Agent must be reset to its initialised state and start position.
It should be noted that a testbed should be as flexible as possible. Imposing too many
limitations on a testbed defeats the purpose of experimentation. The rules on the individual
agents were enforced via the FSM design. These rules are not strict because their limitations
can be broken through multi-agent interactions. This may lead to behaviour that was not
accounted for in the design and may prove beneficial in analysis during experimentation. If
the output of the interaction with another agent is unsuitable, control parameters can be used
to adjust this. This was used to control behaviour when one agent communicated with
another. An example is the case of the robber taking the civilian hostage. This practice is
evident in the next section which focuses on the FSM design.
44
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
7.3 Finite State Machine Design
Before an AI technique can be implemented, its suitability needs to be analysed. This has
been done with the help of a requirements analysis table (Davis 2011) shown in Appendix C.
The table shows suitability against different criteria shown in the first column. It can be seen
that State Machines are the most effective technique.
In relation to the testbed,
Speed and efficiency is a very important as it is a real-time application.
The design needs to be robust because each agent has specific goals and multi agent interaction added randomness to the test bed.
In terms of believability or effectiveness, this testbed requires effectiveness.
The design of the agents needs to be clear. As stated in chapter 5 a clean interface is one of the most important qualities in a good testbed.
Variety was not a very important requirement in the test bed and was ignored.
All the agents needed to consistently achieve their goals in order to conform to the storyline.
Scalability was very important to allow the addition of more characteristics to the agents as the design progressed.
State machines met all these requirements and therefore were chosen for the design.
Finite state machines are state machine albeit with a fixed (‘finite’) number of states,
7.4 Agent Behaviour and Design
The main focus of this section is on the individual Agent FSM designs. Some of the
behaviour of the base agents has also been mentioned briefly. A brief recap of some of the
basic qualities of the agents has also been mentioned.
7.4.1 Base Agent
The base agent is shown in Appendix D. It has been represented analogous to a remote
controlled car because it has no mind of its own. This is the manual agent or entity that has
been mentioned in section 6.3.1 and belongs to the Agent class. It is controlled by the arrow
keys and has no relevance in the overall gameplay. For this reason it does not have its own
state machine.
45
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
7.4.2 Agent0
The Agent0 has been represented by a skeleton shown in Appendix D. This is to signify that it
is a kind of ‘bare bone’ agent. The on screen behaviour of this agent is, checking for free
spaces in four directions. The next step is to select randomly between any available spaces
and move in that direction. This is the default behaviour for the civilians, police and robbers
which are in turn derived from the class. Agent0 contains sense and effect methods which are
in turn overridden by the derived agents. Agent0 also has no involvement in the game. It is
shown on the game screen to demonstrate the agent hierarchy. It also does not employ the use
of a state machine as it is not part of the game scenario.
7.4.3 Police
The architecture of the police was the simplest out of the 3 agents.
The police were divided into 5 states. These were {IDLE, SEEK, WARN, ARREST AND FOLLOW}.
The overall behaviour of the police is explained through the sensory, processing and effecting modules.
Police icon is shown in Appendix D
Police Goals:
As mentioned previously the police were already present in the game at the time of
initialization. It was assumed that they were informed of the robbery prior to the simulation.
The main goals of the police were
i. Enter the bank and search for insurgents
ii. Prevent the robbery by confronting and arresting the robbers
iii. Ensure the safety of civilians
The implementation of these goals is again, a part of the FSM.
Sensory Module:
There were 3 things that the police were required to sense in the game. The first was the
environment and the location of obstacles. This was derived from the parent classes. The
other two things were the civilians and robbers. Note: Concurrent with human behaviour,
the sense module is constantly active and checking for any changes in the environment.
46
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
The sensing can be both voluntary and involuntary. In the case of the police, the sensing
was voluntary because the goals required a search for other agents in the environment. The
main purpose of the sense module was to gather data from the environment. An example of
data is the type of agent sensed.
Pseudo Code for Sensory Module
Processing module:
In terms of C++, the processing code is included in the sense () method. This was done for
ease of programming. In terms of implementation the processing is carried out after sensing.
Where sensing is the gathering of data, processing is the interpretation of data. Processing
decides what the overall state of the world is depending on the sense data. An example of
processing is evaluating whether a civilian is hostage or not. Another example is evaluating
whether a robber has a hostage or not.
Pseudo Code for Processing Module
Effect Module:
47
Police::sense() {
Switch(Get Agent type)
case CIVILIAN:
{
sense civilian;
init pointer to civilian;
case CIVILIAN:
if(a civilian is in range and is not warned)
{
alertcivilian = true //alert civilian is a boolean flag which is used for the next stage
}
Case ROBBER:
{
if(robber has a hostage ){
do nothing
}else if (robber does not have a hostage){
The robber can be arrested;}
}
case ROBBER:
{
sense robber;
init pointer to robber ;
}
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
In terms of C++ code the effecting module has been represented by the update loop of each
agent.
The FSM is part of the effect module which in turn was responsible for the final actions of the
agents. The states are selected according to the output of the processing stage.
The police FSM is shown in Figure 20.
Figure 20 Police FSM
This has been explained in conjunction with some of the implementation in code.
The initial state of the Police is IDLE. It then checks if any other agent is in its line of sight. If
not It processes that no agent is present and the state is changed to SEEK.
In this state the police patrol from door to door of each room searching for civilians or
robbers. If a civilian is detected in sight distance, it processes if it is a hostage or not. If not, it
changes its state to WARN and instructs the civilian to run out of the bank.
Note: The interaction here between the Civilian and Police. The information gathering
at this stage has been extended from the world to other agents.
48
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
If the civilian is a hostage it cannot do anything and continues in the SEEK state. If a robber
is detected in sight distance, it checks whether it has a hostage or not. If not then the state
changes to FOLLOW in which case the policemen chases after the robber. Once it reaches the
ROBBER the state changes to ARREST.
If the ROBBER escapes out of SIGHT distance, the state changes back to SEEK. In ARREST
a robber is taken to jail, which is also indicated on one corner of the map. After this the Police
revert to the IDLE state and the whole process starts all over again.
7.4.4 Civilian
The civilian is shown in Appendix D.
The states for the civilian were {IDLE, WANDER, FLEE, FOLLOW, SAFE}
Civilian Goals:
The civilian goals are as follows
i. Move from one point to another, to simulate daily activities in a bankii. Flee to safety iii. Involuntary goal of following a robber when taken hostage
Sensory Module:
The sensory module for the civilian differs from the police because it is involuntary when
checking for agents. The sense code for the civilian senses only the robber because one of its
goals is to run when a robber is in sight. Instead of sensing the police the civilian
acknowledges a message of warning to run out from the bank.
The civilian was not required to sense the police because there was no requirement to initiate
an interaction by the civilian.
For this reason, the civilian was designed to receive messages from both agents. These
messages were designed to interrupt a fulfilment of a desired final state by the civilian. For
example if a civilian is fleeing, a message can be sent to follow as a hostage. The Pseudo
code for the sense module is therefore fairly simple for the civilian as compared to the effect
module.
49
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Pseudo Code for Sensory Module
Processing Module:
The processing stage of the civilian indicates to the next stage that a robber has been sensed.
Along with this it initiates a pointer to the robber which says that ‘the civilian can now
acknowledge messages from the robber’. How it interprets the messages to change states is
shown in the next section.
Pseudo Code for Processing Module
50
Civilian::sense() {
Switch(Get Agent type)
case Police:
{
sense civilian;
init pointer to robber ;
Case ROBBER:
if(a robber is in sight range)
{
_senseRobber = true //alert civilian is a boolean flag which is used for the next stage
_robber = rob //Initiate a pointer to the robber class so that a robber can send messages.
}
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Effect Module:
The civilian behaviour is an example of partial fulfilment of goal states. Another
characteristic derived from chapter 5. The game engine or testbed allows for easy interaction
between the agents. There is support to allow for one agent to manipulate the workings of
another agent. The civilian was an entity that required such behaviour. A clean design
allowed this without changing any previous code.
The civilian FSM can be seen in figure 21.
Figure 21 Civilian FSM
As with the police, the initial state is IDLE. From this state the civilian can move to three
states FLEE, WANDER and FOLLOW. If the civilian does not detect any robbers in its line
of sight, it will go to a wander state during which is just moves randomly around the
environment.
51
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
From the IDLE and WANDER state, if a civilian sees a robber in its line of sight, the state
will change to FLEE.
If the civilian reaches the safe point, the state becomes SAFE and the game ends for the
civilian.
The most important state in the civilian design is the FOLLOW state. It can be seen that a
civilian can go to this state from all the available states. This is the state in which the civilian
receives a message from the robber that instructs it to FOLLOW. So while it is a fulfilment of
the FOLLOW state, it occurs as a partial fulfilment of all the other states.
For this reason, the warning message from the police to the civilian has not been implemented
in this way because the police cannot warn the civilians whenever they want whereas the
robbers can take a civilian hostage while it is in any state.
For a broader scope once a robber escapes, the civilian is released and it goes to FLEE and
finally to safe when the game ends for the CIVILIAN.
Note that the hostage command is given to the civilian only when the robber reaches the
position of the civilian. This is to demonstrate that the robber has to chase the civilian to take
it hostage.
7.4.5 Robber
The robber was the most complicated design of all the agents. The robber can be seen as a combination of both the civilian and the police classes. Like the policeman it continues on its goal path but at some instant it can be interrupted causing partial satisfaction of goal states.
Where it differs from the civilian is that it cannot be interrupted from all its states. The robber icon is also shown in Appendix D.
The robber states were {IDLE, FLEE, HEIST, ESCAPE, CATCH, SAFE, ARRESTED}
It can be seen that having an FSM as large as this is very hard to manager. For this reason the behaviour of the robber was broken down into two parts. The states can be represented as {IDLE, FLEE, HEIST} leading to {ESCAPE, CATCH, SAFE, ARRESTED}.
The robber goals were
i. Reach the bank vaultii. When bank vault reached, take a hostage to ensure the escapeiii. Avoid all police encounter unless bank vault reached, in which case it tries to escape as
quickly as possible.
52
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Before explaining the robber’s sensory module, it needs to be made clear that not all aspects of agent behaviour could be made flexible. The reason for this is that certain behaviours created dominance of one agent over the other. This point has been elaborated in the FSM section for the robber.
Sensory Module:
The robbers had previous knowledge of the bank vault location from the start. This was because of the pathfinder class. In terms of sensing the robber is the same as the police.
Pseudo Code for Sensory Module
Processing Module:
Due to its complexity, the robber also has the most data to process.
The first thing the robber has in common with all agents is that it senses the agents on the basis of sight distance. If in the processing stage, the sensed agent is still in sight it executes the next command.
The robber had interactions with both civilians and the police. For this reason pointers are initiated to both those agents in order to pass or receive messages. The robber passes a message to the civilian and receives a message from the police.
The robber also has to check whether a civilian has been taken hostage by another robber. In this case it will not attempt to take that civilian hostage.
53
Robber::sense() {
Switch(Get Agent type)
case CIVILIAN:
{
sense civilian;
init pointer to civilian;
}
case POLICE:
{
sense police;
init pointer to civ ;
}
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Pseudo Code for Processing Module
Effect Module:
The robber FSM can be seen in figure 22.
The robber proceeds from the IDLE state to the HEIST state during which it moves towards the bank vault. If it detects a policeman at this time it moves to the FLEE state and tries to run back to the start point. This is repeated until the robber reaches the bank vault. As mentioned earlier it can be seen that these 3 states are sectioned off from the other states in the FSM.
Once the robber reaches the vault it moves into the second section of the FSM. At this point it tries to ESCAPE the bank. During this process if it senses a civilian in sight distance, it proceeds to the CATCH state, takes the civilian hostage and returns to the ESCAPE state and continuous to move towards the exit. If it has a hostage it is guaranteed an escape because the police cannot arrest the robbers in that condition. If it does not have a hostage the robber can be interrupted through a partial satisfaction of its goal states into an ARRESTED state which ultimately leads to the game ending for the robber. The FSM was divided so that the taking of a hostage is only possible in the second half of the FSM. This was to give the police a fair chance. The police can intercept the robber from any state and force it into an ARRESTED state. The police was designed to arrest the robber on reaching the same cell position as the robber.
54
Case:POLICE
If(police is in sight){
_sensePolice = true ;
_police = pol ; // pointer to police
}
Case: CIVILIAN
if(civilian is in sight and not a hostage)
_senseCivilian = true ;
_civilian = civ ;
}else if (civilian is hostage) {
_caughtHostage = true ; //output flag which leads to robber ignoring a hostage in FSM
}
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Figure 22 Robber FSM
To fill in any gaps this chapter has left out it would be beneficial to recap what has been said.
The core behavioural design of the agents works in 3 sections or modules. The first module senses the environment and gathers all the data. The second processing module sets conditional Boolean flags on the basis of this data. These Boolean flags are then used to arbitrate over the FSM in the third effect module. For a clearer understanding inspection of the source code included with this report should shed further light on the subject.
55
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Chapter 8 Emotion ArchitectureOnce the FSM had been implemented, the next step was emotions. At this point a lot had been achieved by the FSM architecture alone.
It is worth noting here that the FSM architecture is somewhat analogous to what is known as a Belief-Desire-Intention Architecture (BDI). Due to its similarity to a reactive goal-based architecture, the BDI architecture can be used to contextualize, some of the considered emotion architectures.
The research on this topic has been on the work of Haddadi and SunderMeyer (O’Hare and Jennings 1996).
Belief can be described as the knowledge the agent has of its environment. In the case of this project, the sense module is somewhat similar to gathering knowledge of the environment.
Desires can refer to the possible set of actions an agent might want to carry out. For example, the list of goals of the agents.
Intention refers to the selection of one guaranteed action out of the desires which will result in an action. For example, State selection and action.
One way of implementing emotions would be for the emotions to affect the desires of an agent. Once an emotion is activated it will only allow a possible set of desires, which in turn will result in specific actions.
In the context of this project this would mean that emotions would be implemented in the processing module and depending on the kind of emotion only certain states would be available in the FSM.
An alternate approach to this implementation model is to allow emotions to govern the
decision making process of the agent (Shihab 2009). Again in the context of this project, this
model will allow emotions to affect the state changes of the FSM. Where the previous BDI
approach would be to put emotions one layer above the Desires, this approach would put it
one layer above a set of intentions available for every desire. This approach assumes that an
agent starts with an emotion. Depending on the emotion, the agent makes a decision, which
leads to an action. The emotion also varies according to the environment and other agents and
their interactions. It can also be affected by a choice of a previous action/or decision.
Even though this approach seems ideal, there was one problem with it. It assumes that an
agent has to start with an emotion and is constantly influenced by this. The initial state of an
agent is dependent on an initialised value of emotion. In the context of this game this is
56
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
unwanted in many situations. It would be more interesting if the effects of emotion can be
shown in comparison to the reactive architecture rather than taking full control of the design.
An approach had to be found that was more suitable for reactive goal based architectures. One
such approach is known as Affective Action Selection and Behaviour Arbitration (AASBA)
(Scheutz 2002). The author’s original implementation is based on behaviour arbitration in
autonomous robots.
This scheme is similar to the first approach mentioned in relation to BDI architectures. The
scheme involves creating a new layer of affective states above the current reactive
architecture. These states are used to arbitrate the behaviours of the agents. The affective
states are designed with their own activation level. If one affective state is active, it controls
the corresponding behaviours in the system.
The means of activation of can vary over different systems and test beds. Scheutz proposed
that affective states be modelled with numerical values and also defined a formula for the
activation of each state. The formula is overly complex and irrelevant for this project but
Scheutz’s research paper can be looked at for more knowledge. The affective states then
compete among themselves for superiority in the system. They form what is called a
“competitive cluster”. One arbitration scheme can be that the affective state with the highest
value controls the behaviour of the system .Events in the system increase the value of one
state while decreasing another. Certain events can also reset the values of both states to 0 so
that the default architecture can behave without the effect of these states. This was beneficial
and also solved the problem that Shihab’s model posed. It allowed the possibility of turning
off emotions in the system.
One problem it had was that since it was a complete external layer of arbitration, it could not
be modified by the agent. This was a serious problem because the extended specification of
the project required agents with control over their emotions and agents with personalities.
It was decided that all agents would start with an emotion of a numerical value 0. This means
that the emotion is off. Emotions would be elicited according to the model in section 3.4.
Numerical threshold levels would be specified for each emotion. Events in the system would
cause an increase in the numerical value. Once the increase reached the threshold value, the
emotion would be active and the FSM in the update method would be controlled according to
the emotion in the processing stage. If the arbitration is part of the processing module, over
time the agent can alter the emotions in order to form a personality.
57
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
This scheme is a combination of the first BDI technique and Scheutz’s technique. While it
puts emotions in the processing module, it still allows for internal control of the arbitration
mechanism. In a testbed, these are good characteristics because the arbitration mechanism
requires complete control. Another added advantage is that emotions can be turned off in the
system during run time allowing the default reactive architecture to take over.
8.1 Civilian emotion model From section 4.3 the most obvious emotion for the civilian was fear. The behaviour of the
civilian running away from the robber was already a representation of fear. This had to be
extended in some way to show a different emotion effect for fear.
It was decided to use a numerical value of 0 for the fear. If a civilian is warned by the police
the fear increases to 0.5 and the civilian runs out of the bank. If the civilian senses a robber
the fear goes to 1 which is the activation level. When the emotion is activated a Boolean flag
in the process module is set to true which then affects the FSM in the effect/update stage.
If an event occurs for example, a policeman then senses a civilian it will lower the level of
fear back to 0.5 allowing the civilian to run away.
Since the civilian does not have many available actions the activation level is basically on or
off. In a different situation the emotion can be incremented gradually with multiple activation
levels and actions.
8.2 Robber emotion modelThe emotion chosen for the robber was anger. Deriving from section 4.3, when a robber is
trying to reach the vault and is constantly blocked by the police, it experiences anger. This is
because the police are blocking the robbers from achieving their goals. If the threshold for
anger is activated a robber chooses to ignore the threat of the police in order to achieve its
goal.
This is an example of what is known as the Nash equilibrium (Nash 1950). What this means
is that without anger the robber and the police are in equilibrium. None of the two can make a
move against each other that will result in an advantage. When influenced by anger the robber
chooses to momentarily gain a disadvantage by risking arrest in the hope of gaining an
advantage later i.e. getting to the vault quicker.
58
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
8.3 Police Emotion model The ideal emotion for the police was disgust. If a policeman was part of the robbery and
allowed the robbers to escape the other policeman would feel disgust. On the activation of
disgust, the police would be less involved in their duties. Instead of SEEK, they would
prominently stay IDLE allowing the robbers to escape easily.
59
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Chapter 9 Conclusion 9.1 Critical Evaluation
Overall, the project provided a wide range of knowledge on many different topics in the field
of games programming and AI design. Invaluable experience was gained which will be
helpful for future projects. I was very happy with my software architecture because even
though it has room for improvement, it does not require complete redesign. A lot of time was
spent on the game engine and for good reason. The game engine took around 4 weeks to
implement. This was a gamble which paid off as the rest of the software implementation was
quicker because of the clean design of the engine.
I felt that the research of the project was sufficient but pinpointing the exact course of action
was the toughest task during the implementation. The multiple topics involved required strict
planning and time management. This was one of the major areas where I had problems. I
believe that I could have utilized my time better for a more efficient outcome.
The biggest drawback was that I was not able to do sufficient testing of the emotion models.
For this reason it has not been mentioned in the report. All the implementation and theory that
has been mentioned was successfully implemented and the explanations were given on
account of the success of the implementation. I cannot however, successfully provided
exhaustive evidence of an implementation for a commercial video game.
7.2 Future Work
Future work involves the refining of the emotion model. The model needs to be extended by implementing more than one emotion successfully.
Other work includes the implementation of multiple emotions at the same time. This was part of the extended specification of the project.
Finally majority of the future work involves refining the current project. If there was more time available I would spend it in improving the standards of the testbed and if time permitted implementing the extended specification.
60
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
7.3 Conclusions
The required outcome of the dissertation was to find out how emotions can make games more
interesting. In terms of theoretical design and research this requirement was achieved. I am
satisfied with the documentation of the project.
I am also satisfied with the design of the software but the fact that I could not refine it to
completion is disappointing.
If I were to start again, I would pay more attention to planning and time management.
In terms of learning outcomes I learnt to successfully implement a hierarchical and systematic
Agent design. I was able to improve on my programming experience in C++. I was also able
to build a small game engine that I intend to use for my next 2D game project.
I learnt to implement some of the basic techniques of AI design. FSMs being one of these
techniques, was the most logical aspect of the design.
The most important thing I learnt was top think in terms of an AI programmer. The
knowledge gained in this report will provide a valuable base for work in the games industry.
61
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
References and Bibliography
1. "Affective Computing" MIT Technical Report #321 (Abstract), 1995
2. .Cannon-Bard theory." Encyclopædia Britannica. Encyclopædia Britannica Online.
Encyclopædia Britannica, 2011. Web. 03 Aug. 2011
3. .Hadadi, A and SunderMeyer,K, 1996 , Foundations of Distributed Artificial
Intelligence ,Wiley-IEEE
4. Agre , PE and Chapman, D, 1987, Pengi:An implementation of a theory of activity, MIT
Intelligence laboratory.
5. Andronache, V. and Scheutz, M. ContentionScheduling: A Viable Action-Selection
6. Bourg, D M, 2004, AI for Game Developers, USA: O’Reilly Publishers
7. Boyle B, Connoly M, Hainey T, 2011, The role of psychology in understanding the impact
of computer games, University of West Scotland
8. Brams J. Steven,1995 Game Theory and Emotions, New York University
9. Buckland, M, 2004, Programming Game AI by Examples, USA: Wordware Publishing
Inc.
10. Champandard , A J, 2003, AI Game Development,USA, New Riders
11. Darwin, Charles (1872). The Expression of Emotions in Man and Animals.
12. Davis, D N, 2000, Minds have personalities-Emotion is the core, AISB2000-AIand
Society, Univrsity of Birmingham.
13. Davis, DN , 2011 , Lecture Notes : Advanced Rendering and AI , University of Hull
14. de Sousa, Ronald. (2010). Emotion. Available:
http://plato.stanford.edu/archives/spr2010/entries/emotion/. Last accessed 15th September
2011.
15. Encyclopaedia Britannica. (2011). Cannon-Bard theory. Available:
http://www.britannica.com/EBchecked/topic/92792/Cannon-Bard-theory. Last accessed
15th September 2011
16. Feist G and Rosenberg, E (2009). Psychology: Making Connections. England: McGraw-
Hill Humanities/Social Sciences/Languages.
17. Feist, Gregory and Rosenberg, Erika, , 2009, Psychology: Making Connections ,
McGraw-Hill Higher Education
63
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
18. Gaulin, Steven J. C. and Donald H. McBurney (2003). Evolutionary Psychology. 2nd ed.
US: Prentice Hall. p 121-142.
19. Hanks S , Pollack E. M, Cohen R.P, 1993, Benchmarks, Test Beds, Controlled
Experimentation and the Design of Agent Architectures, AI magazine volume 14
20. Huitt, W. (2003). THE AFFECTIVE SYSTEM.
Available:http://www.edpsycinteractive.org/topics/affect/affsys.html. Last accessed 15th
September 2011
21. James, William. (1884). “What is an emotion?” Mind, 9, 188-205.
22. Lees, Ma , Logana, B,Theodoropoulosb 2005, Agents, Games and HLA, a University of
Nottingham, b University of Birmingham
23. Lewis, Suzanne C, Computational Models of Emotion and Affect, PhD dissertation,
University of Hull
24. Mechanism for Robotics? In Proceedings of MAICS 2002, AAAI Press, 2002.
25. Millington, I and Funge, J,2004, Artificial Intelligence for Games 2nd Ed.,USA , Morgam
Kauffman Publishers
26. Nash, John (1950) "Equilibrium points in n-person games" Proceedings of the National
Academy of Sciences 36(1):48-49.
27. Nilsson, Nils J, 2000, Artificial Intelligence: A New Synthesis, San Francisco: Morgan
Kauffman Publishers
28. Russell, Stuart J.; Norvig, Peter, 2003. Artificial Intelligence: A Modern Approach. 2nd
ed. New Jersey: Prentice Hall.
29. Schachter, S. and Singer, J. E. , 1962 Cognitive, social and physiological
determinants of emotional states, Psychological Review, 69, 379-399
30. Scheutz, M, 2002, Affective action selection and behaviour arbitration for Autonomous
Robots, University of Notre Dame
31. Shihab K.,2009, Emotional Agents in Computer Games,International Journal of
Computers
64
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Appendix A Original Specification
Affective computation is starting to attract considerable interest in a number of computational
domains. One such area is the use of affect in computational agents that allow more involving
interactive game playing. This project will address how a computational model of emotion
(affect) can be used to make better computer games. A relatively simple scenario (e.g.
predator-prey) can be used to investigate the effect of using different emotions or affect and
perhaps different computational models of emotion or affect.
66
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Appendix B Updated Specification
The updated specification required the selection of a valid computational model of emotion
from the field of psychology. This model of emotion was to be translated into a software test
bed. The software could be designed in any programming language. A scenario had to be
implemented and a storyline had to be selected that could demonstrate emotions in video
games. There was a choice on the architecture of the agent behaviour. Reactive goal based
agents were suggested. Finite State machine were to be used to implement a game that
consisted of reactive goal based agents. Once the default behaviour of the game was achieved,
a model of emotion was to be implemented as a secondary layer on top of the game FSM.
Overall this was to be used to research how Emotions can make games more interesting.
The extended specification included representing agents with personalities and multiple
emotions.
67
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Appendix D Character Icons
Base Agent
Agent0
Police
Civilian
Robber
69
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Appendix E Project Plan
WEEK SCHEDULED TASK UNSCHEDULED TASK
Week 106/07/11
- Concrete storyline for integration with test bed- Software GUI design using SFML library and C++- Agent(class0) state machine design- Define critical scenarios in storyline-Define quantitative and qualitative evaluation factors for the project
- Refining of software test bed- Document work in draft form
Week 213/07/11
- software implementation of class0 agent FSM- -Test bed complete with class0 agent
- Further refining of test bed-Document work in draft form
Week 320/07/201
1
- Accommodate previous delays- Ensure basic FSM functionality- Extend Class to different roles - Finalise documentation for test bed and background theory
- Basic FSM functionality is mandatory .Extensibility of the design needs to be ensured before progressing further
Week 427/07/201
1
- Implement the 3 types of game charactersCivilians, terrorists and 3 tactical team unitsImplement basic character movementDefine individual character type goalsExtend basic FSM to goal based reactive-Implement testing methodologies-Start draft final report structure
- Integrate goals with storyline- Identify individual FSM design for each agent
Week 503/08/201
1
- Accommodate previous delays-Class1a agent FSM design and state diagram-Class1a class diagram-extend class0 diagramCode implementation clas1a agent
- Finalise list of goals to be implemented example attack rescue, kill, take hostage etc.
70
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
-test class0, class1 and class 1a agents-finish final report structure
Week610/08/201
1
-Continue code implementation-Implement different goal classes for agents-Testing and debugging-Single goal Finite state per agent- Finalise documentation and integrate into draft report
- Complete documentation of all work done until this point- Testing and debugging
Week717/08/201
1
WEEK SCHEDULED TASK UNSCHEDULED TASK
Week 824/08/201
1
- Test and analysis emotions in applicable agent-Document test results-Ensure whether emotions are being demonstratedDocument check-Finalise documentation Draft report complete
- Edit test bed to ensure emotions are being demonstrated
Week 91/09/2011
Ensure all applicable emotion and goal scenarios are functioningComplete software for demonstration
Week 108/09/2011
Complete remaining documentationFinalise report submission
71
September 16, 2011
EMOTIONAL AGENTS IN GAME PLAYING
Appendix F Considerations
Legal Considerations
All source code used in the project has been designed from the ground up. If any excerpts of source code have been unknowingly used they have been used from a source that is open and free for all. No copyright media or content has been used in this dissertation.
Ethical Considerations
There are no ethical issues involved in this project.
Social Considerations
There are no people involved in the evaluation of the project
Professional Considerations
An attempt has been made in the best respect to cite all information sources to the best of my ability. The SFML software library used for this project is open source and free for all. All the source code used was written either throughout the course of the degree or the duration of the dissertation period.
73