Individual Project - Final Report

School of Computing, Electronics and Mathematics

PRCO303Final Stage Computing Project

BSc (Hons) Computing & Games Development

Steven Hooper (10398658)

Using Bayesian networks for enemy decision-making in a 2.5D side-scrolling game.

2014/2015

AcknowledgementsA special thanks to:

My close family who have supported me throughout my three years of study.

The Interactive Systems Studio team for their assistance throughout the course, their willingness to help outon a variety of subject matters and their help with helping ideas come to fruition.

My supervisor, Serafim Rodrigues for his guidance and direction regarding the artifical intelligence part of this project.

Those who help myself and others online through tutorials and discussions on websites such as Stack Overflow and CodeProject, offering comprehensive solutions to a wide array of programming problems.

Using Bayesian networks for enemy decision-making in a 2.5D side-scrolling game. 2

Abstract This is a project of two parts, one being a framework for creating and using Bayesian networks to serve as a probability model for making decisions and the other being a game developed in the Unity3D game engine to demonstrate this through a combat system. The enemies decisions are derived from the developed frameworks network.

A Bayesian Network is a probabilistic graphical model consisting of conditional dependencies between nodes, in essence it gives probabilities for queries given partial evidence.

I have aimed to create a reactive combat experience in a side-scrolling game, remedying the issue of stagnant enemy behaviours. I have focused on enabling the enemy to predict the players next move and react appropriately and by doing this I aim to strike a balance between difficulty and enjoyment.

My hope for this system is to reward players for thinking about the moves they take when fighting enemies and to provide a high skill ceiling for those who are interested in involving themselves in the system and learn how to master it.

The game itself looks to provide a visual aesthetic to set it apart from other games in the genre by drawing inspiration from a traditional Japanese art style. This provides users with a visually enticing experience along with the depth of an inspired combat system. To aid this there are fully animated game agents which help make the actions between the player and enemies more apparent.

Whilst the project is intended as a personal learning experience for both game design and a machine learning technique completely new to me a tool has been developed which can be used in other applications.

The tool constructs Bayesian networks from XML and allows users to perform inference, it is applicable for awide range of applications with minimal effort required to make use of it.

I found Bayesian networks to be a very promising way of creating adaptive behaviours though the uses of these networks are limited. In terms of games; most of the environment, game engine and game entities and the information they contain are already set up so inference isn't very helpful however the model can be used for entities not inherently part of a game. An example being the player whose situation can be analyzed to measure intent; this is estimating what the player may do given their current situation.

I found this project to be a worthwhile learning experience which has familiarized myself with topics I have had little to no understanding of and has allowed me to develop a usable tool with a wide scope of usage and game to demonstrate adaptive enemy behaviour.


Table of ContentsCover page 1

Acknowledgements 2

Abstract 3

1. Introduction 6

2. Background 8

3. Objectives 9

4. Tools 104.1 Game engine 104.2 Programming software 124.3 Media software 124.4 Project management software 13

5. Method of approach 135.1 Research 135.2 Languages and libraries 16

5.2.1 C# 165.2.2 XML 165.2.3 Unity engine libraries 16

5.3 Deployment 165.4 Project management 17

5.4.1 Design process 175.4.2 Version control 195.4.3 User story tracking 20

6. Target market 21

7. System design – Decision network 227.1 Features 227.2 System architecture 227.3 Program flow 28

7.3.1 Parsing XML to network 287.3.2 Calculating inference 29

7.4 Class descriptions and relationships 307.4.1 NetworkNode 307.4.2 BayesianNetwork 307.4.3 Inference 317.4.5 Elimintation 317.4.6 Bucket 317.4.7 DemoNetworkForm 31


8. Game design – Beyond the Mountains 328.1 Features and mechanics 328.2 Class descriptions and relationships 33

8.2.1 Unity's MonoBehaviour class 338.2.2 Script folder hierarchy 338.2.3 Camera class: Culling 338.2.4 Camera class: FollowPlayer 338.2.5 Core class: CombatController 338.2.6 Core class: MainController 348.2.7 Core class: StatBar 348.2.8 Decision network class: Selection 348.2.9 Enemies class: EnemyController 348.2.10 GUI class: GUIController 358.2.11 Other class: AnimTriggers 358.2.12 Other class: BackgroundScroller 368.2.13 Other class: HashIDs 368.2.14 Other class: ParticleTrailBehaviour 368.2.15 Player class: PlayerAnimator 368.2.16 Player class: PlayerInput 378.2.17 Player class: PlayerStatistics 378.2.18 Third-party scripts and shaders 37

8.3 Other assets 378.3.1 Asset hierarchy 378.3.2 Animations 388.3.3 Fonts 388.3.4 Materials 398.3.5 Models 398.3.6 Particles 408.3.7 Prefabs 408.3.8 Scenes 408.3.9 Sound 408.3.10 Textures 40

8.4 Game flow 418.4.1 Title screen 418.4.2 User input and combat 42

8.5 Controls 438.5.1 Direct controls 438.5.2 Indirect controls 43

9. Testing 449.1 Self-testing 449.2 User feedback 459.3 Optimization 46

10. Risk management 47

11. Constraints and limitations 48

12. Critical Evaluation 49

13. Conclusion 50


14. References 51

Appendices 52Appendix 1: User guide 53Appendix 2: Project initiation document 55Appendix 3: Interim report 1 62Appenidx 4: Interim report 2 69Appendix 5: Supervisor meetings 81Appendix 6: User feedback 86


1. IntroductionThis report aims to provide a thorough look into the framework and game developed alongside other projects between early October to the end of April.

Major points which will be discussed include:

• The approach taken in planning and development• The design and architecture of the provided systems• The game design process used throughout• Project requirements• Target market and intended uses• Risk management• Constraints and limitations• Evaluation of the provided systems• Self and user testing• Learning outcomes

The interim submissions and other relevant documentation leading up to the creation of the final report areattached either as appendices or submitted as additional material and will be referenced throughout the report.

It is recommended to read through the content provided in the appendices and the disc copies containing the project and its development for further understanding.


2. BackgroundOver the course of this programme I have found two areas of study which particularly interest me, these being the development of games as one of the strongest entertainment mediums and artificial intelligence which continues to surprise and excite people all over the world.

Because of this and the fact they mold together so well it seemed appropriate to begin a project under these two fields.

Last year I looked into and had particular interest in evolutionary computation, the practice of simulating evolution in software by having 'generations' of entities which evolve continuously with the eventual goal ofreaching a near-optimal solution. This is ideal for parameter optimizations and situations where a solution may change due to the flexibility provided through the main genetic operations of crossover and mutation.

This made me think about how this could be applied in a gaming environment, game agents could theoretically develop to some bias according the the aforementioned genetic operations, though through further investigation and my own attempts it seemed to be expensive in terms of computations for what is actually achieved as an end result.

With the goal of game agents adapting and improving the idea of using a conditional probability model to influence decision-making arrived as a possible solution.

With conditional probability, you can be provided with new evidence which changes the beliefs in what happened/what may happen, effecting the existing probabilities established. This can be represented in a graphical model as a network of nodes with different levels of influence on each other.

Due to each nodes conditional dependency this makes for a model which could be used in an environment which is ever changing, including games where the player has profound effect on the game entities they're interacting with, in addition the model can be used to draw commonalities between what may seem like random variables.

This probability model, while on a very large scale could get computationally heavy, would only need to update when an event happens which is relevant to the nodes in the network, and a number of different


Illustration 1: A very simple network of nodes with conditional dependencies indicated by arrows.

networks can be comprised for different object interactions to avoid unecessary calculations.

Furthermore, once initial values are established the model takes care of itself - it is self-contained, the model cares only about the nodes within it with no other operations being necessary and in turn is intuitive to use and visualize.

With this approach to decision-making in mind I thought of a combat system being an appropriate way of making use of this probability model, this will involve enemy game agents making decisions based on playeractions such as defending when the player is attacking.

By keeping track of player details the network can measure intent, this is essentially asking the question of what the player will do next.

Of course the program itself doesn't know this but it can estimate from all the difference connections, something which cannot be done with regular decision trees and similar models.

At the beginning of development I had several influences which effected the direction I wanted to take the game aesthetics and mechanics in. These being the games 'Elsword' which is a popular online side-scroller with distinct character interactions with enemies and games such as 'Apothea', a greek mythological platformer which look at unique themes and visuals to stand out on the indie gaming market, encouraging me to take a different direction in terms of game visuals.

Given that side-scrollers are a saturated market, to give the game some originality I also combined 2d and 3d aspects and played with depth through techniques such as parallex scrolling. By mixing 2d and 3d I could also make key objects in the game stand out in their environment.

3. Objectives

There were a clear set of objectives I wanted to research and develop from early stages in development, these were:

• To introduce myself properly to the field of probability theory and conditional probability.

• Develop Bayesian network(s) with suitable nodes and dependencies.

• Integrate a combat system within the game which utilizes a Bayesian network for enemy decision-making.

• Make the Bayesian network(s) modular enough to be usable for other applications with minimal overhead.

• Improve my object-oriented programming skills, Unity game engine knowledge, mathematical skills and problem solving skills.

• Manage the project suitably throughout using different tools and an appropriate design philosophy.

• Create an aesthetically pleasing level design for a game with appropriate art direction and an established look and feel.

• Look into ways of making the provided algorithms take less time to process.

Whilst these were subject to minor changes I feel these objectives have been achieved, details on their development and design are explained further in the report.


4. Tools

4.1 Game engine

I have chosen Unity 5 for the game engine to develop the game. Not only has Unity become hugely popular for the flexibility in regards of deployment flexibility, such as web players and executables for different platforms and operating systems, it is a very intuitive and feature-rich platform with a thriving community providing tutorials and problem solving.

Some examples of the features it provides are different views of the game, a scene view for editing and game view for deployment. The hierarchy to the left of the views listing all objects within a scene. The built-in math and physics libraries to quicken development of features and the user-friendly way of adding user-created assets such as textures, scripts and sounds to game objects within the game.


Illustration 2: Reference: http://www.learn-cocos2d.com/2014/02/mobile-2d-game-engine-popularity-index-january-2014/.

An indication of Unity's popularity is shown with data retrieved from Stack Overflow looking at the tagged questions asked for mobile development in January 2014

http://www.learn-cocos2d.com/2014/02/mobile-2d-game-engine-popularity-index-january-2014/

http://www.learn-cocos2d.com/2014/02/mobile-2d-game-engine-popularity-index-january-2014/

A relatively new tool to Unity and one of the reasons for me choosing to develop with this game engine wasits 'Mecanim' system for animating different game agents.

Having just begun using it at the beginning of the project I followed the tutorials provided by Unity and established animation controllers for the player and enemy. They are constructed similarly with a base layer for movement animations and then an action layer which contains the states for different attack styles and defending.


Illustration 3: Visual FSM-view of the Player animators base layer, arrows indicate the transitions between states.

Illustration 4: Visual FSM-view of the Player animators action layer.

4.2 Programming software

The main IDE used for programming was Microsoft Visual Studio 2013, this is the most comprehensive software for C# programming and also allows me to test deploy Windows Phone applications which I build from Unity.

MonoDevelop was also used in instances where Visual Studio 2013 hasn't been on the system I'm working on, this is an IDE natively used with Unity which can also be used for C# development.

4.3 Media software

Adobe Photoshop Elements – Used to create the 2d textures for various visuals in the game such as the GUI,backgrounds and materials rendered on 3d models.

GarageBand - Used to compose the music from a set of available instruments, including sound effects and background music.

Audacity – This was a program i was unfamiliar with, I used it for sound editing. I imported the audiio tracks created on GarageBand and changed elements of it like the pitch and volume as well as cutting out unwanted portions of a sound file and trying to make it loop correctly.


Illustration 5: Using Audacity to edit the games soundtrack.

Mirillis Action – This was also newly purchased for this year to record footage for progress updates in high-definition, it also has also support for 60fps recording ideal for game footage. The videos were recorded for progress updates and advertising featured on my blog and Youtube.

3d Studio Max – Used for examining the models in the game in detail, adding optimizations to them and rigging.

4.4 Project management software

GitHub with git bash – This is a website used to store git repositories which is a method of versioning projects which can then be referred to or rolled back to if major issues occur with recent changes. Git bash is a command line tool for performing git operations with my repositories. It allows me to quickly add, modify and push changes to GitHub for synchronization between local and online copies of the project.

Pivotal Tracker – This software provided me with a dashboard which can be used to add user stories, essentially listing tasks a user would expect from the software upon completion. Different properties for these stories can be set such as weightings for a quick indication of how long I expect a task to take to complete.

5. Method of approach

5.1 Research

Coming from little to no knowledge in regards to using Bayesian networks meant the majority of research conducted was based on conditional probability theory as a precursor to using them. There were also new features and tools within Unity I was planning to learn.

The materials used to learn about conditional probability theory, Baye's rule and applying it to graphical models were mostly recordings of lecturers at institutions such as MIT which have become available through MIT Open Course Ware with knowledge being expanded upon through the completion of online excercises.

To get a better understanding of situations where Bayesian networks are used I looked at various articles online, namely 'Bayesian Networks in Video Games' (Cummings, Elzer, Zoppetti), 'An Introduction to Hidden Markov Models and Bayesian Networks (Ghahramani, 2001) and 'Real-World Applications of Bayesian Networks' (Heckerman, Mamdani, Wellman, 1995).

In terms of the game itself there were several new features of Unitys game engine I wanted to make use of, the main one being Unitys 'Mecanim' system. This is used for animating game agents and is essentially a visualized finite state machine with each state containing an animation and its properrties such as its start at end time and curve with support for layering animations on top of eachother and blending them together for smooth transitions with blend trees.

Preparation for this was also done through online tutorials and Unitys documentation archives.

I was mostly familiar with the other tools and resources I was working with including project management tools such as Git but picked up experience along the projects development.


Below are some examples from myself working on excercises and diagrams which were used as reference for when I developed the framework.


Illustration 6: Working through excercises determining probabilities and how they change with evidence.


Illustration 7: Using a tree model to help learn discover probabilities for all possible outcomes

Illustration 8: Intersecting outcomes

5.2 Languages and libraries

5.2.1 C#

An object-oriented language was used for development due their suitability to games and getting applications built much faster and in a more structured way with the drawback being performance and portability, something Unity itself remedies with its porting capabilities.

C# has a large userbase and is one of the languages supported by the UnityEngine namespace which contains all Unitys classes and it's also my personal preference when it comes to development outside of 3dgraphics due to user-friendly native constructs such as properties and events, enumeration support and native garbage-collection.

5.2.2 XML

XML has been used for the files which construct the initial Bayesian network. Due to it being a markup language where its main use is to describe entities through organized nested tags it seemed appropriate to use it for describing the initial Bayesian network which can then be parsed to C#.

5.2.3 Unity engine libraries

The libraries within Unity help to quicken development of many features within the game. I have made use of Unity's Input, GUI, Math, Physics, Animator and GameObject classes primarily, all of which fall under the UnityEngine namespace.

In addition a third-party library called iTween available on Unity's asset store was used to animate 2d GUI elements more easily for fluidity between menus.

5.3 Deployment

The game was built to run on both computers and mobile devices with fairly low system requirements in comparison to many games released today. Because of this there are several different methods of inputs which vary depending on the platform the user is playing on.

For desktops/laptops you can play with a mouse and keyboard though there is support for Xbox 360 and Playstation 3 controllers. For mobile devices multi-touch functionality and an accelerometer is required to be able to fully play the game.

The system requirements to run Unity applications and the supported operating systems are listed below:

• OS: Windows XP+, Mac OS X 10.7+, Ubuntu 10.10+◦ Graphics card: DX9 (shader model 2.0) capabilities; generally everything made since 2004

should work.◦ CPU: SSE2 instruction set support.

• Android:◦ OS 2.3.1 or later; ARMv7 (Cortex) CPU or Atom CPU; OpenGL ES 2.0 or later.

• Windows Phone:◦ OS 8 or later; ARMv7 (Cortex) CPU or Atom CPU; OpenGL ES 2.0 or later.


5.4 Project management

5.4.1 Design process

I felt following a game design process would be appropriate due to the end product being a game, this involves establishing a priority list of game mechanics indicating what needs to be developed in order, the mechanics with highest priority are those which make use of the Bayesian network.

Additionally, a list of game rules which limit the player and their surroundings is necessary to get a more specific idea of how the game will play out.

A game design document has also been developed which is essentially a specification of all the tools and assets used to develop the game as well as the reason for it being created.

Finally, a graphic bible which illustrates the games look and feel was created both for reference and to makemyself stay on the right track when it comes to visual aesthetics.

After having a more detailed plan of my approach to the project I developed a schedule which can be seen on the following page. This outlines deadlines for stages of the design process as well as the actual implementation of the project. It is separated into documentation, implementation and deliverable sections.

The schedule itself was updated manually with each significant addition to the project.


Illustration 9: Influential image board excerpt from the Graphic Bible, illustrates different art stylesand scenery which have interested me.

You can see from the schedule a lot of the planning stage took place in October, the implementation of the project began in late October/early November and there are some stages of development which continue throughout the entirety of the project timeline such as continued research on AI techniques and improving the visuals of the game itself as well as ensuring the code developed is clean and usable in other projects.

In addition to a Game Design process I have also used AGILE methodologies to prepare sprints for every couple of months of development, these can be read from the interim reports attached as appendices.

Using Bayesian Networks for enemy decision-making in a 2.5D side-scrolling game 18

Illustration 10: Schedule used to track general progress throughout project

5.4.2 Version control

There are two Git repositories used to store different versions of both the framework and game which uses it, this is due to them working irrespective of whether they're being used together or not. The chosen version control system was Git as it is the most established and supported with the chosen client being Github which I have used in the past.

The repositories were created early on in development after establishing a barebones level design for the game and with the first revision of a working Bayesian network in a Windows Form Application.

Future commits were pushed as bug fixes, features and optimizations were added with commits being shown below.

I usually worked on the project in intervals throughout the year so I could focus on the project easier rather than having my mind on other modules hence the breaks between weeks. The beginning weeks see little activity due to it being largely theoretical with research, the design process and setting up the tools I was going to use.


Illustration 11: Committing a change to the game's repository and the week-by-week commit log

5.4.3 User story tracking

For a high level view and basic description of each feature and bug I needed to work on I used Pivotal Tracker which is an online agile management tool providing each project with a dashboard of ''user stories', these can be added and modified and make up the tasks required to implement the project.

Each user story can be assigned a type, difficulty weighting and sub tasks for each step required to completeit. This helps for time management as it gives me a quick indication of what needs to be done and how long it is expected to take rather than having to rely on memory.


Illustration 12: A view of the projects dashboard, completed stories on the left

6. Target marketWhilst this project was treated mostly as a learning experience to improve my skills in several areas of interest I expect to build on it and create a full featured game in the future.

There are two major genres which correlate with the gameplay of Beyond the Mountains, these being platformer in terms of how the player explores the world and action in terms of the reactive combat and other, smaller features within the game.

Action-platformers are a tried and tested combination which has been popular for many years and has interest from people of all ages.

Although the target audience is broad, those who are spending less time working, typically those younger than 30 are spending more time playing games with desktop/laptop computers being the most popular platform to play on. This, along with the fact a desktop with Windows is my development platform has resulted in me getting the game working as expected on Windows, Linux and OSX operating systems before moving on to mobile deployment.


Illustration 13: Reference: http://www.marketingcharts.com/online/nearly-all-us-teens-53-of-adults-play-video-games-7114/

The mobile platform is saturated with games which appeal to the casual gamer but typically do not offer an experience where difficulty and quick thinking have such a profound impact on a users success. One of the goals of the game is to make people feel rewarded for being alert and understanding the combat system provided. This should encourage an experience where the player is always able to improve and see the results of the improvement through game visuals.

Looking at the game from a broader perspective with what it has to offer in regards to gameplay, it should interest those wanting combat mechanics with tight controls, a high skill ceiling and an experience they can jump right into, exploring a world with fresh visual aesthetics compared to most games on the desktop and (in particular) mobile market.

By advertising the game with these three components at its core it is hoped to garner interest for both casual and hardcore gamers, an experience that is quick to get into and play with but has room for constant improvement by the player.

In terms of deployment to services which can allow the game to be played at a larger scale I eventually plan to release the game on Kongregate which has established itself as one of the most popular websites for Flash and Unity games as well as the Steam Greenlight service where users determine whether a game is suitable for full release, this can garner a lot of future feedback to take this project from a proof of concept and a result of research to an experience worth purchasing.

7. System design - Decision network

7.1 Features

• XML parser for the construction of a Bayesian network compatible with the latest Windows Runtime compatible with Windows, Linux, OSX, Android, Windows Phone.

• Automatic construction of network after reading valid XML file• Binary nodes sorted in buckets• Probability calculations of nodes given evidence through bucket elimination• Windows form front-end for testing a posterior probability given any amount of priors

7.2 System architecture

The decision network parses XML files from either a chosen file or by directly passing an XML string into it. The XML parser itself is using the recent System.Xml.Linq namespace for up to date classes and functions which compile on the Windows Phone operating system as well as standard operating systems such as Windows XP/7/8/8.1.

While parsing the XML file the program is constructing a network of nodes each with their own id, name, parents, range and conditional probability tables based on the values at different tags.


A conditional probability table (CPT) defines the initial probabilities of each node being in a certain state (forthis project there are binary states to keep things simpler, so 0 or 1) across all the different combinations of probabilities from other nodes and updates whenever new evidence is given. The following diagram illustrates the network currently being used for the combat system for 'Beyond the Mountains' however thesystem supports networks with any amount of nodes and dependencies.


Illustration 14: A part of the BN_Combat.xml file used to construct the combat systems network, details on each node as well as their name are used to construct the network.

Each instance of a node contains its own CPT stored as a 2D array.

Having constructed the network it must then be able to be inferred upon, this is to find an answer to a givenquery based providing incomplete evidence. The formula used to achieve this is expressed as the following:

P(A|B) = P(A)P(B|A) _________________

P(B)

The probability for A given B where A is the query and B is the evidence is the probability of A multiplied by the probability of B given A over the probability of B.

To provide further context, this can be viewed using the following:

posterior = likelihood * prior_______________________ marginal likelihood

The posterior value is the conditional probability that is assigned after the relevant evidence or background is taken into account.

The denominator marginal likelihood is a normalizing constant that ensures the posterior probability adds up to one. The value of one is the standard maximum probability the evidence occurs, this constant can be a different value but then must not change and the summation of the numerator over all possible values of the query must be equal to this constant.

For the sake of clarity a very simple network and set of CPTs is given to help explain how a solution can be found for a given query.


Illustration 15: A graphical model for finding what makes grass wet

Illustration 16: The CPTs for the nodesin the network

With the graphical model and probability tables established we can perform inference to find answers to queries.

The joint probability of all nodes in this model is given asP(C,S,R,W) = P(C) * P(S|C) * P(R|C,S) * P(R|C) * P(W|S,R) where C = Cloudy, S = Sprinkler, R = Rain, W = Wet grass.

Say we wanted to find the probability that the sprinkler is on if the grass is wet, seeing as the sprinkler is notdirectly influenced by wet grass this would be impossible with regular decision trees. What makes networks using Bayes’ rule different is the ability to perform top down and bottom up searching. As well as this, due to each node having a link to at least one other node probabilities can be established between nodes which intuitively have little connection to each other.

The formula for querying if the sprinkler is on if the grass is wet is demonstrated below:

P(S=1|W=1) = P(S=1, W=1) = ∑c,r P(C=c, S=1, R=r, W=1)

_____________________ _____________________ P(W=1) P(W=1)

The formula is taking the summation of all possibility probability combinations for cloudy and rain where sprinkler and wet grass is set to 1 (the program is given that information by you in the query).

Due to the amount of states and nodes a network can have the computation for doing this can get quite demanding, below is a brute force technique of calculating inference with just binary states and four nodes, imagine multiple states and hundreds of nodes - this will give you get an idea of how expensive the operation can be.

(c = 0, r = 0): P(W=1|S=1, R=0) * P(S=1|R=0) * P(R=0|C=0) * P(C=0) *(c = 0, r = 1): P(W=1|S=1, R=1) * P(S=1|C=0) * P(R=1|C=0) * P(C=0) *(c = 1, r = 0): P(W=1|S=1, R=0) * P(S=1|C=1) * P(R=0|C=1) * P(C=1) *(c = 1, r = 1): P(W=1|S=1, R=1) * P(S=1|C=1) * P(R=1|C=1) * P(C=1)

Chance the sprinkler being on if the grass is wet: (0.9 * 0.5 * 0.8 * 0.5) + (0.99 * 0.5 * 0.2 * 0.5) + (0.9 * 0.1 * 0.2 * 0.5) + (0.99 * 0.1 * 0.8 * 0.5) / P(W)

Every time the query P(S=1|W=1) is given the program must recalculate the probabilities, considering the values for each state in the nodes themselves don’t change it would be beneficial to make use of dynamic programming, lowering the amount of computation at the expense of storing values which have been calculated before.

Summations in programming terms are essentially loops, therefore using the brute force method of inference there would need to be nested loops, this makes the problem exponential and in turn not a very scalable solution.


Pseudo code (for P(S=1|W=1):

PRODUCTFOR R = 0 : 1 : 1

FOR C = 0 : 1 : 1PRODUCT = PRODUCT + P(C)*P(R|C)*P(S=1|C)*P(W=1|S=1, R)

ENDEND

To minimize the amount of multiplications we can take terms out, examples below.

Store values of all possible probabilities of w in variable x and r in variable y:

P(S=1) = ∑c∑r∑w P(W|S=1, R) * P(R|C) * P(C) * P(S=1|C)= ∑c∑r P(R|C) * P(C) * P(S=1|C) * ∑w P(W|S=1,R) ←x = 1= ∑c P(C) * P(S=1|C) * ∑r P(R|C) ←y = 1

With this we are left calculating possible probabilities of c= P(S=1|C=0) * P(C=0) + P(S=1|C=1) * P(C=1)

By storing these calculations we can use a sequence of loops rather than nested loops, reducing the computational complexity of the problem from exponential to polynomial.

x = 0y = 0z = 0

FOR W = 0:1:1x = x + P(W|S=1,R)

END

FOR R = 0:1:1y = y + P(R|C) + x

END

FOR C = 0:1:1z = z + P(S=1|C) * P(C) + y

END

As we calculate each variable x,y,z we are cumulatively adding them to each other to find the joint probability of the nodes.

A bucket elimination algorithm is used in the program, these store a nodes CPTs children, id and parents and are used when finding probabilities for a given query and are established in a similar way to the x,y, z variables in the previous pseudo code.


A windows form application is used as the front-end for testing the network with form controls to allow users to test different XML and node configurations before using the program in a more sophisticated environment.

What is done with the recieved probabilities after performing inference is largely up to the user, an exampleof myself using the results of inference is explained when talking about my game demo 'Beyond the Mountains' class descriptions and game design.


Illustration 17: Calculating the probability of adrnealine 1 if lunge is 1 and swipe is 1 from the combat network used in my game

7.3 Program flow

7.3.1 Parsing XML to network


XML file sent to be loaded

Parse XML file intonetwork

Has the file been loaded?

Load XML file

End of file reached?

User enters XML string or XML file

Display node list

Yes

No

Yes

No

7.3.2 Calculating inference


7.4 Class descriptions and relationships

7.4.1 NetworkNode

This class stores all properties and methods for each node in the network.

The data stored is a list of the nodes parents, its CPT as a 2d array, its current evidence determined by a query, its name, id and range (nodes it is connected to).

Functions:• Clear() - Resets evidence of node, called when new inference occurs

7.4.2 BayesianNetwork

This class loads and parses the XML file and builds the network from it, ensuring the data provided is suitable.

Functions:• BayesianNetwork() - Constructor establishing new list of network nodes.• ResetNodes() - Clears evidence of all nodes in list.• SetNodes() - Checks string formatting for node properties are suitable.• BuildFromXml() - Loads XML file, finds elements under root XML tag and calls other functions to

create the nodes from them.• CreateNode() - Checks and adds a nodes properties: range, parents, CPT and ensures probabilities

of node normalize.• IsLoaded() - Uses regex to determine whether string to build network from is a file name or a ready-

loaded XML string. • LowerCaseTags() - Convets XML tags to lower case for consistency.


Illustration 18: Relationships between classes represented as a UML diagram.

7.4.3 Inference

Inherited by Elimination, contains base functions and variables for performing inference.

Functions:• Inference(BayesianNetwork) – Creates local instance of BayesianNetwork class.• GetBelief(string, string) – Overriden by Elimination, intended to return probability of query given

evidence.

7.4.4 Elimination

Inherits inference, performs joint probability calculations for queries given evidence using the stored BayesianNetwork.

Functions:• Elimination(BayesianNetwork) – Calls Inference's constructor and adds a list to store buckets, calls

PrepareBuckets().• PrepareBuckets() - Adds each nodes children and parents and their CPTs to buckets• GetBelief(string, string) – Returns result of query given evidence, calls Sum() to get probability and

normalizes it.• Sum(int, int) – Sums a query given evidence over all possible values, checking CPTs in each bucket

and iterating through each calculation using the nodes parents and children buckets stored in Lists.• FindMaxNodeId(ArrayList) – Gets node with highest ID value for bucket initialization.• FindNodeByName(ArrayList, string) - Safe way of checking whether a node exists a network by

name.

7.4.5 Bucket

This class keeps track of the parents and children of the nodes in the network

Functions:• Bucket(int) – Creates a bucket with a specified id and establishes a List to store parents and children

of network nodes.

7.4.6 DemoNetworkForm

This acts as a front-end for testing different XML configurations getting input from GUI controls.

Functions:• DemoNetworkForm() - Set up form and Lists for new BayesianNetwork and Query which contains

the different evidence strings and values.• LoadNetwork() - Calls function to build network from XML/file name provided.• Infer() - Calls function to infer probability of query given evidence passing the values in text fields as

parameters• IsValid(), IsNameValid() - Check if entered string isn't empty and follows intented structure


8. Game design - Beyond the Mountains

8.1 Features and mechanics

• Player actions with compatibility for keyboard, mouse, joypad and mobile controls consisting of:◦ Walking left/right◦ Running left/right◦ Jumping while running◦ Jumping while still◦ Attacking◦ Slashing◦ Lunging◦ Swiping◦ Blocking

• Player statistics of health, adrenaline and aggression affected by combat exchanges with enemies.• Enemy chasing behaviours with raycasting used for player detection.• Enemy decision-making in combat derived from a given Bayesian network.• Enemy animations using Unitys 'Mecanim' system derived from actions.• Enemy statistics including visible health bars displayed above enemy models.• Physics mechanics for moving around objects and solving simple puzzles.• Hand-drawn backgrounds and objects inspired by a Traditional Japanese art-style.• Parallex scrolling with 2d objects and backgrounds which scroll with the player at different speeds

depending on their depth.• GUI for title screen, settings, about and the HUD displaying a players health and adrenaline.


8.2 Class descriptions and relationships

8.2.1 Unity's MonoBehaviour class

There are several different default functions Unity provides which can be used when a class inherits from MonoBehaviour, these are called in different ways when running the program. There are three default functions used commonly in the program. The Awake() function is always called first and is recommended for initialization, the Start() function is called after which is used to setup relationships between game objects and the Update() function is called every frame for things that need to constantly change based on the game environment.

8.2.2 Script folder hierarchy

8.2.3 Camera class: Culling

Used to establish camera sort mode to orthographic to avoid clipping 2d objects.

Functions:• Awake() - Sets camera sort mode to orthographic.

8.2.4 Camera class: FollowPlayer

Attached to a camera object to follow the player from a certain position.

Functions:• Start() - Gets a reference to the PlayerAnimator and sets position to players x,y position and its z

position to a value determined by the user.• LateUpdate() - Updates the camera to keep following the player by checking whether the player is

moving and then lerping (smooth curve transition) to the players x,y position.

8.2.5 Core class: CombatController

Calculates the combat exchange between an enemy in player checking the position and action both agents are using for determining an outcome (how much health is lost).

Functions:• Awake() - Gets local reference of Player, Animator, and HashID classes.• CalculateCombatExchange() - Checks the player is in range of the enemy, facing the enemy,

determines outcome of exchange by changing player and enemy statistical values.


8.2.6 Core class: MainController

Controls interaction between the title screen and main scene.

Functions:• Start() - Get reference to GUIController and PlayerInput.• Update() - Checks if player has signalled input to restart or return to the title screen and either

resets player position or changes scene to title screen depending on this.

8.2.7 Core class: StatBar

This class is used to represent properties and methods for different statistics such as their maximum and current values and how they can be effected.

Functions:• StatBar() - Establish default values for StatBar.• Setup(float, float, float) - Takes values to replace values for StatBar.• Update() - Sets when a value in the StatBar was last updated• UpdateStatistic(float) – Takes a parameter of how much the current value of the statbar should be

changed, checking the change is appropriate and applying it.

8.2.8 Decision Network class: Selection

The other classes for the Decision network were covered in the previous section, the game needs to find some way of choosing a node after getting inference which is what this class provides. Using a roulette-wheel inspired algorithm ensures the enemy is less predictible.

Functions:• Selection(HashTable) – Gets a hastable of node names and influence, sets a range based on the

maximum key value of hash and creates a hashtable to store probability regions to be randomly selected.

• AssignRegions() - Assigns probability regions for each node, the higher the probabilityof a node occurring is the larger the chance of it being selected with random number generation.

• ChooseRegion() - Generates a random number between 0 and the total range of nodes, checking the difference between the number and each of the hashes probability regions selecting the region closest to the node value.

8.2.9 Enemies class: EnemyController

This class determines enemy behaviour and infers moves for the enemy based off of the Bayesian network.

Functions:• Awake() - Sets up local references to other classes and sets initial values for enemy statistics.• Update() - Checks current state from enemy animator, if the enemy is not dead and the player is in

viewable, move towards the player. If player is in range perform an attack determined by inferring the decision network every animation cycle and calculate the damage of the attack.

• OnCollisionEnter() - Checks for collision between self and objects which can kill the enemy.• CheckAttacking() - Checks if the enemy is in an animation transition.• InferMove() - Infer the Bayesian network of probabilities for a specified set of outcomes, pass these

into a hastable to be selecting from using the roulette wheel algorithm in Selection.


• AddQueryToHash() - Used to store query and evidence data in a hash to be passed to Selection.

8.2.10 GUI class: GUIController

This is the class used for all user interface controls and textures and how they interact with eachother. Unityencourages a single GUI class to prevent unecessary draw calls.

Functions:• Awake() - Get references to required game objects and set initial positions of GUI elements in the

title screen.• Update() - Gets users input from Unity's Input Manager and iterates through selection grid of

clickable buttons depending on the input entered.• OnGUI() - Sets resolution for game scene, establishes a matrix to maintain aspect ratio of GUI

controls if the game scene is resized, displays the different GUI controls. If the game scene is the active scene this class draws the players and enemies statistics as bars with percentages.

• SlideLabel() - Accesses third-party iTween library to perform sliding animations to GUI controls if certain GUI controls have been selected.

• DrawHealth() - Draws the players health bar with texture array stored in a public variable, in increments of 10 to 100 a new element of the texture array is drawn.

• DrawAdrenaline() - Draws the players adrenaline with texture array stored in a public variable, in increments of 10 to 100 a new elements of the texture array is drawn.

• DrawEnemyHealth() - Gets the position of an enemy and draws an enemys health above them usinga texture array.

8.2.11 Other class: AnimTriggers

Triggers are invisible objects placed around the level with a collider, when an object hits this collider it triggers a certain behaviour. The AnimTriggers class is used to indicate when certain animations should be played based on the colliders the player or enemy are hitting.

Functions:• Start() - Gets reference to the Animator.• OnTriggerEnter() - Checking if the game object with this script applied has entered a collider with

the name 'T_JumpDown', if so play the jump down animation.• OnTriggerExit() - This is called when the game object with this script applied is no longer colliding

with the trigger and stops the triggered behaviour.

8.2.12 Other Class: BackgroundScroller

This class is used to scroll 2d objects such as the background and 2d obstacles with the player, their depth isused to determine how fast they scroll with the player.

Functions:• Start() - Saves the initial position of object, gets a reference to the player.• Update() - A condition is used to determine whether the object should scroll at the same speed as

the player (so it never disappears) or scroll at a slower speed than the player so the object goes out of view at a specified point. The difference in position between the player and object is calculated and the object is moved by this difference divided by a specified scroll speed.


8.2.13 Other Class: HashIDs

This class stores the hash values determined by Unity's Mecanim system to store the keys for different animations to be used by game objects.

Functions:• Awake() - Set local variables to correspond with the Animator classes states which contain the

animations. Local variables are also set for the different animation effectors within the Animator class.

8.2.14 Other Class: ParticleTrailBehaviour

This class controls the particle trail object which appears when the player moves.

Functions:• Start() - By default disable the particles and get a reference to the PlayerAnimator class.• Update() - Check whether the PlayerAnimator class is at the state for running, if it is enable the

particle trail.

8.2.15 Player Class: PlayerAnimator

This class is used to get and set animation data from Unity's mecanim system depending on player input and the current game situation.

Functions:• Awake() - Gets references for the Animator, HashIDs, and CombatController classes and sets default

animation speed.• Update() - Gets the current states on the Base and Attack layers in mecanim's finite state machine.

Calls functions determining whether the player is attacking and if so in what style, also calls functons determining whether the player is jumping or falling. Sets a cooldown on the player performing attack animations.

• SlashStateUpdate()/LungeStateUpdate()/SwipeStateUpdate() - Checks if player is currently attackingin a certain style, if not set the Animators attacking boolean to false.

• JumpStateUpdate() – Checks if the player is jumping, if not set the Animators jumping boolean to false.

• FallStateUpdate() - Checks if the player is falling, if so do not allow the player to block, lower the animation speed depending on distance to ground calculated with a raycast.

• IsGroundedCloseDistance()/IsGroundedMediumDistance()/IsGroundedFarDistance() - Raycast determining whether the game object is close/medium/far distance from the ground.

• ResetTimer() - Determines when the player can attack again.

8.2.16 Player class: PlayerInput

This class reads input from keyboard, mice, joypads and mobile device acceloremeters and touch and passes the information to the PlayerAnimator class which decides the animation to play based on it.

Functions:• Awake() - Gets local references and default input type.• Update() - Calls DetectControlType and based on that checks the different inputs of the current


active control.• DetectControlType() - Checks what control type if any are being used.• CheckDesktopInput() - If the control type is mouse and keyboard or joypad listen for the different

inputs which fall under that control type and set parameters in the PlayerAnimator class.• CheckMobileInput() - If the control type is mobile get accelerometer, touch and drag data and set

parameters in the PlayerAnimator class.• CheckDirection() - Orients the player depending on input.

8.2.17 Player class: PlayerStatistics

This class sets up and controls the different player statistics such as health and adrenaline, instantiating StatBar classes to represent the values of each.

Functions:• Start() - Sets up StatBar for each statistic• Getters/setters for each StatBar

8.2.18 Third party scripts and shaders

A third party shader was used for applying a cartoonistic effect to the materials applied to 3d models within the scene. This is used to match the 2d artwork better. It was purchased from Unity's asset store under the name 'Toon Shaders Mobile'.

8.3 Other assets

8.3.1 Asset hierarchy


8.3.2 Animations

The animations used are motion capture data provided in Unity's asset store which are then modified to transition into eachother within the game. The animators which contain the configuration of animations for different game objects are also saved under this folder.

8.3.3 Fonts

The fonts folder contains fonts overlaid onto GUI controls and the game scene to provide information on the HUD and for tutorials.


Illustration 19: This is the Base layer of the Player animator, each state stores animations and theirproperties with arrows indicating the transitions between states.

Illustration 20: Import properties for Fonts in Unity, PhoTai was used as the font for the games logo

Illustration 21: Game logo

8.3.4 Materials

Materials are used to map 2d textures onto 3d objects, after attaching a texture a shader can be applied for different lighting effects to make an object look 3d.

8.3.5 Models

This folder contains the 3d models used in the game, primarily the player and enemies who were chosen to be 3d to stand out. Both the Samurai and Skeleton model models used in the game were obtained from Unity's asset store.


Illustration 22: The properties of the material used for 3d boulders in the game

Illustration 23: The texture mapped on the boulder using a material and cartoon shader

Illustration 24: The 3d models for player and enemies (imported from Unity's asset store)

8.3.6 Particles

These are aesthetic effects which animate within the game and are used to give polish to the visuals.

8.3.7 Prefabs

Prefabs are configured game objects which already have had their scripts and other assets applied to them.

8.3.8 Scenes

This folder contains each of the scenes or stages within the game. As I am animating 2d GUI elements there is only a single scene which is the level.

8.3.9 Sound

This folder contains soundtracks and sound effects used in the game saved in MP3 format.

8.3.10 Textures

This folder contains 2d textures which are used for materials, GUI and backgrounds.


Illustration 25: Texture import settings, the 2d backgrounds and objects are set as sprites. A lower texture resolution is recommended for mobile systems.

8.4 Game flow

8.4.1 Title screen


Illustration 26: Flow diagram representing the title screen.

8.4.2 User input and combat


Illustration 27: Flow diagram representing a very abstract look at userinput and combat interactions.

8.5 Controls

8.5.1 Direct controls

• A/Left joystick/Tilt left – Move the character left.• D/Right joystick/Tilt right – Move the character right.• Left Mouse Button click/Joystick button square/Swipe down – Slash attack.• X/Joystick button triangle/Swipe up – Swipe attack.• Z/Joystick button O/Swipe across – Lunge attack• Right Mouse Button click/L1/Hold touch – Block• Spacebar/Joystick button X/Tap - Jump

8.5.2 Indirect controls

• When the player is not grounded he falls until he reaches another platform.• When the player runs out of life he falls over and dies.


9. Testing

9.1 Self-testing

Testing was performed as I completed each task in both the game and the Decision Network. This involves looking at and setting breakpoints for each method and class as well as making use of the debug messages and other tool the IDE used provides.


Illustration 28: Using breakpoints to check individual functions andlines of code.

Illustration 29: Custom error handling for clearer problem solving.

Illustration 30: Custom errors helpful for developers and users.

9.2 User feedback

Both written and verbal user feedback was collected from colleagues which mostly concerns how effective the Decision Network and game actually are. I welcomed any suggested features they deem appropriate as well as any improvements or suspected bugs to existing parts of the project to then be examined to determine whether it is necessary to take action.

Some examples of feedback recieved are shown below:

"Tilt less to move left and right, it can be awkward to see the screen when moving".

"Backgrounds are disappearing and should scroll with player".

I have also taken onboard advice from my supervisor from our several meetings over the course of the module, keeping a record to refer to which is featured as the appendix 'Supervisor meetings'.

Some of the feedback suggested has been worked on, parallex scrolling has been added to 2d elements in the game so the background scrolls with the player, the volume for the soundtrack has been increased, I have implemented proper visual indications of enemy statistics and applied them to each of the enemies in the level and made the transition between the player walking and running more gradual.

I also made use of the project showcase to get more user feedback, particularly verbally to avoid misunderstandings and have an account of what people think of the project right after they demo it. This feedback will be used to improve the project further including past the project deadlines to serve as a high standard piece of work to put on my portfolio. In addition a Google Form was set up which testers could optionally fill in, the results are featured on the next page.


9.3 Optimization

The previous steps were applicable to making sure the code compiles and the different aspects of the project work as expected however for optimization I used Unity's profiler. The profiler is great for providing analytics on CPU and memory usage, frames per second and the time taken in ms to perform certain operations such as draw calls at different parts of the game.


Illustration 32: The profiler notifies the developer of the different operations being performed and their toll on computer resources when running the game.

Illustration 31: Creating the form with feedback from it formatted in Google Sheets

10. Risk management

Risk Management strategy

Accidental loss of information I had the most important files for the project stored on multiple devices.

Loss of functionality in recent development I made use of version control through git allowing me to reroll back to previous revisions. I also had working versions of the project stored on a secondary system andUSB drive.

Lack of confidence in abilities with certain areasof the project

Research took place in any fields where my skills weren't up to the standard they needed to be. This involved following tutorials on MITOpenCourseWare, completing excercises and using Unity's wiki pages. Other support sites such as StackOverflow were used to help solve problems.

Running out of time/extending deadlines Some aspects of the project took longer than anticipatedanda s a result more time was spent than planned to keep up with my schedule, Pivotal Tracker helped to keepme focused on what needed to be done.

Bugs not caught by compiler Both manually debugging and having other people test the game enabled me to find bugs which arent immediately obvious.

Prioritization of less important elements A priority feature list was established and used as reference to understand what needs to be done first.

Failure to create engaging game which has potential for commercial release

The game has a niche technology for decision making behind it through using the Bayesian network developed as well as being fully animated with a unique art style as selling points.

Failure to delliver Interim Reports on time with sufficient progress

I spread out my work across modules working for one or two weeks on a module at a time with this changing onlyif deadlines were imminent. This helped me make significant steps through each development cycle and prepare an interim report in time.

Failing to develop a Bayesian network system sufficient for real-world use

I tried making the system more modular and user-friendly by having XML parsed to construct the network, this leaves the system very open to different platforms including the web. The solution also makes use of an appropriate data structure to reduce time comlpexity for inference.


11. Constraints and limitationsThe states of the nodes in the Bayesian network are binary, if there were less granularity the results of inference could be more accurate and the network itself would require less nodes.

Some knowledge of how an XML file needs to be formatted for the system to build a network correctly needs to be researched. A tool to help with this should be developed with the possibility of establishing them on a website and sharing different network configurations online.

To make the decision-making an enemy makes more obivously deriven from a machine learning technique more actions could have been added. Due to the importance of animation to visually show each action the player and enemy are initiating it takes a lot longer to create new actions.

It would be helpful to enable scripts to update how much the nodes in the network influence each other after their initial construction. This would allow for more varied behaviours where biases towards nodes canchange.

As the Unity engine is built for 2d and 3d development 2.5d environments can be slightly buggy to work with. Issues have arisen such as the backgrounds clipping with objects at other levels of depth. Whilst changing the camera sort mode to orthographic has helped prevent this for the most part there are still times where clipping issues occur.

No build is currently available for iOS at this time due to the requirement of deploying on a system with OSXoperating system to then recompile in Objective-C.

The background research required for an understanding on conditional probability and Bayes rule meant forless time being spent on the game itself. I didn't want to do more than a barebones level design before setting up a program which creates Bayesian networks and allows inference. As a result the game is fairly simple but serves as a proof of concept.


12. Critical evaluationThe end of this project has seen me complete what I set out to do, I have created a side scroller with intelligent agents through having them adapt their behaviours.

The method I ended up using to achieve this is more suitable as a real world solution than the the early ideas mentioned in the project initiation document and the decision of using Bayesian networks was made early on before development took place on the game and the AI side of it though some minor changes wereapplied to the design process such as the feature list and schedule.

The quality of the system which loads XML as a Bayesian network does work as expected but could be extended upon to allow for more than two states for each node to make the solution more applicable to applications requiring a higher level of accuracy when inferring.

I feel the project as a whole was managed well though there could have been more attention to detail in regards to the schedule as well as more specific details and planning for each user story on Pivotal Tracker including exact times when tasks should be completed rather than just an estimation on how long they will take.

On a minor note regarding version control, using multiple branches for the game and the Bayesian network framework would have been a more elegant way of separating the project than having two repositories as itwould prevent having to work with two repository configurations to perform git operations on.

User feedback was qualititive based; there hasn't been an online release of the game or public surveys usedbut rather detailed feedback from a handful of users. This is positive in terms of I have a better idea on whatneeds to be fixed but it is likely that bugs that would have otherwise been discovered haven't been due to fewer testers.


13. ConclusionWhilst there have been times throughout the project timeline where I had fallen slightly behind due to needing to familiarize myself with the thory behind Bayesian statistics and conditional probability I have developed two software solutions which work reliably on multiple platforms including mobile devices.

I have developed a tool suitable for situations where a system needs results whilst not having conclusive evidence to find it through other means and a game demonstrating this tool in a real-time situation whilst also serving as a piece of work which demonstrates creativity.

I have discovered some appropriate uses of Bayesian networks, whilst they may not always be the optimal solution for adaptive AI they are suitable for measuring player intent and constructing an estimation of what the player is going to do next and are appropriate for real-time environments as they only need to be updated if an object which the network is concerned with has changed its state.

If I were to extend this project for future development I would like to look into more in-depth behaviours such as more actions such as power attacks, hiding and temporary powerups which would make the adaptive behaviours of the enemy a lot more noticeable. It would also be beneficial to develop a tool for helping the user construct XML files in a format the programdeveloped expects.


14. ReferencesBuckland, Richard. 'Lecture 10A: Bucket Sort, Counting Sort - Richard Buckland'. YouTube. N.p., 2015. Web. 25 Apr. 2015.

Cummings, Jarett, Stephanie Elzer, and Gary Zoppetti. 'Bayesian Networks In Video Games'. Web. 22 Apr. 2015.

Cs.ubc.ca,. 'Graphical Models'. N.p., 2015. Web. 25 Apr. 2015.

De Freitas, N,. 'Undergraduate Machine Learning 8: Inference In Bayesian Networks And Dynamic Programming'. N.p., 2015. Web. 25 Apr. 2015.

Demaine, Erik. '19. Dynamic Programming I: Fibonacci, Shortest Paths'. YouTube. N.p., 2015. Web. 25 Apr. 2015.

GHAHRAMANI, ZOUBIN. 'AN INTRODUCTION TO HIDDEN MARKOV MODELS AND BAYESIAN NETWORKS'. International Journal of Pattern Recognition and Artificial Intelligence 15.01 (2001): 9-42. Web.

Heckerman, David, Abe Mamdani, and Michael P. Wellman. 'Real-World Applications Of Bayesian Networks'.Commun. ACM 38.3 (1995): 24-26. Web.

Laramee, Francois. 'The Game Design Process - Game Design - Articles - Articles'. GameDev.net. N.p., 2015. Web. 25 Apr. 2015.

Li, Calvin. 'Dynamic Programming For Programming Competitions'. YouTube. N.p., 2015. Web. 25 Apr. 2015.

Russell, Stuart J, and Peter Norvig. Artificial Intelligence. 3rd ed. Englewood Cliffs, N.J.: Prentice Hall, 1995. Print.

Tsitsiklis, J,. '1. Probability Models And Axioms'. N.p., 2015. Web. 25 Apr. 2015.

Tsitsiklis, J,. '2. Conditioning And Bayes' Rule'. N.p., 2015. Web. 25 Apr. 2015.


AppendicesThe following content consists of full, uncut documents created throughout the design process of this project, some aspects of these documents have already been featured in the final report.

Additionally the documents regarding the game design process are available on the submitted disc.

- User guide- Project Initiation Document - Interim Report 1- Interim Report 2- Supervisory meetings - User feedback (full)


Appendix 1: User guide

Decision Network windows form

1.) Load a valid XML form under File->Open. (Two networks have been provided).

2.) You should see a list of nodes in the network if the network built successfully.

3.) In the textbox under "Type evidence" enter the nodes you want to give as evidence separated by a semicolon, e.g. "slash=1;lunge=1".

4.) In the textbox under "Type query" enter the nodes you want to query a joint probability for, e.g. "aggression=1".

5.) The query and evidence "adrenaline=1", slash=1;lunge=1 is essentially asking the program: what are the chances that an entity is being aggressive if they are performing slash and lunge attacks?

6.) Press 'Get result' and the full query string and probability of the query will show in the lowermost textbox.


Illustration 33: Form showing nodes in a network to the left

Beyond the Mountains

1.) Open the executable for whatever platform you want to play it on, this could be Windows, Windows Phone or Android.

2.) After the splash screen you will see a title screen, from here you can choose to play, adjust settings, read background on the game or exit.

3.) To control your character either press the left/right arrow keys or A/D if using a desktop/laptop or the left joystick if using a Joypad or tilt left/right if using a mobile device.

4.) If you run into skeletons you must fight to survive. You can attack in different styles with the left mouse button, x and z on desktop/laptops, the right joypad buttons or by swiping up/across or down on mobile devices.

5.) To block from enemy attacks either press the left mouse button on desktop/laptops, the left bumper on a Joypad or by holding down your finger on the screen of a mobile device.

6.) In the top left corner you will see your health and adrenaline, if your health runs to zero you will die and need to restart, you can do this by pressing R/Escape on desktop/laptops, Start on a Joypad or the back button of your mobile device. Enemies also have health bars and will die if their bars are depleted.


Illustration 34: The title screen on Beyond the Mountains

Appendix 2: Project Initiation Document

Table of Contents

1. Project overview (Page 56)2. Business case (Page 56)3. Main objectives (Page 57)4. Initial scope (Page 57)5. Method of approach (Page 58)6. Initial project plan (Page 59)

6.1. Control plan (Page 60)6.2. Communication plan (Page 60)

7. Initial risk list (Page 61)8. Quality plan (Page 61)


1. Project overview

For this module I have decided on a game of the side scrolling genre with particular emphasis on artificial intelligence which will be implemented to effect how the enemies in the game both evolve and adapt. As for my contribution towards the gaming and programming industry, I aim for the AI part of the game to be reusable in other games to make it easier to create interesting enemies.

I hope to include other core elements to the game to make it not just a demo but a feasible release, including core game mechanics such as movement, screen effects and a non-linear level design.

I have several reasons for picking this project in particular, in essence it brings together a lot of the skills I have learned in the past two years as well as providing myself with new challenges. I feel it is not only relevant to my programme but will also be a great learning experience which will help me in future projects.I see very few games apply interesting behaviours for the agents in their games, I hope to provide myself and others with resources which will help with this.

2. The business case

As well as building upon my knowledge of programming in C# and object oriented principles, by using the Unity game engine for this project I am provided with other opportunities to learn new skills. In particular, I have never coded substantial AI behaviours, my current practical experience with this is limited to creating asimple A* path-finding algorithm. Despite this, I have researched the topic and realized some of the capabilities the field of AI has in games.

In addition, I will look further into creating an immersive level by trying to improve upon my current skills in 3d modelling as well as choosing an interesting theme and level structure. The modelling work will consist of creating models for things such as the character, enemies and still objects, texturing the model and animating it if necessary. As this isn't the sole focus of the project I may make use of pre-made models if I need to move on to something else, for any models which aren't my own I will be ensuring I have permission to use them and will credit them as someone elses.

I aim to make the game code run as cleanly as I can, making use of technologies such as XML for loading levels and an external database if enough persistent data needs to be stored.

As I'm striving for as much modularity in my AI code as possible, I hope to have this available to plug in to other games via a code library, a stretch goal of mine is to write an article and provide this library for free onCodeProject and whether they accept it or not it will be available to download on my portfolio.I feel AI is still an underused technology in game design, in particular with indie titles who don't have a strong infrastructure available to them, by providing this library for free and showing a working example of it in useI hope individuals or a small team of game developers will make use of it.

There are many programs using AI with the source code available, though many are not directly applicable to games development or come at a cost, in this respect I feel there is a potential consumer base for what I am producing, though the main purpose of the project is my own learning.


3. Main objectives

There are several main objectives for the implementation of this project, in order these are:

1. Creating a bare-bones level design to leverage game mechanics

2. Adding the typical mechanics of side-scroller (player movement, spawning enemies, obstacles)

3. Creating a rudimentary evolutionary computation script with references to UML diagrams and I/O flowchart which evolve certain values.

4. Implement this into the Unity game to evolve enemy statistics

5. Expand upon this by granting enemies certain abilities at difference fitness thresholds

6. Integrate a proper game theme, graphical effects (lighting/shaders/particles) and design of models/backdrop

7. Develop the AI scripts further to ensure a good standard of functionality

8. Introduce use of XML for quick storing and loading of persistent data, also resulting in lower load times

9. Improve all previously mentioned areas of the project, introducing an SQL database to store data if necessary

4. Initial Scope

This section will be used to expand upon the previously mentioned objectives.

All steps mentioned will follow a schedule both in written and graph form, giving me rough guidelines on how long I should be taking on each part of the project in hope that I will stay on top of what needs to be done.

In order to properly test the code I plan to be writing, I will need to have a basic level in place with the relevant scripts attached to objects to analyse how they behave and make improvements. For this reason I will be jumping straight into the game engine and creating the basic level with elements which make up the game, these being a controllable character, a scrolling screen and the introduction of enemies. By doing thisI also have a real example of the beginnings of my game which I can continue to build upon.

The C# scripts will follow, after some research, practice and planning (in the form of UML diagrams and I/O flowcharts), I will be beginning to write scripts which make use of evolutionary algorithms to change values,which can then be applied to enemies in game and tested to see if the enemies object values are changing based on the algorithm in place.

As I feel the enemies are the main feature of the game, they are the first feature I will be going into further detail with, adding more behaviours that will make them more intelligent such as abilities which are enabled at certain thresholds and activated at appropriate times.

By this time, I should have a presentable demo in place and as such a product which can be briefly reviewedby colleagues, and from here I will be mainly taking steps to improve the overall experience of the game, introducing more elaborate graphics, themes and a story.


From here my concern is making the aspects of the game work as expected, improving performance and theAI side of the game.

An article with the code library for the AI will be written and published on my website and sent to CodeProject towards the end of my project if I feel satisfied with the algorithms produced and feel they are at a standard which can benefit others.

5. Method of approach

I will be creating a schedule which I aim to roughly follow in the hope of staying on track with my intended implementation, a Gantt chart will be used to visualize this. I will be using project management software (possibly Pivotal Tracker) throughout the projects development and regularly updating a blog to discuss progress, any issues and how I overcame them as evidence of my learning. I will be keeping in regular contact with my supervisor if I need advice or need someone to review certain areas of my work.

Ideally I would like to follow a game design process, beginning with an analysis on the products features andtarget audience, then having a preliminary design discussing game rules, content and behaviour followed bya product specification with further information on how the features will be implemented with particular focus on AI. From here an initial game design document will be created and updated once significant changes have been made to the implementation. Once the core of the game is created and I am looking more into the games theme and visual aesthetics I will be creating a graphic bible, which essentially describes the look and feel of the game with real examples.

I will need to research any technologies I'm less capable with before applying them, most notably this concerns my 3d modelling skills and familiarity with coding genetic algorithms. Both online and written tutorials will be used to help advance these skills. I have at least some prior knowledge in these two fields so I'm not going in blind.

I know the majority of the tools I will be using, but there is always a chance of other helpful software coming to my attention, the initial list is:

Unity – Game engineVisual Studio - IDEPivotal Tracker – Project management3ds Max – 3d modelling toolGIMP and Photoshop Elements – Image editingWordpress – Blog


6. Project Plan

Stage Deadline Products/deliverables/outcome

1.) Initiation 15/10/14 PID and setting up software

2.) Semi-detailed schedule for references, design proposal

19/10/14 List of when milestones should be achieved, visualized in Gantt chart.Priority list of game features.

3.) Preliminary design and initial Game Design Document

24/10/14 Preliminary design discusses game rules, content and behaviour.Game Design Document will be more detailed discussing many aspects of the project

4.) Bare-bones level design and basic mechanics

26/10/14 Unity project with level featuring character controller, rudimentary camera and enemy spawner

5.) Revising evolutionary computation with planned use

09/11/14 Initial UML/Flowcharts for I/OFurther knowledge on the subject

6.) Initial (basic) AI implementation and integration with game

02/12/14 Series of scripts which evolve values using evolutionary algorithm(s) applied to enemy in game with evidence of it working (at least partially)

7.) Interim submission 03/12/14 Proof of progress

8.) Further progress with AI section of project

Continuous Refining functionality, performance and modularity of the code library

9.) Begin graphic bible 01/01/15 Planning and writing game theme, further details on look and feel of project (GUI included)

10.) Improved models, level design and theme

Continuous Following graphic bible, making adaptations to the graphic bible where necessary to keep relevance

11.) Second interim submission 04/02/15

12.) Final report, electronic poster,finalized GDD

29/04/15 PDF report <10000 words,

13.) Demonstration Week starting 04/05/15

Presentation with necessary assets to show off project

Blog updates and use of project management software will be done throughout the project and referenced in interims/report. Testing will be done throughout the project.


6.1 Control Plan

Continued use of project management software to determine current user stories to be worked on, bugs to be resolves and refinements to make.

A Wordpress blog will be regularly updated discussing progress with different parts of the project, issues I have encountered and how I have overcame them as well as providing a reference to remind myself where Iam in the project.

I will plan meetings with my supervisor if I need something reviewed or need help with a significant problem and record any feedback he has given me (minutes) either on my blog or in a local document.

Interim submissions will be uploaded to the DLE as a detailed reference to where I currently am in development.

I will be trying my best to avoid any potential issues that could arise in the project by using version control software and creating backups, more common risks are discussed in the risk management section.

6.2 Communication plan

As aforementioned, meetings will be mostly based upon whether I am having any issues with what I'm working on, this has been decided and agreed upon from the first meeting which took place on the 3 rd of October.

In addition to this I would like him to review my progress whenever significant progress has been made, he will also be providing feedback for any technical deliverables such as interim submissions.


7. Initial Risk List

Risk Management Strategy

Accidental loss of information Local backups and GitHub repository for regular pushing to/pulling from server

Loss of functionality in recent development Use version control software to revert changes (GitHub)

Lack of confidence in ability with certain areas of project

Revision of any parts of project I'm less capable withbefore attempt of independent implementation

Networking errors (if incorporating database) Ensure software isn't fully dependent on database and can still run largely without it

Running out of time or time left over Project is very open to downscaling/upscaling

Bugs not caught by compiler Regular testing from myself and colleagues

Prioritisation of less important elements Initial schedule should help reassure me of how much time I have to do certain parts of project

8. Initial Quality Plan

Quality check Strategy

Relevant features Research of existing games which have similarities with mine, sticking to features I know I can implement sufficiently and writing a priority list to ensure the more important features are implemented.

Game experience Myself and colleagues will play through the game to ensure there are no obvious design flaws which make the game unsatisfying to play.

Performance I will be running the game on different graphical settings and hardware as well as minimizing complexity of 3d models which is a typical culprit of slowdown.

Accurate to specification I will be creating relevant documents and continuously use project management software as well as have a blog for reference to help ensure the direction I take development remains consistent with my project idea.


Appendix 3: Interim Report 1

Table of Contents

Overall schedule with current progress 63

Sprint 1 (23/09/14 – 24/10/14) 63- Goals 63- Details of tasks undertaken and outcomes 64- Use of resources 64- Deviations and risks 66


Product quality 68

Continued practices 68


Overall schedule with current progress

Sprint 1 (23/09/14 – 24/10/14)

This sprint involved coming up with ideas for the project, submitting a proposal for the project and other formal documentation concerning planning and reviewing the project with my allocated supervisor as well as setting up the software required to implement the project and maintain a sustainable work-flow.

Goals

• Submission and approval of project proposal ✓ • Allocation of a supervisor ✓ • Submission and approval of PID ✓ • Review of project with supervisor ✓• Registration, installation and familiarisation with the following software:◦ GitHub ✓ ◦ Pivotal Tracker ✓ ◦ Unity 4.5 ✓ ◦ Visual Studio 2013 ✓ ◦ Wordpress blog ✓ ◦ LibreOffice Calc ✓ ◦ LibreOffice Writer ✓

• Creation of schedule with automatic updating ✓• Priority feature list ✓• Preliminary game rules ✓


Details of tasks undertaken and outcomes

Having had the project proposal and PID for my project approved and discussed with the supervisor my first task was to set up the software I was going to use, performing installation and creating accounts where necessary and familiarizing myself with the different technologies, the following software was used:• GitHub – version control for backups if something goes wrong.• Pivotal Tracker – breaking down project into manageable user stories.• Unity game engine – the platform used to make the game.• Visual Studio 2013 – my chosen IDE for wirting the necessary scripts.• Wordpress blog – providing less formal information on how I'm getting on.• LibreOffice Calc – to create schedule• LibreOffice Writer – to write the documentation for project

Following this I created a project schedule in LibreOffice Calc, a Gantt chart showing what needs to be done and when where, there are three main sections to this being: • Documentation – game design process and research• Implementation – creation of assets used in the Unity project• Deliverables – each module of work which contributes towards final mark, serves as a reminder for

deadlines

By creating a schedule as a spreadsheet it is much easier to update and follow where I'm currently at as the visual representation using a Gantt chart is automatically updated with the percentages of tasks complete.

Following this schedule and referring back to it where necessary I have began the game design process by writing a priority list for the features in my game which outlines basic mechanics then what could be done to improve them, these features are listed from top to bottom where top is the most important and bottom is the less important.

I also wrote a list of game rules which talks about restrictions the game places on you such as only being able to move in certain directions. These documents will help me stay on track with what I really want to achieve and can be referred to at any time.

Use of resources

For version control I chose to use git with GitHub as I have been using it for around a year now and I'm familiar with its features and interfaces. By using this I can install a local user-friendly program which allows me to push/pull without having to fiddle around with the command line. This is an extremely useful technology for reverting to previous versions of files if something goes wrong, as well as allowing sharing between others if I want testing to be done.


Pivotal Tracker is used to manage user stories, when writing these I aim to look from a consumer perspective on what I would want to see with the proposed project, each story is broken down into smaller tasks and assigned weightings for a rough indication on how long they will take.

The Unity game engine was set up to work on the actual game itself, by using this software I can arrange everything that makes up the project into a structured hierarchy of assets. The IDE to use alongside this is Visual Studio 2013 as its my personal preference, it has all the features I need and a large following if I run into any issues with it.

I had an existing WordPress blog which was created last year, I have added a category and a couple new blog posts to this which is a way of casually informing people with what I'm currently working on and providing exposure to my work.


Deviations and risks

At this early stage as the sprint was mostly planning no major risks or deviations to the project have arisen, my supervisor expressed that his skill-set is mostly tied in with the AI part of the project and I should look into talking to a member of staff from the games development program at some point in the future for guidance and refinements with the quality of the actual game.

As I'm juggling quite a few tools to try and maintain a structured approach to my work-flow I need to be sure to prioritise the actual research and implementation stated in my project PID over recording progress, this is not too much of an issue as I have experience with these tools from previous years of work but will need to bear it in mind.

Sprint 2 (25/10/14 – 19/12/14)Incomplete sprint

This sprint is where I start getting the game up and running as well as do research into the AI component to be used within the game. I will begin to create assets such as scripts for basic game mechanics and some basic artwork. Towards the end of the sprint I aim to have a very simple Bayesian inference implementation building off of an existing library and a presentable test level.

Goals

• Bare-bones level design ✓ • Player model ✓ • Movement mechanics ✓ • Jumping mechanic ✓ • Movement animations ✓ • Follow-camera ✓ • Concept art ✓ • Research into conditional probability, Bayes' Rule and Bayesian inference -• Initial AI scripts with integration -


I began designing the test level which initially just had primitive shapes for models for platforms, characters and obstacles which developed into more detailed models later on.

For the character I am using a rigged model available to use for educational purposes from Unity's asset storewhich was then animated using the Mecanim system in Unity.

I created some very basic mechanics such as player movement (left/right) and jumping, and making the camera follow the player and lerp (change position/rotation in a decremental way) based on user input, these are scripts written in C# and attached to the game objects which require the behaviours.

I developed the art style further to give myself a better idea on what works and because the game theme effects future mechanics and behaviours for both players, enemies and still objects in the scene. I took an interest into the Traditional Japanese art style seeing as it wasn't being used in any games I could find and decided to have it influence the games visuals. For reference I created an image board consisting of art and game screen shots from other side scrolling games

I have also begun looking into the AI component of the game, deciding that I wanted the main purpose of it


to be making enemies adaptive based on player behaviours, mainly concerning the processes of enemies being able to decide when to dodge and attack, similar to the original purpose for including the AI however focusing more on enemy decisions opposed to enemies evolving their abilities. My supervisor and I had a discussion on what methods can be used to achieve adaptive AI and we decided that Bayesian inference was a promising solution.

I began research on conditional probability and Bayes' rule, the precursor to looking into Bayesian inference networks. I followed several tutorials such as videos under MIT OpenCourseWare to familiarize myself with the concepts. I'm writing a document (continuously being worked on) which will be used as a reference to these concepts if I forget something, self-made diagrams have been added to the document to help with this.

Use of resources

I have been working a lot with the Unity game engine again, making use of its multitude of features incorporated within it to help get my game up and running reasonably quickly.

I created the 2d artwork used in the game with Photoshop Elements, a program already set up on my computer.

I recorded my first progress video with EzVid which was then updated to my WordPress blog.

I have been using YouTube for various video tutorials on conditional probability and Bayes' rule.

Continuous updating of project management software (GitHub, Pivotal Tracker, etc.).


There has been a fairly significant change in the way I want the AI component to effect the game. The originalplan was to use a genetic algorithm to evolve enemies, with some research I found that whilst genetic algorithms can be used to implement interesting behaviours it is often computationally demanding and thereare other algorithms which could introduce the kind of behaviours I want for the enemies.

The main dynamic the AI component was supposed to bring was to make enemies adaptive depending on the situation of the player, with this as the primary point of interest discussion with my supervisor lead to the


decision to use Bayesian inference which assigns probabilities to an array of outcomes in a sample space.

Having researched the concept I found it does have existing applications in game development and that it is apopular technology with applications to many areas in industry.

Partially due to my uncertainty with how exactly I was going to use the original idea of genetic algorithms within the game the time of which I developed the art style differs from the original plan stated in the PID as my initial goal was to begin the AI of the project at this point, realizing this was a large task in terms of both research and practical implementation I wanted to do more with the game before jumping into it so there's more to show at an earlier stage in the module and to help guide me on how exactly I approach the enemy behaviours which is hugely relevant to the planned use of Bayesian inference.

By making the decision to prioritise the look and feel of the project earlier than planned I have delayed progress with the AI component of the project which could be seen as a risk though the initial schedule has leeway for these kinds of circumstances and nothing has really been added, rather I'm just choosing to move around tasks.

Product quality

I've taken further steps with the Unity project to make it more professional and presentable such as adding animation and particle effects as well as elaborating on the art style before going into the AI component of the game.

I have created formal documentation which neatly describes my current research into Bayesian inference to use as reference.

The project hierarchy involves sensible naming conventions and documentation being separated from the Unity project files for clarity.

I have had a couple colleagues alongside me test the game and offer critique on what could be changed.

I have kept in contact with my supervisor to stay on the right track, reviewing current practices and listening to any suggestions he may be able to offer me.

Continuous practices

Meetings between myself and my supervisor have been organized when I want to review my progress or need suggestions for certain aspects of the project. There have been three meetings so far, the first discussing the PID, the second talking about the approach to take to create adaptive AI within the game and the third reviewing current progress and what should be included in the interim submission. During these meetings I wrote down some helpful points the supervisor made which are saved as 'minutes' and then forwarded them into the project meeting record.

The blog has been updated with each significant step in progress, it includes recordings and snapshots of the game being played, discussions on certain aspects of the project, some of my upcoming plans, code snippets and reviews of recently added features.

Project management software has also been kept up to date, I am constantly pushing any new progress to GitHub to keep it up to date and adding user stories to Pivotal Tracker before I begin to work on them.


Appendix 4: Interim Report 2

Table of Contents

Overall schedule with current progress 70



Product quality 79

Continued practices 79


Overall schedule with current progress

Using Bayesian Networks for enemy decision-making in a 2.5D side-scrolling game 70

Sprint 2 continued (25/10/14 – 19/12/14)

This sprint is continued from last term where I start getting the game up and running as well as do research into the AI component to be used within the game.

Goals

• Bare-bones level design ✓ • Player model ✓ • Movement mechanics ✓ • Jumping mechanic ✓ • Movement animations ✓ • Follow-camera ✓ • Concept art ✓ • Research into conditional probability, Bayes' Rule and Bayesian inference ✓


Continuing from the last interim this sprint only lasts around two weeks before half term begins, however some additional research was performed in the time between the first interim and the end of the sprint.

I continued learning about conditional probability with some practical examples being completed demonstrating my developing understanding of the subject.


Illustration 35: Intersecting probabilities

Illustration 36: Conditional probability using a tree model


Illustration 37: Bayes rule examples

This research is a precursor to when I want to use my own Bayesian Network for enemy decision making. It's helped me to familiarize myself with common terms, understand the meaning of inference as well as introduce myself to common problems in the field of conditional probability.

Use of resources

Online tutorials were used to help my research, primarily MIT's OpenCourseWare which covers a large range of topics in the field of computing and other technology courses.

The diagrams themselves were simply created in Microsoft Paint and cleaned up from prior versions so I canuse them as a clearer reference in the future.

The research so far has been compiled into one document created using Google Docs.


Illustration 38: Intersecting probabilities 2

I have continued to use Pivotal Tracker and GitHub to update my progress and provide version control for the current version of the game, though no further changes to the game were made in this particular sprint since the last interim.


In this sprint I have yet to really begin working on my coded implementation of a Bayesian Network due to wanting to understand the topic and so I have a more structured approach to tackling the problems I will face. I believe spending the extra time making sure I understand key concepts may put me slightly behind atthis point but provide myself with the knowledge I need going forward.

As for the games progress, it was mentioned in the first interim the games art style was established further along with player animations before initially intended which gives me a workable prototype early on and lets my plans for the projects direction in terms of its look and feel become more apparent.


Illustration 39: Using Google Docs for compiling research

Sprint 3 (06/01/15 – 04/02/15)

This sprint develops both the AI component of the project and the game itself further. I want to look at existing implementations of Bayesian Networks as well as research how Bayesian Networks should be implemented in terms of programming as well as research the causal nature of them and how they can be used efficiently.

For the game, I aim to tighten controls and establish a primitive graphical user interface (GUI) which can be extended upon in the future as well as prepare for the player to be able to interact with the intended Bayesian Network once it is integrated with enemy behaviours.

Goals

• Research Bayesian Networks. ✓ • Find existing implementation of a Bayesian Networks. ✓ • Find algorithms which can be used for suitable computation for Bayesian Networks. ✓ • Establish an initial Bayesian Network. ✓ • Integrate the Bayesian Network into the game -• Add further animations to game. ✓ • Add title screen in game which directs to play and settings. ✓


Throughout this sprint I have been referring to various video and written resources to help develop my understanding in regards to the concept of Bayesian Networks and how they are implemented. A set of nodes are organized into a graphical model (typically a tree) and effect each other through influence. As for what is effected, it takes place in the states within each node, these can be binary or more complex, and through the changes in these states the child nodes states will also update.

Because of the large amount of states the system will potentially need to compute when performing inference (finding probabilities based on incomplete evidence) the problem is that the task is exponential, because of this I have also researched Dynamic Programming, essentially increasing the amount of storage in exchange for less demanding computation. Ultimately this is beneficial due to computations that have already been performed being “remembered” (stored in a dynamic array).

Ideally exponential problems can be scaled down to be performed in polynomial time, for example rather than using nested loops to iterate through many different probability calculation a sequence of loops can beused.


I am working from an existing implementation for CodeProject to create my own Bayesian Network, currently the application allows the user to perform inference on four binary nodes and is executed as a Windows form application. This is coded in C# and ran from Visual Studio, the game engine being used for the game also uses C# as its primary programming language so it can be integrated easier. An image of the application running can be seen below:

As for the game itself, I have added further animations using Unity's Mecanim system. The player can now attack in three styles as well as block and fall from platforms, this is demonstrated below.

The Mecanim system provides a graphical interface to create animation states which play the chosen animations, this is shown below with the current states for the player:


Illustration 41: New animations added to the game

Illustration 40: Performing inference on nodes loaded through XML

What actually happens in these states (besides the animation details) can be further defined through scripting.

As well as introduce new animations I edited existing ones to be more responsive to player input by lowering transition times between the states as well as change their import settings to orientate them properly.

A title screen has also been added to the game and will appear when the game is first loaded. This is shown below:

This title screen is created with Unity's 'OnGUI' function to instantiate GUI elements such as buttons and boxes to be used as backgrounds. I then extended the GUI to animate when the buttons are clicked, when the settings our about options are pressed, a new window slides in from the right representing the


Illustration 42: The current animation states used by the player

Illustration 43: The title screen for the game

corresponding information depending on the button which was pressed.

As the game is best played using a controller, I had to use a particular GUI layout to allow iteration between buttons with analogue control known as a 'SelectionGrid' where the index indicating what GUI element is selected can be updated with joystick controls.

A simple theme song has also been integrated into the game and plays on the title screen, this was composed on Garage Band available on iOS/OSX.

As well as practical development, I have created a Game Design Document which is a specification detailing all aspects of the game in its current stage, detailing things such as the scope of the game, intended platforms, features and resources used.

Use of resources

The Unity3D game engine has been used for the game development, in particular its built in Mecanim system to organize player animations, the scene editor to organize game objects in the game and the inspector to view and configure game object properties.

Pivotal Tracker was used to create and update user stories, GitHub was used for my git repositories online using an ssh key to push changes without needing to log in.

Visual Studio 2013 was used as the IDE for C# code editing, this is true for both the GUI and animation scripts as well as the Bayesian Network program.

GarageBand was used on an iPad Mini to compose the theme music, it was then converted from an m4a fileto an mp3 file online.

Wordpress is being used for my blog, with each significant step in progress I'm aiming to update it using the tools provided.

Various video and written sources were used for research in Bayesian networks, inference, dynamic programming and bucket sort algorithms, links are available below:

A Brief Introduction to Graphical Models and Bayesian Networks

http://www.cs.ubc.ca/~murphyk/Bayes/bnintro.html

Inference in Belief Networks

http://www.codeproject.com/Articles/3320/Inference-in-Belief-Networks

19. Dynamic Programming I: Fibonacci, Shortest Paths

https://www.youtube.com/watch?v=OQ5jsbhAv_M

Dynamic Programming for Programming Competitions

https://www.youtube.com/watch?v=Ad0DKl7HUwI

Lecture 10a: Bucket Sort, Counting Sort - Richard Buckland

https://www.youtube.com/watch?v=3mxp4JLGasE


https://www.youtube.com/watch?v=Ad0DKl7HUwI

https://www.youtube.com/watch?v=OQ5jsbhAv_M

http://www.cs.ubc.ca/~murphyk/Bayes/bnintro.html


The main concern currently is being able to create an effective Bayesian Network without using binary states as I want the players adrenaline value which will be a value from 0 to 1 in increments of 0.01 maximum to be the root node of the model in use, I will explore this further in the next sprint.

I had planned to have some basic integration with the game at the end of the sprint, with recent deadlines in other modules this has unfortunately been delayed but is currently being actively worked on. Seeing as Unity uses C# and JavaScript as its primary programming language there shouldn't be any major difficulties integrating the scripts when they are ready.

When first looking into the possibilities of using a Bayesian Network for enemy decision making I hadn't thought of inference being quite computationally intensive which has had me look into dynamic programming, as this seems to be quite a straightforward topic it's only a small addition to make to my code.

Product quality

I want to ensure the AI component of the project is implemented to a good standard hence the thorough research on key topics relevant to it.

I have continued to keep the game project files organized using sensible naming conventions and the expected filing for the different type of asset used.

I have created a Game Design Document outlining the current progress of the game which will be regularly updated with each significant addition or change.

I have made steady progress with both the game itself and the Bayesian Network which will eventually be integrated with it, testing changes to code in both projects regularly to ensure no loss in functionality or unexpected behaviour.

Continuous practices

My personal blog has been updated further with the key parts of the project I am working on, this provides others with similar interests an insight on what I'm currently doing and some proof of my understanding of the different aspects of the project.

I have had some contact with my supervisor though unfortunately a meeting hasn't taken place since the beginning of the second interim due to unforeseen circumstances with my supervisor and half term.


Project management software has been kept up to date, I have set up another Git repository for the Bayesian Network application which will eventually be integrated with the game, the separate repository will remain for others who are interested in using it with their own projects. I have been creating and updating user stories on Pivotal Tracker which helps schedule the different tasks undertaken as well as serveas a reminder on what needs to be done next.


Illustration 44: Current stories in Pivotal Tracker

Illustration 45: Git information for the new Bayesian Network repository

Appendix 5: Supervisor meetingsStudent: Steven Hooper Supervisor: Serafim Rodrigues Date: 03/10/14

Items Reported

PID and proposal

Topics discussed

Discussed overall project aimsEmphasis on making AI section of project modularCompile to a code library Aim to release article with code library on codeprojectUse a flowchart to clearly define inputs and outputs of libraryEnsure a quality product due to a large portion of the marks being given to front-end aspects Whenever faced with a notable issue, contact supervisor

Action Plan

Modularise code so it can be reused

Propose article to CodeProject

By whom?

Serafim

Serafim

By when?

Throughout

When AI component is finished


Student: Steven HooperSupervisor: Serafim RodriguesDate: 21/11/14

Items Reported

Unity project with moving character and animations, beginning to establish art style in the game with mix of 3d models and 2d game art and particle effects. Established asset hierarchy.

Topics discussed

The approach to the AI component of the gamePossibilities to create adaptive AILook into how physics could be usedBayesian inference, conditional probabilityIn terms of the AI component, start small and scale upNewtons mechanicsTable of actions (sample space in conditional probability)

Action Plan

Research Bayes' Rule, conditional probability and Bayesian inference

By whom?

Serafim

By when?

28/11/14


Student: Steven Hooper Supervisor: Serafim RodriguesDate: 28/11/14

Items Reported

Documentation and diagrams showing research on Bayes' rule and conditional probability.

Topics discussed

Look for practical examples of Bayesian inference specific to games.(http://www.codeproject.com/Articles/3320/Inference-in-Belief-Networks)Work off an existing library to save time working with the fundamentals of Bayesian inference.Review game with Dan Livingstone for guidance on game features, mechanics and presentation.

Action Plan

Create a small example of Bayesian inference in use using an existing code library.

Discuss game with Dan Livingstone

By whom?

Serafim

Serafim

By when?

19/12/14

Before second interim


http://www.codeproject.com/Articles/3320/Inference-in-Belief-Networks

Student: Steven Hooper Supervisor: Serafim RodriguesDate: 02/02/15

Items Reported

Further animations, Game Design Document, initial use of Bayesian Inference implementation, planned interim submission 2, progress update with Dan Livingstone

Topics discussed

Interim report 2 brief review

Discussed how the Bayesian Network will be properly implemented in the game with enemy animations working with it.

Briefly mentioned Dans thoughts on the project

Discussed what coding methodologies are expected

Action Plan

Focus attention of the full integration of Bayesian Network

Further use of Hungarian notation for variable names and properties

By whom?

Serafim

Myself

By when?

Next meeting

ASAP


Student: Steven Hooper Supervisor: Serafim Rodrigues Date: 10/03/15

Items Reported

Mobile version of the project with accelerometer and touch controls

Full integration of Bayesian Network with enemy animations

Topics discussed

Scheduling for project demonstration

The basic framework has followed the plan and been established properly, now it's just building on and improving what's there.

A publishable paper is an option, after completing a draft of the final report review it with Serafim and he will contact the right people to get advice and assistance on publishing a paper.

Key areas of report: How the project is implemented, what development practices were used,why the provided solution is suitable for the problem being solved, mention the new things I have learned in detail, provide proof of user testing.

Action Plan

Email first and secondary supervisors a suggested time for demonstration

Build upon existing framework

Provide proof of user testing

Draft final report

By whom?

Serafim

Serafim

Serafim

Serafim

By when?

ASAP

Next meeting

Next meeting

First week in April


Appendix 6: User feedbackWritten feedback

OK, just played it:

I think the music could stay at the same volume as the main menu, seems a bit on the quiet side otherwise.

I don't get much feedback from combat, as I am able to spam attacks and let the animation run its course, where I might expect it to change based on the enemy, such as stuttering when hit or my sword bouncing of their shield, maybe adding rumble to the controller could help?

Are both skeletons different tests? As I could only damage the one on the right.

Does the meter under the health do anything yet? Looks like it will be some sort of special attack?

There is a bit of delay when jumping which feels a bit odd, maybe a more responsive jump will be more satisfying.

I was able to double jump by spamming the jump button when jumping from the top of a slope and managed to land behind the left skeleton, who then ignored me, maybe add some proximity code where if you are right next to them they will notice you?

Unless you are planning on stealth kills, then maybe just increase their sight to include the air in front and above them.

The analog movement has just two speeds, maybe try fully using the analog so tilting varying amounts varies the speed accordingly, with running still occurring after tilting a certain amount.

I think for me, the main issue was the lack of feedback during combat, everything else was either fine or minor issues which aren't a big deal. Finding ways of adding extra animations, particles, sound effects and even battle music when battling would add a lot to it.

The controls are a bit fidgety, but that's mobiles for you, tilting to move mainly, I did like the swipe attacks, do I jump by pressing both thumbs down or swiping both thumbs, couldn't quite tell.

I'm not sure what I would do about tilt controls, I think there needs to be more of a space between walking and running, I should be able to tilt more before breaking into a run.

I would also consider another method such as either a virtual dpad or holding the left or right edges of the screen to move in that direction. I wasn't expecting so many hidden paths when jumping off random ledges!

Signed:

Verbal feedback

Tilt less to move left and right.

Background should scroll with player.

Signed:


Project showcase


Date post:	21-Jan-2017
Category:	Documents
Upload:	steven-hooper
View:	288 times
Download:	1 times

Individual Project - Final Report

Documents