Player Modeling

James Morrison

Previously and Currently Currently, many

games have only player-selected difficulty- “manually adaptive AI”

This is implemented in many games, as far back as Doom, and also very recent ones

(game play demonstration)

Currently Many games don’t

even implement multiple difficulty levels

This leads to games being criticized for being too easy or too hard.

Manually Adaptive AIThis is may be the most used method of

selecting difficulty, but it is far from best.Consider the average gamer: What

difficulty should be picked? How do you know?

What if the gamer specializes in a given genre of game?What if it’s the same as the game in question?Different?

As challenge has been linked to player satisfaction, what should be done?

The FixThe fix for this problem is automatically

adjusting to challenge the player.This presents a few problems...

How to change the difficultyIs just a single measure enough?

How to know how to adjust the difficultyDetection of player actions/consequences

This is all handled with Player Modeling.

What is Player Modeling?A player model is an abstraction of the

player.A few uses of player modeling:

Model the skill of the player so the game can adjust difficulty

Model weaknesses so they can be exploitedModel actions so the player can be predicted

Creating a Player ModelWhen designing a player model, one has

to decide what traits should be modeled based on the gameFor example, UsesSmokeGrenades works

just fine for Counter-Strike, but not at all in Oblivion

What actions the player can take are a good starting pointi.e. UsesDodge, UsesPowerAttack

Model DetailsDoes a high rating in an attribute mean

that the player is highly proficient or uses it often?

How do you integrate the model into the AI?

Updating and maintaining a model of any significant detail can be quite a task...

Usage vs. ProficiencyExample: UsesSmokeGrenades

A use model would have a high rating for the player if the player uses a high number of smoke grenades

A proficiency model would have a high rating for the player if the player uses smoke grenades intelligently

Use a grenade when retreating, or when outnumbered, for example

Initializing the ModelThere are a few ways of initializing models

for players in a game setting…Initialize with ‘neutral’ values (.5 on a 0-1

scale, for example)Pre-learn using game testers to figure out a

good baseline for the ‘average’ playerSet the levels arbitrarily high or low

Initializing with Neutral Values Initializing with a neutral value would mean

that your AI starts on what is effectively it’s middle difficulty.

This leads to some players being bored (because it’s too easy) or too challenged in the case that it’s too hard.

This also means that all players start as generalists and move from there, instead of the game recognizing their preferences from the start.

Pre-LearningThis would set the values to what the

majority of players find about right when they start the game.

This would require a significant amount of testers, and this also prevents the game from recognizing player’s preferences from the start if they significantly deviate from the average

Arbitrary Values In this initialization, the values are set by

the developer, using their judgment as to how to start the player.

This could lead, as in neutral values, to some players being bored or over-challenged.

This type of setup also allows players to pre-input their play preferences (a la character creation in Oblivion or many games based on the D&D system)

Updating the Model:Two steps:

Deciding when the model needs updatedthis involves finding out when a player has

succeeded or failed at a particular taskThis can be computationally very expensive,

especially in the use of a passive skillCalculating the New value for the model

updateTrait(traitName, observedValue) Often a machine-learning algorithm called the least

mean squares method is used… traitValue=α*observedValue + (1- α)*traitValue Where α is what’s known as the learning rate

An Example Player Model

Using the ModelNow that we have a model, and it’s

initialized, how do we actually use it in our theoretical game?

This brings us back to it’s uses:Model the skill of the player so the game can

adjust difficultyModel weaknesses so they can be exploitedModel actions so the player can be predicted

I’ll go in depth with all three of these…

Adjustment of Difficulty This is the most

obvious of the three, and possibly most useful

As the player progresses, tailor the encounters to the player

Usually, a challenging but beatable encounter is ideal.

Adjustment of Difficulty Usually adjusting the

difficulty to suit the player has the effect of increasing player satisfaction.

When done incorrectly, however, or when the wrong metric is used, it can feel unfair or too easy

Modeling in OblivionThe game Oblivion two sorts of player

modeling behaviors.At the beginning of the game, the player’s

actions are tracked, and a suggested class is offered.

This system works relatively well, and for many players, will give a good class based on how they performed in the beginning of the game.

Demo!

Later in OblivionAfter leaving the tutorial zone in Oblivion,

the game begins keeping track of the character’s level.

This is used as a metric for adjusting the difficulty of encounters within the game.

What are some possible problems with this system?

Yet another gameplay demonstration

Modeling WeaknessesThis is specific to the proficiency type of

modeling…Using a model to detect player

weaknesses can be useful both for helping the player and harming him.

At the beginning of a game, if the player is significantly deficient in some area of gameplay, more tutorial time could be spent on the deficiency.

Later, a deficiency can be used to increase the difficulty of a given encounter.

Modeling ActionsSpecific to the Usage amount type of

modeling… If a player has a high rating in some

aspect, it is likely that is the aspect of the game to be used.

As an example: If a player is known to use a lot of smoke grenades, keep the smoke grenade animations and sounds cached for faster loading times.

Variations and Other UsesPlayer modeling however, has many more

uses then just these few uses and one design.

Besides the obvious changes in how the model learns and changes (with a change in the updateTrait() function) there are many other possible variations.

Hierarchical Player Modeling In a hierarchical model, only the leaf

nodes are actual concrete descriptors.All parent nodes have only derived values

based on their children (usually the average)

An Example Hierarchical Model

Why Have a Hierarchy?The hierarchical model allows more

resolution in the model, while still allowing overarching statements to be made about the player. i.e. the player is a stealth-centric character,

but prefers the sap to the bow.This increased resolution allows the

designers to adapt even further to the playerPerhaps dropping better saps, and fewer

arrows in the above example…

Player Modeling for Interactive Storytelling

Finding player preferences can also be used to direct the flow of the game.

In Knights of the Old Republic, your character moved towards the dark side or light side depending on his actions, and this changed the story and dialogue to some extent.

Theoretically, a system has been designed that will give only player-interested side quests to the player.

Interactive Storytelling In the second case, imagine a choose-

your-own-story game, and where the player never intentionally makes the choices.

With different players acting differently, or the same player playing different characters, the play experience is changed every time

This was studied using the Neverwinter Aurora toolset, and was shown to have higher player-satisfaction values

An example StoryThe story follows very roughly the story of

red riding hood (if the wolf was a troll, and the grandmother was a wizard)

The Results of the StudyThe Study showed

that all user groups found more fun and more influential to the game world

This was especially true of the three specific groups shown.

• The Study showed the certainty factors for 3 groups and the entirety of the study• Groups were

• Females who found the game easy to follow

• Females• Those who found the game easy

to follow • Everyone else

Model SharingAnother possible use for player models

would be sharing the models between games.

This is mostly useful between games of the same genre, or more often, games of the same series (halo 1, 2, 3) as games have different actions available to the player

Model SharingModel sharing would allow players to truly

take up where the left off playing another game.

Players starting ‘PlayerModeled game 2’ would be already modeled and the AI would then have no (or a very small) learning curve.

This reduces the time where the player is uninterested due to the game being too easy or too hard.

In ConclusionPlayer Modeling is not so much an AI

technique, but one that allows customization of the AI for maximum player enjoyment

Player Modeling can also be used to change the story or other aspects of the game to further increase the enjoyment of the player.

In ConclusionPlayer modeling has yet to be fully

implemented into a game, and is still being researched.

Some games have tried implementing simpler types of it, but they tend to fall short in some way.

Thanks for listening!