AI to Control

Pacing in Games

Borut Pfeifer

• White Knuckle Games – Reality’s Edge, 2001-2003

• Radical Entertainment – 2003

• Articles in Game Programming Gems 4, various others

Motivations & Goals

• Bad AI Programmer, No Twinkie – Max Payne, Medal of Honor. Well paced, but too heavily scripted.

• Keep strong pacing without sacrificing replayability.

• Take user’s skill level into account to maintain flow, for a more consistent experience.

• Eliminate the need for arbitrary static difficulty levels & the development they involve – who likes picking easy anyway?

• Create strong pacing while allowing user to roam freely.

Dramatic Tension

Measure of complication

Tension

Time

Climax

Dénouement

Dramatic Tension

In reality…

Tension

Time

Climax

Dénouement

Dramatic Tension

Brenda Laurel - Computers As Theater

Drama as a model for interactivity:

• Enactment - The dramatic model focuses on action, which also defines interactivity

• Intensification – incidents can be selected and arranged to condense time.

• Unity of action – causally linked actions with a strong central action

Flow

Mihaly Csikszentmihalyi - Flow: The Psychology of Optimal Experience

Flow – the state in which a person is so involved in an activity that nothing else seems to matter

Studied thousands of people, including experts (chess masters,

artists, athletes, etc.) to help shed light on the flow experience.

Used “experience sampling” – subjects wore pagers and had to write down how the felt and what they were thinking when they received pages. Over a hundred thousand samples have been collected.

Characteristics of Flow

• Challenging activity that requires skills Stimulates some part of the brain (spatial, visual, logic, memory,

attention, etc.).

• Merging of action and awareness What you’re you’re paying attention to consists of things related to

achieving your goals.

• Clear goals and feedback Player always knows what they need to accomplish, and is receiving

continuous feedback as to how well goals are being achieving

• Concentration on the task at hand A high degree of concentration must be required to achieve the goals.

Characteristics of Flow

• Loss of self-consciousness Because of complete concentration on the task at hand, you stop

thinking about yourself, problems at home or work.

• Transformation of time – compression or expansion.

Doesn’t matter if time is a factor in the experience.

• The paradox of control Typically, the person experiencing flow feels like they have high

degree of control over their activity. It’s more the possibility of

control,

not the actuality of having it. People don’t enjoy the sense of control,

but of exercising control in difficult situations.

Flow – Frustration vs. Boredom

Skill

Challenge

Frustration

Boredom

Flow Channel

A

B D

C

A – The user starts in flow state

Flow – Frustration vs. Boredom

Skill

Challenge

Frustration

Boredom

Flow Channel

A

B D

C

B – The user is learning game controls, is not yet skilled, the game challenges him too quickly, and he gets frustrated. He must learn new skills to return to flow.

Flow – Frustration vs. Boredom

Skill

Challenge

Frustration

Boredom

Flow Channel

A

B D

C

C – The user has learned how to play the game, or starts out adept at it, low challenge leads to boredom. In order to return to flow, the challenge must be increased).

Flow – Frustration vs. Boredom

Skill

Challenge

Frustration

Boredom

Flow Channel

A

B D

C

D – The user returns to flow having improved skills and being able to function at a higher level of challenge.

Application

• Measure the player’s skill level to figure out roughly where in the flow zone the player might be.

• Increase challenge corresponding with improvements in the player’s skill.

• Also adjust challenge based on desired dramatic tension.

System Components

• Dynamic game elements What about the game can change from moment to moment that

affects the difficulty?

• Definition of difficulty progressionWhere are the hard spots and easy spots?

• Measurement of the player’s skill

• Information to tie dynamic elements together There may be an aesthetic component to selecting groups of

dynamic game elements.

• Sequencer that uses all these elementsSelects the elements necessary to achieve the desired difficulty with

respect to the player’s skill.

System Components

Difficulty Sequencer

Pool of availablegame elements

Difficulty Progression

Logic

Created game elements

Player actions

Player SkillMeasurementElement

GroupingInformation

Dynamic Difficulty Elements

For an action/combat game, these would be different types of enemies, their weapons, etc.

Need to evaluate the difficulty of an individual element.Designer defined difficulty rating – allows quick heuristic evaluation to

compensate for a variety of complex factors. Simpler to tune.

More complex mechanic may be neededIf an element is actually comprised of sub-elements (an enemy type with a

weapon, other power-ups), may need a rating for each sub-element.

Difficulty rating scaleNeeds to be defined to correspond with player skill measure, 1 (easiest)

to 10 (hardest) for example.

Defining Difficulty Progression – Two Approaches

Offline – Designers provide hints to the system where the easy and hard spots are.

Example – Location based difficultyDesigners create areas using trigger volumes and associate a difficulty

with an area, as well as a list of available enemy types, spawn points, etc.

Mechanic based - Model difficulty based on some element of the game’s experience

Example – Amount of remaining gameplayAs the player gets closer to completing all the goals in a level, the system

would modulate difficulty as necessary.

Player Skill

We need a measure of the player’s skill to determine the experienced difficulty.

Factors to consider:

• Time (valid for most action-related games).

• Difficulty of elements the player has completed.

• Number of elements the player is in the process of completing.

Player Skill

For a combat oriented game, these would translate to the following:

• Time to defeat a single enemy – indirect relationship

• Designer defined difficulty value for that enemy type – direct relationship

• Number of other enemies the player is currently engaging in combat – direct relationship

Player Skill

Basic Player skill measure =

Enemy difficulty / Time to defeat enemy

Would be nice to have same scale as enemy difficulty:So we add a constant that represents a basic unit of difficulty: the time it

would take a typical novice player (at skill level 1) to defeat a simple

enemy (at difficulty level 1), with the simplest available tools (weapon and

any other power-ups) the first time he plays the game.

Player skill measure =Enemy difficulty /(Time to defeat enemy * basic unit of difficulty)

Player Skill

Need to factor in other ongoing challenges the player is dealing with at the time.

Scale enemy difficulty by number of other opponentsAnother constant, based on usability testing or game design goals.

Example – Set scale per # of enemiesScale the player skill measure by a constant amount per number of

enemies: +1 enemy: x1.125, +2 enemies: x1.25, +3 or more enemies:

x1.5 - If the player has to defeat three enemies at once, defeating any 1 of

them is harder (by 1 ½ times) than if the player faced that enemy alone.

Final Player skill measure = (Enemy difficulty * scale factor for number ofsimultaneous enemies) /(Time to defeat enemy * basic unit of difficulty)

Player Skill – Other Factors to Consider

• Use a simple average (not weighted by recency) of skill measures over the last few minutes of gameplay.

• Prevents player from cheating (quickly changing performance to cause the system to lower difficulty).

• Difficulty modulation should be invisible to the player, otherwise it would removed them from the experience.

• Simpler is better - easier to tune.

• Time is often the key factor in skill measurement for a game with any sort of action component.

• Avoid additional factors (weapon accuracy, damage dealt) that are not directly associated with effectiveness.

Grouping Difficulty Elements

One last bit of information needed, potentially, before selecting elements to define the goal difficulty.

How do we select multiple difficulty elements in a pleasing fashion?

How do we connote a causal relationship between elements?

Usually would come in the form of additional designer input to the system, to control the aesthetics of grouping and content-specific relationships.

Difficulty Sequencer

Goal difficulty = Player’s skill * difficulty modifier

Difficulty modifier comes from the defined difficulty progression at that location/time (very hard, normal, easy, etc.)

Now randomly select from the difficulty element groupings

Select from the available list of elements to match the desired characteristics of the grouping, whose total difficulty sums to the goal difficulty.

Difficulty Sequencer - Example

So if we have 5 difficulty ranges, each range might have these modifiers:

Very Hard - 2.0

Hard - 1.5

Normal - 1.0

Easy - 0.75

Very Easy - 0.5

If the player’s skill is at 8, and we’re in a very hard section, then the goal difficulty is 16.

Note: discrete difficulty settings

Difficulty Sequencer - Example

Let’s say we have the following groupings of enemies:

• One very strong enemy

• Two enemies, one stronger than the other

• Two enemies of equal difficulty

• Three enemies, one stronger than the other two

• Three or more weak enemies

And the following enemies available in this area:

• Commando - Difficulty 10

• Sniper - Difficulty 5

• Infantry - Difficulty 4

• Guard dog - Difficulty 2

Difficulty Sequencer - Example

Randomly select the two enemies of equal strength group The

system can’t create two enemies of equal strength within a small enough

range of the goal - 20 (10+10) or 10 (5+5).

So then select another grouping, two enemies, one of greater strength.

It chooses an enemy of difficulty 10 and one of difficulty 5 Their

total difficulty is sufficiently close to the goal difficulty.

Now it creates the enemies according guidelines specified by the designers (at applicable spawn points, with available weapons/armor).

Difficulty Sequencer – More Factors

The game may require more complex difficulty model – enemies may pick up enhancements that greatly affect their strength (weapons, armor, etc.)

Each individual sub-element then has it’s own difficulty.The sequencer would just have to consider each valid combination of sub-

elements (enemy A with weapon 1, enemy A with weapon 2, etc.).

The difficulty would just be the sum of the individual difficulty elements.

Strengths

• Simpler tuning process – don’t have to tweak each enemy’s position, type, etc.

• Speed & ease of development – well suited for user mods (NeverWinter Nights)

• Can create stronger, more consistent, pacing even in non-linear environments

• Eliminates need for static difficulty levels by taking into account player’s skill throughout the game.

• Enhances replayability by avoiding heavily scripted pacing

Weaknesses

• Relies on semi-arbitrary, heuristic based input to define difficulty scales.

• Must keep mechanics simple, otherwise it can be hard to tune (since they only indirectly affect the player’s experience).

• Difficulty mechanics must be as invisible to the player as possible.

Further work

• Keep track of the rate at which player’s skill is changing – a measure of how much they’re learning.

• Could add requirements for skills dynamically - If the player’s skill set isn’t growing, add tasks that require new skills to succeed.

• As the dynamic elements become more complex, the selection/grouping information becomes more involved.

Feedback - bpfeifer@radical.ca

References

Laurel, Brenda, Computers As Theater, Addison Wesley, 1991

Csikszentmihalyi, Mihaly, Flow: The Psychology of Optimal Experience, Harper Collins, 1990

Mateas, Michael, and Stern, Andrew, “Façade: A One Act Interactive Drama,” Game Developer’s Conference 2003