2013 JUNE 24 - AI DESIGN BREAKDOWN

http://forums.conifergames.com/index.php?topic=158.msg739#msg739

The AI is one of every strategy game's most important features. The bread and butter of the genre is offering

difficult decisions in a replayable environment where success requires constant adaptation. Good AI

enhances all three of these elements - whereas a bad one can derail the entire experience.

In the context of game AI, "good" means more than just "smart." The job of computer opponents is more

than just standing in for humans - they're a part of the world and a gameplay system just like economics,

diplomacy and warfare. Attila the Hun isn't just there to provide a roadblock to victory - he's there to be

Attila the Hun, the frightening warlord who races in with a massive horde of horse archers, burns everything

in sight, and runs off before you can muster a response. Is such a strategy productive? Hopefully! But it's in

many ways that's only a secondary concern.

That having been said, a good AI must also put up a competent fight. This is especially true with features like

combat, when the AI's decisions are front-and-center and mistakes cannot be tucked under the rug. Such

competency requires AI players to craft plans and be capable of course correcting as necessary.

For all of this to come together, you too as AI developer must have a plan. To that end, I've spent much of the

past couple months putting together an insanely detailed breakdown for every AI subsystem and the logic

each will perform. Think of it as a "first draft," where the second will be the translation to programming code.

I've run through repeated thought experiments to test my design and I've scrapped more than one critical

feature when it became clear that intention and results weren't in harmony.

I won't be posting that work today (or ever... it's already well over 200 pages and still growing!). But I know

many of you would like to see how I envision everything fitting together, so what I will be doing is sharing my

basic outline for the components that will make up an AI player. As with all of game development I have no

doubt the details will change a thousand times over, but I'm fairly confident this general structure will be

maintained until the game is completed, and very possibly for several future titles.

- Jon

AI Outline

Info Library (IL) - Not a specific AI system, but a general name for the vast collection of processed data for an

AI player... is that guy stronger than me? How dangerous it that tile? etc.

Grand Strategy AI (GSAI) - Sets the general, game-wide direction taken by the player.

Objectives AI (OAI) - Determines what the AI's priorities are, what opportunities are available, which are

worth pursuing, etc. The Objectives chosen are what shapes the decisions made by all of the following AI

subsystems.

Economic AI (EA) - Allocates Resources, manages production choices and handles misc low-level economic

decisions like choosing Romanization Perks.

Diplomatic AI (DAI) - Manages day-to-day diplomatic interactions, including when to offer trades, what offers

we'll accept and what specific wording is used in all exchanges.

Front AI (FAI) - Allocates Units between different opportunities, and puts in requests with EAI when more are

needed.

Tactical AI (TAI) - Gives specific orders to every Unit, every turn.

Priority - A scale with 20 values that serves as a general measurement for how important something is. Each

label can have a "+" or "-" after it to make its value slightly more or less. So a "Very Good-" Priority would be

one level above a "Good+" Priority, which is itself one level above "Good".

Zone - A small group of tiles, will probably end up being roughly 3x3.

Grand Strategy AI (GSAI)

Summary The GSAI chooses a single GS, which shapes the general game-wide direction an AI player pursues. It dictates

the what, but not the how - it might provide us with the goal of conquest, but it won’t tell us if we’re going to

do it by sea, with mounted Units, etc. Execution details such as these are decided at the OAI level, which also

takes personality traits and the current situation into account.

Details In the early game, the GSAI picks a “precursor” GS, which hints the player in a direction without committing

them to anything specific. Maybe we’re playing a militaristic game, but we don’t know what form that will

take. Or we’re committed to being peaceful with the Romans, but we’re unsure how closely we should tie

ourselves to them since we don’t yet know our proximity.

In the XML, each AI leader is assigned a fixed set of GSs it’s able to choose from. This ensures that every

leader plays consistent with their personality, but doesn’t tie them down to a single strategy like “always

attack everyone you see.”

Factors to Consider The AI’s intrinsic personality is by far the most important. Attila is never going to sit back and play

nice with everyone, no matter what the situation is.

Proximity to the Romans. If we’re really close we probably want to make nice. If we’re too far away

then focusing heavily on them doesn’t really make sense.

Proximity to other barbarians. If we’re completely surrounded by enemies then we probably want to

lie low and play it safe until things shake out.

Availability of resources. Should we completely lack Metal, then it doesn’t make much sense to play

aggressively.

Geography. If we’re close to the water, this opens up the possibility of naval raiding. If we’re tucked

into a defensible corner, then we can afford to be a little less cautious.

Military balance of power. If we fight a major war, lose our entire army and two-thirds of our

population, then it’s probably time to change course.

Relationship with Other Systems The GSAI doesn’t receive input from any other subsystem.

OAI: The sole job of the GSAI is to alter the Priority scoring of Objectives being considered by the OAI. It

makes decisions unilaterally (using data from the IL) and receives no input from anyone.

Objectives AI (OAI)

Summary This is the meat of the player-level AI which sets all short and medium-term goals. Decisions are made here

for everything from who to declare war on, to which resource deposits to try and grab, to when and where to

migrate.

The reason why so much is done in one place is because the AI needs to have a very clear idea of how to

score competing priorities. This can either be done with multiple subsystems that all “speak the same

language,” or handled in a single location. We’re taking the latter approach.

However, what the OAI does not do is manage any details. If we want Metal, then the OAI figures out which

Deposit is worth going after and assigns it a priority, then hands the task off to the FAI to get it done. If there

aren’t enough Units available the FAI is what figures this out, and then makes a Request to the EAI.

Details The OAI utilizes two different types of Objectives, lumped into a single prioritized list:

Goals - Abstract “things we want to do,” like undermining the Goths, getting more Metal, or migrating.

Opportunities - Concrete “things we can do,” like invading the Goths, harvesting the Iron Deposit on Tile X, or

migrating to Region Y.

Objective Creation & Execution First, every single possible Objective is analyzed for its viability, and if it’s legal, prioritized. Ongoing

Objectives are processed first and given a small bias (so that the AI doesn’t change its mind all the time),

followed by new Objectives. The combined list is then sorted.

Next, higher priority Objectives alter the Priority of those Objectives below them in the list - if we’ve decided

we’re going to invade the Romans, it doesn’t make sense to also try to complete a Request for them!

Any Objective with at least Medium- Priority is considered Active, which means it will be acted upon by other

AI systems. Everything below that threshold is completely ignored. Since every possibility is considered and

scored, this is our way of “focusing the AI’s attention.”

Ultimately, the OAI has authority over all other systems. If it says to declare war, that’s what we’re going to

be doing - consequences be damned.

Relationship with Other Systems GSAI: Our current GS alters the Priorities of individual Objectives within the OAI. The “Lie Low” GS

will greatly reduce the Priority assigned to all “Invade Player X” Objectives.

EAI: When the OAI wants a nonmilitary thing built, researched, chosen, etc., it puts in a Request with

the EAI. Construction Requests from the OAI are extremely rare, as all Requests for anything relating

to Units come from the FAI (which, in turn, derives its goals from the OAI - delegation at its finest!).

DAI: The OAI’s Objectives shape the general approach taken by the DAI (“make trades when

possible”) and even occasionally makes specific directives (“try to trade for Metal”). The OAI is

completely in charge of declaring war and making peace. When negotiating peace treaties, the OAI

tells the DAI the general parameters of what is acceptable. The best way to think of it is that the OAI

makes the important decisions, while the DAI handles day-to-day affairs.

FAI: The OAI provides the FAI with 2 things. First, general military goals, such as “capture Gothic

Cities” or “wreak mayhem on the Huns” or “ignore everything else, protect the homeland!” The OAI

also asks for specific missions, such as “capture that Hostile Stronghold” or “claim that Iron deposit.”

The OAI also gets its info from the IL side of the FAI as to whether we should be fighting, and if so,

how.

Economic AI (EAI)

Summary The EAI balances day-to-day management of the economy with fulfilling “Requests” that come from the OAI

and FAI. It’s main job is to manage the allocation of Resources, but it also handles other economic decisions

such as choosing Romanization Perks.

Nearly all actions taken by the EAI will be with the goal of fulfilling Requests, but there will be times when

there are no pressing matters to attend to and the EAI must maintain a healthy holding pattern.

The EAI also makes Requests of the OAI when there are circumstances that prevent it from completing

Requests its received. For example, if the FAI asks the EAI to build Infantry but we have no Metal to do it

with, it’s up to the OAI to rectify that (or not, if there’s more important business to attend to).

Details The main task of the EAI is balancing competing Requests and the long-term health of the kingdom. If we

have a Request to train Units which require Population, we’re unlikely to build them from tiny Cities that

can’t really afford it, unless that really is in fact the best way to complete a very high Priority Request.

Most Requests have a target location, and the EAI factors that in when deciding where to build what. For

example, if we’re planning on launching an invasion it’s better to have Units built nearby than on an island on

the other side of the map.

If the score for a City/Request pairing is high enough, that Request is pushed through. If not, the Request

goes away, and must be re-added the following turn by the subsystem which made it. This ensures other

systems are constantly re-evaluating what they do and don’t need, and allows the EAI to be flexible and

adapt to new circumstances immediately.

The EAI evaluates multiple combinations of actions to see which results in the highest score (just like basically

every action taken by the TAI). This prevents situations where the AI is too “greedy” with accomplishing

Requests. We’d rather the EAI accommodate two “High” Priority Requests than one that’s “Very High” and

another that’s “Insignificant.”

The EAI performs exactly the same for all players in all situations. Personality biases are expressed in the

Objectives chosen by the OAI.

Factors to Consider OAI economic and FAI military Requests

Is a City a particularly good or bad fit?

Proximity to target location

o Population before and after

o Proximity to enemies

o Other (special buildings)

Current and projected Resource accumulation

Current and projected maintenance (e.g. no Food during the winter)

Relationship with Other Systems OAI: The EAI receives Requests from the OAI, and can also make Requests of it. “You want us to train

ships, but we don’t have any Wood!”

DAI: The EAI tells the DAI how much we can (or cannot) afford to give up certain Resources for trade

or peace treaties. However, the decision for whether or not to make the move is ultimately up to the

DAI, as making peace might be a matter of life or death.

FAI: The EAI receives construction Requests from the FAI.

Diplomatic AI (DAI)

Summary The main job of the DAI is to execute on diplomatic directives provided by the OAI (which is where the

important decisions like declaring war and suing for peace are made). The DAI manages day-to-day

interactions of popping up when appropriate, determines the exact language in which a message is to be

delivered, etc.

The DAI also looks for opportunities to complete Objectives within the parameters outlined by the OAI. There

might be an Objective that really wants us to get Iron, and the Diplomatic AI looks for opportunities to trade

for it. If there’s another Objective that tells us to hate on the Goths, we won’t try to trade for their Iron even

it would help us.

Details The DAI tests the list of actions it can perform to see if they make sense at that time, based on what

Objectives have diplomatic ramifications and are in a state where action is permitted. For example, if we

have an Objective to declare war and attack the Goths and the Objective says it’s ready to go, the DAI will fire

away.

A particularly diplomatic AI leader will create diplomacy-specific Objectives, which the DAI then sees and

reacts to.

The DAI will only attempt to make a trade if the active Objectives allow for it. e.g. if both “Get More Metal”

and “Don’t Trade with Attila” are active, we’ll try to trade for Metal, but not with our Hunnic friend.

If another leader offers us a proposal, the DAI determines if it’s worth agreeing to. It analyzes each piece of

the deal to see how valuable it is to us in our specific situation. For example, the value of a Resource or the

cost of declaring war on someone will be tested against our Objectives.

As should be fairly clear, the goal is to look at Objectives as much as possible when deciding on what

diplomatic activities to engage in.

Factors to Consider Current Objectives.

Relations with other leaders.

MAYBE stuff in the IL, although my preference is to use the Objectives whenever possible, as this is

the part of the AI that decides what it does and does not want to do.

Relationship with Other Systems OAI: The DAI receives general diplomatic directives from the OAI via Objectives. e.g. Make nice with

the Western Romans, don’t help the Huns, etc.

EAI: The DAI can ask the EAI it how much of a Resource it’s okay to trade. However, the DAI has the

final authority to make the call.

Front AI (FAI)

Summary The FAI divvies Units up and ensures they’re utilized in the best way possible.

The most important job of the FAI is enacting the whims of the OAI. It might want us to take down a Hostile

Stronghold, or muster an invasion, or improve an Iron Deposit. It’s up to the FAI to figure out what is required

and put the pieces in motion. From there, the TAI then takes over and directs specific moves.

As part of this, the FAI organizes Units into “Fronts,” which are groups with a shared goal. An invasion force

should muster at tile X and launch from tile Y. The homeland is divided into Zones A, B and C, and each needs

a unique level of defense based on the threat posed to and contents of each. We need at least Z in order to

safely escort a Laborer to the Horses over yonder.

Details The FAI creates one Front for each:

Invasion or Mission requested by the OAI.

Zone that the player has an interest in defending.

A Front ends either when the OAI kills off the associated Objective, or when the FAI decides to abandon a

Zone (on defense). Defensive Fronts are recreated every turn, since they don’t keep track of any persistent

data.

Fronts that are not Defensive are classified as either a Mission, if they have a very discrete goal (such as

capturing a Hostile Stronghold), or an Invasion, if its objectives are less defined (go beat up on the Huns).

Collectively, these two are referred to as “I/Ms,” as they tend to be similar to one another and somewhat

unlike Defensive Fronts.

All Fronts are assigned a Priority, which determines how Units are divvied up. For I/Ms, Priority is identical to

the associated Objective. The Priority of a Defensive Front is based on the general defense-related Objectives

in the OAI (“Defend the homeland at all costs!”) and the Value of the Zone in question (based on Assets

contained, strategic usefulness, etc.).

Execution The FAI assigns each I/M a Muster Tile, which is where Units will be collected in preparation for launch, and a

Target Tile, which is what we’re going once the Front is finished mustering.

The OAI can assign a Front Orders which weight the score of actions taken by its Units. If a Front is essentially

just a bucket of Units that have a shared goal, the Orders shape how that goal is to be achieved. Orders drive

low-level behavior, such as recklessness, caution, and preventing specific Improvements from being pillaged.

A Front can be set to Withdraw if the OAI decides to cancel the Objective or abandon a Zone. A Withdrawal

Target Zone is chosen, and Units belonging to that Front head towards it. Units can be pulled off of a

withdrawing Front by another Front if the score is high enough. Since Defensive Fronts are recreated each

turn, when one is set to Withdraw it actually creates a new I/M whose sole purpose is to get to the

Withdrawal Target.

Unit Assignment When deciding how many Units of what type a Front should be given, the FAI estimates the strength and

makeup of enemy forces likely to be encountered. A list of minimum Values deemed necessary in each

Domain (anti-Infantry, anti-Cavalry, anti-Naval, etc.) is generated.

To determine what Units go where, each Unit/Front pairing is scored for fit, factoring in what the

Front needs, the Front’s Priority, distance, general availability or scarcity of Units, etc. Once a big list

has been compiled we link the Unit/Front with the highest score, and then work our way down until

all Units have been assigned.

Each time a Unit is added to a Front, the scores for all remaining Units with that Front are

recalculated. Score drops by a small amount for each Unit added so that Fronts with a marginally

lower Priority get Units - otherwise the big boys would never share.

Once a Front is close to having its minimum requirements satisfied, the penalty for adding Units is

reduced and then eliminated.

If there’s not much to go around and a Front has high requirements (relative other Fronts in the

area) its Priority is reduced. Generally speaking, we prefer fewer fully-staffed Fronts to several

undermanned Fronts.

Based on how well or poorly each Front’s requirements are satisfied, the FAI then delivers Requests to the

EAI to train new Units.

Special Unit Roles Some Defensive Fronts can have part of their requirements satisfied by Reserve “Units.” These don’t actually

exist, but the EAI has set Resources aside and can pop them out at any time.

I/Ms have dedicated Units for Rearguard duty. The FAI evaluates the Assets that must be guarded, the

threats posed to the Front’s flank, etc. and determines how many Units it needs and where they should be

stationed. I/Ms also set aside Units for Recon, which fan out to keep an eye on things, and rejoin the main

force in case something important comes up.

Once the initial FAI design is functioning correctly, logic will then be added allowing Units to be loaned away

by I/Ms that are still mustering to other nearby Fronts. If we’re building up a massive invasion force, this

allows us to temporarily siphon off Units to take out Hostile Strongholds, escort civilians, etc.

Factors to Consider Objectives outlined by the OAI.

Value of each Zone.

Threat posed to each Zone.

Current and future seasonal effects.

Availability of Units of different Domains in a general area.

Relationship with Other Systems OAI: The FAI receives directives from the OAI via Objectives, which also determine what Orders are

given to specific Fronts. The IL side of the FAI provides info to the OAI regarding our military

situation.

EAI: The FAI places Requests in to the EAI when it needs Units for various tasks.

TAI: The FAI informs the TAI the general approach to be taken in each Zone so that it knows how to

score different actions.

Tactical AI (TAI)

Summary This is where individual Units are all given orders.

The TAI runs through (almost) every combination of possible moves and attacks, scores the results, and

chooses the best outcome.

Details A brute force approach is expensive, but necessary. It’s impossible for us to “teach” the AI how to execute a

flawless tactical plan, as there are far too many factors that need to be considered.

In order to cut down on the exponential processing requirements, a series of checks will be made to limit the

number of actions and combos considered. e.g. If there is no danger nearby just take the fastest path to our

destination. Another way we’re speeding up execution is by placing Units into “Dependency Groups,” which

separate out those which are and are not affected by one other. A Scout exploring an island has no bearing

on a siege against Rome, so one need not account for the moves of the other.

Each combination of actions is assigned two scores: Safe, and Risky. A Front’s Orders (generally shaped by

Objectives and leader personality) determine the modifier applied to these scores. A particularly conservative

leader might need to see a significantly higher Risky score in order to go that route.

The score of an individual action is determined by a large set of rules which account for possible battle

results, unseen danger, distance to destination, defensive bonuses, supply, etc. Scoring calculations are

calibrating using our determination as to the Value of the Asset (Unit, Improvement, City, etc.). If moving to a

tile is guaranteed to result in a Unit’s death, the negative score for that move is equal to the Unit’s Value.

Every time an attack takes place or a new enemy Unit is revealed, the Dependency Group’s action combo is

re-scored. This ensures the AI is always using the most up-to-date information and won’t be ambushed.

Factors to Consider The position, number and strength of owned, friendly and enemy Units.

Map conditions:

Supply available

Movement cost

Defense bonus

Unseen Danger present on each tile:

General likelihood that there are hostile Units nearby, based on proximity to enemy staging points,

the strength of those enemies, etc.

Memory of where enemy Units used to be.

What opponents can or might be able to see.

Relationship with Other Systems FAI: The TAI speaks with only the FAI, from which it receives the priorities it should be considering

when making specific moves.

Compilers Footnotes

This article was originally posted by Jon Shafer on the Conifer Games forums on 24th June 2013. The article

was then posted on reddit (/r/gamedev) by mm04. Some other guy then copy and pasted it into an

undisclosed Microsoft product, formatted, and then exported to PDF on 17th February 17 2015.