Resource Management in complex environments: an application to

Real Time Strategy Games

Thiago Andrade(tas3@cin.ufpe.br)

Federal University of PernambucoInformatics Center

Recife - PE

Geber Ramalho(glr@cin.ufpe.br)

Sérgio Queiroz(srmq@cin.ufpe.br)

November 10, 2014

Resource management

“Managing resources is the process of using resources in the most efficient manner” (Business Dictionary)

• Resources are present in human activities– Tangible: Equipment, money, people– Intangibles: Time and Space

Real Time Strategy Games

• Military simulations where players use real-time resources to achieve goals. Examples: Starcraft, Age Of Empires, etc.

• Real Time• Dynamic • Uncertain• Partially observable• Strong and multiple interaction between variables

Real Time Strategy Games

• Resources: Gas, Gold, Stone, Wood, etc.

• Investment Items: Army units, Buildings, Technology, etc.

Resource problems

• Limited• Many choice options

– Train a soldier or develop a technology • Quantities to consider

– x soldiers or y horsemans• Difficult evaluation options (multicriteria)

– soldier type1 best against E1, soldier type2 best against E2. Which one should I train?

Management problems

• Dependency between items– Soldier depends on Castle

• Time– Deprecated decisions after some period of time

• Context– Peace, War, etc.

• The problems mingle!• Solution muse achieve: correctness,

adaptability, completeness, performance

State of art

• Most of AI games are scripted– Most games don’t have the ability to deal with unfamiliar

situations (Kovarsky & Buro, 2006)

• Build-order– The order on which units are created in a RTS game

(Churchill & Buro, 2011)– Static: In some cases discards the context of the game

• Build-order strategy (MCCoy & Mateas, 2008)– Complete AI: Tactical, Resource, Strategy, Income,

Production

PICFlex

• PICFlex: Resource Management based on Investment Policy Flexible and Contextual

• An investment policy P is described as:

• Behavior– One specific instance of an investment policy– Initial, Defensive and Aggressive

• Ex. P = {[army,40%], [building, 30%], [upgrade, 20%], ...}

PICFlex

• Choice of investment policy– Human player perceives nuances of the game and adopt different

strategies. Therefore it is not recommended to have a single policy

• Investment policy choice function (F)

where P is the current investment policy, C is the game context and P’ is the new policy.

• Fs simple – Simple choice function• Fa adequate – Suitable choice function

PICFlex

• Execution Strategy (S)– In the long term, the adoption of the policy will provide a

"balance" in spending by target goals– The goal can work as ceiling preventing investments that

exceed it

• Sg generous – Generous strategy (cancels the policy)• Sr rigid – Rigid strategy• So observed growth – Observed growth strategy

Players

• Player 1 - Random investiments• Do not have an Investment Policy

• Player 2 - Random fixed Investiment Policy• Investment policy of random values changing during the game• St generous – Generous Strategy (always returns true for investments)

• Player 3 – Fixed IP– Fixed Investment Policy defined by RTS experts– Sr rigid Rigid execution strategy

• Player 4 – Fixed IP2– Balanced growth pattern defined by RTS experts– Army investment policy defined by RTS experts

Players

• Player 5 – Adapted IP Switching Investment policy– Policy adapted to the context– Fa adequate Choose more context adherent behavior– Sr rigid Rigid execution strategy

• Player 6 - Flexible Adapted IP – Flexible Spending Policy– Fa adequate Choose more context adherent behavior– Sg observed growth Stragetey execution of observed growth

investment policy. – The goals are flexible.

Experiments

• Objective: evaluate techniques

• Setup: against native AI Starcraft Broodwar

• Metric: game score

• 150 maches

• Two baseline agents– Player 1: Random Investments– Player 2: Random Investment Policy

Comparisons

• Comparative implementations of PICFlex

Player Victories Success Avg. Score

1 - Rnd Inv 23 15,33% -1851,92

2 - Rnd IP 4 2,67% -1863,62

3 - Fixed IP 31 20,67% 1839,42

4 - Fixed IP2 38 25,33% 2389,47

5 - Adap IP 40 26,67% 7062,02

6 - FlexAdap IP 63 42,00% 10801,01

Results – FlexAdap Inv. Policy

Alternative Hypothesis Null Hypothesis a p-value

Ha11: P6 > P5 Hn11: P6 = P5 0,05 0,01637

Ha12: P6 > P4 Hn12: P6 = P4 0,05 7,888e-05

Ha13: P6 > P3 Hn13: P6 = P3 0,05 3,737e-07

Ha14: P6 > P2 Hn14: P6 = P2 0,05 1,415e-12

Ha15: P6 > P1 Hn15: P6 = P1 0,05 1,152e-11

Analysis

• Investment policy showed its value when it was defined by experts in the Fixed Investment Policy increasing from 2.67% to 20.67% success rate

• Balanced Growth is used since Fixed IP2, but its success rate was only 25.33%

• Fa adequate was used on Adap IP, but its success rate was only 26,67%

• Sg observed growth: responsible for the increase in the success rate

• Sr rigid: the rigid goals prevented important investments

Contributions

• Main contributions

– New approach of Resource Management based on the concept of Investment Policy, validated in RTS

– Study of Resource Management problems

Future Work

• Compare PICFlex with a solution that use pure build-order

• Create new implementations of functions F and S that improve game score

• Implement PICFlex with other bot and compare to this work’s implementation

• Improve PICFlex performance: 40% of CPU time is used with AI

• Implement PICFlex on complex environments other than Real Time Strategy Games