Contents lists available at SciVerse ScienceDirect
Applied Soft Computing
j ourna l ho mepage: www.elsev ier .com/ locate /asoc
n evolutionary algorithm based on Nash Dominance for Equilibrium Problemsith Equilibrium Constraints
ndrew Kohnstitute for Transport Studies, 36-40 University Road, University of Leeds, Leeds LS2 9JT, United Kingdom
r t i c l e i n f o
rticle history:eceived 29 July 2010eceived in revised form 20 June 2011ccepted 31 August 2011vailable online 9 September 2011
a b s t r a c t
This paper introduces an evolutionary algorithm for the solution of a class of hierarchical(“leader–follower”) games known as Equilibrium Problems with Equilibrium Constraints (EPECs). In onemanifestation of such games, players at the upper level who assume the role of leaders, are assumed toact non cooperatively to maximize individual payoffs. At the same time, each leader’s payoffs are con-strained not only by their competitor’s actions but also by the behaviour of the followers at the lower
eywords:ash Equilibriumquilibrium Problems with Equilibriumonstraintsransportation systems managementlectricity markets
level which manifests in the form of an equilibrium constraint. By a redefinition of the selection criteriaused in evolutionary methods, the paper demonstrates that the solution for such games can be foundvia a simple modification to a standard evolutionary multiobjective algorithm. We give a proposed algo-rithm (NDEMO) and illustrate it with numerical examples drawn from both the transportation systemsmanagement literature and the electricity generation industry underlying the applicability of NDEMO inmultidisciplinary contexts.
. Introduction
A major trend in the provision of transportation services andacilities has been deregulation coupled with the private sectorlaying a larger role. Whether it occurs in highway [60] or transit62], entities providing such services face competition from othersith similar offerings. It is of interest to regulators to understandow such organizations make decisions on their service levels inhis deregulated environment.
In this environment, the service levels provided are an outcomef a non-cooperative Nash game [39] amongst the players. Howevern transportation, this game possesses a feature that distinguishest from the classic Nash game: The players’ actions are constrainedy a condition defining equilibrium in the transportation system12]. Users of the transportation network make their route choiceecisions by choosing routes that are the lowest cost accordingo Wardrop’s Equilibrium Principle [58] and their route choice isarameterized in the decision variables of these firms. Thereforehis is a hierarchical (i.e. leader–follower) game with the firms aseaders at the upper level engaged in a Nash game and travelerss followers at the lower level routing according to an equilib-
ium condition. Thus the terms “firms”, “leaders” and “players” areynonymous in this context.
The game just described is an instance of a broader class of Equi-librium Problems with Equilibrium Constraints (or EPECs) [35,36].EPECs have emerged as an area of research [2,10,57] in mathemat-ics applicable to transportation systems management and otherdisciplines [22,36]. This paper focuses on the determination of equi-librium values of the strategic variables for each profit maximizingleader when in competition with others.
This paper is an extension of the earlier work by the presentauthor [29] but has been enhanced in two key areas. Firstly onthe theoretical aspect, we strengthen the theoretical justificationof the proposed algorithm. Secondly, on the practical aspect, wedemonstrate the applicability of our algorithm to the examples ofEPECs that arise not only within transportation systems manage-ment but also those arising in the electricity generation industry todemonstrate that our proposed algorithm is indeed applicable inmultidisciplinary contexts.
The rest of this paper is organized as follows. In the nextsection we outline the literature of the leader follower gameparadigm that forms the basis of this research. Section 3 subse-quently focuses on the notions associated with the non-cooperativeNash game underlying the behaviour of the leaders in the EPEC.Section 4 reviews both the deterministic (i.e. gradient based) andevolutionary approaches for computing NE. Section 5 elucidatesthe Nash Domination criteria developed in [33] and provides an
algorithm. Section 6 presents numerical examples of the solu-tion of EPECs utilizing the concept of Nash Domination. Section7 concludes the paper with a summary and directions for furtherresearch.
Fig. 1 gives a pictorial representation of what has come to benown as the Stackelberg game [56]. It is a model of the mar-et structure whereby a single leader is able to gain increasedrofits by anticipating the reactions of the rest of the market partic-
pants (known as the “followers”). In the field of mathematics, thetackelberg game is referred to as a Mathematical Program withquilibrium Constraints (MPEC) and has been investigated in detaily a number of researchers (see [31,41]). The characteristic uni-ying feature of MPECs is that in addition to general constraints,here exists a constraint specifying equilibrium in some paramet-ic system. The key point to note is that the followers are assumedo take the single leader’s decision variables as exogenous whenptimizing their individual objectives [32].
This equilibrium constraint is also present in the case of a mul-iple leader follower game shown in Fig. 2. Though both modelsossess in common the hierarchical feature, the key differenceetween Figs. 1 and 2 is that multiple leaders are present in the
atter and these leaders are assumed to play a game amongst them-elves. We thus seek the equilibrium points of the game playedy these upper level leaders. Hence as an extension of the MPEC,ig. 2 illustrates the more general class of Equilibrium Problemsith Equilibrium Constraints (EPECs).
In this multi-leader generalization of the Stackelberg gamerticulated e.g. in [35], researchers have conjectured that thereould be two possible potential behaviours of the leaders at thepper level [42]. At one extreme, leaders could act cooperativelynd this results in a multiobjective problem subject to an equi-ibrium constraint at the lower level (MOEPEC) [61]. At the otherxtreme, the leaders can act non-cooperatively and play a Nash
ame amongst themselves resulting in a Non Cooperative EPECNCEPEC). In one of the numerical examples, we will revisit theistinction between the MOEPEC and NCEPEC. For the main part
Fig. 2. Multiple leader follower game (EPEC).
ng 12 (2012) 161–173
of this paper though we concentrate exclusively on the situationin which leaders act non-cooperatively with the objective of max-imizing personal gain.
Casting our present work within the broader research context,the existence of the binding equilibrium condition distinguishesthe games we describe herein from standard Nash Games. In par-ticular [12] have pointed out that the NCEPEC is a special caseof a Generalized Nash Equilibrium Problem as described in (e.g.[19,24,52]).
3. Nash Equilibrium
Much of the game theory literature deals with games that areeither zero sum where victory or gains for one player is exactlybalanced by the defeat or losses for the other (as in games suchas checkers [3]) or where the actions of players are constrained tobe in a discrete set (such as the binary options of confess/do notconfess in games like the Prisoner’s Dilemma [49]). However thesolution algorithms proposed for these are generally not applicableto NCEPECs. In such games, the payoffs to the players are continuousand the strategic decision variables are subsets of the real line (asdescribed in Chapter 6 of [59]).
Consider the leaders’ problem in the NCEPEC. This is a single shotnormal form game with a set N of players indexed by i ∈ {1, 2, . . .,n} and each player can play a strategy si ∈ Si which all players areassumed to announce simultaneously. S =
∏ni=1Si is the collective
action space for all players. It is convenient to denote s−i as thecombined strategies of all players in the game excluding that ofplayer i i.e. s−i≡ (s1, . . ., s(i−1), s(i+1), . . ., sn). So with a slight abuse ofnotation, we have that s ≡ (si, s−i). We emphasize that the notation(si, s−i) does not imply that the components of s are reordered suchthat si becomes the first block. We refer to s as a strategy profile ofall players in the game. Let Ui(s) be the payoff to player i, i ∈ N if s isplayed.
Definition 1 ([39]). A combined strategy profile s∗ =(s∗1, s∗2, . . . , s∗n) ∈ S is a Nash Equilibrium (NE) for the gameif:
Definition 1 emphasizes the fact that at a NE no player can ben-efit (increase individual payoffs) by unilaterally deviating from itscurrent strategy. Hence each player is doing the best taking intoaccount what the competitors are doing [16]. The NE problem isthe determination of strategies that satisfy Eq. (1).
4. Computation of Nash Equilibrium
4.1. Deterministic approaches
In a game, the optimal strategy for a player is governed by thebest response function. If Ui(s) is continuously differentiable, thenthe best response function for player i is given by dUi(si, s−i)/dsi = 0[16,59]. The NE is the intersections of these best response functionsfor all players which amounts to finding solutions to n simultaneousequations i.e. solving dUi(si, s−i)/dsi = 0, ∀ i ∈ {1, 2, . . ., n} [7,59].
While useful for providing insights into the behaviour of play-ers, the analytical method is not feasible for realistic problems andeven less so for NCEPECs due to the binding equilibrium condi-tion. Thus the practical approach for finding NE is by using variantsof fixed point iteration (e.g. non-linear Gauss–Siedel) [25,57] orby formulating it as a Complementarity Problem [26]. Applica-
tions of these methods are found in (e.g. [18,30]). Convergenceof these algorithms rely on the payoff functions being continu-ously differentiable and possessing diagonally dominant Jacobians[16, Theorem 4.1, p. 280]. However, if the payoff functions of the
mputi
plDmprtcd
4
nrtt
emsg(mtbaTtnpoobe
em(tsee[etdotetd
5
5
awucuoc
A. Koh / Applied Soft Co
layers are not concave, there may exist NE that satisfy Eq. (1)ocally but not globally. This is known as a “local NE trap” [54,efinition 3, p.306]. There is thus a parallel with the literature onulti-modal function optimization where the potential for multi-
le optima cannot be ignored. Thus apart from their differentiabilityequirements, another drawback of deterministic approaches ishat they can fall prey to the local NE trap, an occurrence cru-ially dependent on the starting point used in these algorithms. Foretails of these and other deterministic methods, see [13,14,38].
.2. Evolutionary methods
Due to the proven ability of evolutionary algorithms to deal withon-smooth and non-differentiable functions as evidenced by theireported success in escaping local optima and potentially local NEraps, evolutionary counterparts of deterministic fixed point itera-ion methods were proposed in [48,50,53].
In particular, the motivation of the work reported in [53] was tomploy the NE paradigm as an alternative to multiobjective opti-ization. In this work the authors provided an example which
uggested that the NE point is on the Pareto Frontier which wasenerated by a standard evolutionary multiobjective optimizationEMO) algorithm. It was stated in [53] that the EMO required much
ore computing resources to generate the Pareto Frontier andhe Nash Genetic Algorithm that these authors proposed woulde robust for finding at least one solution and is hence usefuls an alternative. However there is a need to exercise caution.hough there exists games where the NE is also Pareto Optimal,his is generally not the case. Since the NE fundamentally assumeson-cooperative behaviour between players with each maximizingersonal rather than collective interests, it is clearly possible thatne player can be made better off without making another worseff and thus in this case the NE is not Pareto Optimal. This fact haseen demonstrated in [21] and will also be shown in a numericalxample to be presented later in Section 6.
A parallel research strand has been the exploitation of co-volution since it was first demonstrated [44] for tacklingulti-dimensional function optimization. Several sub populations
one representing each problem dimension) are evolved simul-aneously to avoid premature convergence and to widen theearch of the problem space. Ideas from co-evolution have beenxported into algorithms designed for the detection of NE; hereach sub population encodes the strategies of individual players6,43,47]. However doubts have been cast on the performance of co-volutionary methods. In [54], the co-evolutionary algorithm hado be hybridized with local search techniques to enable successfuletection of NE. [27] developed a co-evolutionary particle swarmptimization method which attempted to detect the NE by learninghe best response functions of the players. Instead of using the co-volutionary paradigm of previous works, a novel idea exploitinghe concept of Nash Dominance was proposed [33] to find NE asiscussed in Section 5.
. Nash Domination
.1. Theoretical foundations
At their most abstract level, evolutionary multi-objective (EMO)lgorithms [4,11] apply stochastic operators to a parent populationith the aim of evolving a fitter child population to solve vector val-ed optimization problems. Subsequently, in the selection phase, a
omparison is made between a chromosome x from the parent pop-lation and a chromosome y from the child population on the basisf fitness and the weaker of the two is discarded. This is entirelyonsistent with the principle of survival of the fittest. Given that
ng 12 (2012) 161–173 163
one of the objectives of EMO is to identify the entire Pareto front[11], fitness is assigned based on Pareto Domination (PD): x ParetoDominates y if x is strictly no worse off than y in all objectives andx is better than y in at least one objective [11, Definition 2.5, p. 28].
[33] define a concept analogous to PD called Nash Dominationfor the NE problem. A chromosome in this context represents thestrategies of all N players concatenated into a row vector i.e. a strat-egy profile. Then instead of using PD to compare two chromosomesi.e. two strategy profiles, Nash Domination operates by counting thenumber of players that can benefit if each player switches strate-gies in turn. The fewer the number of players that can profit fromunilaterally deviating from one profile compared to the other, thecloser the former is to a NE following Definition 1.
Consider two strategy profiles {x, y} ∈ S, (x ≡ (x1, . . ., xn), y ≡ (y1,. . ., yn)) and introduce an operator k : S × S → N associating the car-dinality of a set defined by 2:
The set thus defined by (2) comprises the players that wouldpotentially benefit by playing yi when everyone else plays x−i. Thetotal number of players in this set is given by k(x, y). A similar inter-pretation applies, mutatis mutandis, for k(y, x). The procedure issummarized in Algorithm 1. Note that in order to evaluate k(x, y)and k(y, x), the payoff to each player, individually, from deviatinghas to be computed. Following this procedure outlined in Algorithm1, one of the following outcomes must be true [33, Remark 4, p.365]:
1. k(x, y) < k(y, x) → x Nash Dominates y or2. k(y, x) < k(x, y) → y Nash Dominates x or3. k(x, y) = k(y, x) → x and y are Nash Non Dominated (NND) with
respect to each other.
Algorithm 1 (Nash Domination comparison).Initialize k(x, y) = 0, k(y, x) = 0for i = 1 to n do
Lemma 1. All Nash Non Dominated (NND) chromosomes are NE.
Proof. See [33, Proposition 9, p. 366]. �
The theoretical basis of the Nash Domination Comparison proce-dure proposed in [33] and outlined in Algorithm 1 is in fact foundedon the Nikaido Isoda (NI) function. This function as given in Eq. (3) isa mathematical tool that plays a key role in the study of NE problems[5,12,20,24]. Consider again two strategy profiles {x, y} ∈ S, thenthe interpretation of � (x, y) is as follows: each summand showsthe increase in payoff a player will receive by unilaterally deviatingand playing a strategy y while other players play according to x.
� (x, y) =n∑
i=1
[Ui(yi, x−i) − Ui(x)] (3)
The interpretation of � (y, x) is analogous: each summand inthis case is the increase in payoff a player will receive by unilat-erally deviating and playing a strategy x while other players playaccording to y. � (x, y) is everywhere non-positive for all feasible ywhen x is a NE profile, a result that follows directly from Definition
1 because at a NE no player can increase their payoff by unilaterallydeviating. Thus the NI function plays the role of a “merit function”measuring the proximity of a strategy to NE. In other words, thecloser � (x, y) when compared to � (y, x), is to 0, the closer x is to
1 mputing 12 (2012) 161–173
aNgd
5
DPHtAttB(
At
ttDafipsppis
Table 1Example of chromosome encoding of a strategy profile in a hypothetical game with2 players and 2 strategic variables per player.
Player 1’s strategies Player 2’s strategies
64 A. Koh / Applied Soft Co
NE compared to y. Without explicitly using the NI function, theash Domination procedure suggested in [33] achieves the sameoal by instead counting the number of players that can profitablyeviate.
.2. The NDEMO algorithm
Based on Lemma 1, we can find the NE by checking for Nashominance when comparing chromosomes. This replaces the usualareto Dominance check when using a standard EMO algorithm.ence instead of locating the Pareto Front, we collect its analogue:
he Nash Non Dominated Front to which the population converges. proposed Nash Domination Evolutionary Multiplayer Optimiza-
ion (NDEMO) algorithm is given in Algorithm 2. NDEMO is based onhe method of [51] which relies on Differential Evolution (DE) [46].y modification of this selection criteria, any other EMO algorithmsee [4,11] for alternatives) can be used.
lgorithm 2 (Nash Domination evolutionary multiplayer optimiza-ion).1: Input: NP, Maxit , �, DE Control Parameters, payoff functions2: it ← 03: Randomly initialize a population of NP parent strategy profiles P4: Evaluate payoffs to players with P5: while it < Maxit or P not converged do6: for j = 1 to NP do7: use Algorithm 3 to create child strategy profiles vector y8: Cit
j← y
9: end for10: Evaluate payoffs to players with C11: T ←12: for j = 1 to NP do13: x ← Pit
j
14: y ← Citj
15: use Algorithm 1 to carry out Pairwise Nash DominationComparison between x and y to determine k(x, y) and k(y, x)
16: if k(x, y) < k(y, x) then17: discard y18: T ← x19: else if k(y, x) < k(x, y) then20: discard x21: T ← y22: else23: T ← x24: T← y25: end if26: end for27: if |T|> NP then28: Randomly trim T until NP remain29: end if30: Randomly choose a chromosome m from T31: Compute Euclidean norm between m and every member in T32: if maximum of norm ≤ � then33: Terminate34: else35: P(it+1) ← T36: it ← it + 137: end if38: end while39: Output: Nash Non Dominated Solutions
NDEMO operates as shown in Algorithm 2. The user specifieshe maximum number of iterations Maxit, the population size NP,he convergence tolerance, �(> 0), control parameters required inE, namely Mutation Factor F and Probability of Crossover CR [46]nd a procedure to compute payoffs. Initial parent strategy pro-les P are generated randomly. A hypothetical example of such arofile is shown in Table 1. Each chromosome is a vector in D dimen-ions with D being equal to the number of strategy variables per
layer multiplied by the number of players (assuming that everylayer has the same number of strategy variables). In the hypothet-
cal example given in Table 1 since there are two player with twotrategic variables each, we have that D is 4.
Variable 1 2 1 2
Value 2.75 20.14 0.126 30.133
Child strategy profiles C are created by applying the DE operatorsvia the stochastic combination of randomly chosen parents as dis-cussed in [46]. Algorithm 3 uses the “DE/rand/1/bin” [46] strategywhich means that the child vector is formed by adding 1 differencevector to a randomly selected population member and then under-goes binary crossover. Whilst the mutation procedure (line 11 ofAlgorithm 3) is performed on the entire vector, it must be notedthat crossover (line 13 of Algorithm 3) operates dimension-wise.In lines 14 of Algorithm 3, rand(0, 1) is a pseudo random numberbetween 0 and 1 and intr(1, D) is a pseudo random integer between1 and D. Finally to enforce bound constraints, we also utilize themethod suggested in [45] (line 16 of Algorithm 3) so that if thechild vector produced violates the bound constraints, it is reset toa point half way between its pre-mutation value and the boundviolated.
Algorithm 3. Creating a child vector via Differential Evolution1: Input: Current Population P2: Input: Mutation Factor F , Probability of Crossover CR3: Input: Lower Bounds LBd and Upper Bounds UBd in each dimension d4: Randomly choose 3 integers: r1, r2, r3 between 1 and NP such that:5: r1 /= j, r2 /= r1 /= j and r3 /= r2 /= r1 /= j,6: x ← Pj
7: a ← Pr1
8: b ← Pr2
9: c ← Pr3
10: Mutation: Produce a mutant vector z via a stochastic combination ofdonor vectors
11: z ← a + F(b − c)12: for d = 1 to D do13: Crossover
14: yd ←{
zd if rand(0, 1) < CR ∨ d = intr(1, D)xd otherwise
15: Enforce Bound Constraints
16: yd ←{
(xd + LBd)/2 if yd < LBd
(xd + UBd)/2 if yd > UBd
yd otherwise17: end for18: Output: child vector y
At each generation, parent and child strategy profiles arecompared one by one pairwise, following the Nash DominationComparison procedure of Algorithm 1. Those chromosomes that areNND are stored in a temporary population T. However, this meansthat the size of T, shown in line 27 of Algorithm 2 as |T|, may exceedNP. If this happens, we randomly trim T so that there will always beonly NP parents for the next generation (lines 27 to 29 of Algorithm2). To check convergence to the NE, randomly select a chromosomem and compute the Euclidean norm between m and every memberof T (lines 30 to 31 of Algorithm 2). If the maximum distance is lessthan �, the population is judged to have converged to a NE and thealgorithm can terminate. Otherwise the counter is increased andthe process is repeated.
6. Numerical examples
In this section six numerical examples occurring in threemultidisciplinary contexts are provided to demonstrate the appli-
cability of NDEMO to solving realistic problems. Table 2 givesthe parameters used for the numerical experiments. Note thatthough we allowed for a maximum of 400 iterations, all the exam-ples required less than this to meet the specified termination
A. Koh / Applied Soft Computing 12 (2012) 161–173 165
Table 2Parameters used in the NDEMO for all numerical experiments.
Control parameter
Mutation factor F 0.45Probability of crossover CR 0.35Population size NP 50Maximum number of iterations Maxit 400Termination tolerance � 0.0001
Table 3Production cost function parameter specification for players.
Firm i ωi �i �i
1 10 5 1.22 8 5 1.13 6 5 1.0
tuw
6
ppcenNiata
at
6
CoMprptr
Hdaaip
Table 4Example 1: Case 1 – Comparison of the results of NDEMO with results published inliterature.
the two points marked with a � correspond to the two solutionsreported in [37] (see Table 5) which were obtained using a deter-ministic non smooth method. If negotiations between the leaders
4 4 5 0.95 2 5 0.8
olerance � of 0.0001. All numerical experiments were conductedsing MATLABTM 7.8 running on a 32 bit WindowsTM XP machineith 4 GB of RAM.
.1. Examples from production of homogeneous product
The first example presented arises when firms compete in theroduction of a homogeneous product. The purpose of this exam-le is three fold. Firstly, we demonstrate that NDEMO successfullyonverges to previously reported results for games without anyquilibrium constraint (i.e. when the game is not hierarchical inature) and thus show that NDEMO can be applied to standardash games. Secondly, we use this example to demonstrate an
nstance of an MOEPEC when the players are assumed to cooper-te. Finally, we wish to compare the solution of the MOEPEC withhe NCEPEC for the purpose of emphasizing the distinction between
Non Cooperative Equilibrium and a Cooperative Equilibrium.The player dependent parameters (ωi, �i and �i) shown in Table 3
re found in [34]. These will be the parameters that we will use forhe 3 case studies for this example.
.1.1. Case 1: Example 1 as a Cournot–Nash gameHere we consider the situation in which the firms engage in a
ournot–Nash game amongst themselves. Because of the absencef a hierarchical structure, this game is neither a NCEPEC nor aOEPEC. However its inclusion serves to demonstrate that the pro-
osed algorithm is able to detect the NE and replicate the reportedesults in [18] and [30] where deterministic methods were pro-osed. In this setting, each firm maximizes individual profits fromhe sale of the homogeneous good (given as the difference betweenevenues and production costs) as given by (4).
Ui(q) = P(q)qi︸ ︷︷ ︸Revenues
− ωiqi +(
�i
�i + 1
)�i−1/�i xi
(�i+1)/�i︸ ︷︷ ︸Production Costs
where P(q) = 50001/1.1
(5∑
i=1
qi
)−(1/1.1)(4)
owever the price (P(q)) and hence individual firm revenues isependent not only on their on individual production levels butlso on that of their competitors. Using the parameters of NDEMO
s mentioned earlier, Table 4 reports results obtained from apply-ng NDEMO to this problem and also compares it against the resultsublished in the literature.
6.1.2. Case 2: Example 1 as a MOEPECLet us now consider the example from [37] where a separate
problem to be described, using the same parameters as in Case 1,is cast as a MOEPEC. In [37] it was assumed that Firms 1 and 5become the leaders in the game and cooperate to maximize indi-vidual profits again following (4) while the followers (firms 2, 3and 4) play a Nash game amongst themselves. This therefore givesrise to a MOEPEC.1 Given the production levels of the leaders andtreating these as exogenous, the followers seek to individually max-imize their profits using (4). Assuming that the payoff functionsare continuously differentiable (a condition easily verifiable for thisexample) the first order conditions for a profit maximum for eachof the followers are defined by (5):
CP
⎧⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎩
fi =∂Ui
∂qi
≥ 0
∂Ui
∂qi
qi = 0
qi ≥ 0
⎫⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎭
i ∈ {2, 3, 4} (5)
It is easy to see that (5) in fact defines a Complementarity Prob-lem (CP) [13,26,30] which when written in generic form is to findq ∈ R
n where f : Rn → R
n such that:
f (q) ≥ 0
qf (q) = 0
q ≥ 0
(6)
As the leaders (firms 1 and 5) cooperatively maximize their prof-its, the actions of the followers leads to (5) which is imposed as animplicit nonlinear constraint on the leaders’ actions. The resultingMOEPEC can be written as a vector optimization problem (with Tdenoting the transpose) in (7).
Maxq1,q5 [U1(q1, q−1), U5(q5, q−5)]T
subject to
q1, q5 ≥ 0
{q2, q3, q4} → sol CP
(7)
In (7) “sol CP” emphasizes that the production levels of thefollowers is the solution of the (nonlinear) CP given by (5). TheMultiobjective Self Adaptive Differential Evolution (MOSADE) [23]algorithm was used to generate the Pareto Front corresponding to(7) and the resulting front is shown in Fig. 3. In doing so, we inte-grated within MOSADE, the PATH Solver from [9] to solve the CP (i.e.(5)) for each vector of the production levels of the leaders. On Fig. 3,
1 This is in fact a MultiObjective Equilibrium Problem with ComplementarityConstraints (MOEPCC) but the MOEPCC is a special case of the MOEPEC and thedistinction does not affect our ensuing discussions.
166 A. Koh / Applied Soft Computing 12 (2012) 161–173
0 200 400 600 800 1000 12000
100
200
300
400
500
600
700
Pro
fit: L
eade
r 2
wt
6
tatu5fa
P
P
mrwN
aiOtw
TT
Table 6Example 1: Case 3 – Production levels and profits for leaders in NCEPEC.
Leader 1 Leader 2
Production Profit Production Profit
97.70 950.56 42.14 414.72
820 840 860 880 900 920 940 960 980 1000
380
400
420
440
460
480
Profit: Leader 1P
rofit
: Lea
der
2
Profit: Leader 1
Fig. 3. Example 1 – Case 2: Pareto Front for MOEPEC.
ere allowed under prevailing anti-trust legislation, we conjecturehat this Pareto Front would play a key role in these negotiations.
.1.3. Case 3: Example 1 as a NCEPECUsing the same parameters as in the previous two cases, assume
hat these same leaders, firms 1 and 5 do not cooperate as they weressumed to do in Case 2 but instead play a Nash game amongsthemselves. The optimization problem facing each leader individ-ally is given in (8) and (9) respectively. As the leaders (firms 1 and) individually maximize their profits, the production levels of theollowers leads to the complementarity problem which is imposeds an implicit nonlinear constraint on the leaders’ actions.
layer 1/Leader 1
⎧⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎩
Maxq1 U1(q1, q−1)
subject to
q1 ≥ 0
q5 = q5
{q2, q3, q4} → sol CP
(8)
layer 5/Leader 2
⎧⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎩
Maxq5 U5(q5, q−5)
subject to
q1 ≥ 0
q1 = q1
{q2, q3, q4} → sol CP
(9)
Critically compared to the MOEPEC, in the NCEPEC, when opti-izing their individual profits, each leader searches for the best
esponse to the other firm’s production level. Integrating NDEMOith the PATH Solver [9] to resolve the CP as before, we applyDEMO to solve the resulting NCEPEC.
The result produced by the NDEMO algorithm is given in Table 6nd this solution (in profit space) is marked with × on Fig. 4. As
llustrated in Fig. 4, the non cooperative outcome is not Paretoptimal. It is obvious that any one of the leaders can be made bet-
er off (i.e. increase individual profits) without making the otherorse off. For example, holding the profit from Leader 1 fixed at
able 5he two solutions reported in [37] and indicated on Fig. 3 with �.
Solution 1 Solution 2
Profit of Leader 1/Firm 1 840.86 978.89Profit of Leader 2/Firm 5 485.63 410.97
Fig. 4. Example 1 – Case 3: The NCEPEC solution (×) is not on the Pareto Front.
× of 950.56, one can move upwards (in the direction of the arrow)towards the Pareto Front and hence increase the profit of Leader2 without reducing the profit accruing to Leader 1. This outcomehighlights the key difference between the MOEPEC and the NCEPECand the proposed NDEMO algorithm is designed for the latter. Ourfinding is similar to that concluded in [21]2 who also found thatthe result obtained by the Nash Genetic Algorithm [53] lies insidethe Pareto Front generated by a conventional EMO algorithm whenoptimizing a problem arising in the steel forging industry. In ourexample, the primary reason that the Nash point lies inside thePareto Front is attributable to the assumption of non-cooperativebehaviour between the leaders.
6.2. Examples from private sector participation in the operationof toll roads
The next three examples presented are typical of situationswhen private profit maximizing firms compete with one anotherin the operation of toll roads. The private firms, acting as leaders,set their strategic decision variables and the followers (who arein effect the highway users) optimize their route choice accord-ing to Wardrop’s Equilibrium Condition [58]. We seek therefore tocompute the Nash Equilibrium strategic variables of these games.3
We define the notation for a mathematical statement of theproblem:
A: the set of directed links in a traffic network,B: the set of links which are subject to tolls B ⊂ A,
Q: the set of origin destination (O-D) pairs in the network,v: the vector of link flows v = [va] a ∈ A,: the vector of link tolls with a = 0, ∀ a /∈ B,
2 Fig. 2 in [21] is analogous to Fig. 4 in this paper.3 We measure tolls in units of seconds and revenues/profits in seconds following
conventions in the transportation planning literature [40].
A. Koh / Applied Soft Computing 12 (2012) 161–173 167
F(
oi
M
w
c
td[ta
6
spiWNwv
6
1bTl
TE
Table 8Example 3 – NCEPEC tolls (s).
Firm Road NDEMO Method of [28]
1 7 140.94 140.962 10 137.51 137.56
The three links (numbered 9, 10 and 11) subject to tolls andcapacity enhancements are shown as dashed lines in Fig. 6. Detailsof the link parameters and the functional forms of the traveldemand relationships can be found in [60] where this was solved
ig. 5. Highway Network with 18 directed links from [28] for Examples 1 and 2links labeled are road numbers).
c(v): the vector of monotonically non decreasing travel costs as afunction of link flows on that link only,c = [ca(va)] a ∈ A,: the vector of generalized travel cost for each OD pair = [q],q ∈ Q,ı: the continuous and monotonically decreasing demand func-tion for each O-D pair as a function of the generalized travel costbetween OD pair q alone, ı = [ıq], q ∈ Q,ı−1: the inverse demand function giving the highway travel costas a function of the demands and˝: feasible region of flow vectors, defined by a linear equationsystem of flow conservation constraints.
For simplicity suppose that each player is able to set tolls onlyn a single link in the network then each seeks to maximize itsndividual revenue (payoff) as given in (10)
axi
Ui(�) = vi(�)i, ∀i ∈ 1, 2 . . . n (10)
here vi is obtained by solving the variational inequality (11) i.e.
Hence for a specified toll vector , the solution of the Varia-ional Inequality defined by 11 results in a vector of link flows andemands (v∗, ı∗ ∈ ˝) that satisfies Wardrop’s Equilibrium principle58] of route choice (see [8,55]). It is well known that the varia-ional inequality in (11) can be solved by means of a standard trafficssignment algorithm once the vector of tolls have been input [8].
.2.1. Example 2This example is taken from [28]. Let roads number 7 and 10
hown in Fig. 5 be the only toll roads operated by two independentlayers. This example was solved as a complementarity problem
n [28] and via a Coevolutionary Particle Swarm Algorithm in [27].e employed NDEMO with the parameters given in Table 2 and
DEMO took 12 minutes to converge to the tolls shown in Table 7hich agrees with previous results. The standard deviation of the
ariables at convergence were both less than 0.0001.
.2.2. Example 3Next we consider the situation when, in addition to roads 7 and
0 being tolled as in Example 2, another player maximizes payoffsy charging tolls on road 17. The results are reported in Table 8.he standard deviation of the variables at convergence were bothess than 0.0001. Although NDEMO again successfully converged
able 7xample 2 – NCEPEC tolls (s).
Firm Road NDEMO [27] [28]
1 7 141.36 141.36 141.372 10 138.28 138.29 138.29
3 17 711.25 712.88
to the NE (as verified by solving it as a complementarity prob-lem following the method described in [28]), this time NDEMOtook 19 minutes to meet the same convergence criteria. Thus withone additional player, the time taken has increased by 40% overthe 2 player case. The increase in computing time stems from thedomination checking procedure of Algorithm 1 combined with thehierarchical nature of the game. This arises from the need to solvea traffic assignment problem for each unilateral deviation (so as toobtain k(x, y) and k(y, x)).
6.2.3. Example 4In this example we consider the 11 link network from [60] where
each player has two strategic variables. In this case, beyond col-lecting the toll revenues, each player also has to finance capacityexpansion of the network link operated. In addition to the notationintroduced earlier, we redefine B to be the set of links which aresubject to both tolls and capacity enhancements, B ⊂ A. Further let
represent the vector of link capacity enhancements with ˇa = 0,∀ a /∈ B.
The payoff to player i, i ∈ N is the difference between the tollrevenue obtained by collecting tolls from traffic using the link andthe amortized cost of providing the capacity enhancements, I(ˇi).Thus is a parameter that transfers the costs of the project into unitperiod costs. Mathematically, the resulting choice of the strategicvariables for each player may be represented by the optimizationproblem in (12):
Maxi,ˇi
Ui(, ˇ) = vi(, ˇ)i − ˛I(ˇi), ∀i ∈ N (12)
where vi is obtained by solving the variational inequality represent-ing Wardrop’s User Equilibrium Condition (13)
Fig. 6. Highway Network with 11 directed links from [60] for Example 3 (linkslabeled are road numbers), Dash lines indicate links (9, 10 and 11) which are subjectto tolls and capacity expansion.
168 A. Koh / Applied Soft Computing 12 (2012) 161–173
Table 9Example 4 – NCEPEC tolls and capacities.
NDEMO [60]
Firm Link Toll (s) Capacity (vehicles) Profit (s/h) Toll (s) Capacity (vehicles) Profit (s/h)
ia a heuristic gradient based procedure. The results in [60] areompared with those produced by NDEMO in Table 9. At con-ergence, the standard deviation of the population of toll andapacity enhancement variables are all less than 0.0001. AlthoughDEMO took about 18 minutes to converge to the specified tol-rance (which is similar to the time taken in Example 2) thisnding suggests that increasing the number of strategic variableser player did not have a significant effect on the performance of thelgorithm.
We provide plots of the mean and standard deviation of the
opulation at each iteration to illustrate the convergence of NDEMOor this problem. The left and right panes of Figs. 7–9 show the
eans and standard deviation of the population of toll variables
0 50 100 150 200 250 3004
5
6
7
8
9
10
11
12
NDEMO Iteration
Mea
n of
Tol
ls o
n Li
nk 9
(P
laye
r 1)
Fig. 7. Example 4 – Mean and standard de
0 50 100 150 200 250 3004
4.5
5
5.5
6
6.5
7
7.5
8
8.5
NDEMO Iteration
Mea
n of
Tol
ls o
n Li
nk 1
0 (P
laye
r 2)
Fig. 8. Example 4 – Mean and standard dev
.98 2.97 61.88 25.92
for each player over the 300 iterations of the algorithm. Similarplots are provided in Figs. 10–12 of the population of the capacityvariables for each player.
6.3. Examples from the electricity generation industry
In deregulated electricity markets, electricity generating com-panies (“GENCOs”) submit bids of the quantities of electricity theypropose to supply to meet the market demand to maximize their
profits. These bids are then cleared by the Independent SystemOperator (ISO). However the price and individual profits are notonly dependent on their individual bids but also that of their com-petitors and the prices are not known until the market clearing
0 50 100 150 200 250 3000
1
2
3
4
5
6
7
NDEMO Iteration
Sta
ndar
d D
evia
tion
of T
olls
on
Link
9 (
Pla
yer
1)
viation of toll for Player 1 on Link 9.
0 50 100 150 200 250 3000
1
2
3
4
5
6
NDEMO Iteration
Sta
ndar
d D
evia
tion
of T
olls
on
Link
10
(Pla
yer
2)
iation of toll for Player 2 on Link 10.
A. Koh / Applied Soft Computing 12 (2012) 161–173 169
0 50 100 150 200 250 3004
5
6
7
8
9
10
11
12
NDEMO Iteration
Mea
n of
Tol
ls o
n Li
nk 1
1 (P
laye
r 3)
0 50 100 150 200 250 3000
1
2
3
4
5
6
7
NDEMO Iteration
Sta
ndar
d D
evia
tion
of T
olls
on
Link
11
(Pla
yer
3)
Fig. 9. Example 4 – Mean and standard deviation of toll for Player 3 on Link 11.
0 50 100 150 200 250 30080
100
120
140
160
180
200
220
240
260
Mea
n of
Cap
aciti
es o
n Li
nk 9
(P
laye
r 1)
0 50 100 150 200 250 3000
20
40
60
80
100
120
140
160
180
Sta
ndar
d D
evia
tion
of C
apac
ities
on
Link
9 (
Pla
yer
1)
d devi
iNti
NDEMO Iteration
Fig. 10. Example 4 – Mean and standar
s performed by the ISO [1,15,17]. This results in a NCEPEC andDEMO can be applied to such “pool based bidding” games. The last
wo examples in this paper illustrate the performance of NDEMO
n this context.
0 50 100 150 200 250 300180
185
190
195
200
205
210
215
220
225
NDEMO Iteration
Mea
n of
Cap
aciti
es o
n Li
nk 1
0 (P
laye
r 2)
Sta
ndar
d D
evia
tion
of C
apac
ities
on
Link
10
Fig. 11. Example 4 – Mean and standard devia
NDEMO Iteration
ation of capacity for Player 1 on Link 9.
6.3.1. Example 5: Two Bus Model with 2 playersTwo players, indexed by i, i ∈ {1, 2}, submit bids to generate elec-
tricity to maximize individual profits. As shown in (14), this is given
by the difference between revenues and costs from production ofqi units of electricity.
Ui(qi, q−i) = �∗i qi − ci(qi), i ∈ {1, 2} (14)
0 50 100 150 200 250 3000
50
100
150
NDEMO Iteration
(Pla
yer
2)
tion of capacity for Player 2 on Link 10.
170 A. Koh / Applied Soft Computing 12 (2012) 161–173
0 50 100 150 200 250 3000
50
100
150
200
250
NDEMO Iteration
Mea
n of
Cap
aciti
es o
n Li
nk 1
1 (P
laye
r 3)
0 50 100 150 200 250 3000
50
100
150
NDEMO Iteration
Sta
ndar
d D
evia
tion
of C
apac
ities
on
Link
11
(P
laye
r 3)
Fig. 12. Example 4 – Mean and standard deviation of capacity for Player 3 on Link 11.
Fig. 13. Two bus network model from [54], Players 1 and 2 are located at G1 and G2r
dbwIo
ı
s
ı
ı
−
Ti(rattrft
elot
Fig. 14. Three bus network model from [7,54]. Players 1, 2 and 3 are located at G1,
Ui(qi, q−i) = �∗i qi − ci(qi), i ∈ {1, 2, 3} (16)
Table 10Parameters of the demand function for Two Bus Model [54].
Table 11Example 5 – Two Bus Model NCEPEC bid quantities (MW/h).
Player 1 2
espectively.
However, as mentioned before, the prices �∗1, �∗2 can only beetermined by the solution of a market clearing problem carried outy the ISO [15,17]. Specific to the Two Bus Model shown in Fig. 13ith Players 1 and 2 located at buses G1 and G2 respectively, the
SO’s market clearing task is embodied in the solution of the systemf equations in (15) for given bid submissions i.e. {q1, q2}.
Max1,ı2,�1
B1(ı1) + B2(ı2) (15a)
ubject to
1 − q1 + �1 = 0 (15b)
2 − q2 − �1 = 0 (15c)
TMax ≤ �1 ≤ TMax (15d)
he objective function (15a) of the market clearing problem is max-mization of the total benefits, the equality constraints in (15b) and15c) represent Kirchoff’s Law and the inequality constraint (15d)epresents the transmission limits on the line. The prices, �∗1 and �∗2,re given by the Lagrange multipliers of the equality constraints inhe market clearing problem (15b) and (15c) respectively. Withinhe scope of this paper it is the market clearing problem (6.3.1) thatepresents the equilibrium constraint facing each player which is aunction not only of their own bids but also that of their competi-or’s.
The benefit functions at each node/bus and the cost functions for
ach player from [54] are shown in Table 10. The line transmissionimit TMax is 80. Table 11 compares the results from [54] with thatbtained by NDEMO where 200 iterations were required to achievehe tolerance � of 1e−4.
G2 and G3 respectively.
6.3.2. Example 6: Three Bus Model with 3 playersIn this example, we consider the three player model from [7] and
the 3 bus network used is shown in Fig. 14. Three players submitbids to generate electricity to maximize individual profits from thegeneration of electricity according to (16).
[54] 148 148Results from NDEMO:
Mean 148.1267 148.1542Standard deviation 0.000289 0.000097
A. Koh / Applied Soft Computing 12 (2012) 161–173 171
Table 12Parameters of the demand function for Three Bus Model [7].
Fig. 15. Example 6 – Mean and standard deviation of bids for Player 1 on Bus 1.
0 50 100 150 200 250 300900
950
1000
1050
1100
1150
1200
NDEMO iteration
Mea
n of
bid
s by
pla
yer
2 (q
2)
0 50 100 150 200 250 3000
50
100
150
200
250
300
350
400
NDEMO iteration
Sta
ndar
d D
evia
tion
of b
ids
by p
laye
r 2
(q2
)
ard de
p(pisp(
ı
s
2
−−−
of the population of each player’s bids, qi, i ∈ {1, 2}, over the itera-tions.
Table 13Example 6–Three Bus Model NCEPEC bid quantities (MW/h).
Player 1 2 3
Fig. 16. Example 6 – Mean and stand
Once again the prices are determined by the market clearingroblem given in (6.3.2). The equality constraints (17b), (17c) and17d) represent the dc powerflow equations. The market clearingrice tuple (the so-called locational marginal prices) �∗
i, i ∈ {1, 2, 3}
s, as before, given by the Lagrange multiplier of these equality con-traints. In this system of equations (6.3.2), �1 and �2 represent theowerflows on lines AC and −AC respectively. The last constraint17e) is the transmission limit on the line.
Max1,ı2,ı3
B1(ı1) + B2(ı2) + B3(ı3) (17a)
ubject to
�1 − �2 = q1 − ı1 (17b)
�1 + 2�2 = q2 − ı2 (17c)
�1 − �2 = q3 − ı3 (17d)
TMax ≤ �1 ≤ TMax (17e)
viation of bids for Player 2 on Bus 2.
The line transmission limit TMax is 100 and the individual benefitfunctions at each node/bus and the cost functions for each playerare shown in Table 12 as reported in [7].
Table 13 compares the results from [7] with that obtained byNDEMO where 300 iterations were required to achieve the toler-ance � of 1e−4. Figs. 15–17 show the mean and standard deviation
[7] 1105 1046 995Results from NDEMO:
Mean 1105.396 1046.238 995.177Standard deviation 0.00104 0.00053 0.0008815
172 A. Koh / Applied Soft Computing 12 (2012) 161–173
0 50 100 150 200 250 300750
800
850
900
950
1000
NDEMO iteration
Mea
n of
bid
s by
pla
yer
3 (q
3)
0 50 100 150 200 250 3000
50
100
150
200
250
300
350
400
NDEMO iteration
Sta
ndar
d D
evia
tion
of b
ids
by p
laye
r 3
(q3)
ard de
7
ir(eftoe
ipTtIiiastloCNafMtsaptl
sdtIpfa
uc
Fig. 17. Example 6 – Mean and stand
. Conclusions
In this paper, we proposed an evolutionary algorithm for solv-ng EPECs by extending the procedure suggested in [33]. Theesulting Nash Domination Evolutionary Multiplayer OptimizationNDEMO) algorithm enabled us to handle Nash games where play-rs encounter a system equilibrium constraint. We highlighted theact that the critical Nash Domination procedure used in NDEMOo select between parent and child chromosomes is in fact the-retically rooted in the well established Nikaido Isoda functionxtending the original contribution of [33].
To assess the performance of NDEMO, six examples were givenn this paper. The first,broken down into three case studies, usedarameters from a well documented 5 player Cournot–Nash model.he three case studies of the first example were given to underlinehe salient points of the market structure of competition assumed.n the first case study, we assumed that the players were compet-ng non cooperatively but on an equal footing and this resultedn a standard Cournot–Nash game for which NDEMO could bepplied. In the second and third case studies, two players pre-ented themselves as “market leaders” and this results in eitherhe cooperative EPEC which is a MultiObjective Equilibrium Prob-em with Equilibrium Constraints (MOEPEC) (second case study)r the Non Cooperative Equilibrium Problem with Equilibriumonstraints (NCEPEC) (third case study). The proposed algorithm,DEMO, is designed for the latter case and conventional evolution-ry multiobjective optimization (EMO) algorithms could be usedor the former. This example highlights the difference between a
OEPEC and a NCEPEC, with the former arising from the assump-ion of cooperative behaviour amongst the leaders and the lattertems from assuming that the leaders engage in a Nash gamemongst themselves. In both cases the strategies the leaders canlay are subject to the actions of the followers which manifests inhe form of an implicit nonlinear constraint on the actions of theeaders.
Three numerical examples illustrating competition in privateector participation in highway transportation and two examplesrawn from pool based bidding in the electricity generation indus-ry were further used to demonstrate the performance of NDEMO.n all instances, it was clear that NDEMO successfully converged toreviously reported results in the literature and underscores theact that the proposed algorithm is suitable for multidisciplinary
pplications.
While the examples suggest that this could be a potentiallyseful method for EPECs, we stress the need, in the pairwiseomparison, to compute the payoff to each player, one by one,
viation of bids for Player 3 on Bus 3.
from deviating. This implies that the computational complexity ofNDEMO increases significantly as the number of players increase asevidenced by the increase in computational times required in ourexamples. However, increasing the strategic variables available toeach player did not significantly increase the time taken to solvethe problem.
Further research would consider the effects of the controlparameters of NDEMO on the speed of convergence to NND solu-tions. In this research we have used control parameters of theembedded Differential Evolution operators suggested in [51]. Nev-ertheless these parameters are in no way regarded as perfect andit is hypothesized that well chosen parameters may reduce the runtime of the NDEMO algorithm.
Acknowledgements
The research reported here is funded by the Engineering andPhysical Sciences Research Council of the UK under the “Compet-itive Cities” Grant EP/H021345/1. The author would like to thanktwo anonymous referees for their comments and suggestions onimproving the presentation of an earlier draft.
The author would like to express his gratitude to HosseinHajimirsadeghi (Control and Intelligent Processing Center of Excel-lence, University of Tehran, Iran), Professor Ross Baldick (Universityof Texas at Austin, USA) and Dr You Seok “Peter” Son for clarificationof the examples pertaining to the electricity generation industry. Allerrors and omissions remain the responsibility of the author.
References
[1] J. Bower, D.W. Bunn, Model-based comparisons of pool and bilateral marketsfor electricity, Energy Journal 21 (3) (2000) 1–29.
[2] M. Cervinka, Hierarchical structures in equilibrium problems, PhD Thesis,Charles University, Prague, Czech Republic, 2008.
[3] K. Chellapilla, D. Fogel, Evolving neural networks to play checkers with-out expert knowledge, IEEE Transactions on Neural Networks 10 (6) (1999)1382–1391.
[4] C. Coello-Coello, G. Lamont, Applications of Multi-objective Evolutionary Algo-rithms, World Scientific, Singapore, 2004.
[5] J. Contreras, M. Klusch, J.B. Krawzyck, Numerical solutions to Nash–Cournotequilibrium in electricity markets, IEEE Transactions on Power Systems 19 (1)(2004) 195–206.
[6] T. Curzon Price, Using co-evolutionary programming to simulate strategicbehavior in markets, Journal of Evolutionary Economics 7 (3) (1997) 219–254.
[7] L.B. Cunningham, R. Baldick, M.L. Baughman, An empirical study of appliedgame theory: transmission constrained Cournot behavior, IEEE Transactionson Power Systems 2 (1) (2002) 166–172.
[9] S.P. Dirkse, M.C. Ferris, The PATH solver: a non-monotone stabilization schemefor mixed complementarity problems, Optimization Methods and Software 5(2) (1995) 123–156.
10] A. Ehrenmann, Manifolds of multi-leader Cournot equilibria, OperationsResearch Letters 32 (2) (2004) 121–125.
11] K. Deb, Multi-objective Optimization Using Evolutionary Algorithms, JohnWiley, Chichester, 2001.
12] F. Facchinei, C. Kanzow, Generalized Nash equilibrium problems, Annals ofOperations Research 175 (1) (2010) 177–211.
13] F. Facchinei, J.S. Pang, Finite Dimensional Variational Inequalities and Comple-mentarity Problems, vol. 1, Springer, New York, 2003.
14] F. Facchinei, J.S. Pang, Finite Dimensional Variational Inequalities and Comple-mentarity Problems, vol. 2, Springer, New York, 2003.
15] D. Gan, D.V. Bourcier, A single-period auction game model for modelingoligopolistic competition in pool-based electricity markets, in: IEEE PowerEngineering Society Winter Meeting, 2002, pp. 101–106.
16] D. Gabay, H. Moulin, On the uniqueness and stability of Nash-equilibria in noncooperative games, in: A. Bensoussan, et al. (Eds.), Applied Stochastic Controlin Econometrics and Management Science, North Holland, Amsterdam, 1980,pp. 271–293.
17] J.D. Glover, M.S. Sarma, T. Overbye, Power Systems Analysis and Design, Thom-son, Toronto, 2008.
18] P.T. Harker, A variational inequality approach for the determination ofoligopolistic market equilibrium, Mathematical Programming 30 (1) (1984)105–111.
19] P.T. Harker, Generalized Nash games and quasi-variational inequalities, Euro-pean Journal of Operational Research 54 (1) (1991) 81–94.
20] A. Haurie, J. Krawczyk, Optimal charges on river effluent from lumped anddistributed sources, Environmental Modelling and Assessment 2 (3) (1997)177–189.
21] B.M. Hodge, F. Pettersson, N. Chakraborti, Re-evaluation of the optimal oper-ating conditions for the primary end of an integrated steel plant usingMulti-objective Genetic Algorithms and Nash Equilibrium, Steel Research Inter-national 77 (7) (2006) 459–461.
22] X. Hu, D. Ralph, Using EPECs to model bilevel games in restructured elec-tricity markets with locational prices, Operations Research 55 (5) (2007)809–827.
23] V.L. Huang, A.K. Qin, P.N. Suganthan, M.F. Tasgetiren, Multi-objective optimiza-tion based on self-adaptive differential evolution algorithm, in: Proceedings ofIEEE CEC, 2007, pp. 3601–3608.
24] A. von Heusinger, C. Kanzow, Relaxation methods for generalized Nash equi-librium problems with inexact line search, Journal of Optimization Theory andApplications 143 (1) (2009) 159–183.
25] K. Judd, Numerical methods in Economics, MIT Press, Cambridge, MA, 1998.26] S. Karamardian, Generalized complementarity problems, Journal of Optimiza-
tion Theory and Applications 8 (3) (1971) 161–168.27] A. Koh, Coevolutionary particle swarm algorithm for bi-level variational
inequalities: applications to competition in highway transportation networks,in: R. Chiong, S. Dhakal (Eds.), Natural intelligence for scheduling, planning andpacking problems, Springer, Berlin, 2010, pp. 195–217.
28] A. Koh, S. Shepherd, Tolling, collusion and equilibrium problems with equilib-rium constraints, European Transport (Trasporti Europei) 43 (2010) 3–22.
29] A. Koh, Nash Dominance with applications to equilibrium problems with equi-librium constraints, in: X.-Z. Gao, A. Gaspar-Cunha, M. Köppen, G. Schaefer, J.Wang (Eds.), in: Soft Computing in Industrial Applications: Algorithms, Inte-gration, and Success Stories, Advances in Intelligent and Soft Computing, vol.75, Springer Verlag, Berlin, 2010, pp. 71–79.
30] M. Kolstad, L. Mathisen, Computing Cournot–Nash equilibrium, OperationsResearch 39 (5) (1991) 739–748.
31] Z. Luo, J. Pang, D. Ralph, Mathematical Programs with Equilibrium Constraints,Cambridge University Press, Cambridge, 1996.
32] S. Leyffer, T. Munson, Solving multi-leader–common-follower games, Opti-mization Methods and Software 25 (4) (2010) 601–623.
33] R.I. Lung, D. Dumitrescu, Computing Nash equilibria by means of evolution-ary computation, International Journal of Computers, Commmunications andControl III (2008) 364–368.
35] B.S. Mordukhovich, Optimization and equilibrium problems with equilibriumconstraints, Omega 33 (5) (2005) 379–384.
36] B.S. Mordukhovich, Variational analysis and generalized Differentiation, II:Applications. Grundlehren der mathematischen wissenschaften, vol. 331,Springer, Berlin, 2006.
37] B.S. Mordukhovich, J.V. Outrata, M. Cervinka, Equilibrium problems with com-plementarity constraints: case study with applications to oligopolistic markets,Optimization 56 (4) (2007) 479–494.
38] A. Nagurney, Network Economics: A Variational Inequality Approach, in:Advances in Computational Economics, Kluwer, Boston, 1999.
39] J. Nash, Equilibrium points in N-person games, Proceedings of the NationalAcademy of Science USA 36 (1) (1950) 48–49.
40] J. Ortuzar, L. Willumsen, Modelling Transport, Wiley, Chichester, 1990.41] J. Outrata, M. Kocvara, J. Zowe, in: Nonsmooth Approach to Optimization Prob-
lems with Equilibrium Constraints, Kluwer, Boston, 1998.42] J.V. Outrata, A note on a class of equilibrium problems with equilibrium con-
straints, Kybernetika 40 (5) (2004) 585–594.43] J.P. Pedroso, Numerical solution of Nash and Stackelberg equilibria: an evolu-
tionary approach, in: Proceedings of SEAL’96, 1996, pp. 151–160.44] M.A. Potter, K. De Jong, A cooperative coevolutionary approach for function
optimization, in: Proceedings of PPSN III, Springer, Berlin, 1994, pp. 249–257.45] K. Price, An Introduction to Differential Evolution, in: D. Corne, M. Dorigo, F.
Glover (Eds.), New Techniques in Optimization, McGraw Hill, London, 1999,pp. 79–108.
46] K. Price, R. Storn, J. Lampinen, Differential evolution: a practical approach toglobal optimization, Springer, Berlin, 2005.
47] M. Protopapas, E. Kosmatopoulos, Determination of sequential best replies inn-player games by genetic algorithms, International Journal of Applied Math-ematics and Computer Science 5 (1) (2009) 19–24.
48] R. Rajabioun, E. Atashpaz-Gargari, C. Lucas, Colonial competitive algorithm asa tool for Nash equilibrium point achievement, in: Proceedings of ICCSA, LNCS5073, Springer, Berlin, 2008, pp. 680–695.
49] A. Rapoport, A. Chammah, Prisoner’s Dilemma, University of Michigan Press,Ann Arbour, Michigan, 1965.
50] K. Razi, S.H. Shahri, A.R. Kian, Finding Nash equilibrium point of nonlinear non-cooperative games using coevolutionary strategies, in: Proceedings of ISDA,2007, pp. 875–882.
51] T. Robic, B. Filipic, DEMO: differential evolution for multiobjective problems,in: Proceedings of EMO2005, LNCS 3410, Springer, Berlin, 2005, pp. 520–533.
52] J.B. Rosen, Existence and uniqueness of equilibrium point for concave N persongames, Econometrica 33 (3) (1965) 520–534.
53] M. Sefrioui, J. Periaux, Nash genetic algorithms: examples and applications, in:Proceedings of IEEE CEC, 2000, pp. 509–516.
54] Y. Son, R. Baldick, Hybrid coevolutionary programming for Nash equilibriumsearch in games with local optima, IEEE Transactions on Evolutionary Compu-tation 8 (4) (2004) 305–315.
55] M.J. Smith, The existence, uniqueness and stability of traffic equilibria, Trans-portation Research Part B 13 (4) (1979) 295–304.
56] H.H. von Stackelberg, The theory of the market economy, William Hodge, Lon-don, 1952.
57] C. Su, Equilibrium problems with equilibrium constraints: stationarities, algo-rithms and applications. PhD Thesis, Stanford University, California, USA, 2005.
58] J.G. Wardrop, Some theoretical aspects of road traffic research, Proceedings ofInstitution of Civil Engineers Part II 1 (36) (1952) 325–378.
59] J.N. Webb, Game theory: decisions, interaction and Evolution, Springer, London,2007.
60] H. Yang, X. Feng, H. Huang, Private road competition and equilibrium with traf-fic equilibrium constraints, Journal of Advanced Transportation 43 (1) (2009)
21–45.
61] J.J. Ye, Q.J. Zhu, Multiobjective optimization problem with variational inequalityconstraints, Mathematical Programming Series A 96 (1) (2003) 139–160.
62] L. Zubeita, A network equilibrium model for oligopolistic competition in citybus services, Transportation Research Part B 32 (6) (1998) 413–422.
