Morfismos, Vol 9, No 1, 2005

VOLUMEN 9NÚMERO 1

ENERO A JUNIO DE 2005 ISSN: 1870-6525

MorfismosComunicaciones EstudiantilesDepartamento de Matematicas

Cinvestav

Editores Responsables

• Isidoro Gitler • Jesus Gonzalez

Consejo Editorial

• Luis Carrera • Samuel Gitler• Onesimo Hernandez-Lerma • Hector Jasso Fuentes

• Miguel Maldonado • Raul Quiroga Barranco• Enrique Ramırez de Arellano • Enrique Reyes

• Armando Sanchez • Martın Solis• Leticia Zarate

Editores Asociados

• Ricardo Berlanga • Emilio Lluis Puebla• Isaıas Lopez • Guillermo Pastor

• Vıctor Perez Abreu • Carlos Prieto• Carlos Renterıa • Luis Verde

Secretarias Tecnicas

• Roxana Martınez • Laura Valencia

Morfismos puede ser consultada electronicamente en “Revista Morfismos”de la direccion http://www.math.cinvestav.mx. Para mayores informes dirigirseal telefono 50 61 38 71.

Toda correspondencia debe ir dirigida a la Sra. Laura Valencia, Departa-mento de Matematicas del Cinvestav, Apartado Postal 14-740, Mexico, D.F.07000 o por correo electronico: [email protected].

VOLUMEN 9NÚMERO 1

ENERO A JUNIO DE 2005ISSN: 1870-6525

Informacion para Autores

El Consejo Editorial de Morfismos, Comunicaciones Estudiantiles del Departamento deMatematicas del CINVESTAV, convoca a estudiantes de licenciatura y posgrado a someterartıculos para ser publicados en esta revista bajo los siguientes lineamientos:

• Todos los artıculos seran enviados a especialistas para su arbitraje. No obstante, losartıculos seran considerados solo como versiones preliminares y por tanto pueden serpublicados en otras revistas especializadas.

• Se debe anexar junto con el nombre del autor, su nivel academico y la instituciondonde estudia o labora.

• El artıculo debe empezar con un resumen en el cual se indique de manera breve yconcisa el resultado principal que se comunicara.

• Es recomendable que los artıculos presentados esten escritos en Latex y sean enviadosa traves de un medio electronico. Los autores interesados pueden obtener el for-mato LATEX2ε utilizado por Morfismos en “Revista Morfismos” de la direccion webhttp://www.math.cinvestav.mx, o directamente en el Departamento de Matematicasdel CINVESTAV. La utilizacion de dicho formato ayudara en la pronta publicaciondel artıculo.

• Si el artıculo contiene ilustraciones o figuras, estas deberan ser presentadas de formaque se ajusten a la calidad de reproduccion de Morfismos.

• Los autores recibiran un total de 15 sobretiros por cada artıculo publicado.

• Los artıculos deben ser dirigidos a la Sra. Laura Valencia, Departamento de Matemati-cas del Cinvestav, Apartado Postal 14 - 740, Mexico, D.F. 07000, o a la direccion decorreo electronico [email protected]

Author Information

Morfismos, the student journal of the Mathematics Department of the Cinvestav, invitesundergraduate and graduate students to submit manuscripts to be published under thefollowing guidelines:

• All manuscripts will be refereed by specialists. However, accepted papers will beconsidered to be “preliminary versions” in that authors may republish their papers inother journals, in the same or similar form.

• In addition to his/her affiliation, the author must state his/her academic status (stu-dent, professor,...).

• Each manuscript should begin with an abstract summarizing the main results.

• Morfismos encourages electronically submitted manuscripts prepared in Latex. Au-thors may retrieve the LATEX2ε macros used for Morfismos through the web sitehttp://www.math.cinvestav.mx, at “Revista Morfismos”, or by direct request to theMathematics Department of Cinvestav. The use of these macros will help in theproduction process and also to minimize publishing costs.

• All illustrations must be of professional quality.

• 15 offprints of each article will be provided free of charge.

• Manuscripts submitted for publication should be sent to Mrs. Laura Valencia, De-partamento de Matematicas del Cinvestav, Apartado Postal 14 - 740, Mexico, D.F.07000, or to the e-mail address: [email protected]

Lineamientos Editoriales

“Morfismos” es la revista semestral de los estudiantes del Departamento de Mate-

maticas del CINVESTAV, que tiene entre sus principales objetivos el que los estu-diantes adquieran experiencia en la escritura de resultados matematicos.

La publicacion de trabajos no estara restringida a estudiantes del CINVESTAV;deseamos fomentar tambien la participacion de estudiantes en Mexico y en el extran-

jero, ası como la contribucion por invitacion de investigadores.

Los reportes de investigacion matematica o resumenes de tesis de licenciatura,maestrıa o doctorado pueden ser publicados en Morfismos. Los artıculos que apare-

ceran seran originales, ya sea en los resultados o en los metodos. Para juzgar esto,el Consejo Editorial designara revisores de reconocido prestigio y con experiencia enla comunicacion clara de ideas y conceptos matematicos.

Aunque Morfismos es una revista con arbitraje, los trabajos se consid-

eraran como versiones preliminares que luego podran aparecer publicados

en otras revistas especializadas.

Si tienes alguna sugerencia sobre la revista hazlo saber a los editores y con gustoestudiaremos la posibilidad de implementarla. Esperamos que esta publicacion pro-picie, como una primera experiencia, el desarrollo de un estilo correcto de escribirmatematicas.

Morfismos

Editorial Guidelines

“Morfismos” is the journal of the students of the Mathematics Department of

CINVESTAV. One of its main objectives is for students to acquire experience inwriting mathematics. Morfismos appears twice a year.

Publication of papers is not restricted to students of CINVESTAV; we want to

encourage students in Mexico and abroad to submit papers. Mathematics researchreports or summaries of bachelor, master and Ph.D. theses will be considered forpublication, as well as invited contributed papers by researchers. Papers submitted

should be original, either in the results or in the methods. The Editors will assignas referees well–established mathematicians.

Even though Morfismos is a refereed journal, the papers will be con-

sidered as preliminary versions which could later appear in other mathe-

matical journals.

If you have any suggestions about the journal, let the Editors know and we willgladly study the possibility of implementing them. We expect this journal to foster, as

a preliminary experience, the development of a correct style of writing mathematics.

Morfismos

Contenido

Approximation of general optimization problems

Jorge Alvarez-Mena and Onesimo Hernandez-Lerma . . . . . . . . . . . . . . . . . . . . . . 1

Linear programming relaxations of the mixed postman problem

Francisco Javier Zaragoza Martınez . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

A nonmeasurable set as a union of a family of increasing well-ordered measur-able sets

Juan Gonzalez-Hernandez and Cesar E. Villarreal . . . . . . . . . . . . . . . . . . . . . . . 35

Noncooperative continuous-time Markov games

Hector Jasso-Fuentes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Morfismos, Vol. 9, No. 1, 2005, pp. 1–20

Approximation of general optimization problems∗

Jorge Alvarez-Mena Onesimo Hernandez-Lerma†

Abstract

This paper concerns the approximation of a general optimiza-tion problem (OP) for which the cost function and the constraintsare defined on a Hausdorff topological space. This degree of gen-erality allows us to consider OPs for which other approximationapproaches are not applicable. First we obtain convergence re-sults for a general OP, and then we present two applications ofthese results. The first application is to approximation schemesfor infinite-dimensional linear programs. The second is on the ap-proximation of the optimal value and the optimal solutions for theso-called general capacity problem in metric spaces.

2000 Mathematics Subject Classification: 90C48.Keywords and phrases: minimization problem, approximation, infinitelinear programs, general capacity problem.

1 Introduction

A constrained optimization problem (OP) is, in general, difficult tosolve in closed form, and so one is naturally led to consider ways toapproximate it. This in turn leads to obvious questions: how ”good”are the approximations? Do they ”converge” in some suitable sense?These are the questions studied in this paper for a general constrainedOP, where general means that the cost function and the constraintsare defined on a Hausdorff topological space. This degree of generality

∗Invited Article.†Partially supported by CONACyT grant 37355-3.

1

2 Jorge Alvarez–Mena and Onesimo Hernandez–Lerma

is important because then our results are applicable to large classesof OPs, even in infinite-dimensional spaces. For instance, as shown inSection 3, we can deal with approximation procedures for infinite linearprogramming problems in vector spaces with (dual) topologies whichare Hausdorff but, say, are not necessarily metrizable.

To be more specific, consider a general constrained (OP)

IP∞ : minimize f∞(x) : x ∈ F∞,

and a sequence of approximating problems

IPn : minimize fn(x) : x ∈ Fn.

(The notation is explained in section 2.) The questions we are interestedin are:

(i) the convergence of the sequence of optimal values min IPn —orsubsequences thereof— to min IP∞, and

(ii) the convergence of sequences of optimal solutions of IPn —orsubsequences thereof— to optimal solutions of IP∞.

We give conditions under which the convergence in (i) and (ii) holds—see Theorem 2.3. We also develop two applications of these results.The first one is on aggregation (of constraints) schemes to approximateinfinite-dimensional linear programs (l.p.’s). In the second applicationwe study the approximation of the optimal value and the optimal solu-tions for the so-called general capacity (GC) problem in metric spaces.

This paper is an extended version of [2] which presents the maintheoretical results concerning (i) and (ii), including of course detailedproofs. Here, we are mainly interested in the applications mentionedin the previous paragraph. The main motivation for this paper wasthat the convergence in (i) and (ii) is directly related to some of ourwork on stochastic control and Markov games [1, 3], but in fact generalOPs appear in many branches of mathematics, including probabilitytheory, numerical analysis, optimal control, game theory, mathematicaleconomics and operations research, to name just a few [4, 5, 12, 13, 14,15, 16, 17, 18, 19, 21, 22, 23].

The problem of finding conditions under which (i) and (ii) hold isof great interest, and it has been studied in many different settings —see e.g. [7, 8, 11, 14, 15, 16, 17, 19, 21, 22, 23] and their references. In

Approximation of general optimization problems 3

particular, the problem can be studied using the notion of Γ-convergence(or epi-convergence) of sequences of functionals [6, 10]. However, theapproach used in this paper is more direct and generalizes several knownresults [6, 10, 11, 19, 21, 22] —see Remark 2.4. Even more, Example 4.7shows that our assumptions are strictly weaker than those considered inthe latter references. Namely, in Example 4.7 we study a particular GCproblem in which our assumptions are satisfied, but the assumptionsconsidered in those references fail to hold. The GC problem has beenpreviously analyzed in e.g. [4, 5, 13] from different viewpoints.

The remainder of the paper is organized as follows. In section 2we present our main results on the convergence and approximation ofgeneral OPs. These results are applied in section 3 to the aggregationschemes introduced in [15] to approximate infinite l.p.’s. In section 4our results are applied to the GC problem, and a particular case of theGC problem is analyzed.

2 Convergence of general OPs

We shall use the notation IN := 1, 2, . . ., IN := IN ∪ ∞ andIR := IR ∪ ∞,−∞.

Let X be a Hausdorff topological space. For each n ∈ IN, consider afunction fn : X → IR, a set Fn ⊂ X , and the optimization problem

IPn : Minimize fn(x)

subject to : x ∈ Fn.

We call Fn the set of feasible solutions for IPn. If Fn is nonempty, the(optimum) value of IPn is defined as inf IPn := inffn(x) | x ∈ Fn;otherwise, inf IPn := +∞. The problem IPn is said to be solvable ifthere is a feasible solution x∗ that achieves the optimum value. In thiscase, x∗ is called an optimal solution for IPn, and the value inf IPn is thenwritten as min IPn = fn(x∗). We shall denote by Mn the minimum set,that is, the set of optimal solutions for IPn.

To state our assumptions we will use Kuratowski’s [20] concept ofouter and inner limits of Fn, denoted by OLFn and ILFn, re-


spectively, and defined as follows.

OLFn := x ∈ X | x = limi→∞ xni, where ni ⊂ IN

is an increasing sequence such that xni∈ Fni

for all i.

Thus a point x ∈ X is in OLFn if x is an accumulation point of asequence xn with xn ∈ Fn for all n. On the other hand, if x is thelimit of the sequence xn itself, then x is in the inner limit ILFn,i.e.

ILFn := x ∈ X | x = limn→∞ xn,

where xn ∈ Fn for all but a finite number of n′s.

In these definitions we may, of course, replace Fn with any othersequence of subsets of X . Also note that IL· ⊂ OL·.

We shall consider two sets of hypotheses.

Assumption 2.1

(a) The minimum sets Mn satisfy that

(1) OLMn ⊂ F∞.

(b) If xniis in Mni

for all i and xni→ x (so that x is in OLMn),

then

(2) lim infi→∞

fni(xni

) ≥ f∞(x).

(c) For each x ∈ F∞ there exist N ∈ IN and a sequence xn withxn ∈ Fn for all n ≥ N , and such that xn → x and lim

n→∞fn(xn) =

f∞(x).

Assumption 2.2 Parts (b) and (c) are the same as in Assumption 2.1.Moreover

(a) The minimum sets Mn satisfy that

(3) ILMn ⊂ F∞.


Note that Assumption 2.1(c) implies, in particular, F∞ ⊂ ILFn,but the equality does not hold necessarily. In fact, in section 4 we givean example in which Assumption 2.1 is satisfied, in particular F∞ ⊂ILFn, but F∞ = ILFn (see Example 4.7).

On the other hand, note that Assumptions 2.2 (a),(c) yield that

ILMn ⊂ F∞ ⊂ ILFn.

Theorem 2.3 (a) If Assumption 2.1 holds, then

(4) OLMn ⊂ M∞.

In other words, if xn is a sequence of minimizers of IPn, and asubsequence xni

of xn converges to x ∈ X, then x is optimal forIP∞. Furthermore, the optimal values of IPni

converge to the optimalvalue of IP∞, that is,

(5) min IPni= fni

(xni) → f∞(x) = min IP∞.

(b) Suppose that Assumption 2.2 holds. Then

ILMn ⊂ M∞.

If in addition ILMn is nonempty, then

(6) min IPn → min IP∞.

Proof: We only prove (a) because the proof of (b) is quite similar.

To prove (a), let x ∈ X be in the outer limit OLMn. Then thereis a sequence ni ⊂ IN and xni

∈ Mnifor all i such that

(7) xni→ x.

Moreover, by Assumption 2.1(a), x is in F∞. To prove that x is in M∞,choose an arbitrary x′ ∈ F∞ and let x′n and N be as in Assumption2.1(c) for x′, that is, x′n is in Fn for all n ≥ N , x′n → x′, and fn(x′n) →f∞(x′). Furthermore, if ni ⊂ IN is as in (7), then the subsequencex′ni

of x′n also satisfies

(8) x′niis in Fni

, x′ni→ x′, and fni

(x′ni) → f∞(x′).


Combining the latter fact with Assumption 2.1(b) and the optimalityof each xni

we get

f∞(x) ≤ lim infi→∞

fni(xni

) (by (2))

≤ lim infi→∞

fni(x′ni

)

= f∞(x′) (by (8)).

Hence, as x′ ∈ F∞ was arbitrary, it follows that x is in M∞, that is,(4) holds.

To prove (5), suppose again that x is in OLMn and let xni∈ Mni

be as in (7). By Assumption 2.1(c), there exists a sequence x′ni∈ Fni

that satisfies (8) for x instead of x′; thus

f∞(x) ≤ lim infi→∞

fni(xni

) (by (2))

≤ lim supi→∞

fni(xni

)

≤ lim supi→∞

fni(x′ni

)

= f∞(x) (by (8)).

This proves (5).

In Theorem 2.3 it is not assumed the solvability of each IPn. ThusOLMn might be empty; in fact, it might be empty even if eachIPn is solvable. In this case, the (convergence of minimizers) inclusion(4) trivially holds. In the convergence of the optimal values (5) and(6), unlike the convergence of minimizers, it is implicitly assumed thatOLMn is nonempty.

Remark 2.4 (i) Parts (a) and (b) of Theorem 2.3 generalize in par-ticular some results in [21, 22] and [11, 19], respectively. Indeed, usingour notation, in [11, 19, 21, 22] it is assumed that the cost functions fnare continuous and converge uniformly to f∞. On the other hand, withrespect to the feasible sets Fn, in [11] it is assumed that ILFn = F∞,whereas in [19, 21, 22] it is required that Fn → F∞ in the Hausdorffmetric. These hypotheses trivially yield the following conditions:

(C1) The inner and/or the outer limit of the feasible sets Fn coincidewith F∞, i.e.

(9) ILFn = F∞


or

(10) OLFn = ILFn = F∞.

(C2) For every x in X and for every sequence xn in X converging tox, it holds that

(11) limn→∞

fn(xn) = f∞(x).

However, instead of (10) and (11) we require (the weaker) Assump-tion 2.1, and instead of (9) and (11) we require (the weaker) Assumption2.2.

(ii) Theorem 2.3 generalizes the results in [6, 10], where it is usedthe notion of Γ-convergence. Indeed, [6, 10] study problems of the form

(12) minx∈X

Fn(x).

Each of our problems IPn can be put in the form (12) by letting

Fn(x) :=

!fn(x) if x ∈ Fn,∞ if x /∈ Fn,

and then, when the space X is first countable, the assumptions in [6,10] can be translated to this context as follows: the sequence Fn Γ-converges to F∞ —see Theorems 7.8 and 7.18 in [10]. On the otherhand, when X is first countable, the sequence Fn Γ-converges to F∞

if and only if

(C3) For every x in X and for every sequence xn in X converging tox, it holds that

lim infn→∞

Fn(xn) ≥ F∞(x).

(C4) For every x in X there exists a sequence xn in X converging tox such that

limn→∞

Fn(xn) = F∞(x).

See Proposition 8.1 in [10]. It is natural to assume that fn(x) < ∞for each x ∈ Fn and n ∈ IN, and that F∞ is nonempty. In this case,(C3) implies part (b) of Assumptions 2.1 and 2.2, (C4) implies part (c),and (C3) together with (C4) imply part (a). Indeed, the last statement


can be proved as follows. Let x ∈ X be in OLMn. Then there is asequence ni ⊂ IN and xni

∈ Mnifor all i such that xni

→ x. Now,as F∞ is nonempty we can take x′ in F∞. For this x′, let x′n ∈ X beas in (C4), so that

F∞(x) ≤ lim infi→∞

Fni(xni

) (by (C3))

≤ lim infi→∞

Fni(x′ni

) (because xniis in Mni

)

= limi→∞

Fn(x′n) (by (C4))

= F∞(x′) < ∞ (by (C4)).

Hence x is in F∞.

On the other hand, if in addition we assume that fn(x) ≤ K for allx ∈ Fn, n ∈ IN and someK ∈ IR, then (C3) and (C4) imply (10). In fact,(C4) implies the inclusion F∞ ⊂ ILFn ⊂ OLFn, and (C3) togetherwith the uniform boundedness condition imply the reverse inclusion.

In the next two sections we present applications of Theorem 2.3. Wealso show, in Example 4.7, a particular problem in which Assumption2.1 is satisfied, but the assumptions considered in [6, 10, 11, 19, 21, 22]do not hold.

3 Approximation schemes for l.p.’s

As a first application of Theorem 2.3, in this section we consider theaggregation (of constraints) schemes introduced in [15] to approximateinfinite linear programs (l.p.’s). (See also [17] or chapter 12 in [16]for applications of the aggregation schemes to some stochastic controlproblems.) Our main objective is to show that the convergence of theseschemes can be obtained from Theorem 2.3.

First we introduce the l.p. we shall work with. Let (X ,Y) and(Z,W) be two dual pairs of vector spaces. The spaces X and Y areassumed to be endowed with the weak topologies σ(X ,Y) and σ(Y,X ),respectively. Thus, in particular, the topological spaces X and Y areHausdorff. We denote by ⟨·, ·⟩ the bilinear form on both X × Y andZ ×W.

Let A : X → Z be a weakly continuous linear map with adjoint


A∗ : W → Y, i.e.

⟨x,A∗w⟩ := ⟨Ax,w⟩ ∀ x ∈ X , w ∈ W.

We denote by K a positive cone in X . For given vectors c ∈ Y andb ∈ Z, we consider the (primal) l.p.

LP : Minimize ⟨x, c⟩(13)

subject to: Ax = b, x ∈ K.(14)

A vector x ∈ X is said to be a feasible solution for LP if it satisfies(14), and we denote by F the set of feasible solutions for LP. Theprogram LP is called consistent if it has a feasible solution, i.e. F isnonempty.

The following assumption ensures that LP is solvable.

Assumption 3.1 LP has a feasible solution x0 with ⟨x0, c⟩ > 0 and,moreover, the set

∆0 := x ∈ K|⟨x, c⟩ ≤ ⟨x0, c⟩

is weakly sequentially compact.

Remark 3.2 Assumption 3.1 implies that the set ∆r := x ∈ K|⟨x, c⟩ ≤ r is weakly sequentially compact for every r > 0, since ∆r =(r/⟨x0, c⟩)∆0.

Lemma 3.3 If Assumption 3.1 holds, then LP is solvable.

For a proof of Lemma 3.3, see Theorem 2.1 in [15].

If E is a subset of a vector space, then sp(E) denotes the spacespanned (or generated) by E.

Aggregation schemes. To realize the aggregation schemes themain assumption is on the vector space W.

Assumption 3.4 There is an increasing sequence of finite sets Wn inW such that W∞ := ∪∞

n=1Wn is weakly dense in W, where Wn =sp(Wn).


For each n ∈ IN, let Zn be the algebraic dual of Wn, that is, Zn :=f : Wn → IR | f is a linear functional. Thus (Zn,Wn) is a dual pairof finite-dimensional vector spaces with the natural bilinear form

⟨f,w⟩ := f(w) ∀ w ∈ Wn, f ∈ Zn.

Now let An : X → Zn be the linear operator given by

(15) Anx(w) := ⟨Ax,w⟩ ∀ w ∈ Wn.

The adjoint A∗n : Wn → Y of An is the adjoint A∗ of A restricted to Wn,

that is, A∗n := A∗|Wn

. Finally, we define bn ∈ Zn by bn(·) := ⟨b, ·⟩|Wn.

With these elements we can define the aggregation schemes as fol-lows. For each n ∈ IN,

LPn : Minimize ⟨x, c⟩

subject to : Anx = bn, x ∈ K,(16)

which is as our problem IPn (in section 2) with fn(x) := ⟨x, c⟩ and Fn

the set of vectors x ∈ X that satisfy (16). The l.p. LPn is called anaggregation (of constraints) of LP. Moreover, from Proposition 2.2 in[15] we have the following.

Lemma 3.5 Under the Assumptions 3.1 and 3.4, the l.p. LP∞ isequivalent to LP in the sense that (using Lemma 3.3)

(17) minLP = minLP∞.

The following lemma provides the connection between the aggrega-tion schemes and Theorem 2.3.

Lemma 3.6 The Assumptions 3.1 and 3.4 imply that the aggregationschemes LPn satisfy Assumption 2.1.

Proof: To check parts (a) and (c) of Assumption 2.1, for each n ∈ INlet xn ∈ Fn be such that xn → x weakly in X . Thus, by definition of theweak topology on X , ⟨xn, y⟩ → ⟨x, y⟩ for all y ∈ Y, which in particularyields

limn→∞

⟨xn, c⟩ = ⟨x, c⟩.


This implies part (b) of Assumption 2.1, and also that the sequence xnis in the weakly sequentially compact set ∆r for some r > 0 (see Remark3.2). In particular, x is in K, and from (16) and the definitions of An

and bn we get

A∞x(w) = limn→∞

Anxn(w) = limn→∞

bn(w) = b∞(w) ∀ w ∈ W∞.

Thus x is in F∞, which yields that OLFn ⊂ F∞, and so Assumption2.1(a) follows.

Finally, to verify part (c) of Assumption 2.1, choose an arbitraryx ∈ F∞. Then, by (15) and the definition of bn,

A∞x(w) = ⟨Ax,w⟩ = ⟨b, w⟩ = b∞(w) ∀ w ∈ W∞.

In particular, if w ∈ Wn for some n ∈ IN, the latter equation becomes

Anx(w) = bn(w).

Hence Anx = bn. It follows that F∞ ⊂ Fn for all n ∈ IN and, moreover,the sets Fn form a nonincreasing sequence, i.e.

(18) Fn ⊇ Fn+1 ∀ n ∈ IN,

which implies part (c) of Assumption 2.1.

To summarize, from Lemma 3.6 and Theorem 2.3, together with(17) and (18) we get the following.

Theorem 3.7 Suppose that Assumptions 3.1 and 3.4 are satisfied.Then

(a) The aggregation LPn is solvable for every n ∈ IN.

(b) For every n ∈ IN, let xn ∈ Fn be an optimal solution for LPn.Then, as n → ∞,

(19) ⟨xn, c⟩ ↑ minLP∞ = minLP,

and, furthermore, every weak accumulation point of the sequencexn is an optimal solution for LP.


Proof: (a) It is clear that Assumption 3.1 also holds for each aggre-gation LPn. Thus the solvability of LPn follows from Lemma 3.3.

(b) From Lemma 3.6, we see that Theorem 2.3(a) holds for theaggregations LPn. Hence to complete the proof we only need to verify(19). To do this, note that (18) yields minLPn ≤ minLPn+1 for eachn ∈ IN, and, moreover, the sequence of values minLPn is bounded aboveby minLP∞. This fact together with (5) and Lemma 3.5 give (19).

Theorem 3.7 was obtained in [15] using a different approach.

Remark 3.8 In the aggregation schemes LPn, the vector spaces Zn

and Wn are finite-dimensional for n ∈ IN, and so each LPn is a so-called semi-infinite l.p. Hence Theorem 3.7 can be seen as a result on theapproximation of the infinite-dimensional l.p. LP by semi-infinite l.p.’s.On the other hand, a particular semi-infinite l.p. is when the vectorspace X of decision variables (or just the cone K) is finite-dimensional,but the vector b lies in an infinite-dimensional space W [5, 14, 18]. Inthe latter case, the aggregation schemes would be approximations toLP by finite l.p.’s.

4 The GC problem

The general capacity (GC) problem is related to the problem ofdetermining the electrostatic capacity of a conducting body. In fact,it originated in the mentioned electrostatic capacity problem —see, forinstance, [4, 5].

Let X and Y be metric spaces endowed with their correspondingBorel σ-algebras B(X) and B(Y ). We denote by M(Y ) the vector spaceof finite signed measures on Y , and by M+(Y ) the cone of nonnegativemeasures in M(Y ).

Now let b : X → IR, c : Y → IR, and g : X × Y → IR be nonnegativeBorel-measurable functions. Then the GC problem can be stated asfollows.

GC: Minimize

!

Y

c(y) µ(dy)

subject to :

!

Y

g(x, y) µ(dy) ≥ b(x) ∀x ∈ X, µ ∈ M+(Y ).


In this section we study the convergence problem (see (i) and (ii) insection 1) in which g and c are replaced with sequences of nonnegativemeasurable functions gn : X×Y → IR and cn : Y → IR, for n ∈ IN, suchthat gn → g∞ =: g and cn → c∞ =: c uniformly.

Thus we shall deal with GC problems

GCn : Minimize

!

Y

cn(y) µ(dy)

subject to :

!

Y

gn(x, y) µ(dy) ≥ b(x) ∀x ∈ X, µ ∈ M+(Y ),(20)

for n ∈ IN. For each n ∈ IN, we denote by Fn the set of feasible solutionsfor GCn, that is, the set of measures µ that satisfy (20), but in addition"Ycn dµ < ∞.

Convergence. We shall study the convergence issue via Theorem2.3. First, we introduce assumptions that guarantee the solvability ofthe GC problems. We shall distinguish two cases for the cost functionscn, the bounded case and the unbounded case, which require slightlydifferent hypotheses. For the bounded case we suppose the following.

Assumption 4.1 (Bounded case) For each n ∈ IN:

(a) Fn is nonempty.

(b) The function gn(x, ·) is bounded above and upper semicontinuous(u.s.c.) for each x ∈ X.

(c) The function cn is bounded and lower semicontinuous (l.s.c.). Fur-ther cn is bounded away from zero, that is, there exist δn > 0 suchthat cn(y) ≥ δn for all y ∈ Y .

In addition,

(d) The space Y is compact.

For the unbounded case, we replace parts (c) and (d) with an inf-compactness hypothesis.

Assumption 4.2 (Unbounded case) Parts (a)-(c) are the same as inAssumption 4.1. Moreover,


(d) For each n ∈ IN, the function cn is inf-compact, which means that,for each r ∈ IR, the set y ∈ Y |cn(y) ≤ r is compact. Further, cnis bounded away from zero.

Observe that the inf-compactness condition implies that cn is l.s.c.

We next introduce the assumptions for our convergence and approxi-mation results. As above, we require two sets of assumptions dependingon whether the cost functions cn are bounded or unbounded. (See Re-mark 4.8 for alternative sets of assumptions.)

Assumption 4.3 (Bounded case)

(a) (Slater condition) There exist µ ∈ F∞ and η > 0 such that

!

Y

g∞(x, y) µ(dy) ≥ b(x) + η ∀ x ∈ X.

(b) gn → g∞ uniformly on X × Y .

(c) cn → c∞ uniformly on Y .

Assumption 4.4 (Unbounded case) Parts (a) and (b) are the same asin Assumption 4.3. Moreover

(c) cn ↓ c∞ uniformly on Y .

Before stating our main result for the GC problem we recall somefacts on the weak convergence of measures (for further details see [9] orchapter 12 in [16], for instance).

Definition 4.5 Let Y , M(Y ) and M+(Y ) be as at the beginning ofthis section. A sequence µn in M+(Y ) is said to be bounded if thereexists a constant m such that µn ≤ m for all n. Let Cb(Y ) be the vectorspace of continuous bounded functions on Y . We say that a sequenceµn in M(Y ) converges weakly to µ ∈ M(Y ) if µn → µ in the weaktopology σ(M(Y ), Cb(Y )), i.e.

!

Y

u dµn →

!

Y

u dµ ∀u ∈ Cb(Y ).


A subsetM0 ofM+(Y ) is said to be relatively compact if for any sequenceµn in M0 there is a subsequence µm of µn and a measure µ inM+(Y ) (but not necessarily in M0) such that µn → µ weakly. In thelatter case, we say that µ is a weak accumulation point of µn.

We now state our main result in this section.

Theorem 4.6 Suppose that either Assumptions 4.1 and 4.3, or 4.2 and4.4 hold. Then

(a) GCn is solvable for every n ∈ IN.

(b) The optimal value of GCn converges to the optimal value of GC∞,i.e.

(21) minGCn −→ minGC∞.

Furthermore, if µn ∈ M+(Y ) is an optimal solution for GCn foreach n ∈ IN, then the sequence µn is relatively compact, andevery weak accumulation point of µn is an optimal solution forGC∞.

(c) If GC∞ has a unique optimal solution, say µ, then for any µn inthe set of optimal solutions for GCn, with n ∈ IN, the sequenceµn converges weakly to µ.

For a proof of Theorem 4.6 the reader is referred to [1].

We shall conclude this section with an example which satisfies ourhypotheses, Assumption 2.1, but the hypotheses used in [6, 10, 11, 19,21, 22] do not hold.

Example 4.7 This example shows a particular GC problem in the un-bounded case, in which our assumptions are satisfied, but the condi-tions used in [6, 10, 11, 19, 21, 22] do not hold. Hence, this exampleshows that our assumptions are strictly weaker than those considered in[6, 10, 11, 19, 21, 22].

We first compare our hypotheses with those in [11, 19, 21, 22]. Inour notation, the latter conditions are as follows (see Remark 2.4).


(i) For each sequence µn ∈ M+(Y ) such that µn → µ weakly we have

limn→∞

∫

Y

cn dµn =

∫

Y

c∞ dµ.

(ii) ILFn = F∞.

Consider the spaces X = Y = [0, 2], and for each n ∈ IN let gn ≡ 1, and

cn(x) :=

⎧⎨

⎩

1 if x = 0,1x

if x ∈ (0, 1],1 if x ∈ (1, 2].

Let b ≡ 0. With these elements the set Fn of feasible solutions for eachproblem GCn is given by

Fn := µ ∈ M+([0, 2]) :

∫gn dµ = µ([0, 2]) ≥ b,

∫cn dµ < ∞.

As the cost functions cn are unbounded, we consider the Assumptions4.2 and 4.4, which are obviously true in the present case, and which inturn imply Assumption 2.1 —see Lemma 3.11 in [1]. Next we show that(i) and (ii) do not hold.

Let µ be the lebesgue measure on Y = [0, 2], and for each n ∈ IN letµn be the restriction of µ to [1/n, 2], i.e. µn(B) := µ(B ∩ [1/n, 2]) forall B ∈ B(Y ). Thus µn is in Fn for each n ≥ 2, and µn → µ weakly.Therefore µ is in ILFn, but µ is not in F∞ because

∫c∞ dµ = ∞.

Hence (ii) does not hold.

Similarly, let µ′n := (1/kn)µn with kn := 1+ ln(n). Then µ′

n is in Fn

for all n ≥ 2, and µ′n → 0 =: µ′ weakly, but

∫cn dµ′

n = 1 −→∫

c∞ dµ′ = 0.

Thus (i) is not satisfied.

Now we compare our assumptions with those in [6, 10]. This can bedone because, as M([0, 2]) is metrizable, the space X = M([0, 2]) is firstcountable. See Remark 2.4.

We take X,Y, cn, gn and Fn as above, but now we take b = 1/2. Asin the former case, Assumption 2.1 holds. Now we slightly modify theset of feasible solutions by

Fn := µ ∈ M+([0, 2]) :

∫gn dµ = µ([0, 2]) ≥

1

2,

∫cn dµ < 1.


Notice that !Fn = ∅ and !Fn ⊂ Fn for all n ≥ 2. The sequence of mod-ified GC problems, say "GCn, also satisfies Assumption 2.1. Indeed,parts (a) and (b) of Assumption 2.1 hold because the minimum sets

have not changed (Mn = #Mn), and part (c) is true since it holds forF∞, and !F∞ ⊂ F∞. Next we show that condition (C3) in Remark 2.4does not hold. For each n ∈ IN, let

Fn(µ) :=

$ %cndµ if µ ∈ !Fn,

∞ if µ /∈ !Fn.

Moreover, for each n ∈ IN, let µ′′n be the restriction of µ to [1 + 1/n, 2],

and let µ′′ be the restriction of µ to [1, 2]. Hence we have µ′′n([0, 2]) =

(n− 1)/n ≥ 1/2 and%cndµ′′

n = (n− 1)/n < 1 for all n ≥ 2. Therefore,

µ′′n is in !Fn for each n ≥ 2, and µ′′

n → µ′′ weakly. Thus µ′′ is in IL !Fn,but µ′′ is not in !F∞ because

%c∞ dµ′′ = 1. Hence

lim infn→∞

Fn(µ′′n) = lim inf

n→∞

&cndµ

′′n = 1 < ∞ = F∞(µ′′),

and so (C3) is not satisfied. It follows that the Fn do not Γ−convergeto F∞, that is, the assumptions in [6, 10] do not hold.

Remark 4.8 Suppose that cn → c∞ uniformly on Y . Then the follow-ing holds.

• If c∞ is bounded away from zero, then so is cn for all n sufficientlylarge. Hence, in part (b) of Theorem 4.6 it suffices to require (only)that c∞ is bounded away from zero, for both cases, bounded andunbounded.

• If the sequence cn is uniformly bounded away from zero (thatis, there exists δ > 0 such that, for each n ∈ IN, cn(y) ≥ δ for ally ∈ Y ), then also c∞ is bounded away from zero.

On the other hand, if cn → c∞ uniformly and gn → g∞ uniformly, thenthe following holds.

• If the Slater condition holds for GC∞ (see Assumption 4.3 (a)),then GCn also satisfies the Slater condition for all n large enough.It follows that, for each n ≥ N , GCn is consistent, i.e., Fn = ∅.Then Assumption 4.3 imply part (a) of Assumptions 4.1 and 4.2,for each n ≥ N . Hence, in part (b) of Theorem 4.6, Assumptions4.1(a) and 4.2(a) are not required.


• If the Slater condition uniformly holds for the sequence GCn(that is, for each n ∈ IN, there exist µn ∈ F∞ and η > 0 such that

!

Y

gn(x, y) µn(dy) ≥ b(x) + η ∀ x ∈ X),

then the Slater condition also holds for GC∞.

Jorge Alvarez–Mena 1

Programa de Investigacion enMatematicas Aplicadas yComputacion, IMP,

A.P. 14–805,Mexico, D.F. 07730Mexico

[email protected]

Onesimo Hernandez-Lerma

Departamento de Matematicas,CINVESTAV-IPN,A.P. 14-470,

Mexico D.F. 07000,[email protected]

References

[1] Alvarez-Mena J.; Hernandez-Lerma O., Convergence of the optimalvalues of constrained Markov control processes, Math. Meth. Oper.Res. 55 (2002), 461–484.

[2] Alvarez-Mena J.; Hernandez-Lerma O., Convergence and approxi-mation of optimization problems, SIAM J. Optim. 15 (2005), 527–539.

[3] Alvarez-Mena J.; Hernandez-Lerma O., Existence of Nash equili-bria for constrained stochastic games, Math. Meth. Oper. Res. 62(2005).

[4] Anderson E. J.; Lewis A. S.; Wu S. Y., The capacity problem,Optimization 20 (1989), 725–742.

[5] Anderson E. J.; Nash P., Linear Programming in Infinite-Dimensional Spaces, Wiley, Chichester, U. K., 1987.

[6] Attouch H., Variational Convergence for Functions and Operators,Applicable Mathematics series, Pitman (Advanced publishing Pro-gram), Boston, MA, 1984.

1Current address: Departamento de Ciencias Basicas, UAT, Apartado Postal 140,Apizaco, Tlaxcala 90300, Mexico.


[7] Back, K., Convergence of Lagrange multipliers and dual variablesfor convex optimization problems, Math. Oper. Res. 13 (1988), 74–79.

[8] Balayadi A.; Sonntag Y.; Zalinescu C., Stability of constrained opti-mization problems, Nonlinear Analysis, Theory Methods Appl. 28(1997), 1395–1409.

[9] Billingsley P., Convergence of Probability Measures, Wiley, NewYork, 1968.

[10] Dal Maso G., An Introduction to Γ-convergence, Birkhauser,Boston, MA, 1993.

[11] Dantzig G. B.; Folkman J.; Shapiro N., On the continuity of theminimum set of a continuous function, J. Math. Anal. Appl. 17(1967), 519–548.

[12] Dontchev A. L.; Zolezzi T., Well-Posed Optimization Problems,Lecture Notes in Math. 1543, Springer-Verlag, Berlin, 1993.

[13] Gabriel J. R.; Hernandez-Lerma O., Strong duality of the gen-eral capacity problem in metric spaces, Math. Meth. Oper. Res.53 (2001), 25–34.

[14] Goberna M. A.; Lopez M. A., Linear Semi-Infinite Optimization,Wiley, New York, 1998.

[15] Hernandez-Lerma O.; Lasserre J. B., Approximation schemes forinfinite linear programs, SIAM J. Optim. 8 (1998), 973–988.

[16] Hernandez-Lerma O.; Lasserre J. B., Further Topics on Discrete-Time Markov Control Processes, Springer-Verlag, New York, 1999.

[17] Hernandez-Lerma O.; Lasserre J. B., Linear programming ap-proximations for Markov control processes in metric spaces, ActaAppl. Math. 51 (1998), 123–139.

[18] Hettich R.; Kortanek K. O., Semi-infinite programming: theory,methods, and applications, SIAM Review 35 (1993), 380–429.

[19] Kanniappan P.; Sundaram M. A., Uniform convergence of convexoptimization problems, J. Math. Anal. Appl. 96 (1983), 1–12.

[20] Kuratowski K., Topology I, Academic Press, New York, 1966.


[21] Schochetman I. E., Convergence of selections with applications inoptimization, J. Math. Anal. Appl. 155 (1991), 278–292.

[22] Schochetman I. E., Pointwise versions of the maximum theoremwith applications in optimization, Appl. Math. Lett. 3 (1990), 89–92.

[23] Vershik A. M.; Telmel’t V., Some questions concerning the approx-imation of the optimal values of infinite-dimensional problems inlinear programing, Siberian Math. J. 9 (1968), 591–601.

Morfismos, Vol. 9, No. 1, 2005, pp. 21-34

Linear programming relaxationsof the mixed postman problem

Francisco Javier Zaragoza Martınez 1

Abstract

The mixed postman problem consists of finding a minimum costtour of a connected mixed graph traversing all its vertices, edges,and arcs at least once. We prove in two different ways that the lin-ear programming relaxations of two well-known integer program-ming formulations of this problem are equivalent. We also givesome properties of the extreme points of the polyhedra defined byone of these relaxations and its linear programming dual.

2000 Mathematics Subject Classification: 05C45, 90C35.Keywords and phrases: Eulerian graph, integer programming formula-tion, linear programming relaxation, mixed graph, postman problem.

1 Introduction

We study a class of problems collectively known as postman problems [6].As the name indicates, these are the problems faced by a postman whoneeds to deliver mail to all streets in a city, starting and ending hislabour at the city’s post office, and minimizing the length of his walk.In graph theoretical terms, a postman problem consists of finding aminimum cost tour of a graph traversing all its arcs (one-way streets)and edges (two-way streets) at least once. Hence, we can see postmanproblems as generalizations of Eulerian problems.

The postman problem when all streets are one-way, known as the di-rected postman problem, can be solved in polynomial time by a network

1This work was partly funded by UAM Azcapotzalco research grant 2270314 andCONACyT doctoral grant 69234.

21

22 Francisco Javier Zaragoza Martınez

flow algorithm, and the postman problem when all streets are two-way,known as the undirected postman problem, can be solved in polynomialtime using Edmonds’ matching algorithm, as shown by Edmonds andJohnson [3]. However, Papadimitriou showed that the postman problembecomes NP-hard when both kinds of streets exist [9]. This problem,known as the mixed postman problem, is the central topic of this paper.

We study some properties of the linear programming relaxations oftwo well-known integer programming formulations for the mixed post-man problem — described in Section 3. We prove that these linearprogramming relaxations are equivalent (Theorem 4.1.1). In particular,we show that the polyhedron defined by one of them is essentially aprojection of the other (Theorem 4.1.2). We also give new proofs of thehalf-integrality of one of these two polyhedra (Theorem 4.2.1) and ofthe integrality of the same polyhedron for mixed graphs with vertices ofeven degree (Theorem 4.2.2). Finally, we prove that the correspondingdual polyhedron has integral optimal solutions (Theorem 4.3.1).

2 Preliminaries

A mixed graph M is an ordered triple (V (M), E(M), A(M)) of threemutually disjoint sets V (M) of vertices, E(M) of edges, and A(M) ofarcs. When it is clear from the context, we simply write M = (V,E,A).Each edge e ∈ E has two ends u, v ∈ V , and each arc a ∈ A has a head

u ∈ V and a tail v ∈ V . Each edge can be traversed from one of itsends to the other, while each arc can be traversed from its tail to itshead. The associated directed graph M = (V,A∪E+ ∪E−) of M is thedirected graph obtained from M by replacing each edge e ∈ E with twooppositely oriented arcs e+ ∈ E+ and e− ∈ E−.

Let S ⊆ V . The undirected cut δE(S) determined by S is the set ofedges with one end in S and the other end in S = V \S. The directed cut

δA(S) determined by S is the set of arcs with tails in S and heads in S.The total cut δM (S) determined by S is the set δE(S) ∪ δA(S) ∪ δA(S).For single vertices v ∈ V (M) we write δE(v), δA(v), δM (v) instead ofδE(v), δA(v), δM (v), respectively. We also define the degree of Sas dE(S) = |δE(S)|, and the total degree of S as dM (S) = |δM (S)|.

A walk from v0 to vn is an ordered tuple W = (v0, e1, v1, . . . , en, vn)on V ∪ E ∪ A such that, for all 1 ≤ i ≤ n, ei can be traversed fromvi−1 to vi. If v0 = vn, W is said to be a closed walk. If, for any twovertices u and v, there is a walk from u to v, we say that M is strongly

Linear relaxations of the mixed postman problem 23

connected. If W is closed and uses all vertices of M , we call it a tour,and if it traverses each edge and arc exactly once, we call it Eulerian.If e1, . . . , en are pairwise distinct, W is called a trail. If W is a closedtrail, and v1, . . . , vn are pairwise distinct, we call it a cycle.

Given a matrix A ∈ Qn×m and a vector b ∈ Qn, the polyhedron

determined by A and b is the set P = x ∈ Rm : Ax ≤ b. A vectorx ∈ P is called an extreme point of P if x is not a convex combinationof vectors in P \ x. For our purposes, P is integral if all its extremepoints have integer coordinates, and it is half-integral if all its extremepoints have coordinates which are integer multiples of 1

2 .Let S be a set, and let T ⊆ S. If x ∈ RS, we define x(T ) =

!

t∈T xt.The characteristic vector χT of T with respect to S is defined by theentries χT (t) = 1 if t ∈ T , and χT (t) = 0 otherwise. If T = S we write1S or 1 instead of χS , if T consists of only one element t we write 1tinstead of χt, and if T is empty we write 0S or 0 instead of χ∅. Ifx ∈ Rn, the positive support of x is the vector y ∈ Rn such that yi = 1if xi > 0, and yi = 0 otherwise, and it is denoted by supp+(x). Thenegative support supp−(x) is defined similarly.

3 Integer programming formulations

Let M = (V,E,A) be a strongly connected mixed graph, and let c ∈QE∪A

+ . A postman tour of M is a tour that traverses all edges and arcsof M at least once. The cost of a postman tour is the sum of the costs ofall edges and arcs traversed, counting repetitions. The mixed postman

problem is to find the minimum cost of a postman tour. We present twointeger programming formulations of the mixed postman problem.

3.1 First formulation

The first integer programming formulation we give is due to Kappaufand Koehler [7], and Christofides et al. [1]. Similar formulations weregiven by other authors [3, 5, 10]. All these formulations are based onthe following characterization of mixed Eulerian graphs.

Theorem 3.1.1 (Veblen [11]) A connected, mixed graph M is Eulerian

if and only if M is the disjoint union of some cycles.

Let M = (V,A ∪ E+ ∪ E−) be the associated directed graph of M .For every e ∈ E, let ce+ = ce− = ce. A nonnegative integer circulation


x of M (a vector on A∪E+ ∪E− such that x(δ(v)) = x(δ(v)) for everyv ∈ V , for more on the theory of flows see [4]) is the incidence vector of apostman tour of M if and only if xe ≥ 1 for all e ∈ A, and xe+ +xe− ≥ 1for all e ∈ E. Therefore, we obtain the integer program:

MMPT1(M, c) = min c⊤AxA + c⊤Ex+E+ c⊤Ex

−E

(1)

subject to

x(δ(v))− x(δ(v)) = 0 for all v ∈ V,(2)

xa ≥ 1 for all a ∈ A,(3)

xe+ + xe− ≥ 1 for all e ∈ E, and(4)

xa ≥ 0 and integer for all a ∈ A ∪E+ ∪ E−.(5)

Let P1MPT (M) be the convex hull of the feasible solutions to the

integer program above, and let Q1MPT (M) be the set of feasible solutions

to its linear programming relaxation:

LMMPT1(M, c) = min c⊤AxA + c⊤Ex+E+ c⊤Ex

−E

(6)

subject to

x(δ(v))− x(δ(v)) = 0 for all v ∈ V,(7)

xa ≥ 1 for all a ∈ A,(8)

xe+ + xe− ≥ 1 for all e ∈ E, and(9)

xa ≥ 0 for all a ∈ A ∪ E+ ∪ E−.(10)

3.2 Second formulation

The second integer programming formulation we give is due to Nobertand Picard [8]. The approach they use is based on the following char-acterization of mixed Eulerian graphs.

Theorem 3.2.1 (Ford and Fulkerson [4, page 60]) Let M be a con-

nected, mixed graph. Then M is Eulerian if and only if, for every subset

S of vertices of M , the number of arcs and edges from S to S minus

the number of arcs from S to S is a nonnegative even number.

The vector x ∈ ZE∪A+ is the incidence vector of a postman tour of

M if and only if xe ≥ 1 for all e ∈ E ∪ A, x(δE∪A(v)) is even for all


v ∈ V , and x(δA(S)) + x(δE(S)) ≥ x(δA(S)) for all S ⊆ V . Thereforewe obtain the integer program:

MMPT2(M, c) = min c⊤x(11)

subject to

x(δE∪A(v)) ≡ 0 (mod 2) for all v ∈ V,(12)

x(δA(S)) + x(δE(S)) ≥ x(δA(S)) for all S ⊆ V, and(13)

xe ≥ 1 and integer for all e ∈ E ∪A.(14)

Note that the parity constraints (12) are not in the required formfor integer programming; however, this can be easily solved by notingthat, for all v ∈ V ,

(15) x(δE∪A(v)) ≡ x(δA(v)) + x(δE(v))− x(δA(v)) (mod 2),

and introducing a slack variable sv ∈ Z+ to obtain the equivalent con-straint

(16) x(δA(v)) + x(δE(v))− x(δA(v))− 2sv = 0 for all v ∈ V.

Let P2MPT (M) be the convex hull of the feasible solutions to the

integer program above, and let Q2MPT (M) be the set of feasible solutions

to its linear programming relaxation:

LMMPT2(M, c) = min c⊤x(17)

subject to

x(δA(S)) + x(δE(S))− x(δA(S)) ≥ 0 for all S ⊆ V and(18)

xe ≥ 1 for all e ∈ E ∪A.(19)

Note that the constraints (12) were relaxed to x(δE∪A(v)) ≥ 0 forall v ∈ V , but these constraints are redundant in the linear programLMMPT2(M, c). We reach the same conclusion if we use the formula-tion with slacks and we discard them.

4 Linear programming relaxations

In the previous section we gave two integer programming formulationsfor the mixed postman problem, as well as their linear relaxations. Oneof the first questions we might ask is whether one of the relaxationsis better than the other or they are in fact equivalent. We answer this


question by showing in two rather different ways that the relaxations areequivalent. A third, different proof is due to Corberan et al [2]. Withthis result in hand, we study some of the properties of the extremepoints of the set Q1

MPT (M) of solutions to our first formulation.

4.1 Equivalence

We give two proofs that LMMPT1(M, c) and LMMPT2(M, c) are essen-tially equivalent. Our first result says that solving both linear programswould give the same objective value.

Theorem 4.1.1 For every x1 ∈ Q1MPT (M) there exists x2 ∈ Q2

MPT (M)such that c⊤x1 = c⊤x2, and conversely, for every x2 ∈ Q2

MPT (M) thereexists x1 ∈ Q1

MPT (M) such that c⊤x1 = c⊤x2. Moreover, in both cases,

x1a = x2a for all a ∈ A and x1e+

+ x1e+

= x2e for all e ∈ E.

Proof: First note that x1e = x2e for all e ∈ A, and x1e+

+ x1e−

= x2efor all e ∈ E imply c⊤x1 = c⊤x2 for every vector of costs c. (⇒) Letx1 ∈ Q1

MPT (M) and define x2 as above. It is clear that x2 ∈ RE∪A+ , so

we only have to prove (18). Let S ⊆ V , then

0 ≤ 2x1(δB(S))(20)

=!

v∈S

"

x1(δ(v))− x1(δ(v))#

+ 2x1(δB(S))(21)

= x1(δA(S)) + x1(δB(S)) + x1(δB(S))− x1(δA(S))(22)

= x2(δA(S)) + x2(δE(S)) − x2(δA(S)).(23)

(⇐) Let x2 ∈ Q2MPT (M) and assume x2 is rational. Let N be a

positive integer such that each component of x = Nx2 is an even integer.Consider the graph MN that contains xe copies of each e ∈ E∪A. Notethat MN is Eulerian, and xe ≥ N for all e ∈ E ∪ A. Hence we candirect some of the copies of e ∈ E in one direction and the rest in theother (say xe+ and xe− , respectively) to obtain an Eulerian tour of MN .Therefore, x ∈ Q1

MPT (MN ), xe ≥ N for all e ∈ A, and xe+ + xe− ≥ N

for all e ∈ E, and hence x1 = 1Nx ∈ Q1

MPT (M). Note that x1 satisfiesthe properties in the statement. !

Theorem 4.1.1 implies that, for every vector c, LMMPT1(M, c) =LMMPT2(M, c), that is, it is equivalent to optimize over either polyhe-dron. Our second result goes a bit further: we show that Q2

MPT (M) is


essentially a projection of Q1MPT (M). Let A be the incidence matrix of

the directed graph D = (V,A), and let D be the incidence matrix of thedirected graph D+ = (V,E+). Let Q3

MPT (M) be the set of solutions

x ∈ RA∪E∪E+∪E−

of the system:

AxA +D(xE+ − xE−) = 0V(24)

xE − xE+ − xE− = 0E(25)

xA ≥ 1A(26)

xE ≥ 1E(27)

xE+ ≥ 0E(28)

xE− ≥ 0E(29)

Note that this system is a reformulation of (7)–(10) where all the con-straints have been written in vector form, and we have included anadditional variable xe for each edge e. The following is a consequenceof Theorem 4.1.1, but we give a different proof.

Theorem 4.1.2 The projection of the polyhedron Q3MPT (M) onto

xE+ = 0E and xE− = 0E is Q2MPT (M).

Proof: Let Q be the projection of Q3MPT (M) onto xE+ = 0E and

xE− = 0E (which can be obtained with an application of the Fourier-Motzkin elimination procedure), that is, let

Q = x ∈ RA∪E : (A⊤zV + zA)⊤xA+(zB + zE)

⊤xE ≥ z⊤A1A+ z⊤E1E , ∀z ∈ R,

where

R = (zV , zB, zA, zE) ∈ RV ∪E+∪A∪E : zA ≥ 0A, zE ≥ 0E and zB ≥ |D⊤zV |.

We verify first that (18) and (19) are valid inequalities for Q:

(18) Let S ⊆ V , and consider the element of R given by zV = χS ,zB = χδE(S), zA = 0A, and zE = 0E . This implies the constraint(χS)⊤AxA + (χδE(S))⊤xE ≥ 0, that is, x(δE(S)) + x(δA(S)) −x(δA(S)) ≥ 0.

(19) Let a ∈ A, and consider the element of R given by zV = 0V ,zB = 0E , zA = 1a, and zE = 0E . This implies the constraint1⊤a xA ≥ 1⊤a 1A, that is, xa ≥ 1. Let e ∈ E, and consider theelement of R given by zV = 0V , zB = 0E, zA = 0A, and zE = 1e.This implies the constraint 1⊤e xE ≥ 1⊤e 1E , that is, xe ≥ 1.


Now we verify that every element of R can be written as a nonneg-ative linear combination of the following elements of R:

(S1) For S ⊆ V , let zV = χS , zB = χδE(S), zA = 0A, and zE = 0E .

(S2) For S ⊆ V , let zV = −χS, zB = χδE(S), zA = 0A, and zE = 0E .

(A) For a ∈ A, let zV = 0V , zB = 0E , zA = 1a, and zE = 0E .

(E1) For e ∈ E, let zV = 0V , zB = 0E , zA = 0A, and zE = 1e.

(E2) For e ∈ E, let zV = 0V , zB = 1e, zA = 0A, and zE = 0E .

If any component of zA or zE is positive, we can use (A) or (E1) toreduce it to zero, so we only consider the set of solutions of zB ≥ |D⊤zV |with zB and zV free. Let S+ = supp+(zV ), and let S− = supp−(zV ).If both S+ and S− are empty, then we can reduce the components ofzB using (E2). Otherwise, assume that S+ is nonempty and that theminimal positive component of zV is 1. For every edge e ∈ δE(S+) withendpoints u ∈ S+, v /∈ S+ we have

(30) (zB)e ≥ |(D⊤zV )e| = |(zV )u − (zV )v| ≥ |(zV )u| = (zV )u ≥ 1.

Therefore, the vectors

(31) z∗B ≡ zB − χδE(S+) and z∗V ≡ zV − χS+

satisfy z∗B ≥ |D⊤z∗V | and have fewer nonzero components. So we canreduce (zB , zV ) using (S1). Similarly, if S− is nonempty, we can reduce(zB , zV ) using (S2). !

4.2 Half-integrality

Now we explore the structure of the extreme points of Q1MPT (M). To

start, we offer a simple proof of the following result due independentlyto several authors. We say that e ∈ E is tight if xe+ + xe− = 1.

Theorem 4.2.1 (Kappauf and Koehler [7], Ralphs [10], Win [12]) Eve-ry extreme point x of the polyhedron Q1

MPT (M) has components whose

values are either 12 or a nonnegative integer. Moreover, fractional com-

ponents occur only on tight edges.


Proof: Let x be an extreme point of Q1MPT (M). We say that a ∈ A

is fractional if xa is not an integer. Similarly, we say that e ∈ E isfractional if at least one of xe+ or xe− is not an integer. Let F = e ∈E ∪A : e is fractional. We will show that F ⊆ E, and that each e ∈ Fis tight. Assume that for some v ∈ V , dF (v) = 1. Let e be the uniqueelement of F incident to v. Since the total flow into v is integral theonly possibility is that e ∈ E. Moreover, both xe+ and xe− must befractional. If e is not tight, the vectors x1 and x2 obtained from xreplacing the entries in e+ and e− by

(32)x1e+

= xe+ + ϵ x1e−

= xe− + ϵ,x2e+

= xe+ − ϵ x2e−

= xe− − ϵ

(where ϵ = minxe+ , xe− , 2(xe+ + xe− − 1) > 0) would be feasible,with x = 1

2(x1 + x2), contradicting the choice of x. Hence e is a tight

edge, and satisfies xe+ = xe− = 12 . Delete e from F and repeat the

above argument until F is empty or F induces an undirected graphwith minimum degree 2. (Deletion of e does not alter the argumentsince it contributes 0 flow into both its ends.) Suppose F contains acycle C. Assign an arbitrary orientation (say, positive) to C. We saythat an arc in C is forward if it has the same orientation as C, and wecall it backward otherwise. Partition C as follows:

C+A

= e ∈ C ∩ A : e is forward,(33)

C−A

= e ∈ C ∩ A : e is backward,(34)

C=E = e ∈ C ∩ E : e is tight,(35)

C>E = e ∈ C ∩ E : e is not tight,(36)

and define

ϵ+ = mine∈C+

A

⌈xe⌉ − xe,(37)

ϵ− = mine∈C−

A

xe − ⌊xe⌋,(38)

ϵ= = mine∈C=

E

xe+ , xe−,(39)

ϵ> = mine∈C>

E

⌈xe⌉ − xe, xe − ⌊xe⌋,(40)

ϵ1 = minϵ+, ϵ−, 2ϵ=, ϵ>.(41)


The choice of C implies ϵ1 > 0. Now we define a new vector x1 asfollows:

(42) x1e =

⎧

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎩

xe + ϵ1 if e ∈ C+A

or e is forward in C>E

xe − ϵ1 if e ∈ C−A

or e is backward in C>E

xe +12ϵ

1 if e is the forward copy of an edge in C=E

xe −12ϵ

1 if e is the backward copy of an edge in C=E

xe otherwise.

This is equivalent to pushing ϵ1 units of flow in the positive directionof C, and therefore it is easy to verify that x1 ∈ Q1

MPT (M). Similarly,define ϵ2 and a vector x2 using the other (negative) orientation of C.But now x is a convex combination of x1 and x2 (in fact, by choosingϵ = minϵ1, ϵ2 and pushing ϵ units of flow in both directions we wouldhave x = 1

2 (x1 + x2)) contradicting the choice of x. Therefore F is

empty. !

A similar idea allows us to prove a sufficient condition for Q1MPT (M)

to be integral. A mixed graph M = (V,E,A) is even if the total degreedE∪A(v) is even for every v ∈ V .

Theorem 4.2.2 (Edmonds and Johnson [3]) If M is even, then the

polyhedron Q1MPT (M) is integral. Therefore the mixed postman problem

can be solved in polynomial time for the class of even mixed graphs.

Proof: Let x be an extreme point of Q1MPT (M). We say that a ∈ A

is even if xa is even. We say that e ∈ E is even if xe+ − xe− is even.For a contradiction, assume x is not integral, and define F as in theproof of Theorem 4.2.1. Let N = e ∈ E ∪ A : e is even. Note thatby Theorem 4.2.1, F ⊆ N . Hence N is not empty. We show now thatM [N ] has minimum degree 2, and hence contains a cycle C. Let v ∈ V .If dF (v) ≥ 2 then certainly dN (v) ≥ 2. If dF (v) = 1 then

(43) x(δ(v))− x(δ(v)) =∑

a∈δA(v)∪δA(v)

±xa +∑

e∈δE(v)

±(xe+ − xe−)

is the sum of an even number of integer terms (one term per arc a ∈δA(v)∪δA(v) and one term per edge e ∈ δE(v)), and one of them is equalto zero (the one in δF (v)); therefore another term must be even. Thesame argument works for a vertex v not in V (F ), that is, dF (v) = 0,with at least one element of N incident to it, that is, dN (v) ≥ 1.


As before, assign an arbitrary (positive) orientation to C and parti-tion it into the classes C+

A, C−

A, C=

E , C>E. Note that all e ∈ C \C=

E satisfyxe ≥ 2. Hence the vector x1 defined as

(44) x1e =

⎧

⎪

⎪

⎪

⎪

⎨

⎪

⎪

⎪

⎪

⎩

xe + 1 if e ∈ C+A

or e is forward in C>E,

xe − 1 if e ∈ C−A

or e is backward in C>E,

xe +12 if e is the forward copy of an edge in C=

E ,xe −

12 if e is the backward copy of an edge in C=

E ,xe otherwise,

as well as the vector x2 obtained from the negative orientation of C,belong to Q1

MPT (M) and satisfy x = 12(x

1 + x2). This contradictionimplies that F must be empty. !

4.3 Dual integrality

Now we consider the dual of the linear relaxation LMMPT1 (6-10):

DMMPT1(M, c) = 1⊤z(45)

subject to

yu − yv + za ≤ ca for all a ∈ A with tail u and head v,(46)

yu − yv + ze ≤ ce for all e ∈ E with ends u and v,(47)

−yu + yv + ze ≤ ce for all e ∈ E with ends u and v,(48)

yv free for all v ∈ V, and(49)

ze ≥ 0 for all e ∈ A ∪ E.(50)

Theorem 4.3.1 Let M = (V,E,A) be strongly connected, and let c ∈ZE∪A+ . Then DMMPT1 has an integral optimal solution (y∗, z∗).

Proof: Since LMMPT1 is feasible and bounded, then DMMPT1 isalso feasible and bounded. Furthermore, both problems have optimalsolutions. Choose an extreme point optimal solution x∗ of the primal.Without loss of generality, we can assume that not both x∗

e+, x∗

e−are

positive, unless e ∈ E is tight. We construct an integral solution (y∗, z∗)to the dual satisfying the complementary slackness conditions:

1. for all (u, v) = a ∈ A, x∗a > 0 implies y∗u − y∗v + z∗a = ca,

2. for all u, v = e ∈ E, x∗e+

> 0 implies y∗u − y∗v + z∗e = ce,

3. for all u, v = e ∈ E, x∗e−

> 0 implies −y∗u + y∗v + z∗e = ce,


4. for all a ∈ A, z∗a > 0 implies x∗a = 1, and

5. for all e ∈ E, z∗e > 0 implies x∗e+

+ x∗e−

= 1.

Note that x∗a > 0 for all a ∈ A, hence condition (1) implies thaty∗u − y∗v + z∗a = ca for all (u, v) = a ∈ A. Also note that, for any e ∈ E,at least one of x∗

e+> 0 and x∗

e−> 0 holds. Moreover, the only case

in which both hold is when e is a fractional tight edge. In this case,conditions (2) and (3) imply that y∗u = y∗v and z∗e = ce. Hence, to obtaina feasible solution to the dual satisfying complementary slackness, wecan set z∗e = ce for each fractional tight edge e, and then contract eachconnected component (Vi, Fi) of the fractional graph (V, F ) into a singlesuper-vertex vi, creating a new dual variable yvi for it. Once we are donewith the rest of the construction, we set y∗v = y∗vi for each vertex v ∈ Vi.

At this point, all remaining edges e satisfy that either xe+ = 0 orxe− = 0. Delete the arc whose variable is zero, and let D = (V ′, A′) thedirected graph thus obtained. Observe that the restriction x of x∗ to thearcs of D is an optimal integer circulation of D with costs c restricted tothe arcs of D. But the minimum cost circulation problem has integraloptimal dual solutions. Let (y, z) ∈ ZV ′∪E′

be one such solution. Let y∗

be the extension of y as described in the previous paragraph. Let z∗ bethe extension of z obtained as follows. For each a ∈ A \ A′ let z∗a havethe integer value implied by condition (1). For each e /∈ F let z∗e havethe integer value implied by either condition (2) or (3).

Now, using the interpretation of (y, z) as a potential in D, it is nothard to verify that the vector (y∗, z∗) satisfies (4) and (5), and hence itis an integral optimal solution to DMMPT1. !

5 Open problems

One of the most interesting open problems is that of a full characteri-zation of integrality of the polyhedron Q1

MPT (M). Another interestingoption is to add a set of valid inequalities to obtain a tighter relaxation.For example, we can add the well-known odd-cut constraints to obtainanother polyhedron O1

MPT (M), and ask again for a full characteriza-tion of integrality of this polyhedron. Finally, we may ask whether ourknowledge about the extreme points of the primal and dual polyhedracould lead us to a primal-dual approximation algorithm for the mixedpostman problem.


AcknowledgementsThe author would like to thank Bill Cunningham, Joseph Cheriyan,

Jim Geelen, Bertrand Guenin, Miguel Anjos, and Antoine Vella at theUniversity of Waterloo for their continuous support and their insightfulcomments during countless discussions.

Francisco Javier Zaragoza MartınezDepartmento de Sistemas,Universidad Autonoma Metropolitana, Unidad Azcapotzalco

Av. San Pablo 180, Edificio H 2do Piso,Col. Reynosa Tamaulipas,Deleg. Azcapotzalco, 02200, Mexico, D.F.

[email protected]

References

[1] Christofides N.; Benavent E.; Campos V.; Corberan A.; Mota E.,An optimal method for the mixed postman problem, Lecture Notesin Control and Inform. Sci. 59 (1984), 641–649.

[2] Corberan A.; Mota E.; Sanchis J. M., A comparison of two differ-

ent formulations for arc routing problems on mixed graphs, avail-able online in Comput. Oper. Res., 2005.

[3] Edmonds J.; Johnson E. L., Matching, Euler tours and the Chi-

nese postman, Math. Programming 5 (1973), 88–124.

[4] Ford L.R. Jr.; Fulkerson D. R., Flows in Networks, PrincetonUniversity Press, Princeton, N.J., 1962.

[5] Grotschel M.; Win Z., A cutting plane algorithm for the windy

postman problem, Math. Programming Series A, 55 No.3 (1992),339–358.

[6] Guan M. G., Graphic programming using odd or even points, Chi-nese Math 1 (1960), 273–277.

[7] Kappauf C. H.; Koehler G. J.,The mixed postman problem, Dis-crete Appl. Math. 1 No.1-2 (1979), 89–103.

[8] Nobert Y.; Picard J.-C., An optimal algorithm for the mixed Chi-

nese postman problem, Networks 27 No.2 (1996), 95–108.

[9] Papadimitriou C. H., On the complexity of edge traversing, J.ACM 23 No.3 (1976), 544–554.


[10] Ralphs T. K., On the mixed Chinese postman problem, Oper. Res.Lett. 14 No.3 (1993), 123–127.

[11] Veblen O., An application of modular equations in analysis situs,Ann. of Math. 2 No.14 (1912/1913), 86–94.

[12] Win Z., On the windy postman problem on Eulerian graphs, Math.Programming Series A, 44 No.1 (1989), 97–112.


A nonmeasurable set as a unionof a family of increasing

well–ordered measurable sets ∗

Juan Gonzalez-Hernandez Cesar E. Villarreal

Abstract

Given a measurable space (X,A) in which every singleton is mea-surable and which contains a nonmeasurable subset, we prove theexistence of a nonmeasurable set which is the union of a well-ordered increasing family of measurable sets.

2000 Mathematics Subject Classification: 28A05, 06A05.Keywords and phrases: well order, measurable space.

1 Introduction

Using the well order principle (Zermelo’s theorem) we prove, for a verygeneral measurable space (X,A), that there exists a well ordered family(under the inclusion) of measurable sets whose union is nonmeasurable.This study is motivated by the determination of the existence of so-lutions in a Markov decision problem with constraints (see [3] for thistopic). The problem we faced was to find an optimal stochastic kernelsupported on a measurable function. This led us to try to extend thedomain of a measurable function on the union of a well–ordered familyof measurable sets. However, the measurability may be missed for theunion of the family, as we show below.

We also give an example of a set A contained in a measurable spacewhere each singleton is measurable, but nevertheless A can not be ex-pressed as a well–ordered union of measurable sets.

∗Work partially sponsored by CONACYT grant SEP-2003-C02-45448/A-1 andPAICYT-UANL grant CA826-04.

35

36 J. Gonzalez-Hernandez and C. E. Villarreal

Let us start by recalling some basic terminology and the statementof the well order principle.

Let X be a set.

(a) A relation ≼ is called a partial order on X if it is reflexive, anti-symmetric and transitive. In this case, X is said to be partiallyordered by ≼.

(b) Let A be a subset of X. If there exists x ∈ A such that x ≼ a forall a ∈ A, then x is called the first element of A (with respect tothe partial order ≼).

(c) A partial order ≼ on X is called a total order if for each x, y ∈ X

we have x ≼ y or y ≼ x.

(d) A total order ≼ in a set X is called a well order if every nonemptysubset of X has a first element. In this case, X is said to be wellordered.

Theorem 1.1 (Well order principle) Let X be a set. There is awell order ≼ in X.

The proof of this principle can be found, for instance, in [1, Wellordering theorem] or [2].

2 The result

Theorem 2.1 Let (X,A) be a measurable space such that, for eachx ∈ X, the set x is measurable, and X contains a nonmeasurable set.Then there is a collection I of measurable subsets of X, well ordered bycontention (⊂), such that

!C∈I C is nonmeasurable.

Proof: Let A ⊂ X be a nonmeasurable set. By the well order principle,there is a well order ≼ in A. Denote by ≺ the relation a ≺ b ⇐⇒ (a ≼ b

and a = b).For each d ∈ A let us define Ad := x ∈ A : x ≼ d. Set E := Ad :

d ∈ A and note that this set is well ordered by ⊂. If all the Ad aremeasurable, then we take I = E . Otherwise, there is a d∗ ∈ A such thatAd∗ is nonmeasurable. Let A′ = d ∈ A : Ad is nonmeasurable. SinceA′ ⊂ A is nonempty, there exists the first element d′ of A′. Now, Ad′ is

A nonmeasurable set as a union of well-ordered measurable sets. 37

nonmeasurable and so is Ad′ \d′. Moreover, taking I = Ad : d ≺ d′,

we have

Ad′ \ d′ = d ∈ A : d ≺ d′ =

!

d≺d′

Ad =!

C∈I

C,

and, therefore, we can conclude that the set"

C∈I C is nonmeasurable.Noting again that I is well ordered by ⊂, the proof is complete.

3 An example

We shall give an example of a measurable space in which each singletonis measurable, but there exists a nonmeasurable set A that is not theunion of measurable sets in a well ordered family (under ⊂).

For every set B, let #B denote the cardinality of B and 2B thepower set of B.

Let X be a set such that #X > #IR (we can take X = 2IR, forinstance). Define the σ-algebra A as the family of subsets A of X suchthat A ∈ A ⇐⇒ A is countable or X \ A is countable. We can takeA ⊂ X such that #A > #IR and #(X \ A) > #IR. Let I be a well-ordered index set, and assume that (Ai)i∈I is any strictly increasing netof measurable sets such that

"i∈I Ai = A. As each X \ Ai ⊃ X \ A

is uncountable, each Ai is countable. From Theorem 14, p. 179 in [2],we can see that #I = #A > #IR, so the set J := i ∈ I : #j ∈I : j ≼ i > #IN is nonempty. Let i∗ be the first element of J andobserve that #j ∈ I : j ≼ i∗ > #IN. Now, by the axiom of choice(see [1] or [2]), for each i ∈ I we can choose xi ∈ Ai \

"j≺iAi, such

that the sets j ∈ I : j ≼ i∗ and"

j≼i∗xj have the same cardinality.However,

"j≼i∗xi ⊂

"j≼i∗ Aj = Ai∗ , and so #Ai∗ ≥ #j ∈ I : j ≼

i∗ > #IN; that is to say, the set Ai∗ is uncountable, and we arrive ata contradiction because each Ai is countable. Hence, A cannot be theunion of measurable sets in a well ordered family.

We would like to conclude by posing a question. Consider the mea-surable space (IR,M), where M is the Lebesgue σ-algebra, and let A

be an arbitrary nonmeasurable subset of IR (for an example of a non-Lebesgue measurable set see [4]). Is it always possible to express A asthe limit of an increasing net (Ai)i∈I of elements in M for some wellordered set I?

38 J. Gonzalez-Hernandez and C. E. Villarreal

Juan Gonzalez-HernandezDepartamento de Probabilidad y

Estadıstica,IIMAS-UNAM,A. P. 20-726,Mexico, D. F., 01000,[email protected]

Cesar E. VillarrealDivision de Posgrado en Ingenierıa

de Sistemas,FIME-UANL,A. P. 66450,San Nicolas de los Garza, N. L.,[email protected]

References

[1] Halmos P. R., Naive Set Theory, Van Nostrand Reinhold, NewYork, 1960.

[2] Just W.; Weese M., Discovering Modern Set Theory I, The Basics,American Mathematical Society, Providence, RI, 1996.

[3] Piunovsky A. B., Optimal Control of Random Sequences in Pro-blems With Constraints, Kluwer Academic Publisher, Dordrecht,1997.

[4] Royden H. L., Real Analysis, Macmillan, New York, 1968.


Noncooperative continuous-timeMarkov games ∗

Hector Jasso-Fuentes

Abstract

This work concerns noncooperative continuous-time Markov gameswith Polish state and action spaces. We consider finite-horizonand infinite-horizon discounted payoff criteria. Our aim is to givea unified presentation of optimality conditions for general Markovgames. Our results include zero-sum and nonzero-sum games.

2000 Mathematics Subject Classification: 91A25, 91A15, 91A10.Keywords and phrases: Continuous-time Markov games, noncoopera-tive games.

1 Introduction

Continuous-time Markov games form a class of dynamic stochastic gamesin which the state evolves as a Markov process. The class of Markovgames includes (deterministic) differential games, stochastic differentialgames, jump Markov games and many others, but they are usually stud-ied as separate, different, types of games. In contrast, we propose here aunified presentation of optimality conditions for general Markov games.In fact, we only consider noncooperative games but the same ideas canbe extended in an obvious manner to the cooperative case.

As already mentioned, our presentation and results hold for gen-eral Markov games but we have to pay a price for such a generality;namely, we restrict ourselves to Markov strategies, which depend only

∗Research partially supported by a CONACYT scolarship. This paper is partof the author’s M. Sc. thesis presented at the Department of Mathematics ofCINVESTAV-IPN.

39

40 Hector Jasso-Fuentes

on the current state. More precisely, at each decision time t, the playerschoose their corresponding actions (independently and simultaneously)depending only on the current state X(t) of the game. Hence, this ex-cludes some interesting situations, for instance, some hierarchical gamesin which some players “go first”.

Our references are mainly on noncooperative continuous-time games.However, for cooperative games the reader may consult Filar/Petrosjan[2], Gaidov [3], Haurie [5] and their references. For discrete time gamessee, for instance, Basar/Oldsder [1], Gonzalez-Trejo et al. [4].

A remark on terminology: The Borel σ-algebra of a topologicalspace S is denoted by B(S). A complete and separable metric space iscalled a Polish space.

2 Preliminaries

Throughout this section we let S be a Polish space, andX(·) = X(t), t ≥0 a S-valued Markov process defined on a probability space (Ω,F , IP).Denote by IP(s, x, t, B) := IP(X(t) ∈ B|X(s) = x) for all t ≥ s ≥ 0,x ∈ S and B ∈ B(S), the transition probability function of X(·).

2.1 Semigroups

Definition 2.1 Let M be the linear space of all real-valued measurablefunctions v on S := [0,∞) × S such that

!

SIP(s, x, t, dy) |v(s, y)| < ∞ for all 0 ≤ s ≤ t and x ∈ S.

For each t ≥ 0 and v ∈ M , we define a function Ttv on S as

Ttv(s, x) :=!

SIP(s, x, s+ t, dy) v(s+ t, y).(1)

Proposition 2.2 The operators Tt, t ≥ 0, defined by (1), form a semi-group of operators on M , that is,

(i) T0 = I, the identity, and

(ii) Tt+r = TtTr.

For a proof of this proposition see, for instance, Jasso-Fuentes [7], Propo-sition 1.2.2.

Noncooperative continuous-time Markov games 41

2.2 The extended generator

Definition 2.3 Let M0 ⊂ M be the family of functions v ∈ M forwhich the following conditions hold:

a) limt↓0 Ttv(s, x) = v(s, x) for all (s, x) ∈ S;

b) there exist t0 > 0 and u ∈ M such that

Tt|v|(s, x) ≤ u(s, x) for all (s, x) ∈ S and 0 ≤ t ≤ t0.

Now let D(L) ⊂ M0 be the set of functions v ∈ M0 for which:

a) the limit

Lv(s, x) : = limt↓0

[Ttv(s, x)− v(s, x)]

t

= limt↓0

1

t

!

SIP(s, x, s + t, dy)[v(s + t, y)− v(s, x)](2)

exists for all (s, x) ∈ S,

b) Lv ∈ M0, and

c) there exist t0 > 0 and u ∈ M such that

|Ttv(s, x)− v(s, x)|

t≤ u(s, x)

for all (s, x) ∈ S and 0 ≤ t ≤ t0.

The operator L in (2) will be referred to as the extended generatorof the semigroup Tt, and the set D(L) is called the domain of L.

The following lemma (which is proved in [7], Lemma 1.3.2, for in-stance) summarizes some properties of L.

Lemma 2.4 For each v ∈ D(L), the following conditions hold:

a) d+

dt Ttv := limh↓0 h−1[Tt+hv − Ttv] = TtLv,

b) Ttv(s, x)− v(s, x) =" t0 Tr(Lv)(s, x) dr,

c) if ρ > 0 and vρ(s, x) := e−ρsv(s, x), then vρ is in D(L) and

Lvρ(s, x) = e−ρs[Lv(s, x) − ρv(s, x)].


2.3 Expected rewards

Let X(·) = X(t), t ≥ 0 be as in the previous paragraphs, that is, aMarkov process with values in a Polish space S and with transition prob-abilities IP(s, x, t, B) for all t ≥ s ≥ 0, x ∈ S and B ∈ B(S). Recallingthe Definitions 2.1 and 2.3 the semigroup defined in (1) becomes

Ttv(s, x) = IEsx[v(s+ t,X(s + t))],

where IEsx(·) := IE[ · |X(s) = x] is the conditional expectation givenX(s) = x. Similarly, we can rewrite part b) of Lemma 2.4 as

IEsx[v(s+ t,X(s + t))]− v(s, x) = IEsx

!" t

0Lv(s+ r,X(s + r))dr

#

(3)

for each v ∈ D(L). We shall refer to (3) as Dynkin’s formula. Theextended generator L of the semigroup Tt will also be referred to asthe extended generator of the Markov process X(·).

The following fact will be useful in later sections.

Proposition 2.5 Fix numbers ρ ∈ IR and τ > 0. Let R(s, x) andK(s, x) be measurable functions on Sτ := [0, τ ]×S, and suppose that Ris in M0. If a function v ∈ D(L) satisfies the equation

ρv(s, x) = R(s, x) + Lv(s, x)(4)

on Sτ , with the “terminal” condition

v(τ, x) = K(τ, x),(5)

then, for every (s, x) ∈ Sτ ,

v(s, x) = IEsx

!" τ

se−ρ(t−s)R(t,X(t))dt + e−ρ(τ−s)K(τ,X(τ))

#

.(6)

If the equality in (4) is replaced with the inequality ”≤” or ”≥”, thenthe equality in (6) is replaced with the same inequality, that is, ”≤” or”≥” respectively.

Proof: Suppose that v satisfies (4) and let vρ(s, x) := e−ρsv(s, x).Then, by (4) and Lemma 2.4 c), we obtain

Lvρ(s, x) = e−ρs[Lv(s, x)− ρv(s, x)]

= −e−ρsR(s, x).(7)


Therefore, applying Dynkin’s formula (3) to vρ and using (7),

IEsx

!

e−ρ(s+t)v(s + t,X(s+ t))"

− e−ρsv(s, x)

= −IEsx

#$ t

0e−ρ(s+r)R(s+ r,X(s + r))dr

%

(8)

= −IEsx

#$ s+t

se−ρrR(r,X(r))dr

%

.

The latter expression, with s+ t = τ , and (5) give

IEsx

&

e−ρτK(τ,X(τ))'

− e−ρsv(s, x)

= −IEsx

#$ τ

se−ρrR(r,X(r))dr

%

.

Finally, multiply both sides of this equality by eρs and then rearrangeterms to obtain (6).

Concerning the last statement in the proposition, suppose that in-stead of (4) we have ρv ≥ R+ Lv. Then (7) becomes

−e−ρsR(s, x) ≥ Lvρ(s, x)

and the same calculations in the previous paragraph show that theequality in (6) should be replaced with “≥”. For “≤”, the result isobtained similarly.

Observe that the number ρ in Proposition 2.5 can be arbitrary, butin most applications in later sections we will require either ρ = 0 orρ > 0. In the latter case ρ is called a “discount factor”.

On the other hand, if the function R(s, x) is interpreted as a “rewardrate”, then (6) represents an expected total reward during the timeinterval [s, τ ] with initial condition X(s) = x and terminal reward K.This expected reward will be associated with finite-horizon games. Incontrast, the expected reward in (11), below, will be associated withinfinite-horizon games.

Proposition 2.6 Let ρ > 0 be a given number, and R ∈ M0 a functionon S := [0,∞)× S . If a function v ∈ D(L) satisfies

ρv(s, x) = R(s, x) + Lv(s, x) for all (s, x) ∈ S(9)

and is such that, as t → ∞,

e−ρtTtv(s, x) = e−ρtIEsx [v(s+ t,X(s + t))] → 0,(10)


then

v(s, x) = IEsx

!" ∞

se−ρ(t−s)R(t,X(t))dt

#

(11)

=" ∞

0e−ρtTtR(s, x)dt.

Moreover, if the equality in (9) is replaced with the inequality “≤” or“≥”, then the equality in (11) should be replaced with the same inequa-lity.

Proof: Observe that the equations (9) and (4) are essentially the same,the only difference being that the former is defined on S and the latteron Sτ . At any rate, the calculations in (7)-(8) are also valid in thepresent case. Hence, multiplying both sides of (8) by eρs and thenletting t → ∞ and using (10) we obtain (11). The remainder of theproof is as in Proposition 2.5.

3 The game model and strategies

For notational case, we shall restrict ourselves to the two-player situ-ation. However, the extension to any finite number ≥ 2 of players iscompletely analogous.

3.1 The game model

Some of the main features of a (two-player) continuous-time Markovgame can be described by means of the game model

GM := S, (Ai, Ri)i=1,2, La1,a2(12)

with the following components.

• S denotes the game’s state space, which is assumed to be a Polishspace.

• Associated with each player i = 1, 2, we have

(Ai, Ri)(13)

where Ai is a Polish space that stands for the action space (orcontrol set) for player i. Let

A := A1 ×A2, and K := S ×A.


The second component in (13) is a real-valued measurable functionRi on

[0,∞) ×K = [0,∞)× S ×A = S ×A (S := [0,∞) × S),

which denotes the reward rate function for player i. (Observethat Ri(s, x, a1, a2) depends on the actions (a1, a2) ∈ A of bothplayers.)

• For each pair a = (a1, a2) ∈ A there is a linear operator La

with domain D(La), which is the extended generator of a S-valuedMarkov process with transition probability IPa(s, x, t, B).

The game model (12) is said to be time-homogeneous if the re-ward rates are time-invariant and the transition probabilities are time-homogeneous, that is,

Ri(s, x,a) = Ri(x,a) and IPa(s, x, t, B) = IPa(t− s, x,B).

Summarizing, the game model (12) tells us where the game lives(the state space S) and how it moves (according to the players’ actionsa = (a1, a2) and the Markov process associated to La). The rewardrates Ri are used to define the payoff function that player i (i = 1, 2)wishes to “optimize”— see for instance (15) and (16) below. To do thisoptimization each player uses, when possible, suitable “strategies”, suchas those defined next.

3.2 Strategies

We will only considerMarkov (also known as feedback) strategies, namely,for each player i = 1, 2, measurable functions πi from S := [0,∞) × S

to Ai. Thus, πi(s, x) ∈ Ai denotes the action of player i prescribed bythe strategy πi if the state is x ∈ S at time s ≥ 0. In fact, we willrestrict ourselves to classes Π1, Π2 of Markov strategies that satisfy thefollowing.

Assumption 3.1 For each pair π = (π1,π2) ∈ Π1 × Π2, there exists astrong Markov process Xπ(·) = Xπ(t), t ≥ 0 such that:

a) Almost all the sample paths of Xπ(·) are right-continuous, withleft-hand limits, and have only finitely many discontinuities in anybounded time interval.


b) The extended generator Lπ of Xπ(·) satisfies that

Lπ = La if (π1(s, x),π2(s, x)) = (a1, a2) = a.

The set Π1×Π2 in Assumption 3.1 is called the family of admissiblepairs of Markov strategies. A pair (π1,π2) ∈ Π1 × Π2 is said to bestationary if πi(s, x) ≡ πi(x) does not depend on s ≥ 0.

Clearly, the function spaces M ⊃ M0 ⊃ D(L) introduced in Section2 depend on the pair π = (π1,π2) ∈ Π1 × Π2 of strategies being used,because so does IPπ. Hence, these spaces will now be written as Mπ,Mπ

0 , D(Lπ), and they are supposed to verify the following conditions.

Assumption 3.2 a) There exist nonempty spaces M ⊃ M0 ⊃ D,which do not depend on π, such that, for all π = (π1,π2) ∈ Π1×Π2

M ⊂ Mπ , M0 ⊂ Mπ0 , D ⊂ D(Lπ)

and, in addition, the operator Lπ is the closure of its restrictionto D.

b) For π = (π1,π2) ∈ Π1×Π2 and i = 1, 2, the reward rate Ri(s, x, a1, a2)is such that Rπ

i is in M0, where

Rπi (s, x) := Ri(s, x,π1(s, x),π2(s, x)).

Sometimes we shall use the notation

Rπi (s, x) := Ri(s, x,π1,π2) for π = (π1,π2), i = 1, 2.(14)

If the game model is time-homogeneous and the pair (π1,π2) is station-ary, then (14) reduces to

Rπi (x) := Ri(x,π1(x),π2(x)) = Ri(x,π1,π2).

Throughout the remainder of this paper we consider the game modelGM in (12) under Assumptions 3.1 and 3.2.

4 Noncooperative equilibria

Let GM be as in (12). In this work, we are concerned with the followingtwo types of payoff functions, where we use the notation (14). For eachpair of strategies (π1,π2) ∈ Π1 ×Π2 and each player i = 1, 2:


• The finite-horizon payoff

V iτ (s, x,π1,π2) : = IEπ1,π2

sx

!" τ

se−ρ(t−s)Ri(t,X(t),π1,π2)dt

(15)

+e−ρ(τ−s)Ki(τ,X(τ)) ]

where 0 ≤ s ≤ τ , x ∈ S, Ki is a function in M (the space inAssumption 3.2 a)), and ρ ≥ 0 is a “discount factor”. The timeτ > 0 is called the game’s horizon or “terminal time”, and Ki is a“terminal reward”.

• The infinite-horizon discounted payoff

V i(s, x,π1,π2) := IEπ1,π2sx

!" ∞

se−ρ(t−s)Ri(t,X(t),π1,π2)dt

#

(16)

where s ≥ 0, x ∈ S, and ρ > 0 is a (fixed) discount factor.

Each player i = 1, 2 wishes to “optimize” his payoff in the followingsense.

Definition 4.1 For i = 1, 2, let V iτ be as in (15), and define Sτ :=

[0, τ ] × S. A pair (π∗1 ,π

∗2) ∈ Π1 × Π2 of admissible strategies is said to

be a noncooperative equilibrium, also known as a Nash equilibrium, iffor all (s, x) ∈ Sτ

V 1τ (s, x,π

∗1 ,π

∗2) ≥ V 1

τ (s, x,π1,π∗2) for all π1 ∈ Π1(17)

andV 2τ (s, x,π

∗1 ,π

∗2) ≥ V 2

τ (s, x,π∗1 ,π2) for all π2 ∈ Π2.(18)

Hence, (π∗1 ,π

∗2) is a Nash equilibrium if for each i = 1, 2, π∗

i maxi-mizes over Πi the payoff function V i

τ of player i when the other player,say j = i, uses the strategy π∗

j .For the infinite-horizon payoff function in (16), the definition of Nash

equilibrium is the same as in Definition 4.1 with V i and S := [0,∞)×S

in lieu of V iτ and Sτ , respectively.

Zero-sum games. For i = 1, 2, let Fi(s, x,π1,π2) be the payoff func-tion in either (15) or (16). The game is called a zero-sum game if

F1(s, x,π1,π2) + F2(s, x,π1,π2) = 0 for all s, x,π1,π2,


that is, F1 = −F2. Therefore, if we define F := F1 = −F2, it followsfrom (17) and (18) that player 1 wishes to maximize F (s, x,π1,π2) overΠ1, whereas player 2 wishes to minimize F (s, x,π1,π2) over Π2, so (17)and (18) become

F (s, x,π1,π∗2) ≤ F (s, x,π∗

1 ,π∗2) ≤ F (s, x,π∗

1 ,π2)(19)

for all π1 ∈ Π1 and π2 ∈ Π2, and all (s, x). In this case the Nashequilibrium (π∗

1,π∗2) is called a saddle point.

In the zero-sum case, the functions

L(s, x) := supπ1∈Π1

infπ2∈Π2

F (s, x,π1,π2)(20)

andU(s, x) := inf

π2∈Π2

supπ1∈Π1

F (s, x,π1,π2)(21)

play an important role. The function L(s, x) is called the game’s lowervalue (with respect to the payoff F (s, x,π1,π2)) and U(s, x) is the game’supper value. Clearly, we have

L(s, x) ≤ U(s, x) for all (s, x).(22)

If the upper and lower values coincide, then the game is said to have avalue, and the value of the game, call it V(s, x) is the common value ofL(s, x) and U(s, x), i.e.

V(s, x) := L(s, x) = U(s, x) for all (s, x).

On the other hand, if (π∗1 ,π

∗2) satisfies (19), a trivial calculation

yieldsU(s, x) ≤ F (s, x,π∗

1 ,π∗2) ≤ L(s, x) for all (s, x),

which together with (22) gives the following.

Proposition 4.2 If the zero-sum game with payoff function F has asaddle point (π∗

1 ,π∗2), then the game has the value

V(s, x) = F (s, x,π∗1 ,π

∗2) for all (s, x).

The next proposition gives conditions for a pair of strategies to bea saddle point.


Proposition 4.3 Suppose that there is a pair of admissible strategiesπ∗1, π

∗2 that satisfy, for all (s, x),

F (s, x,π∗1 ,π

∗2) = sup

π1∈Π1

F (s, x,π1,π∗2)

(23)

= infπ2∈Π2

F (s, x,π∗1 ,π2).

Then (π∗1 ,π

∗2) is a saddle point.

Proof: Let (π∗1 ,π

∗2) be a pair of admissible strategies that satisfy (23).

Then, for all (s, x), from the first equality in (23) we obtain

F (s, x,π∗1 ,π

∗2) ≥ F (s, x,π1,π

∗2) for all π1 ∈ Π1,

which is the first inequality in (19). Similarly, the second equality in(23) yields the second inequality in (19), and it follows that (π∗

1,π∗2) is

a saddle point.

In the next section we give conditions for a pair of strategies to be asaddle point, and in Section 6 we study the so-called nonzero-sum caseas in (17), (18).

5 Zero-sum games

In this section we study the existence of saddle points for the finite-horizon and infinite-horizon payoffs in (15) and (16), respectively.

Finite-horizon payoff

As in (19)-(21), the finite-horizon payoff (15), in the zero-sum case,does not depend on i = 1, 2. Hence, we have the payoff

Vτ (s, x,π1,π2) : = IEπ1,π2sx

!" τ

se−ρ(t−s)R(t,X(t),π1,π2)dt

+e−ρ(τ−s)K(τ,X(τ)) ] .

This function Vτ plays now the role of F in (19)-(23). Recall that theAssumptions 3.1 and 3.2 are supposed to hold.


Theorem 5.1 Consider ρ ∈ IR and τ > 0 fixed. Moreover let R(s, x, a1, a2)and K(s, x, a1, a2) be measurable functions on Sτ × A, where Sτ :=[0, τ ] × S and A := A1 × A2. Suppose that for each pair (π1,π2) ∈Π1×Π2, the function R(s, x,π1,π2) is in M0. In addition, suppose thatthere is a function v(s, x) ∈ D and a pair of strategies (π∗

1 ,π∗2) ∈ Π1×Π2

such that, for all (s, x) ∈ Sτ ,

ρv(s, x) = infπ2∈Π2

R(s, x,π∗1 ,π2) + Lπ∗

1,π2v(s, x)(24)

= supπ1∈Π1

R(s, x,π1,π∗2) + Lπ1,π∗

2v(s, x)(25)

= R(s, x,π∗1 ,π

∗2) + Lπ∗

1,π∗

2v(s, x)(26)

with the boundary condition

v(τ, x) = K(τ, x) for all x ∈ S.(27)

Then

a) v(s, x) = Vτ (s, x,π∗1 ,π

∗2) for all (s, x) ∈ Sτ ;

b) (π∗1 ,π

∗2) is a saddle point and v(s, x) is the value of the game.

Proof:

a) Comparing (26)-(27) with (4)-(5), we conclude that part a) followsfrom Proposition 2.5.

b) Assume for a moment that, for all (s, x) ∈ Sτ and all pairs (π1,π2)of admissible strategies, we have

Vτ (s, x,π1,π∗2) ≤ v(s, x) ≤ Vτ (s, x,π

∗1 ,π2)(28)

If this is indeed true, then b) will follow from part a) together with (19)and Proposition 4.2. Hence it suffices to prove (28).

To this end, let us call F (s, x,π1,π2) the function inside the bracketsin (24)-(25), i.e.

F (s, x,π1,π2) := R(s, x,π1,π2) + Lπ1,π2v(s, x).(29)

Interpreting this function as the payoff of a certain game, it followsfrom (24)-(26) and the Proposition 4.3 that the pair (π∗

1 ,π∗2) is a saddle

point, that is, F (s, x,π∗1 ,π

∗2) = ρv(s, x) satisfies (19). More explicitly,


from (29) and the equality F (s, x,π∗1 ,π

∗2) = ρv(s, x), (19) becomes: for

all π1 ∈ Π1 and π2 ∈ Π2,

R(s, x,π1,π∗

2) + Lπ1,π∗

2 v(s, x) ≤ ρv(s, x) ≤ R(s, x,π∗

1 ,π2) + Lπ∗

1,π2v(s, x).

These two inequalities together with the second part of Proposition 2.5give (28).

Infinite-horizon discounted payoff

We now consider the infinite-horizon payoff in (16), which in thezero-sum case can be interpreted as

V (s, x,π1,π2) = IEπ1,π2sx

!" ∞

se−ρ(t−s)R(t,X(t),π1,π2)dt

#

.

Exactly the same arguments used in the proof of Theorem 5.1 butreplacing Proposition 2.5 with Proposition 2.6, give the following resultin the infinite-horizon case.

Theorem 5.2 Suppose ρ > 0. Let R(s, x, a1, a2) be as in Assumption3.2 b). Suppose that there exist a function v ∈ D and a pair of strategies(π∗

1 ,π∗2) ∈ Π1 ×Π2 such that, for all (s, x) ∈ S := [0,∞) × S,

ρv(s, x) = infπ2∈Π2

R(s, x,π∗1 ,π2) + Lπ∗

1,π2v(s, x)

= supπ1∈Π1

R(s, x,π1,π∗2) + Lπ1,π

∗

2v(s, x)

= R(s, x,π∗1 ,π

∗2) + Lπ∗

1,π∗

2v(s, x)(30)

and, moreover, for all (s, x) ∈ S and (π1,π2) ∈ Π1 ×Π2,

e−ρtIEπ1,π2sx [v(s + t,X(s+ t))] → 0 as t → ∞.(31)

Then

a) v(s, x) = V (s, x,π∗1 ,π

∗2) for all (s, x) ∈ S;

b) (π∗1 ,π

∗2) is a saddle point for the infinite-horizon discounted payoff,

and v(s, x) is the value of the game.

Proof: Comparing (30)-(31) with (9)-(10) we can use Proposition 2.6to obtain a).

To obtain b), we follow the same steps used in the proof of Theorem5.1 but replacing Proposition 2.5 with Proposition 2.6, and Sτ with S.


6 Nonzero-sum games

An arbitrary game which does not satisfy the zero-sum condition iscalled a nonzero-sum game. In this section we are concerned with theexistence of Nash equilibria for nonzero-sum continuous-time Markovgames with the payoff functions (15) and (16).

Finite-horizon payoff

For i = 1, 2, let V iτ (s, x,π1,π2) be the finite-horizon payoff in (15).

In this setting, the following theorem gives sufficient conditions for theexistence of a Nash equilibrium — see Definition 4.1.

Theorem 6.1 Suppose that for i = 1, 2, there are functions vi(s, x) inD and strategies π∗

i ∈ Πi that satisfy, for all (s, x) ∈ Sτ , the equations

ρv1(s, x) = maxπ1∈Π1

R1(s, x,π1,π∗2) + Lπ1,π∗

2v1(s, x)

(32)

= R1(s, x,π∗1 ,π

∗2) + Lπ∗

1,π∗

2v1(s, x)

and

ρv2(s, x) = maxπ2∈Π2

R2(s, x,π∗1 ,π2) + Lπ∗

1,π2v2(s, x)

(33)

= R2(s, x,π∗1 ,π

∗2) + Lπ∗

1,π∗

2v2(s, x),

as well as the boundary (or “terminal”) conditions

v1(τ, x) = K1(τ, x) and v2(τ, x) = K2(τ, x) for all x ∈ S.(34)

Then (π∗1 ,π

∗2) is a Nash equilibrium and for each player i = 1, 2 the

expected payoff is

vi(s, x) = V iτ (s, x,π

∗1 ,π

∗2) for all (s, x) ∈ Sτ .(35)

Proof: From the second equality in (32) together with the first bound-ary condition in (34), the Proposition 2.5 gives (35) for i = 1. A similarargument gives of course (35) for i = 2.

On the other hand, from the first equality in (32) we obtain

ρv1(s, x) ≥ R1(s, x,π1,π∗2) + Lπ1,π∗

2v1(s, x) for all π1 ∈ Π1.


Thus using again Proposition 2.5 we obtain

v1(s, x) ≥ V 1τ (s, x,π1,π

∗2) for all π1 ∈ Π1,

which combined with (35) for i = 1 yields (17). A similar argumentgives (18) and the desired conclusion follows.

Infinite-horizon discounted payoff

Let us now consider the infinite-horizon payoff V i(s, x,π1,π2) in(16). The corresponding analogue of Theorem 6.1 is as follows.

Theorem 6.2 Suppose that, for i = 1, 2, there are functions vi(s, x) ∈D and strategies π∗

i ∈ Πi that satisfy, for all (s, x) ∈ S, the equations(32) and (33) together with the condition

e−ρtT π1,π2

t vi(s, x) → 0 as t → ∞

for all π1 ∈ Π1, π2 ∈ Π2, i = 1, 2, and (s, x) ∈ S. Then (π∗1 ,π

∗2) is a

Nash equilibrium for the infinite-horizon discounted payoff (16) and theexpected payoff is

vi(s, x) = Vi(s, x,π∗1 ,π

∗2) for all (s, x) ∈ S, i = 1, 2.

We omit the proof of this theorem because it is essentially the sameas the proof of Theorem 6.1 (using Proposition 2.6 in lieu of Proposition2.5).

7 Concluding remarks

In this paper we have presented a unified formulation of continuous-timeMarkov games, similar to the one-player (or control) case in Hernandez-Lerma[6]. This formulation is quite general and it includes practicallyany kind of Markov games, but of course it comes at price because wehave restricted ourselves to Markov strategies, which are memoryless.In other words, our players are not allowed to use past information;they base their decisions on the current state only. This is a seriousrestriction that needs to be eliminated, and so it should lead to futurework.

Acknowledgement

Thanks to Prof. Onesimo Hernandez-Lerma for valuable commentsand discussions on this work.


Hector Jasso-FuentesDepartamento de Matematicas,

CINVESTAV-IPN,A.P. 14-470,Mexico D.F. 07000,

Mexico.

References

[1] Basar T.; Olsder G.J., Dynamic Noncooperative Game Theory,Second Edition, SIAM, Philadelphia, 1999.

[2] Filar J.A.; Petrosjan L.A., Dynamic cooperative games, Interna-tional Game Theory Review 2 (2000), 47–65.

[3] Gaidov S.D., On the Nash-bargaining solution in stochastic diffe-rential games, Serdica 16 (1990), 120–125.

[4] Gonzalez-Trejo J.I.; Hernandez-Lerma O.; Hoyos-Reyes L.F., Mini-max control of discrete-time stochastic system, SIAM J. ControlOptim. 41 (2003), 1626–1659.

[5] Haurie A., A historical perspective on cooperative differentialgames, in: Advances in Dynamic Games and Applications (Maas-tricht, 1998), 19–29, Birkhauser Boston, 2001.

[6] Hernandez-Lerma O., Lectures on Continuous-Time Markov Con-trol Processes, Sociedad Matematica Mexicana, Mexico D.F., 1994.

[7] Jasso-Fuentes H., Noncooperative continuous-time Markov games,Tesis de Maestrıa, CINVESTAV-IPN, Mexico D.F., 2004.

Morfismos, Comunicaciones Estudiantiles del Departamento de Matematicas delCINVESTAV, se termino de imprimir en el mes de diciembre de 2005 en el tallerde reproduccion del mismo departamento localizado en Av. IPN 2508, Col. SanPedro Zacatenco, Mexico, D.F. 07300. El tiraje en papel opalina importada de 36kilogramos de 34 × 25.5 cm consta de 500 ejemplares en pasta tintoreto color verde.

Apoyo tecnico: Omar Hernandez Orozco.

Contenido

Approximation of general optimization problems

Jorge ´ amreL-zednanreHomisenOdnaaneM-zeravlA . . . . . . . . . . . . . . . . . . . . . . 1

Linear programming relaxations of the mixed postman problem

Francisco Javier Zaragoza Martınez . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

A nonmeasurable set as a union of a family of increasing well-ordered measur-able sets

laerralliV.EraseCdnazednanreH-zelaznoGnauJ . . . . . . . . . . . . . . . . . . . . . . . 35

Noncooperative continuous-time Markov games

Hector Jasso-Fuentes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Date post:	06-Mar-2016
Category:	Documents
Upload:	morfismos-department-of-mathematics-cinvestav
View:	215 times
Download:	1 times

Morfismos, Vol 9, No 1, 2005

Documents