+ All Categories
Home > Documents > Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT...

Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT...

Date post: 10-Jul-2020
Category:
Upload: others
View: 0 times
Download: 0 times
Share this document with a friend
33
From CVaR to Uncertainty Set: Implications in Joint Chance Constrained Optimization * Wenqing Chen Melvyn Sim Jie Sun § Chung-Piaw Teo. Abstract In this paper we review the different tractable approximations of individual chance constraint problems using robust optimization on a varieties of uncertainty set, and show their interesting con- nections with bounds on the condition-value-at-risk CVaR measure popularized by Rockafellar and Uryasev. We also propose a new formulation for approximating joint chance constrained problems that improves upon the standard approach. The standard approach decomposes the joint chance constraint into a problem with m individual chance constraints and then applies safe robust opti- mization approximation on each one of them. Our approach builds on a classical worst case bound for order statistics problem, and is applicable even if the constraints are correlated. We provide an application of the model on a network resource allocation network with uncertain demand. The new chance constrained formulation led to more than 8-12% improvement over the standard approach. * The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122 and R-314-000-068-122. NUS Business School, National University of Singapore. Email: [email protected] NUS Business School, National University of Singapore and Singapore MIT Alliance. Email: [email protected] § NUS Business School, National University of Singapore and Singapore MIT Alliance. Email: [email protected] NUS Business School, National University of Singapore and Singapore MIT Alliance. Email: [email protected].
Transcript
Page 1: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

From CVaR to Uncertainty Set: Implications in Joint Chance

Constrained Optimization ∗

Wenqing Chen† Melvyn Sim‡ Jie Sun§ Chung-Piaw Teo. ¶

Abstract

In this paper we review the different tractable approximations of individual chance constraint

problems using robust optimization on a varieties of uncertainty set, and show their interesting con-

nections with bounds on the condition-value-at-risk CVaR measure popularized by Rockafellar and

Uryasev. We also propose a new formulation for approximating joint chance constrained problems

that improves upon the standard approach. The standard approach decomposes the joint chance

constraint into a problem with m individual chance constraints and then applies safe robust opti-

mization approximation on each one of them. Our approach builds on a classical worst case bound

for order statistics problem, and is applicable even if the constraints are correlated. We provide an

application of the model on a network resource allocation network with uncertain demand. The new

chance constrained formulation led to more than 8-12% improvement over the standard approach.

∗The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants

R-314-000-066-122 and R-314-000-068-122.†NUS Business School, National University of Singapore. Email: [email protected]‡NUS Business School, National University of Singapore and Singapore MIT Alliance. Email: [email protected]§NUS Business School, National University of Singapore and Singapore MIT Alliance. Email: [email protected]¶NUS Business School, National University of Singapore and Singapore MIT Alliance. Email: [email protected].

Page 2: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

1 Introduction

Data uncertainties prevail in many real world linear optimization models. If ignored, the so called

“optimal solution” obtained by solving a model using the “nominal data” or point estimates can be-

come infeasible in the model when the true data differs from the nominal one. We consider a linear

optimization model as follows

Z(z) = min c′x

s.t. A(z)x ≥ b(z)

x ∈ X,

(1)

where X ⊆ <n is a polyhedron and the data entries, A(z) ∈ <m×n and b(z) ∈ <m are uncertain and

affinely dependent on a vector of primitive uncertainties, z,

A(z) = A0 +N∑

k=1

Akzk

b(z) = b0 +N∑

k=1

bkzk.

To overcome such infeasibility, Soyster [24] introduced a worst case model that ensures that its solutions

remains feasible for all possible realization of the uncertain data. Soyster proposed the following model

[24],

min c′x

s.t. A(z)x ≥ b(z) ∀z ∈ W,

x ∈ X,

(2)

where

W = {z : −z ≤ z ≤ z} z, z > 0

is the interval support of the primitive uncertainties z. Soyster [24] showed that the model can be rep-

resented as a polynomially sized linear optimization model. However, Soyster’s model can be extremely

conservative in addressing model where the violation of constraints may be tolerated as a tradeoff for

better attainment in objective.

Perhaps the most natural way of safeguarding a constraint against violation is to control its violation

probability. Such a constraint is known as a probabilistic or a chance constraint, which was first

introduced by Charnes, Cooper, and Symonds [11]. A chance constrained model is defined as follows

Zε = min c′x

s.t. P(A(z)x ≥ b(z)) ≥ 1− ε

x ∈ X,

(3)

1

Page 3: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

where the chance constraint requires all the m linear constraints to be jointly feasible with probability

at least 1− ε, where ε ∈ (0, 1) is the desired safety factor.

Chance constrained problems can be classified as individual chance constrained problem when m = 1,

and joint chance constrained problem when m > 1. It is well known that under multivariate normal

distribution, an individual chance constrained problem is second-order cone representable. In other

words, the optimization model becomes a second-order cone optimization problem (SOCP), which is

computationally tractable, both in theory and practice (see for instance Alizadeh and Goldfarb [1]).

However, for general distributions, chance constrained problems are computationally intractable. For

instance, Nemirovski and Shapiro [20] noted that evaluating the distribution of a weighted sum of

uniformly distributed independent random variables is already NP-hard.

Needless to say, joint chance constrained problems are clearly harder than individual chance con-

straint problems. For instance, with only right hand side disturbances, we can transform an individual

chance constrained problem to an equivalent linearly constrained problem. However, this property does

not necessarily apply in a joint chance constrained problem. In fact, convex joint chance constrained

problems are hard to come by. For instance, with only right hand side disturbances, a joint chance

constrained problem is convex only when the distributions is logconcave. Solution techniques for solv-

ing such problems includes supporting hyperplane, central cutting plane and reduced gradient methods

(see for instance Prekopa [22] and Mayer [17].)

The intractability of chance constrained problem using exact probability distributions has spurred

recent interests in robust optimization in which data uncertainties are described using uncertainty sets.

Moreover, robust optimization often requires only a mild assumption on probability distributions such

as known supports, W, covariances and other forms of deviation measures, notably the forward and

backward deviations derived from moment generating functions proposed by Chen, Sim and Sun [13].

For some practitioners, this could be viewed as an advantage over having to obtain the entire joint

probability distributions of the uncertain data. One of the goals of robust optimization is to provide a

tractable approach for obtaining a solution that remains feasible in the chance constrained model (3) for

all distributions that conform to the mild distributional assumption. Hence, such solutions are viewed

as “safe” approximations of the chance constrained problems.

Robust optimization has been rather successful in constructing safe approximation of individual

chance constrained problems. Given an uncertainty set, U , the robust counterpart of an individual

2

Page 4: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

linear constrained problem with affinely dependent primitive uncertainties z is defined as

a(z)x ≥ b(z) ∀z ∈ U .

Clearly, the Soyster’s model (2) is a special case of robust counterpart in which the uncertainty set U is

chosen to be the worst case support set, W. For computational tractability, the chosen uncertainty set

U is usually in the form of tractable convex representable sets such as second-order cone representable

ones and even polytopes. Variants of uncertainty sets include symmetrical ones such as a simple ellipsoid

proposed by Ben-Tal and Nemirovski [3, 4] and independently by El-Ghaoui et al. [15] and a normed

constrained type polytope proposed by Bertsimas and Sim [8]. More recently, Chen, Sim and Sun [13]

proposed an asymmetrical uncertainty set that generalizes the symmetric ones. All these models are

computationally attractive in the form of SOCPs or even in the form linear optimization problems. In

the recent work of Nemirovski and Shapiro [20], they incorporated moment generating functions for

providing safe and tractable approximations of an individual chance constrained problem. Despite the

improved approximation, it is not readily second-order cone representable, and hence computationally

more expensive. Other forms of deterministic approximation of an individual chance constrained prob-

lem includes using Chebyshev’s inequality, Bernstein’s inequality, Hoefding’s inequality to bound the

probability of violating individual constraints. See, for example, Pinter [21].

Besides deterministic approximations of chance constrained problems, there is a body of works on

approximating chance constrained problems using Monte Carlo sampling (see for instance Calafiore and

Campi [10] and Ergodan and Iyengar [14]). However, the solutions obtained via sampling appears to be

conservative compared to those obtained using deterministic approximation. See computation studies

in Nemirovski and Shapiro [20] and also Chen, Sim and Sun [13].

While robust optimization has been pretty successful in approximating individual chance constrained

problems, it is rather unsatisfactory in approximating joint chance constrained problems. The “standard

method” for approximating a joint constrained problem is to decompose a joint chance constrained

problem into a problem with m individual chance constraints. Clearly, by Bonferroni’s inequality, a

sufficient condition for ensuring feasibility in the joint chance constrained problem is to ensure that the

total sum of violation probabilities of the individual chance constraints is less than ε. The natural choice

proposed in Nemirovski and Shapiro [20] and also Chen, Sim and Sun [13] is to divide the violation

probability equality among the m individual chance constraints. To the best of our knowledge, prior

to this work, we do not know of any systematic approach for selecting better allocation of the safety

factors among the individual chance constraints. Unfortunately, this approach ignores the fact that the

3

Page 5: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

individual chance constraints could be correlated, and hence the approximation obtained, using the best

allocation of the safety factors, could be extremely conservative. This motivates our research to achieve

better approximations of joint chance constrained problems. We build instead on a classical result on

order statistics (cf. Meijilson and Nadas [18]) to bound the probability of violation for the joint chance

constraint. We show that by choosing the right multiplers which can be used in conjunction with this

classical inequality, we can derived an improved approximation to the above method (using Bonferroni’s

inequality) for the joint chance constraint problem.

Our specific contributions in this paper include the followings:

1. We review the different tractable approximations of individual chance constraint problems using

robust optimization and show their interesting connections with bounds on the condition-value-

at-risk CVaR measure popularized by Rockafellar and Uryasev [23].

2. We propose a new formulation for approximating joint chance constrained problems that improves

upon the standard approach using Bonferroni’s inequality.

3. We provide an application of the model on a network resource allocation problem with uncertain

demand and study the performance of the new chance constrained formulation over the approach

using Bonferroni’s inequality.

The rest of the paper is organized as follows. In section 2, we focus on robust optimization approx-

imation of individual chance constrained problems. In section 3, we propose a new approximation of

joint chance constrained problem. In Section 4, We anlalyze the efficacy of joint chance constrained

problem through a computational study of emergency supply allocation network. Finally, we conclude

this paper in Section 5.

Notations We denote random variables with tilde sign, such as x. Bold face lower case letters represent

vectors, such as x and bold face upper case letters represent matrices, such as A. In addition, we denote

x+ = max{x, 0}.

2 Individual Chance Constrained Problems

In this section, we will establish the relation between bounds on the condition-value-at-risk (CVaR)

measure popularized by Rockafellar and Uryasev [23] and the different tractable approximations of

individual chance constrained problems using robust optimization. For simplicity, we consider a linear

4

Page 6: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

individual chance constraint as follows

P(y(z) ≤ 0)

)≥ 1− ε, (4)

where y(z) are affinely dependent of z,

y(z) = y0 +N∑

k=1

ykzk,

and (y0, y1, . . . , yN ) are the decision variables. To illustrate the generality, we can represent the following

chance constrained problem

P(a(z)′x ≥ b(z))

)≥ 1− ε,

where

a(z) = a0 +N∑

k=1

akzk

b(z) = b0 +N∑

k=1

bkzk,

by enforcing the following affine relations

yk = −ak ′x + bk ∀k = 0, . . . , N.

The chance constrained problem (4) is not necessarily convex in its decision variables, (y0, y1, . . . , yN ).

A step towards tractability is by convexifying the chance constrained problem (4) using the conditional-

value-at-risk (CVaR) measure, ρ1−ε(v), which is a functional on a random variable v defined as follows

ρ1−ε(v) ∆= minβ

{β +

1εE

((v − β)+

)}.

The CVaR measure is a special class of optimized certainty equivalent (OCE) risk measure introduced

by Ben-Tal and Teboulle [6] and is popularized by Rockafellar and Uryasev [23] as a tractable alternative

for solving value-at-risk problems in financial applications. Recent works of Bertsimas and Brown [7]

and Natarajan et al. [19] have uncovered the relation between financial risk measures and uncertainty

sets in robust optimization. Our interest in CVaR measure is due to Nemirovski and Shapiro [20], who

have established that the following CVaR constrained problem,

ρ1−ε(y(z)) ≤ 0 (5)

is the tightest convex approximation of an individual chance constrained problem. However, despite its

convexity, it remains unclear how we can evaluate the CVaR measure precisely. The key difficulty lies in

5

Page 7: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

the evaluation of the expectation, E((·)+), which involves multi-dimension integration. Such evaluation

is typically analytically prohibitive above the forth dimension. Although it is possible to approximate

CVaR using sampling average approximation, its solution may not be a safe approximation of the chance

constrained problem (4). Furthermore, sampling average approximation of the CVaR measure relies on

full knowledge of the underlying distributions, z, which may become a practical concern due to the

limited availability of independent stationary historical data.

2.1 Bounding E((·)+)

Providing bounds on E((·)+) is pivotal in developing tractable approximations of individual and joint

chance constrained problems. We show next different ways of bounding E((·)+) using mild distributional

information of z, such as supports, covariances and deviation measures. The results in bounding E((·)+)

has also been presented in Chen and Sim [12]. For completeness, we list some of the known bounds on

E((·)+).

The primitive uncertainties, z may be partially characterized using the forward and backward de-

viations, which are recently introduced by Chen, Sim and Sun [13].

Definition 1 Given a random variable z with zero mean, the forward deviation is defined as

σf (z) ∆= supθ>0

{√2 ln(E(exp(θz)))/θ2

}(6)

and backward deviation is defined as

σb(z) ∆= supθ>0

{√2 ln(E(exp(−θz)))/θ2

}. (7)

The forward and backward deviations are derived from the moment generating functions of z and can

be bounded from the support of z.

Theorem 1 ( Chen, Sim and Sun [13]) If z has zero mean and distributed in [−z, z], z, z > 0, then

σf (z) ≤ σf (z) =z + z

2

√g

(z − z

z + z

)

and

σb(z) ≤ σb(z) =z + z

2

√g

(z − z

z + z

),

where

g(µ) = 2 maxs>0

{φµ(s)− µs

s2

},

6

Page 8: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

and

φµ(s) = ln

(es + e−s

2+

es − e−s

).

Moreover the bounds are tight.

Assumption U: We assume that the uncertainties {zj}j=1:N are zero mean random variables, with

positive definite covariance matrix, Σ. Let W = {z : −z ≤ z ≤ z} denote the smallest compact convex

set containing the support of z. Of the N primitive uncertainties, the first I random variables, that

is, zj , j = 1, . . . , I are stochastically independent. Moreover, the corresponding forward and backward

deviations (or their bounds used in Theorem 1) are given by pj = σf (zj) > 0 and qj = σb(zj) > 0

respectively for j = 1, . . . , I, and we denote P = diag(p1, . . . , pI) and Q = diag(q1, . . . , qI).

Theorem 2 (Chen and Sim [12]) Suppose the primitive z satisfies Assumption U. The following func-

tions are upper bounds of E ((y0 + y′z)+).

(a)

E ((y0 + y′z)+) ≤ π1(y0,y) ∆=(

y0 + maxz∈W

z′y)+

= minr,s, t

{r | r ≥ y0 + s′z + t′z, s− t = y, s, t ≥ 0, r ≥ 0

}.

The bound is tight whenever y0 + y′z ≤ 0 for all z ∈ W.

(b)

E ((y0 + y′z)+) = y0 + E ((−y0 − y′z)+)

≤ π2(y0, y)∆= y0 +

(−y0 + max

z∈W(−y)′z

)+

= minr,s, t

{r | r ≥ s′z + t′z, s− t = −y, s, t ≥ 0, r ≥ y0

}.

The bound is tight whenever y0 + y′z ≥ 0 for all z ∈ W.

(c)

E ((y0 + y′z)+) = 12 (y0 + E|y0 + y′z)|) ≤ π3(y0, y) ∆= 1

2y0 + 12

√y0

2 + y′Σy.

(d)

E ((y0 + y′z)+) ≤ infµ>0µe E

(exp(y0+y′z

µ ))

≤ π4(y0,y) ∆=

infµ>0

{µe exp

(y0

µ + ‖u‖222µ2

)}if yj = 0 ∀j = I + 1, . . . , N

+∞ otherwise

where uj = max{pjyj ,−qjyj}.

7

Page 9: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

(e)

E ((y0 + y′z)+) ≤ y0 + infµ>0µe E

(exp(−y0−y′z

µ ))

≤ π5(y0, y) ∆=

y0 + infµ>0

{µe exp

(− y0

µ + ‖v‖222µ2

)}if yj = 0 ∀j = I + 1, . . . , N

+∞ otherwise

where vj = max{−pjyj , qjyj}.

Remark : Observe that πi(y0,y), i = 1, . . . , 5 are convex and positive homogenous functions, that is,

πi(ky0, ky) = kπi(y0, y) ∀k ≥ 0. (8)

Furthermore,

πi(y0,0) = y+0 . (9)

More importantly, Chen and Sim [12] shows that the bound can be strengthened further by suitably

decomposing (y0, y) into (yi0, y

i), and using a linear combination of the bounds πi(yi0, y

i) to obtain a

stronger bound.

Theorem 3 (Chen and Sim [12]) Suppose πi(y0, y), for all i ∈ L, is an upperbound to E(y0 + y′z)+,

πi(y0, y) is convex and positive homogenous. Define

πL(y0, y) ∆= minyl0,yl

l∈Lπl(yl0, yl)

s.t.∑

l∈Lyl0 = y0

l∈Lyl = y.

Then

0 ≤ E((y0 + y′z)+

) ≤ πL(y0, y) ≤ minl∈L

πl(y0, y). (10)

Moreover, πL(y0, y) inherits the convexity and positive homogenous properties of the individual functions

πi(y0, y), i ∈ L.

For details, the interested reader may refer to Chen and Sim [12].

Proposition 1 Under Assumption U and suppose π(y0,y) is an upperbound to E(y0 + y′z)+, then

π(y0, y) = 0 (11)

only if

y0 + maxz∈W

y′z ≤ 0. (12)

8

Page 10: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

Proof : Note that

0 = π(y0, y) ≥ E((y0 + y′z)+

) ≥ 0.

Suppose

y0 + y′z∗ = y0 + maxz∈W

y′z > 0

for some z∗ ∈ W. Since the objective function is linear, we can assume WLOG that z∗ is an extreme

point in W.

Let Bε(z∗) denote an open ball with radius ε around z∗, with

y0 + y′z > 0 for all z ∈ Bε(z∗).

Since E ((y0 + y′z)+) = 0, we must have

P (Bε(z∗)) = 0.

Thus the support for z lies in the convex hull W ′ of the (closed) set W \ Bε(z∗). z∗ /∈ W ′ since it is

an extreme point in W. This contradicts our earlier assumption that W denote the smallest convex set

containing the support for z.

2.2 Bounds on CVaR and Robust Optimization

There are several attractive proposals of robust optimization that approximate individual chance con-

strained problems which we have mentioned. In such a proposal, the solution, (y0, y) to the following

robust counterpart

y0 + maxz∈U

y′z ≤ 0

guarantees that

P(y0 + y′z ≤ 0) ≥ 1− ε. (13)

Clearly, the choice of uncertainty set depends on the underlying assumption of primitive uncertainty.

Another approach of approximating the chance constraint problem is to provide an upper bound of

ρ1−ε(y0 +y′z), so that if the bound is nonnegative, the chance constraint (13) will also be satisfied. For

a given upperbound π(y0, y) to E(·)+, we define

η1−ε(y0, y) ∆= minβ

{β +

1επ(y0 − β, y)

}.

Clearly,

ρ1−ε(y0 + y′z) = minβ

{β +

1εE

((y0 + y′z − β)+

)} ≤ η1−ε(y0,y)

9

Page 11: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

and a sufficient condition for satisfying (13) is

η1−ε(y0,y) ≤ 0. (14)

Note that if the epigraph of π(·, ·) can be approximated by a second-order cone, the constraint (14) is

also approximately second-order cone representable.

We show next that the two appraoches are esentially equivalent.

Theorem 4 Under Assumption U and suppose π(y0, y) is an upperbound to E(y0 + y′z)+, π(y0, y) is

convex and positive homogenous, with π(y0,0) = y+0 , then

η1−ε(y0, y) = y0 + maxz∈U(ε)

y′z.

for some convex uncertainty set U(ε).

Proof : The set {(u, y0, y) : u ≥ π(y0, y)} is a convex cone as it is the epigraph of a convex positive

homogeneous function. K ∆= cl {(u, y0, y) : u ≥ π(y0, y)} is thus a closed convex cone. We show next

that K is pointed.

Note that E(y0 + y′z)+ = 0 ensures that y0 + maxz∈W y′z ≤ 0 under Assumption U. Suppose

(u, y0, y) ∈ K and (−u,−y0,−y) ∈ K. Since π(y0,y) ≥ 0 and π(−y0,−y) ≥ 0, we must have u ≥0,−u ≥ 0, i.e., u = 0. This forces π(y0, y) = π(−y0,−y) = 0. Hence by Proposition 1,

y0 + maxz∈W

y′z ≤ 0, −y0 + maxz∈W

(−y)′z ≤ 0.

This is possible only when y0 = 0 and y = 0. Hence K is a pointed cone.

The dual cone by K∗ ∆= {(v, z0, z) : (v, z0,z) · (u, y0, y) ≥ 0 ∀ (u, y0,y) ∈ K} is thus also a closed

convex pointed cone. Thus both K and K∗ have non-empty interior.

Suppose

η1−ε(y0, y) = minβ{β + π(y0 − β, y)/ε}

is unbounded. Let {βn} be a sequence of numbers with limn→∞ βn = −∞, and limn→∞ {βn + π(y0 − βn, y)/ε} =

−∞. We may assume βn < 0 for all n.

limn→∞ {βn + π(y0 − βn, y)/ε} = lim

n→∞

{βn + (−βn)π

(y0

−βn+ 1,

y

−βn

)/ε

}= ∞,

since

limn→∞π(

y0

−βn+ 1,

y

−βn) = π(1,0) = 1,

10

Page 12: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

and ε < 1. This is a contradiction. Thus η1−ε(y0, y) must be bounded.

We have, using strong duality theorem, that

η1−ε(y0, y) = minβ,u

{β + u/ε : (u, y0 − β, y) ∈ K}

= minβ,u

{β + u/ε : (u,−β,0) ÂK (0,−y0,−y)}

= max{y0z0 + y′z : (v,−z0,−z) ∈ K∗, v = 1/ε, z0 = 1

}

= max{y0 + y′z : (1/ε,−1,−z) ∈ K∗}

Hence

η1−ε(y0, y) = y0 + maxz∈U(ε)

y′z,

with

U(ε) ∆= {z : (1/ε,−1,−z) ∈ K∗} .

For the functions πi(y0,y), i = 1, . . . , 5, the corresponding uncertainty sets can be computed explic-

itly. Consider the following uncertainty sets:

U1(ε)∆= W,

U2(ε)∆= {z | z = (1− 1/ε)ζ, for some ζ ∈ W} ,

U3(ε)∆=

{z | ‖z‖2 ≤

√1− ε

ε

},

U4(ε)∆=

{z | ∃s, t ∈ <I , (z1, . . . , zI) = s− t, ‖P−1s + Q−1t‖ ≤

√−2 ln ε

},

U5(ε)∆=

{z | ∃s, t ∈ <I , (z1, . . . , zI) = s− t, ‖Q−1s + P−1t‖ ≤ 1− ε

ε

√−2 ln(1− ε)

}.

Corollary 1

ηi1−ε(y0, y) ∆= min

β

{β +

1επi(y0 − β, y)

}= y0 + max

z∈Ui(ε)y′z.

Proof :

Uncertainty Set U1:

η11−ε(y0,y) = min

β

(β +

π1(y0 − β, y)ε

)

= minβ

(β +

1ε(y0 − β + max

z∈Wy′z)+

)

= y0 + maxz∈U1

y′z.

11

Page 13: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

Uncertainty Set U2:

η21−ε(y0,y) = min

β

(β +

π2(y0 − β, y)ε

)

= y0 + minβ

(β +

π2(−β, y)ε

)

= y0 + minβ

{β +

((maxz∈W

(−y)′z + β

)+

− β

)}

= y0 + minβ

{β(1− 1/ε) +

((maxz∈W

(−y)′z + β)+)}

= y0 + (1/ε− 1)minβ

{−β +

11− ε

((maxz∈W

(−y)′z + β)+)}

= y0 + (1/ε− 1)maxz∈W

y′(−z) + (1/ε− 1) minβ

(−β +

11− ε

(β)+)

= y0 + maxz∈U2

y′z.

Uncertainty Set U3:

η31−ε(y0,y) = min

β

(β +

π3(y0 − β, y)ε

)

= minβ

(β +

y0 − β +√

(y0 − β)2 + y′Σy

)

= y0 +√

1− ε

ε

√y′Σy

= y0 + maxz∈U3

y′z,

where the second equality follows from choosing the optimum β,

β∗ = y0 +√

y′Σy(1− 2ε)2√

ε(1− ε).

Uncertainty Set U4:

For notational convenience, we denote

yI = (y1, . . . , yI)

yI = (yI+1, . . . , yN ).

η41−ε(y0, y) = min

β

(β +

π4(y0 − β, y)ε

)

= minβ,µ,u

(β +

µe exp(y0−β

µ + ‖u‖222µ2 )

2ε| u ≥ PyI ,u ≥ −QyI , yI = 0

)

= minµ,u

(y0 +

‖u‖22

2µ2− µ ln ε | u ≥ PyI , u ≥ −QyI , yI = 0

)

= minu

(y0 +

√−2 ln εu0 | P−1u ≥ yI , Q

−1u ≥ −yI , yI = 0, ‖u‖2 ≤ u0

)

= y0 + maxz∈U4

y′z,

12

Page 14: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

where the second and third equalities follow from choosing the tightest β∗ and µ∗, that is

β∗ = y0 +‖u‖2

2

2µ2− µ ln ε− µ,

µ∗ =‖u‖2√−2 ln ε

.

The last equality is the result of strong conic duality and has been derived in Chen, Sim and Sun [13].

Uncertainty Set U5:

Following from the above exposition,

η51−ε(y0,y) = min

β

(β +

π5(y0 − β, y)ε

)

= minβ,µ,v

(β +

y0 − β + µe exp(−y0−β

µ + ‖v‖222µ2 )

2ε| v ≥ −PyI , v ≥ QyI ,yI = 0

)

= minµ,v

(y0 + (

1ε− 1)(

‖v‖22

2µ2− µ ln(1− ε)) | v ≥ −PyI , v ≥ QyI ,yI = 0

)

= minv

(y0 +

1− ε

ε

√−2 ln(1− ε)‖v‖ | | P−1v ≥ −yI , Q

−1v ≥ yI , yI = 0)

= y0 + maxz∈U5

y′z.

We show next that the uncertainty set corresponding to the stronger bound πL(y0, y) can also be

obtained in similar way.

Proposition 2 Let Ui, i ∈ L, be compact uncertainty sets such that their intersections

UL =⋂

i∈LUi,

has a non-empty interior. Then

maxz∈UL

y′z = minyi,i∈L

(∑

i∈Lmaxzi∈Ui

y′izi |∑

i∈Lyi = y

).

Proof : We observe that the problem

max y′z

s.t. z ∈ ULis equivalently

max y′z

s.t. zi = z

zi ∈ Ui ∀i ∈ L.

(15)

13

Page 15: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

By strong duality, we have

maxz{y′z : z = zi, i ∈ L}

= minyi,i∈L

{∑

i∈Ly′izi :

i∈Lyi = y

}.

Hence, the problem (15) is equivalent to

maxz∈UL

y′z = maxzi∈Ui,i∈L

{min

yi,i∈L

{∑

i∈Ly′izi |

i∈Lyi = y

}}.

Observe the set UL is a compact set with nonempty interior. Hence, maxz∈U y′z is therefore finite.

Furthermore, there exists finite optimal primal and dual solutions zi and yi, i ∈ L that satisfy strong

duality. Hence, we can exchange “max” with “min”, so that

maxz∈U

y′z = minyi,i∈L

{max

zi∈Ui,i∈L∑

i∈Ly′izi |

i∈Lyi = y

}

= minyi,i∈L

{∑

i∈Lmaxzi∈Ui

y′izi |∑

i∈Lyi = y

}.

Theorem 5 Suppose z satisfies Assumption U. Let

UL(ε) ∆=⋂

l∈LUl(ε).

and suppose UL(ε) has an non-empty interior. Then

ηL1−ε(y0, y) = y0 + maxz∈UL(ε)

y′z.

Proof :

For notational convenience, we ignore the representation of uncertainty sets as functions of ε. Observe

that for any ε ∈ (0, 1), the sets Ui(ε) are compact and contain 0 in their interiors.

ηL(y0,y) = minβ

(β +

πL(y0 − β, y)ε

)

= minβ,yl0,yl,l∈L

β +

l∈L

(πl(yl0 − βl, yl)ε

)|

l∈Lyl = y,

l∈Lyl0 = y0,

l∈Lβl = β

= minyl0,yl,l∈L

l∈Lminβl

(βl +

πl(yl0 − βl,yl)ε

)|

l∈Lyl = y,

l∈Lyl0 = y0

= minyl0,yl,l∈L

l∈L

(yl0 + max

z∈Ul(ε)y′lz

)|

l∈Lyl = y,

l∈Lyl0 = y0

= y0 + minyl,l∈L

l∈L

(max

z∈Ul(ε)y′lz

)|

l∈Lyl = y

= y0 + maxz∈UL(ε)

y′z,

14

Page 16: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

where the last inequality is due to Proposition 2.

Hence, the different approximations of individual chance constrained problems using robust opti-

mization are the consequences of applying different bounds on E((·)+). Notably, when the primitive

uncertainties are characterized only by their means and covariance, the corresponding uncertainty set

is an ellipsoid of the form U3. See, for instance, Bertsimas et al. [9] and El-Ghaoui et al. [16]. When

I = N , that is all the primitive uncertainties are independently distributed, Chen, Sim and Sun [13]

proposed the asymmetrical uncertainty set

UA(ε) = W︸︷︷︸=U1(ε)

⋂U4(ε),

which generalizes the uncertainty set proposed by Ben-Tal and Nemirovski [5]. Noting that UA(ε) ⊆U{1,2,4,5}(ε), we can therefore improve upon the approximation using the uncertainty set U{1,2,4,5}(ε).

However, in most application of chance constrained problems, the safety factor, ε is relatively small. In

which case, the uncertainty sets of U2(ε) and U5(ε) are usually exploded to engulf the uncertainty sets

of W and U4(ε), respectively . For instance, under symmetric distributions, that is P = Q and z = z,

it is easy to establish that for ε < 0.5, we have

U{1,2,4,5}(ε) = U1(ε)︸ ︷︷ ︸=W

⋂U2(ε)︸ ︷︷ ︸⊇W

⋂U4

⋂U5︸︷︷︸⊇U4

= UA(ε).

3 Joint Chance Constrained Problem

Unfortunately, the notion of uncertainty set in classical robust optimization does not carry forward as

well in addressing joint chance constrained problems. We consider a linear joint chance constraint as

follows,

P(yj(z) ≤ 0, j ∈M

)≥ 1− ε, (16)

where M = {1, . . . ,m}, yj(z) are affinely dependent of z,

yj(z) = y0j +

N∑

k=1

ykj zk j ∈M.

(y01, . . . , y

N1 , . . . , y0

m, . . . , yNm) being the decision variables. For notational convenience, we represent

yj = (y1j , . . . , y

Nj ),

so that yi(z) = y0i + y′iz and denote

Y = (y01, . . . , y

N1 , . . . , y0

m, . . . , yNm),

15

Page 17: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

as the collection of decision variables in the joint chance constrained problem. By suitable affine con-

straints imposed on the decision variables Y and x, we can represent the joint chance constraint in

Model (3) in the form of constraint (16).

It is not surprising that a joint chance constraint is more difficult to solve than an individual one.

For computational tractability, the common approach is to decompose the joint constrained problem

into a problem with m individual constraints of the form

P(yi(z) ≤ 0

)≥ 1− εi, i ∈M. (17)

By enforcing Bonferroni’s inequality on their safety factors,

i∈Mεi ≤ ε. (18)

any feasible solution that satisfies the set of individual chance constrained problem will also satisfy

the corresponding joint chance constrained problem. See for instance, Chen, Sim and Sun [13] and

Nemirovski and Shapiro [20]. Consequently, using the techniques discussed in the previous section, we

can then build tractable safe approximations as follows

η1−εi(y0i , yi) ≤ 0, i ∈M. (19)

The main issue with using Bonferroni’s inequality is the choice of εi. Unfortunately, the problem

becomes non-convex and possibly intractable if εi are made variables and enforcing the constraint (18)

as part of the optimization model. As such, it is natural to choose, εi = ε/m as proposed in Chen, Sim

and Sun [13] and Nemirovski and Shapiro [20].

In some instances, Bonferroni’s inequality may be rather conservative even for an optimal choice of

εi. For instance, suppose yi(z) are completely correlated, such as

yi(z) = δi(a0 + a′z), i ∈M (20)

for some δi > 0, the least conservative choice of εi is εi = ε for all i ∈ M, which would violate the

condition (18) imposed by Bonferroni’s inequality. As a matter of fact, it is easy to see that the least

conservative choice of εi while satisfying Bonferroni’s inequality is εi = ε/m for all i = 1, . . . ,m. Hence,

if yi(z) are correlated, the efficacy of Bonferroni’s inequality will possibly diminish.

We propose a new tractable way for approximating the joint chance constraint problem. Given

a vector of positive constants, α ∈ <N , α > 0, an index set J ⊆ M, an upperbound π(y0, y) for

16

Page 18: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

E ((y0 + y′z)+), we define the following function,

γ1−ε(Y , α,J ) ∆= minw0,w

minβ

(β +

1επ(w0 − β, w)

)

︸ ︷︷ ︸=η1−ε(w0,w)

+1ε

i∈Jπ(αiy

0i − w0, αiyi −w)

.

The next result shows we can use the above function to approximate a joint chance constrained problem.

Theorem 6 (a) Suppose z satisfies Assumption U, then

ρ1−ε

(maxi∈J

{αiyi(z)})≤ γ1−ε(Y , α,J ).

Consequently, the joint chance constraint (16) is satisfied if

γ1−ε(Y ,α,J ) ≤ 0 (21)

and

y0i + max

z∈Wy′iz ≤ 0 ∀i ∈M\J . (22)

(b) For fixed α, the epigraph of the function γ1−ε(Y , α,J ) with respect to Y is second-order cone rep-

resentable and positive homogenous. Similarly, for a fixed Y , the epigraph of the function γ1−ε(Y , α,J )

with respect to α is second-order cone representable and positive homogenous.

Proof : (a) Under Assumption U, the set W is the support of the primitive uncertainty, z, hence, the

robust counterpart (22) implies

P(y0i + y′iz > 0) = 0, ∀i ∈M\J .

Hence, with α > 0, we have

P(y0

i + y′iz ≤ 0, i ∈M)

= P(y0

i + y′iz ≤ 0, i ∈ J)

= P(

maxi∈J

{αiy0i + αiy

′iz} ≤ 0

).

Therefore, it suffices to show that if Y is feasible in the constraint (21), then the CVaR measure,

ρ1−ε

(maxi∈J

{αiyi(z)})≤ 0.

Using the classical inequality (cf. Meijison and Nadas [18]) that

E(

maxi=1,...,n

Xi − β

)+

≤ E (Y − β)+ +n∑

i=1

E (Xi − Y )+ , for any r.v. Y, (23)

17

Page 19: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

we have

ρ1−ε

(maxi∈J

{αi(y0i + y′iz)}

)

= minβ

{β +

1εE

((maxi∈J

{αi(y0i + y′iz)} − β

)+)}

≤ minβ,w0,w

β +

E

((w0 − β + w′z)+

)+

i∈JE

((αy0

i − w0 + (αiyi −w)′z)+)

≤ minβ,w0,w

β +

π(w0 − β, w) +

i∈Jπ(αy0

i − w0, αiyi −w)

= γ1−ε(Y , α,J ) ≤ 0.

(b) For a fixed α, the corresponding epigraph can be expressed as

Y1 = {(Y , t) : γ1−ε(Y , α,J ) ≤ t} =

(Y , t) :

∃w0, r0, . . . , rm ∈ <, w ∈ <N

r0 + 1ε

∑i∈J ri ≤ t

η1−ε(w0, w) ≤ r0

π(αiy0i − w0, αiyi −w) ≤ ri ∀i ∈ J

.

Since the epigraphs of η1−ε(·, ·) and π(·, ·) are second-order cone representable, the set Y1 is also second-

order cone representable. For positive homogeneity, we observe that since π(·, ·) is positive homogenous,

we have that for all k ≥ 0,

γ1−ε(kY ,α,J )

= minβ,w0,w

β +

π(w0 − β, w) +

i∈Jπ(kαiy

0i − w0, kαiyi −w)

= k minβ,w0,w

1

kβ +

π

(1kw0 − 1

kβ,

1kw

)+

i∈Jπ

(αiy

0i −

1kw0, αiyi −

1kw

)

= k minβ,w0,w

β +

π(w0 − β, w) +

i∈Jπ(αiy

0i − w0, αiyi −w)

= kγ1−ε(Y ,α,J ).

Similarly, the same exposition applies when Y is fixed and α being the decision variable.

Remark : Note that the constraints (22) do not depend on the values of αj for all j ∈M\J . Speaking

intuitively, we can perceive αj = ∞ for all j ∈ M\J . However, to avoid dealing with infinite entities,

we define the set J as part of the input to the function γ1−ε(·, ·, ·). Throughout this paper, we will

restrict the focus of α to only elements corresponding to the indices in the set J . Unfortunately, the

function γ1−ε(Y , α,J ) is not jointly convex in both Y and α. Nevertheless, for a given Y , it is a

tractable convex function with respect to α and in the attractive form of SOCP. We will later exploit

this property for improving the choice of α.

18

Page 20: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

The inequality obtained using (23) is tight when the variables yi(z), i = 1, . . . , n are negatively

correlated. More specifically, if the sets

Si∆= {z : yi(z) ≥ β} , i = 1, . . . , n

are mutually disjoint, then

E(

maxi

yi(z)− β

)+

=n∑

i=1

E (yi(z)− β)+ ,

and hence the inequality (23) cannot be tighten further substantially. Interestingly, by introducing the

parameters α and random variable w0 +w′z, our approach is also able to handle the situation when the

variables are positively correlated. In the example (20) where yi(z), i ∈ M are completely positively

correlated, the following condition

η1−ε(a0, a) ≤ 0

is also sufficient to guarantee feasibility in the joint chance constrained problem. Choosing αi = 1/δi > 0,

we see thatγ1−ε(Y ,α,M)

= minw0,w

(η1−ε(w0,w) +

{ ∑

i∈Mπ(αiy

0i − w0, αiyi −w)

})

= minw0,w

(η1−ε(w0,w) +

{ ∑

i∈Mπ(αiδia

0 − w0, αiδia−w)

})

≤ η1−ε(a0,a) +1ε

{ ∑

i∈Mπ(a0 − a0, a− a)

}

= η1−ε(a0,a) ≤ 0.

Therefore, we see that the new bound is potentially better than the application of Bonferroni’s inequality

on individual chance constraints. By choosing the right combination of (α,J ), we can prove a stronger

result as follows.

Theorem 7 Let εi ∈ (0, 1), i ∈M and∑

i∈M εi ≤ ε. Suppose Y satisfies

η1−εi(y0i , yi) ≤ 0 ∀i ∈M,

then there exists α > 0, and a set J ⊆ M such that (Y , α,J ) are feasible in the constraints (21) and

(22).

Proof : Let βi be the optimal solution to

minβ

(β +

1εi

(π(y0

i − β, yi)))

︸ ︷︷ ︸=η1−εi

(y0i ,yi)

.

19

Page 21: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

Since η1−εi(y0i ,yi) ≤ 0 and that

π(y0i − βi, yi) ≥ E

((y0

i − βi + y′iz)+)≥ 0,

we must have βi ≤ 0. Let J = {i|βi < 0},

αj = − 1βj

∀j ∈ J .

Since βj = 0 for all j ∈M\J , we have

0 ≤ π(y0i , yi) ≤ 0 ∀i ∈M\J

From Proposition 1, it follows that

y0i + y′iz ≤ 0 ∀z ∈ W, ∀i ∈M\J

which satisfies the set of inequalities in (22).

For i ∈ J , the constraint η1−εi(y0i , yi) ≤ 0 is equivalent to

1−βi

π(y0i − βi, yi) ≤ εi

Since the function π(·, ·) is positive homogenous, we have

1−βi

π(y0i − βi, yi) = π

(1−βi

y0i + 1,

1−βi

yi

)

= π(αiy

0i + 1, αiyi

)

≤ εi ∀i ∈ J .

Finally,

γ1−ε(Y , α,J )

= minβ,w0,w

β +

π(w0 − β,w) +

i∈Jπ(αiy

0i − w0, αiyi −w)

≤ −1 +1ε

π(−1 + 1,0) +

i∈Jπ(αiy

0 + 1, αiy − 0)

= −1 +1ε

i∈Jπ(αiy

0 + 1, αiy)

≤ −1 +1ε

i∈Jεi ≤ 0,

where the first inequality is due to the choice of β = −1, w0 = −1, w = 0 and the last inequality

follows from∑

i∈M εi ≤ ε.

20

Page 22: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

3.1 Optimizing over α

Consider a joint chance constrained model as follows

Zε = min c′x

s.t. P(yi(z) ≤ 0, i ∈M) ≥ 1− ε

(x, Y ) ∈ X,

(24)

in which X is efficiently computable convex set, such as a polyhedron or a second-order cone repre-

sentable set. Given a set of constant, α > 0 and a set J , we consider the following optimization

model.Z1

ε (α,J ) = min c′x

s.t. γ1−ε(Y , α,J ) ≤ 0

y0i + max

z∈Wy′iz ≤ 0 ∀i ∈M\J

(x, Y ) ∈ X.

(25)

Under Assumption U, suppose Model (25) is feasible, the solution (x, Y ) is also feasible in Model (24),

albeit more conservatively.

The main concern here is how to choose α and J . A likely choice, is say αj = 1/m, for all j ∈ Mand J = M. Alternatively, we may use the classical approach by decomposing into m individual chance

constraint problem with εi = ε/m. Base on Theorem 7, we can find a feasible α > 0 and set J such

that Model (25) is also feasible.

Our aim is to improve upon the objective by minimizing γ1−ε(Y , α,J ) over αj , j ∈ J , resulting in

greater slack in the model (25). Hence, this approach will lead to improvement in the objective, or at

least will not increase the value.

Given a feasible solution, Y in Model (25), our aim is to improve upon the objective by readjusting

the set J and the weights αj , j ∈ J , that will result in greater slack in the model (25) over the solution,

Y . We define the following set,

K(Y ) ∆={

i : y0i + max

z∈Wy′iz > 0

}.

Note that we can obtain the set K(Y ) by solving the following linear optimization problem

minm∑

i=1

si

s.t. y0i + max

z∈Wy′iz ≤ si,

(26)

so that K(Y ) = {i : s∗i > 0}, s∗ being its optimal solution.

21

Page 23: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

Since Y is feasible in Model (25), we must have K(Y ) ⊆ J . If the set K(Y ) is nonempty, we

consider the following optimization problem over αj , j ∈ K(Y ),

Z1α(Y ) = min γ1−ε(Y , α,K(Y ))

s.t.∑

j∈K(Y )

αj = 1

αj ≥ 0 ∀j ∈ K(Y ).

(27)

By choosing π(y0,y) ≤ π1(y0, y), we can ensure that the objective function of Problem (27) is finite.

Moreover, since the feasible region of Problem (27) is compact, the optimal solution for αj , j ∈ K(Y )

is therefore achievable.

Proposition 3 Assume there exists (Y , α,J ), α > 0, such that γ1−ε(Y ,α,J ) ≤ 0. Let α∗ be the

optimum solution of Problem (27).

(a)

Z1α(Y ) ≤ 0.

(b) Moreover, the solution α∗ satisfies,

α∗i > 0 ∀i ∈ K(Y ).

Proof : (a) Since K(Y ) ⊂ J , and under the assumption that there exists (Y ,α,J ), α > 0, such that

γ1−ε(Y , α,J ) ≤ 0, by using the same α, we observe that

γ1−ε(Y ,α,K) ≤ γ1−ε(Y ,α,J ) ≤ 0.

Due to the positive homogenous property of Theorem 6(b), we scale α by a positive constant so that it

is feasible in Problem (27). Hence, the result follows.

(b) Suppose there exists a nonempty set G ⊂ K(Y ) such that α∗i = 0,∀i ∈ G, we will show that the

following holds,

y0i + max

z∈Wy′iz ≤ 0 ∀i ∈ K(Y )\G,

which is a contradiction. We have argued that Z1α(Y ) ≤ 0. Let k ∈ G, that is, α∗k = 0. Observe that

22

Page 24: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

for some suitably chosen (β, w0,w),

0 ≥ γ1−ε(Y , α∗,K(Y ))

= β +1ε

π(w0 − β, w) +

i∈K(Y )

π(α∗i y0i − w0, α

∗i yi −w)

= β +1ε{π(w0 − β, w) + π(−w0,−w)}+

i∈K(Y )\{k}π(α∗i y

0i − w0, α

∗i yi −w)

≥ β +1ε

{E

(w0 + w′z − β

)+ + E(−w0 −w′z

)+}

≥ β + 1ε (−β)+,

where the second equality is due to α∗k = 0. Since, ε ∈ (0, 1), the inequality β + 1ε (−β)+ ≤ 0 is satisfied

if and only if β = 0. We now argue that

π(y0i , yi) = 0 ∀i ∈ K(Y )\G (28)

which, from Proposition 1, implies

y0i + max

z∈Wy′iz ≤ 0 ∀i ∈ K(Y )\G.

Indeed, for any l ∈ K(Y )\G, we observe that

0 ≥ β +1ε

π(w0 − β, w) +

i∈K(Y )

π(α∗i y0i − w0, α

∗i yi −w)

=1ε

π(w0, w) +

i∈K(Y )

π(α∗i y0i − w0, α

∗i yi −w)

Substituting β = 0,

≥ 1ε

{π(w0,w) + π(α∗l y

0l − w0, α

∗l yl −w)

}

≥ 1ε

{π(α∗l y

0l , α

∗l yl)

}

=α∗lε

π(y0l , yl) ≥ 0.

Hence, the equality (28) is achieved by noting that α∗l > 0.

We propose an algorithm for improving the choice of α and the set J . Again, we assume that we

can find an initial feasible solution of Model (25).

Algorithm 1 .

Input: Y

1. Solve Problem (26) with Input Y . Obtain optimal solution s∗.

23

Page 25: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

2. Set K(Y ) := {i|s∗j > 0, j ∈M}.

3. Solve Problem (27) with Input Y . Obtain optimal solution α∗. Set J := K(Y ).

4. Solve Model (25) with Input (α,J ). Obtain optimal solution (x∗, Y ∗). Set Y := Y ∗.

5. Repeat Step 1 until a termination criterion is met or until J = ∅.

6. Output solution (x∗,Y ∗).

Theorem 8 In Algorithm 1, the sequence of objectives obtained by solving Model (25) is non-increasing.

Proof : Starting with a feasible solution of Model (25), we are assured that there exists (Y , α,J ),

α > 0, such that γ1−ε(Y , α,J ) ≤ 0. With Proposition 3(b), the condition in Step 3 ensures that

α∗j > 0 for all j ∈ J . Moreover, Proposition 3(a) ensures that the updates on α and J do not affect the

feasibility of its previous solution (x, Y ) in the Model (25). Hence, its objective value will not increase.

The implementation of Algorithm 1 may involve perpetual updates of the set J and result in

reformulating Problem 25. A practical solution is to ignore the set J and solve the following model,

Z2ε (α) = min c′x

s.t. γ1−ε(Y ,α,M) ≤ 0

(x,Y ) ∈ X,

(29)

for a given α ≥ 1 such that 1′α = M , where M is a large number. The updates of α is done by solving

Z2α(Y ) = min γ1−ε(Y , α,M)

s.t.∑

j∈J αj = M

α ≥ 1.

(30)

The algorithm is also simplified as follows,

Algorithm 2 .

Input: Y

1. Solve Problem (30) with Input Y . Obtain optimal solution α∗. Set α = α∗

2. Solve Model (29) with Input α. Obtain optimal solution (x∗, Y ∗). Set Y = Y ∗.

3. Repeat Step 1 until a termination criterion is met.

24

Page 26: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

4. Output solution (x∗,Y ∗).

The following result is straightforward.

Theorem 9 Assume Y is feasible in Model (29) for some α ≥ 1 and 1′α = M . Then, the sequence

of objectives obtained by solving Model (25) in Algorithm 2 is non-increasing.

Like most “Big M approaches”, the quality of the solution improves with larger values of M . How-

ever, M cannot be too large that it results in numerical instability of optimization problem. Although,

the Big M approach does not provide the theoretical guaranteed improvement over classical approach

using Bonferroni’s inequality, it seems to perform very well from our numerical studies.

4 Computations studies

We consider a resource allocation problem on a network in which the demands are uncertain. The

network we consider is an directed graph with node set V, |V| = n and arc set E , |E| = r. At each

node, i, i ∈ V, we decide on the quantity of resource xi to allocate, which will incur a cost of ci per unit

resource. When the demands di, i ∈ V are realized, resources at the nodes or from neighboring nodes

are used to meet the demands. The goal is to minimize the total allocation cost subjected to a service

level constraint of meeting all demands with probability at least 1− ε. We assume that the resource at

each node i can only be transshipped across to its outgoing neighboring nodes defined as

N−(i) ∆= {j : (i, j) ∈ E},

and received from its incoming neighboring nodes defined as

N+(i) ∆= {j : (j, i) ∈ E}.

Transhipment of resources received from other nodes is prohibited.

In our model, we ignore operating costs such as the transhipment costs. One of such applications

is with regards to allocation of equipment such as ambulances or time critical medical supplies for

emergency response to local or neighboring demands. The costs associated with their procurement is

more significant than the operating cost of transhipment, which may occur rather infrequently. We list

the notations of the model as follows

ci : Unit cost of having one resourse at node i, i ∈ V ;

di(z) : Demand at node i, i ∈ V as a function of the primitive uncertainties z;

xi : Quantity at resource at node i, i ∈ V;

wij(z) : Transshipment quantity from node i to node j, (i, j) ∈ E in respond to realization of z.

25

Page 27: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

The problem can be formulated as a joint chance constrained problem as follows,

min c′x

s.t. P

xi +∑

j∈N+(i)

wji(z)−∑

j∈N−(i)

wij(z) ≥ di(z) i = 1, . . . , n

xi ≥∑

j∈N−(i)

wij(z) i = 1, . . . , n

w(z) ≥ 0

≥ 1− ε

x ≥ 0, w(z)

(31)

We assume that the demand at each node are independently distributed and represented as

dj(z) = d0j + zj ,

where zj are independent zero mean random variables with unknown distribution.

By introducing new variables, we can transform the model (31) to the “standard form” model as

followsmin c′x

s.t. xi +∑

j∈N+(i)

wji(z)−∑

∈N−(i)

wij(z) + r(z) = di(z) i = 1, . . . , n

xi + si(z) =∑

j∈N−(i)

wij(z) i = 1, . . . , n

w(z) + t(z) = 0

y(z) =

r(z)

s(z)

t(z)

P(y(z) ≤ 0) ≥ 1− ε

x ≥ 0, r(z), s(z), t(z), y(z),w(z).

(32)

Note that the dimension of y(z) is m = 2n + r.

The transhipment variables w(z) is an arbitrary function of z. In order to obtain a bound on

Problem 31, we apply the linear decision rule on the transhipment variables w(z) advocated in Ben-Tal

eta al. [2] and Chen, Sim and Sun [13] as follows,

w(z) = w0 +n∑

j=1

wj zj .

26

Page 28: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

Under the assumption of linear decision on w(z) and with suitable affine mapping, we have

r(z) = r0 +∑n

j=1 rj zj

s(z) = s0 +∑n

j=1 sj zj

t(z) = t0 +∑n

j=1 tj zj

y(z) = y0 +∑n

j=1 yj zj ,

which are affine functions with respect to the primitive uncertainty, z. Hence, we transform the problem

from one with infinite variables (optimizing over functional) to a restricted one with polynomial number

of variables. Therefore, we can apply our proposed framework to obtain an approximate solutions to

Problem (32).

In our test problem, we generate 15 nodes randomly positioned on a square grid and restrict to

the r shortest arcs on the grid in terms of Euclidean distances. We assume ci = 1. For the demand

uncertainty, we assume that d0j = 10 and the demand at each node, dj(z) takes value from zero to 100.

Therefore, we have zj ∈ [−10, 90]. Using Theorem 1, we can determine the bounds on the forward and

backward deviations, which yields pj = 42.67 and qj = 30.

For the evaluation of bounds, we use L = {1, 2, 4, 5}. We formulate Model using an in-house devel-

oped software, PROF (Platform for Robust Optimization Formulation). The Matlab based software is

essentially an SOCP modeling environment that contains reusable functions for modeling multiperiod

robust optimization using decision rules. We have implemented bounds for the CVaR measure and

expected positivity of a weighted sum of random variables. The software calls upon CPLEX 10.1 to

solve the underlying SOCP.

We first solve the problem using the classical approach by decomposing the joint chance constrained

problem into m constraints of the form (19), with εi = ε/(2n + r). We denote the optimal solution as

xB and its objective as ZB. Subsequently, we use Algorithm 2, the big M approach, with M = 106,

to improve upon the solution. We report results at the end of twenty iterations. Here, we denote the

optimal solution as xN and its objective as ZN . We also benchmark against the worst case solution,

which corresponds to all the demands at its maximum value. Hence the worse case solution is xWi = 100

for all i ∈ W and ZW = 1500.

Figure 1 shows an illustration of the solution. The size of the hexagon on each location, i reflects

upon the quantity xi. Each link refers to two directed arcs in opposite directions. We present solutions in

Table 1. It is interesting to note that the solutions obtained using the classical approach has significant

resources needed at nodes 5, 10, 12 and 13, which forms a complete graph with node 15. After several

27

Page 29: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

iterations of optimization, the new solutions centrally locates the resources at node 15, diminishing the

resources needed at nodes 5, 10, 12 and 13.

Node 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

xB 14 61 73 100 13 213 136 112 7 161 27 8 9 61 161

xN 18 41 77 100 1 257 82 59 15 2 11 0 0 41 337

Table 1: Resource allocation: 15 nodes, 50 arcs (rounded to nearest integer)

In Table 2, we compare the relative improvement of ZN against ZB and ZN against ZW . The

new method has 8 − 12% improvement compared with classical approach of applying Bonferroni’s

inequality and has 30−42% improvement compared with the worst case solution. We also note that the

improvement generally increases over the classical approach when the number of connectivity increases.

This is probably due to the increases correlation among the constraints as connectivity increases. Even

though minimum distributional information are provided, this experiment shows that the new method

solves the joint chance constrained problem more efficiently.

In addition, we find that the improvement increases as the number of facilities increases. Moreover,

we tested the convergence rate of Algorithm 2. Figure 2 shows that the improvement is made mostly

in the first several steps.

# of Nodes # of Arcs ZW ZB ZN (ZW − ZN )/ZW (ZB − ZN )/ZB

15 50 1500 1158.1 1043.3 30.45% 9.91%

15 60 1500 1059.7 968.1 35.46% 8.64%

15 70 1500 1027.3 929.5 38.03% 9.52%

15 80 1500 1009.3 890.1 40.66% 11.81%

15 90 1500 989.1 865.7 42.29% 12.48%

Table 2: Comparisons among Worst case solution ZW , Solution using Bonferroni’s inequality ZB and

Solution using new approximation ZN .

28

Page 30: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

12

3

4

5

6

7

8

9

10

11

1213

14

15

Solution using Bonferroni’s inequality

12

3

4

5

6

7

8

9

1011

12 13

14

15

Solution using New Method

Figure 1: Inventory allocation: 15 nodes, 50 arcs

29

Page 31: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

0 5 10 15 200.82

0.84

0.86

0.88

0.9

0.92

0.94

0.96

0.98

1

Iteration steps

ZN

/ZB

ε=0.001ε=0.01ε=0.1

Figure 2: A sample convergence plot

5 Conclusion

In this paper, we propose a general technique to deal with joint chance constrained optimization prob-

lems. The standard approach decomposes the joint chance constraint into a problem with m individual

chance constraints and then applies safe robust optimization approximation on each one of them. Our

approach builds on a classical worst case bound for order statistics problem, where the bound is tight

when the random variables are negatively correlated. By introducing new parameters (α, w0,w,J )

into the worst case bound, we enlarge the search space so that our approach can also deals with pos-

itively correlated variables, and improves upon the solution obtained using the standard approach via

Bonferroni’s inequality.

The quality of solution obtained using this approach depends largely on the availability of good

upperbound π(y0,y) for the function E ((y0 + y′z)+). As a by product of this study, we show that

any such bound satisfying convexity, positively homogeneity, and with π(y0,0) = y+0 , can be used to

construct an uncertainty set to develop a robust optimization framework for (single) chance constrained

problems. This provides a unified perspective on the choice of uncertainty set in the development of

robust optimization methodology.

30

Page 32: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

References

[1] F. Alizadeh, D. Goldfarb (2003): Second-order cone programming, Math. Programming, 95, 3-51.

[2] Ben-Tal, A., A. Goryashko, E. Guslitzer and A. Nemirovski. (2004): Adjusting Robust Solutions

of Uncertain Linear Programs, Math. Programming, 99(2), 351-376.

[3] Ben-Tal, A. Nemirovski (1998): Robust convex optimization. Math. operations research, 23, 769-

805.

[4] Ben-Tal, A. Nemirovski (1999): Robust solutions to uncertain programs, Operations research let-

ters, 25, 1-13.

[5] A. Ben-Tal, A. Nemirovski (2000): Robust solutions of linear programming problems contaminated

with uncertain data, Mathmatical programming, 88, 411-424.

[6] A. Ben-Tal, A., and M. Teboulle (1986): Expected utility, penalty functions and duality in sto-

chastic nonlinear programming, Management Science, 32, 1445-1466.

[7] Bertsimas, D. and Brown, D.B. (2006): Constructing uncertainty sets for robust linear optimiza-

tion. Woeking Paper

[8] D. Bertsimas, M. Sim (2004): Price of robustness, Operations research, 52, 35-53.

[9] D. Bertsimas, D. Pachamanova and M. Sim (2004): Robust Linear Optimization under General

Norms, Operations Research Letters, 32, 510-516.

[10] G. Calafiore, M. C. Campi (2004): Decision making in an uncertain environment: the scenario-

based optimization approach, working paper.

[11] Charnes, A.,Cooper, W. W., Symonds, G. H. (1958): Cost horizons and certainty equivalents: an

approach to stochastic programming of heating oil, Management Science 4, 235-263.

[12] Chen, W., Sim, M., (2006): Goal Driven Optimization, Working paper.

[13] Chen, X., Sim, M., Sun, P. (2005): A robust optimization perspective of stochastic programming,

to appear in Operations Research.

[14] Ergodan, G. and Iyengar, G. (2006): Ambiguous chance constrained problems and robust opti-

mization, Math Programming Series B, Special issue on Robust Optimization.

31

Page 33: Optimization Online - From CVaR to Uncertainty Set ...⁄The research is supported by Singapore MIT Alliance, NUS Risk Management Institute, NUS academic research grants R-314-000-066-122

[15] El-Ghaoui, L. F. Oustry, H. Lebret (1998): Robust solutions to uncertain semidefinite programs,

SIAM journal of optimization, 9, 33-52.

[16] El Ghaoui, L., M. Oks, and F. Oustry (2003): Worst-Case Value-at-Risk and Robust Portfolio

Optimization: A Conic Programming Approach, Operations Research, 51(4): 543-556.

[17] J. Mayer (1998): Stochastic linear programming algorithms, Gordon and Breach, Amsterdam.

[18] Meilijson, I., A. Nadas (1979): Convex majorization with an application to the length of critical

path. Journal of Applied Probability 16, 671-677.

[19] Natarajan, K., Pachmanova, D., and Sim, M. (2006): Constructing risk measures from uncertainty

sets. Working Paper, NUS Business School.

[20] Nemirovski, A. Shapiro, A., (2004): Convex approximation of chance constrained programs, Work-

ing Paper, Georgia Tech, Atlanta.

[21] Pinter, J. (1989): Deterministic Approximations of Probability Inequalities, ZOR - Methods and

Models of Operations Research, 33, 219-239.

[22] A. Prekopa (1995): Stochastic programming, Kluwer, Dordrecht.

[23] T. Rockafellar, S. Uryasev (2000): Optimization of Conditional Value-at-Risk, Journal of risk, 2,

21-41.

[24] A. L. Soyster (1973): Covex programming with set-inclusive constraints and applications to inexact

linear programming, Operations research, 21, 1154-1157.

32


Recommended