Sobel 1975 Water Resources Research

VOL. 11, NO. 6 WATER RESOURCES RESEARCH DECEMBER 1975

Reservoir Management Models MATTHEW J. SOBEL

School of Organization and Management, Yale University, New Haven, Connecticut 06520

The optimal policies for several discrete time control models of reservoir storage are characterized. Each characterization is exploited to simplify the numerical solution of heretofore formidable problems. Some of the results exploit analogies between multi-item inventory theory and systems of multiple reservoirs. In particular, it is observed that the 'linear decision rule' label in the water resource literature or- dinarily arises in contexts where either (1) a myopic policy is optimal, or (2) a multiple-reservoir optimal release policy, as a function of the vector of reservoir contents, possesses a Jacobian matrix whose values are between zero and one. Either (1) or (2) impressively reduces the computational burden that would otherwise be carried.

This paper analyzes the structure of optimal policies for several discrete time control models of reservoir storage. Most of the models are stochastic and they are prompted by operating problems of regulating the amounts of water discharged from reservoirs. However, the design problem of selecting the capacity of a reservoir induces two models in the paper and, of these, one is deterministic. The form of an optimal policy is characterized for each model and then exploited to simplify the numerical solution of heretofore formidable problems. Several sections of the paper exploit analogies between multiple-reservoir systems and multi-item inventory systems. As a consequence, the form of some of the results below should be familiar to readers acquainted with inventory theory.

Several purposes are served by proliferating the number of available reservoir models. Actual reservoir systems display heterogeneous characteristics, and so a richer set of models permits adequate representation of more of these systems. Furthermore, it permits the same reservoir system to be modeled in several ways. Thus dichotomies in this paper such as deterministic/stochastic, independent/correlated, and chance constraints/explicit costs are not presented as a tax- onomy of actual reservoirs. Instead, they are different ways of idealizing the same problem of managing a water storage process.

SVMMA}

The following section investigates the reservoir design problem of computing the smallest reservoir capacity that will ac- commodate a sequence of predicted inflows. It is shown that a hand calculation, simpler than previously proposed algorithms, will solve this problem. Moreover, some stochastic versions of the reservoir capacity problem can be solved equal- ly easily.

The paper continues by developing an analogy between models of multiple-reservoir systems and of 'multi-item inventory' models. Cost minimization (or net revenue max- imization) reservoir models are very closely related to inventory models about whose optimization so much is known. The analogy is exploited in a stochastic model of a class of multiple-reservoir systems. This model has a simple myopic solution, and so the sequential problem degenerates to a sequence of single-period problems whose numerical solution is easily obtained. The 'myopic' solution specifies a point (or vec-

Copyright (D 1975 by the American Geophysical Union.

tor) that receives different labels in different literatures: 'linear decision rule' (water resources), 'base stockage policy' (operations research), and 'single critical number' (operations research). In the subsequent section the structure of optimal policies is explored for multiple-reservoir systems that may lack a myopic solution. Optimal release policies, as functions of reservoir contents, are shown to possess a Jacobian matrix whose values are between zero and one. This property is shown to accelerate computations impressively.

For expository convenience, the previously mentioned stochastic models assume that inflows in different periods are ind endent. This assumption is relaxed in the final section of the paper where it is shown that the preceding algori.thms and policy structures extend to general stochastic processes of inflows and demands.

NOTATION AND CONVENTIONS

Vector notation is used in the sections of the paper concerned with multiple-reservoir systems. However, inner products are the only multiplications needed. If a = (a, ..., ai) and = (, ..., i) are real I vectors, then a. or .a is written to denote the inner product -'t--[ ottlt. The notation ej is used for the I vector having all zeros except one in the jth component, j = 1, ..., I.

All functions that arise are assumed to attain their minima when they are constrained to compact domains. It is sufficient that the functions be lower semicontinuous, and this restric- tion poses no practical difficulties. Whatever functions are differentiated will possess one-sided derivatives. For defi- niteness, therefore, derivatives are assumed to be taken from the left.

When I' is a finite set of real numbers or a real-valued function that attains a constrained minimum, the notation min I' denotes the smallest member of the set or lowest value of the function. Similarly, if I' is a finite set of real numbers, then max I' denotes the largest member of the set. It is convenient to write (a) + for max {a, 0} when a is a real number. Ifa = (a, ", at) is a real I vector, then (a) + denotes the I vector whose ith component is (at) +. The notation P{A} indicates the probability of an event A.

M any quantities in the following models will vary with time, and the subscript t denotes the time period to which a variable pertains. If the variable refers to the ith reservoir in a collec- tion of I reservoirs, then it carries a superscript of i. For example, st denotes the quantity of water stored in reservoir i at the end of period t. If the superscript i is absent from such a

767

768 SOBEL: RESERVOIR MANAGEMENT MODELS

variable, then it denotes the relevant I vector in period t, e.g., st = ..., sD.

The notation list presents most of the symbols used in the paper. Generally, capital letters denote the random counter- parts to lowercase letters, and these capitals are absent from the list. Exceptions to this rule, such as I, K, and T, are on the list. The latter part of the paper makes some use of dynamic programing in general and inventory theory in particular. Some quantities which are scalars until that point thereafter are functions. For example, the random drawdown in period t is denoted Xt(u) as a function of freeboard u at the beginning of the period. Asterisks are sometimes used to denote an optimal policy, Xt*( ) being an example. Such variants of a lowercase symbol, xt in this case, are not shown in the list.

One last notational peculiarity attends the mixing of water resource models with inventory theory. The maximal drawdown has been labeled f in past water models, and the expected cost of an optimal inventory policy has been denoted ft( ) in inventory theory. Both these usages occur in the sequel, but at most one of them is used in any given section. In fact, the first occurrence offt( ) is well beyond the last use off to denote maximal drawdown.

SMALLEST STORAGE CAPACITY FOR PREDICTED NEEDS: A CHEBYSHEV PROBLEM

This section concerns a deterministic model, namely, a storage process in which inflows and demand are treated as being known in advance. The problem, concerning a planning horizon of T periods, is to determine (1) the smallest storage capacity that accommodates the inflows and permits the pro- jected demands to be met and (2) the corresponding schedule of discharges. Such deterministic models arise in subtasks in simulation programs having random elements. The optimization has been posed by ReVelle et al. [1969] as a linear programing problem. It will be shown here that simple hand calculations lead to a solution.

Thomas' 'sequent peak' algorithm [Harvard Water Resources Group, 1963] (see Fiering [1967] or Loucks [1970]) solves a problem related to ours. It is the special case of (1)-(4) below when ft = co and mt= 0 for all t and it is 'assumed that the inflow and drafts will be repeated in successive sets of cy- cles of T-years each' [Harvard Water Resources Group, 1963, pp. 1-6]. This last assumption is riot made below. In some applications it is an attractive feature, while in others it is not sought. In some cases, the assumption forces an increase in the requisite capacity.

Let st denote the quantity of water in storage at the end of period t and let rt be the predicted inflow during the period. Then st_ and st are connected by

St = St- + rt -- Xt (1) where xt denotes the drawdown, the quantity discharged during period t.

In reservoirs whose surface area does not vary significantly with storage volume, leakage and evaporation also are essen- tially independent of storage olume. Then rt can be regarded as inflow net of leakage and evaporation, and the algebraic sum is not a function of storage volume.

Of course (1) is invalid if the right side exceeds the capacity c of the reservoir, and so an elaboration of (1) is

st = min {c, st_ + rt -- xt} (2) the overflow, if any, being (st_ + rt -- xt -- c) +. The physical

constraints are

0

SOBEL: RESERVOIR MANAGEMENT MOVERS 769

future periods r > t. Let Lt denote the sum of mt and water required to be held back in t for future use. A policy for which xt > st-1 + rt -- Lt in a period t leads to infeasibility in some period r > t. Then Lr = mr Lt = mt + (Lt+l + qt+l - rt+l - mr) + (8)

t = 1, ..., T- 1

is verified inductively starting with t = T- 1 and proceeding from t + 1 to t. The minimax capacity problem with (7) in place of (4) has an optimal solution satisfying

(ft - xt)(st - Lt) = 0 t = 1,..., T (9) The proofs of (9) and (5) are the same after recognizing that (9) is equivalent to

xt = min {ft, st-1 + rt -- Lt} t = 1," ', T (10) The computations underlying (8) and (10) (or (6) if rt -- qt

and mt --> mt+l for all t) are trivial by hand. They are followed by the recursion (1) for sl, "', st. Then the minimally sufficient capacity is c = max {So, '", st}.

Table 1 presents a numerical example of the applications of (1), (8), and (10). The data, the computations for (8), (10), and (1), and the end products, the draft and storage decisions, are given.

STOCHASTIC MINIMAX CAPACITY

An inspection of (5) and (6) shows that the value taken by an optimal discharge quantity xt does not depend on the data for any period later than t. In fact, if qt -< xt --< ft and mt --< st are time-varying restrictions, then rt -- qt for all t and rnl _> rn2 > " > mr imply that

xt = min Ift, st-1 + rt - mr} (11) is optimal. Here the decision xt in period t is unaffected by knowledge of events in periods later than t. Thus the myopic character of (6) is retained by (11) which implies that a variety of stochastic minimax capacity problems have an easy solution. Suppose that a stochastic process (Q1, F1, M1, R0, "', (Qr, Fr, Mr, Rr) determines (qt, ft, mr, rt) for t = 1, ..., T. The storage levels S1, "', Sr and discharges ,Y1, '", ,Yr are controlled random variables. Let

St = St-1 + Rt-Xt (12)

and suppose that the values st-i, mr, rt, qt, and ft taken by St-i, Mr, Rt, Qt, and Ft are known when the value of Xt is selected. A stochastic version of rt -- qt and mt -- mt+l for all t is

P{Rt- Qt Mt - Mt+l t= 1,'.., T} = 1 (13) There is some decision rule X1, "', Xr such that

PlMt+l < St Qt < xt < Ft t = 1,..., T} = 1 (14) Assumptions (13) and (14)guarantee that a feasible rule exists (with probability one). Let X denote the set of rules (X1, "', Xr) satisfying (14). The stochastic version of (6) is

't* = min {Ft, St_i* q- Rt - Mt} (15) where So* -- So and

St* : St-i* q- Rt - Xt* t = 1, "', T Second Result

Then the optimality and myopia of (11) imply that (15) is feasible and induces minimax capacity under assumptions (13) and (14). This claim is true with probability one. As a consequence, among all rules satisfying (14) the one given by (15) minimizes the expectation of the maximum stored quantity.

Let ht be an outcome (or 'realization' or 'point in the sample space') of the history

Ht -- (So, Q1, F1, M1, X1, R1, "', St-2, Qt-1, Ft-1, Mt_l, )'t-1, Rt-1, St-1, Qt, Ft, Mt) (16)

It is physically impossible for policy X1, '", Xr to have Xt depend on more information than is contained in Hr. Suppose (14) is replaced by the chance constraint

and P{mt St "- St _ l q- rt -- mr(mr) (17a) qt --< X't(ht) ft} - Olt (175)

for all ht and t = 1, ..., T. The values of mr, rt, qt, and st-1 are included in ht and the values of at are assumed to satisfy 0 (at ( 1, t = 1, ...,T.

The restrictiveness of (17) compared to (14) is more illusory than real. Suppose (13) is valid and that X1, '", Xv is a policy different from (.15) and let So, "', St be the storage levels generated by the Xt. It will be shown that maxt {St*t

TABLE 1. A Simple Minimax Capacity Problem

Data

Maximum Minimum Month Draft Draft

t ft qt

Minimum Computations Inflow Storage

rt mt Lt + + qt + - rt + i Lt $t- i + rt - Lt

Decisions

Dratt Xt

Storage St

0 1 10 8 2 10 8 3 10' 8 4 10 8 5 11 9 6 13 11 7 15' 13 8 17' 15 9 14 12

10 12 10 11 11' 8 12 10' 8

0 10 1' -1 1' 9 9 1 10 1' -3 1' 10 10 1 15 1 3 4 12 10 6 21 1 16 17' 10 10 17 17 I 23 24* 10 10 24 13 3 22 25* 12 12 25t

9 2 16 18 16 15 19 6 1 6 7 18 17 8 8 1 1 2* 14 14 2

10 1' -4 1' 11 11 1 12 0 -2 0 13 11 2 11 1 1 12 10 3

*Binding constraint. 'Minimal c = s6 = 25 = max {s0,'", s.}.


_< maxt {St} with probability one. If Xt(ht) < Xt*(ht) for all ht and t, then St 2 St* for all ht and t; so

P{max {St*} max {St}} = 1 t t

and X* is superior to X. The only other possibility is that there is the smallest integer t and history ht such that

Xt(ht) > Xt*(ht) = min {ft, st- * + rt - mr} Third Result

Then either (1) Xt(ht) ft, and Xt is infeasible, or (2) Xt(ht) St-* + rt -- mt = st- + rt -- mr; so mt > st = st- + rt -- X(h ), P{M(h ) _< &(h, X ), and Q(h) _< X(h ) _< F(h)} = 0, which violates (17); so Xt is again infeasible. Therefore (13) implies that X* in (15) is optimal for the stochastic minimax problem with chance constraints.

COMMENT ON CHANCE CONSTRAINTS AND POLICY STRUCTURE

'Chance-constrained' programing combines dynamic cost minimization with probability constraints on some operating characteristics that would result from a chosen policy. In- equality (17) is an example of a chance constraint. Several recent reservoir management models (see LeClerc and Marks [1973] and its references) have included chance constraints, but their authors have ignored the well-known [Derman and Klein, 1965] possibility that a randomized policy dominates every unrandomized policy. For a trivial example, suppose there are two actions labeled 0 and 1. Exactly one of these two must be taken every period subject to the constraint that action I be taken at least 100% of the time. Action j incurs a cost of $j each time that it is taken, and the objective is to minimize the average cost per period. Action I taken every period is the only unrandomized stationary policy that satisfies the 100% constraint, and the associated average cost is $1. However, the randomized st. ationary policy that takes action 0 with probability I - and action 1 with probability also satisfies the constraint, and its average cost is $ < $1.

Water resource models have not yet applied chance constraints to joint events. For example, if et is the event being constrained in (17), then one may seek a solution that satisfies

P{ UI . U/ '" UI r} > a0 in addition to (17). This observation is pertinent in most water resource contexts; methods for solving the resulting optimization problems are discussed by Miller and Wagner [1965].

FORMULATION OF OPERATING PROBLEMS: APPLICABILITY OF INVENTORY THEORY

The benefits and costs associated with a particular reservoir design depend partly on the modes of operation that the design permits. In other words, design problems and operating problems are interrelated because the design of a reservoir affects the composition of the class of feasible operating rules. However, this interdependence will not be made explicit henceforth, and the remainder of the paper is devoted to deter- mining the size of drafts in successive periods. This section of the paper examines the formulation of operating problems and concludes that their structure is shared by models found in 'inventory theory.'

Beginning with the seminal work by Massd [1946] (which also originated dynamic programing), a sizable literature on the determination of optimal sequential decision rules for

reservoir releases has developed. Little [1955], Gessford and Karlin [1958], and Amir [1967] significantly advanced Mass6's results. As a generalization of their models, consider an inter- connected system of I reservoirs having capacities C , ..., C I and let C = (0, ..., CI).

Elaborating on the previous notation, let sd denote the amount of water in the ith reservoir at the end of period t, let xd denote the amount of water released from it, and let denote the random amount of water that flows into it. Let st = (St , ''', Stl), Rt = (Rt 1, ''', Rtl), and xt = (xt', '", xt I) so that (2) is valid with the present vector notion if rt is replaced by Rt and the min in (2) is taken separately for each component i = 1, ..., I.

It is convenient to use st_ - xd, namely, the storage after discharge before inflow, as the decision variable instead of xt t. Let

Yt = St-1 i -- Xt Yt = (Yt 1, '' ', Yt I) so that (2) is equivalent to

st - min {C, Yt q- Rt} (18) If water cannot be pumped from one reservoir to another, then the constraint on xt is 0 < xt < st_x or

0 __ Yt -- St- (19) If water can be pumped from any reservoir to any other during the same period then the constraint is weaker, namely,

i E Yt i st-t _ 0 _< y t C (20) i i

If water can be pumped only from some reservoirs to others, then the constraints will be intermediate between (19) and (20). For specificity in the sequel the 'no pumping' assumption (19) will be made. However, the essential properties of the following results are preserved under (20) or any mixture of (19) and (20). For ease of exposition the random vectors R, Ro., ... are assumed to be independent. This assumption will be relaxed at the end of the paper; if this were not possible, the results would be valueless.

There is a one-to-one relation between the contents of a reservoir and its freeboard. Let ud = C - s,_d denote the freeboard in reservoir i at the beginning of period t and let u, = (ut , "', ud). Similarly, let v,' = C - yd which can be in- terpreted as the freeboard after discharging xd before the inflow Rt occurs:

l)? = C i -- yt i = C i -- St_ + Xt t = Ut + Xt Now (18) and (19) are equivalent to

ut+ = (vt - R) + u < v < C (21) The purpose of the transformation (s, y) --, (u, v) is to observe that (21) is formally the recurrence equation for successive inventory levels in a 'lost sales' inventory system. If vd is the amount of 'goods' available to satisfy 'demand' Rt , then the

i is -- R? if that difference is subsequent 'inventory level' ut+ nonnegative. If Rt t tt t, then the excess demand is lost (lost

= 0 as a result There is a huge literature on op- sales) and ut+ . timal sequential ordering rules (the 'order' is vd - ut ) in inventory systems (as evidenced by the surveys by Veinott [1966] and Clark [1972]). Therefore the appearance of (21) suggests that inventory theoretic results and methodology may be ap- plicable to reservoir management. That conjecture is verified in following sections.

SOBEL: RESERVOIR MANAGEMENT MODELS 771

COSTS AND BENEFITS DEPENDING ONLY ON RESERVOIR LEVELS

Continuing from (21), let Gt(u, v) denote the expected cost in period t (net of revenues) of a vector xt = v - u of discharges if storage at the beginning of the period is st_ = C - u. Detailed models for the construction of Gt( , ) are presented by many authors such as Massd [1946], Little [ 1955], Gessford and Karlin [1958], Amir [1967], and Roefs [1968]. Su and Deininger [1972] present models in which the expected net costs in each period t depend only on the reservoir level, and so there are functions g ( ), '", gr( ) such that

gt(v) Gt(u, tg) t = 1, 2,'' ', T where T is the planning horizon. The problem in this case is to choose v, ..-, vr sequentially subject to (21) so as to

T

min E gt(vt) (22) t----1

where E denotes expectation. If the constraints (21) could be ignored, then the problem

would have the following trivial myopic solution. Let tit be the biggest (lexicographically) nonnegative vector in [0, C] that minimizes gt( ); an optimal unconstrained solution is vt = tit (so xt = tit - C + st-O for all t = l, ..., T. The constraints (21) jeopardize this solution with the possibility that ut ; tit for some t. That eventuality is precluded by the assumption

tit-dr _< tit+ t = 1, ..., T- 1,(23) where dt= (dr , "', dt ), dt being the minimum possible inflow to reservoir i in period t, and' so dt is the largest number satisfying P{Qt _> dt t} = 1. Of course, u < ti < ti2 < < tit is sufficient for (23).

To see that (23) implies that vt = tit for all t is feasible, first observe that u _< ti implies that v = ti satisfies (21) for t = 1. Suppose that vt = tit is feasible; then

ut+ = (vt - Rt) + = (tit - Rt) + < (tit - dr) + (24) with probability one by definition of dr. For the ith component in (24), either ut - - dt < 0 = (ut - - cltt) + < ut+ -

-t is nonnegative by definition) or 0 < (ut - (because ut + - _ -t from (23). The inductive conclu- dt) + = ut - -dt < Ut+

sion, with probability one, is that (23) ensures ut < tit, and so vt = at is feasible for all t = 1, ..., T. Optimality of vt = tit for all t follows from the fact that the imposition of constraints (21) cannot reduce the cost of an optimal solution to a previously unconstrained problem. Fourth Result

In summary, (23) implies that vt = tit for t = 1, ..., T is optimal for the problem of (21) and (22). The key step is to change the decision variable from xt to either Yt or t9t. In either case, given st_ or ut, there is a one-to-one relation with xt. When (23) is valid, changing the decision variable replaces a burdensome T-period dynamic programing recursion in an l- dimensional state space with T separate /-dimensional minimization problems. If gt( ) is the same for all t, i.e., the cost structure is 'stationary,' then just one /-dimensional minimization is needed. The whole problem is said to have a myopic solution because vt = tit does not stem from a recursion. The results above for problems (21) and (22) with assumption (23) make up a special case of properties that Veinott [1965] derived for multi-item inventory systems. Clark [1972] surveys recent research on multi-item inventory systems.

Example 1. The problems (21) and (22) encompass costs and revenues as linear functions of discharge quantities x, ' ', xr. This fact was first exploited by Veinott and Wagner [1965] (who mention that Martin Beckmann had recognized it earlier) in an inventory problem. Suppose At' is the cost (net of benefits) of a unit quantity released from reservoir i during period t. Let At = (At , '' ', At ') so that At'xt (inner product) is the net cost of xt. Let bt(ut) denote the net cost of having st_ = C - ut in storage at the start of period t. Then the total cost K during T periods, a random variable, is

T

K = [At'xt -'[- O t(ut+)l t--1

T

= - t--.1

Letting Ar+ -= 0 and using (21) and ut = (vt_ - Rt_O +, T

---- ' )+ t t t=2

+ + T

.... + t--.--1

+ - - Therefore

T

EK = Y'. gt(vt) -- A' u t=l

where A-u is uninfluenced by the decisions v, ..., vr and

gt(v) = At.v - At+.E(v - Rt) + + Ebt[(v - Rt) +] (25) Then the problem of feasibly minimizing EK is equivalent to (22).

Example 2. As a special case of (25), suppose only a single reservoir is being regulated so that I = 1 and also At = A, bt(u) = B(u - 3,) 2, and P{Rt _< r} = r/C (0 _< r < C) for all t = 1, .., T. It is assumed that A > 0, B > 0, 3, > 0, and 2B(C - 3,) < A. For t < T, (25) takes the form

fo fo Cgt(v) = --A (v -- r) dr--[- B (v -- r --)2 dr -'l- B3'2(C -- v) --I- CAr (25')

for 0 < v < C. Straightforward calculus shows that this function is concave on [0, C] (because of the inequality above). Therefore its minimum on [0, C] occurs either at v = 0 or at v = C. The assumption A > 0, B > 0, and 3, > 0 ensures that gt( ) is minimized at v = 0 if t < T. Because A r+ -= O, sub- stitution in (25) for t = T yields (25') except that the first term is absent. It is straightforward to show that gr( ) is convex on [0, C] and minimized at 0. Then in the notation of (23), tit = 0 for all t; so (23) is satisfied if u = 0, i.e., So = C. Therefore if the reservoir is full at the outset, then an optimal policy con- sists of discharging xt = C - st_ - tit = C - C - 0 = 0 at the beginning of each period t.

Example 3. The problem analyzed by Su and Deininger [1972] may exemplify (25). They consider a single reservoir (Lake Superior) in which the decision variable is the quantity discharged and the expected costs (net of benefits) depend only on the level of the reservoir at the beginning of a period.


FORMULATION OF OPERATING PROBLEMS: DYNAMIC PROGRAMING

Recall the notation Gt(u, o) for the single-period expected cost of a vector x = v - u of discharges if initial storages are st_ = C - u. The preceding section explained that the computation of an optimal policy is often easy if Gt depends essen- tially only on v. This section concerns more complex models in which that assumption cannot be made. It is convenient to let p(a, fi) = (a - fi)+ for I vectors a and fl. Then from (21),

Ut+x = P(t;t, Rt) At the beginning of each period t an outcome ht of the

history Ht is observed, and then vt. is chosen (xt = vt - ut). A reservoir release policy for period t is thus an I vector-valued function Vt( ) ofht such that 0 < ut < Vt(h) < C for all possible outcomes ht of Ht (ut is specified in ht). Let Vt = (Vt, '", Vr) denote an arbitrary release policy in periods t, t + 1, ... T and let ft(Vt[ ht) be the conditional expected net cost in those periods from following policy Vt given that the history Ht = hr. That is

T

lb,)= i=t

The criterion of optimality will be ft( [ ). More precisely, a policy V* = (V*, ..., Vr*) is'defined to be optimal if

ft(Vt*lht) < ft(Vt ht) (26) for all ht and Vt, t = 1, ..., T, where Vt* = (Vt*, "', Vr*). A straightforward inductive proof (starting with t: T) shows that if there exists an optimal policy, then ft(Vt*lht) depends on ht only through ut, t = 1, ..., T, and ft(Vt*lht) -= ft(ut), the {ft} satisfying

ft(u) = min {G,(u, v) '-k EJ,+,[p(v, R,)]}, (27) u_v_ c

0_u_C,t = 1,"',T(fr+( )=0).Conversely, ifthereisa sequence {ft( )} satisfying (27), then there exists an optimal policy V* = (V*,. ., Vr*), Vt*(ht) -- Vt*(ut) being any value of v at which the minimum in (27) is attained.

The recursive equation (27) will be used to characterize an optimal policy. A virtue of the dynamic programing formulation thus far is the absence of any heuristic devices such as the 'principle of optimality.'

There is a considerable literatur,e on optimal economic growth under uncertainty and optimal consumption and sav- ings by an individual during a lifetime. Equation (27) with 0 < v _( u replacing u _( v _( C is formally equivalent to the recursive equations of 'capital accumulation' which pervade that literature. However, it is reasonable to assume that p( , r) is a concave function (for each r) in the capital accumulation con- text whereas (v - r) + lacks that useful property. References to the capital accumulation literature and results for (27) when p( , r) is concave can be found in part 1 of the volume edited by Szegb' and Shell [1972].

MONOTONE POLICIES

The objective of this section is to demonstrate that there is an optimal discharge policy Xt*(s), as a function of reservoir levels s, such that 0 < aXt*'(s)/asj < 1, i,j = 1, ...,I. Then a higher storage level induces a larger discharge, but the increment in the discharge does not exceed the increment in storage. This property will be exploited computationally in the following section.

It is useful to introduce the notation

Jt(u, 1)) = Gt(u, l)) q- Eft+xLo(/), Rt) ] so (27) can be written

it(u) = min Jt(ll, t)) (0 _< U _< C) (29) u_v _ c

It is assumed that Gt( , ) is a weakly subadditive function on its domain {(u, v): 0 _ u _ r _ C} A, namely, that cross- partial derivatives of Gt, with respect to uj and ra, are non- positive. Specifically, henceforth it is assumed for each t that

0 _> Gt(u + bey, v + 'Yen) - Gt(u, l; + 'Yen) -- Gt(u + 6el, V) + Gt(u, v) (30)

where b, 'Y > 0 such that all four arguments in (30) lie in A. Example 1 above is a special case of the usual situation

where Gt( , ) contains one term depending on reservoir levels and another term depending on discharge quantities. Specifically, if

Gt(u, v) = at(v - u) + bt(u) (31) then nonnegativity of the cross-partial derivatives of at( ) ensures (30). The cross partials are nonnegative, for instance, if at(u- v) -= at*Y][v'- u ] with at* being convex.

The second term in (28), Eft+[o(v, Rt)], is trivially weakly subadditive in (u, v), and a sum of weakly subadditive functions is itself weakly subadditive. Therefore (30) and (28) imply for each t that Jt( , ) is weakly subadditive. It follows from (29) and Topkis [1968] that there is an optimal policy Vt*( ) for (29) such that

Vt'(u) Vt(u' ) 0 _( u _( u' _( C (32) To interpret (32), let s and s' be I vectors such that 0 < s' < s O, it is convenient, following Veinott [1972], to rewrite (29) with the decision variable as x = v - u instead of v and the state variable as s = C - u instead of u. Let wt(s) --- ft(C - s) so that wt(s) = min {Gt(C-- s, C-- s Jr- x)

O_z_x

+ EWt+l(min {C, s- x n t- Rt})} (34) Assumption (30) is too weak to ensure fhe desired property for Xt*( ). Suppose also that separation (31) applies to Gt( , ) so that

Gt(C - s, C- s + x) = at(x) + bt(C- s) It is assumed for each t that at and bt have nonnegative cross- partial derivatives or, more generally, that

at(x + be:) - at(x) bt(x + be:) - bt(x) (35) are nondecreasing functions of xn for any k j, j - 1, ., I, b > 0, and 0 < x < x + bej < C. It follows from (34), (35), and Topkis [1968] that there is an optimal policy Xt*( ) for (34)

(28)

SOBEL: RESERVOIR MANAGEMENT MODELS 773 such that

Xt*(s') < Xt(s) 0 _< s' _< s _< C (36) Fifth Result

Therefore optimal discharge quantities are increasing functions of the storage quantities. Inequality (36) implies that Xt*( ) is-continuous except at upward jumps and that these are the only kind ofjumps possible. However, (33) implies that only downward jumps are possible. Taken together, (33) and (36) imply that Xt*( ) is continuous. Also (36) asserts monotonicity, and so ,It*( ) is left-differentiable on (0, C], and all the elements of the Jacobian matrix of Xt*( ) have values between 0 and 1, i.e..

0 < O Xt*(x) _ Os i _< 1 0 < s 0 when I = 1 in groundwater management models. The mathematical structure underlying many of Burt's papers (see Burr [1970] for a bibliography) is closely related to (27) and therefore to capital accumulation models. A numerical example that illustrates the computational benefits of monotone optimal policies is presented after the following section.

ACCELERATED COMPUTATIONS FOR MONOTONE POLICIES

This section explains how (33), (36), and (37) can be exploited to accelerate the computation of an optimal policy. It is assumed that variables have been discretized for computations and that C i is an integral multiple of the common unit storage and release quantity for reservoir i, i = 1, ..., I. Then (37) is equivalent to Xt*'(s + ej) {Xt*'(s), Xt*'(s) + 1} (38)

i = 1, ...,I j = 1, ...,I In words, suppose that Xt* is known at some lattice point s, 0 < s < C. Let s' = s + ej be an adjacent lattice point such that

S ' s k .i

s ' = s-'[ ' 1 k = j Now ,Yt*(s + e;) is an optimal quantity to release from reservoir i if the contents of the reservoirs are given by s + e;. Thus (38) asserts that either Xt*(s + e;) is the same as Xt*(s) or else it is Xt*(s) + 1. Therefore the I vector Xt*(s + e;) must lie on one of the 2 lattice points x adjacent to Xt*(s) such that x _> Xt*(s). Formally, let M denote the set of I vectors, each of whose components is zero or one. Thus M contains 2 elements. Now (38) is equivalent to

Xt*(s + e.) {Xt*(s) + m' rn M} (39) By using v - u = x in (29), the effect of (39) is ft(u- ej) = min l Jr(u- ej, Vt*(u) - m): rn M} (40)

for 0 < u < C andj = 1, ..., I. For each t, (40) suggests a recursion in u starting with u = C. Necessarily, Vt*(C) - C because u < v < C. The lattice points in the/-dimensional rect-

angle [0, C] can be generated sequentially by starting at u = C and then repeatedly subtracting/-dimensional unit vectors ej. There are

II (C q -- 1)-- 1 i--1

lattice points in [0, C] not including the point C. From (40), at each of these points at most 2 alternatives need to be compared; so for each t, Vt*( ) is determined with fewer than

2r II (C' -3- 1) (41) i--1

comparisons of alternative decisions. A straightforward computation using (29), by comparison, requires searching the lattice points in [u, C] for each u. The rectangle [u, C] contains

I

II (c'- u' + i=l

points, so summing over u [0., C], there are I

Z u [0, tY] i=l

= 2-' II ( C+ 2)( C+ 1) (42) comparisons when the monotone structure of Vt*( ) is not exploited. Some values of the ratio of (41) to (42) are presented in Table 2. The computational improvement of (40) with respect to (29) becomes more pronounced as I and C increase.

Suppose that (30) is valid but (31) is invalid, or (31) applies but the cross partials of at and bt are not all nonnegative. Then in the discrete case, Vt*(u) < Vt*(u + e), but Vt*'(u + e) > Vt*(u) + 1 is possible for some u, i, and j. The problem is still simpler than (29) but computationally much more burdensome than (40).

EXAMPLE

The following example illustrates the use of (40). Suppose I = 1; so a single reservoir is being controlled, its capacity is C = 3, and the planning horizon has T = 3 periods. Benefits and costs in a period t are assumed to derive from a demand Dt for downstream use of water and the level of the reservoir at the beginning of each period. Suppose that the net cost of downstream use is

4(Dr - xt) + + 2(xt - Dr) + and that the net cost associated with reservoir levels is (st-1 - 2) '. The demands D1, Do., and Ds are assumed to be independent random variables with the same distribution:

d P{Dt = d} 0 0.1 1 0.4 2 0.3 3 0.1 4 0.1

TABLE 2. Ratio of Number of Alternative Decisions per Stage in Accelerated Versus Straightforward Algorithms,

O=C ..... Ct=c

Number of

Reservoirs I

Size of Each Reservoir c

5 10 50

1 0.571 0.333 0.078 2 0.327 0.111 0.006 5 0.187 0.037 2.6 X 10 -6


TABLE 3. Values of G(u, v) in the Example u v=O v=l v=2 v=3

0 7.8 4.4 3.4 4.2 1 6.8 3.4 2.4 2 7.8 4.4 3 10.8

Then Gt(u, v) takes the following form: Gt(u, t;) = 4E(Dt - t; + u) + + 2E(v - u - Dr) +

+ (3 - u - 2) ' (43) Using the distribution of Dt to evaluate the expectations results in the values for G(u, v) (see Table 3). To illustrate the computation of G(u, v), consider G(2, 3):

For example, Xa(2) = 2 + Va(3 - 2) - 3 = -1 + Va(1) = -1 + 2- 1.

It is necessary to evaluate Ef[(v - Rs) +] because Jr(u, v) = St(u, v) + Eft+[(v - Rt) + ] (45)

It is assumed that R, R., and Rs are independent random variables with the following common distribution'

r P{Rt = r} 0 0.3 1 o. 1 2 0.1 3 0.3 4 0.2

Use of this distribution and the values off(u) results in the fol-

G(2, 3) = 4[P{Ot = 0}(0 -- 3 + 2) + + P{Ot = 1}(1 -- 3 '-I- 2) + + P{O = 2}(2 -- 3 + 2) + + P{ D = 3}(3 -- 3 + 2) + + P{ D = 4}(4 -- 3 + 2)+1 + 2[P{ D, = 0}(3 - 2 - 0) + + P{Dt = 1}(3- 2- 1) + + P{Dt = 2}(3- 2-- 2) + + P{D = 3}(3- 2- 3) + q- P{Dt = 4}(3 -- 2-- 4)+1 q- (3-- 2-- 2) '

= 410.1(0) q- 0.4(0) q- 0.3(1) -3- 0.1(2) q- 0.1(3)1 q- 210.1(1) q- 0.4(0) q- 0.3(0) q- 0.1(0)q-0.1(0)1 q- 1' = 3.4 q- 1 = 4.4

The weak subadditivity of G(u, v) can be verified either with (43) or Table 3. For example,

[G(2, 3) - G(2, 2)] - [G(1, 3) - G(1, 2)] = -3.4- (-1) = -2.4 N 0

Because T = 3 and fr+( ) -- 0, we have J3(u, v) = G(u, v). Therefore the recursion specified beneath (40) takes the form in Table 4 for t - 3. To illustrate the computation, consider

/a(1) = min {J a(1, Va(2)), Ja(1, Va(2)- 1)} = min { Ja(1, 3), Ja(1, 2)} = min {G(1, 3), G(1, 2)} = min {4.2, 3.4} = 3.4

The minimum is attained at J3(1, 2) = G(1, 2), and so Va(1) = 2. In order to find the corresponding optimal discharge policy,

namely, Xa(s2), we use the identity t)t = C -- st- + Xt

so

Xt(st-O = st_ + Vt(C- st_) - C (44) In the present case, this yields the following policy:

s,. Xs(s,) 0 0 I 1 2 1 3 2

lowing tabulation of Ef[(v - RO +]:

0 1 2 3

For example, for v = 3,

Era[(3- Rs) +]

- ] 3.4 3.4 3.7 5.7

= fa(3)P{Rs = 0} q- /a(2)P{Rs = 1} '-I- fa(1)P{Rs = 2} + fa(0)P{Rs > 2}

= 10.8(0.3) + 4.4(0.1) + 3.4(0.1) +'3.4(0.5) = 5.7 Using this equation and the above tabulation in (45) for

t = 2 results in values for J.(u, v) (see Table 5). For example, J.(1, 2) = G(1, 2) + Ef[(2 - R0 + ] = 3.4 + 3.7 = 7..1. Table 5 is used to computef(u) in the same fashion that Table 1 was used to compute f(u). See Table 6 for the specific recursion. The optimal discharge policy associated with V.(u) is determined with (44) and follows:

s, X4s,) 0 0 I 1 2 1 3 2

TABLE 4. Recursive Computation off(u) TABLE 5. Valises of d,.(u, v) in the Example

u Computation offa(u) f(u) V3(u) v=0 v=l v=2 v=3

3 G(3, 3) = 10.8 10.8 2 min {G(2, 3), G(2, 2)} - min {4.4, 7.8} 4.4 1 min {G(1, 3), G(1, 2)} = min {4.2, 3.4} 3.4 0 min {G(10, 2), G(0, 1)} = min {3.4, 4.4} 3.4

3 0 11.2 7.8 7.1 9.9 3 1 10.2 7.1 8.1 2 2 11.5 10.1 2 3 16.5

SOBEL: RESERVOIR MANAGEMENT MODELS 775

TABLE 6. Recursive Computation off(u) Computation off(u) f(u) Vs(u)

ds(3, 3) = 16.5 16.5 3 rain {ds(2, 3), d:(2, 2)} = rain {10.1, 11.5} 10.1 3 min {ds(l, 3), ds(l, 2)} = rain {.8.12, 7.1} 7.1 2 rain {ds(O, 2), ds(O, 1)} = rain {7.1, 7.8} 7.1 2

TABLE 8. Recursive Computation of f(u) Computation off(u) f(u) V(u)

d(3, 3) = 21.0 21.0 3 min {d(2, 3), d(2, 2)} = min {14.6, 15.8} 14.6 3 rain {d;(1, 3), d(1,2)} = rain {12.6, 11.4} 11.4 2 rain {dffO, 2), d(O, 1)} = rain {11.4, 11.5} 11.4 2

Tables 7-9 present the remaining computations in the same order in which they are performed.

v A[(v - 0 7.1 1 7.1 2 8.0 3 10.2

CORRELATED INFLOWS

This section is concerned with the impact of conditional dependence in the inflow sequence of random I vectors Rx, R2, ", Rr. The preceding stochastic models already encompass interdependence of the I components of Rt = (Rt x, ''', Rtt). Also, the stochastic minimax capacity model allowed dependence among Rx, "', Rr. In general, the impact of dependence is to expand the dimension of the state space but to leave unchanged the general form of an optimal policy. This assertion will be illustrated for the stochastic models in preceding sections.

Let Wt(Rx, '' ', Rt-x) denote a statistic of Rx, "', Rt-x that is sufficient for Rt, namely,

P{Rt < alWt} = P{Rt < alRx, ..., Rt_x} (46) Such a statistic always exists because (Rx, "', Rt-O is itself sufficient. Let ft be the set of all possible values of Wt. Then instead of (27) or (28) and (29) the generic problem is

Jr(u; co) = min Jr(u, v; co) (47) u _< v _< C

Jr(u, v; w) = Gt(u, v; w) + Eft+x(p[v, Rt], t[w, Rt]) (48) (O_


NOTATION

at(x) net cost in period t of drawdown vector x. At net unit cos: of drawdown from reservoir i during

period t. bt(u) net storage cost of having C - u in storage at start of

period t. c reservoir capacity as endogenous variable.

exogenous capacity of reservoir i. dt minimum possible inflow to reservoir i in period t. Dt random downstream demand for water during

period t. E expectation (of a random variable).

ft(u) expected cost of an optimal policy during periods t through T if C - u is in storage at the start of period t.

ft maximal drawdown during period t (sometimes not indexed by t).

gt(l)) Gt(u, v) when it does not depend on u. Gt(u, v) expected net cost in period t of a vector v - u of

drawdowns if storage levels at the beginning of the period are C - u.

ht history of events up to the beginning of period t. i generic index number of reservoir superscripted on

other variables. I number of reservoirs in system.

Jr(u, t)) expected cost during periods t, ..., T of drawdown u - v in period t, when the initial storage was C - u, followed by an optimal policy.

K total (random) cost during T periods. Lt minimal storage in period t plus water obliged to be

held back for future use. mt minimal storage at end of period t (sometimes not

indexed by t). M set of all I vectors each of whose components is zero

or one.

N expected costs irrelevant to the decision problem. P probability measure. qt minimal drawdown during period t (sometimes not

indexed by t). rt inflow during period t. st quantity of water in storage at end of period t. t generic index number of period. T length of planning horizon;

ud freeboard in reservoir i a,t beginning of period t. vt freeboard in reservoir i after discharge but before in-

flow in period t. w(x) f(c- x).

Wt( ) sufficient statistic. xt drawdown during period t. yt t quantity of water in reservoir i during period t after

drawdown before inflow, equal to st_x - xt . at minimum probability level of a chance constraint.

p(v, r) storage at end of period if inflow is r and storage was C - v just before inflow occurred.

3_ domain of the single-period cost function Gt( , ). Acknowledgments. This manuscript was partially supported by

NSF grant GK-38121 and was presented at the Puerto Rico meeting of the Institute of Management Sciences/Operations Research Society of America in October 1974.

REFERENCES

Amir, R., Optimum operation of a multi-reservoir water supply system, Ph.D. thesis, Stanford Univ., Stanford, Calif., 1967.

Burt, O. R., Optimal resource use over time with an application to groundwater, Manage. $ci., 11, 80-93, 1964.

Burt, O. R., Groundwater storage control under institutional restrictions, Water Resour. Res., 6(6), 1540-1548, 1970.

Clark, A. J., An informal survey of multi-echelon inventory theory, Nay. Res. Logist. Quart., 19(4), 621-650, 1972.

Derman, C., and M. Klein, Some remarks on finite horizon Marko- vjan decision models, Oper. Res., 13, 272-278, 1965.

Eisel, L. M., Comments on 'The linear decision rule in reservoir management and design' by Charles ReYelle, Erhard Joeres, and William Kirby, Water Resour. Res., 6(4), 1239-1241, 1970.

Fiering, M., Streamflow Synthesis, Harvard University Press, Cambridge, Mass., 1967.

Gessford, J., and S. Karlin, Optimal policy for hydroelectric operations, in Studies in the Mathematical Theory of Inventory and Production, chap. 11, edited by K. J. Arrow, S. Karlin, and H. Scarf, Stanford University Press, Stanford, Calif., 1958.

Griliches, Z., Distributed lags: A survey, Econometrica, 35(1), 16-49, 1967.

Harvard Water Resources Group, Operations research in water quality management, Div. of Eng. and Appl. Phys. Harvard Univ., Cambridge, Mass., !963.

Iglehart, D. L., Capital accumulation and production for the firm: Op- timal dynamic policies, Manage. Sci., 12, 193-205, 1965.

LeClerc, G., and D. H. Marks, Determination of the discharge policy for existing reservoir networks under differing objectives, Water Resour. Res., 9(5), 1155-1165, 1973.

Little, J. D.C., The use of storage water in a hydroelectric system, J. Oper. Res. Soc. Amer., 3, 187-197, 1955.

Loucks, D. P., Some comments on linear decision rules and chance constraints, Water Resour. Res., 6(2), 668-671, 1970.

Mass6, P., Les Rdserves et la Rdgulation de l'Avenir Dans la Vie Economique, 2 vols., Hermann, Paris, 1946.

Miller, B. L., and H. M. Wagner, Chance constrained programming with joint constraints, Oper. Res., 13(6), 930-945, 1965.

ReVelle, C., E. Joeres, and W. Kirby, The linear decision rule in reservoir management and design, 1, Development of the stochastic model, Water Resour. Res., 5(4), 767-777, 1969.

Roefs, T. G., Reservoir management: The state of the art, Rep. 320- 3508, IBM Sci. Center, Yorktown, N.Y., 1968.

Russell, C. B., An optimal policy for operating a multipurpose reservoir, Oper. Res., 20(6), 1181-1189, 1972.

Sobel, M. J., Chebyshev optimal waste discharges, Oper. Res., 9(2), 308-322, 1971.

Su, S. Y., and R. A. Deininger, Modelling the regulation of Lake Superior under uncertainty of future water supplies, in Proceedings of the International Symposium on Uncertainties in Hydrologic and Water Resource Systems, vol. 2, pp. 555-575, University of Arizona, Tucson, 1972.

SzegiS, G. P., and K. Shell, Mathematical Methods in Investment and Finance, North Holland, Amsterdam, 1972.

Topkis, D. M., Ordered optimal solutions, Ph.D. thesis, Stanford Univ., Stanford, Calif., 1968.

Veinott, A. F., Jr., Optimal policy for a multi-product, dynamic, non- stationary inventory problem, Manage. Sci., 12(3), 206-222, 1965.

Veinott, A. F., Jr., The status of mathematical inventory theory, Manage. Sci., 12(11), 745-777, 1966.

Veinott, A. F., Jr., Sub-additive functions on a lattice in inventory theory, paper presented at 19th International Meeting, Inst. of Manage. Sci., Houston, Tex., April 1972.

Veinott, A. F., Jr., and H. M. Wagner, Computing optimal (s, $) inventory policies, Manage. Sci., (5), 525-552, 1965.

(Received February 5, 1974; revised February 13, 1975; accepted March 20, 1975.)

Date post:	04-Nov-2015
Category:	Documents
Upload:	ahmedomaryusuf
View:	16 times
Download:	0 times

Sobel 1975 Water Resources Research

Documents