Dubious Tales from

Deep Dark Waters Past(Being a very selective history of hydro management

and NZEM origins.. with some observations on various kinds of bias)

E.G.Read with contributions from S. Starkey

Presented to EPOC Winter WorkshopAuckland, New Zealand

5 September 2013

CaveatThis presentation should not be

quoted, or relied upon

It has been largely prepared from memorywhich is inevitably selective and biased

(and not aided by the fact that any files I still have are currently in post-quake limbo-land)

Some Overview Papers • J.G. Culy, E.G. Read, and B. Wright: "Structure and Regulation of the New

Zealand Electricity Sector", in R Gilbert and E Kahn (eds.) International Comparison of Electricity Regulation, Cambridge University Press, 1996, p. 312-365.

• E.G. Read, J.G. Culy, and S.J. Gale: "Operations Research in Energy Planning for a Small Country", European Journal of Operational Research, vol. 56, 1992, p. 237-248.

• E.G. Read: "OR Modelling for a Deregulated Electricity Sector", International Transactions in Operations Research, vol. 3, no. 2, 1996, p. 129-138.

• E.G. Read: "Electricity Sector Reform in New Zealand: Lessons from the Last Decade” Pacific Asia Journal of Energy Vol 7, No 2, 1997, p. 175-191

Episode 1: Guidelines

1958 Meremere:– The Basic Rule Curve/ Forbidden Zone/Guideline– Basically a critical probability calculation

(as in the classic “newsagent” problem) – Backwards projection of minimum storage to

survive “Design Dry Year” (e.g. 1:20 if marginal cost of running out is 20 times marginal cost of fuel)

1967 Marsden:- Multiple Guidelines

Guideline Diagram (From Read and Boshier 1989)

Episode II: VALWAT/STAGE

Experimented with SDP involving a variety of forward simulation strategies (Programmed by Chris Lusk c.1979, reported by Read and Boshier, 1989)

Lead to abandonment of:– Deterministic LP model (Boshier and Lermit, 1977)

– Deterministic decomposition model (Read, 1983)

Produced STAGEDeveloped 1980ish, reported by Boshier et al (1983)Based on Swedish model by Stage and Larsson :– Uses “Marginalistic SDP”

- Setting MWVt = marginal value of release in t = E{MWVt+1 }…. Subject to bounds

– But with about 6 months “forward simulation” - Setting MWVt =E{ MWVt+26 , or spill/shortage value if storage bound reached sooner}

– Trying to capture effects of correlation - Spreading trajectories over a wider range - So typically raising expected MWVs (see later)

Optimal (SDP/STAGE) guidelines

Optimal deterministic guidelines assuming average inflows

Optimal guidelines from averaging deterministic optima for historical inflows

Optimal guidelines assuming naïve future management

Biases?The optimum is actually rather flat:– The two extreme solutions performed badly, but – While the SDP But was preferred for its

conservatism – It actually produced similar simulated

performance to the average of deterministic policies

Why do these biases occur, though?

The System Marginal Cost Curveis basically convex

Optimal management will try to control variability of marginal cost as best it can

Bias arises due to sub-optimal/unrealistic control of

variability

True expected marginal cost with optimal (well controlled) degree of variation

Under-estimated expected marginal cost from expected situation /perfect foresight

Over-estimated expected marginal cost due to sub-optimal (poorly controlled) degree of variation

Episode III: RESOP/PRISM>SPECTRA

“2-D Constructive Dual Dynamic Programming”– so named retrospectively by Read and Hindsberger(2010)– Conceived by Read (1984ish)– Programmed by Culy, Davies, and many more over the

years– Reported by Read et al(1987)

RESOP reservoir optimisation module:– Originally developed within a long term planning model

(PRISM) – Soon used for operational planning and much more – PRISM was later re-developed as SPECTRA

Basic Concept: “Guideline augmentation” by inserting “flats”

corresponding to varying utilisation level of one thermal station

“Uncertainty Adjustment” to produce new expected MWV curve

Another source of bias: decision period length

Notice that, a longer decision period means:– The flats get wider in proportion – The uncertainty adjustment interpolates over a

wider range

But the MWV curve is still basically convex, so:– The Expected MWV curve may be expected to rise– And also lose its detailed structure– Release decisions become more “moderate”

(since we could not stick with more extreme release levels for any length of time)

This was very evident in early RESOP experiments

A more recent example(using the model of Dye et al, 2012)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 210

200

400

600

800

1000

1200

Bi-monthly monthly

And another source of bias: Interpolation

True expected marginal cost

Over-estimated expected marginal cost due to naïve linear interpolation on a (relatively coarse) grid

Yet another source of bias: assuming independence

Cumulative pdf of

independent in flows

Effect of correlation

Counteracting biases in SPECTRAIndependent inflows:– Too optimistic (lowering MWV)

Linear interpolation on a fairly coarse grid– Too pessimistic/cautious (raising MWV)

Originally a coarse enough grid was used to give a cautious biasBut tuning was possible using a finer grid– And it was tuned to achieve the 1:20 criterion exactly– In 1992, which was worse than a 1:20 event– So we should have run out of water – And nearly did

After the 1992 crisis…Operational reliability standard raised:– From 1:20 DDY with 7% load reduction– To 1:60 DDY without load reduction (The operational optimum being quite flat due to excess supply due to overbuilding in NZED/MoE period)

Heuristic added to account for correlation in optimisation phaseby:

– Inflow pdf spreading σ'= σ(t) / √t

where σ(t) is the s.d of t week cumulative flow distribution

– So assuming independent inflows in optimisation gives effective cumulative inflow s.d of:

σ‘(t) = t * σ‘ / √t = σ(t)

Heuristic added to account for correlation simulation/operation phase

by:Augmenting storage level by projected inflow deviation when looking up MWV/release table: s‘(t) ≈ s(t)+

Where (r) is the r week correlation coefficient–And more sophisticated models are possible

And r is chosen to reflect some reasonable “influence” period– Which should depend on how close we are

to storage limits

Episode IV: Decentralisation

The 1992 crisis provided some impetus for change, by denting ECNZ’s reputation,– Quite unfairly IMHO

But some kind of “competitive” market was always on the agenda– And ECNZ’s dominance was obviously a much

more significant issue

Impact of decentralisation?

Less precise coordination– Less detailed information– Less sophisticated modelling

Greater risk aversion– Less pooling available to meet contract

commitments

More innovation????????Greater diversity of perspective/technique– Lower overall risk

Benefits of Diversity?

full

full

empty

empty

Manager 1

Manager 2

Diverse strategies reduce probability of extremes

Both empty

Both full

A Comment on Bias in Performance Evaluation

Performance evaluation seems much more difficult than most OR papers recogniseClassic paradigm:– I make assumptions/collected data/built a model – I show that my policy is better than yours… … given my assumptions/data/model – I get my work published, and you get no say in it

But reality is very different…… not least in that it is neither known nor agreed

What About Reality?Even if reality was as simple as an (agreed) inflow distribution:– We will not observe that reality, only randomised

outcomes – Policies will tend to be judged by what happened, not

what might have happened

Observers differ widely in their assessment of:– Underlying driving forces– Current data values – Forecasts– What ‘should’ drive decision-making – Their personal situation and stake – The interpretation of historical events

A Rational Approach?Given this variety – What seems “optimal” to one analyst may seem totally

irrational to another.– And vice versa!!– We should be (a)ware of (the extent to which we are)

privileging our own perspectives when comparing our own recommendations with those of others

That leaves us with a much more difficult task:– Assessing internal consistency with stated assumptions– Assessing robustness of outcomes in terms of a range

of criteria, across a range of possible assumptions

Or…

We could just entrust decision-making to those who have most at stake –And hope for “robust” outcomes

Knowing that these will always seem sub-optimal – From (almost) everyone’s perspective

Reservoir Management Outcomes?We should expect storage coordination to look “wrong”, from any individual perspectiveBut Tipping and Read (2010) • Tuned a model to show that, in aggregate, hydro was

operating as if using a plausible looking MWV curve • Subsequently tested to find storage policy– About as cautious as that under MoE – Somewhat more cautious than that under ECNZ– As should be expected?

BUTNone of this was really

what motivated the NZEM reforms

Advantages were seen in increased decision-making diversity– But maybe only enough to offset loss in

coordination efficiency

The real issue was creating a competitive market as a means of controlling cost and prices– Rather than relying on “Government”

At the time…History and common sense both seemed to suggest that it was politically impossible for a Government (of any stripe):– To make unbiased growth forecasts for the

economy (and hence for electricity)– To back away from plans and promises that turned

out to be unwise– To set fair, honest and realistic prices for industry,

commerce, or domestic consumers – To resist biasing technology choices

The fear was that: • Gains in operational efficiency at ECNZ would

be lost by gradual reversion to public sector norms, but particularly

• Forecasting, planning and pricing would be re-politicised– With potentially severe consequences for

allocative and investment efficiency– Which are where most of the sector’s costs are

incurred

Concluding Perspective Harker (2013) claims that the true energy component of domestic power bills has risen in recent years– Almost to the level reached under the MoE

in 1982

But….1982 was the only year, in the MoE period, in which

- After rising by 124% in 12 months- Prices (briefly) reached LRMC levels- As calculated using cost projections from SCM and

MWD- Most of which turned out to be significant under-

estimates30 years later, with cheap gas and hydro both gone:– The New Zealand electricity sector is still

(apparently) producing at those prices– And actually paying its own way!

If Dr Harker is correct …

I am personally astounded

How did such a marvellous thing ever come about?

Maybe Max was right?

More ReferencesJ.F. Boshier and R. J Lermit: A Network Flow Formulation for Optimum Reservoir Management of the New Zealand Power Generating System , NZOR vol5 #2., 1977, p85-10085J.F. Boshier, G.B. Manning, and E.G. Read: "Scheduling Releases from New Zealand's Hydro Reservoirs" Transactions of the Institute of Professional Engineers in New Zealand, vol. 10, no. 2/EMCh, July 1983, p.33-41.S. Dye, E.G. Read, R.A Read, S.R. Starkey “Easy Implementations of Generalised Stochastic CDDP Models for Market Simulation Studies” Proceedings 4th IEEE/Cigré International Workshop on Hydro Scheduling in Competitive Markets. Bergen, Norway, 2012B. Harker “Chairman's Address”, TrustPower AGM, July 2013 https://www.nzx.com/companies/TPW/announcements/239025E.G. Read: "Reservoir Release Scheduling for New Zealand Electricity - A Non-Linear Decomposition Algorithm", New Zealand Operational Research, vol. 11, no. 2, July 1983, p.125-142.E.G. Read, J.G. Culy, T.S. Halliburton, and N.L. Winter: "A Simulation Model for Long-term Planning of the New Zealand Power System", in G.K. Rand (ed.) Operational Research 1987, North Holland, p.493-507.E.G. Read: "A Dual Approach to Stochastic Dynamic Programming for Reservoir Release Scheduling", in A.O. Esogbue (ed.) Dynamic Programming for Optimal Water Resources System Management, Prentice Hall NY, 1989, p.361-372.E.G. Read and J.F. Boshier: "Biases in Stochastic Reservoir Scheduling Models", in A.O. Esogbue (ed.) Dynamic Programming for Optimal Water Resources System Management, Prentice Hall NY, 1989, p.386-398.E. G. Read and M. Hindsberger “Constructive Dual DP for Reservoir Optimisation” in S. Rebennback, P.M. Pardalos, M.V.F. Pereira and N.A. Iliadis (eds) Handbook on Power Systems Optimisation Springer, 2010, Vol I p3-32 J. Tipping and E. G. Read “Hybrid bottom-up/top-down modelling of prices in hydro-dominated power markets” in S. Rebennback, P.M. Pardalos, M.V.F. Pereira and N.A. Iliadis (eds) Handbook on Power Systems Optimisation Springer, 2010, Vol II, p213-238