Integrated multiscale models for the optimal integration of renewable
and distributed electricity generation
Panos Parpas∗
February, 2010
Abstract
This paper identifies the challenges involved with addressing the integration of renewable and
distributed electricity generation within the existing energy infrastructure. We outline a tentative
framework to address these challenges using a multiscale modeling methodology. The integrating
theme of the proposed methodology is suggested by the space–time separation grounded in the en-
gineering and physical properties of large power networks. The issue of multiple scales is addressed
using the framework of singular perturbation theory for the control of continuous time problems.
We show that the uncertainties that exist in energy planning problems, in particular load demand
uncertainty and uncertainties due to the power source, are highly structured. This structure can
be exploited to a great advantage that provides insight into the nature of the problem. At the
same time this structure can be used to perform a dimensionality reduction to the original model
and therefore allowing useful computation to be performed. We discuss two different case studies.
The first case study is concerned with the coupling of a capacity expansion model with a unit
commitment model. The question addressed here is how to incorporate hourly dynamics in the
capacity expansion model. The latter model is mainly concerned with investment decisions that are
taken only every few years and their effects are long lived. The hourly dynamics describe electricity
load demand and the intermittency of wind generation. This case study is thus concerned with
the large scale integration of renewables in an existing portfolio of power generation. We show
how to link the two models along their state dimension. The second case study addresses some of
the challenges when considering the large scale integration of distributed generation or microgrids.
This problem has a spatial aspect that is not present in the first case study and so we use it as a
∗Energy Initiative, M.I.T, Cambridge, MA. 02139. [email protected]
1
template to illustrate how one could approach problems with a strong spatial component using a
multiscale framework.
1 Introduction
Due to the manifold of technoeconomic issues that arise in energy planning, the electricity industry
has for a long time relied on complex optimization models to perform its planning. Examples of such
models include unit commitment (decisions on which generators to use), economic dispatch (decide
the optimal level of use for each generator), capacity expansion (decide the optimal generation mix
to meet demand), hydro/thermal scheduling models, optimal power flow models and so on. While
there exists a large body of literature to address the range of challenges faced by the energy industry,
the environment in which these models are used is becoming increasingly complex. This complexity
is driven primarily by two factors. The first factor is the deregulation and re-regulation of electricity
markets. The second driver is the increasing pressure placed on the electricity industry to address
the problem of meeting demand in a sustainable manner. In order to deal with this complexity either
simplified models or heuristic methods are typically used. These approximations, are to a large extend,
uncontrolled and may have a substantial impact on the nature of the obtained solution. In particular
most models ignore the fact that many features, that have important ramifications for the selection of
the optimal policy, exist along multiple temporal and spatial scales. The impact of the optimization
methodology, and the potential inefficiencies of the old heuristics can be seen by two examples taken
from [Bloom, 2009]. The first is from Red Electrica de Espana (rEE), the Spanish national power
distributor. They switched from using heuristic models to an exact optimization methodology and
saved about 50k€-100k€ daily, and reduced emissions by about 2.5% annually (about 100,000 tons of
CO2). A second examples is a large system operator in the east of the United States that saved about
$200 million annually by solving the unit commitment problem by a branch and bound algorithm as
opposed to the heuristic method they used before. These examples illustrate the impact that ad hoc
heuristics can have on the energy industry. Incorporating large amounts of intermittent renewable
generation resources or the integration of distributed sources of electricity generation, makes an already
complex problem more complex. It is therefore important to have robust models that can scale, both
computationally and conceptually, with the increasingly complex issues faced in the energy industry.
2
This paper identifies the challenges involved with addressing the integration of renewable and
distributed electricity generation within the existing energy infrastructure. We outline a tentative
framework to address these challenges using a multiscale modeling methodology. The integrating
theme of the proposed methodology is suggested by the space–time separation grounded in the engi-
neering and physical properties of large power networks. Before we delve into the proposed modeling
methods it is necessary to decompose the implications of the two drivers and sources of the complexity
in which planning decision need to be taken. We refer the interested reader to [Claudio et al., 2008]
for a more complete discussion of these issues. Below we summarize the issues that have repercussions
on the planning models discussed in this paper.
2 Background
The deregulation of the industry means that utilities now have to compete in an open market envi-
ronment. The degree of deregulation varies from country to country, and from state to state in the
US. Despite these variations between regions, as well as the fact that real-time system operation is
still largely a centralized activity, deregulation has completely changed the way electricity generation
and long term planning are performed. In a regulated environment the utilities would pass on the
costs of any risks in generation costs directly to consumers. This approach is no longer possible in an
open market. Consequently, utilities see generation plants as an asset that needs to be risk managed
and optimally utilized. The focus is no longer on cost minimization, but on how to deliver a quality
product in an economically viable manner. However, electricity is different from other assets since
large scale storage is not economically feasible and so it needs to be generated and transmitted at the
same time it is needed. Thus the utility needs to maintain an equilibrium between generation and
demand in a way that ensures the safety and stability of the system, while maximizing asset utilization
at the same time. An important implication of the non-storability of electricity for long term planing,
is that it is not enough to know the total electricity needed, but also when it is needed. A system
under constant demand has dramatically different optimal design than a system facing a demand pro-
file with substantial variations over time. This property of electrical power system design remains the
same even if the total demand required is the same but the demand profile over time is different. One
way to address both the increasing volatility in generation costs as well as the way utilities respond
3
to variations in load is via investing in a portfolio of generation facilities. For example, nuclear plants
have large upfront investment costs, generally lower fuel costs, and are ideal for serving base load.
On the other end of the spectrum, gas turbines are cheap to build but have high fuel costs, and can
be used to meet peak demand. Utilities have also ventured, with mixed results [Parsons, 2008], into
trading. The reason behind the change of business model was either for hedging, risk management
purposes or so as to establish an alternative source of income. Thus considering a large and diverse
number of technologies and economic activities is an important facet of long term planning in the
energy sector. In a nutshell, the consequences of deregulation to capacity expansion decisions are the
increased need to incorporate robust methods for handling uncertainty and the importance of model-
ing load variations. Both of these factors are important if a substantial increase in system efficiency is
to be achieved. All these new facets of the capacity expansion problem need to be considered in view
of many possible current and evolving technologies. It is no longer possible to model each decision
in isolation. A system wide view encompassing not only physical, operational, and safety constraints
but also economic and, as it will be discussed next, environmental constraints as well, is required.
Until recently, and to a large extend even now, the environmental costs of producing electricity have
not been incorporated fully into the price consumers see. The process of internalizing these costs into
the price is currently under way. This requires utilities to think differently about some technologies,
and their role in helping make the transition into a system that can grow sustainably. For example, if
a utility wants to investigate the possibility of including a large amount of wind generation capacity
into their power portfolio, then due to the intermittent nature of wind, daily variations in load and
daily variations in wind speeds become more important. These daily variations will affect the way the
utility manages its reserves, and therefore effects the whole portfolio of generating capacity. Since the
output from a wind turbine increases by the cube of wind speed, using average wind speed will always
underestimate the amount of output available from wind and this can lead to an inefficient system
design. Again a system wide view needs to be taken. Currently, there exist little understanding on
how to, rigorously, capture these daily effects into long term planning.
The rest of this paper is structured as follows: in Section 3 we describe the general framework
we propose in order to capture some of the multiscale challenges described above. In Section 4 we
discuss a case study that couples capacity expansion with a unit commitment model. The results from
4
section 4 are taken from [Parpas and Webster, 2010]. In Section 5 we attempt to apply the methods
from [Parpas and Webster, 2010] to a distributed network planning problem. The analysis in 5 is both
speculative and exploratory in nature. We conclude in Section 6.
3 Overview of the proposed framework
In Figure 1 we illustrate the multiscale nature of decisions that need to be taken in the power systems
sector. In an intraday scale a utility will be concerned with the safe operation of the power generating
plants in order to meet demand on a second by second basis. Problems such as voltage control
and frequency control are used for this purpose. As we move up on the temporal scale, pricing,
risk management and capacity expansion problems are considered. The last box shows the range
of issues that are important on the larger socioeconomic scale that power systems belongs to. The
Info
rm
ati
on
Arr
iva
l
Seconds /
Weeks< Second
Weeks/
Years
Years /
Decades
Engineering /
Control
Unit commitment
Economic Dispatch
Pricing
Stability Control
Frequency Control
Voltage Control
Physics Engineering /
Economics
Generation Expansion
Network Expansion
Maintenance
Fuel Management
Power Management
Risk Management
Pricing
Bidding Strategies
Macroeconomics /
Development
Equilibrium models
System dynamics
Environment
Security
Policy
Size
Generator /
Transmission line
Plant, Region
Country
World / Economy
Figure 1: Multiscale models and decision on the power system sector.
deregulation and environmental concerns described in Section 2 make it necessary for utilities and
policy makers to use models that can bridge the scales between the different facets of power system
planning. Decisions taken by one agent in one scale (e.g. what technology mix to invest in) affect issues
that have their dynamics rooted at a different scale (e.g. environment). What an economist might call
an externality (e.g. greenhouse gas emissions) can also be thought of as a problem of interacting scales
[Cash and Moser, 2000]. This is because the environmental phenomenon (greenhouse gas emissions)
is managed at a scale (utility companies) that have no or little incentives to incorporate the temporal
and spatial dynamics of the phenomenon into their planning process. At the same time, policy
5
instruments (e.g. carbon taxes) are designed with the purpose of mitigating long term environmental
impacts but also have an impact on the choice of technologies and amount of innovation at the firm
level. This type of technological change is not captured in large economic models concerned with
the long time scales. The problem is more complicated than what is shown in Figure 1. Agents on
different scales use different models and with different resolutions. For example, an economist may
use a detailed model of the US economy but only an approximated model for the EU economy. This
multi-model environment introduces additional complications that is beyond the scope of this paper.
Hitherto the problem of bridging the scales has been handled using ad-hoc methods e.g. [Wing, 2006,
Bohringer and Rutherford, 2008]. The objective of this paper is to describe a methodology to address
some aspects of this problem.
The modeling approach proposed in this paper is based on the observation that information per-
taining the issues on the left of Figure 1 arrive at a much faster rate than the information concerning
the issues on the right of the figure. The idea is then to use this property of the problem to identify
an integrated way of approaching these issues that is self-consistent, and transparent. The primary
objective is not to produce more accurate numbers by incorporating intraday variations, but rather
define the representational structures that will allow qualitative and quantitative reasoning about the
effects of different time scales to take place. At the same time this structure can be used to perform
a dimensionality reduction on the original model and therefore allowing useful computation to be
performed.
Figure 2(a) shows a pictorial representation of this research programme. Given two models M1 and
M2 that have different dynamics, the aim is to construct bridges along the spatial, temporal and state
dynamics of the two models. Figure 2(b) shows a different approach to integrated modeling where
each model has its own dynamics and the two models interact when needed, e.g. when the output of
one model is used as the input to the other. The proposed approach uses the conceptual framework of
figure 2(b). Both approaches offer viable solutions to integrated modeling. In order to be specific we
only discuss two case studies. These two problems provide a partial implementation of the programme
outlined above. The first case study is concerned with the coupling of a capacity expansion model with
a unit commitment model. The question addressed here is how to incorporate hourly dynamics in the
capacity expansion model. The latter model is mainly concerned with investment decisions that are
6
Time
State
Space
M1 M2
(a) Integrated multiscale model-ing
Time
State
Space
M1
Time
State
Space
M2
(b) Integrated modelling
Figure 2: Two approaches on integrated modelling.
taken only every few years and their effects are long lived. The hourly dynamics describe electricity
load demand and the intermittency of wind generation. This case study is thus concerned with the
large scale integration of renewables in an existing portfolio of power generation. The objective is to
show how to link the two models along their state dimension.
The second case study addresses some of the challenges when considering the large scale integration
of distributed generation or microgrids. This problem is addressed from the point of view of a system
operator that may be concerned with assessing the impact of various technologies or deciding the
optimal size of a microgrid. This problem has a spatial aspect that is not present in the first case
study and so we use it as a template to illustrate how one could approach problems with a strong
spatial component using a multiscale framework.
3.1 Literature Review
3.1.1 Literature related to Case Study I
Both deterministic and stochastic models exist that address different aspects of the capacity expansion
problem in the electricity industry. Capacity expansion problems have generated a large amount of
literature. This is mainly because expansion problems are applicable to many areas but also because
they are a good testbed for new modeling ideas. We will only discuss models that address the effect
of the different time scales. Even though some of the work discussed below does not address capacity
expansion directly, we believe that the most relevant papers to this work are the ones that address
multiple scales since their ideas could be used in the capacity planning problem.
In [Sen et al., 2006] the problem of incorporating different scales for addressing risk management
7
problems such as buying and selling forwards for fuel are addressed in conjunction with the intraday
unit commitment problem. Their model does not address intraday effects from intermittent sources.
They formulate the problem as a multistage stochastic programming problem. The resulting large scale
mixed integer linear programming problem is solved using a nested column decomposition algorithm.
In [Epe et al., 2009] the authors do address the problem of dealing with the intermittency of wind.
Again a multistage stochastic programming approach is taken and the resulting large scale optimization
problem is solved using a recombining tree methodology. In [Pritchard et al., 2005] the operation of a
hydro-electric reservoir is addressed. Their problem also has multiple scales since the supply of power
occurs intraday, in hourly intervals, but the management of the reservoir occurs over monthly scales.
The problem is formulated as a dynamic programming problem. By approximating the decision to
have some desirable properties the different scales can be decomposed. In [Powell et al., 2009] the
problem of multiple scales as well as the intermittency of wind and solar are handled using stochastic
dynamic programming. In order to solve the intractable dynamic programming problem approximate
dynamic programming is used. We refer to the review article in [Wallace and Fleten, 2003] and the
recent book [Weber, 2005] for a more complete overview of both stochastic programming approaches
as well as approaches based on dynamic programming. What all these papers have in common is that
they address the existence of multiple scales using some sort of algorithmic framework. They either
use a decomposition algorithm, approximate dynamic programming, or find some other way to relax
the non-anticipativity constraints in order to make the problem tractable.
To make operational the uncertainty structures present in this class of problems we make use of
the tools from singular perturbation theory for MDPs in continuous time. In some respects some
models already take advantage of this structure. For example the widely used MARKAL model
[Seebregts et al., 2001] uses the concept of a “typical” load to overcome the onerous requirement to
optimize over all possible loads. This type of heuristic is useful, but as we are trying to develop models
to understand the behavior of systems we do not observe it is important to understand why heuristics
work, when they fail, and what can be done instead. For example it is not clear how to extend these
heuristics to handle wind intermittency, or demand elasticity (a major objective of demand response
programs).
The main results for multiscale MDPs are summarized in two excellent books [Sethi and Zhang, 1994,
8
Yin and Zhang, 1998]. Therefore we only comment on the relations between this work and the lit-
erature. In [Jiang and Sethi, 1991] a similar model to ours is proposed. The model is motivated
by a manufacturing system with machines that have failure rates that occur on fast time scales.
However, their model does not address the issue of capacity expansion and demand is also determin-
istic. In [Zhang et al., 1997] another model similar to ours is proposed and is again studied in the
context of manufacturing systems. Their model also has fixed capacity and deterministic demand.
Even though the problem of capacity expansion with multiple scales has been studied, see Ch 10
in [Jiang and Sethi, 1991], their model does not address capacity expansion for Markov Chains with
weak interactions and multiple scales. Moreover all the papers described above are concerned mainly
with manufacturing systems, and in order to carry the results over to the power system sector we
had to devise a new framework. This is because in order to capture some of the basic features of the
energy planning models we needed a model that can handle a Markov chain with weak interactions
both on capacity (to model wind intermittency for example) as well as on the demand dynamics (to
model the demand load dynamics). We also need to handle uncertainties that are not changing fast,
such as the long term demand growth for electricity. In Section 4 we only provide an outline of these
extensions. The details can be found in [Parpas and Webster, 2010].
3.1.2 Literature related to Case Study II
The literature on network planning for power systems is vast. We refer the interested reader to
[Wallace and Fleten, 2003] and [Georgiadis et al., 2008] for a review. However there are currently
no studies that attempt to address the problem of optimal planning in the context of large scale
penetration of distributed generation technologies. There are a few studies that discuss the prob-
lem of a single network [Hawkes and Leach, 2009, Abu-Sharkh et al., 2006]. Some heuristic methods
for this problem are discussed in [Mendez et al., 2006]. In our analysis we rely on the results from
[Chow and Kokotovic, 1985]. The model discussed in [Chow and Kokotovic, 1985] pertains issues at a
smaller scale than the ones we propose to study. Also the work in [Chow and Kokotovic, 1985] has no
controls and is concerned with the deterministic case. We are interested in stochastic optimal control
problems.
9
4 Case Study I: State aggregation for capacity expansion
Before we delve into the details of the model, we first motivate our construction by looking at some of
the characteristics of some real electricity demand data. Hourly electricity load data time series have
a strong periodic component. By far the most widely used method to analyze such data sets is a two
step process. The first step involves identifying the periodic component in order to be able to remove
the trends that are related to time of day, weekend, and any other purely temporal elements. The
second step is to fit some stochastic process to the remaining component (we call this the stochastic
component). Hourly load data for 2007 were used from the PJM midatlantic region (PJM.E). This
data is freely downloadable from the PJM website.
Electricity hourly load demand is modeled as:
H(t) = D(t) + η0(t), t ∈ [0, T ]. (4.1)
The deterministic component is given by,
D(t) = a + bt +
N∑
j=0
cj cos(2πφjt + lj) t ∈ [0, T ]. (4.2)
Thus the deterministic component accounts for a linear trend in demand. The seasonal fluctuations
are super imposed around this linear trend. In our numerical experiments we used N = 5. The
deterministic harmonics of (4.2) are obtained from the peaks of the Fourier transform of the hourly
load data. In a more realistic model these parameters would be uncertain. The term η0 represents
the stochastic component. In order to identify a suitable stochastic process for η0 we fit a maximum
likelihood Markov Chain to the residual component. The residual component is given by,
r(t) = A(t) − D(t),
where A(t) represents the actual data (8760 points). The resulting transition matrix of the Markov
Chain is shown in Figure 3. The states in the Markov Chain represent the amount by which the
hourly load is above or below the deterministic periodic component (for ease of exposition the axis
shows the state number rather than the actual state). What is immediately obvious from this figure
10
is that apart from a few cases the Markov Chain will tend to stay around the same state. Of course
the variations around the state, as well as far away from the diagonal are important since they will
influence the amount of reserves. Figure 3(b) shows the result of the same process but with using
a Markov Chain with only ten states. Using typical states, optimizing over these and summing the
results, essentially assumes a transition matrix as in Figure 3(c) (an identity matrix). However using
typical states will always underestimate the costs of running such a system (this follows from Jensen’s
inequality and will be true for convex models, such as the one studied in this paper). This means that
a certain amount of guesswork will always be required in order to find the level of reserves, or any
other quantity of interest. Therefore, it seems that assuming the problem to be reducible to a few
typical states oversimplifies the problem. In view of the drive towards more efficient management of
increasingly scarce resources it is important to attempt to find better ways of incorporating hourly load
demand that leads to more realistic yet tractable models. Introducing uncertainty into demand load
management will give us a better understanding both of the true costs of the system as well as help
identify strategies to manage the dynamic equilibrium between supply and demand. At the same time
it is obvious from the figure that the data does not completely lack structure. To see the difference, a
process with no structure is shown in Figure 3(d). Therefore assuming a completely general structure
is also not appropriate. Methods that do not take advantage of this highly specialized feature of the
problem are essentially implicitly assuming a structure such as the one shown in Figure 3(d).
We followed a similar procedure for the wind data. We obtained wind data from the NCDC
[NCDC, 2009] website. The wind data contains the wind speeds at an hourly time interval over the
years 2007–2008, in the Buzzard’s bay area in south Massachusetts. The resulting Markov Chain for
wind uncertainty is shown in Figure 4.
The information structure proposed in this paper is shown in Figure 1. We start from the
bottom layer that has all the detailed intraday variations. This information is combined into ag-
gregate states. Finally some other probabilities are devised that control the transitions between
aggregate states. In order to make the structure in Figure 1 operational, we deploy the tools of
singular perturbation theory. Below we describe the main components of this framework. More infor-
mation can be found in [Yin and Zhang, 1998, Zhang et al., 1997, Phillips and Kokotovic, 1981] and
[Parpas and Webster, 2010].
11
(a) A Markov chain with 200 states (b) A Markov chain with 10 states
(c) (d) A structure free transition matrix
Figure 3: Markov transition matrices. (a)-(b) are obtained from a maximum likelihood fit to hourlytime series. (c) is the structure implicitly assumed when considering typical states. (d) was obtainedby a maximum likelihood fit to a white noise process.
12
(a) 20 states (b) 50 states
Figure 4: Markov transition matrices for wind uncertainty. (a)-(b) are obtained from a maximumlikelihood fit to hourly time series
Motivated by the data above we will assume a specific structure for the uncertainties representing
load demand and wind availability. To this end, the transition function is assumed to satisfy:
dP
dτ= p(τ)[Q + ǫW ],
p(0) = I.
(4.3)
where Q = diag(Q1, . . . , Qm). For the purposes of this section, we assume that the m aggregate states
have been identified by some statistical procedure. The ǫ parameter is used to make explicit the
assumption that the transitions inside one of the aggregate states are much more common than the
transitions between aggregate states. Since ǫ is assumed to be small, if we run the system for a small
amount of time then the W matrix will have a small role to play. However, if we run the system for
long periods then the transitions between aggregate states become significant. In order to capture the
latter effects, even as ǫ approaches 0, we stretch the time dimension t = ǫτ . This stretching of time
is what sets singular perturbations apart from regular perturbations. With this change of time (4.3)
13
becomes:
dP
dt= p(t)[
1
ǫQ + W ],
p(0) = I.
We note that we have not made any approximations so far. All we have assumed is that we can
identify the different aggregate states, and that we can specify an ǫ > 0 as a small parameter. This
parameter is used as a measure of the separation of scales. We are now in a position to fully specify
our model.
4.1 The model
We assume that the decision maker can use any of the N generating plants to meet demand
dx(t)
dt=
N∑
i
ui(t) − z(t) + η0(t, ǫ) x(0) = x. (4.4)
Where ui represents the output from the ith plant. x(t) represents the amount of energy not served.
We assume that some suitable penalty is imposed for meeting demand i.e. the cost of each unit of x
is high compared to the cost of production. z(t) represents the demand, and it is assumed to have the
following dynamics,
dz(t)
dt= f(z(t), ξ0(t)) z(0) = z. (4.5)
In (4.4) η0 is the fast scale Markov process that represents the intraday variations in hourly loads.
In (4.5) ξ0 is also assumed to be a finite state Markov process, but this process changes at a slower
rate. The latter process is used to represent uncertainties such as the long term trend demand for
electricity. Information about long term demand arrives at longer time intervals. Therefore it is not
appropriate to model this type of uncertainty as a fast scale process. We use the notation η and ξ, for
the fast, and slow scale dynamics throughout the paper.
We also assume that the decision maker can invest in each of the generating facilities with the
aim of increasing capacity. Some of the plants are allowed to have zero capacity to start with. The
14
cumulative investment, in the ith plant, is given by:
dyi(t)
dt= πi(t) yi(0) = yi, i = 1, . . . , N.
By imposing appropriate upper bounds on the rate of investment, πi, we can ensure that new capacity
cannot appear overnight. We assume that once total investment in plant i reaches level Ki then new
capacity becomes available. Each plant can be expanded once, and the level of investment required is
assumed to be deterministic. Both of these assumptions can be relaxed, but with significant increase
in the complexity of the analysis.
The total available capacity in the ith plant is assumed to be a Markov process, and it is given by:
ηi(ǫ, t) =
ηi1(ǫ, t) t ≤ τi
ηi2(ǫ, t − τi) t > τi
Where τi represents the stopping time:
τi = inf{t | yi(t) = Ki} ∧ T.
The states of ηi1(ǫ, t) and ηi
2(ǫ, t) are given by the set Si1 and Si
2 respectively. Where
Si1 = {si
1,1, si1,2, . . . , s
i1,mi
},
similar notation is used for Si2. We also assume that there exists a one-to-one mapping between states
in Si1 and Si
2, this will be denoted by κ : Si1 → Si
2. We use the notation Si to denote the states of
available capacity in the ith plant.
The Markov process η(t, ǫ) = [η0(t, ǫ), . . . , ηN (t, ǫ)] is assumed to have a generator given by:
dpη(t)
dt= pη(t)[W +
1
ǫQ] (4.6)
Where Q = diag(Q1, . . . , Ql). Furthermore both W and {Qi}li=1 are assumed to be irreducible. The
generator for the ξ variables can be much more general since we will not perform any approximation
15
on this process. The dynamics of ξ are given by,
dpξ(t)
dt= pξ(t)R.
A policy (u(t), π(t)) will be called admissible if it satisfies the dynamics specified above (including
the stochastic bounds on available capacity). Moreover an admissible policy needs to be adapted to
the filtration Ft. The set of all Ft-adapted processes is denoted by Aǫt. For notational simplification
we use w(t) = [x(t), y(t), z(t)] wherever possible.
The objective function and the complete problem are given below.
Jǫs(w, η, ξ;u, π) = E
{N∑
i=1
∫ T
s
e−ρ(t−s)Gi(xs, πi(s), ui(s), ξ(s))ds + e−ρ(T−s)φ(XT )
}
vs(w, η, ξ) = min Jǫs(w, η, ξ;u, π)
dx(t)
dt=
N∑
i
ui(t) − z(t) + η0(t, ǫ) x(s) = x
dz(t)
dt= f(z(t), ξ0(t)) z(s) = z
dy(t)
dt= π(t), y(s) = y (Hǫ)
yi(t) ≤ Ki
0 ≤ ui(t) ≤ ηi(t, ǫ)
(u(t), π(t)) ∈ Aǫt
Where Aǫt denotes the set of admissible controls. Note that the objective is allowed to depend on the
slow Markov process ξ. This may be used to represent uncertainties in operating costs (e.g. fuel costs,
emission costs etc.).
16
4.2 Asymptotic Analysis
As ǫ goes to zero, it can be shown [Parpas and Webster, 2010] that the value function of (Hǫ) is the
solution of the following aggregate problem.
vs(w, k, ξ) = min Js(w, k, ξ;u, π)
dx(t)
dt=
∑
j∈Mη(t)
λη(t)j
( N∑
i
uη(t),ji (s) − z(t) + η
η(t),j0
)x(s) = x
dz(t)
dt= f(z(t), ξ0(t)) z(s) = z
dy(t)
dt= π(t), y(s) = y (H)
yi(t) ≤ Ki
uη(t),ji (t) ∈ Ci(η(t), j)
η(s) = k.
Note that the value function is not a function of η but rather of the current aggregate state the process
is in. Ck(i, j) is the feasible set associated with the ith state inside the jth aggregate set of states for
technology k. In the case with no startup costs then this will simply be given by:
0 ≤ uη(t),ji (t) ≤ η
η(t),ji .
The expectation is taken with respect to the aggregated Markov Chain. That is the Markov Chain
that specifies the probability of moving between clusters of states. The complete derivation of the
approximate problem can be found in [Parpas and Webster, 2010]. The important feature in (H)
is that we only consider aggregate sets of states. This has significant computational advantages.
Additional numerical experiments from [Parpas and Webster, 2010] show that the algorithm used to
solve (H) requires a lot less iterations to converge than when the same algorithm is used on the
original problem. This is because of the so called smoothing property for such problems. We refer the
interested reader to [Parpas and Webster, 2010] for the details.
17
5 Case Study II: Spatial aggregation for distributed generation plan-
ning
Traditionally the physical flow of electricity has been fixed and centrally managed. Large generating
plants (hydro, nuclear or fossil-fired) generate power, and through the high voltage transmission net-
work, it reaches the low voltage network and eventually gets distributed to consumers. The economics
of power generation made this mode of operation efficient. Deregulation, concerns over environmental
impacts of power generation and security issues are challenging this traditional structure of power
generation. Distributed generation is the term used to describe a set of technologies that are located
close to the load demand centers. Since these distributed resources are not centrally managed they
pose a fresh set of challenges for the modeling of power grids. The term microgrid is used to de-
scribe a set of decentralized technologies operating to provide heat and power to a small set of mostly
residential customers. The technologies used in microgrids include both renewables sources such as
solar, wind, but also fossil-fuel based generators. A distinctive feature of microgrids is the empha-
sis on polygeneration. A microgrid can operate in isolation to the large national grid. It may also
operate in tandem with the main grid. The tandem operation will be needed if the microgrid can
not generate enough local power to satisfy local demand, or when too much power is generated and
then it can be sold back to the main grid. More information on the engineering aspects of distributed
generation can be found in [Masters, 2004]. Most studies have looked at the impact of a single mi-
crogrid (e.g. [Hawkes and Leach, 2009, Abu-Sharkh et al., 2006]). With some notable exceptions (see
[Mendez et al., 2006]) the large scale integration of microgrids has not been sufficiently analyzed. The
aim of this section is to show how a large number of microgrids can be analyzed together using similar
techniques as in Section 4.
5.1 The model
The first step in extending the techniques of section 4 is to endow the Markov process, representing the
load demand or any other uncertain quantity such as the availability of wind, with a spatial structure.
This class of stochastic processes are called random fields, see e.g. [Whittle, 1986]. Random fields
have been studied extensively. The optimal control of Markov random fields has received very little
attention compared with the standard Markov decision processes. Most of the work on the control of
18
spatial Markov processes is reviewed in [Chornei et al., 2006].
Let Γ = (V,Be, Bp) denote an undirected graph, where V denotes the set of vertices, and Bp
the set of physical edges, and Be the electrical edges. A physical edge exists between each node if
there is conditional dependency (in the sense defined below) between two nodes. An electrical edge
between two nodes means that a node is electrically connected with another node. Obviously this
is an idealization of how a large electrical network might be connected, however when attempting to
incorporate large amounts of intermittent wind generation it becomes important to separate the two
concepts. The graph is illustrated in Figure 5. The electrical graph should be easy to identify given
the actual configuration of the network. The graph related to the physical aspects of the network may
be different, even though one would expect that nodes that are close to each other should belong to the
same neighborhood. Each node on the graph will represent a single source of distributed generation.
Depending on the application this may be down to a single microgrid or a cluster of microgrids.
By k ↔ j we denote that vertex j is connected to vertex k. For each k ∈ Bp we define the set
Np(k) = {j ∈ Bp | k ↔ j} as the set of all neighboring vertices to vertex k. We use Np(k) = Np(k)∪k.
We use similar notation for Be. We further assume that the graph can be split into l different areas.
We use Mk to denote the set of nodes in area k.
At each node on the graph we use cIi (cE
i ) to denote the number of internal (external) connections
at node i. We define,
cI = mini{cI
i } cE = maxi
{cEi } ǫ1 =
cE
cI.
Therefore as ǫ1 gets smaller, one would expect to be able to analyze each area of the graph separately
without much loss of information. Next we define sparsity parameters for each area. We use sIk (sE
k )
to denote the number of internal (external) connections in area k. We use m = mink{|Mk|}. As in
the case of the node sparsity parameter we also define,
sI = mink
{sIk} sE = max
k{sE
k } ǫ2 =sE
mcI.
Intuitively, one would expect to be able to analyze each area separately when ǫ1 and ǫ2 are small
enough. Similar techniques as in the previous section could be used. In what follows we formulate a
simple network in order to make this operational.
19
The stochastic process ηi(t, ǫ) will be used to represent the random variables at vertex i. We will
assume the following,
P [ηi(τ, ǫ) = η | Ft] = P [ηi(τ, ǫ) = η | ηj(t, ǫ), j ∈ Np(i)], τ > t, (5.1)
where Ft =⊗
k∈V Fk. It is clear that the preceding equation is a generalization of the familiar
Markov condition to the case when the problem has spatial structure. See [Whittle, 1986] for more
details regrading the properties of such processes. It is natural to expect this type of processes to be
useful for the modeling of wind availability. Demand uncertainty can be assumed to be conditionally
independent from its neighbors (after the temporal component has been eliminated by the methods
described earlier). Consider the following equations describing the state of vertex i in area k
(a) Physical Graph (b) Electrical Graph
Figure 5: The two types of graphs representing the spatial structure of the model.
dxki
dt= rǫ
k,i(t) +∑
j∈Ne(i)
Lij , (5.2)
where,
rǫk,i(t) =
ni∑
j=1
ujk,i(t) − η
i,k0 (t, ǫ),
represents the power produced locally by the ith microgrid. As before u represent the power output,
and each node is assumed to have an ni number of generators. Lij represents the net injection of power
between nodes i and j. In general Lij would be a control. However in order to make the ensuing
20
analysis tractable we will assume that,
Lij = Lij(xi − xj).
Note that this function must satisfy an energy balance condition,
∑
i
∑
j∈Ne(i)
Lij = 0.
To help make the analysis concrete we assume the following,
Lij =
λij(xj − xi) if i ↔ j
0 otherwise
The general case of nonlinear functions can be handled in a similar way as in [Chow, 1982]. Under
these assumptions the system in (5.2) can now be written as,
dxki
dt= rǫ
k,i(t) +∑
j∈Mk
∑
j 6=i
λij(xkj − xk
i ) +
l∑
m6=k
∑
j∈Mm
λij(xkj − xk
i ). (5.3)
Motivated by the decomposition above, we define LI ∈ Rn×n to be a matrix containing the internal
connections in an area. This implies that LI must be block diagonal. The diagonal entries in this
matrix are chosen to satisfy,
LIii = −
∑
j∈Mk
λij ∀i ∈ Mk ∀k.
The off-diagonal entries of LI are chosen to equal λij provided that i and j belong to the same group.
Similarly we define LE ∈ Rn×n to be a matrix containing external connections. Thus LE satisfies the
following two conditions,
ΛEij =
λij if i 6= j and i ∈ Mk, j ∈ Mm m 6= k
−∑
m6=k
∑j∈Mm
λij if i = j, i ∈ Mk
0 otherwise.
21
Note that by definition L = LI + LE . With the definitions given above we can decompose the system
into its external and internal connections.
dx
dt= rǫ(t) + (LI + LE)x(t), . (5.4)
Define the following aggregate variable,
yk =
|Mk|∑
i=1
xki
|Mk|=
1
|Mk|1T|Mk|
xk,
where 1n denotes a vector of ones of dimension n. We can vectorize the expression above as follows,
y = Cx.
Where,
C , M−1UT ∈ Rl×n,
U , diag(1|M1|, . . . ,1|Ml|) ∈ Rn×n,
M , diag(|M1|, . . . , |Ml|) ∈ Rl×l.
Next we define the first node in each neighborhood as a reference node,
zki−1 = xk
i − xki−1 i = 2, . . . ,Mk.
We will use z = Gx, to denote the equation above in vector form. The matrix G is defined in the
obvious way. It is shown in the Appendix that after some change of variables the following scaled
system can be derived,
dy
dτ= G1r
ǫ(t) + A1y + A2z
ǫ2dz
dτ= G1r
ǫ(t) + ǫ1B1y + B2z.
It can be seen from the system above that ǫ2 is a singular perturbation parameter, while the parameter
ǫ1 serves the role of a regular perturbation parameter. The significance of the system in 5.1 is that it
22
represents a controlled approximation of the aggregated dynamics of the original system. As there are
three small parameters for this problem1, the asymptotic analysis as these parameters tend to zero is
much more complicated. Despite the increased complexity there are several ways one could proceed.
One could solve the optimal control problem by letting ǫ approach to zero. At this limit the dynamics
representing the uncertainties in net power flow in each area would be obtained. The solution will
contain two sources of error. The first is due to taking the limit ǫ → 0. The second source of error is
due to using the aggregated dynamics instead of the correct dynamics. Error estimates for the latter
source of error could then be derived using standard techniques (see e.g. [Chow and Kokotovic, 1985]).
Due to space limitations we omit the detailed implementation of this approach. The details will appear
elsewhere.
6 Conclusions
Optimization models have been used in the electric power industry for a long time. Environmental
considerations and the development of new distributed generation technologies make it necessary to
consider the properties of power networks at multiple scales and from the point of view of multiple
agents. This paper proposed a tentative framework to address some of these challenges.
Appendix
In the Appendix we show how the perturbed dynamics in (5.1) are derived. These calculation were
done in [Chow and Kokotovic, 1985]. One difference in the computations below is the existence of
control variable representing the net injection of power in a node. The other difference is the existence
of uncertainty. Using the same notation as in Section 5, we obtained the following two expressions for
the nodes on the graph,
y = Cx z = Gx,
It follows from [Aoki, 1968] that we can derive the following expression for x,
x = Uy + Wz,
1Remember that ǫ is the singular perturbation parameter for the uncertainties around net power injection.
23
where W = GT (GGT )−1. It follows that,
dy
dt= C
dx
dt= C(rǫ(t) + (LI + LE)x(t))
= C(rǫ(t) + (LI + LE)(Uy(t) + Wz(t)))
= Crǫ(t) + CLEUy(t) + CLEWz(t).
(6.1)
Similarly, for z we obtain the following,
dz
dt= G
dx
dt= G(rǫ(t) + (LI + LE)x(t))
= Grǫ(t) + GLEUy(t) + (GLEW + GLIW )z(t).
(6.2)
The significance of the reformulation of the system above is that the matrix with the internal links
LI appears only in the last expression in the system above. Next we observe that if ǫ1 and ǫ2 are
sufficiently small2 then one would expect the norm of (GLEW + GLIW ) to be much larger than the
other matrixes in (6.1) and (6.2). This motivates the following change of time t1 = ǫ1t. We also rescale
the matrices to that everything is of O(1). We use the following definitions,
A1 ,CLEU
cIǫ2A2 ,
CLEW
cIǫ2
B1 ,GLEU
cIǫ1B2 ,
G(LI + LE)W
cI.
G1 ,G
cI
With the rescaled dynamics introduced above we obtain the following dynamics,
dx
dt1= ǫ2G1r
ǫ(τ) + ǫ2A1y + ǫ2A2z
dz
dt1= G1r
ǫ(τ) + ǫ1B1y + B2z.
The final system is obtained by rescaling the system above in its slow time scale,
τ = ǫ2t1
2Remember that ǫ1 is the perturbation parameter representing the ratio of external to internal connection in aneighborhood. The other parameter ǫ2 represents the permutation parameter that represents the sparsity of the graphin relation to the number of nodes and number of internal connections.
24
dx
dτ= G1r
ǫ(t) + A1y + A2z
ǫ2dz
dτ= G1r
ǫ(t) + ǫ1B1y + B2z.
References
[Abu-Sharkh et al., 2006] Abu-Sharkh, S., Arnold, R., Kohler, J., Li, R., Markvart, T., Ross, J.,
Steemers, K., Wilson, P., and Yao, R. (2006). Can microgrids make a major contribution to UK
energy supply? Renewable and Sustainable Energy Reviews, 10(2):78–127.
[Aoki, 1968] Aoki, M. (1968). Control of large-scale dynamic systems by aggregation. IEEE Trans-
actions on Automatic Control, 13(3):246–253.
[Bloom, 2009] Bloom, J. (2009). Optimization Applications in the Energy and Power Industries.
http://www-01.ibm.com/software/websphere/industries/energy/.
[Bohringer and Rutherford, 2008] Bohringer, C. and Rutherford, T. (2008). Combining bottom-up
and top-down. Energy Economics, 30(2):574–596.
[Cash and Moser, 2000] Cash, D. and Moser, S. (2000). Linking global and local scales: designing
dynamic assessment and management processes. Global Environmental Change, 10(2):109–120.
[Chornei et al., 2006] Chornei, R., Daduna, H., and Knopov, P. (2006). Control of spatially structured
random processes and random fields with applications. Springer Verlag.
[Chow, 1982] Chow, J. (1982). Time-scale modeling of dynamic networks with applications to power
systems. Springer-Verlag.
[Chow and Kokotovic, 1985] Chow, J. and Kokotovic, P. (1985). Time scale modeling of sparse dy-
namic networks. IEEE Transactions on Automatic Control, 30(8):714–722.
[Claudio et al., 2008] Claudio, C., Gomez-Exposito, A., Conejo, A., Canizares, C., and Antonio, G.
(2008). Electric Energy Systems: Analysis and Operation. CRC.
[Epe et al., 2009] Epe, A., Kuchler, C., Romisch, W., Vigerske, S., Wagner, H., Weber, C., and Woll,
O. (2009). Optimization of Dispersed Energy Supply-Stochastic Programming with Recombining
Scenario Trees. Optimization in the Energy Industry, page 347.
25
[Georgiadis et al., 2008] Georgiadis, M., Pistikopoulos, E., and Kikkinides, E. (2008). Energy Systems
Engineering: Volume 5: Energy Systems Engineering. Wiley-VCH.
[Hawkes and Leach, 2009] Hawkes, A. and Leach, M. (2009). Modelling high level system design and
unit commitment for a microgrid. Applied Energy, 86(7-8):1253–1265.
[Jiang and Sethi, 1991] Jiang, J. and Sethi, S. (1991). A state aggregation approach to manufacturing
systems having machine states with weak and strong interactions. Operations Research, 39(6):970–
978.
[Masters, 2004] Masters, G. (2004). Renewable and efficient electric power systems. Wiley, Inter-
science.
[Mendez et al., 2006] Mendez, V., Rivier, J., Fuente, J., Gomez, T., Arceluz, J., Marin, J., and
Madurga, A. (2006). Impact of distributed generation on distribution investment deferral. In-
ternational Journal of Electrical Power and Energy Systems, 28(4):244–252.
[NCDC, 2009] NCDC (2009). National Climatic Data Center,
http://www.ncdc.noaa.gov/oa/ncdc.html.
[Parpas and Webster, 2010] Parpas, P. and Webster, M. (2010). A singular perturbation approach to
multiscale power system capacity expansion. Submitted.
[Parsons, 2008] Parsons, J. (2008). Do Trading and Power Operations Mix? The Case of Constellation
Energy Group 2008. Massachusetts Institute of Technology, Working Papers, Center for Energy and
Environmental Policy Research.
[Phillips and Kokotovic, 1981] Phillips, R. and Kokotovic, P. (1981). A singular perturbation ap-
proach to modeling and control of Markov chains. IEEE Transactions on Automatic Control,
26(5):1087–1094.
[Powell et al., 2009] Powell, W., George, A., Lamont, A., and Stewart, J. (2009). SMART: A Stochas-
tic Multiscale Model for the Analysis of Energy Resources, Technology and Policy. Preprint.
[Pritchard et al., 2005] Pritchard, G., Philpott, A. B., and Neame, P. J. (2005). Hydroelectric reservoir
optimization in a pool market. Math. Program., 103(3, Ser. A):445–461.
26
[Seebregts et al., 2001] Seebregts, A., Goldstein, G., and Smekens, K. (2001). Energy/environmental
modeling with the MARKAL family of models. In Proc. Int. Conf. on Operations Research (OR
2001).
[Sen et al., 2006] Sen, S., Yu, L., and Genc, T. (2006). A stochastic programming approach to power
portfolio optimization. Oper. Res., 54(1):55–72.
[Sethi and Zhang, 1994] Sethi, S. P. and Zhang, Q. (1994). Hierarchical decision making in stochastic
manufacturing systems. Systems & Control: Foundations & Applications. Birkhauser Boston Inc.,
Boston, MA.
[Wallace and Fleten, 2003] Wallace, S. and Fleten, S. (2003). Stochastic programming in energy. In
A., R. and A., S., editors, Handbooks in OR & MS, chapter 10. Elsevier Science.
[Weber, 2005] Weber, C. (2005). Uncertainty in the electric power industry: methods and models for
decision support. Springer Verlag.
[Whittle, 1986] Whittle, P. (1986). Systems in stochastic equilibrium. John Wiley & Sons, Inc. New
York, NY, USA.
[Wing, 2006] Wing, I. (2006). The synthesis of bottom-up and top-down approaches to climate policy
modeling: Electric power technologies and the cost of limiting US CO2 emissions. Energy Policy,
34(18):3847–3869.
[Yin and Zhang, 1998] Yin, G. G. and Zhang, Q. (1998). Continuous-time Markov chains and ap-
plications, volume 37 of Applications of Mathematics (New York). Springer-Verlag, New York. A
singular perturbation approach.
[Zhang et al., 1997] Zhang, Q., Yin, G., and Boukas, E. (1997). Controlled Markov chains with weak
and strong interactions: asymptotic optimality and applications to manufacturing. Journal of
Optimization Theory and Applications, 94(1):169–194.
27