1 Electricity Generation Efficiency Measures: Fixed Proportion Technology Indicators Darold T Barnum (corresponding author) Department of Managerial Studies Department of Information & Decision Sciences University of Illinois at Chicago (MC 243) 601 South Morgan Street Chicago, IL 60607-7123 USA (v) 1-312- 996-3073 (f) 1-312-996-3559 [email protected]John M Gleason Department of Information Systems and Technology College of Business Administration Creighton University Omaha, NE 68178 USA (v) 1-402-330-2332 [email protected]Darold T Barnum is Professor of Managerial Studies and Professor of Information and Decision Sciences, University of Illinois at Chicago. He earned his Ph.D. in Business and Applied Economics at the University of Pennsylvania. His recent research focuses on DEA theory and application. He has published in journals such as Ecological Economics, Socio-Economic Planning Sciences, Management Science, Industrial and Labor Relations Review, Applied Economics, Interfaces, International Transactions in Operational Research, IEEE Transactions, and Journal of Transportation Engineering. His forthcoming article in Ecological Economics is a joint environmental and cost efficiency analysis of electricity generation. John M Gleason is Professor Emeritus of Decision Sciences, College of Business Administration, Creighton University, Omaha, NE. He earned his D.B.A. in with double majors in Quantitative Business Analysis and Logistics from the Indiana University Graduate School of Business. His recent research focuses on DEA theory and application. He has published in a variety of journals, including Socio- Economic Planning Sciences, Management Science, International Transactions in Operational Research, IEEE Transactions on Engineering Management, OMEGA, Industrial and Labor Relations Review, Decision Sciences, Risk Analysis, International Journal of Industrial Engineering, Applied Economics, Journal of Transportation Engineering, Interfaces, and Government Information Quarterly. He has received honors and awards for research productivity from governmental, professional, and university organizations.
Darold T Barnum is Professor of Managerial Studies and Professor of Information and Decision Sciences, University of Illinois at Chicago. He earned his Ph.D. in Business and Applied Economics at the University of Pennsylvania. His recent research focuses on DEA theory and application. He has published in journals such as Ecological Economics, Socio-Economic Planning Sciences, Management Science, Industrial and Labor Relations Review, Applied Economics, Interfaces, International Transactions in Operational Research, IEEE Transactions, and Journal of Transportation Engineering. His forthcoming article in Ecological Economics is a joint environmental and cost efficiency analysis of electricity generation. John M Gleason is Professor Emeritus of Decision Sciences, College of Business Administration, Creighton University, Omaha, NE. He earned his D.B.A. in with double majors in Quantitative Business Analysis and Logistics from the Indiana University Graduate School of Business. His recent research focuses on DEA theory and application. He has published in a variety of journals, including Socio-Economic Planning Sciences, Management Science, International Transactions in Operational Research, IEEE Transactions on Engineering Management, OMEGA, Industrial and Labor Relations Review, Decision Sciences, Risk Analysis, International Journal of Industrial Engineering, Applied Economics, Journal of Transportation Engineering, Interfaces, and Government Information Quarterly. He has received honors and awards for research productivity from governmental, professional, and university organizations.
Abstract Purpose – Labor, capital and energy inputs are not substitutable for each other in the generation of electricity. This conflicts with the Data Envelopment Analysis (DEA) requirement of input substitutability. This paper seeks to demonstrate the conflict’s impact on DEA efficiency estimates, and to develop efficiency indicators appropriate for fixed proportion technologies. Design/methodology/approach – The paper illustrates the effects of nonsubstitutable inputs on efficiency measurement, develops efficiency measures appropriate for fixed proportion technologies, and, using a sample of 70 electricity generation plants, compares the new measures’ estimates with those of conventional DEA measures. Findings – When compared to efficiency measures designed for fixed proportion technologies, DEA badly overestimates the relative efficiency of some plants and badly underestimates the relative efficiency of others. Research limitations/implications – The new measures may have undiscovered shortcomings. Hopefully, other researchers will seek to improve them, as well as developing additional new indicators. Practical implications – The paper’s findings suggest that conventional DEA models should no longer be used to estimate the efficiency of electricity generation plants. Only efficiency indicators accounting for fixed proportion technologies should be used to estimate plant efficiency. Originality/value – Because of the energy and environmental crises we are facing, it is critical that power plant efficiency be validly measured. Thus, governmental policy makers, utility managers and other decision makers will base their actions to improve efficiency on correct information. Replacing widely-used DEA models with more valid efficiency measures will help achieve this goal. Keywords: Fixed factor proportion technology, Data envelopment analysis, Efficiency measurement, Electricity generation plants, Energy production efficiency Classification: Research paper
In their survey of energy and environmental efficiency studies, Zhou, Ang and Poh (2008) report that
a growing number of publications use Data Envelopment Analysis (DEA). They conclude that
“Considering the importance of energy efficiency study and the ability of DEA in combining multiple
factors, it is reasonable to believe that DEA [will] play a more important role in energy efficiency studies
in [the] future (Zhou et al., 2008, p. 11).” Further, they find that the largest number of these DEA
efficiency studies concern electric utilities.
The large and growing body of DEA studies of electric utilities is commendable, given the critical
importance of energy efficiency and pollution control, and the role played by electricity generation in
both (Welch and Barnum, 2009). However, as described later in this introduction, DEA theory requires
that resource inputs be substitutable for each other, but electricity generation inputs are not substitutable.
Therefore, DEA electricity generation applications conflict with DEA theory. Whether DEA estimates of
electricity generation efficiency are materially affected by nonsubstitutable inputs has not been studied.
The first essential step for successfully improving electricity generation efficiency is to validly
measure efficiency. If efficiency is not measured correctly, then, based on the invalid information,
governmental policy makers, utility managers and other decision makers may respond in ineffective or
even harmful ways. In short, it is essential that efficiency be correctly measured before effective
solutions for its improvement can be implemented. Therefore, it is worthwhile to reconsider the DEA
efficiency measures being used, and, if they prove to be significantly biased, then to suggest alternative
efficiency measures for this industry.
1.1 DEA theory
DEA theory assumes substitutability among inputs. To elucidate, recall that an isoquant identifies the
various combinations of two substitutable inputs that yield a constant output when employed by efficient
producers. An isoquant can exist only if inputs are substitutable, and, further, the various possible input
combinations must form a convex set (Petersen, 1990). Farrell (1957, p. 256) states (italics added), “The
isoquant SS' represents the various combinations of the two factors that a perfectly efficient firm might
use to produce unit output. . . . The curve SS', then, will be taken as the estimate of the efficient isoquant.”
Charnes, Cooper and Rhodes (1978, p. 437) emphasize (italics added) that their “analysis has employed
two assumptions which we shall refer to as the 'isoquant' and 'ray assumptions', respectively. . . . The
former, i.e., the isoquant assumption representation, is critical and it may not be relaxed . . . .” Banker,
Charnes and Cooper (1984) identify the set of axioms required by DEA, including the convexity
postulate (p. 1081) which can only be true if the inputs (and outputs) are substitutable: “If
and ( , ) , 1,..., ,j jX Y T j n∈ = 0jλ ≥ are nonnegative scalars such that 1
1,nji
λ=
=∑ then
( 1 1,n n
j j j jj j ), .X Yλ λ= =∑ ∑ T∈ ” Färe, Grosskopf and Lovell (1994, pp. 39-44) state “We are now
prepared to introduce three important subsets of the boundaries of more sets and less sets. These subsets
form the reference [isoquant] sets relative to which efficiency is measured.”
1.2 DEA applications to electricity generation
Inputs in DEA studies of electricity production most frequently include installed electricity generation
capacity as a measure of capital, average number of employees as a measure of labor, and BTUs or
physical units (such as tons of coal) as a measure of fuel. We identified 24 DEA papers published
between 1983 and 2008 that studied electricity generating plants. Of the 24, 18 used installed generator
capacity as a proxy for capital input, 20 used number of employees as proxy for labor input, and, as a
proxy for fuel input, 20 used either some measure of heat production such as BTUs (13) or physical units
(7). Thus, by far, the most common configuration of inputs in past studies is installed generator capacity,
number of employees, and BTUs.
As we demonstrate empirically later in this paper, these three inputs cannot be substituted for each
other in the production of a fixed amount of electricity output. Employees cannot be substituted for
BTUs, BTUs cannot be substituted for generator capacity, and generator capacity cannot be substituted
4
5
for employees. That is, these inputs cannot replace each other in the production of a given amount of
output, because there is no factor substitutability. Such production technologies often are referred to as
Fixed Factor Proportion Technologies (Beattie and Taylor, 1985).
1.3 Purpose and organization of this paper
The purpose of this paper is to demonstrate the impacts of DEA theory’s substitutability requirement on DEA
application’s efficiency estimates for electricity generation plants. First, we provide a detailed illustration of the
effects of nonsubstitutable inputs on the measurement of efficiency. Next, we statistically analyze a sample of 70
electricity generating plants with data for the five years 2003-2007, to test whether or not capital, labor and fuel are
substitutable for each other in the production of electricity. Then, we develop an additive efficiency measure, the
Fixed Proportion Additive (FPA) measure, which deals with fixed proportion technologies. After doing so, we use
the preceding electricity generation plant data to compare the FPA measure’s efficiency estimates to those of the
conventional DEA Additive efficiency measure. Next, we develop a ratio measure, the Fixed Proportion Ratio
(FPR), for fixed proportion technologies. Again using the preceding data, we compare the FPR efficiency estimates
to those of the Charnes-Cooper-Rhodes (CCR) measure and the Enhanced Russell Measure (ERM). Finally we
present our conclusions.
2. Illustration of the effects of nonsubstitutability
Consider a hypothetical case of one output and two inputs (a) when the inputs are substitutable, and
(b) when the inputs are not substitutable. Suppose one unit of output is produced by the DMUs with
various combinations of the inputs (Figure 1).
If the two inputs shown in Figure 1 are substitutable, then the input best-practice frontier is
represented by the piecewise isoquant A-B-C-D-E (ignoring composite DMU F for the time being).
Conventional DEA models also would report that these five DMUs are efficient.
If the two inputs are not substitutable, then at most one of these five DMUs could be efficient. For
example, if it were true that, to produce one unit of output, a minimum of 1.28 units of input 1 and 0.18
units of input 2 were needed, then the DMU at point C would be efficient because it combines the
required amounts and proportion (1.28/0.18 = 7.11) of the inputs needed to efficiently produce one unit of
6
output. It would not be possible for a DMU to produce one unit of output with the inputs available at
point D (1.48, 0.14). Indeed, at point D, the required ratio of inputs would result in 0.14 * 7.11 = 1.00
unit of input 1 and 0.14 units of input 2 being used (because only 0.14 units of input two are available),
thereby wasting 0.48 units of input 1 (which has 1.48 units of input available). Furthermore, the amount
of output produced would be only 0.14/0.18 = 0.78 units.
The preceding example is merely illustrative, of course, because at present we don’t know whether the
inputs are nonsubstitutable or, even if they are, what the correct ratio of inputs would be. Indeed, because
we know that DMUs A, B, C, D and E all produced one unit of output, it might appear that the empirical
evidence suggests that these two inputs are substitutable for each other.
But, substitutability is not necessarily present for two reasons. The first is the presence of noise in the
data. Because of the ubiquitous presence of noise, it is inevitable that several DMUs will be on an
illusory piecewise isoquant frontier, even if the inputs are not substitutable. The second reason is that a
DMU is unlikely to be equally efficient in its use of both inputs. In Figure 1, for example, if inputs are
truly nonsubstitutable, the supposed piecewise isoquant frontier may be the result of A being the most
efficient of all DMUs in the use of input 1 but less efficient in the use of input 2, and E being the most
efficient of all DMUs in the use of input 2 but less efficient in the use of input 1. Substitutability, or the
lack thereof, can be identified by logic and/or by statistical testing, but not by a deterministic empirical
examination of an alleged best-practice frontier.
In fact, the two inputs in Figure 1 are nonsubstitutable. One is installed generator capacity in
megawatts (mW) and the other is number of full-time-equivalent employees for a set of electricity
generating plants, with each plant’s two inputs divided by its megawatt-hours (mWh) of output so they
represent the inputs used to produce one megawatt-hour of output. It would not be possible to maintain
constant output by decreasing generating capacity while increasing employment to compensate, or by
decreasing the number of employees while increasing generating capacity to compensate. The number of
employees needed is directly related to plant capacity (Woodruff et al., 2005). Further, simple regression
7
of one of the inputs on the other (with output equal to 1 for all DMUs) shows their relationship to be
positive and statistically significant [t = 3.69; P(t > 3.69) < 0.0005]. Since output is equal for all DMUs,
the relationship would have to be negative if the inputs were truly substitutes. Therefore, the illusory
piecewise isoquant in Figure 1 is due to noise in the data and/or differences in the relative efficiency with
which DMUs employ each input. (We offer more sophisticated statistical analyses later in the paper,
simultaneously considering all three inputs and using statistical panel data analysis on five years of data,
and reach the same conclusion.)
2.1 Estimating the Point Frontier
When inputs are truly nonsubstitutable and there is only one output, as is the case in Figure 1, the
best-practice frontier consists of a single point that we call the point frontier. This frontier is estimated
by the composite DMU at point F in Figure 1. Although not shown on the graph, the frontiers of the
reference set increase vertically and horizontally from point F, forming a right-angle or L-shaped
reference set frontier. However, the only Pareto-Koopmans efficient subset of the reference set frontier is
the point frontier, because only at that point can none of the inputs or outputs be improved without
worsening other inputs or outputs.
For substitutable inputs, the minimum level of each input is conditioned on the level of the other input.
However, for nonsubstitutable inputs, the minimum level of each input needed to produce a given amount
of output is not influenced by the other input. So, if all DMUs’ outputs are equal, when inputs are
nonsubstitutable it is only necessary to find the minimum level of each input. As is true for DEA, this
frontier estimation method assumes that a composite DMU can be used to identify a point on the efficient
frontier that is attainable by an actual DMU.
Therefore, we can estimate the point frontier for one unit of output when inputs are nonsubstitutable
from the DMUs using the minimum amounts of each input, that is, from the DMUs establishing the
boundaries of the right-angle reference set frontier. In Figure 1, DMU A uses the least of input 1 (1.16)
and DMU E uses the least of input 2 (0.13), so a fully efficient composite DMU would use 1.16 units of
input 1 and 0.13 units of input 2, as shown by point F on the graph. Of course, if a particular DMU is the
most efficient in the use of both inputs, then that one DMU alone would determine the point frontier. For
example, if point F represented an actual DMU instead of a composite DMU, then that actual DMU
would reflect the point of maximum efficiency.
This point frontier also can be estimated with the same linear programming methodology used by
DEA. For DEA with substitutable inputs, when output for all DMUs is equal, the most efficient level of
each input is estimated based on the lowest observed value of one input given the observed values of the
other inputs. As stated in Cooper, Seiford and Tone (2007b, p. 7), “no point on this [Pareto-Koopmans]
frontier line can improve one of its input values without worsening the other.”
In the general case, suppose that for each DMU j ( 1,..., )j J= there are data on M inputs
1( ,... ) Mjm j jMx x x += R∈ and on N outputs 1,..., )j jNy( N
jny y += R∈ . If there are two inputs (M = 2) and
only one output (N = 1), and if that output is equal for all DMUs, then DEA scores can be computed with
the linear program 1-4, where DMUk is the target. This model is the conventional CCR model adapted
for one equal output.
minθλ
(1)
subject to 1 1jy = 1,2,...,j J= (2)
0jλ ≥ 1,2,...,j J= (3)
1
J
jm j kmj
x xλ θ=
≤∑ 1, 2m = (4)
One point on the Pareto-Koopmans efficient subset frontier is estimated by the inputs 1,k k 2x x of any
DMU k with 1θ = and no slack for either input. Now, let us rewrite linear program 1-4 by expanding
equation set 4 to the individual equations 8 and 9. Thus, linear program 5-9 more explicitly illustrates the
fact that the minimum θ attainable for either input is influenced by the value of the other input. Because
of DEA’s convexity assumption, for inputs defining the Pareto-Koopmans efficient subset frontier with
output constant, the lower the value of one input, the higher must be the value of the other. That is, if one 8
input is decreased then it is necessary to substitute the other input in sufficient quantities to keep output
constant. And, of course, it still is true that if a DMU’s 1θ = , then its inputs estimate one point on the
efficient frontier if neither contains slack.
minθλ
(5)
subject to 1 1jy = 1,2,...,j J= (6)
0jλ ≥ 1,2,...,j J= (7)
1 11
J
j j kj
x xλ θ=
≤∑ (8)
2 21
J
j j kj
x xλ θ=
≤∑
(9)
If, however, inputs are not substitutable and output is fixed, then the efficient value for one input is
not conditioned on the value of the other input. Consequently, the points on the Pareto-Koopmans
efficient frontier must be estimated separately for each input. To estimate the value for the first input,
linear program 5-9 would be run J times excluding equation 9. The input value corresponding to the
efficient target DMU(s) would be the value of the first coordinate of the point frontier. Similarly, to
estimate the value for the second input, linear program 5-9 would be run J times excluding equation 8.
The input value corresponding to the efficient target DMU(s) would be the value of the second coordinate
of the point frontier. No slack could occur, so these two input values define coordinates of the sole
efficient point on the reference set frontier.
2.2 Comparison of efficiencies from the two frontier estimates
9
As can be seen from Figure 1, if the inputs are truly nonsubstitutable then DEA over-estimates the
efficiency of DMUs A-E by identifying them as 100 percent efficient when they are not. The five DMUs
on the substitutable (piecewise-isoquant) frontier are substantially less efficient if they are measured
against F on the nonsubstitutable (point) frontier. Thus, we need to develop metrics to measure efficiency
when a fixed factor technology is present. We do so later in this paper, and then apply the metrics to a
sample of electricity generating plants. Before developing the metrics, therefore, we will present our
sample and statistically test it for substitutability of inputs.
10
3. Statistical testing for input substitutability
The data set is a matched sample of 70 electricity generating plants for the five years 2003-2007,
gathered from the Federal Energy Regulatory Commission’s Form 1 annual reports (2008). To assure
that the technologies being used were roughly the same, we limited our sample to plants that obtained at
least 95 percent of their energy from coal in at least one of the five years, and obtained all of their
remaining energy from gas and oil. We eliminated plants burning any amount of other fuels, such as tires
or garbage, and of course eliminated plants with energy sources such as nuclear, wind and hydroelectric.
We only used plants that supplied data for all of the necessary variables for all five years, and also
eliminated all plants that reported data that clearly were incorrect. We started with the entire set of plants
reporting data on Form 1, and, after applying the preceding screening criteria, 70 plants remained.
Because by far the most common configuration of inputs in previous papers were installed generator
capacity, number of employees, and BTUs, we use these three as measures of capital, labor and fuel.
And, as almost universal in previous papers, we use annual mWh as our output measure.
If the inputs are substitutable, then, holding output constant, the relationships between them will be
negative to a statistically significant degree. If the inputs are nonsubstitutable, then, holding output
constant, the relationships between them will be either not statistically significant, or positive to a
statistically significant degree. Therefore, in order to identify the relationships, we need to conduct two
regressions, in both cases holding output constant. In the first, we regress employment on capacity and
BTUs, holding output (mWh) constant. In the second, we regress capacity on employment and BTUs,
holding output constant.
We can analyze these data with statistical panel data analysis (PDA) because we have five years of
matched plant data available. We estimate the unobserved effects with a random effects model instead of
a fixed effects model because of its greater power to detect differences, and because the lack of changes
in installed generating capacity over the five years involved would make it impossible to estimate the
11
effect of capital otherwise. If any biases are present, they would generally make hypothesized
relationships appear to be stronger than is actually that case, and this would become a concern only if
there were negative relationships among the independent variables.
The regression results reported in Tables 1 and 2 are not surprising. The number of employees is
positively related to the installed generating capacity to a statistically significant degree, which would be
expected. The number of employees is positively related to BTUs of fuel but the relationship is not
statistically significant. This is not surprising because we know that staffing is directly related to installed
generator capacity, not to the particular level of fuel consumption. Further, we would not expect the level
of inefficiency in the use of labor to necessarily be the same as the level of inefficiency in the use of fuel.
(If the levels of inefficiency for the two inputs were identical except for random variations, then they
would show a statistically-significant positive relation in the regression.) Finally, the installed generator
capacities of plants are positively related to the number of BTUs consumed to a statistically significant
degree, which, since we are holding output constant, is indicative of some positive correlation between
fuel and capital inefficiencies.
Most important, there are no negative relationships, statistically significant or otherwise, and negative
relationships would be necessary if the factors could be substituted for each other, holding output
constant. It would be possible but is unnecessary to develop more complex and econometrically
sophisticated models. Not only are the outcomes what were expected, but also it is unlikely to the
extreme that the applicable regression coefficients would change signs from positive to negative and be
statistically significant in the negative direction. Using a different proxy for capital might make some
difference, but we have chosen to demonstrate the case with the measure of capital most commonly used
in DEA analyses of electricity generating plants.
In sum, none of the inputs are substitutable for each other, because none reflect negative relationships
of any form when output is held constant. Some of the inputs are complements (positive relationships) to
a statistically significant degree and others show non-statistically significant positive relationships.
12
Therefore we can assume a fixed factor proportion technology is present as we compare the reported
efficiencies using conventional DEA models and fixed factor proportion models.
Next, we develop a metric that we call the Fixed Proportion Additive (FPA) measure, because its
efficiency estimates can be compared directly to those produced by the DEA Additive measure.
4. Additive efficiency measure to deal with nonsubstitutability
4.1 Indicator development
In order to estimate each DMU’s efficiency relative to the point frontier in Figure 1, one simple
possibility would be a variation of the DEA Additive model (Cooper et al., 2007b, p. 94). With the
Additive model, the efficient point for a target DMU is the furthest point on the piecewise isoquant
frontier where neither of its inputs has increased and its output has not decreased. The rectilinear distance
between the target DMU and that point measures the DMU’s inefficiency.
We call our variation the Fixed Proportion Additive (FPA) model, because it assumes that the inputs
and outputs occur in fixed proportions. Like the Additive model, the degree of inefficiency is measured
as the rectilinear distance between the target DMU and the efficient point. But, for the FPA model, the
efficient point is the point frontier rather than a point on a piecewise isoquant frontier.
Significantly, the only difference between the two models is the location of the point from which
inefficiency is measured. This can be seen in Figure 2, which employs piecewise isoquant and point
frontiers identical to those in Figure 1. As a result, direct comparisons of the two models’ efficiency
scores can be made, because both models’ efficiency metric is exactly the same. Therefore, valid
estimates can be made of the bias in DEA scores when inputs and outputs are nonsubstitutable.
As can be seen in Figure 2, the Additive model indicates that DMUs A, B, C, D and E are efficient,
while the FPA model indicates that none of these DMUs is efficient, as would be expected based on the
Additive and FPA-estimated frontiers. For the target DMU, the Additive model’s inefficiency score is
0.10 and the FPA model’s inefficiency score is 0.22, more than twice as high.
The input-oriented FPA efficiency score for the target DMUk can be obtained for each target DMUk
from a set of 1,2,...,j J= DMUs by the use of equation 10.
1 11
( / ) ( / )j
M
k km k jmm
FPA I x y x yMin=
⎡ ⎤− = −⎢ ⎥
⎣ ⎦∑ j (10)
We next apply equation 10 to electricity generating plant sample data involving three inputs and one
output, and compare the resulting efficiency estimates to those obtained with the Additive model. .
4.2. Comparing the efficiencies reported by the additive and FPA models
So that the scores of the Additive and the FPA models will be directly comparable, we first divide
each DMU’s three inputs by its output. Therefore, the values used in the FPA model (equation 10) and
the Additive model (equations 11-14) are identical, so the resulting sums of the slacks for the target DMU
k are directly comparable.
1
maxM
k ii
Add s−=
= ∑ (11)
Subject to 1
J
jm j m kmj
x s xλ −
=
+ =∑
1,2,...,m M= (12)
1 1jy = 1,2,...j J= (13)
0jλ ≥ (14) For 2007, using the FPA scores as the base, the Additive model reported efficiencies that were
42.4 percent greater on the average, ranging from 3.6 percent greater to 100 percent greater. The two
models measure efficiency the same way and only differ on their identification of efficient points based
on whether or not the inputs are substitutable. In this case we know that the FPA model is correct
because the inputs are not substitutable. Thus, if the Additive model is (inappropriately) applied to these
data, it overestimates true efficiency by over 40 percent on the average, but the efficiency of some DMUs
is only slightly overestimated while the overestimation is very large for others. In short, in the presence
13
of nonsubstitutable inputs, the conventional Additive efficiency estimates are extremely biased and
exhibit strikingly low precision.
5. Ratio measure to deal with nonsubstitutability
We developed the FPA measure primarily because it would be completely comparable to a conventional
DEA measure, but it is of little value in practice. A much more valuable measure would be one akin to a
radial measure, because it would identify the proportional efficiency of target DMUs. In this section, we
develop such a measure, the Fixed Proportion Ratio (FPR) measure, to deal with nonsubstitutability.
5.1 Efficiency of a single output/input combination
In order to measure a DMU’s relative degree of inefficiency in the use of an input to produce an
output, the indicator needs to be normalized by some base. Our motivation was a comparison between
the Additive and ERM measures. The Additive model is based on the value of slacks, while the ERM
model normalizes the slacks by utilizing a DMU’s output slack divided by its output, and its input slack
divided by its input. Although we do not use slack directly, we do use the concept of normalizing actual
values in order to develop a measure of relative inefficiency.
Thus, for each output/input combination, we compute the normalized output/input ratio by dividing
the target DMU’s output/input ratio by that of the DMU j that is the most efficient for that particular
output/input ratio. Equation 15 illustrates the efficiency of target DMU k’s input m and output n. The
input and output in the numerator are from the target DMU k, and the input and output in the denominator
are from the DMU that has the maximum output/input ratio for that specific output/input combination.
/( / )kn km
kmnjn jmj
y xeffMax y x
⎡ ⎤⎢ ⎥=⎢ ⎥⎣ ⎦
(15)
Because a fixed ratio of the various types of input is needed to produce output, all inputs must be
positive. Also, to avoid division by zero, at least one jny must be positive. So, the range of efficiency
scores for each output/input pair is [0 , and at least one DMU will achieve an efficiency score of 1. ,1]
14
5.2 Overall efficiency of the target DMU
Because the target DMU’s efficiency would usually be different for each output/input combination, its
average efficiency can be computed as the mean of its individual efficiencies. Thus, for each of target
DMU k’s output/input ratios, equation 15 is used to compute the normalized efficiency measure for that
ratio. If there are m inputs and n outputs, then there will be m n× efficiency measures of the form .
So, for each DMU k in a set of J DMUs, the mean of its
kmneff
m n× efficiency measures is computed, which
yields a partially normalized efficiency measure for that DMU. Then, each DMU k’s partially normalized
efficiency measure is divided by the maximum partially normalized efficiency measure, which yields a
normalized efficiency measure in [0, 1]. We call this the Fixed Proportion Ratio (FPR) measure:
kFPR = 1 1
1 1
(1 / )
(1 / )
M N
kmnm n
M N
jmnj m n
MN eff
Max MN eff
= =
= =
⎛ ⎞⎜ ⎟⎝ ⎠
∑ ∑
∑ ∑
(16)
Because of the nature of the formula’s construction, it is easy to determine which of the target DMU’s
output/input pairs are the sources of inefficiency. Further, if an analyst wished to weight some pairs more
heavily than others, the formula could easily be modified to accommodate this.
5.3 FPR characteristics
All inputs must be positive, but this is not a constraint because no output can be produced by a fixed-
proportion technology otherwise. Likewise, because of the fixed-proportion output technology, all
outputs would be positive for a fully efficient DMU, but, for an inefficient DMU, outputs would only
have to be non-negative. The inputs and outputs are units-invariant but not translation invariant;
however, for a fixed-proportion technology, translation could not be contemplated in any case. As
already noted, FPR has a range over [0, 1]. Technical, mix and scale efficiencies are included in the FPR
measure. Finally, the measure is monotone decreasing in each input excess and output shortfall.
FPR envelops the data. Each individual output/input ratio of a target DMU is divided by the maximum
ratio for that particular output/input pair, which yields its efficiency relative to the most efficient pair. The
15
16
mean of each DMU’s individual ratios is divided by the maximum mean ratio, yielding its mean efficiency
relative to the maximum mean efficiency. As a result, the formula complies with the DEA convention of
basing efficiency on the most efficient observations in the set of units being analyzed.
5.4 Application to electricity generating plant efficiency measurement
Next, we compare scores of the FPR efficiency indicator with those of the Charnes-Cooper-Rhodes
(CCR) measure and the Enhanced Russell Measure (ERM) with constant returns to scale. (ERM and its
variations are also known as the Russell Measure, the Färe/Lovell Efficiency Index and the Slacks Based
Measure. But, as Cooper, Huang, Li, Parker and Pastor (2007a) note, ERM is the most firmly embedded
in the literature.)
Because the FPR measure incorporates both technical and scale efficiencies, we compare it to
conventional DEA models that also do so, rather than considering models that assume variable returns to
scale and therefore measure only technical efficiency.
Recall that our first measure, the FPA indicator, is directly comparable to the ADD model because
both use the same metric, with the only difference being to location of the efficient point. When
comparing these two measures, the ADD mean estimate was 142 percent of the FPA mean estimate,
compelling evidence that the ADD measure indeed biases efficiency scores upward when inputs or
outputs are nonsubstitutable.
However, the FPR, CCR and ERM indicators each compute their own unique efficiency metric, so
none of the three’s efficiency estimates are directly comparable to the others. In particular, differences in
the average efficiency estimates among the three are not too meaningful. Much more important would be
less-than-high correlation between the indicators, and substantial differences in DMU rankings based on
indicator values. If these symptoms are present, it indicates that the relative efficiencies of DMUs are
affected by the presence of fixed proportion technologies, making comparisons between DMUs invalid
when indicators assuming substitutability are utilized.
17
The data set used is the sample of 70 electricity generating plants, with output again being annual
kilowatt hours and the three inputs again being installed generator capacity, employees, and BTUs of fuel.
The three efficiency measures are computed from 2007 data.
As can be seen in Figure 3, FPR and ERM track closely until their DMU efficiency estimates
reach 70 percent, but from that point on there is significant variation with ERM always being higher and
usually substantially higher. This probably is caused by the availability of substitutions in the ERM
measure that allow a DMU to compare more favorably with its benchmarks, and such favorable
comparisons are more likely to exist when the target DMU is more efficient in general. The CCR
measure is much higher than the other two at all of the reported efficiency levels except for the highest
few. Moreover, it shows much greater variation than the other two, and, although not shown, has
extremely large slacks. This might be expected since CCR measures only weak efficiency (that is, it does
not consider inefficiency reflected in slacks), while ERM measures strong efficiency (which does
incorporate all inefficiencies including slacks). Unlike the CCR which estimates efficiency much higher
than the FPR at all efficiency levels, the ERM efficiency estimates are lower than those of the FPR for
several of the lowest efficiency levels and higher than those of the FPR for higher efficiency levels.
For all 70 DMUs, the R-square value between FPR and ERM is 0.92, but, for the 24 DMUs with
efficiency scores above 0.70, the R-square is only 0.46. The R-square value between FPR and CCR is
0.83 for all 70 DMUs, and 0.33 for the 24 DMUs with highest efficiency. Also, for the highest 24, the
Spearman rank coefficients between FPR and ERM is 0.76 and between FPR and CRR is 0.63, with the
Kendall rank coefficients being 0.61 and 0.50 respectively.
Perhaps of more concern is the fact that the rank order of some DMUs’ FPR scores is
substantially different from their CRR and ERM ranks. The ERM ranks ranged from 15 higher to 7 lower
than the FPR ranks, while the CCR ranks ranged from 29 higher to 24 lower.
Thus, based on the results of this sample, it appears that CCR is an unacceptable efficiency
measure when inputs are nonsubstitutable because of its very large upward bias and very low precision at
18
all efficiency levels. And, while ERM is a reasonably good measure at lower efficiency levels, at higher
levels its estimates become increasingly biased upward and decreasingly precise when compared to FPR.
Unfortunately, almost all DEA efficiency studies of electricity generating plants have used radial
measures, sometimes involving a Joint Production Model when undesirable outputs are considered, with
only a few using slacks-based measures. Thus, it is likely that most publications to date report electricity
generating plant efficiency estimates that are significantly biased, imprecise, and report inaccurate
efficiency rankings. Given the energy and environmental crises we are facing, this problem is of even
greater concern if such studies are used for policy or operating decisions.
6. Methods permitting the use of conventional DEA models
There are several methods that permit the use of conventional DEA models without suffering the bias
and precision problems described in this paper. One method is to aggregate nonsubstitutable variables
using their prices as weights. We believe that this solution is a good one if prices are available and can be
adjusted for price differences over time and among DMUs. These models also deal with the problem of a
DMU being more efficient in its production of one type of output than another, or more efficient in the
use of one type of input than another. For electricity generation, this solution would amount to using total
operating costs plus depreciation as the sole input variable. Another benefit of using monetary value
rather than physical inputs is that it would also account for the efficiency with which the plant commits its
various generators to production (the unit commitment problem), and the levels at which they are
operated (Powell et al., 1977).
Another solution is to use conventional DEA models but utilize only one of the nonsubstitutable
inputs. Because fixed-ratio variables are nonsubstitutable, they will increase and decrease together, so
one can serve as a rough proxy for all. The problem with this approach is that it does not account for
differences in a DMU’s efficiency in producing different outputs or in using different inputs. But, in the
absence of comparable prices, it may be the best choice available if one wishes to use conventional DEA
19
models. For electricity generation, it would seem that the best sole input variable would be fuel,
measured either in heat units or physical units, given that labor and capital basically are fixed.
7. Conclusions
This paper identifies the effects on efficiency estimates when conventional DEA models are applied
to electricity generating plants, which employ a fixed factor proportion technology. Using electricity
generating plant data, we found that inputs do occur in fixed proportions. As a result, the Additive and
CCR efficiency estimates are substantially biased upward and provide little precision in comparison to the
Fixed Proportion Additive and Fixed Proportion Ratio measures. ERM seems somewhat more robust to
fixed input proportion relationships at low efficiency levels, and at the lowest efficiencies it
underestimates true efficiency. At higher efficiency levels, ERM overestimates true efficiency, and
provides increasingly imprecise estimates and improperly ranked DMUs as efficiency increases.
Based on our sample data, therefore, it would appear that conventional DEA models should not be
used to measure the efficiency of electricity generation plants using the typical proxies for capital, labor
and energy inputs. Conventional DEA models should be used only if the nonsubstitutable inputs are
aggregated or if only one of the nonsubstitutable inputs is utilized.
We develop two new indicators, most importantly the FPR measure, for use when a fixed proportion
technology is present. The primary theoretical strength of the FPR indicator is that it avoids the DEA
requirement that inputs be substitutable. Its use would not be appropriate if substitutions were possible,
however, because it ignores substitutability and therefore would bias true efficiencies downward.
The FPR measure may have shortcomings which we have not discovered, and, hopefully, other
researchers will seek to improve it and to develop additional indicators for measuring efficiency under
fixed proportion technologies. But, as already noted, the use of DEA measures is of doubtful validity
when the three conventional inputs of labor, capital and energy are utilized, and cannot be depended on to
make precise and unbiased estimates with reasonably valid efficiency rank ordering.
20
References
Banker, R.D., Charnes, A., and Cooper, W.W. (1984), "Some models for estimating technical and scale inefficiencies in data envelopment analysis", Management Science, Vol. 30 No. 9, pp. 1078-1092.
Beattie, B.R. and Taylor, C.R. (1985), The Economics of Production, Wiley, New York.
Charnes, A., Cooper, W.W., and Rhodes, E. (1978), "Measuring the efficiency of decision making units", European Journal of Operational Research, Vol. 2 No. 6, pp. 429-444.
Cooper, W.W., Huang, Z., Li, S.X., Parker, B.R., and Pastor, J.T. (2007a), "Efficiency aggregation with enhanced Russell measures in data envelopment analysis", Socio-Economic Planning Sciences, Vol. 41 No. 1, pp. 1-21.
Cooper, W.W., Seiford, L.M., and Tone, K. (2007b), Data Envelopment Analysis: A Comprehensive Text With Models, Applications, References and DEA-Solver Software, Springer, New York.
Färe, R., Grosskopf, S., and Lovell, C.A.K. (1994), Production Frontiers, Cambridge University Press, Cambridge, England.
Farrell, M.J. (1957), "The measurement of productive efficiency", Journal of the Royal Statistical Society, Series A, Vol. 120 No. 3, pp. 253-290.
Federal Energy Regulatory Commission (2008), "Form 1 - Electric Utility Annual Report", available at: ftp://eforms1.ferc.gov/f1allyears/ (accessed 31 December 2008).
Petersen, N.C. (1990), "Data Envelopment Analysis on a relaxed set of assumptions", Management Science, Vol. 36 No. 3, pp. 305-
Powell, J., Gleason, J., and Burton, J. (1977), "Power generator scheduling by dynamic programming", European Journal of Operational Research, Vol. 1 No. 3, pp. 154-160.
Welch, E.W. and Barnum, D.T. (2009), "Joint environmental and cost efficiency analysis of electricity generation", Ecological Economics, In press, http://ssrn.com/abstract=1331628.
Woodruff, E.B., Lammers, H.B., and Lammers, T.F. (2005), Steam Plant Operation, McGraw-Hill, New York.
Zhou, P., Ang, B.W., and Poh, K.L. (2008), "A survey of data envelopment analysis in energy and environmental studies", European Journal of Operational Research, Vol. 189 No. 1, pp. 1-18.
