1

This is a PDF file of an unedited manuscript that has been accepted for publication in Journal of the

Operational Research Society. The manuscript will undergo copyediting, typesetting, and review of the

resulting proof before it is published in its final form. Please note that during the production process errors

may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

The final version will be available at: http://dx.doi.org/10.1057/jors.2012.19

Super-efficiency and stability intervals in additive DEA

M C Gouveia

ISCAC, Quinta Agrícola, Bencanta, 3040-316 Coimbra, Portugal, and

INESC Coimbra, Rua Antero de Quental, 199; 3000-033 Coimbra, Portugal

mgouveia@iscac.pt

L C Dias

Faculdade de Economia, Univ. Coimbra, Av. Dias da Silva 165, 3004-512 Coimbra, Portugal, and

INESC Coimbra, Rua Antero de Quental, 199; 3000-033 Coimbra, Portugal

ldias@inescc.pt

C H Antunes

DEEC-FCT Universidade de Coimbra-Pólo II, 3030 Coimbra, Portugal, and

INESC Coimbra, Rua Antero de Quental, 199; 3000-033 Coimbra, Portugal

cantunes@inescc.pt

ABSTRACT. This study addresses the problem of finding the range of efficiency for each Decision

Making Unit (DMU) considering uncertain data. Uncertainty in the DMU coefficients in each factor

(input or output) is captured through interval coefficients (i.e., these are uncertain but bounded). A two-

phase additive Data Envelopment Analysis (DEA) model for performance evaluation is used, which is

adapted to include the concept of super-efficiency to provide a robustness analysis of the DMUs in face

of uncertain information, assessing whether each DMU is surely efficient, potentially efficient, or surely

inefficient for the uncertainty intervals specified. Another contribution is to present how a maximal

stability hyper-rectangle can be computed for each DMU such that its efficiency status does not change

when the coefficients vary within that interval.

KEYWORDS. Data Envelopment Analysis; Multi-Criteria Analysis; Super-efficiency; Robustness

Analysis; Uncertainty; Stability intervals.

http://dx.doi.org/10.1057/jors.2012.19

2

Introduction

Data Envelopment Analysis (DEA), originally developed by Charnes et al. (1978), is a nonparametric

approach based on linear programming to evaluate observations representing the performances of all units

(Decision Making Units - DMUs) under evaluation. Each DMU is characterized by the "consumption" of

multiple inputs for the "production" of multiple outputs. The different DEA models seek to determine

which of n DMUs form the efficient frontier (or envelopment surface) in Pareto-Koopmans sense. These

evaluations result in a performance score that ranges between zero and unity that represents the “degree

of efficiency” obtained by DMUs.

In real-world evaluation problems, information is generally subject to several sources of uncertainty,

resulting from data scarcity, difficulties of data estimation or data collection, or even to contradictory

information from distinct sources. Therefore, decision aid models must cope with this data uncertainty by

using models capable of providing robust conclusions, i.e., recommendations that are somehow

“immune” to plausible data instantiations. The use of interval coefficients is a very flexible modelling

tool for capturing this type of data uncertainty (i.e., the precise performances of the DMUs are unknown

but bounded within an interval) since it does not impose stringent requirements about probability or

possibility distributions. There are two main perspectives to deal with uncertainty in the context of our

work. One, which is generally encompassed under the designation imprecise DEA (IDEA) (Cooper et al.,

1999, 2001a, 2001b), studies how to deal with imprecise data such as bounded data, ordinal data and ratio

bounded data in DEA and results in a non-linear and non-convex DEA model. By using scale

transformations and variable changes (Zhu, 2003), or only variable transformations (Despotis and Smirlis,

2002), this non-linear model can be transformed into an equivalent linear programming problem. The

other perspective, which is more related to the approach proposed in this paper, deals with computing

stability intervals for uncertain coefficients so that results do not change (Zhu, 1996, 2001; Seiford and

Zhu, 1998a, 1998b).

This paper addresses this type of uncertainty in the context of a two-phase method developed by Gouveia

et al. (2008), which was inspired on the additive DEA model proposed by Charnes et al. (1985) as well as

the additive model with oriented projections presented by Ali et al. (1995). In Gouveia et al.’s model, the

DMUs are treated as alternatives of a multiple criteria decision model (an additive multi-attribute utility

model), each alternative being evaluated in a number of distinct criteria. The two-phase method provides

an efficiency measure of each DMU by calculating a criterion weighting vector and, if necessary,

obtaining the projected point.

There are two main contributions in this work. One contribution is to present how the range of efficiency

for each DMU can be computed in presence of interval values for the DMU coefficients in each

input/output. This allows performing a robustness analysis of each DMU under evaluation, assessing

whether each DMU is surely efficient, potentially efficient, or surely inefficient for the uncertainty

intervals specified. Another contribution is to present how a maximal stability hyper-rectangle can be

computed for each DMU such that its efficiency status does not change when the coefficients vary within

3

that interval. This is confronted with the idea of taking a super-efficiency score as a proxy of robustness,

after adapting the original model of Gouveia et al. (2008) to consider the concept of super-efficiency.

Following this introduction, section 2 briefly describes DEA models, with emphasis on the additive DEA

model and the weighted additive model. In section 3, the concept of super-efficiency is reviewed. Section

4 introduces the two-phase method of Gouveia et al. (2008) with the modifications to include the super-

efficiency concept. In section 5, the ranges of efficiency are computed and the robustness of each DMU is

analyzed by considering a two-dimensional pedagogical example. In section 6, an example with real

world data is provided for exploiting the insights from this robustness analysis. Concluding remarks are

presented in section 7.

Data Envelopment Analysis

The set of n DMUs to be evaluated is )},...,1(:{ njjDMU = . Each DMUj consumes m different inputs

),...,1( miijx = to produce p different outputs ),...,1( prrjy = . jX (the jth column of nḿX ) denotes the

vector of inputs consumed by DMUj. A similar notation is used for outputs, Yj. The input data is

represented by matrixnḿX and the output data by matrix np´Y . 1 denotes the summation vector( )

T1,...,1 .

There are two main types of DEA models that provide a measure of relative efficiency for each DMU

according to the returns to scale considered (Seiford and Zhu, 1999b): Constant Returns-to-Scale (CRS)

models, such as the CCR model (see Charnes et al., 1978), and Variable Returns to Scale (VRS) models,

such as the BCC model (see Banker et al., 1984) and the additive model (ADD) of Charnes et al. (1985).

The additive model

In CCR and BCC models we need to distinguish between input-oriented and output-oriented models. The

additive model combines both orientations in a single model, which can be formulated as follows:

ADD Xk,Yk( )( )

min zk = - 1s+1e( )

s.t. Yl - s= Yk,

- Xl - e= -Xk,

1l =1,

l ³ 0, e ³ 0, s³ 0.

The ADD model returns a non-positive value *kz , which allows checking the relative efficiency of the

DMU k under analysis. If the value obtained is negative, then the DMU under analysis is operating

inefficiently in some factors. This value is the symmetric of the sum of the distances in each dimension to

the envelopment surface (L1 distance).

If DMU k is inefficient (i.e., it does not lie on the efficient frontier defined by the set of DMUs) the model

4

identifies a projected point ( )kk YX ˆ,ˆ on the efficient frontier. If the optimal value of the primal ADD

model is zero the point ( )kk YX , belongs to the efficient frontier, that is ( )kk YX ˆ,ˆ = ( )kk YX , . This is a

necessary and sufficient condition for efficiency, Charnes et al. (1985).

The projected point can be characterized, in an alternative way, as sYeXYX kkkk ,ˆ,ˆ , from the

primal constraints. This ADD model measures the excess of inputs, e , and the deficit of outputs, s , in

which the DMU k operates when confronted with the DMUs that operate on the efficient frontier.

Ali et al. (1995) presented a variant of additive model with oriented projections, which is henceforth

called weighted additive model.

The envelopment formulation for this model is:

ADDW Xk,Yk,uk, vk( )( )

min zk = - uks+ vke( )

s.t. Yl - s= Yk,

- Xl - e= -Xk,

1l =1,

l ³ 0, e ³ 0, s³ 0.

The parameters uk, v

k(which are fixed before the model is solved) have play an important role that is

clearly seen in the primal formulation. The vectors uk, v

k( ) are the coefficients of the objective function

and thus define the relative weight attributed to one unit of each slack (for a discussion on the role of

weights and value judgments in DEA (see Thanassoulis et al., 2004)). The weight vectors provide and

determine the directions of the projection. Setting uk

= 1and vk

= 1 in the primal weighted additive

problem leads to the original additive model.

Super-efficiency and sensitivity analysis in DEA

Andersen and Petersen (1993) developed an extended DEA measure in which the basic idea is to compare

the DMU under evaluation with a linear combination of all other DMUs in the reference set. This means

that the production possibility set is reduced by not considering the DMU being evaluated, which allows

efficient DMUs to become super-efficient and have different super-efficiency scores. In other words, the

DMUs can increase the input vector (or decrease the output) to some extent while preserving efficiency.

We can also perceive DEA models as projection mechanisms and the projections of the inefficient DMUs

on the efficient frontier depend on the scales used to measure each input or output. Super-efficiency is

very sensitive to the projection mechanisms and tends to favour “extreme” solutions (Bouyssou, 1999).

The set of DMUs can be partitioned into two groups: frontier (efficient) DMUs and non-frontier

(inefficient) DMUs. The frontier DMUs consist of DMUs in the set E (extreme efficient), set E’ (efficient

but not an extreme point) and the set F (weakly efficient or frontier point but with non-zero slacks). The

5

super-efficiency model identifies the classification of a given DMU and, using some extensions of the

super-efficiency model, a sensitivity analysis of the conclusions about efficiency can also be performed.

However, under certain conditions the process of determining the super-efficiency score can lead to an

infeasible linear program. Based on Thrall (1996), a necessary, but not sufficient, condition for

infeasibility is that an excluded DMU is extreme efficient. Dulá and Hickman (1997) and Seiford and Zhu

(1999a) reported a necessary and sufficient condition for infeasibility in an input-oriented CCR super-

efficiency model: the excluded DMU has the only zero value for any input, or the only positive value for

any output, among all DMUs in the reference set. Infeasibility cannot arise in an output-oriented CCR

super-efficiency model. Infeasibility also occurs in the BCC super-efficiency model, when an efficient

DMU under evaluation cannot reach the frontier formed by the remaining DMUs via increasing the inputs

(or decreasing the outputs). Infeasibility arises in either orientation whenever there is no reference DMU

for the excluded one.

Many DEA researchers have addressed the sensitivity of the results to data perturbations and the

robustness of the efficiency scores resulting from these perturbations, based on super-efficiency DEA

approaches. Continuing the work of Zhu (1996), Seiford and Zhu (1998a) developed a sensitivity analysis

procedure to determine stability regions for possible increases in all inputs and for possible decreases in

all outputs within which the efficiency of a specific efficient DMU remains unchanged. Seiford and Zhu

(1998b) extended the method by Zhu (1996) and Seiford and Zhu (1998a) to the worst-case scenario,

where the same maximum percentage data changes for deteriorating the efficiency of a DMU under

analysis and the data changes for improving the efficiencies of the other DMUs simultaneously are

calculated. In this work, the authors also concluded that the relationship between the infeasibility and

stability of efficiency classification, discovered in Seiford and Zhu (1998a), remains for the simultaneous

data variations case and for all basic DEA models. Generalizing these results, Zhu (2001) considered that

the data perturbation in the DMU under analysis and the data perturbations in the remaining DMUs can

be different when all the remaining DMUs improve their efficiencies at the expense of deteriorating the

efficiency of the efficient DMU under analysis. Necessary and sufficient conditions for preserving

efficiency were provided.

Our approach differs from the sensitivity and robustness analysis presented by previous authors in several

aspects. It has been developed for the additive two-phase model of Gouveia et al (2008). The uncertainty

in the coefficients in each factor (input or output) is captured through interval coefficients and converted

into utility scales (which are always to be maximized). An optimistic efficiency measure and a pessimistic

efficiency measure are computed. Additionally, a tolerance threshold for each efficient DMU is

determined, that is a maximum tolerance in the factor scores for which the DMU’s efficiency status

changes. Using these two types of efficiency measures, we can classify the DMUs as surely efficient,

potentially efficient, or surely inefficient. Unlike the standard super-efficiency models, with specific

orientations, the model proposed in this paper projects the DMUs in any direction in a way that minimizes

the distance of the unit under evaluation to the best of all units (excluding the one in evaluation), and

6

therefore no infeasibility concerns arise.

Adaptation of the two-phase method to compute super-efficiency scores

The two-phase method developed by Gouveia et al. (2008) is a variant of the additive DEA model with

oriented projections (Ali et al., 1995), which uses concepts developed in the field of multiple criteria

decision analysis (MCDA) under imprecise information (Athanassopoulos and Podinovski, 1997; Dias

and Clímaco, 2000).

We consider the DMUs as alternatives of a multiple criteria evaluation model, each one being evaluated

in a number of distinct criteria. Each criterion corresponds to an input or an output factor in DEA models.

A direction of preference is associated with each criterion: increasing for outputs and decreasing for

inputs. The method uses an additive utility function to aggregate the utilities associated with each

alternative, based on the Multi-Attribute Utility Theory (MAUT) (see Keeney and Raiffa, 1976).

Adapting to this context, the purpose of MAUT will be to assess the utility of each alternative,

considering that the larger the utility the better. MAUT is also aimed at simplifying the task of building

the utility functions when evaluating the alternatives that are described by multiple attributes. In addition

the decision maker's task is facilitated because he can focus the attention on one attribute at a time and

then make the aggregation of attributes, rather than make judgments directly on the global utility (the

concept of attribute in this theory is equivalent to our concept of criterion). This overcomes the problem

of the scales associated with the ADD model, since all the input and output measures are translated into

utility units. Moreover, the weights used in the aggregation gain a specific meaning: they are the scale

coefficients of the utility functions. Weights are chosen to benefit each DMU as much as possible, rather

than being fixed beforehand as in the model by Ali et al. (1995). Finally, the efficiency measure assigned

to each DMU gains an intuitive meaning: it corresponds to a “min-max regret” (utility loss) measure.

Considering that the alternatives are the DMUs to be evaluated according to q criteria, we assume that the

utility of each alternative is given by an additive MAUT model

q

cjccj DMUuwDMUu

1

, where

wc ³ 0,"c =1,...,q and wc =1c=1

q

å (by convention). The scale coefficients qww ,...,1 are the weights of the

utility functions.

The use of this model requires that the original input and output scales have to be converted into utility

scales and there are several techniques for questioning the decision maker, in order to construct the utility

functions compatible with their answers (see von Winterfeldt and Edwards, 1986). Hence, after being

converted into utilities all criteria are treated as outputs.

Gouveia et al. (2008) proposed a two-phase method to incorporate preferences in the ADD model. In this

study the two-phase method is adapted to consider the super-efficiency concept. For that purpose the

following problem is solved:

7

d,wmin dk

s.t. wcuc DMU j( )c=1

q

å - wcuc DMUk( )c=1

q

å £ dk, j =1,...,n, j ¹ k

wc =1c=1

q

å ,

wc ³ 0,"c =1,...,q

(1)

The score denotes the distance defined by the utility difference to the best of all alternatives

(excluding the one under evaluation). The aim of our approach is, for DMU k, to calculate the vector w of

utility function weights that minimizes the distance (the utility difference) of this unit to the best one

(note that the best alternative will also depend on w), excluding itself from the reference set. Regarding

the model of Gouveia et al. (2008), the only change is to exclude one constraint in which the DMU k

under evaluation was compared to itself (which is achieved by introducing j≠k in problem (1)). This

change implies that is now allowed to become negative.

This approach is identical to that described in Gouveia et al (2008), and starts by finding the weights (the

variables of problem (1)) that most benefit the DMU under consideration to have the worst utility loss

(also a variable of problem (1)). Then, the “weighted additive” problem can be solved using the optimal

weighting vector wc* , resulting from (1), to compute the projected point in case of the DMU is inefficient.

Phase 1: Convert inputs and outputs into utility scales. Compute the efficiency measure, , of each

DMU, k = 1,…,n, and the corresponding weighting vector.

Phase 2: If dk* ³ 0 then solve the “weighted additive” problem (2), using the optimal weighting vector

resulting from phase 1, , and determine the corresponding projected point of the DMU under

evaluation.

minl,s

zk = - wc*sc

c=1

q

å

s.t. l juc DMU j( )j=1, j¹k

n

å - sc = uc DMUk( ), c = 1,...,q

1l = 1,

l ³ 0, s³ 0.

(2)

If the optimal value of the objective function in (1) is not positive, then the DMU k under evaluation is

efficient. Otherwise it is inefficient and is the minimum difference of utility to the best DMU (i.e., the

DMU with higher global utility). However, with the adaptation made to the original method we can also

discriminate the efficient units. If < 0, then the DMU is in the set E (extreme efficient); if = 0 and

all the slacks are null in phase 2, then the DMU belongs to E’ (efficient but not an extreme point); if =

dk*

dk*

dk*

wc*

dk*

dk*

dk* dk

*

dk*

8

0 and not all the slacks are null in phase 2, then the DMU belongs to set F (weakly efficient or frontier

point but with non-zero slacks).

Using this measure, we can assess the extent to which an efficient DMU may worsen its utility while

remaining efficient. This allows analyzing the robustness of the classification of a DMU as an efficient

unit in face of uncertain information regarding factor coefficients. As becomes more negative, we

expect the efficient DMU to be more robust to changes in input and output levels.

Robustness analysis and stability intervals

Consider that the value cjp (performance of DMU j in factor c) is uncertain but bounded within the range

U

cjcj

L

cj pp p . This implies ),()()( jU

cjcj

L

c DMUuDMUuDMUu if the factor c is an output, or

),()()( jU

cjcj

L

c DMUuDMUuDMUu if the factor c is an input.

Given interval performances on each factor (input or output), it is possible to compute an optimistic and a

pessimistic efficiency measure dk* for each DMU, k =1,…,n, using the first phase of the two-phase

method. As an example, let us consider that all performances are bound to intervals

pcjL = pcj 1-d( ) £ pcj £ pcj 1+d( ) = pcj

U, with d for instance equal to 5%, 10%, or 20%. For this work, we

consider that all performances are applied the same toleranced , but we may consider that the tolerance is

applied only to a subset of these (inputs or outputs).

To present the methodology proposed in this paper, let us consider as an illustration the data in Table 1

(displayed in Fig. 1). Taken from Gouveia et al. (2008), these data have been modified by adding DMU9,

which is weakly efficient, to portray more possibilities. For this illustration let us assume that inputs and

outputs are converted into “utilities” in a linear way (in practice these functions may be constructed with

the clients of the study, reflecting their value system, see Almeida and Dias (2012)). So, for the plausible

higher tolerance value considered (in this case = 20%), and for each c =1,…,q, we choose a value

and , and then we compute the utilities for each unit

using:

uc DMU j( ) =

pcj - McL

McU - Mc

L, if factor c is an output

McU - pcj

McU - Mc

L, if factor c is an input

ì

í

ïïï

î

ïïï

, j =1,..., n,c =1,..., q.

dk*

McL

< min pcjL, j =1,...,n{ } McU > max pcjU, j =1,...,n{ }

9

Table 1: Test data

DMU Y1 Y2

1 10 2

2 9 5

3 6 7

4 3 8

5 4 4

6 8 1

7 5 6

8 4 6.5

9 2 8

McL

1 0

McU

12 10

Considering the nominal values for each DMU and applying the two-phase method (1) and (2), we can

build Table 2 and rank the DMUs in terms of optimal utility loss dk* :

DMU1 ≻ DMU2 ≻ DMU3 ≻ DMU4 ≻ DMU9 ≻ DMU8 ≻ DMU7 ≻ DMU6 ≻ DMU5.

The lower the value of dk* the better, and if dk

* is negative then the DMU is efficient (DMUs 1 to 4).

DMU9 has dk*= 0 but it is not strictly efficient, since one of the slacks is not null. The remaining DMUs

have dk*> 0 and hence are not efficient.

The projections of inefficient DMUs were obtained considering the weighting vector that resulted from

phase 1, in which the linear program (1) is solved for each DMU. Among the efficient DMUs (DMUs 1-

4) the first two have a larger margin to decrease their performance than the remaining ones.

Table 2. Efficiency measure and other results for each DMU

DMU dk*

*

1w *

2w s1*

s2*

1 -0.091 1.000 0.000

2 -0.074 0.579 0.421

3 -0.032 0.355 0.645

4 -0.020 0.216 0.784

5 0.250 0.423 0.577 0.455 0.100 2=1

6 0.163 0.767 0.233 0.091 0.400 2=1

7 0.096 0.423 0.577 0.227 0.000 2=0.5; 3=0.5

8 0.085 0.268 0.732 0.000 0.117 3=1

9 0.000 0.000 1.000 0.091 0.000 4=1

To compute the optimistic efficiency measure we consider the best value of the intervals for the DMU

10

being evaluated and the worst value of the intervals for all other DMUs. The reverse is considered to

compute the pessimistic efficiency measure. Let jLc DMUu denote the minimum utility that DMU j

attains in factor c given its uncertain performance, note that and

in the next expressions:

ucL DMU j( ) =

pcjL - Mc

L

McU - Mc

L, if factor c is an output

McU - pcj

U

McU - Mc

L, if factor c is an input

ì

í

ïïï

î

ïïï

, j =1,..., n,c =1,..., q

Let jUc DMUu denote the maximum utility that DMU j attains in factor c given its uncertain

performance:

ucU DMU j( ) =

pcjU - Mc

L

McU - Mc

L, if factor c is an output

McU - pcj

L

McU - Mc

L, if factor c is an input

ì

í

ïïï

î

ïïï

, j =1,...,n,c =1,..., q

To compute the optimistic efficiency measure dkopt * for DMU k the following LP similar to (1) is solved:

min dkopt

s.t. wcucL

DMU j( )c=1

q

å - wcucU

DMUk( )c=1

q

å £ dkopt

, j = 1,...,n, j ¹ k

wc = 1c=1

q

å ,

wc ³ 0,"c =1,..., q

To compute the pessimistic efficiency measure dkpes* for DMU k the following LP similar to (1) is

solved:

min dkpes

s.t. wcucU

DMU j( )c=1

q

å - wcucL

DMUk( )c=1

q

å £ dkpes

, j = 1,...,n, j ¹ k

wc = 1c=1

q

å ,

wc ³ 0, "c = 1,..., q

With this analysis we can assess the robustness of the DMU under consideration because despite being

assessed in a pessimistic way it may keep its efficiency status. A DMU is said to be robust to changes in

its factors if the DMU remains in the same status after the change. Therefore, the DMU is robustly

McL

< min pcjL, j =1,...,n{ }

McU > max pcj

U, j =1,...,n{ }

11

efficient in the range of uncertainty considered. These results are displayed in Table 3.

Table 3. Lower and upper limits for the utility loss dkopt *

, dkpes*é

ëêù

ûú, for each DMU

DMU 5% 10% 20%

1 [-0.177;-0.005] [-0.264;0.082] [-0.436;0.255]

2 [-0.138;-0.009] [-0.203;0.056] [-0.333;0.185]

3 [-0.095;0.031] [-0.158;0.094] [-0.284;0.219]

4 [-0.087;0.048] [-0.160;0.116] [-0.320;0.251]

5 [0.199;0.301] [0.148;0.352] [0.046;0.454]

6 [0.097;0.229] [0.018;0.295] [-0.145;0.428]

7 [0.037;0.155] [-0.024;0.213] [-0.146;0.331]

8 [0.024;0.147] [-0.038;0.209] [-0.161;0.332]

9 [-0.080;0.080] [-0.160;0.154] [-0.320;0.283]

DMUs 1 and 2 are efficient and remain in this state when a tolerance of 5% is considered. DMUs 3 and 4

do not remain efficient for this tolerance value. For this tolerance, DMUs 1 and 2 are surely efficient,

DMUs 3, 4, and 9 are potentially efficient, and DMUs 5-8 are surely inefficient. If the intervals of

uncertainty are defined by a tolerance of 10% or 20%, then there are no surely efficient DMUs.

Figure 1 shows the efficient frontier, given the nominal values. The pessimistic evaluation of efficiency

for DMU1 considering the 5% tolerance is portrayed by the dashed line. The rectangles define the region

of tolerance. In this case, DMU1 is in its worst performance level while the remaining units are at their

best performance levels, and it remains efficient.

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

9 43

2

16

78

5

2u

1u0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Figure 1. The unit isoquant spanned by the pessimistic evaluation of DMU1

Figure 2 shows DMU7's optimistic assessment considering a tolerance of 10%, which is portrayed by the

dashed line. Hence, DMU7 is in its best performance while the remaining units are at their worst for this

tolerance values and, with this type of analysis, theDMU7 that was in the set of inefficient units

(considering the nominal values of DMUs) is now in the set of efficient ones.

12

0.2

9 48 7

5

6

32

1

2u

1u

0.1

0.3

0.4

0.5

0.9

0.6

0.7

0.8

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Figure 2. The unit isoquant spanned by the optimistic evaluation of DMU7

A different type of analysis that can be carried out is to compute the maximum tolerance such that the

efficient DMUs maintain efficiency, i.e., to compute a stability interval in terms of a parameter

affecting all the factors:

dmax = d : dkpes*(d) = 0{ }

This can be easily accomplished by a bisection technique (see Table 4) if the utility functions are

monotonous. Table 5 shows that for a tolerance greater than 5.2% the DMU1 is no longer guaranteed to

be efficient. For DMU2 the tolerance threshold for being robustly efficient is 5.6%.

Table 4: Computing δmax

using a bisection technique

Let d denote the maximum tolerance value (in this paper d = 0.2)

Let e denote the precision used (in this paper e = 0.001).

d := 0;

while (d - d )

13

Table 5. Efficiency threshold for the DMUs 1, 2, 3 and 4

DMU 1 2 3 4

δmax

5.2% 5.6% 2.5% 1.4%

It is noteworthy that DMU1 had a measure of efficiency highest than DMU2, but in this analysis this

DMU emerges as being the most robust as it maintains the range of efficiency for a higher level of

uncertainty on performances. Concerning DMUs 3 and 4, the tolerance threshold for which they are

surely efficient is lower.

An example with real world data

This section revisits an empirical example of Cooper et al. (2006) displayed in Table 6, aiming to

illustrate the insights that can be obtained from the approach proposed in this paper. There are 12

hospitals and 6 factors: number of doctors and nurses and the relative unit costs of doctors and nurses in

terms of inputs, and two outputs identified as number of outpatients and inpatients (each in units of 100

persons/month). In this case we also consider that the higher tolerance value is = 20%, and for each c

=1,…,q, we choose a value and .

Table 6: Test data

Inputs Outputs

Doctor Nurse Outpatients Inpatients

DMU Number Cost Number Cost Number Number

1 20 500 151 100 100 90

2 19 350 131 80 150 50

3 25 450 160 90 160 55

4 27 600 168 120 180 72

5 22 300 158 70 94 66

6 55 450 255 80 230 90

7 33 500 235 100 220 88

8 31 450 206 85 152 80

9 30 380 244 76 190 100

10 50 410 268 75 250 100

11 53 440 306 80 260 147

12 38 400 284 70 250 120

McL

10 200 100 50 70 40

McU

70 750 400 150 320 180

For the purpose of this illustration, the performances of the 12 hospitals are converted into utilities, using

McL

< min pcjL, j =1,...,n{ } McU > max pcjU, j =1,...,n{ }

14

a linear transformation, as described in the previous section, and considering the tolerance d = 5%, 10%,

or 20%.

Table 7 shows the efficiency measure, weights, slack and λ values, for each DMU considering the

nominal values, resulting from the application of the two-phase method (section 4). From these results we

can conclude that only three hospitals are working inefficiently. This efficiency measure allows to

discriminate the efficient units using the super-efficiency concept and rank all DMUs:

DMU11≻DMU5≻DMU1≻DMU2≻DMU12≻DMU10≻DMU4≻DMU9≻DMU7≻DMU6≻DMU3≻DMU8.

The DMUs 11, 5, 1, 2, 12, 10, 4, 9 and 7 are efficient. With the slack values obtained by solving problem

(2) and introducing the weight vector resulting from stage (1), we get the projections of inefficient

DMUs.

Table 7. Efficiency measure and other results, for each DMU

DMU dk*

*

1w *

2w *

3w *

4w *

5w *

6w s1*

s2*

s3*

s4*

s5*

s6*

1 -0.0949 0.261 0.000 0.274 0.000 0.000 0.466

2 -0.0903 0.000 0.060 0.856 0.000 0.084 0.000

3 0.0246 0.092 0.000 0.361 0.017 0.530 0.000 0.000 0.160 0.000 0.112 0.043 0.044 2=0.76;10=0.17; 12=0.04

4 -0.0125 0.000 0.000 0.461 0.000 0.415 0.124

5 -0.0952 0.000 0.524 0.000 0.476 0.000 0.000

6 0.0227 0.000 0.000 0.448 0.030 0.395 0.127 0.132 0.083 0.000 0.045 0.042 0.038 2=0.10;10=0.90

7 -0.0004 0.211 0.000 0.273 0.000 0.516 0.000

8 0.0479 0.050 0.000 0.364 0.208 0.085 0.293 0.035 0.201 0.000 0.143 0.001 0.055 5=0.63; 11=0.07; ;λ12=0.30

9 -0.0103 0.477 0.171 0.000 0.000 0.094 0.257

10 -0.0224 0.000 0.000 0.420 0.000 0.580 0.000

11 -0.1929 0.000 0.000 0.000 0.000 0.000 1.000

12 -0.0800 0.347 0.000 0.000 0.159 0.475 0.019

Table 8 displays the results for each DMU considering the optimistic (for lower limits of the ranges) and

pessimistic (for upper limits of the ranges) perspectives, using the first phase of the two-phase method

with = 5%, 10%, or 20%.

15

Table 8. Lower and upper limits for the utility loss dkopt *

, dkpes*é

ëêù

ûú, for each DMU

DMU 5% 10% 20%

1 [-0.149;-0.043] [-0.206;0.007] [-0.321;0.092]

2 [-0.140;-0.040] [-0.191;0.007] [-0.306;0.089]

3 [-0.035;0.082] [-0.095;0.131] [-0.220;0.220]

4 [-0.075;0.049] [-0.138;0.106] [-0.270;0.218]

5 [-0.162;-0.037] [-0.229;0.021] [-0.362;0.137]

6 [-0.066;0.108] [-0.156;0.192] [-0.336;0.360]

7 [-0.081;0.075] [-0.164;0.142] [-0.331;0.268]

8 [-0.018;0.115] [-0.084;0.181] [-0.217;0.285]

9 [-0.072;0.051] [-0.135;0.111] [-0.280;0.230]

10 [-0.118;0.072] [-0.275;0.217] [-0.407;0.327]

11 [-0.288;-0.097] [-0.384;-0.002] [-0.574;0.189]

12 [-0.163;0.002] [-0.247;0.082] [-0.420;0.231]

According to the rank previously made and for a tolerance of 5%, DMUs 11, 5, 1 and 2 are surely

(robustly) efficient. The DMUs 12, 10, 4, 9 and 7 do not remain always efficient for the same tolerance

value. For a tolerance of 10% only DMU11 maintains the efficiency status.

Using a bisection technique we compute the maximum tolerance value for which the efficient DMUs

maintain the efficiency status. Observing the stability interval limits shown in Table 9,and comparing

with the rank established based on the nominal values of DMUs, we can conclude that the most robust

DMU is DMU11, which coincides with the analysis based on super-efficiency. But the second most

robust one is DMU1, which was ranked after DMU5 in the super-efficiency ranking.

Table 9. Efficiency threshold for the efficient DMUs

DMU 1 2 4 5 7 9 10 11 12

δmax 9.2% 9.0% 1.0% 8.2% 0.0% 0.8% 1.1% 10.1% 4.9%

This type of analysis can be better understood when accompanied by the graph in Figure 3 where the

behaviour of dkpes*(pessimistic assessment) is illustrated. In fact DMU11has a better efficiency measure

and maintains the efficiency for a greater level of uncertainty in performances (δmax

= 10.1%). It is also

possible to see that despite some DMUs had a better level of efficiency, with respect to the nominal

situation δ = 0%, they are overtaken by others with a lower efficiency measure when more uncertainty in

the performance of these units is assumed. This is the case of DMU10, which has a lower dkpes* for a

tolerance of less than 5.4% and it is surpassed by DMU7 for higher tolerance values.

16

Figure 3. Values approximated by linear interpolation of dkpes*

for DMUs 1, 2, 4, 5,7,9,10,11 and 12

We can also observe that for linear utility functions the curve that represents the optimal value dkpes* is

concave. In Appendix Awe prove that if all utility functions are linear, then the function that describes

how dkpes*changes with is concave.

Concluding remarks and future work

This work provides a robustness analysis of each DMU in presence of interval data. A preliminary

assessment of the robustness of each DMU is obtained using the first phase of a two-phase method with a

modification to include the super-efficiency of efficient units. This method projects the DMUs in any

direction in a way that minimizes the distance of this unit to the best of all (excluding the one under

evaluation), and therefore no infeasibility occurs.

Assuming that the values of the DMU performances in each factor (inputs and outputs) are not known

exactly, but an interval of values for these performances can be established, it is possible to calculate an

efficiency range for each DMU. The efficiency scores for the DMU under analysis are computed

considering its coefficients in the most unfavourable/favourable bounds and all the other DMU’s

coefficients in their most favourable/unfavourable bounds, in order to assess the DMU’s robustness. This

process enables to classify the DMUs as surely efficient, potentially efficient, or surely inefficient.

The maximum tolerance d such that the efficient DMUs maintain efficiency is also computed, by using a

bisection technique. An illustrative example shows that, according to this robustness measure, the DMU

with the highest super-efficiency score is not necessarily the most robust one, i.e., the one with widest

stability intervals.

17

In future work we intend to include in the model a way to calculate this tolerance given different types of

utility functions.

Appendix A

Let us suppose that the under analysis, DMU k, is improved by dz% while all other DMUs are changed

by dz% in the opposite direction. The problem in phase 1 is, for this constant dz:

Considering that c=1,...,a are output factors and c=a +1,...,qare input factors, we have:

min dk* dz( )

s.t. wcpcj (1-dz)- Mc

L

McU - Mc

Lc=1

a

å - wcpkj (1+dz)- Mc

L

McU - Mc

Lc=1

a

å +

wcMc

U - pcj (1-dz)

McU - Mc

Lc=a+1

q

å - wcMc

U - pkj (1+dz)

McU - Mc

Lc=a+1

q

å ≤ dk* dz( ) ,jk (1’)

wc.1=1

wc ³ 0 , dk* dz( ) free

LEMMA 1.

Let dk* dz( ) , wc

*be an optimal solution to (1’). If the performance of DMU k in factor c, pkj , is altered in

dz +q( )%, with in the set of real, then the optimal value to (1’) is at most

dk*

dz( ) -

w*q

McU - Mc

Lpcj + pkj( )

c=1

a

å +w

*q

McU - Mc

Lpcj + pkj( )

c=a+1

q

å .

PROOF

Problem with DMU k altered in dz +q( )%:

min dkq dz( )

s.t. wcq pcj (1-dz -q )- Mc

L

McU - Mc

Lc=1

a

å - wcq pkj (1+dz +q )- Mc

L

McU - Mc

L+

c=1

a

å

wcq Mc

U - pcj (1-dz -q )

McU - Mc

Lc=a+1

q

å - wcq Mc

U - pkj (1+dz +q )

McU - Mc

Lc=a+1

q

å ≤ dkq dz( ) , jk (2’)

wcq .1=1, wc

q ³ 0 , dkq dz( ) free

Let dk* dz( ) , wc

*be an optimal solution to (1’). Let nkkB ,..,1,1,...,1 be the set of indices of

18

DMUs for which there is no slack in (1’). Note that B , otherwise dk* dz( ) , wc

*would not be

the optimal solution (it would be possible to have a smaller dk* dz( ) . So,

wc*pcj

McU - Mc

Lc=1

a

å -wc

*pcjdz

McU - Mc

Lc=1

a

å -wc

*pkj

McU - Mc

Lc=1

a

å -wc

*pkjdz

McU - Mc

Lc=1

a

å -

wc*pcj

McU - Mc

Lc=a+1

q

å +wc

*pcjdz

McU - Mc

Lc=a+1

q

å +wc

*pkj

McU - Mc

Lc=a+1

q

å +wc

*pkjdz

McU - Mc

Lc=a+1

q

å = dk*

dz( ),"j Î B

wc*pcj

McU - Mc

Lc=1

a

å -wc

*pcjdz

McU - Mc

Lc=1

a

å -wc

*pkj

McU - Mc

Lc=1

a

å -wc

*pkjdz

McU - Mc

Lc=1

a

å -

wc*pcj

McU - Mc

Lc=a+1

q

å +wc

*pcjdz

McU - Mc

Lc=a+1

q

å +wc

*pkj

McU - Mc

Lc=a+1

q

å +wc

*pkjdz

McU - Mc

Lc=a+1

q

å < dk*

dz( ),"j Ï B

Let

Dj

= wc* pcj (1-dz -q )- Mc

L

McU - Mc

Lc=1

a

å - wc* pkj (1+dz +q )- Mc

L

McU - Mc

L+

c=1

a

å

wc* Mc

U - pcj (1-dz -q )

McU - Mc

Lc=a+1

q

å - wc* Mc

U - pkj (1+dz +q )

McU - Mc

Lc=a+1

q

å

= dk*

dz( ) -

wc*q

McU - Mc

Lpcj + pkj( )

c=1

a

å +wc

*q

McU - Mc

Lpcj + pkj( )

c=a+1

q

å .

Therefore,

"j Î B, Dj

= dk*

dz( ) -

wc*q

McU - Mc

Lpcj + pkj( )

c=1

a

å +wc

*q

McU - Mc

Lpcj + pkj( )

c=a+1

q

å .

"j Ï B, Dj

< dk*

dz( ) -

wc*q

McU - Mc

Lpcj + pkj( )

c=1

a

å +wc

*q

McU - Mc

Lpcj + pkj( )

c=a+1

q

å .

Thus, dkq dz( ) = dk

*dz( ) -

wc*q

McU - Mc

Lpcj + pkj( )

c=1

a

å +wc

*q

McU - Mc

Lpcj + pkj( )

c=a+1

q

å and wcq

= wc*is an

admissible solution to (2’), and it is found an upper bound for the optimal value of (2’). Whereas

the optimal value to (2’) is denoted by dk*(d

z+q ), then,

dk*(d

z+q ) £ dk

*dz( ) -

wc*q

McU - Mc

Lpcj + pkj( )

c=1

a

å +wc

*q

McU - Mc

Lpcj + pkj( )

c=a+1

q

å .

19

Using LEMMA 1. we are now able to prove that the dk* dz( ) is a concave function of dz, as it was our

purpose.

PROPOSITION 1.

dk* dz( ) , the solution to LP (1’), is a concave function of dz

PROOF

Let zyx

,, be possible values for the tolerance applied to the performances of each factor,

considering any DMU k, such that A=dy -dxand dz = tdx + t 1- t( )dy , with t Î 0,1] [.

Given proposition 1, if we consider =-(1-t)A, we have:

dk*(d

z- (1- t)A) £ dk

*dz( ) +

wc* 1- t( ) A

McU - Mc

Lpcj + pkj( )

c=1

a

å -wc

* 1- t( ) A

McU - Mc

Lpcj + pkj( )

c=a+1

q

å and, on the other hand,

if we do=t.A, we have: dk*(d

z+ t.A) £ dk

*dz( ) -

wc*t.A

McU - Mc

Lpcj + pkj( )

c=1

a

å +wc

*t.A

McU - Mc

Lpcj + pkj( )

c=a+1

q

å ,

with A=dy -dxand dz = tdx + t 1- t( )dy , t Î 0,1] [.

So, considering dx = dz- 1- t( ) A and dy = dz+ t.A and given what was stated earlier, we get

dk*(d

x) £ dk

*dz( )+

wc* 1- t( ) A

McU - Mc

Lpcj + pkj( )

c=1

a

å -wc

* 1- t( ) A

McU - Mc

Lpcj + pkj( )

c=a+1

q

å and

dk*(d

y) £ dk

*dz( ) -

wc*t.A

McU - Mc

Lpcj + pkj( )

c=1

a

å +wc

*t.A

McU - Mc

Lpcj + pkj( )

c=a+1

q

å .

Considering now the definition of concave function and the upper limits obtained, we have:

t.dk*

dx( ) + 1- t( ).dk

*dy( ) £ t dk* dz( ) +

wc* 1- t( ) A

McU - Mc

Lpcj + pkj( )

c=1

a

å -wc

* 1- t( ) A

McU - Mc

Lpcj + pkj( )

c=a+1

q

åé

ë

êê

ù

û

úú+

(1- t) dk*

dz( ) -

wc*t.A

McU - Mc

Lpcj + pkj( )

c=1

a

å +wc

*t.A

McU - Mc

Lpcj + pkj( )

c=a+1

q

åé

ëê

ù

ûú = dk

*dz( )

It follows that the function dk* dz( ) is concave.

20

REFERENCES

Ali A I, Lerme C S and Seiford L (1995). Components of efficiency evaluation in Data Envelopment

Analysis. European Journal of Operational Research 80: 462-473.

Almeida P N and Dias L C (2011). Value-based DEA models: application-driven developments. Journal

of the Operational Research Society 63: 16-27, 2012.

Andersen P and Petersen N C (1993). A procedure for ranking efficient units in data envelopment

analysis. Management Science 39: 1261-1264.

Athanassopoulos A D and Podinovsky V V (1997). Dominance and potential optimality multiple criteria

models decision analysis with imprecise information. Journal of the Operational Research Society 48:

142-150.

Banker R D, Charnes A and Cooper W W (1984). Some models estimating technical and scale

inefficiencies in Data Envelopment Analysis. Management Science 3: 1078-1092.

Bouyssou D (1999). Using DEA as a tool for MCDM: some remarks. Journal of the Operational

Research Society 50: 974-978.

Charnes A, Cooper W W and Rhodes E (1978). Measuring the efficiency of decision making units.

European Journal of Operational Research 2: 429-444.

Charnes A, Cooper W W, Golany B, Seiford L and Stutz J (1985). Foundations of Data Envelopment

Analysis for Pareto-Koopmans efficient empirical production functions. Journal of Econometrics 30: 91-

107.

Cooper W W, Seiford L M and Tone K (2006). Introduction to Data Envelopment Analysis and its uses.

Springer: New York.

Cooper W W, Park K S and Yu G (1999). IDEA and AR-IDEA: models for dealing with imprecise data

in DEA. Management Science 45: 597–607.

Cooper W W, Park K S and Yu G (2001a). An illustrative application of IDEA (imprecise data

envelopment analysis) to a Korean mobile telecommunication company. Operations Research 49: 807-

820.

Cooper W W, Park K S and Yu G (2001b). IDEA (imprecise data envelopment analysis) with CMDs

(column maximum decision making units). Journal of the Operational Research Society 52: 176–181.

Despotis D K and Smirlis YG (2002). Data envelopment analysis with imprecise data. European Journal

Operational Research 140: 24-26.

Dias L C and Clímaco J N (2000). Additive Aggregation with Variable Interdependent Parameters: the

VIP Analysis Software. Journal of the Operational Research Society 51: 1070-1082.

Dulá J H and Hickman B L (1997). Effects for excluding the column being scored from DEA

envelopment LP technology matrix, Journal of the Operational Research Society 48: 1001-1012.

Gouveia MC, Dias L C and Antunes CH (2008). Additive DEA based on MCDA with imprecise

information. Journal of the Operational Research Society 59: 54-63.

21

Keeney R L and Raiffa H (1976). Decisions with multiple objectives: preferences and value tradeoff.

Wiley: NewYork.

Seiford L M and Zhu J (1998a). Stability regions for maintaining efficiency in data envelopment analysis.

European Journal of Operational Research 108: 127-139.

Seiford L M and Zhu J (1998b). Sensitivity analysis of DEA models for simultaneous changes in all the

data. Journal of the Operational Research Society 49: 1060-1071.

Seiford L M and Zhu J (1999a). Infeasibility of super efficiency data envelopment analysis models.

INFOR 37: 174-187.

Seiford L M and Zhu J (1999b). An investigation of returns to scale in data envelopment analysis. Omega

27: 1-11.

Thrall R M (1996). Duality, classifications and slacks in DEA. Annals of Operations Research 66: 109-

138.

Thanassoulis E, Portela M C and Allen R (2004). Incorporating value judgments in DEA. In: Cooper W

W, Seiford L M and Zhu J (eds). Handbook on Data Envelopment Analysis, Kluwer: Boston, pp 99-137.

Von Winterfeldt D and Edwards W (1986). Decision analysis behavioural research. Cambridge

University Press: New York.

Zhu J (1996). Robustness of the efficient DMUs in data envelopment analysis. European Journal of

Operational Research 90: 451-460.

Zhu J (2001). Super-efficiency and DEA sensitivity analysis. European Journal of Operational Research

129: 443-455.

Zhu J (2003). Imprecise data envelopment analysis (IDEA): A review and improvement with an

application. European Journal of Operational Research 144: 513-529.