Farm sustainability assessment:
some procedural issues
M. Andreoli1, R. Rossi2 and V. Tellarini3
1 University of Pisa,Dip. Economia aziendale,
Via C. Ridolfi 10, 56124 Pisa, ItalyE-mail: [email protected]
2 Tuscany Region,Dip. delle Politiche Territoriali e Ambientali,
Area Tutela e valorizzazione delle risorse ambientaliVia di Novoli 26, 50127 Florence, ItalyE-mail:[email protected]
3 University of Pisa,Dip. Economia dell'Agricoltura dell'Ambiente Agro-forestale e del Territorio,
Via del Borghetto 80, 56124, Pisa, ItalyE-mail: [email protected]
Abstract
This article discusses some procedural issues relating to a multicriterial assessment of farm sustainability,
based on the criteria proposed by the European Union Concerted Action on ‘The Landscape and Nature
Production Capacity of Sustainable/Organic Types of Agriculture’. Two main problems are stressed: 1)
the treatment of basic information used for evaluating farm performances as regards the criteria and 2) the
difficulties in evaluating a case study farm. Firstly, the problem of implementing multicriterial analyses
when using qualitative ordinal data and discrete quantitative data is faced, stressing the importance of
1
2
3
4
5
6
789
10
1112131415
16171819
20
21
22
23
24
25
26
clearly defining and applying procedures that can be transferred and repeated. This is due to the fact that
almost all research contributions describe in detail multicriterial methods and results, but give little space
to the problem of collecting and analysing basic information. Nevertheless, final results heavily depend on
the way basic information has been gathered and processed in order to obtain the indices that have been
used for the assessment. The lack of standards and of procedure description does not make it possible to
compare results of assessments and to judge their suitability to the aim of farm sustainability assessment.
Secondly, the problem of finding external points of reference for judging a case-study farm is confronted.
Case studies can be important as ‘models’ for other farms. Indeed, it is easier to persuade farmers to adopt
farming styles and decisions that somebody else has already successfully implemented rather than to adopt
unexplored ways of managing their farms. This asks for reliable methods to assess a single farm, but
almost all multicriterial methods only provide a tool for ranking a set of objects, e.g., farms, from the best
to the worst. Conclusions provide some comments on the usefulness of these approaches.
Keywords: sustainable farming, multicriterial analysis, Tuscany, landscape production, Italy, case-study
assessment
27
28
29
30
31
32
33
34
35
36
37
38
39
40
1. Introduction
The use of multidimensional approaches, e.g. based on multicriterial analysisl has been a major
improvement in respect to reductionistic approaches typical of a culture too much based on specialisation
(Tellarini et al., paper presented at the EU Concerted Action on Landscape and Nature production
Capacity meeting held in Wageningen, 1996). Studying phenomena from a holistic point of view means
taking into account all their relevant facets. Although a holistic approach consents to achieve a full
understanding of phenomena, it asks for tools capable to cope with multidimensional problems. From an
applied point of view the importance of a multidimensional approach in setting up interventions for
agriculture is apparent when considering that policies aiming to steer agricultural production or to
subsidise farms do not only affect economic and productive results but affect also, e.g., the quality of
environment and landscape. The effects of farming on environmental pollution and landscape quality have
been studied in Italy, e.g., by Pennacchi et al., 1994 and 1998, Accademia Agricoltura, 1991, Chiusoli,
1994. Policies having only one aim, such as supporting farmers’ income as the ‘old’ Common
Agricultural Policy (CAP), have often resulted not only in reaching, and sometimes only partially, the
intended goal, but they have caused other unforeseen ‘side-effects’. According to Croci-Angelini (1995),
CAP has resulted in deepening regional disparities, while Baldock and Beaufoy (1993) concluded that
rationalised intensive agriculture has been associated with damage and destruction of the environment,
natural and seminatural habitats and (visual) landscapes. The negative effects that can result from farming
have increased the need for sustainable farming practices. A review of the meaning and evolution of
sustainability in agriculture has been recently provided by Polinori (1998).
A checklist for ‘Sustainable Landscape Management’ has been produced as the final report of the EU
Concerted Action on ‘The Landscape and Nature Production Capacity of Sustainable/Organic Types of
Agriculture’ (van Mansvelt and van der Lubbe, 1999). This checklist provides an inventory of indices that
might be relevant when analysing farming activity impacts. These criteria, ‘using a unifying concept
derived from Maslow’s study on human motivation translated to the landscape and perceived as a
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
reflection of the priorities and motivations leading the actions of people’ (van Mansvelt and van der
Lubbe, 1999), have been organised in six main fields: a) Environment, b) Ecology, c) Economy, d)
Sociology, e) Psychology, and f) Physiognomy/Cultural Geography. Due to the variety of relevant fields,
sustainable farming has to be analysed using a multidimensional approach. This, however, implies the
need to cope with criteria expressed in different units of measurement and with data that are not
homogeneous as regard to the level of precision. This asks first of all for a very careful treatment of the
data used for building the indices on which to base the final assessment of farm performance, and
secondly for a rational choice of the methodology to be used for reaching an ‘overall judgement’ (Colorni
and Laniado, 1988, 1992). In this context, ‘overall judgement’ indicates a summary of all the performance
that the object of the analysis has shown for all the relevant criteria.
This article attempts to systemise a series of considerations relating to the above problems, which where
stimulated by some of the contributions of the members of the EU Concerted Action on Landscape and
Nature Production.
2. The importance of ‘a priori’ clarification of rules and procedures
According to Tellarini (1995), in social science empirical research it is possible to distinguish two
different phases: the first, called ‘private phase’, which concerns research organisation, data gathering,
data verification and data processing; and the second, called ‘public phase’, which involves summarising
and commenting results. The first phase is defined as private, because it is very seldom fully described by
the researcher, since this would take too much space, especially in the case of a multidisciplinary and
multicriterial approach. Thus, when presenting multicriterial analyses, quite often only the list of criteria
that have been used is provided, without giving any explanation on the way the basic data have been
gathered and transformed into indices (e.g., environmental impact criteria in Ciani et al., 1993). According
to Colorni and Laniado (1992), the Environmental Impact Assessments performed during the ‘80s “were,
in fact, more ‘surveys’ than assessments. Moreover, such ‘surveys’ were performed according to different
points of view, with no reference to a common standard: this makes comparison of different studies
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
difficult and, even, worse, means that it is often impossible for a public authority to really check the
adequacy of the impact study”. In the same way, the lack of a common standard and of the information
needed for fully understanding how criteria have been built does not allow a rational use of many of the
studies on the impact of farmers’ choices, especially on non-economic parameters. Consequently,
although the importance of ‘a priori’ clarification of rules relating to a scientific method may seem an
obvious concern, nevertheless, in our opinion, it is important to underline that:
A) The use of qualitative data requires greater attention in the description of hypotheses adopted and
of the procedure used for building criteria, since qualitative data are more difficult to be
interpreted objectively than quantitative data. In other words, in our opinion, it is easier to
evaluate the difference between a 1.000 and a 2.000 Euro monthly income than to judge how great
is the difference between a good or a normal level of ‘offer of sensory qualities, such as colours,
smells and sounds’;
B) Although it is very seldom possible to fully describe in an article the procedures leading to the
building of criteria used for an assessment, nevertheless it is necessary that before starting an
analysis researchers fully state the procedures for gathering and processing basic information.
These procedures should accommodate for the specific requirements of qualitative and
quantitative data processing. If during the analysis one or several procedures would demonstrate
not to be suitable, it is necessary to go back and start over again. Following a stated procedure
ensures consistency in data gathering and processing.
2.1. The problem of processing qualitative data
When facing a multicriterial analysis, researchers very often have to cope with qualitative variables. Many
of the parameters proposed by the EU Concerted Action members for evaluating farm performance (van
Mansvelt and van der Lubbe, 1999), such as landscape completeness or wholeness, are qualitative.
Moreover, in many cases the cost of quantitative information is so high that, although it might be possible
to measure a phenomenon exactly, it is preferable to use a ‘discrete scale’ (e.g., income classes) rather
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
than a continuous scale, or even to use qualitative data, provided that they can be ordered (Andreoli and
Tellarini, 1999). In the latter case, researchers have to translate qualitative ordinal information into
numerical codes due to the requirements of software for multicriterial analysis. However, researchers
should remember that only methods capable to cope with qualitative ordinal data, e.g., concordance
absolute index, would give correct results. For building concordance indices it is necessary to compare
every possible pair of objects for each criterion and to check if the first has a better, worse or equal
performance than the other ones (Colorni and Laniado, 1992). Consequently, this method can not be used
when the analysis is performed on one case study. The problem of dealing with only a single case-study
farm will be discussed later.
Let us take the case of erosion in the analysis of two case study farms by Rossi et al. (1997, 1999). The
erosion analysis was performed by using a five-step scale, since the quality of information was judged
insufficient for a finer scale, where each step was represented by a symbol that was associated to a real
situation. The observed situations and associated symbols were the following:
• Clear absence of erosion ++
• Absence of erosion with some uncertainty +
• Minimal erosion (without consequences) +/-
• Moderate erosion -
• Severe erosion --
When transforming qualitative ordinal data into numerical codes and processing them with multicriterial
methods, researchers should make sure that: a) numerical codes are attributed in a rational way, ranking
qualitative data, e.g., from the best to the worst and attributing to them decreasing, or increasing,
numerical codes, and b) the method used for performing multicriterial analysis is suitable for processing
qualitative ordinal information, as in the case of concordance absolute index method.
In the above described erosion case (Rossi et al., 1997, 1999), provided that data are considered
qualitative ordinal, the translation into numerical codes of the symbols can be done, e.g., as follows:
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
Symbol ++ + +/- - --
Value 1.00 0.75 0.50 0.25 0.00
In this case values have been obtained by giving score ‘1’ to the best situation and score ‘0’ to the worst
one and finding the three intermediate values in such a way that the scale has a ‘constant stepping’. This
method is very similar to normalisation procedures, which will be described later. However, when
transforming qualitative ordinal data, it is only important that numerical codes can allow ranking
situations from the best to the worst, independently from how much a situation differs from the next one.
Thus, any scale with decreasing or increasing values can be accepted, independently from the ‘stepping’.
2.2. Using continuous or discrete quantitative data
If in the case of erosion the above symbols represent a quantitative phenomenon expressed as a discrete
scale, the proposed conversion would not any longer be correct, in so far as the situation of clear absence
of erosion with some uncertainty is much closer to that of clear absence of erosion than to that of minimal
erosion (Andreoli et al., 1998). Again, this difference is smaller than that between moderate erosion and
severe erosion. In other words, the proposed numerical conversion is correct only if erosion data are
processed as qualitative ordinal data. If the initial information is processed as quantitative data, the scale
between clear absence of erosion and severe erosion must be divided in a way that more correctly reflects
the differences in the impact of the erosion levels (Andreoli and Tellarini, 1999). Fig. 1 provides a
graphical representation of a possible numerical conversion of the above symbols in the case of qualitative
ordinal information (graphic on the left-hand side) and quantitative information (graphic on the right hand
side).
Fig. 1 - Conversion of symbols relating to real situations into numerical codes, in the case of qualitative
ordinal and quantitative discrete data.
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
2.3. From indices expressed in physical units to indices expressed in terms of Utility
Performing a multicriterial analysis based on continuous quantitative data implies confronting the problem
that criteria are expressed in different units of measurement. Measurement units are not relevant if data are
qualitative ordinal because they are used only to compare, for each criterion, if one object of the analysis
has a better, worse or equal performance than another one. On the contrary, in the case of quantitative data
it should be taken into account how much a value differs from another one. If all values are transformed
into a common unit of measurement, by means of normalisation or other procedures, it is possible to reach
an ‘overall judgement’ for every object of analysis by summing up all the values it has scored for the
relevant criteria.
One of the most common ways for normalising the values of a criterion consists (Colorni and Laniado,
1988):
a) in giving score 0 to the lowest value observed in the analysis for that criterion;
b) in giving score 100 to the highest observed in the analysis for that criterion;
c) in calculating all the intermediate values by means of a linear transformation.
This kind of normalisation has the advantages of always obtaining, for each criterion, positive values
ranging from 0 to 100, but it is subject to two main critics. First of all, the normalised value given to an
object is strictly depending on which other objects are considered in the analysis; in other words,
normalised values for a group of objects of analysis could change if a new object is added or if one of the
previous is eliminated from the analysis (Colorni and Laniado, 1988). Secondly, as seen in the above
described example of erosion, very seldom a linear and automatic transformation of values consents to
adequately represent differences existing between ‘real situations’.
Conversion of data expressed in physical units into a common measurement unit can also be done by
transforming criteria into goals or ‘objectives’ (Colorni and Laniado, 1992). This means expressing
criteria in terms of ‘satisfaction’ or ‘utility’ resulting from the physical value of the criterion itself, e.g.
evaluating the satisfaction resulting from one, or several, levels of farm incomes or from varying levels of
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
pollutant concentration rather than measuring them in thousands of Euro or in p.p.m. Thus, rather than
transforming criteria in monetary terms, as in the case of Cost-Benefits-Analysis (Dasguta and Pearce,
1975), the common unit of measure chosen is ‘Utility’. The concept of Utility is often used in economic
analysis, e.g., for describing consumers’ behaviour. Indeed, while entrepreneurs are supposed to aim to
profit maximisation, consumers are supposed to aim at maximising the utility resulting from the
consumption of products and services (Samuelson and Nordhaus, 1993). When parameters are expressed
in terms of Utility, high values have always a ‘positive meaning’ and low values have a ‘negative
meaning’, while this is not true if working with physical units. From this point of view, working with
Utility values is easier because it is not necessary to remember how an index is defined (or calculated) for
knowing if a high value is desirable, or not.
When it is possible to set a target (e.g., an optimal share of fodder crops or a satisfactory level of income)
for every parameter, the transformation of conventional data into Utility can be done by giving score ‘1’
when the target is achieved and score ‘0’ when it is not. Since this method provides a too rough
measurement scale - only two values are allowed - it is usually necessary to find an alternative procedure.
When quantitative continuous physical data are available, it is possible to have a Utility function that is
continuous, rather than dichotomous. Given that the relationship between physical and utility values is
very seldom linear, it is necessary to define it case by case. Between the concentration of a pollutant in
p.p.m. (parameter in physical terms) and the Utility associated with it; e.g., there is an inverse relationship
so that as pollution increases Utility decreases. This relationship is not linear, since it is assumed that the
level of pollution has no negative effects on the environment, as long as it is very limited. As the pollutant
concentration increases, the quality of the environment worsens, at first quite slowly and then ever more
rapidly. In other cases, e.g., when the density of a natural population is involved, there is no consistently
positive or negative relationship between the physical parameter (e.g., expressed as number of
animal/hectares) and the Utility value. When the density is low its increase determines an increase in
Utility, in that the species is reaching optimum density levels; then there is a range of optimum density
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
within which the Utility function maintains its maximum level, but beyond which the satisfaction level
decreases again (Andreoli and Tellarini, 1999).
The use of Utility function could be criticised in so far as there could be subjectivity in building them. As
Bosshard (1997) states <<experiences in landscape planning, especially in the last few years, confirm
epistemological consideration, viz. that a model for evaluating cannot be ‘objective’ - in the sense of being
generally valid. Rather, every validation is individually dependent on at least the following three premises:
1. temporary, culturally dependent ideas of values;
2. the prevailing physical situation;
3. the personal standpoint of the participants, including that of the experts, with respect to the
presentation of the problem.>>
This statement does not only apply to the problem of building Utility functions for parameters, but above
all affects the problem of deciding the relative importance (weight) to be given to each criterion in
comparison with the other ones. Subjectivity in transforming physical values in satisfaction values - as the
importance given to each criterion - could be limited by applying a procedure capable to accommodate for
these causes of variability. In other words, in our opinion, a ‘satisfactory’ level of objectivity and
comparability of results might be reached, if, e.g., rational procedures and benchmarks for transforming
physical values in utility values are defined. In the same way, although weighting is a subjective process,
it is possible to limit its subjectivity by giving guidelines and a rational procedure for attributing weights.
A method for attributing weights taking into account the features of impacts (temporary/permanent,
local/national, short/long term) and impacted resources (renewable/not renewable, common/rare,
strategic/not strategic) is proposed in Schmidt di Friedberg (1987). Finally, only the exact knowledge of
the hypotheses on which data conversion in Utility values and weighting of criteria have been performed
can allow readers to judge on the reliability of an analysis. Indeed, the quality of results of an assessment
does not depend only on the methodology used for the evaluation, but it heavily depends also on the way
data used for the assessment have been obtained.
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
3. Analysing a single case-study
Analysing a single case study is in some way more difficult than analysing a set of objects, in so far as it is
not possible to perform comparisons between objects. Thus, analysing a single case study does not allow
using qualitative ordinal data because there is not any suitable object to compare data with. Moreover, this
means that it is not possible to normalise values, due to the lack of internal reference points. Indeed,
having only one value for each criterion (the one of the case study), the concepts of minimum, maximum
and average do no longer have any meaning. Consequently, when analysing only one case study, the
transformation of criteria into a common unit of measurement has to be done by means of Utility
functions. This because Utility functions are (or could be) based on external reference points. Due to the
fact that Utility data have to be used as quantitative ones, the conversion from physical to Utility units has
to be done very carefully. Thus, in our opinion, the conversion should start by defining a procedure that:
sets external points of reference for the minimum and maximum values of the scale, namely the
physical situations that correspond to value ‘0’ and value ‘1’ of the Utility function. This process is
similar to the one of calibrating a thermometer scale, where value 0 is given to the situation of melting
ice and value 100 is given to the situation of boiling water. Varying benchmarks should/could be used
for every region. Indeed, according to Hendriks et al. (in press), external reference values may or must
differ for different landscape types/regions; since an external point of reference can not be global, but it
must be filled in regionally (see also Rossi et al., 1997). A Utopic region is needed as guiding image
for farm development;
does not apply automatic conversions implying a linear transformation of data, but it tries to define
values that are representative of differences in satisfaction relating to real situations. From this point of
view, if it is not possible to reconstruct the whole Utility function, it is sufficient to be capable to find
the Utility level to be attributed to the case study.
It is important to note that what stated, as regards conversion procedures is not only valid for the analysis
of a single case-study farm, since the same principles can be adopted when a set of objects are analysed. In
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
fact, while when assessing one farm it is only necessary to place a single value in the range defined by the
0-1 external points of reference, in the case of a set of objects there is a number of values to be
transformed that corresponds to the number of objects. In both cases, in our opinion, it is important to
discuss the way external reference points could be chosen.
Individuating the values against which to calibrate the scale means deciding which situations to use as
references for the maximum and minimum points on the scale. An ‘objective procedure’ for individuating
reference points could be the one of taking the best situation achievable in the long term for each
parameter as a reference for the maximum Utility. In this case the term of comparison for judging a case
study would be a ‘Utopic farm’. The Utopic condition is not so much tied to the achievement of a
predetermined maximum target for a single parameter (which might actually be possible for real farms), as
to the possibility of reaching the maximum value of all indicators contemporarily. Indeed the concept of
Utopic Farm is similar to the one of ‘Ideal Point’ often used in the case of multiple criteria analysis (e.g.,
Romero and Rehman, 1989), which is characterised by the contemporary achievement of all the optimum
values (individually achievable) for conflicting objectives. Using Utopic values as reference points allows
for differences due to the specific region under examination in so far as it is possible to refer to a situation
that expresses the absolute maximum possible of that parameter, independently of the area where the case-
study is located (absolute or general Utopia), or to refer to a relative maximum, expressing the maximum
level actually possible in that particular region (relative or local Utopia). The choice of a local (or relative)
Utopia or a general (or absolute) Utopia conditions the reading of the results, as well as the possibilities of
comparison when evaluations of different situations are required. So whereas evaluations expressed
against the standard of a general Utopia are directly comparable, since they use the same scale, those
expressed according to the standard of a local Utopia indicate the position of the farm with respect to the
maximum result obtainable in the reference region, so that the scale is calibrated with a maximum value
that varies according to context.
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
Since it is the whole performance, and not that one regarding a single parameter, to indicate how much
the case-study farm differs from a Utopic farm, it is important to describe how this ‘overall judgement’ on
farm performance can be reached. The easiest way of doing it is to sum up all the Utility values scored by
each farm, after multiplying them for their weights. In this case each weight represents the relative
importance given to a criterion in comparison with the other ones. It should be noted that some researchers
are against weighting criteria because weights are the result of <<a subjective, uncertain and conflictual
operation>> (Colorni and Laniado, 1992) and consequently they might be unreliable. However, not using
weights when summing up the performance scored for criteria means giving to all of them the same
weight, i.e., weight 1; this is again a subjective decision and probably less correct than explicitly giving
weights. In this context, in our opinion, it is more suitable to try to control subjectivity, e.g., by giving
guidelines for weight attribution (e.g., as in Schmidt di Friedberg, 1987) or by checking how much the
results of the analysis are depending on the chosen set of weights, than avoiding using them. In other
words, if subjectivity is unavoidable, it is at least possible to try to control it and to explicitly state the
hypothesis that can be considered as subjective in order to make the analysis as ‘transparent’ as possible
(Colorni and Laniado, 1992). Since weights are strictly depending on the socio-economic and
environmental context where the analysis is placed, it is not possible to find a weighting system that could
be generally valid in every situation. It is apparent, e.g., that developing countries where people still suffer
for starvation are more concerned in productive problems of agriculture than in those of landscape
preservation. On the contrary, in ‘rich’ countries, environment and landscape are given an increasing
interest, in comparison with the problem of agricultural production, which nowadays is often higher than
needed. Thus, if a situation implies a level for a criterion which is below the minimum required, nobody
would be ready to compensate a decrease in this criterion with an increase in another one, which is less
important or which currently has a satisfactory level. Once that physical survival requirements, or needs
considered strictly necessary, have been met, it is possible to ‘trade’ between criteria, exchanging the
‘surplus’ of a criterion for an increase in another one. Thus, the trade-off between objectives (represented
by criteria) heavily depends on their initial values. Indeed, according to a marginalistic approach
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
(Samuelson and Nordhaus, 1993) usually the importance of an improvement in a criterion is increasingly
lower when passing from a mere matching of requirements to increasing levels of surpluses. The above
statement is, in our opinion, perfectly coherent with the Maslow’s approach to human motivation used as
unifying concept in the EU Concerted Action on The Landscape and Nature Production Capacity of
Organic/Sustainable Types of Agriculture (van Mansvelt, 1997, van Mansvelt and van der Lubbe, 1999).
The use of weighted sum as method for assessing the overall farm performance and of score 1 as the
maximum Utility value results in giving to the Utopic farm an overall judgement of 1. This because
weights are recalculated in such a way that their sum is always 1. Thus the performance scored by a case-
study farm should be read taking into account that the maximum possible level of the overall judgement
(i.e., the one of the Utopic farm) is 1. In other words, if a real case-study farm would have an overall
judgement of 0.78, this would mean that its performance is 78% (0.78/1) of the maximum possible,
namely the overall judgement of the Utopic farm.
However, it should be remembered that, as shown above, exactly defining what Utopia is can be
problematic, especially as regards the choice of whether to take as reference the maximum values possible
for the various parameters (not always easy to establish) or those that can be considered maximum in the
examined context. Indeed, while the Utopic value for the erosion parameter might be objectively
generalised in ‘clear absence of erosion’, this is not the case for parameters such as farm income, where
Utopia might be characterised by extremely high values, completely incongruent with the context of the
farm under study. To set the external reference for 0 score could be still harder, since using a ‘too bad’
external reference point for score 0 might result in underestimating differences between the other
situations. Moreover, the distance between actual farm and Utopia depends on the units of measurement
adopted, or rather, on the weighting system used. In other words, using different vectors of weights, the
distance of a case-study farm from Utopia or ‘perfection’ may vary considerably.
Finally, it is important to remember that Utopia is, by definition, Pareto dominant on all the actual or
potential farm situations. <<A Pareto optimal solution is a feasible solution for which an increase in the
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
value of one criterion can only be achieved by degrading the value of at least one other criterion>>
(Romero and Rehman, 1989). Consequently, a situation is Pareto dominant when it is not worse for all
parameters and better for at least one. Since Utopia is characterised by scoring the maximum value for
each criterion, this means that real farms could match its performance but not perform better. Thus, Utopia
could not be used as a ‘second object of analysis’ for performing a multicriterial analysis based on
qualitative ordinal data.
Since farmers could consider the Utopic performance to be ‘out of reach’, researchers could consider
using a reference point that it is closer to the real case-study situation. From this point of view, another
way of calibrating the scale could be the one of using as reference points targets that could be achieved by
the case-study farm in the short or long run. In this way the judgement would consist in an assessment of
what the performance of the farm is in comparison with its potential performance in the long or short run.
In other words, with this kind of approach, it could be possible to judge how much efficient a farm is,
being the ‘inefficiency’ defined as distance between the case-study farm real situation and its potentiality.
Here too, it is essential to understand the type of reference to be used as external term of comparison, a
problem that, as in the previous case, brings us back to that of the calibration of the scale. The use of a
potential value rather than a Utopic one leads, however, to even greater problems of definition, depending
on which of the following courses is chosen:
To consider the case-study farm as a homogeneous part of the region in which it is located. In this case
the ‘local Utopia’ could be used as the term of external comparison, i.e. the best performance
theoretically obtainable in that context. This course is open to two main criticisms. Firstly, the
potentiality of the farm is not necessarily that of the surrounding territory. Indeed, with regard to
economic performance, e.g., if the size of the farm is atypical of the area, farm actual potentiality could
be quite different from that of the surrounding farms. Secondly, that reference is still made to a Utopic
rather than to a potential situation in that account is not taken of the fact that the various objectives are
conflicting. In other words, the maximum potential value obtainable for an individual parameter might
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
coincide with the Utopic one, in so far as Utopia refers to the contemporary achievement of the
maximum value for all parameters. Thus by trying to include in the ‘potentiality’ the concept of
conflicting objectives, it is much more difficult to individuate the set of maximum values for the
various objectives that may be contemporarily reached. When analysing a set of farms, a possible way
of calculating farm potential in a homogeneous context could be that of considering a selected case-
study as benchmark for the comparison, after checking that farms under study have the chance of
performing as well as the case-study farm. Although this ‘applied’ potentiality might solve the problem
of finding a set of reference values, nevertheless it might underestimate the ‘theoretical’ potentiality.
Despite the possible criticisms, the local Utopia approach is easy to apply and extend to other farms in
so far as it does not ask for repeating a double evaluation for them all, i.e. actual and potential
situations. However, the adjective ‘potential’ might be misleading since, as we have seen, it is more a
question of local Utopia (or of comparison with case-studies) rather than the specific potentials of the
farm under study.
To consider the real potentialities of the farm under examination, that need not necessarily coincide
with those of the surrounding territory for all parameters. The application of this type of approach
involves two rather difficult problems. First of all, it involves the need to carry out a double evaluation,
one of the actual situation and another of the potential situation of the farm. In other words, unlike for
Utopia with its common reference scale for the whole area, here the potentiality of the farm is
considered to be specific of the farm itself. Secondly, as in the previous case, the difficulty of defining
the potentialities of a farm with regard to a series of criteria relative to objectives that can not be
pursued contemporarily. So, unlike analyses in which only one parameter is evaluated, here there
might not be just one but many potential situations depending on the priority given to the achievement
of the various objectives. This results in great difficulty in the individuation of the potential situation to
be taken as referent. Moreover, unlike the previous case, it is not possible to use case studies as
external references in so far as farm features are not similar to the one of the context.
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
In conclusion, we believe that it is much more difficult to determine the margin of improvement of
overall farm efficiency by making use of targets potentially achievable by the actual farm than to
individuate the distance from a situation of local Utopia, even if the former method is formally more
correct. This due to the above mentioned fact that the main difference between a potential situation and
Utopia consists in not being capable to pursue and achieve contemporarily an excellent evaluation for
conflicting objectives.
4. Concluding remarks
Performing a farm sustainability assessment is not an easy task, especially if all the relevant effects of
farmers’ choices have to be taken into account. From this point of view, although the increasing interest of
researchers and the whole society are bringing about many studies on this topic, there is still a long way to
go. The checklist of criteria proposed by the EU Concerted Action on Landscape and Nature Production
constitutes a first step in this direction, providing an inventory of criteria that could be relevant for farm
sustainability assessment. Of course, since the checklist is supposed to be valid at European level,
researchers have to select every time which criteria to use and which ones are not suitable for an analysis
performed in a specific context. The second step that should be done is providing guidelines and standards
for using the criteria. This involves two different sets of problems. Firstly, the framework provided by the
EU Concerted Action members for sustainability assessment is quite complex. Thus, even if this approach
guarantees the reliability of results, nevertheless it asks for a very expensive and time consuming data
gathering. From this point of view it might be very interesting to have ‘shortcuts’, i.e., simplified
procedures for gathering information that guarantee a ‘satisfactory’, although not optimal, level of quality
of information while greatly reducing the effort needed for data collection. Secondly, it would be
important to dispose of surveys based on standard procedures, capable to provide researchers with the
reference points for calibrating criteria scales for a variety of contexts, characterised by specific socio-
economic, environmental, etc., features. This kind of research is not always very much appreciated, since
it asks for a lot of time and efforts and it only provides information for further research. In our opinion,
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
however, this kind of research is important since it allows to perform further analyses, whose results can
be compared. Moreover, since in the analysis of case-study farms researchers could be more easily biased
from their opinion on the farms they have selected, the use of standard procedures is to be strongly
advised. This article has confronted some of the issues relating to procedures that could be used for
implementing farms assessment using a multicriterial approach, and it has tried to stress the main
problems that can cause surveys and analyses not to be reliable or comparable. This with the aim of
promoting a discussion leading to the definition of standards that could be employed not only in
theoretical research, but also in applied research.
Acknowledgements
This research has been supported by National Research Council under contributions n. 94.00965.CT06
and n. 95.03251.CT06 and by the University of Pisa.
References
Accademia Nazionale di Agricoltura, 1991. Agricoltura e Ambiente, Edagricole, Bologna.
Andreoli, M., De Simone, A., Rossi, R. and Tellarini, V., 1998. Una proposta di percorso per la
valutazione di realtà aziendali o comprensoriali in base ad un set di criteri di tipo qualitativo, Il
Borghetto, Pisa.
Baldock, D. and Beaufoy, G., 1993. Nature conservation and new directions in the EC common
agriculture policy. Report for the Ministry of Agriculture, Nature Management and Fisheries, The
Netherlands. Arnhem and London: Institute for European Environmental Policy.
Bosshard, A., 1997. What does objectivity mean for analysis, valuation and implementation in agricultural
landscape planning? A practical and epistemological approach to the search for sustainability in
‘agri-culture’, Agric. Ecosyst. Environ., 63/2,3: 133-143.
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
Ciani, A., Boggia A. and Marinozzi, G., 1993. Metodologie di valutazione di alternative di parchi: il caso
del Parco del Nera, Genio Rurale, 11: 46-54.
Chiusoli, A., 1994. La rinaturalizzazione del paesaggio agrario: una esigenza ambientale, culturale e
civile, Genio Rurale, 4: 42-51.
Colorni, A. and Laniado, E., 1988. VISPA, Software Territoriale e Ambientale, CLUP, Milano.
Colorni, A. and Laniado, E., 1992. SILVIA: a decision support system for environmental impact
assessment, in Colombo, A.G. (Editor), Environmental Impact Assessment, ECSC, EEC, EAEC,
Brussels and Luxembourg, printed in the Netherlands, pp. 167-180.
Croci-Angelini, E., 1995. Effectiveness and redistribution of the regional policy in the European
Community, in Sotte, F. (Editor), The regional dimension in agricultural economics and policies,
proceedings of the 40th Seminar of the European Association of Agricultural Economists, Grafica
Tiburtina, Rome, pp. 251-274.
Dasguta, K. and Pearce, D.W., 1975. Analisi costi benefici, teoria e pratica, ISEDI, Milano.
Feliziani, R., 1997. Valutazione di alternative di gestione: il caso del Parco Nazionale dei Monti Sibillini,
Genio Rurale, 12: 25-32.
Hendriks, K., Stobbelaar D.J. and van Mansvelt J.D., 1999. The appearance of agriculture. Assessment of
landscape quality of (organic and conventional) horticultural farms in West-Friesland, Agric.
Ecosyst. Environ., in press.
Pennacchi, F., (Editor), 1994. La riforma Mac Sharry. Effetti nelle aziende R.I.C.A., CNR-RAISA n.
2224, Quaderni dell’Istituto di Economia e Politica Agraria, Tipografia dell’Università degli Studi
di Perugia, Perugia,.
Pennacchi, F., (Editor), 1998. Sostenibilità Efficienza e Successo Aziendale. Una valutazione nelle
aziende della R.I.C.A., CNR-RAISA, Quaderni dell’Istituto di Economia e Politica Agraria,
quaderno n. 24, Tipografia dell’Università degli Studi di Perugia, Perugia,
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
Polinori, P., 1998. Agricoltura e Sostenibilità, in Pennacchi, F., (Editor), Sostenibilità Efficienza e
Successo Aziendale. Una valutazione nelle aziende della R.I.C.A., CNR-RAISA, Quaderni
dell’Istituto di Economia e Politica Agraria, quaderno n. 24, Tipografia dell’Università degli Studi
di Perugia, Perugia, pp. 3-37.
Romero, C. and Rehman, T., 1989. Multiple Criteria Analysis for Agricultural Decision, Elsevier,
Amsterdam.
Rossi, R., Nota, D. and Fossi, F., 1997. Landscape and nature production capacity of organic types of
agriculture: examples of organic farms in two Tuscan landscapes, Agric. Ecosyst. Environ., 63/2,3:
159-171.
Rossi, R. and Nota, D., 1999. Nature and landscape production potentials of organic types of agriculture: a
check of evaluation criteria and parameters in two Tuscan farm-landscapes, Agric. Ecosyst.
Environ., in press.
Samuelson, P.A. and Nordhaus, W.D., 1993. Economia, italian edition of Economics, 14th edition,
Zanichelli, Bologna, 1993.
Schmidt di Friedberg, P., (Editor), 1987. Gli indicatori ambientali, Franco Angeli, Milano.
Stobbelaar, D.J. and van Mansvelt, J.D., 1999. The process of landscape evaluation. Introduction to
second special Agric. Ecosyst. Environ. issue of the Concerted Action: the Landscape and Nature
Production Capacity of Organic/Sustainable Types of Agriculture, Agric. Ecosyst. Environ., in
press.
Tellarini, V., 1995. Approcci metodologici, in Cannata, G., (Editor) Aziende e famiglie nella collina e
montagna appenniniche. Studi di casi, CNR-RAISA, FrancoAngeli, Milano, pp. 23-38.
van Mansvelt, J.D. 1997. An interdisciplinary approach to integrate a range of agro-landscape values as
proposed by representatives of various disciplines, in Agric. Ecosyst. Environ., 63/2,3: 233-250.
van Mansvelt, J.D., and van der Lubbe, M.J., 1999. Checklist for Sustainable Landscape Management,
Elsevier, Amsterdam.
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
Errata – corrige on print – proof
Page Row Errata Corrige1 8-9 from beginning abstract way basic information that has way how basic data has
1 10 from beginning abstract and their suitability and the possibility to judge their suitability
3 row 11 (right side) farms by (Rossi farms (see Rossi3 row 34 (right side) 1997, 1999 1997; Rossi and Nota, 19993 row 38 ish (right side) Symbol ++ + + - - - - Symbol ++ + + - - -
PROTO: the symbol before the last is a single dash “-“ and not a double one “- -“4 table caption qualitative ordinal qualitative ordinal (A)4 table caption quantitative discrete quantitative discrete (B)4 par. 2.2 row 5 of text closer to that closer to that PROTO:no extra space4 par. 2.3 row 22 (point b) highest observed highest value observed5 row 1 (left side) advantages advantage5 row 36 (left side) >From this point From this point5 row 8 (right side) with it; with it,5 row 36 (right side) (1) values values;5 row 37 (right side) (1) situation situation;5 row 40 (right side) (1) problem problem.7 row 10 (right side) than in those than with those7 row 36 (right side) 1999). 1999; Stobbelaar and van Mansvelt,
1999).8 row 40 (left side) >From this point From this point8 row 43 (left side) shorter or longer run short or long run9 row 26 (left side) to dispose off to have10 References: ELIMINATE: Feliziani, 199710 References: KEEP: Stobbelaar and van Mansvelt, 199910 References: ADD: Andreoli, M., Tellarini, V., 1999. Farm sustainability evaluation: methodology
and practice, Agric. Ecosys. Environm., in press
(1) It is a direct quotation
IN THE TEXT I have changed all case study and case studies in case-study and case-studies because in my Oxford dictionary is always spelt in this way
484
485
486487488489
490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521
522
Fig. 1 - Conversion of symbols relating to real situations into numerical codes, in the case of qualitative ordinal and quantitative discrete data.523
524