A Proposed Methodology to Assess the Quality of Public Use Management in Protected Areas

A Proposed Methodology to Assess the Quality of Public UseManagement in Protected Areas

Maria Munoz-Santos • Javier Benayas

Received: 9 April 2011 / Accepted: 29 March 2012 / Published online: 5 May 2012

� Springer Science+Business Media, LLC 2012

Abstract In recent years, the goal of nature preservation

has faced, almost worldwide, an increase in the number of

visitors who are interested in experiencing protected areas

resources, landscapes and stories. Spain is a good example

of this process. The rapidly increasing numbers of visitors

have prompted administrations and managers to offer and

develop a broad network of facilities and programs in order

to provide these visitors with information, knowledge and

recreation. But, are we doing it the best way? This research

focuses on developing and applying a new instrument for

evaluating the quality of visitor management in parks.

Different areas are analyzed with this instrument (78 semi-

quantitative indicators): planning and management capac-

ity (planning, funding, human resources), monitoring,

reception, information, interpretation, environmental edu-

cation, training, participation and volunteer’s programs.

Thus, we attempt to gain a general impression of the

development of the existing management model, detecting

strengths and weaknesses. Although Spain’s National Parks

constituted the specific context within which to develop the

evaluation instrument, the design thereof is intended to

provide a valid, robust and flexible method for application

to any system, network or set of protected areas in other

countries. This paper presents the instrument developed,

some results obtained following its application to Spanish

National parks, along with a discussion on the limits and

validity thereof.

Keywords Quality assessment � Visitors management �National Parks � Evaluation

Introduction

Natural areas have long been used for recreation (hunting,

fishing, hiking, etc.). Current levels of demand, however,

based upon a process that started halfway through the XX

century (Newsome and others 2002; Eagles and others

2002; Monz and others 2010), are unprecedented. More

than ever before, these natural areas have become impor-

tant places for leisure and enjoyment, a fact that appears to

reveal a new way in which the population relates to nature,

rather than just a passing fashion (Archer and Wearing

2003; Eagles 2004). Protected areas therefore provide

currently recognized and valuable recreational, cultural and

educational services (Manning and others 1999; Millen-

nium Ecosystem Assessment 2003). Nevertheless, these

places can only benefit us and fulfill conservation objec-

tives if they are well managed (Dudley and others 1999).

Thus, correct management of public use plays a vital role

with regard to optimizing the enjoyment of these services

and conserving the ecosystems that provide them.

In recent years, the evaluation of planning and man-

agement of protected areas has become one of the relevant

and intensely debated aspects in national and international

forums dealing with nature conservation and management

(e.g., evaluation of management effectiveness has been

included in the Work Program in protected areas adopted

by the Convention on Biological Diversity in 2004)

(Leverington and others 2010). There is an increasing need

M. Munoz-Santos (&)

Universidad Autonoma de Madrid, Fundacion F.G.Bernaldez,

Modulo 08. Edificio de Ciencias Dcho. 504.5,

28049 Madrid, Spain

e-mail: [email protected]

J. Benayas

Department of Ecology, Universidad Autonoma de Madrid,

Madrid, Spain

123

Environmental Management (2012) 50:106–122

DOI 10.1007/s00267-012-9863-0

to convey the results obtained both to society and to senior

staff and government ministries (Moore and others 2003).

A recent study conducted by Leverington and others (2010)

shows that at international level there are now over 8,000

evaluations of effectiveness of protected area management,

in 100 countries, and over 50 methodologies employed.

Within the scope of visitor management, there is an

important body of study dealing with monitoring and

evaluation. Over the last 50 years numerous studies have

developed the field of Recreation Ecology (see the review

essay by Monz and others 2010), as well as different

models of visitor impact monitoring, in which the infor-

mation provided by different indicators enables manage-

ment measures to be adopted (Newsome and others 2002;

Manning and Lawson 2002; Moore and others 2003).

There is also a related stream of research about the deter-

mination of standards of quality. As a result, these stan-

dards are being included in different outdoor recreation

planning and management frameworks (e.g., Limit of

Acceptable Change, Visitor Experience and Resource

Protection) (Manning and Freymund 2004). With regard to

the evaluation of communication and environmental edu-

cation programs in parks, there is a growing body of lit-

erature, although this is as yet limited (see the review

essays by Absher and Bright 2004 and Marion and Reid

2007). As for experiences in evaluation of the quality of

services and facilities offered, there is an increasing num-

ber of quality control systems, among these, the evaluation

program Visitor Services Project of the National Park

Service, USA (Le and others 2004, 2005).

In Spain, protected areas occupy 12.1 % of the land and,

as occurs at the international level, these areas are

attracting an increasing number of visitors. The 2 million

visits to these areas in 1982 have now risen to at least 26

million (EUROPARC-Espana 2009). The different

administrations and managers have reacted to the pro-

gressive increase in the number of visits since the 1980s by

means of big investments and the development of a wide

network of facilities, activities and services intended to

facilitate awareness and enjoyment of the area by these

visitors. Likewise, there has been an increase in the amount

of private companies offering complementary nature-rela-

ted activities (Blanco 2006). The challenge facing the

managers of Spain’s protected areas is a truly complex one.

They have to guarantee the conservation of these areas,

promoting visitor satisfaction and providing benefits to the

local population (39 % of Spain’s population is concen-

trated in the municipalities containing protected areas).

And these challenges must all be met within a scenario of

rapid increase, both in the number of protected areas

declared and in the amount of visits (in 1987 only 1.7 % of

the country was protected, in 2009 12.1 %) (EUROPARC-

Espana 2009).

Following the growth and consolidation phase, steps are

being taken toward a new stage characterized by the greater

attention paid to quality processes in management and to

the contribution of protected areas to ecosystem services

(EUROPARC-Espana 2009). In recent years, different

initiatives have arisen relating to evaluation of protected

area management and to the improvement of monitoring

systems aimed at providing rigorous evaluation informa-

tion (Mallarach and others 2004; EUROPARC-Espana

2010).

With regard to evaluating visitor management, despite

the existence of some partial studies on the evaluation of

certain facilities and services (e.g., Benayas and others

2000; Sureda and others 2002; EUROPARC-Espana 2006),

specific evaluations of visitor management in Spanish

parks are practically non-existent. Most of them consist of

a series of quantitative indicators integrated within the

general evaluation programs. In this context, there is a

general call for systematic evaluation programs of visitor

management to be initiated in order for it to be more

effective with regard to fulfillment of objectives.

Objectives

The objective of the research involves developing and

applying an instrument for evaluating the quality of visitor

management in parks making up the Spanish National Park

System (NPS). And thus, attempt to gain a general

impression of the development of the existing model,

detecting strengths and weaknesses. Although Spain’s

National Parks constituted the specific context within

which to develop the evaluation instrument, the design

thereof is intended to provide a valid, robust and flexible

method for application to any system, network or set of

protected areas in other countries (provided that our

methodology needs to be tailored to each specific national

or regional context).

Study Area: The Spanish National Park System

In Spain, the National Park status offers the highest degree

of protection to a natural area and also enjoys the greatest

recognition and prestige. A total of 347,081.66 ha of the

50,488,490 comprising the national territory have been

declared National Parks (14 in total, 13 at the time of the

study), the first of these in August 1918 (Fig. 1).

Spain constitutes an important tourist destination at the

international level (3rd most visited country in 2009 and

4th in 2010) (UNWTO 2010). Tourism in the National

Parks has developed gradually over the past 20 years and

in 2007 the NPS reported 10.8 million visits. The National

Environmental Management (2012) 50:106–122 107

123

Park attracting most visitors (Teide National Park) reports

3.5 million visits annually, a figure that surpasses the

number of international visits to countries such as Costa

Rica (UNWTO 2010) Fig. 2.

The NPS total budget was 223 million euros in 2009

(OAPN 2010), an average of 642.8 €/ha protected, 100 %

public funds. Abundant resources are destined for man-

agement, compared with the rest of the country’s protected

areas (e.g., the average budget for natural parks in Spain is

34.80 €/ha) (EUROPARC-Espana 2009).

Up to 2006, management was shared by the National

and Regional governments. Currently, the Regional Gov-

ernments are responsible for managing the National Parks,

and the National Government for coordinating the System.

At the time this study was conducted, the NPS availed only

of some partial, non systematical, visitor studies and not in

all the parks. No study had been undertaken to compre-

hensively and comparatively analyze the efforts on visitor

management that are made by the National Parks.

Methods: The Evaluation Instrument

Many of the traditional methods for evaluation of visitor

management tend to be based upon the perception of

users, generally by means of satisfaction questionnaires.

This is partly due to the difficulty involved in identifying

indicators and standards (Moore and Polley 2007). Vis-

itor surveys can provide meaningful and useful infor-

mation, greatly contributing to the management system.

But it can also present associated problems and biases

(Stewart and Cole 2003; Laven and others 2005; Moore

and Polley 2007). According to these authors, if the

results thereof are generalized to the absolute, the risk

arises of basing decisions on the perception of one single

kind of actors involved (visitors), whose appraisal, albeit

relevant, does not necessarily reflect all the aspects of the

problem.

The objectives and scope of work presented herein

require a global view. Thus, the evaluation proposal was

founded upon different methodological proposals, based in

turn on systematic selection of indicators and objective

data for management evaluation. The utility of this kind of

methodologies for evaluation based on objective indicators

has been proved in different studies worldwide (De Faria

1993; Cayot and Cruz 1998; Cifuentes and others 2000;

Hockings and others 2000; Sureda and others 2002; Ervin

2003; Stolton and others 2003; Mallarach and others 2004;

Velasquez and others 2004; TNC 2004, and more recently

Stolton and others 2007).

Fig. 1 Study area. Spanish National Parks Network

108 Environmental Management (2012) 50:106–122

123

Our methodology was developed by people with

expertise in visitor management (academics, recreation

professionals and managers) and was adapted to the orga-

nizational structure and management needs of the Spanish

National Parks System and to the information available or

that could be compiled at the time of the study. Due to the

lack of available information to assess the efficiency or the

effectiveness, it was decided that the object of evaluation

should be Quality.

There is much literature about the concepts of ‘‘quality’’

and ‘‘satisfaction’’ in the leisure, tourism and business lit-

erature (e.g., Mackay and Crompton 1990; Tian-Cole and

others 2002). For purposes of this research we understood

quality according to the definition proposed by EUROP-

ARC (the European Federation of National and Natural

Parks): ‘‘intrinsic conditions and qualities necessarily

inherent in resources, facilities and services of a protected

natural area in order for public use activities to unfold in a

satisfactory manner for visitors and for the Park Agency,

always within the social and conservation objectives

assigned to public use’’ (EUROPARC-Espana 2006). Much

discussion has occurred regarding the operational definition

of satisfaction, as it is a multi-dimensional concept (Man-

ning 1999), influenced by situational variables including

resource, social and management settings, being these

influences mediated by the subjective evaluations of indi-

viduals (Whisman and Hollenhorst 1998). For the purposes

of this study ‘‘satisfactory manner’’ means to enhance both

acceptable levels of resource and experiential conditions

(Laven and others 2005).

Our methodological proposal is therefore specifically

designed for systems of protected areas, which enables

results to be expressed, both in relative terms of compari-

son (this park is better than that one because…) or in

absolute terms (this study area scored highly because

it responds to a determined standard with which to make

the comparison). It was designed and applied during

2006–2008.

Design of the Evaluation Instrument

The evaluation instrument was designed by a panel of 5

experts, both academic (2 university lecturers, 1 graduate

student) and professionals (1 consultant, 1 protected area

manager). They all had previous experience in evaluation

of visitor management in protected areas in the last 5 years

and possessed knowledge of the National Parks network. It

was applied by way of a pilot experience to two National

Parks.

Selection of Analysis Categories and Sub-Categories

Firstly, based upon different documents of the Spanish

NPS dealing with visitor management policy,

Formulation of objectives: QUALITY ASSESSMENT

Design of the evaluation instrumentPanel of experts

METHODOLOGY FOR GATHERING

INFORMATION

- Written documents - Structured and semi-

structured interviews - Semi-structured and

unstructured interviews- Closed and regulated

questionnaires for appraisal of facilities and services

- Participant observation - Field notes- Photographic records

EVALUATION INSTRUMENT

78 semi-quantitative indicators (values 1-4)

Nine programmes: - Planning and

management capacity- Monitoring - Reception facilities- Information- Interpretation- Environmental education- Participation- Volunteers

CONCLUSIONS AND RECOMMENDATIONS FOR

MANAGEMENT

Selection of categories and sub-categories, indicators and criteria for evaluation

Proposal for information -gathering instruments and

techniques

Gathering information in 2 Parks and review of the methodological proposal

Application of the evaluation instrumentto the National Parks network

3 external evaluators

Synthesis of information gathered

Visits to each national park

Individual appraisal of the indicators

Results sharing and collective appraisal

Calculation of categories and sub-categories

Fig. 2 Methodology


123

organizational structure and legislation, park management

effectiveness literature and reference documents as Eagles

and others (2002) (document that later served as a basis for

Hyslop and Eagles (2007) ‘ideal’ visitors policy framework

checklist) the members of the panel established the most

important components to be evaluated (categories and sub-

categories). These categories were established according to

the management needs of the Spanish NPS. Therefore other

possible categories, relevant in other Park Systems, were

not given a category status but included within other

broader categories. See discussion below for further anal-

ysis on the categories.

The nine categories considered to be vital with regard to

obtaining a global view of public use quality for the NPS

were the following:

1. Planning and management capacity: analyses aspects

making the management possible (planning, human

resources, financial resources).

2. Monitoring and control: includes the capacity of the

managers to conduct a monitoring of activities of

visitors and companies and of impact control.

3. Reception facilities program: analyses aspects related

with the capacity and quality of the infrastructures

available for visitor management.

4. Information program: analyses the quality of the

information means and resources offered by the parks

(both personal and non-personal).

5. Interpretation program: performs in-depth analysis of

the quality of the interpretation resources and services

provided by the parks as management tools (both

personal and non-personal).

6. Environmental education program: analyses the formal

quality of educational programs for the local

population.

7. Training programs: analyses training programs for

park and external staff.

8. Participation program: analyses the formal suitability

of the mechanisms for participation of society in park

management.

9. Volunteer program: analyses the quality of the volun-

teer programs offered.

Different sub-categories and indicators were established

for each of them (see Table 1).

Selection of Indicators

Indicators are the most commonly used instruments for

developing evaluation methodologies, as they enable large

amounts of information to be synthesized in a small

number of relevant indexes, which serves to simplify the

analyses at different scales or levels (MacCool and Stankey

2004). They also favor both the identification of key

elements of pressure upon our scope of management,

enabling us to prioritize interventions therein, to evaluate

tendencies according to the objectives established and to

redefine these if this were to be necessary. The set of

interrelated indicators used in an evaluation method is

known as the indicator system. These should be coherent,

respond to common objectives, be subjected to the same

selection criteria and should also serve for decision-taking,

sharing the same structure.

As a reference document for the establishment of the

indicators and the quality standards, we used the Man-

agement Plan for the Spanish National Parks System

(Royal Decree 1803/199, 1999) which establishes specific

public use guidelines and goals for the System. We also

reviewed general literature concerning protected areas

evaluation based on indicators (e.g., De Faria 1993; Cayot

and Cruz 1998; Cifuentes and others 2000; Hockings and

others 2000; Sureda and others 2002; Mallarach and others

2004; Velasquez and others 2004; TNC 2004), reference

literature regarding visitor management (e.g., Hendee and

others 1990; Hammit and Cole 1998; Hornback and Eagles

1999; Benayas 2000; Manning and Lime 2000; Eagles and

others 2002; Newsome and others 2002) and specific lit-

erature regarding facilities, communication, interpretation

and education in protected areas (e.g., Ham 1992; Stokking

and others 1999; Weiler and Ham 2001; Sureda and others

2002; Knapp and Benton 2004; Hamu and others 2004;

Spanish Ministry of Environment 2005; Hesselink and

others 2007).

For each of the indicators we constructed a set of quality

conditions or criteria for the evaluation, in which the

optimum conditions is awarded the maximum value (4) and

the one furthest from optimum is given the minimum one

(1). Furthermore, within each category we assigned a

weighting to each indicator, according to the degree of

importance assigned thereto by the panel of experts. Four

point scale was preferred so the evaluators had to make a

definite choice rather than choose neutral or intermediate

positions. These scales demand much more from respon-

dents, but avoid the non-commital responses.

The panel first worked with a large amount of indicators

(120) which were successively subjected to analyses and

refining processes until the final list was established. For

each of the indicators representativeness, ease of collection

of the information and objectivity in its application were

discussed and appraised by the team of experts in a plenary

session. Likewise, the appraisal criteria for each one of

them were established by means of agreement. In the

selection of the indicators, future reviews and/or extensions

of the methodology proposed were considered. Following

this process, 100 indicators were selected, and subse-

quently tested in two pilot parks. Once the pilot tests were

conducted, indicators presenting low levels of agreement


123

Table 1 Categories, sub-categories, indicators and weighting

Category Sub-category Indicators (weighting)

Planning and management

capacity

1. Planning 1.1. Level of development of basic planning (3)

1.2. Level of development of planning of the visitor management area (3)

1.3. Level of development of other complementary planning tools (tourism certifications, quality

certifications) (1)

1.4. Level of development of the research program associated with visitor management (2)

1.5 Level of coordination with other tourism institutions (1)

2. Financial resources 2.1. Budget stability of the visitor management area (2)

2.2. Suitability of the budget to the needs of the visitor management area (2)

2.3. Capacity for self-financing of activities (1)

3. Human resources 3.1. Availability of staff with technical VM functions (2)

3.2. Availability of staff for visitor attention (guides) (2)

3.3. Availability of staff for law enforcement (1)

3.4. Availability of staff with VM administration functions (1)

3.5. Consolidation of the VM team (1)

3.6. Labor-related stability of the team (1)

3.7. Degree of development of internal communication procedures (1)

3.8. Level of training of the VM team (1)

Monitoring and control 4. Monitoring and control 4.1. Availability of updated information on visitor typology (1)

4.2. Availability of updated information on number of visitors (1)

4.3. Availability of updated information on satisfaction with the visit (1)

4.4. Presence of effective visitor regulation mechanisms (1)

4.5. Visitor impact control and monitoring (1)

4.6. Level of quality control of contracts (1)

Reception facilities program 5. Facilities 5.1. Suitability of number and distribution of facilities (2)

5.2. Degree of accessibility for handicapped people to the facilities provided (1)

5.3. Existence of environmental management measures in facilities (1)

5.4. Adaptation to needs of timetables and opening schedules (1)

5.5. Level of maintenance of facilities (1)

5.6. Fulfillment of basic needs (1)

5.7. Coordination with other facilities provided by other organisms (1)

6. Network of trails 6.1. Diversity of itineraries according to park characteristics (1)

6.2. Level of impact (1)

Information program 7. Signs 7.1 Signs of accesses (1)

7.2. Signs of limits (1)

7.3. Signs of facilities and services (1)

7.4. Signs in trails (1)

7.5 Signs of regulations and safety (1)

7.6. State of conservation of signs (1)

7.7. Intrusion of signs and visual impact (1)

7.8. Image of the Network (0,5)

8. Personal information 8.1. Suitability of number and distribution of information points (1)

8.2. Extent (% visitors informed by park staff in relation to total) (1)

8.3. Attention to different languages (1)

8.4. Adaptation to needs of the park and visitors: type of information offered (1)

9. Non-personal information 9.1. Diversity of publications offered (1)

9.2. Availability of publications (1)

9.3. Languages of publications (1)

9.4. Suitability of information offered in publications in relation to park and visitor characteristics (1)

9.5. Suitability of information offered through the web in relation to park and visitor characteristics (1)


123

among evaluators (more than one point of difference) were

detected and reformulated in a more objective way. Others

were eliminated due to lack of information or interest for

management (feedback was received from the managers).

Subsequent to application to pilot parks, indicators were

reduced to 78 (see Table 1): Planning and management

capacity (16), Monitoring and control (6), Reception

facilities program (9), Information program (17),

Table 2 Example of scoring system for indicator 1.2

Ind. 1.2. Degree of development of public use planning

Definition

The indicator refers to the existence of specific planning for the visitor management area

Criteria

1. There is no specific public use planning

2. There is a planning draft in its early stages, or guidelines and/or independently approved specific programs

3. There is an advanced planning draft

4. Complete public use planning is in force and available

Source of information and calculation

Documents, interviews

Table 1 continued

Category Sub-category Indicators (weighting)

Interpretation program 10. Personal interpretation 10.1. Percentage of visitors in relation to total (1)

10.2. Total diversity of guided visits offered (1)

10.3. Mechanisms of promotion, marketing and reserves (direct management)

10.4. Adaptation of visits to park and visitor characteristics (direct management) (1)

10.5. Knowledge of interpretation techniques (direct management) (2)

10.6. Quality of message transmitted (direct management) (2)

10.7. Mechanisms of promotion, marketing and reserves (contracts) (1)

10.8 Adaptation of visits to park and visitor characteristics (contracts) (1)

10.9. Knowledge of interpretation techniques (contracts) (2)

10.10. Quality of message transmitted (contracts) (2)

11. Visitor centers and

museums

11.1. Suitability of number and distribution of VC and museums(1)

11.2. Percentage of visitors in relation to total (1)

11.3. Functionality and design of facilities (1)

11.4. Quality of exhibition design (1)

11.5. Quality of message transmitted (1)

11.6. Functioning. Adaptation of timetables and opening schedules to needs (1)

12. Outdoor exhibitions 12.1. Suitability of number and distribution of panels, viewpoints, etc. (1)

12.2. Design (1)

Environmental education

program

13. EE program 13.1. Diversity of sub-programs (1)

13.2. Diversity of target groups (1)

13.4. Diversity of techniques employed (1)

13.5. Degree of innovation of educational actions (1)

13.6. Evaluation of educational actions (1)

13.7. Degree of development of school programs (1)

Training program 14. Internal training 14.1. Degree of development of internal training program (1)

15. External training 15.1. Degree of development of external training program (1)

Participation program 16. Participation program 16.1. Degree of development of participation program (1)

Volunteer program 17. Volunteer program 17.1. Availability of resources and infrastructures (1)

17.2. Continuity of actions (1)

17.3. Level of relationship with management (1)


123

Table 3 Example of scoring system for indicator 10.4

Ind. 10.4. Knowledge of interpretation techniques (park rangers)

Definition

The indicator refers to the knowledge of interpretation techniques by the guides in charge of the service

Criteria

1. The guides are generally unaware of the interpretation techniques (mean value from 1.0 to 1.5 points)

2. The guides show a medium-low level of knowledge of the interpretation techniques (mean value from 1.6 to 2.5 points)

3. The guides show a medium-high level of knowledge of the interpretation techniques (mean value from 2.6 to 3.5 points)

4. The guides show a high level of knowledge of the interpretation techniques (mean value from 3.6 to 4.0 points)

Source of information and calculation

In this case, it is a complex calculation indicator. The evaluators:

1st They make at least three guided visits offered by the staff of the National Park

2nd They complete, individually for each visitor, an ‘‘evaluation of the guided visit’’ questionnaire at the Likert 1–4 scale. In the case of

‘‘interpretation techniques’’ the questionnaire contains the following criteria (Ham 1992): The guide …Looked at the group while he talked

Spoke to the group in a sufficiently loud voice

Adapted to the languages of the group members

Spoke to the whole group and ensured that they could all see and hear

Spoke clearly

Was humorous in his explanations

Related the information to the life and personality of the group

Paid attention to possible eventualities

Stimulated participation with questions

Stimulated the use of two or more senses

Managed to involve the people

Availed of support material (guides, binoculars, photos, notebook,…)

Used vocabulary suitable to the level of the group

Used suitable anecdotes and examples

Used metaphors and analogies

Used his hands to attract attention

Repeated important comments by the visitors

The playfulness factor is of vital importance

Used languages, expressions or local accent

Used materials in his explanations (stones, feathers, …)

Was creative and original in his communication method

3rd Share the results. In the case of divergence in an answer scoring over 1 point among the judges, an interjudges agreement must be reached

4th The mean value is calculated and the indicator is completed

Table 4 Information collection sources and techniques

Technique Description

Semi-structured interviews with managers In-depth interviews with the manager in charge of visitor management in each park. The contents

and procedures are organized in advance (58 questions, 9 categories, duration of approximately 2

hours). They were conducted in person by 2 of the evaluators and the answers were recorded and

transcripted.

Semi-structured interviews with public use

staff

Interviews with guides, educators, informing subjects, guards, etc. These were semi-structured,

enabling greater flexibility and freedom. The interviewer was in charge of the contents, sequence

and formulation. They were conducted in person by 2 evaluators.

Participant observation and regulated

evaluation questionnaires

Information on the quality of the interpretation programs (exhibitions, panels and guided visits)

was obtained by means of participant observation in all the activities and resources of each of the

parks. In order to conduct this observation in a structured fashion, specific regulated questions

were prepared with quality criteria for the different facilities and services provided. These

questionnaires were designed by adapting them to the requirements of the study and were based

upon the existing literature. They were completed by 3 evaluators.

Analysis of documentation All the parks’ annual reports, monitoring reports, technical reports, files, plans, programs, etc.,

were requested and analyzed by one evaluator.

Others Filed notes, individual reports and photographs.


123

Interpretation program (17), Environmental education

program (6), Training program (2), Participation program

(1), Volunteer program (3).

Two examples of indicators are included (Tables 2, 3).

The first one is the most frequently used. The second type

is used to evaluate the interpretation services and facilities

and requires regulated checking-lists for information

gathering (questionnaires, 1–4 Likert scales).

Information Collection: The Evaluation Team

Information was gathered by three external evaluators with

experience in similar evaluation studies on visitor man-

agement in protected areas (two within the university

context and one as a consultant). To gather the information,

an evaluation visit was made, lasting at least one week, to

each of the parks, and as many possible sources of infor-

mation were used, along with information collection

techniques (Table 4). In order to minimize the disadvan-

tages associated with the use of external evaluators, the

variables that required greater internal knowledge of the

institution (for example, the indicator referring to the Suf-

ficiency of funds) are included as direct questions in the

interview with the manager of each park.

Data Processing and Interpretation of Results

Information Synthesis

During the visits to the parks, interviews, questionnaires,

participant observation, etc. were done and documents with

data on visitor use levels, budgets, etc. were collected.

Following these visits, a descriptive file was created by one

of the judges which contained fundamental information

thereon (existing programs, available resources and facili-

ties, budgets, etc.). This file was delivered to the other

judges.

Individual Appraisal of the Indicators

Three evaluators were instructed in the purpose and use of

the tool. Training in the use of the tool included developing

consistent agreement on decision rules for assessing the

indicators. Each evaluator used the descriptive file inde-

pendently, together with their own annotations and

questionnaires.

Results Sharing and Collective Appraisal

Once the individual work had been done, the appraisal for

each indicator was shared with the other evaluators. In the

event of divergences in the appraisal of an indicator the

mean of the appraisals was used. If exceptionally the

divergence exceeded more than one point the evaluators

should comment the reasons for the divergence to confirm

they were using the same decision rules, and, if the dif-

ference was due to a problem with the interpretation of the

indicator, then the decision rules were jointly reviewed

again by the experts so as to reach an acceptable level of

agreement. Once this procedure was implemented no dif-

ferences higher than one point remained. As indicators

presenting low levels of interjudges agreement were

detected in the pilot tests and were reformulated or elimi-

nated, this happened less than 10 times, all regarding the

interpretation programs (further discussion below).

Calculation of Categories and Sub-categories

For the calculation of the sub-categories, we employed the

weighted averages of each of the indicators they comprise

(see Table 1).

Results

Study Case: Application of the Evaluation Model

to the Spanish National Park System

The evaluation presented attempted to establish a broad

perspective of visitor management in National Parks. The

results obtained provide information on the health of the

institution, detecting problems to which possible opera-

tional solutions are to be found and identifying good and

bad management practices.

The information obtained is of great value and has

provided us with interesting management conclusions.

Results were delivered to the National Parks managers both

as a report and in several discussion forums. Some of the

main results are now described:

Quality of the Visitor Management Model

The results of the study show a good level of average

quality (70 % of the optimum) of visitor management in

the National Parks. None of the categories analyzed present

values close to the maxima or minima of the range of

possible scores (Table 5). The results obtained reveal dif-

ferences among Parks. The scope of variation of global

appraisal ranges from 83 % for the best appraisal to 58 %

for the worst one.

Financial Resources vs Planning

– The category obtaining the lowest results for the whole

System is planning (mean = 2.36, r = 0.81). The high


123

Ta

ble

5M

ain

resu

lts

of

the

app

lica

tio

no

fth

eev

alu

atio

nm

od

elto

the

Sp

anis

hN

atio

nal

Par

kS

yst

em

Su

b-c

ateg

ory

Par

kn

ame

Mea

nN

PS

SD

AIG

SN

EV

GA

RD

ON

OR

DT

EI

TA

BL

PIC

AC

AC

AL

CA

BT

IMIA

G

Fin

anci

alre

sou

rces

3.6

03

.60

2.8

03

.40

3.6

03

.20

3.6

03

.20

3.2

03

.20

3.6

03

.20

2.8

03

.31

0.2

9

Sig

ns

2.3

33

.47

3.8

72

.93

3.8

73

.73

4.0

03

.07

3.2

33

.73

3.0

73

.07

2.5

43

.30

0.5

3

Tra

ils

3.5

03

.50

3.5

03

.00

3.5

03

.50

3.0

03

.50

3.5

03

.50

2.5

02

.50

3.5

03

.27

0.3

9

Per

son

alin

fo.

3.2

53

.25

3.0

03

.50

3.5

02

.50

3.5

03

.00

2.7

53

.25

3.0

03

.00

2.5

03

.08

0.3

4

Tra

inin

g(e

xte

rnal

)4

.00

3.0

03

.00

3.0

03

.00

4.0

03

.00

3.0

02

.00

2.0

02

.00

4.0

03

.00

3.0

00

.71

Rec

epti

on

3.5

02

.63

3.0

03

.38

3.1

32

.88

3.1

33

.13

2.7

52

.88

3.2

53

.13

2.0

02

.98

0.3

8

Hu

man

reso

urc

es3

.70

3.3

02

.70

3.0

03

.00

2.7

03

.10

2.8

02

.80

2.9

03

.00

1.9

03

.00

2.9

20

.41

Vo

lun

teer

pro

g.

3.6

73

.67

3.0

03

.00

2.3

32

.67

2.0

02

.67

3.0

03

.00

3.0

01

.00

3.0

02

.77

0.7

0

No

np

erso

nal

info

rmat

ion

3.2

03

.00

3.0

02

.40

2.4

02

.40

2.8

02

.80

2.6

03

.20

2.4

02

.60

2.2

02

.69

0.3

3

Per

son

alin

terp

reta

tio

n3

.63

2.8

82

.00

2.8

11

.00

3.5

02

.63

2.8

83

.00

1.5

02

.71

2.8

63

.38

2.6

80

.76

Mo

nit

ori

ng

3.5

02

.67

2.3

32

.83

3.0

02

.00

2.3

32

.00

2.6

72

.17

2.8

33

.00

2.3

32

.59

0.4

4

Tra

inin

g(i

nte

rnal

)4

.00

3.0

03

.00

2.0

03

.00

2.0

02

.00

2.0

02

.00

3.0

03

.00

2.0

02

.00

2.5

40

.66

No

np

ers

inte

rpre

tati

on

2.5

02

.00

3.5

03

.50

4.0

04

.00

3.0

02

.00

2.0

03

.00

1.0

01

.00

1.0

02

.50

1.1

0

En

vir

on

men

tal

edu

cati

on

2.1

73

.17

3.8

33

.33

1.8

33

.17

1.8

33

.00

2.8

32

.33

1.1

72

.50

1.3

32

.50

0.8

1

Vis

ito

rce

nte

rs2

.82

2.0

82

.88

2.9

42

.50

2.3

12

.59

2.5

72

.92

1.5

22

.54

2.3

91

.49

2.4

30

.48

Par

tici

pat

ion

3.0

03

.00

3.0

03

.00

3.0

02

.00

2.0

02

.00

2.0

02

.00

2.0

02

.00

2.0

02

.38

0.5

1

Pla

nn

ing

3.9

02

.90

2.3

02

.40

3.3

03

.20

1.5

02

.30

2.3

02

.00

1.3

02

.20

1.1

02

.36

0.8

1

Mea

n3

.31

3.0

12

.98

2.9

72

.94

2.9

32

.71

2.7

02

.68

2.6

62

.49

2.4

92

.30

2.7

8-

Per

cen

tag

e(%

)8

2.7

57

5.1

87

4.5

77

4.1

57

3.4

77

3.1

86

7.6

66

7.5

36

6.9

96

6.4

46

2.3

16

2.2

85

7.6

06

9.5

5-

Th

eco

lum

ns

sho

wth

ed

iffe

ren

tN

Ps

and

the

row

sth

ed

iffe

ren

tsu

b-c

ateg

ori

esan

aly

zed

.In

ord

erto

emp

has

ize

the

mai

nd

iffe

ren

ces,

they

are

org

aniz

edd

ecre

asin

gly

fro

mle

ftto

rig

ht

and

top

to

bo

tto

m


123

number of visits to these areas make this situation

especially serious. Although managers are becoming

increasingly aware of the advantages of good planning,

it appears that the urgent matters left no time for the

important ones and in most cases greater priority has

been given to the inflow of visitors and the problems

they cause than to planning.

– The category obtaining the highest score is financial

resources (mean = 3.31, r = 0.29). In objective terms,

the budgets available for management in national parks

are much higher than the average for other protected

areas in Spain. Notwithstanding this overall sufficiency

of funding, managers pointed out that budget items

were not always properly balanced. In particular, they

reported some problems due to underfunding of human

resources.

Do We Know What We are Doing?

– The mean for the category of monitoring and control is

2.59 (r = 0.44). In recent years, good systems have

been developed for quantifying visits and for typifying

them. There are, however, practically no visitor studies

(satisfactions, disappointments, suggestions …) (mean

ind. 4.3 = 1.69, r = 0.95). Even though the method

here used is based on external appraisal and not in

visitors’ inputs, as justified above, visitor studies are

also necessary for managers to have a permanent

feedback.

– Problems were identified relating to quality control of

the services provided by contracted companies (mean

ind. 4.6 = 2.08, r = 0.64).

On the Facilities Provided

– The System’s reception facilities program presents a

good level of quality for the sub-category trails

(mean = 3.27, r = 0.39) which is somewhat lower for

the rest of the facilities (mean = 2.98, r = 0.38).

– No serious problems were detected relating to impacts

on trails (mean ind. 6.2 = 3.23, r = 0.44). In very few

cases, episodes were identified that required urgent

restoration measures; in other cases, these had been

adopted, with frequent use of buffering facilities.

– The general facilities are good (mean ind. 5.1 = 3.00,

r = 0.58), with problems only in the two recently

created Parks. Problems were detected which related

only to accessibility for handicapped people, thus

reflecting the lower priority given to these aspects in

the Parks (and in Spanish society as a whole) until the

last decade.

– Problems were detected in relation to the coordination

of the supply of facilities with those of other admin-

istrations (mean ind. 5.7 = 1.92, r = 0.95). On occa-

sions, we detected overabundance of facilities in a

given territory, with duplicated functions.

Communication and Environmental Education

As a whole, the results obtained exhibit high quality values

for the signs, and personal information programs, lower

ones for the volunteer, information training and non-per-

sonal information programs and low values for the pro-

grams relating to environmental education, participation

and interpretation through non-personal media.

Information

– The high values for the personal information

(mean = 3.08, r = 0.34) and signs (mean = 3.30,

r = 0.53) programs, reflect the large effort invested in

these themes in recent years. The data show, however,

that the role of the System web page is still underused

for these purposes (mean ind. 9.5 =2.00, r=0). Like-

wise, other places or facilities next to the Parks but not

depending upon the National Parks Administration (but

other parks or tourism administrations) remain unused;

if coordinated and supervised, these could provide

useful information services to National Parks’ visitors.

Interpretation

– On average, personal interpretation is valued for the

System higher than any other means of non-personal

interpretation (mean = 2.68, r = 0.76).

– Although the visitor centers greatly fulfill their recep-

tion facilities and information functions, they present

problems in relation to communication of interpretation

messages (mean = 2.43, r = 0.36). The quality of the

facilities (location, design, safety, etc.) and functioning

(timetables, maintenance, environmental management,

etc.) of these centers show high values; nonetheless, the

aspects relating to the functionality thereof (support for

management, interpretation, fulfillment of basic needs,

etc.) exhibit lower values.

– With regard to the personal interpretation, our results

show clear differences between the quality of services

managed directly by the Parks or of those managed by

enterprises. Knowledge of interpretation techniques

(ind. 10.5, 10.9) and quality of contents and delivery of

the message (ind. 10.6 and 10.10) score higher in all

cases relating to services provided by Park guides. On

the contrary, results show that contracted companies


123

avail of better marketing systems and reserves of the

services they provide (ind. 10.3 and 10.7).

Environmental Education

– Our results show a mean value for the category of the

whole of the System of 2.5 (r = 0.81), thus revealing

great variability among Parks. Five parks have very

basic environmental education programs aimed at

schoolchildren only. Other parks have made an effort to

diversify their programs and broaden their target

groups.

Participation and volunteers

– All 13 parks avail of constituted public participation

organs, but public participation, however, plays a scant

role in management (mean = 2.38, r = 0.51). The

parks generally have infrastructures and staff to deal

with the needs of the volunteer programs

(mean = 2.77, r = 0.70), although these occasionally

involve an extra workload. Only in one park are these

activities not conducted due to a lack of sufficient

human resources.

Discussion

There is a growing body of literature available on program

evaluation beginning with Theobald’s seminal text on

evaluation of Recreation and Park Programs (Theobald

1979). Many authors have reviewed evaluative concepts,

and have discussed the importance of evaluation (see for

example Sanders 1994; Hockings and others 2000; Weiler

and Ham 2001; Ervin 2003). There are many reasons for

evaluation and they vary for each situation. Evaluating

management of a protected area is not only a way of

identifying problems, but also informs us whether the

programs and strategies chosen work (Hockings 2000). The

literature is growing and there is general agreement

between the scholars that evaluation is a key aspect in the

pursuit of more effective administration. As a result,

assessments are increasingly being conducted by govern-

ment agencies (Leverington and others 2010). But, despite

all, the fact is not all parks evaluate systematically their

policies, programs or resources. That is the case of the

Spanish National Park System. In a country with little

tradition in the evaluation of the results of public policies,

evaluation of management of protected areas in general,

and of visitor management in particular, remains an issue

to be addressed. In this context, the present study and the

results obtained provide a great amount of information

which enables us to address quality evaluation of the public

use of the system studied and provides a useful instrument

for the management thereof.

Therefore, the objective of this paper does not only

involve analyzing the results of the management of public

use of the parks studied, but also to check the validity and

potentiality of the evaluation instrument applied. In this

context, further discussion of some key aspects follows.

Method Selection

The evaluation model used in this research is a standard

approach based on indicators, using independent evalua-

tors. It is based on available management literature, both in

the selection of the categories and indicators. It differs

from other similar methodologies (such as Cayot and Cruz

1998; Ervin 2003; TNC 2004) in that it has been specifi-

cally tailored to address public use issues in depth, and, as

discussed above, in its using of independent evaluators

instead of park managers opinions or visitors opinions. The

use of the instrument developed presents big advantages:

a) It enables large amounts of information to be summa-

rized in a small number of relevant indexes which

serves to simplify the analyses at different scales or

levels.

b) Facilitates comparison among different parks.

c) Enables good practices or opportunities for improve-

ment to be detected.

d) Periodic application thereof can provide information

on the evolution of visitor management over time,

showing achievements and advances, as well as the

tendencies followed.

e) Although the design of the methodology was adjusted

for application to a specific parks system, it can easily

be adapted and employed in other protected areas.

Despite the big advantages offered by this kind of semi-

quantitative evaluation instruments, however, they also

involve a series of limitations that should be considered on

interpreting the results obtained. Firstly, the design of the

whole instrument is conditioned by two factors: its neces-

sary application to a heterogeneous set of parks, and the

possibility of obtaining the information in a reasonably

easy manner.

This involves difficulties mainly within the scope of

participation, volunteer and environmental education pro-

grams where, in order to establish the real quality thereof,

one must avail of in-depth evaluations. In the case of

environmental education programs, one example is the

need for an evaluation that would entail attending the

interventions and conducting prior and subsequent evalu-

ations. The evaluation instrument designed can only make


123

a formal appraisal of these programs and their results and

must therefore be interpreted bearing this in mind (for

example, changes in program users’ behavior are not

monitored). Likewise, the evaluation instrument lacks

indicators of the effect of the visitor management model on

the sustainable development of the parks.

The use of independent evaluators provides greater

neutrality, but causes problems inherent to information

access, particularly due to the high number of indicators

used. For purposes of this research, and in order to state the

reliability of the instrument, a panel of evaluators was used,

which is an expensive approach. The cost could be mini-

mized using a more participatory approach, including park

staff in its application (see below).

Selection of the Categories and Indicators

Throughout the evaluation process, the method and eval-

uation team chosen, along with the criteria for selecting the

indicators, will have a great impact upon the result

obtained (MacCool and Stankey 2004).

Overall goals and general policies for visitation in

Spanish National Parks (e.g., to provide visitors with

information and education, not to charge fees for entrance

or for basic information services) are stated in the Man-

agement Plan for the Spanish National Parks System

(Royal Decree 1803/199, 1999) and redefined in each

park’s management plan. But we found big differences

among parks. Some of the parks had well developed

management plans but others did not have any. Therefore

the general goals stated by the Management Plan for the

Spanish National Parks System (which has a legal status,

and must be met by all the parks) were used as a basis for

the selection of the categories and indicators.

Quality of visitor management has been assessed for:

Planning and management capacity (planning, funding,

human resources); monitoring, reception, information,

interpretation, environmental education, training, partici-

pation and volunteer’s programs. As said above these cat-

egories were selected and adapted specifically for the

Spanish National Park System. Hyslop and Eagles (2007)

listed 30 different categories of visitor management policy

in an ideal framework for visitors management. Table 6

shows the comparison between the categories listed by

these authors and the categories used in this study.

Twenty four of the thirty categories used by Hyslop and

Eagles can be referred to our categories. Some of them

were included not as a category, but in the form of indi-

cators. As an example, there is no category for ‘Fees’ or

‘Marketing and Competition for Visitation’ as in the

Spanish NPS fees are not as important as in other systems.

There is a general policy stating that park access should be

free and that basic services (information, interpretation,…)

should have no cost for visitors and that complementary

services, as food and accommodation, should be provided

outside the parks. Therefore we decided not to establish a

whole category but an indicator (indicator 2.3. analyzes the

capacity for self-financing of the activities). Yet there are

six categories not considered by us, some do not fit with the

attributions of the individual Spanish National Parks (e.g.

use of an established visitor management framework),

others were not deemed relevant for visitor management in

the Spanish context when the study was conducted (e.g.,

backcountry trips) but could be included in further studies.

Identifying Current Key Issues for the Spanish National

Parks

We have presented some results obtained following the

application of the method to Spanish National Parks. The

instrument has proved itself as a good tool to identify

management key issues, detecting weaknesses and

strengths:

– Overall quality: the results shows how, although there

is room for improvement, the general situation can be

considered acceptable. The four parks with best general

appraisal also avail of certification systems and of

formal mechanisms for integration in the tourism

dynamics of the area. It could be that these certification

systems do raise the overall quality of visitor manage-

ment, or that parks with good level of management are

less resistant to the use of externally monitoring

certification systems.

– Planning models: Planning exhibits deficiencies despite

availing of appreciable financial resources; what guar-

antee is there that these resources are being invested in

the best way possible? Not planning means that many

decisions are taken due to urgent needs by management

or for political reasons. This vicious circle has been

criticized before (Blanco 2001; Corraliza and others

2002; Rosabal 2005): the lack of planning and the high

number of visits at determined times of the year mean

that resources and budget funds increase in the months

with most visits (when daily needs prevent planning)

and staff are not re-hired at times of fewer visits, rather

than being used for these planning tasks. Efforts have

been attempted in recent years aimed at mitigating

these tendencies, with specific funds and resources for

planning, but it could be that the current planning

models are not able to function in a period of rapid

change. Planning takes too long and is too expensive.

Based on the results, an in deep reflexion about the

planning approach should be made.

– Control systems: The results show how there is hardly

any control of fulfillment of the conditions set out in


123

administration contracts, and this becomes a particular

problem in relation to interpretation services. Our

results show that the increased quality of these

programs requires greater control of the contracted

companies. There is a need for monitoring processes of

these companies in order to check the functioning of

the service, but also for design of systems in order for

the administration to evaluate the services provided and

to redirect these whenever they are unsatisfactory or, in

extreme cases, not to renew the contract.

– Communication and education. A management instru-

ment to be discovered: Despite the advances made in

this respect, most agencies responsible for the man-

agement of protected areas do not avail of their own

Communication, Education and Public Awareness

strategy (Carabias and others 2003). This is the case

of the National Parks System. The inexistence of a

shared strategy has given rise to a high degree of

heterogeneity in the quality of the programs of the

different parks. The results show that environmental

education is still not used by managers as a manage-

ment instrument integrated in planning, but as a

complementary set of activities not always related with

management priorities. This situation has been

denounced in previous studies, documents and conclu-

sions of seminars and workshops (Sureda and others

2002; VVAA 2004; Spanish Ministry of Environment

2005; EUROPARC-Espana 2006). On average, per-

sonal interpretation is valued higher than any other

non-personal interpretation media. This situation is

common to the studies for evaluating the quality of

interpretation conducted annually by the USA National

Park Service (NPS 2005; Le and others 2004, 2005).

Despite these results, big investments in facilities are

Table 6 Comparison between Hyslop and Eagles (2007) ideal topics addressed through policy in an ideal framework for visitor management

and the categories, sub-categories and indicators used in the Spanish NPS evaluation

Category Sub category Indicator Not included

1. Goals or objectives of visitation *

2. Visitor use plan *

3. Use of an established visitor management framework *

4. Permitted/encouraged visitor levels and uses *

5. Conflict management *

6. Methods of transportation *

7. Trails and markings *

8. Noise restrictions *

9. Restricted items *

10. Zoning and temporary area restrictions *

11. Accessibility (for the disabled) *

12. System of reservation *

13. Dates and hours of operation *

14. Length of stay *

15. Fees and pricing *

16. Visitor education and interpretation *

17. Risk management *

18. Emergency response *

19. Backcountry trips *

20. Enforcement of rules and laws *

21. Facilities *

22. Accommodation *

23. Waste management *

24. Retail services and concessions *

25. Human resources required for visitation *

26. Marketing and competition for visitation *

27. Measurement of economic impacts of visitation *

28. Visitor use monitoring (numbers and activities *

29. Assessment of visitor satisfaction *

30. Assessment of attainment of objectives *


123

still prioritized. The high levels of investment in recent

years in visitors centers (up to 11 million euros per

center) do not correspond with the quality of the

interpretation exhibitions, or the changes these cause in

the visitors. Again, a deep reflection on the model that

is being followed should be made.

On the Reliability and Validity of the Instrument

As already said, the use of an expert panel to evaluate a

program based on a standard set of criteria is an appro-

priate, but expensive approach. Given that a new instru-

ment was developed, to assess its reliability, all three

evaluators were used not only in the selection and initial

validation of the indicators but through all the process.

Pilot tests were undertaken, to assess the feasibility con-

siderations surrounding data gathering and the internal

validity of the instrument (as discussed in the methodology

section great care was taken to ensure the rigor of the

procedure). All indicators presenting low inter-rater reli-

ability in this phase of the study were detected and refor-

mulated in a more objective way in order to minimize the

divergences. This process was done twice. Therefore the

final instrument was build as robust as possible (even if the

problems for some of the categories cited above remained).

During the subsequent application of the instrument to

all the National Parks no divergences occurred between the

judges but with the criteria for evaluating the interpretation

programs. Due to the special characteristics of these pro-

grams, and the in-depth evaluations that were conducted

(assessing aspects regarding communication, knowledge of

techniques, message, etc.) the indicators used for this cat-

egory were more complicated and information demanding.

In further studies a review of this category should be made,

in order to simplify the indicators. Nevertheless, the fact

that, in a consistent way, variations in measurements when

taken by different evaluators proved to be low also in this

category supports the overall inter–rater reliability of the

instrument.

The validity of the instrument is also supported by the

results discussed above:

– The parks with best general appraisal also avail of

certification systems.

– The lowest rated park is the ‘‘youngest’’ one, created

only two years prior to the study, and with an incipient

management system.

– The heterogeneous results between the parks are

consistent with the lack of a detailed visitor’s manage-

ment policy for the whole System.

– Several of the problems and key issues detected, were

pointed out by several authors previously in other

partial studies.

– Regarding face validity it may be pointed out, that the

park managers and staff generally agree with the study

(when explained what is measured and what is not

measured by the indicators)

Therefore we argue that the developed instrument ful-

fills its objective of assessing the management of public use

of the system studied. Overall, content validity is deemed

satisfactory, considering the broad range of categories and

indicators used. Regarding to its external validity we think

the method, with few modifications, could be used in other

parks and systems, therefore presenting a good level of

transferability. Even though it is a methodological proposal

specifically designed for systems or networks of protected

areas, it could be also used in individual parks.

As pointed out above, an evaluation panel formed only

by external experts is an expensive approach, and a more

reasonable way to gain the information the researchers are

attempting to obtain could be the participation of the

managers and staff of the parks, using it as a formative tool.

By means of participation of the key public use actors,

evaluation can become a critical exercise of reflection upon

the models created.

Concluding Remarks

This paper proposes a tool for defining reliable quality

standards and indicators, to be used in the different public

use models and for the design of systematic evaluation

strategies. This is a difficult task which cannot be suc-

cessfully addressed exclusively within the academic scope;

it should be shared by as many of the stakeholders involved

as possible. Therefore, although the instrument has proven

to constitute a useful and valuable evaluation tool, it pre-

sents certain shortcomings derived from the characteristics

of the study, which was external and limited by time and

funds. There is a need to gradually address this problem, in

each case adapting the indicators to more objective criteria

through the participation of the managers involved and by

incorporating new economic and social indicators. There is

also a need for greater participation by park staff in their

application, for training purposes. By means of participa-

tion of the key public use actors, evaluation can become a

critical exercise of reflection upon the models created.

Acknowledgments The present paper is part of the research project

‘‘Evaluation and funding of public use in protected natural areas. The

case of the Spanish Network of National Parks’’, conducted at the

Dept. of Ecology of Madrid’s Autonoma University and financed by

the Spanish Ministry of the Environment’s Organismo Autonomo de

Parques Nacionales, project 102/2002. The authors are grateful for the

funding and would like to thank all the staff of the protected areas for

their help. Thanks are also due to the anonymous reviewers for their

helpful suggestions to improve this manuscript.


123

References

Absher JD, Bright AD (2004) Communication research in outdoor

recreation and natural resources management. In: Manfredo MJ,

Vaske JJ, Bruyere BL, Field DR, Brown PJ (eds) Society and

natural resources: a summary of knowledge. Missouri, Jefferson,

pp 117–126

Archer D, Wearing S (2003) Self, space, and interpretive experience.

Journal of Interpretation Research 8(1):7–23

Benayas J, Blanco R, Gutierrez J (2000) Evaluacion de la calidad de

las visitas guiadas a espacios naturales protegidos. Topicos en

Educacion Ambiental 2(5):69–78

Benayas J (Coord.) (2000) Manual de buenas practicas del monitor de

Naturaleza: espacios naturales protegidos de Andalucıa. Junta de

Andalucıa. Sevilla, Spain, p 250

Blanco R (2001) La Situacion de Partida del uso Publico en la red de

Parques Nacionales. Un diagnostico para mejorar la gestion.

Proceedings II jornadas tecnicas sobre uso publico en la Red de

Parques Nacionales. Almagro 12–14 de noviembre de 2001

Blanco R (2006) El turismo de naturaleza en Espana y su plan de

impulso. Estudios Turısticos 169–170:7–38

Carabias J, De La Maza J, Cadena R (Coord.) (2003) Capacidades

necesarias para el manejo de areas protegidas. America Latina y

el Caribe. The Nature Conservancy, Arlington, Virginia

Cayot L, Cruz F (1998) Manual para la evaluacion de la eficiencia de

Manejo del Parque Nacional Galapagos. Servicio Parque Nacional

Galapagos. Instituto Ecuatoriano Forestal y de Areas Naturales y

Vida Silvestre, Puerto Ayora, Islas Galapagos, Ecuador

Cifuentes M, Izurieta A, de Faria H (2000) Measuring protected area

management effectiveness. WWF, GTZ, IUCN, Costa Rica

Corraliza JA, Garcıa J, Valero E (2002) Los Parques Naturales en

Espana: conservacion y disfrute. Mundiprensa, Madrid

De Faria H (1993) Elaboracion de un procedimiento para medir la

efectividad del manejo de areas silvestre protegidas y su

aplicacion en dos areas protegidas de Costa Rica. Thesis

Magister Scientiae. CATIE, Turrialba, Costa Rica

Dudley N, Hockings M, Solton S (1999) Measuring the effectiveness

of protected areas management. In: Dudley N, Solton S (eds)

Partnerships for protection. Earthscan, London

Eagles PFJ (2004) Trends affecting tourism in Protected Areas. In:

Working papers of the Finnish Forest researching Institute.

Working Papers of the Finnish Forest Research Institute 2,

18–26

Eagles PFJ, McCool SF, Haynes CDA (2002) Sustainable tourism in

protected areas: guidelines for planning and management. IUCN

Gland, Cambridge, p 183

Ervin J (2003) WWF: rapid assessment and prioritization of protected

area management (RAPPAM) methodology. WWF Gland,

Switzerland

EUROPARC-Espana (2006) Evaluacion del papel que cumplen los

equipamientos de uso publico en los espacios naturales prote-

gidos. Fundacion Fernando Gonzalez Bernaldez, Madrid, p 96

EUROPARC-Espana (2009) Anuario EUROPARC-Espana del estado

de los espacios naturales protegidos. Fundacion Fernando

Gonzalez Bernaldez, Madrid, p 160

EUROPARC-Espana (2010) Herramientas para la evaluacion de las

areas protegidas: modelo de memoria de gestion. Fundacion

Fernando Gonzalez Bernaldez, Madrid, p 121

Ham S (1992) Environmental interpretation: a practical guide for people

with big ideas and small budgets. Fulcrum Pub, Colorado, p 431

Hammit WE, Cole DN (1998) Wildland recreation: ecology and

management, 2nd edn. Wiley, New York, p 341

Hamu D, Auchincloss E, Goldstein W (eds) (2004) Communicating

protected areas, commission on education and communication.

IUCN, Gland, p 312

Hendee JC, Stanley GH, Lucas RC (1990) Wilderness management.

North American Press, Fulcrum publishing, Golden Colorado, p 540

Hesselink F, Goldstein W, Paul P; Garnett T, Dela J, Alm A (2007)

Communication, education and public awareness, a toolkit for

the convention on biological convention. Montreal, p 331

Hockings M (2000) Evaluating protected area management. a review

of systems for assessing management effectiveness of protected

areas. School of natural and Rural Systems Management,

Queensland

Hockings M, Stolton S, Dudley N, Phillips A (Eds) (2000) Evaluating

effectiveness—a framework for assessing the management of

protected areas. Best Practice Protected Area Guidelines Series

No. 6. IUCN, Gland, p 121

Hornback KE, Eagles PFJ (1999) Guidelines for public use

measurement and reporting at parks and protected areas. IUCN,

Gland, p 90

Hyslop KE, Eagles PFJ (2007) Visitor management policy of

protected areas in Canada and the United States. Leisure/Loisir

31(2):475–499

Knapp D, Benton MS (2004) Elements to successful interpretation: a

multiple case study of five national parks. Journal of Interpre-

tation Research 9(2):9–26

Laven DN, Manning RE, Krymkowski DH (2005) The relationship

between visitor-based standards of quality and existing condi-

tions in parks and outdoor recreation. Leisure Sciences

27(157–173):2005

Le Y, Littlejohn M, Hoger J, Hollenhorst S (2004) Serving the visitor

2003. Report on visitors to the National Park System. National

Park Service Visitor Services Project. University of Idaho Park

Studies Unit, Moscow, p 30

Le Y, Littlejohn M, Hoger J, Hollenhorst S (2005) Serving the visitor

2004. Report on visitors to the National Park System. National

Park Service Visitor Services Project. University of Idaho Park

Studies Unit, Moscow, p 32

Leverington F, Lemos Costa K, Pavese H, Lisle A, Hockings M

(2010) A global analysis of protected area management effec-

tiveness. Environmental Management 46:685–698

MacCool SF, Stankey GH (2004) Indicators of sustainability:

challenges and opportunities at the interface of science and

policy. Environmental Management 33(3):294–305

MacKay KJ, Crompton JL (1990) Measuring the quality of recreation

services. Journal of Park and Recreation Administration

8(3):47–56

Mallarach JM, Vila J, Varga D (Eds) (2004) El PEIN deu anys

despres: balanc i perspectives. Diversitas 50:29–4. Catedra de

Geografia i Pensament Territorial, University Girona, Girona

Manning RE (1999) Studies in outdoor recreation: search and

research for satisfaction, 2nd edn. Oregon State University

Press, Corvallis, p 374

Manning RE, Freymund WA (2004) Visual research methods to

measure standards of quality for parks and outdoor recreation.

Journal of leisure research 36(4):557–579

Manning RE, Lawson SR (2002) Carrying capacity as informed

judgment: the values of science and the science of values.

Environmental Management 30:157–168

Manning RE, Lime DW (2000) Defining and managing the quality of

wilderness recreation experiences. In: McCool SF, Cole DN,

Borrie WT, O’Loughlin J (compilers) Wilderness science in a

time of change conference. Proceedings of USDA, Forest

Service, May 23–27, 1999 Missoula, Montana, Ogden

Manning RE, Valliere W, Minteer B (1999) Values, ethics, and

attitudes toward National forest management. Society and

natural resources 12(5):421–436

Marion JL, Reid SE (2007) Minimising visitor impacts to protected

areas: the efficacy of low impact education programs. Journal of

Sustainable Tourism 15(1):5–27


123

Millennium Ecosystem Assessment (2003) Ecosystems and human

well-being: a framework for assessment. Island Press, Washing-

ton, DC, p 245

Ministerio Medio de Ambiente Espana de (Spanish Ministry of

Environment) (2005) Orientaciones para planes de CECoP

(Comunicacion, Educacion, Concienciacion y Participacion) en

humedales espanoles. Ministerio de Medio Ambiente, Madrid,

p 96

Monz CA, Cole DN, Leung Y, Marion JL (2010) Sustaining visitor

use in protected areas: future opportunities in recreation ecology

research based on the USA experience. Environmental Manage-

ment 45:551–562

Moore SA, Polley A (2007) Defining indicators and standards for

tourism impacts in protected areas: cape range National Park,

Australia. Environmental Management 39:291–300

Moore SA, Dmithe AJ, Newsome DN (2003) Environmental perfor-

mance reporting for natural areas tourism: contributions by

visitor impact management frameworks and their indicators.

Journal of sustainable tourism 11(4):348–375

Newsome D, Moore S, Dowling RK (2002) Natural area tourism

ecology. Impacts and management. Channel View Publications,

Clevedon

OAPN (2010). Annual Report 2010. Organismo Autonomo de

Parques Nacionales. Ministerio de Medio Ambiente. Madrid

Rosabal P (2005) >Como es nuestra gestion? La evaluacion de

efectividad de la gestion en areas protegidas como mecanismo

para mejorar nuestras acciones. Proceedings X Congreso

EUROPARC-Espana. Fundacion Fernando Gonzalez Bernaldez.

Madrid, p 104

Royal Decree 1803/199 (1999) Management Plan for the Spanish

National Parks System

Sanders JR (1994) The program evaluation standards: how to assess

evaluations of educational programs 2nd Ed. Sage Publications,

Thousand Oaks

Stewart WP, Cole DN (2003) On the prescriptive utility of visitor

survey research: a rejoinder to manning. Journal of Leisure

Research 35:119–127

Stokking K, van Aert L, Meijberg W, Kaskens A (1999) Evaluating

environmental education. IUCN, Gland, p 134

Stolton S, Hockings M, Dudley N, MacKinnon K, Whitten T (2003)

Reporting progress in protected areas: a site-level management

effectiveness tracking tool. World Bank/WWF Alliance for

Forest Conservation and Sustainable Use

Stolton S, Hockings M, Dudley N, MacKinnon K, Whitten T,

Leverington F (2007) The management effectiveness tracking

tool (METT) reporting progress at protected. Area Sites: Second

Edition. WWF International, Gland

Sureda Negre J, Oliver Trobat MF, Castells Valdivielso M (2002)

Avaluacio dels equipaments d’educacio i d’interpretacio ambi-

ental a les Illes Balears. Palma. Ed, Sintes, p 210

The Nature Conservancy (2004) Measuring success: the parks in Peril

site consolidation scorecard manual. The Nature Conservancy,

Arlington, p 56

Theobald WF (1979) Evaluation of recreation and park programs.

Wiley, New York, p 204

Tian-Cole S, Crompton JL, Wilson VL (2002) An empirical

investigation of the relationships between service quality,

satisfaction and behavioural intentions among visitors to a

wildlife refuge. Journal of Leisure Research 34(1):1–24

UNWTO (2010) Tourism Highlights 2010. UNWTO World Tourism

Barometer. http://www.unwto.org Accessed Dec 2010

Various Authors (2004) Proceedings I Encuentro Nacional de Guıas

Interpretadores del Patrimonio de Parques Nacionales. Parque

Nacional del Teide. Octubre de 2004, p 33 (unpublished)

Velasquez M, Guerrero P, Villegas T (2004) Parque Nacional

Galapagos. Evaluacion de la Efectividad del Manejo

(1996–2004). Ministerio del Ambiente, Parque Nacional Gala-

pagos, Ecuador, p 74

Weiler B, Ham SH (2001) Tour guides and interpretation in

ecotourism. Chapter 35 in Weaver D (ed) The encyclopedia of

ecotourism. CABI Publishing, Wallingford, pp 549–563

Whisman S, Hollenhorst S (1998) A path model of whitewater

boating satisfaction on the Cheat River of West Virginia.

Environmental Management 22:109–117


123

http://www.unwto.org

Date post:	26-Aug-2016
Category:	Documents
Upload:	javier
View:	219 times
Download:	0 times