In 2003, the European Union (EU) launched the Action Plan for Forest Law Enforcement, Governanceand Trade (FLEGT). The Action Plan aims to combat illegalities in the forest sector, using trade as theleverage to involve producers, traders and consumers in its implementation. A key instrument emergingfrom the Action Plan is Voluntary Partnership Agreements (VPAs). These are bilateral trade agreementsbetween the EU and wood exporting countries, to ensure that all wood exports were produced and tradedin accordance with the laws of the originating country. Six such agreements were negotiated so far andwith their implementation nearing, it is important to develop methodology for monitoring key impactsof a VPA. The VPAs and the Action Plan specifically outline the requirement to monitor their impacts onthe livelihoods of forest dependent communities. For this an effective VPA impact monitoring systemhas to be built that fits with the context and capacity of the country. This paper aims to contribute to
these developments by proposing an indicator framework for a country-level VPA impact monitoring.The framework comprises of 46 indicators designed to capture changes across ten key impact areas ofVPAs. The paper also discusses requirements and possible challenges of VPA impact monitoring as wellas its implications to good forest governance in the VPA countries and offers principles for cost-effectiveVPA impact monitoring.
Illegal logging and associated domestic and international traden illegally produced timber is perceived to pose significant obsta-les to the achievement of sustainable development. It results inerious environmental, economic, social and governance damagesContreras-Hermosilla, 2002). In environmental terms, it leads toegative environmental impacts including deforestation, forestegradation and loss of species and emission of greenhouse gasesContreras-Hermosilla, 2002; Tacconi, 2007). In economic terms,llegal logging and the related trade hinder economic development.he market value of products resulting from illegal logging exceedsS$ 15 billion annually (Nellemann, 2012). Another US$5 billion
s estimated to be lost annually by governments due to evadedaxes, fees, and other revenues associated with legal forestry.
n social terms, illegal logging can be linked to contributing tooverty and national and regional armed conflicts (Kaimowitz,
2003). In governance term, illegal logging weakens the rule of lawand enhances corruption (Smith et al., 2003).
The European Union (EU) is a major global importer of timberand wood products. Studies estimate that a considerable propor-tion (16–19 percent) of timber imported into the EU could befrom illegal sources (Nellemann, 2012). Thus, the EU considers ita responsibility to address illegal logging and related poor gover-nance in the timber producing countries. To address the problemsassociated with illegal logging, the European Commission (EC) pre-pared and adopted an EU Action Plan for Forest Law Enforcement,Governance and Trade (FLEGT) in 2003 (European Commission,2003). The Action Plan defines a set of processes and measuresthrough which the EC proposes to restrict illegal logging andrelated trade (Othman et al., 2012). One of the key strategies isto provide support for improved governance in wood-producing(or processing or exporting) countries and to introduce a licens-ing scheme to secure that only legal timber enters the EU throughVoluntary Partnership Agreements (VPAs) between the EU and the
government of a timber producer country (known as ‘VPA country’).
Implementation of the VPA may cause multiple impacts inthe VPA countries. The Action Plan explicitly notes the impor-tance of monitoring the impacts of the VPAs in both the EU
ria and Indicators for sustainable forest management (Howell et al.,2008) is equivalent to that of ‘key impact area’.
When determining what shall be monitored and reported, itis useful to make a contextual analysis of the VPA and the VPA
Stage 4: Ho w to go about data col lec�on
Stage 3: Iden�fyi ng indicators to monitor imp acts
Stage 2: What is to be monitored: Agreeing on impacts
Stage 1: Building understanding of impact monitoring and planning the process
88 Y.T. Tegegne et al. / Ecologic
nd VPA countries to: “. . . monitor the evolving impact of therogramme on forest sector stakeholders, including forest-based
ndustries in the EU and wood-producing countries, and gov-rnments and local communities in wood-producing countries”European Commission, 2003). The VPA also outlines a requirementf monitoring and assessing the impacts of the agreement. This iso ensure that the VPA is meeting its objectives and not causingnexpected negative impacts (e.g. on livelihoods or on trade).
The impacts of an intervention or political decisions like VPAould be well assessed and monitored with the help of an indicator-ased framework that covers all relevant impact areas of VPAs.
good indicator framework should provide a robust way todentify positive impacts and help to identify any unintended neg-tive impacts, and thereby help to implement targeted mitigationeasures (Kishor and Rosenbaum, 2003). The use of appropriate
ndicators provides an efficient way of monitoring and commu-icating results of policy decisions and management activitiesNiemeijer, 2002). However, a methodology framework of rele-ant, practical and applicable indicators, which adequately coverhe most important impact areas of VPAs implementation, is stillacking.
This paper aims to contribute to these developments by propos-ng an indicator framework for a widely accepted, country-levelPA impact monitoring system. This paper is based on review-
ng and analysis of initiatives (e.g. CIFOR, 1999; Davis et al., 2013;TTO, 2005; Kishor and Rosenbaum, 2012; Lawson and MacFaul,010; Proforest, 2012) that have developed indicators, indicatorrameworks or the processes behind. Important inferences havelso been drawn from the analysis of official documents suchs all of the existing VPA agreements to date. This informationas supplemented by personal experience and knowledge from
nvolvement of the authors in activities for the EU FLEGT Facility.his paper is structured around the following main themes: VPAmpact monitoring; a discussion of proposed five stages for VPAmpact monitoring; proposed indicator framework and discussionsn requirements and possible challenges of VPA impact monitorings well as its implications to good governance in the VPA countries.
. VPA impact monitoring
Through a VPA, EU and the VPA country agree to establish andaintain necessary support to guarantee that timber and timber
roducts exported to the EU are from guaranteed legal sourcesnd have a so-called FLEGT license, which shows that all rele-ant laws of the producer country are complied with (Europeanommission, 2007). An important element of VPA is the Legalityssurance System (LAS), which sets out the tracking proceduresy which a VPA country will ensure that timber products originaterom legal sources (European Commission, 2007).
The first VPA negotiation to be formally concluded was withhana in late 2009 (Beeko and Arts, 2010). Until May 2014, six VPAegotiations between the EU and timber producing countries haveeen finalized: Ghana, Cameroon, Liberia, Republic of Congo, theentral African Republic and Indonesia (EU FLEGT Facility, 2013).hese countries are in the process of preparation for VPA imple-entation, which means that impact monitoring systems should
oon be in place. Negotiations are on-going with the Democraticepublic of Congo, Honduras, Gabon, Guyana, Laos, Malaysia andietnam; several other countries have also expressed interest.
The implementation of a VPA may have multiple potentialmpacts (Beeko and Arts, 2010; Ramcilovic-Suominen et al., 2010).
or instance, the VPA negotiation process itself positively impactedorest governance in VPA countries by creating more opportu-ity for dialogue and providing a framework to strengthen forestontrol, transparency and accountability (Othman et al., 2012).
icators 46 (2014) 487–494
However, there have been concerns that the implementation ofa VPA could harm the current and future livelihood of forestdependent communities (Beeko and Arts, 2010). An indicatorbased impact monitoring system could be a means that providesthe required information and evidence for appropriate decision-making related to managing the impacts of a VPA. However, mostexperience of building indicator frameworks relate to monitoringof sustainable forest management; there is so far no framework inplace for the VPA impact monitoring and its specific, but variable,conditions.
3. Proposed five stage approach to VPA impact monitoringsystem
The VPA involves a theoretical commitment to impact moni-toring, which must be followed by an effective, practical systemtailored to a country’s context and capacity. This involves a widerange of stakeholders and practitioners. It is important to build acommon understanding of the importance of impact monitoring,what is possible, how it is to be achieved, who is responsible andhow the processes are to be institutionalized. The authors of thispaper propose to approach the development of a VPA impact mon-itoring system into five stages (Fig. 1). In a first stage, the processmust begin by building an understanding of the need and approachto impact monitoring.
In a second stage, stakeholders should identify what shall bemonitored and reported. This is a very important stage as it is anopportunity to make the intended impacts of the VPA as explicit aspossible.
A recap of the original objectives and expected impacts of aVPA is necessary. This is important as many different stakeholdersbrought their own sets of issues to the negotiation table, whichalso colour their expectations towards VPA implementation. Fur-thermore, the team that negotiated a VPA may not be the same asthe one that should develop the impact monitoring. At this stage,important documents including the VPA text, relevant nationalpolicies and regulations such as land and tenure policies, and exist-ing and any on-going forest-sector monitoring initiatives should bereviewed and analysed. This could help briefing stakeholders withthe mapping of potential impact areas of a VPA.
A VPA can have impacts on environment, economical, social andgovernance aspects, resulting in large numbers of impacts whichcan be categorized into ‘key impact areas’. A key impact area isa group of individual impacts which relate to the same aspect ofinterest to structure and enable a better comprehension of the largeamounts of impacts. In practice, the concept of ‘criteria’ in the Crite-
Stage 5: Test and refine the preliminary indicator framework
Fig. 1. Five proposed stages to establish a VPA impact monitoring system.
Box 1: A good indicator for VPA impact monitoringneeds to be SMARTI:
• Specific: an indicator should monitor what they claim tomonitor (i.e. the specific impacts of VPA) and are not con-founded by other factors.
• Measurable: ideally indicator should be reliable, comparableand contextually appropriate. For this qualitative, quantita-tive or binary measurements can be used.
• Achievable: the indicator should be realistic in terms of appli-cability, cost and complexity of data collection and systemsthat are in place. Technological innovations can have an influ-ence in the long run.
• Relevant: an indicator must provide information useful to theVPA objectives and assumptions and help guide decisionsthat key users will need to make.
• Time-bound: indicator should describe when there is changeand track it over time.
• Interpretable: an indicator should give an indicator valueeasy to understand and interpret across range of end users.
ii
cIVeps
MTs(biv(ataitr
oabTfotgicmit
dfit
(e.g. through LAS implementation) or as part of other processes.
• Source: Castro (2011) and Drucker (1954).
mplementation because certain aspects of reporting and monitor-ng are legally binding requirements under the agreement.
In the third stage of VPA impact monitoring, appropriate indi-ators under each key impact areas are selected and developed.ndicators for VPA impact monitoring need to reflect the progress ofPA implementation, pick up early warning signs when undesirableffects would occur, identify problem areas, and allow for appro-riate modifications and timely corrections of VPA implementationtrategies.
Effective indicators have the following characteristics: Specific,easurable, Accurate, Realistic, Timely and Interpretable (SMARTI).
he monitoring framework being in a sum of its components, ithould generally follow the characteristics. The SMARTI attributesBox 1) were proposed e.g. by Castro (2011) and Drucker (1954)ut also recent studies that focused more strongly on developing
ndicators provided us with useful criteria such as policy rele-ance, quality and affordability of data and existing set of indicatorsOlsson et al., 2009; Pülzl et al., 2012). Although the SMARTIttributes should govern the selection and development of indica-ors, a pragmatic approach will be required where indicators withll these characteristics are difficult to identify. As a result, somendicators that do not meet all of the criteria may be included inhe indicator set because some impacts of VPAs would otherwiseemain unaddressed.
The fourth stage defines, for each indicator, data sources, meth-ds and guidelines for practical data collection and calculation,nd these should be documented in such a way that the data cane interpreted unambiguously by anyone doing data processing.his also includes instructions on metadata. Data can be sourcedrom statistics, company and sector reports, independent surveys,ther rural or sectoral development programmes, or other repor-ing initiatives. Relevant data for VPA impact monitoring can beenerated by monitoring embedded in the VPA itself (e.g. the LAS,ndependent third-party audits and independent monitoring byivil society). Additionally, useful data may be produced by otheronitoring programmes that operate independently of the VPA,
ncluding monitoring in the context of REDD+, biodiversity protec-ion, regional and rural development.
A fifth stage tests and refines the indicators with the stakehol-ers who will use them. It is crucial to know whether the indicator
ramework and supplementary information are suitable in practicen the VPA country. Any user responsible for reporting and moni-oring needs to be able to unambiguously collect and calculate the
icators 46 (2014) 487–494 489
required data for each indicator within the VPA impact monitor-ing system. Identifying complications during the test phase willallow for reiterations and adjustments to the VPA impact moni-toring system. Following this stage, a clear overview should havebeen made of system accuracies and needed resources, allowingfor a realistic planning for a full implementation of the monitoringsystem.
4. Building an indicator framework for VPA impactmonitoring
4.1. Proposed key impact areas of VPA
Proforest (2012) identified nine potential key impact areasthrough discussions with the European Commission, European For-est Institute (EFI) and experts from other organizations. In thisstudy, authors took the initial list by Proforest of impact areasand re-formulated, checked and expanded the list so that it cancover the requirements and assumptions of the FLEGT Actions Planand the VPAs. Such criteria include relevance of the impact areato the context of the VPA implementation, ease of availability;cost and time consideration in setting baseline for the monitor-ing framework for a particular impact area; and the logic andease in addressing the attribution gap between VPA implemen-tation outputs and expected impacts. The impact areas can beconsidered in the wider context of sustainability and improved gov-ernance. The proposed key impact areas are categorized into twogroups of different nature: (i) forests, market and livelihoods and(ii) governance. The first group – forests, market and livelihoods– focuses on changes on the ground (i.e. impacts on the forests,the market and on the local population) and includes key impactareas such as forest conditions; economic development; domes-tic market development; and livelihoods and poverty. The secondgroup – governance – comprises elements describing the actionsand processes enabling to achieve the desired impacts of the VPAimplementation on the ground and includes key impact areas suchas effectiveness of stakeholders’ involvement; institutional effec-tiveness and efficiency; accountability and transparency; illegallogging; law enforcement and compliance and rights of access andforest tenure. These impact areas provide useful input for nationaldiscussions but should not be considered a blueprint because it isessential to have an open, participatory, stakeholder-led discus-sion to develop a nationally agreed set of impacts. The rationaleand assumptions for each of the suggested impact areas are brieflyelaborated in Box 2.
4.2. Proposed list of indicators
This paper presents a set of 46 indicators that could be usedto capture evidence of change across the ten key impact areas(Fig. 2). The indicators are drawn from a number of existing indica-tor sets, including – among others – VPAs, CIFOR (1999), Davis et al.(2013), ITTO (2005), Kishor and Rosenbaum (2012), Lawson andMacFaul (2010) and Proforest (2012). The main criteria in selectionof indicators from the different sources were whether an indicatormeasures impacts very much in line with the overarching objectiveand underlying assumptions of FLEGT Action Plan and the VPAs.Another objective was that they would rely as much as possible onexisting or planned operational data sources, so that the proposedframework would be cost-effective. Hence, most of these indicatorscan either be measured as a direct result of VPA implementation
Not all the indicators fulfil all the SMARTI requirements (e.g.cost), but each indicator included in the framework is accompa-nied by sub-indicators and checked if the indicators fulfil most of
A—Forests, market and livelihoods1. Forest conditions
The FLEGT Action Plan contains the assumption thatstronger enforcement of national laws, including thoserelating to forest management planning and logging oper-ations, will ultimately lead to more sustainable forestmanagement. This impact area focuses on how VPAsinfluence forest conditions and its implications towardssustainable forest management.
2. Economic developmentEconomic development is a strong incentive for improve-
ment in governance in the forest sector. VPA partnercountries would like to see improvements in the contri-bution of their forest sectors to the national economicdevelopment.
3. Domestic market developmentAlthough the primary aim of a VPA is to address timber
exports to the EU, most countries are interested in usingthem also to address broad governance challenges in theforest sector. VPA countries have included considerationsto support reforms of their domestic market.
4. Livelihoods and povertyAs stated by the EU FLEGT Action Plan, ‘the challenge
is to ensure that actions to address illegal logging, par-ticularly enhanced law enforcement, do not target weakgroups, such as the rural poor, while leaving powerful play-ers unscathed’ (2003: Section 4.1.1, p. 6.). The EU and theVPA countries will want to see what has been achieved onpoverty alleviation.
B—Governance5. Effectiveness of stakeholders’ involvement
VPAs require relevant stakeholders are appropriatelyand adequately represented. This means that stakehol-ders have the right to make their own decisions onwho represents them, that their representatives have theopportunity to make contributions and that those contrib-utions are considered.
6. Accountability and transparencyThe actors involved in VPA implementation must be
accountable for their actions. The assumption is thatgreater transparency leads to improved accountability andbetter law enforcement. This will improve the effective-ness of different stakeholders.
7. Illegal loggingThe legality definition for each country provides a
benchmark for monitoring illegal logging. This impactarea considers the VPAs’ impact on illegalities in timberoperations and volume of legally produced timber trade.
8. Institutional effectiveness and efficiencyThis impact area addresses whether government insti-
tutions are effectively and efficiently carrying out theirmandates in relation to the forest sector. The result isimprovement in programmes and in the quality of servicesand institutions.
9. Rights of access and forest tenureThis is related to the VPAs’ objective of strengthening
regulation to secure and protect access and ownershiprights, and ensuring equitable benefit sharing.
10. Law enforcement and complianceVPAs aim to strengthen forest law enforcement and
compliance. This impact areas focus on VPAs’ impact onforest offences and the extent of implementation of andcompliance with laws and regulations.
icators 46 (2014) 487–494
the SMARTI requirements. However, indicators with all these char-acteristics may be difficult to identify in some areas. As a result,some indicators that do not meet all of the SMARTI requirementsare included in the indicator set (Fig. 2) because they cover impactsof VPAs that would otherwise remain unaddressed.
The indicators proposed here are intended to be generic andto be adapted to a VPA country’s specific needs and context. Theindicators are neither exclusive nor exhaustive and some indicatorsmay not be practical or applicable for all VPA countries. The setcan be iteratively developed by adding or substituting indicators orby including indicators to probe more deeply and introduce finerlayers of detail on relevant aspects.
In some situations, it may be necessary to provide a very briefsummary of the impacts attributable to VPA implementation. Insuch cases, given that FLEGT concerns Forest Law Enforcement,Governance and Trade and its overarching objective of reducingpoverty, the indicators in the summary could be chosen in connec-tion to the impact areas: livelihoods and poverty; illegal logging;economic development; institutional effectiveness and forest con-ditions. The method for preparing the summary would benefit fromstakeholder endorsement. Even though these indicators reflectsome important aspects, they cannot paint the full picture of VPAimpacts.
4.2.1. Forest conditions indicatorsThe long-term aim of the FLEGT Action Plan is to promote
sustainable forest management in timber producing countries bysupporting improved forest law enforcement and governance. Inhere, the assumption with the VPAs is that better adherence toexisting forest and environmental legislation will lead to improvedforest management planning and logging operations and ultimatelysustainable forest management. The VPA also clarifies and enforceslegislation, including that related to the protection of forests andthe identification and demarcation of protected areas. These con-siderations have been taken into account in the development of theenvironmental indicators suggested here (Fig. 2).
4.2.2. Economic development and domestic markets indicatorsEconomic development indicators measure economic impacts
of VPAs – as bilateral trade agreements – which aim to ensure andreinforce the economic contribution of forests to national develop-ment and in most VPAs a consideration of legality in the domesticmarket trade. The VPAs aim to ensure that the economic rentsdue from forest use are made so as to contribute to national andlocal development. They aim to reassure the EU markets that allimports from that country are verified legally compliant and in sodoing, increase or maintain EU (and international) market for highvalue exports from VPA Partner Countries. Additionally, domesticmarkets in many VPA countries consume an increasing proportionof illegal timber (Lawson and MacFaul 2010). The importance ofdomestic markets in terms of the national economy and livelihoodscannot be ignored and should be included in impact monitoringfrom the start. Therefore, the economic indicators need to address– among others – the VPAs impacts on volume and value of exportsto EU and international markets, and forest sectors’ contribution tothe overall national economy of a VPA country, but also focus onhow VPAs could affect the domestic timber market and the devel-opment of government timber procurement policies. All together,ten economic indicators have been proposed (Fig. 2).
4.2.3. Livelihoods and poverty indicatorsFive indicators proposed have been designed to capture the
impacts of VPAs on livelihoods of forest-dependent communities.The VPA is expected to have effects on livelihoods and poverty lev-els (Ramcilovic-Suominen et al., 2010; Beeko and Arts, 2010). Oneassumption behind the VPA is that “improved resource governance
esults in positive socio-economic outcomes and poverty allevia-ion for communities dependent on forest resources” (EFI, 2009).PA processes could improve livelihood conditions and reduceoverty through: creating employment opportunities, securingccess to forest resources, improved and equitable distribution ofevenue to local stakeholders and ensuring social agreements areegotiated in the interests of affected communities (Hobley anduchy, 2011).
.2.4. Governance indicatorsThe VPAs are framed by the assumption that greater
ransparency leads to improved accountability and better lawnforcement (FERN, 2013). This assumption has to be tested atountry-level, given the variations in the level of transparency inifferent countries, and the differences in systems to ensure trans-arency. Different actors involved in VPA implementation must beccountable for their actions. There should be mutual accountabil-ty between different actors including civil society, government andhe private sector. The LAS plays a role in increasing accountabilityy requiring documentation, chain of custody control, verificationnd independent audits. VPAs include detailed requirements aboutransparency and information which must be made publicly avail-ble (FERN, 2013).
The second key impact area that was decidedly proposed under
he governance aspect is that of ‘illegal logging’. The FLEGT Actionlan was designed to address the pervasive problems associatedith illegal logging and to promote trade in legally produced tim-
ers. The underlying assumptions under this key impact area are
k for VPA impact monitoring.
(i) that existing policies, laws and regulations will be strengthenedand enforced to support sustainable forest management, and (ii)that a functioning LAS reduces the level of illegal logging.
VPA implementation requires new institutional arrangementsall of which are expected to function in a way that con-tributes to achieving VPA goals. Successful implementation of VPAsthus require improved performance of government institutionsinvolved in the VPA, increased efficiency of business processes andimproved coordination amongst those responsible for regulationand control of forest management and timber traceability, amongothers. Here, the assumption is that more effective and efficientinstitutions lead to improved forest conditions and livelihoods, andthat greater effectiveness reduces costs.
In addition, the VPAs encourage the development and use ofmulti-stakeholder dialogues. All relevant stakeholders should beable to participate actively, influence outcomes and should demandaccountability of decision-makers at all levels. Therefore, poten-tial indicators have been designed to assess whether a VPA hasincreased the capacity of stakeholders to participate and enhancedstakeholders’ role in the VPA processes (Fig. 2).
Another important aspect of forest governance is tenure andaccess. VPAs aim to strengthen regulations that secure and pro-tect access and ownership rights. They also aim to ensure equitablebenefit sharing. Potential indicators have been proposed to test the
assumption of clear rights of access and forest tenure will reducedisputes, illegal activities and promote investment and how thesemay change after VPA implementation. Finally, further emphasis isplaced on how VPAs could lead to changes in law enforcement and
4 al Ind
ca
5
psaattloV
5
ipaaicc
tdtikdr
ttspsasspteoiSm
iita2
tmtVmvm
92 Y.T. Tegegne et al. / Ecologic
ompliance in a VPA country. These issues have been taken intoccount in the development of governance indicators (Fig. 2).
. Discussion
Monitoring the impacts of VPAs is the responsibility of botharties of the agreement. The VPAs and the FLEGT Action Planpecifically outline the requirement to monitor the impacts of thegreement. The VPA countries are under time pressure to developn effective VPA impact monitoring system that fits with the con-ext and capacity of the country. In this section, the authors discusshe requirements of VPA impact monitoring, the possible chal-enges to VPA impact monitoring, and finally address implicationsf participatory VPA impact monitoring to forest governance in thePA countries.
.1. Important considerations for VPA impact monitoring
A number of conditions and considerations should be takennto account by a VPA country while developing and implementingarticipatory, cost-effective VPA impact monitoring. The consider-tions, among others, include conducting VPA impact monitorings an iterative and consultative process. The considerations alsonclude that VPA impact monitoring should take account of the spe-ific national context and be built in legally binding aspects. Theseonsiderations are briefly discussed below.
First, developing a VPA impact monitoring system should beaken as an iterative process (Fig. 1). Although the process isepicted as a linear sequence of consecutive stages, in realityhere may be a need for iterations between steps when develop-ng the system. Necessary iterations may include adjustments toey impacts, selection and development of indicators, methods ofata collection, data sources for each indicator or the analysis andeporting of the results.
Second, as the VPA derives its legitimacy from its nature ashe product of a multi-stakeholder process, the development ofhe impact monitoring system too must capitalize on the multi-takeholder mechanism and be managed as an open democraticrocess. The involvement and interaction of a range of actors andtakeholders are of paramount importance for the developmentnd acceptance of the monitoring system. Representatives of alltakeholder groups must be involved from the outset and at alltages, so that they can participate in shaping the process androvide knowledge and content. Such participation will enhancehe credibility and reliability of the VPA impact monitoring system,nsure that it capitalizes on local knowledge and understandingf the country context (Mendoza and Prabhu, 2005) and help toncrease acceptance of the results once the system is implemented.takeholder participation may also enhance stakeholders’ commit-ent to carrying out data collection and monitoring.It is important that the whole stakeholder process is structured
n such a way that stakeholders with no or only a little formal train-ng in defining impacts and indicators can grasp the concepts andhe process, feel comfortable sharing their inputs and knowledge,nd be able to contribute their expertise with relative ease (Hobley,013).
Key impact areas will emerge from the objective embedded inhe VPA and from the changes assumed to result from its imple-
entation. It is essential that stakeholders are involved in defininghe key impact areas, or disagreements will likely arise as to which
PA objectives are relevant for monitoring and how they should beonitored. The monitoring framework will only fulfil its expected
alue when it is endorsed and supported by all stakeholders. Thiseans that not only should in-country stakeholders agree on the
icators 46 (2014) 487–494
impact areas for monitoring, but also that agreement between apartner country and the EU is needed.
As the VPA text – and therefore its objectives – is not fullydefined until negotiations are concluded, it may be difficult to iden-tify key impact areas and ways to monitor them early in the VPAnegotiation process. Defining key impact areas should be made apriority as early as possible—maybe as part of the last negotiationsessions or just after the agreement is initialled. This would alsohelp the definition and assessment of the baseline. When the mon-itoring framework is developed during the implementation phaseof the agreement, representatives of the key stakeholders groupsshould be given a chance to revisit and clarify the key changes thattheir constituents wanted from the VPA. Identifying such expectedchanges will help define impact areas.
When defining indicators for each key impact area, it is abso-lutely necessary to first analyse the VPA and its annexes, whichcontain numerous references to potential indicators. Facilitate datacollection and assessment as much as possible by relying on exist-ing operational data sources. Such indicators can measure directresults of VPA implementation (e.g. through LAS implementation)or indirect results that can be affected also through other processes.
The ten key impact areas and 46 indicators proposed in thispaper are not blue print. Countries have to agree on and adopt aset of their own key impact areas (and set of indicators) that theywould want to monitor. This enhances the acceptability and credi-bility of the monitoring system. The countries are recommended tofollow specific criteria to select, develop and screen the initial listof impact areas and indicators into a manageable and methodologi-cally informed final list of impact areas and indicators. In this aspect,the SMARTI attributes could play a role. Stakeholders should eval-uate the degree to which each indicator meets each of the SMARTIattributes by assigning a score, for instance.
Ghana – the first country to sign and ratify the VPA agreement– recently formed a multi-stakeholder team on VPA impact mon-itoring. In August 2013, the team met to discuss on VPA impactareas to be built into a monitoring framework. Three criteria wereframed into a weighting matrix for screening an initial list of keyimpact areas. The criteria include relevance of the impact area to thecontext of the VPA implementation, availability of information andattribution. The team came up with a final list of 8 key impact areas.It is expected that the Ghanaian team will discuss and endorse theinitial list of key impact areas with their counterparts from EU in2014.
As the key impact areas are the arbitrary outcome/selection ofa political process, it is important that the agreement on the keyimpact areas is a well supported consensus amongst the stake-holder groups. Absence of such consensus would undermine theacceptance of any results from the implementation of the impactmonitoring system.
A third important consideration is building of the VPA impactmonitoring into legally binding aspects. When determining whatwill be monitored and reported, it is useful to remember that theVPA and its related commitments form the basis of VPA impactmonitoring, and therefore that certain aspects of reporting andmonitoring are legally binding requirements under the agreement.Hence, part of the content for VPA impact monitoring will be foundin the VPA document, particularly in the core articles, but also inannexes addressing, for example, the legality definition or crite-ria for assessing the LAS. The more the features of the VPA impactmonitoring system reflect the issues arising in VPA negotiations,and the more familiar stakeholders are with the content of theVPA document and key issues, the easier and faster the whole pro-cess is likely to be. This could also help developing country-specificcriteria or to modify the SMARTI attributes so that country-specific
issues are taken into consideration during selecting and developingindicators.
al Ind
5
iwupha(pR
oiacw
apiomsooioctttat1oa
tbaboiti
5
psepttekhpm(kti
Y.T. Tegegne et al. / Ecologic
.2. Possible obstacles for VPA impact monitoring
First, achieving appropriate level of stakeholder groups’ partic-pation, and particularly ensuring that stakeholders concerns are
ell addressed is challenging. The VPA process has provided a gen-ine and positive context for mobilization and participation of allarties to the decision-making. Since the signature of the VPAs;owever, there have been signs that this momentum has deflatednd is at risk of being lost as the implementation has been slowDooley and Ozinga, 2011). The depletion of momentum can inart be explained also by a shift of attention and resources towardsEDD+.
A second obstacle for successful VPA impact monitoring is a lackf existing data, particularly for defining a baseline. The data andnformation that VPA countries have may not be adequate or reli-ble enough to serve as a basis for meaningful monitoring. This willomplicate measuring progress through comparison of any futureith any past situation.
Third, a main challenge for any kind of impact monitoring isttributing particular impacts or observed changes directly to theolicy implementation (i.e. the VPA in this case). Such attribution
s always a difficult task (Dickson and Kapos, 2012) because thebserved changes could be due to other large scale policy imple-entation (e.g. agricultural policy, REDD+ strategy, forest policy),
olely or in combination with a VPA. The reliability and credibilityf a VPA impact monitoring methodology depends for large partn its capacity to explain and demonstrate attribution. An exam-nation is needed to discriminate between changes that might beccurring because of a VPA implementation on the one hand, andhanges that are occurring directly, in part or indirectly because ofhe VPA implementation. The challenge is then to choose indica-ors that relate to these impacts. Two approaches can help addresshis challenge. First, already agreed VPAs contain a lot of referencesbout what they want to achieve and what not, pointing to the iden-ification of potential indicators. Second, theory of change (Weiss,995), impact pathway analysis (GEF, 2010) and the impact modelf Metz (2005) could assist in determining which impacts can bettributed to VPA.
Fourth, in line with the principles of the VPA being both coun-ry driven and country specific, VPA impact monitoring needs toe very much a country driven or steered process. This introduces
challenge in terms of reflecting on impacts that go beyond theounds of specific countries and that can be aggregated to reflectn the impacts of VPAs more generally. Finally, corruption, politicalnterference, institutional capacity and lack of competent exper-ise in the VPA countries could pose some challenges for effectivemplementation of a country-level VPA impact monitoring system.
.3. Implications to good forest governance
Some of the key attributes of good forest governance are trans-arency and accountability, participation, and equitable benefitharing (Davis et al., 2013; Kishor and Rosenbaum, 2012). Gath-ring and sharing information and dialogue are key features ofarticipatory VPA impact monitoring processes, which contributeo accountability and transparency. Through a VPA impact moni-oring process relevant stakeholders become more aware of howquitably the benefits are shared within the community and whereey problems and gaps lie. VPA impact monitoring may help stake-olders from private and civil society to strengthen or developartnerships with public and other stakeholders. It can also pro-ote dialogue among stakeholders with different interests or have
had) adversarial relations. It further contributes to improving thenowledge base and creating joint analysis that also gives meaningo the partnership. Furthermore, the proposed key impact areas andndicators enable to monitor the changes on the ground, but also
icators 46 (2014) 487–494 493
take note of the governing processes such as civil society participa-tion, institutional effectiveness and improve law enforcement andcompliance.
6. Conclusion
Each VPA includes a commitment to monitor the impacts ofVPAs, but the approach for doing so has not yet been agreed.The indicator framework proposed in this paper could be usedas a tool to assess whether VPA is having the desired objectivesand impacts. The proposed indicator framework could serve as aninput for international discussions but could also be applicable toa country-specific VPA impact monitoring. The framework is quitecomprehensive, and given its scope can be tailored to fit country-specific condition.
VPA impact monitoring should raise positive impacts. It alsofacilitates early detection of unintended negative impacts, andthereby enabling to take corrective or mitigation measures, so thatthe problem can be dealt with cost-effectively. This paper offers thefollowing principles for cost-effective VPA impact monitoring:
• Root the VPA impact monitoring into what really matters as out-lined in the legally binding agreement;
• Use best applicable practices and data when possible;• Invest time in the selection of SMARTI indicators;• Use participatory VPA impact monitoring systems to prioritize
effort and to build consensus; and• Use data generated by monitoring embedded in the VPA.
Finally, the size of the effort and reach of VPA impact monitor-ing must be in balance with the implementation capacities of VPAcountries. Further research should help to identify issues that arenot captured by the proposed framework but relevant to the imple-mentation of the VPA, such as capacity building, the stakeholderengagement processes, training, awareness raising, anti-corruptionmeasures, and similar.
Acknowledgments
The contents for this paper evolved from the work by the authorsof this paper on a VPA impact monitoring Sourcebook for practi-tioners for the EFI EU FLEGT Facility. The authors are grateful totwo anonymous reviewers for their comments on the manuscript.
Appendix A. Supplementary data
Supplementary data associated with this article can befound, in the online version, at http://dx.doi.org/10.1016/j.ecolind.2014.07.020.
References
Beeko, C., Arts, B., 2010. The EU-Ghana VPA: a comparative policy analysis of itsdesign. Int. For. Rev. 12, 221–230.
Castro, M.F., 2011. Defining and Using Performance Indicators and Targets in Gov-ernment M&E Systems. The World Bank, Washington, DC.
CIFOR, 1999. Guidelines for Developing, Testing and Selecting Criteria and Indicatorsfor Sustainable Forest Management. CIFOR, Jakarta, www.cbd.int/doc/case-studies/tttc/tttc-00143-en.pdf (accessed June 2013).
Contreras-Hermosilla, A., 2002. Law Compliance in the Forestry Sector: An overview.World Bank Institute, Washington, DC, USA.
Dooley, K., Ozinga, S., 2011. Building on forest Governance reforms through FLEGT:the best way of controlling forests’ contribution to climate change? Reciel 20(2.).
Drucker, P., 1954. The Practice of Management. Harper and Row, New York, NY.
FI, 2009. What is a Voluntary Partnership Agreement? The European Unionapproach. EFI Policy Brief 3. European Forest Institute, Joensuu, Finland.
U FLEGT Facility, 2013. http://www.euflegt.efi.int/vpa-countries (accessed January2013).
uropean Commission, 2003. Communication from the Commission to the Coun-cil and European Parliament: Forest Law Enforcement, Governance and Trade(FLEGT) Proposal for an EU Action Plan. European Commission.
uropean Commission, 2007. FLEGT Briefing Notes: Forest Law Enforcement, Gov-ernance and Trade, 1–7. European Commission.
ERN, 2013. Improving Forest Governance: A comparison of FLEGT VPAs and theirImpact. FERN, www.fern.org/improvingforestgovernance (accessed October2013).
EF, 2010. Impact pathways analysis: a framework for enhanced sustainable landmanagement project design. In: Document Submitted by the GEF MSP Measur-ing Impacts from SLM—Development of a Global Indicator System. GEF.
obley, M., 2013. Poverty Impact Assessment for Voluntary Partnership Agreements.European Forest Institute, Joensuu, Finland.
obley, M., Buchy, M., 2011. FLEGT and Poverty Alleviation: the Role of VPAs. Euro-pean Forest Institute, Joensuu, Finland.
TTO, 2005. Revised ITTO Criteria and Indicators for the Sustainable Management ofTropical Forests, ITTO Policy Development Series No. 15. ITTO.
aimowitz, D., 2003. Forest law enforcement and rural livelihoods. Int. For. Rev. 5,199–210.
ishor, N., Rosenbaum, K., 2003. Indicators to monitor progress of forest law enforce-ment and governance initiatives to control illegal practices in the forest sector.Int. For. Rev. 5, 211–218.
ishor, N., Rosenbaum, K., 2012. Assessing and Monitoring Forest Governance: AUser’s Guide to a Diagnostic Tool. Program on Forests (PROFOR), Washington,DC.
awson, S., MacFaul, L., 2010. Measuring the Response to Illegal Logging: Lessonsfor FLEGT Partnership Agreements. Chatham House, www.illegal-logging.info/uploads/CHillegalloggingpaperwebready1.pdf (accessed May 2013).
etz, M., 2005. Monitoring Policy Impacts (MPI): The Eight Methodo-logical stepsfor MPI. Food and Agriculture Organization of the United Nations, FAO, Rome.
icators 46 (2014) 487–494
Mendoza, G.A., Prabhu, R., 2005. Combining participatory modelling and multicri-teria analysis for community-based forest management. For. Ecol. Manage. 207,145–156.
Nellemann, C., 2012. Green carbon, black trade: illegal logging, tax fraud and laun-dering in the worlds tropical forests. In: A Rapid Response Assessment. INTERPOLEnvironmental Crime Programme. United Nations Environment Programme,GRIDArendal, www.grida.no.
Niemeijer, D., 2002. Developing indicators for environmental policy: data-drivenand theory-driven approaches examined by example. Environ. Sci. Policy 5,91–103.
Olsson, J.A., Bockstaller, C., Stapleton, L.M., Ewert, F., Knapen, R., Therond, O., Geni-aux, G., Bellon, S., Correira, T.P., Turpin, N., Bezlepkina, I., 2009. A goal orientedindicator framework to support integrated assessment of new policies for agri-environmental systems. Environ. Sci. Policy 12, 562–572.
Othman, M., Leal, I., Devers, D., Turunen, L., 2012. FLEGT voluntary partnership agree-ments. In: Broekhoven, G., Herman, S., v. Scheliha, S. (Eds.), Moving Forward withForest Governance. Tropenbos International, Wageningen, The Netherlands, pp.109–116.
Proforest, 2012. VPA Impact Monitoring: Options Paper and Indicator. Proforest.Pülzl, H., Prokofieva, I., Berg, S., Rametsteiner, E., Aggestam, F., Wolfslehner, B., 2012.
Indicator development in sustainability impact assessment: balancing theoryand practice. Eur. J. Forest Res. 131, 35–46.
Ramcilovic-Suominen, S., Gritten, D., Saastamoinen, O., 2010. Concept of liveli-hood in the FLEGT voluntary partnership agreement and the expectedimpacts on the livelihood of forest communities in Ghana. Int. For. Rev. 12,361–369.
Smith, J., Obidzinski, K.K., Suramenggala Subarudi, I., 2003. Illegal logging, collusivecorruption and fragmented governments in Kalimantan, Indonesia. Int. For. Rev.5 (3), 293–302.
Tacconi, L., 2007. Illegal Logging: Law Enforcement, Livelihoods and the TimberTrade. Earthscan, London, pp. 301.
Weiss, C.H., 1995. Nothing as practical as good theory: exploring theory-basedevaluation for comprehensive community initiatives for children and fami-lies. In: Connell, J.P., Kubisch, Anne C., Schorr, Lisbeth B., Weiss, Carol H. (Eds.),Approaches to Evaluating Community Initiatives: Concepts, Methods, and Con-texts. Aspen Institute, Washington, DC.
