SustainabilityA-TestHow to select tools
for a sustainability assessment?
Marjan van Herwijnen VUA-IVM Amsterdam
OECD workshop onSustainability Assessment Methodologies
14-15 January 2008, Amsterdam, the Netherlands
Content
• Overview and goals of the project• The result: a webbook• Selecting tools• Conclusions
SustainabilityA-Test: Overview
• EU project (DG research) in 6th Framework;• Duration: March 2004 - August 2006• Budget: 1.3 million euro• 18 European project partners
Purpose of SustainabilityA-Test …
• … is to scientifically underpin the use of assessment tools in integrated assessments (IA) for sustainable development (SD).
• This purpose has been made concrete by:– Making a tool inventory– Making logical tool groups– Link each group to the policy cycle
• Results are made available through internet
The webbook: home page
The webbook: Overview
The webbook: Book of references
The webbook: Case study
The webbook: About
Selecting tools
• Tool overview• Evaluation framework• Overall framework• Role of tools• Tool selection process B
Tool overview (Feb. 2005)
Physical assessments
MFAecologicalfootprint
Stakeholder analysisand participatory processes multi-agent repertory grid
interactivebackcasting
Environmentalappraisal SIA SEA
vulnerabilityassessment
Multi-criteriaanalysis
Non-compensatory
methods
Multi-attribute utility
theory
Linearadditivemodels
Scenario andsimulation
Use existing scenarios Develop new scenarios
Physical models Socio/economic models
Integrated models exploratory anticipatory baseline
policy qualitative
quantitative
Monetaryassessments(cost-benefitAnalysis)
Market Approaches to Estimating the WTP
Tools?
Revealed-preference Studies
hedonicproperty valuetravel cost
Non-market Approaches to Estimating the WTP
contingentvaluation
ModellingPartial
economic sectormodels
generaleconomy
models
hydrologicalmodels
biochemicalcycle models
climate models
vegetationmodels
public healthmodels
Physical models Socio/economic models qualitativesystem analysis
modelsland usemodelsIA
models scenariobuilding and planning
tools
Integrated models
demographicmodels
= procedural tool
= tool
= group or sub-group of tools
Legend
indicatorbased assessment
TRUA
Other tools?
CLRC Other tools?
Analyticalhierarchyprocess
Outrankingmethods
FuzzyMCA
focus groups ICT-tools
Tool overview (Sept. 2005)
Multi-criteria analysis tools
Non compensatoryPartial compensatory
PROME-THEE NAIADE Regime Dominance
method
Compensatory
Multi-attribute
value theory
Analytichierarchyprocess
Weightedsummation
Assessment frameworksStrategic
environmentalassessment
EU impact assessment
Integrated sustainabilityassessment / Transition
management
Environmentalimpact
assessment
Cost-benefit and cost-effectiveness analysis tools
Cost-benefit analysis and monetary evaluation tools
Hedonicpricing
Travel costs
Costs of illness
Avertingexpenditures
Contingent valuation
Contingent behaviour
Conjointchoice
questions
Cost-effectiveness
analysis
Marketmethods
Model tools
Family of bio-physical models
Family of socio-economic models
Hydrology models
Climate models
Biogeo-chemical models
General economy models
Partial economy models
Demo-graphic models
Public health models
Family of integrated models
Integrated assessment
models
Qualitative system analysis
models
Land use models
Scenario building and
planning tools
Accounting tools, physical analysistools and indicator sets
Physical analysistools
Economy-wide MFA
Global land use
accounting
Ecologicalfootprint
Lifecycleanalysis
Indicator setsIndicator sets for
vulnerabilityassessments
Indicator sets for assessments
Accounting tools
Measure of economicwelfare
ISEW
Sustainablenationalincome
Genuinesavings
NAMEA
Scenario tools
Trends Cross Impact
Relevance trees and morphologic
analysis
Modelling, simulating,
gaming
Interactivebrainstorming
Scenario workshops
Integrated foresightmanage-ment
model
Rankingmethod
Participatory tools
ConventionalIT based
Electronicfocus
groups
Repertorygrid
techniqueTIDDD Interactive
backcastingConsensus conference
Focus groups
DelphiSurvey
In-depthinterviews
Citizen’sjury
Tool overview (Jan. 2006)
Evaluation framework
The evaluation framework includes aspects for:• Policy processes
– Recognition of the problem;– …– Evaluation of the implemented policy option.
• Sustainable development– Lists of environmental, social and economic aspects;– Crosscutting aspects.
• Operational aspects– Input/output aspects;– Transparency aspects;– Tool characteristics.
Evaluation: database
Evaluation: result
tool recogn. invest. expl. analysis. selection. implem. eval. discont. inter-gen. de-)coup. adapt. (ir-)rever. distr. global. spacial. envir. socialLUC 2 2 2 2 2 2 2 3 2 1 1 3 2 2 1 0 3IA 2 2 2 2 2 2 3 3 1 1 1 1 1 1 1 1 3QSA 1 1 1 1 1 3 3 3 2 2 2 2 2 2 2 2 2SBP 1 1 1 1 2 2 3 3 1 1 1 1 1 1 1 2 2Climate 2 2 2 2 3 3 3 3 1 3 2 1 3 1 3 1 3Hydrology 2 2 2 2 3 3 2 2 2 3 2 1 2 2 1 1 3BGC 2 2 2 2 3 3 2 2 2 3 2 1 2 2 1 1 3GEM 3 2 2 1 2 3 2 2 2 1 2 3 2 2 3 3 3Demography 1 1 3 3 3 3 1 3 1 3 2 2 3 1 3 3 0PEM 3 2 2 1 2 3 2 2 3 2 2 3 2 2 2 2 3Health 2 2 2 2 2 2 2 3 2 0 2 3 2 1 2 1 1NAMEA 1 1 3 2 3 3 1 1 1 1 1 3 1 1 1 1 1economy-wide 1 1 1 2 5 5 1 5 1 1 2 1 2 1 2 1 3GLUA/TRUA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0EF 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3CLARC 1 2 2 2 2 3 2 3 2 3 3 3 3 2 3 2 1LCA 1 3 2 3 2 2 2 2 2 1 4 2 2 2 2 1 3ScenA - Appl 1 1 1 2 2 3 3 3 2 3 3 3 2 2 2 2 2ScenA - new 1 1 2 2 2 3 3 3 2 3 3 3 2 2 2 2 2CBA 0 0 1 1 1 0 1 0 2 2 0 2 3 4 1 1 1CEA 3 3 2 1 1 2 1 2 3 2 2 3 3 2 2 2 2MEW 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0ISEW 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0SNI 1 1 2 3 3 3 3 3 3 1 1 1 2 2 3 1 3Genuine Saving 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0MCA 2 2 3 3 1 3 3 2 2 2 2 2 2 2 2 1 1MAVT 3 2 2 3 1 3 3 3 2 2 2 2 2 1 2 1 1WSum 3 2 2 3 1 3 3 3 2 2 2 2 2 1 2 1 1AHP 1 1 1 1 1 3 3 3 2 2 2 2 2 1 2 1 1PROMETHEE 2 2 2 3 1 3 3 3 2 2 2 2 2 1 2 1 1NAIADE 3 2 3 3 1 1 1 2 2 2 2 2 2 2 2 3 3DomM 3 2 3 3 1 3 3 3 2 2 2 2 2 1 2 1 1(S)IA 3 1 3 1 1 3 3 3 2 2 4 2 2 3 4 2 2SEA 3 2 1 1 3 3 1 3 0 0 0 0 0 0 0 1 5IAM 1 2 3 3 2 3 3 3 2 2 2 3 1 2 2 1 2VALSA 1 1 2 3 2 3 3 3 2 1 1 3 2 3 3 2 2SIndbAT 1 2 3 1 2 3 1 3 3 1 1 3 1 1 1 1 2eFG 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0RGT 1 1 1 1 2 3 1 3 5 5 5 5 5 5 5 2 1TIDDD 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0BC 1 2 1 1 1 0 2 2 2 2 2 2 2 2 2 2 2CC 2 1 1 1 2 4 1 2 3 3 3 3 2 3 2 1 0FG 1 1 1 1 2 1 1 3 3 3 3 3 2 2 2 2 2TM 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Development of an ‘overall framework’
• Key-question:– Which tools to use in an ‘IA for SD’?
• Two sub-questions:– Which tasks can be distinguished in an ‘IA for SD’?– Which tools can support these tasks?
Which tasks can be distinguished in an ‘IA for SD’?
• Nearly all IAs distinguish 4 phases:I. Problem analysesII. Finding solutionsIII. AnalysisIV. Follow-up
• Irrespective of the type of IA, the tasks to be done in each phase are much alike
Generic framework for assessments
Problem recognition
Investigating problem / conflicting
assumptions
Identification of possible solutions
Analysis of policy
proposals
Selection of policy option
Implementation
Discontinuation
Assessimpacts
I – Problemanalysis
II – Finding solutions
III – Analysis
IV – Follow up
Compare policy options
Evaluation
Impact assessment:describe the problem, the objectives and policy options
Impact assessment:describe how future monitoring and assessment could be organised
Four phases
Chapter 4, p. 40
I – Problem analysis
II – Finding solutions
III – Analysis
IV – Follow up
AIMTo understand the problem and to frame it, whilst accounting for different views on it
USEFUL TOOLSTools that can steer the process of mobilising knowledge and articulating values, by
means of stakeholder participation (experts, policy makers, lay people).
AIMTo identify all possible options so as to act
on the problem as defined in Phase I
USEFUL TOOLSScenario analysis tools and tools that can
support scenario development
AIMCharacterising as far as possible the details of the plausible scenarios and policy interventions
developed in Phase II, with the final aim of selecting options for implementation
USEFUL TOOLSAnalytical tools, such as models, indicator sets, cost–benefit analysis tools and physical analysis
tools
AIMLearning and monitoring/evaluation
USEFUL TOOLSLearning: participatory toolsMonitoring and evaluation:
analytical tools
Raw data and statistics
Indicator toolsEx-post assessment
Full characterizationof scenarios to enable
comparison
Supporting scenario construction
Supporting problem framing
Model tools
–Compare different alternativeDefinition of criteria–
Cost-benefit analysis (CBA) and cost effectiveness
analysis (CEA) tools
–Compare different
alternativesDefinition of criteria–Multi-criteria
analysis tools (MCA)
–Reference for the
application of other toolsVisioning (including uncertainty analysis)
Providing the future perspectives to problem
framingScenario tools
Quality assuranceProviding the context for and improve robustness of MCA, CBA and CEA
Support scenario buildingProblem framingParticipatory tools
Phase IVFollow-up
Phase IIIAnalysis
Phase IIFinding solutions
Phase IProblem analysis
The role of tools in assessments
Which tool to select within a tool group?
Dominance methodRegimeNAIADEPROMETHEEAHPWeighted summationMAVT
Mixed dataQualitative dataQuantitative dataNon-compensatory
Partial-compensatoryCompensatory
Type of dataDecision rule
Example: Multi-criteria analysis
• To support user to find a best suitable tool use different entry-points to find tool information:– Tool overview– Book of references– Tool search
• Base tool search on the following three questions:– Where are we in the policy process (which phase)?– What tasks are to be done in that (and the subsequent)
phase(s)?– Which tools can be deployed to support these tasks?
• Webbook address: www.SustainabilityA-Test.net
Summary and conclusions
