Date post: | 26-Dec-2015 |
Category: |
Documents |
Upload: | elwin-mclaughlin |
View: | 215 times |
Download: | 0 times |
The Story of Washington’s Murder(according to Bill Sutherland)12th Dec 1799 Inspects plantation in snow13th Sore throat14th Dec 2:00 AM Breathing difficulties • Mr Rawlins, Estate overseer, provided mixture molasses, vinegar• Mr Rawlins ordered by Washington to remove 0.75 pint blood 10:00 AM Dr James Craik, personal physician, arrives• Preparation dried beetles applied to throat• 1 pint blood removed• No improvement so another 1 pint blood removed• Vinegar in water gargle – near suffocation• 2 more pints blood removed3:00 PM Dr Elisha Dick, prominent physician arrives• 1.8 pints blood removed• Pulse low10:10pm Died
Total > 6.5 pints removed
Pierre Charles Alexandre Louis(1787-1872)
• Introduced numerical methods to examine the effectiveness of medical interventions
• Showed blood letting ineffective
How Practitioners Make Decisions Myth Anecdote Observation Comparison Replicated RRC
AcademicNow
Practitioners
Future Academic
Practitioners
Documented
Source: Sutherland 2007
Measuring Effectiveness
An Overture
Measuring Conservation Effectiveness SummitMay 5-6, Palo Alto, California
Research On Over 220 MeasuresSystems in Different Fields
3542
55
30
56
0
10
20
30
40
50
60
Business &Mngmt
Education &Soc Serv
InternationalDvlpmt
Public Health& Pop
Envt &Conservation
Ap
pro
ac
he
s R
ev
iew
ed
Business & ManagementFamily Tree
1500
1900
1970
1980
1990
2000
EFFECTIVENESSACCOUNTING & CERTIFICATION
Balanced Scorecard
Cost Accounting
Activity Based Costing
Double-EntryBookkeeping
Engineering Standards
Auditing by Accountants
Process Certification
Scientific Mngmt
Total Quality Mngmt
Managing for Results
Six Sigma
Process Reengineering
Performance Benchmarking
Learning Organization
Community of Practice
Outcome Evaluation
Operations Research
Reflective Practice
Social Learning
STATUS ASSESSMENT
Stock Market Index
Economic Production
Econ Welfare Indicators
Outcome Evaluation
Accounting Standards
BenchmarkingChaordic Systems
Q1. What Is Measuring Effectiveness?
1500
1900
1970
1980
1990
2000
EFFECTIVENESSACCOUNTING & CERTIFICATION
STATUS ASSESSMENT
Status Question:How are Species and Ecosystems Doing?
?
Effectiveness QuestionAre Our Actions Leading to Desired Results?
?
Certification Question:Do We Meet External Standards?
?
Why We Are Focused on Effectiveness
EFFECTIVENESSACCOUNTING & CERTIFICATION
STATUS ASSESSMENT
??The Focus
for This Summit
Q2. What is the Unit for Measuring Effectiveness?
Actions
Threats
Conservation TargetWildlife and Habitat
Direct
Indirect
Costs of measuring change
Level of
confid
en
ce
Tim
e t
o s
ee a
n im
pact
Outputs
Outcomes
Impacts
Q3. How Do You Measure Effectiveness?
1500
1900
1970
1980
1990
2000
EFFECTIVENESSACCOUNTING & CERTIFICATION
STATUS ASSESSMENT
External Summative Evaluations
Participatory Formative
Evaluations
Project Cycle Based
Monitoring
Need to Integrate Measuring Effectivenessinto an Iterative Project Cycle
ImplementManagement &
MonitoringPlans
D
Develop aMonitoring
Plan
C
Develop aManagement Plan:Goals, Objectives,
& Activities
B
Start
Clarify Group'sMission
Iterate
Use Results toAdapt & Learn
EDevelop
ConceptualModel Basedon Local SiteConditions
A
AnalyzeData and
CommunicateResults
E
The AMProjectCycle
Check
DoAdapt
Plan
The Basic Systematic Performance Management (SPM) Cycle
Biodiversity ConservationProjects Come In All Shapes and Sizes
1. A community fishing ground in the Pacific2. A TNC preserve or provincial park3. Coordinated efforts to manage state and federal
funding for Puget Sound or Lake Ontario4. The WWF Markets Initiative5. A funder’s grantmaking strategy for the California
Coastal waters, or the Coral Triangle
CMP AWF CI TNC WCS WWFBiodiversity Targets
Focal Targets
Conservation Outcomes
Focal Conservation Targets
Landscape Species
Long-Term Goals
Threats Threats Pressures Threats Threats Threats
Objectives Milestones Objectives Targets Project Targets
An Excerpt from the CMP’s Rosetta Stone Analysis
Similar Concepts, Different Terms
Conservation Measures Partnership’s Open Standards Are One Flavor of SPM
• Developed by leading orgs & agencies
• Draws on many fields• Open source &
common language• Used around the world
Q4. How Much Should We Invest inMeasuring Effectiveness?
SPM does not specifyM&E Design !!
The “Burden of Proof” Depends on Risks,Costs, and Timeframe for Decision Making
Generally Invest More in Measures When:• Stakes are high (high cost of error or inaction)• Potential to leverage learning• Costs of measures are low relative to actions
MarislaFoundation
PackardFoundation
Q5. Should We Mandate Measuring Effectiveness?The Bell Curve: Treating Cystic Fibrosis
A Surprise – The Best Get BetterIt’s the centers in the top quartile that are improving fastest….they are at risk of
breaking away. What the best may have, above all, is a capacity to learn and adapt – and to do so faster than everyone else.
The Bell Curve Leads to Uncomfortable Questions Will being in the bottom half be used against doctors in lawsuits? Will we be
expected to tell our patients how we score? Will our patients leave us? Will those at the bottom be paid less than those at the top? The answer to all these questions is likely yes.
Poor
BelowAvg
AboveAvg
Best
Distribution of Cystic FibrosisTreatment Center Success
Rates
Ultimately We Are Not Competing Within Conservation, But Against Other Social Causes
ConservationWorthy Cause A Worthy Cause B
Q7. How Do We Roll Out Measuring Effectiveness Both Internally….
After a guy gave an ‘air ball’Six Sigma presentation…the standard joke
was that the guy “decided to leave” before his elevator reached the ground floor.
Q7. How Do We Roll Out Measuring Effectiveness Both Internally….and Externally?
If We Create a Culture of Data Sharing…
?
Q8. How Do We Continue this Conversation Going Forward?
Our Fundamental Hypothesis
-
0 %
100 %
- Failure
+ Success
+Re
so
urc
es f
or
Co
ns P
roje
cts
Our Fundamental Hypothesis
--
0 %
100 %
- Failure
+ Success
++
Re
so
urc
es f
or
Co
ns P
roje
cts
Investment inSPM
Measuring Effectiveness Can Transformthe Practice of Conservation
• Practitioners in field using measures results to plan and implement effective projects
• Managers and donors getting better information to improve work
• All stakeholders learning from each other• Increased public support for conservation• Ultimately, better conservation outcomes