presented by presented by

Professor Ron JohnstonProfessor Ron Johnston

Australian Centre for InnovationAustralian Centre for Innovation

IPTS SeminarIPTS Seminar

17 October 200517 October 2005

SevillaSevilla

Evidence-based Policy Making:Evidence-based Policy Making:Some IssuesSome Issues

Recent HeadlinesRecent Headlines

““Big Brother not required by First Minister” –Big Brother not required by First Minister” –The ScotsmanThe Scotsman

Tender – foreign economic aid-related Tender – foreign economic aid-related services – EUR-OPservices – EUR-OP

““Check unethical health practices” – The Check unethical health practices” – The HinduHindu

““Sure Start – What a Waste” – Daily MailSure Start – What a Waste” – Daily Mail ““Danger – major quango ahead” – The Danger – major quango ahead” – The

ScotsmanScotsman

An Example – the Campbell An Example – the Campbell CollaborationCollaboration

The challenge – safety and efficacyThe challenge – safety and efficacy Disasters – Disasters – whole-language reading, driver education whole-language reading, driver education

at school, ‘scared straight’ justice, ‘3 strikes, you’re out’at school, ‘scared straight’ justice, ‘3 strikes, you’re out’

So, experiment first, make policy laterSo, experiment first, make policy later Their approach - systematic evaluative Their approach - systematic evaluative

review of policy interventions, selection of review of policy interventions, selection of most effective, and publication of most effective, and publication of authoritative guidanceauthoritative guidance

What is EBPM?What is EBPM?

An approach that helps people make well-An approach that helps people make well-informed decisions about policies, informed decisions about policies, programmes and projects by putting the programmes and projects by putting the best available evidence from research at the best available evidence from research at the heart of policy development and heart of policy development and implementation implementation (CERI/OECD, 2004)(CERI/OECD, 2004)

As opposed to OBPMAs opposed to OBPM

Relies heavily on either the selective use of Relies heavily on either the selective use of evidence or on the untested views of evidence or on the untested views of individuals or groups, often inspired by individuals or groups, often inspired by ideological standpoints, prejudices or ideological standpoints, prejudices or speculative conjecture speculative conjecture (CERI/OECD, 2004)(CERI/OECD, 2004)

Experience from HistoryExperience from History

Some ObservationsSome Observations

““Intelligence has nothing to do with politics” Intelligence has nothing to do with politics”

(Winston Churchill)(Winston Churchill) Policy-making is the art of looking for trouble, Policy-making is the art of looking for trouble,

finding it, mis-diagnosing it, and then applying the finding it, mis-diagnosing it, and then applying the wrong remedies wrong remedies

(Groucho Marx)(Groucho Marx) If you don’t know where you are going, any road If you don’t know where you are going, any road

will get you there will get you there

(Lewis Carroll)(Lewis Carroll)

Identification ofneeds and options

Policy formulation

Institutionalprocess

Policy deployment

Policy monitoringand evaluation

Prospective studies andexploration of policy options

Support to the conception anddevelopment of EU policiesEx-ante Impact Assessment

Supportto the introduction and

implementation of policiesSupport to O.M.C.

Support to policy revisionEx-post Impact Assessment

Support to thedecision-making process

Identification ofneeds and options

Policy formulation

Institutionalprocess

Policy deployment

Policy monitoringand evaluation

Prospective studies andexploration of policy options

Support to the conception anddevelopment of EU policiesEx-ante Impact Assessment

Supportto the introduction and

implementation of policiesSupport to O.M.C.

Support to policy revisionEx-post Impact Assessment

Support to thedecision-making process

But we are faced with different types But we are faced with different types of problemsof problems

Type 1 – responsibility for solving a problem rests Type 1 – responsibility for solving a problem rests solely with governmentsolely with government

Type 2 – responsibility for solving a problem rests Type 2 – responsibility for solving a problem rests with both the government and the governedwith both the government and the governed

Type 3 – no feasible solution to a problem exists, Type 3 – no feasible solution to a problem exists, so government and governed must work together so government and governed must work together to deal with a situation that neither can change, at to deal with a situation that neither can change, at least in the short term least in the short term

Why is EBPM Needed Now?Why is EBPM Needed Now?

Complexity, uncertainty, speed of changeComplexity, uncertainty, speed of change Better informed public with greater demandsBetter informed public with greater demands Key policy issues overlapKey policy issues overlap An inter-connected, inter-dependent worldAn inter-connected, inter-dependent world National issues become international issuesNational issues become international issues Growing demand on solutions that work across Growing demand on solutions that work across

boundariesboundaries Demand to bring about visible change in the real Demand to bring about visible change in the real

worldworld

EBPM is embedded in concept of professional EBPM is embedded in concept of professional policy makingpolicy making

Key competencies:Key competencies: Forward lookingForward looking Outward lookingOutward looking Flexible and innovativeFlexible and innovative Evidence-basedEvidence-based Involve key stakeholdersInvolve key stakeholders InclusiveInclusive Joined upJoined up Continuous improvementContinuous improvement Evaluation-based learningEvaluation-based learning

Key Issues – IKey Issues – IThe Nature of Policy-MakingThe Nature of Policy-Making

The Interface of Policy-making with PoliticsThe Interface of Policy-making with Politics Power/Influence versus RationalityPower/Influence versus Rationality The Power of MythsThe Power of Myths

Key Issues – IIKey Issues – IIConstraints on Policy-Maker’s Access to EvidenceConstraints on Policy-Maker’s Access to Evidence

Time demands preclude researchTime demands preclude research Information overloadInformation overload Information evaluationInformation evaluation Information gapsInformation gaps You don’t know what evidence you need You don’t know what evidence you need

or don’t haveor don’t have Where is the cross-cutting evidence for Where is the cross-cutting evidence for

cross-cutting policy?cross-cutting policy?

Key Issues – IIIKey Issues – IIIThe Nature of EvidenceThe Nature of Evidence

Information or knowledgeInformation or knowledge Codified or tacitCodified or tacit Expert or interest-basedExpert or interest-based ReliabilityReliability RelevanceRelevance TranslationTranslation FitFit

Key Issues – IVKey Issues – IVThe Role of EvidenceThe Role of Evidence

1.1. ‘‘Straightforward’ issues – get the factsStraightforward’ issues – get the facts

2.2. Complex issues – facts/expert judgments Complex issues – facts/expert judgments are always challengeableare always challengeable

3.3. Instrumental versus TransformativeInstrumental versus Transformative

4.4. Agenda settingAgenda setting

5.5. Providing a language of discourseProviding a language of discourse

6.6. Active learningActive learning

A Case StudyA Case Study

Desert KnowledgeDesert Knowledgein Central Australiain Central Australia

Implications for IPTSImplications for IPTS

Policy makers need to establish what they Policy makers need to establish what they will need to knowwill need to know

The challenge of joining up – structures for The challenge of joining up – structures for horizontal research (eg Canada)horizontal research (eg Canada)

Creating policy knowledge poolsCreating policy knowledge pools Establishing and applying good practiceEstablishing and applying good practice