Walking the tightrope: Pursuing real-world impacts from research in an academic environment

David PannellCentre for Environmental Economics and PolicySchool of Agricultural and Resource Economics

For this PPT see www.davidpannell.net under “Talks”

Balancing act

Real-worldimpact

Academicimpact

What matters?

Academic impact· Sophistication

· Originality/innovation

· Citations

· Academic reputation

· Evidence/rigour

Real-world impact· Simplicity

· Usefulness/relevance

· Opinions of trusted others

· Trust, credibility, relationships

· Clarity, persuasiveness

Academic impact

· ERA

· Assessed by established senior academics

· Usually value rigour over relevance

· Usually uni-disciplinary

· Journal quality (A*, A, B, C)

· Citations

· Academic reputation

Real-world impact

· Growing interest

· Perception: we need to do better at convincing government about benefits of research

· ARC discussing how to include real-world impact in ERA

· UK’s Research Excellence Framework: 20% of funding based on “impact” from 2014.

Trial by universities, 2012

· Group of Eight (Go8) and Aust Technology Network of Universities (ATN)

· Each university submitted cherry-picked case studies (165 submissions)

· Evaluated by people from industry & government

· 24 ‘best’ selected

Example – research project

2000: Salinity was a hot topic

$1.4 billion of public funding

I was shocked

· Poor design of the program

· Program developers seemed to have been unaware of crucial areas of salinity research and their implications

· No chance of any significant benefits

My response· Media

· Discussion papers

· Presentations

· Submissions

Tried to help them

· Developed INFFER (Investment Framework for Environmental Resources)A tool for integrating the science with other infoDevelop logical, evidence-based environmental

projectsAssess value for moneyPrioritise projects

Strategy· Extensive input by users

· Make tools as simple as possible

· Provide training and help desk for users

· Clear documentation aimed at non-experts

· Public critiques of existing approaches

· Attempt to influence gov’t agencies to change the signals

Regional NRM application

International application

Policy impacts

· Senate inquiry (2006)Recommended use of INFFER

· NRM Ministerial Council (2007)Endorsed new set of principles for investment in

salinity

· Victorian Government, Biodiversity White Paper “INFFER will be utilised for the next five years”.

· Caring for our Country Influenced design of project template

Example – blog

“Pannell Discussions”

· Started in 2004

· Theme: environmental economics, agricultural economics, policy, etc.

· 250 posts so far

· Each is a mini-discussion paper (500-1000 words)

· Often references my own research

· About one every two weeks

“Pannell Discussions”

· Subscribers receive notification of new posts

· 640 subscribers

· New posts are tweeted (130 followers)

· Popular posts get about 1000 readers

Real-world impacts

· Less obvious than for INFFER

· Readers have a better understanding of economics than they would have

· Some reduction in confusion, misconceptions, prejudices

· Greater awareness of specific tools & concepts

· Increased profile for myself and UWA in the community

Costs?

· It takes time INFFER: lots!Pannell Discussions: about 1 hour per week

· Some academics might not consider these endeavours to be very academically respectable

· Some aspects are difficult, stressful, frustrating

· Is it worth it?Real world – definitely yesAcademically – yes, but …

Academic benefits (sample of 1!)· Journal papers generated

Directly part of the INFFER work: 17Related/stimulated by: 16

· But, different sorts of papersResponding to identified real-world needsCan be out of left field relative to the existing lit More inter-disciplinary papersMore synthesis/commentary type of papersMostly, it’s not the type of research that gets into

the most prestigious disciplinary journals

Academic benefits

· CitationsThese papers get relatively well citedOne INFFER-related paper is the most cited paper

over the last 8 years in Land Economics (one of the leading international journals in my field)

Academic benefits

· Prizes/awards

· INFFEREureka Prize for

Interdisciplinary ResearchAARES Quality of Research Discovery Award

· Pannell DiscussionsAARES Quality of Research Communication Award

· GeneralPerhaps made some contribution to my Federation

Fellowship

Academic benefits

· Opportunities generated

· Invited onto steering committee of major EU project, thanks to blog

· Reputation for useful research easier to get funding (unsolicited approaches offering $)

Would I recommend it?

· Not for everyone

· Need to Get a buzz out of making a differenceHave strong communication skillsEnjoy the various challenges Be prepared/able to make the timeNot be too obsessed with academic prestige

· If project pushes for change, need toEnjoy learning about how things work in the real

worldBe resilient and persistent

Resilience/persistence needed

· People will suspect your motives

· People with a vested interest in the status quo will attack you

· People will misunderstand, misinterpret, and totally misrepresent what you are saying

· Nobody reads more than a page

· Nobody knows about your discipline

· Everybody is too rushed to do things properly

Resilience/persistence needed

· Everybody thinks they are doing a good job, even if they clearly aren’t

· People think evidence and analysis is optional

· It reduces their flexibility for decision making

· People will pursue objectives you think are inappropriate

· Significant change takes years

· You have to repeat yourself ad nauseam

Resilience/persistence needed

· You’ll see the same mistakes made repeatedly

· If you succeed, it could be more because of relationships and trust than the quality of your evidence or logic

· Even if you convince some people in the system of your position, people higher up who know absolutely nothing about it will over-rule them

· The person you’ve been cultivating will change jobs

Implications for universities

· Don’t expect everyone to do it

· Some high achievers for real-world impact might not be academic high achievers

· Perhaps an initiative to free up some time for selected people

· Don’t set rigid requirements for “quality” based only on academic criteria

Implications for disciplines

· Broaden perspective on what constitutes quality

· Broaden who judges

· Avoid rigour-mortis

· Be open to multi-disciplinary work

Resources

· Pannell, D.J. and Roberts, A.M. (2009). Conducting and delivering integrated research to influence land-use policy: salinity policy in Australia, Environmental Science and Policy 12(8), 1088-1099.http://dpannell.fnas.uwa.edu.au/dp0803.htm

· Pannell, D.J. (2004). Effectively communicating economics to policy makers. Australian Journal of Agricultural and Resource Economics 48(3), 535-555. http://dpannell.fnas.uwa.edu.au/j78ajare.pdf

Resources

· Weible et al. (2012). “Understanding and influencing the policy process”, Policy Science 45, 1-12. http://link.springer.com/article/10.1007%2Fs11077-

011-9143-5

Pannell Discussions (Blog posts)· 150 – Why don’t environmental managers use

decision theory?http://www.pannelldiscussions.net/2009/04/150-wh

y-dont-environmental-managers-use-decision-theory/

· 136 – Engaging with policy: tips for researchershttp://www.pannelldiscussions.net/2008/09/136-eng

aging-with-policy-tips-for-researchers/

Resources

· A relevant blog post by ecologist Brian McGill on “What it takes to do policy-relevant science” http://dynamicecology.wordpress.com/2013/05/14/

what-it-takes-to-do-policy-relevant-science/

· Video: Ben Martin (U Sussex) “Science Policy Research - Can Research Influence Policy? How? And Does It Make for Better Policy?”http://upload.sms.csx.cam.ac.uk/media/747324

For this PPT see www.davidpannell.net under “Talks”

Other needs (for policy impact)· Need some demand pull

· Seek a product champion

· Understand potential users

· Understand the chain from research to impact for your issue

A chain from research to impact: Information for policy· Research

· Something useful is learned (or isn’t)

· New information influences policy (or doesn’t)

· Policy change is implemented (or isn’t)

· If policy aims to change behaviour, people respond as intended (or don’t)

· Changes (relative to no research) result – social, environmental or economic benefits (or not)

Other needs (for policy impact)· Need “absorptive capacity” in the organisation

· The political circumstances need to be right. You can’t change ideological positions of govt.

· Timing. Grasp opportunities.

· Good communicationSimplicity, brevity, clarityAvoid jargon, maths, complex graphs