Science and Technology Policy and Research Activity

Evaluation.

Prof. Chris Phillips, Experimental Solid State Group, Physics Dept., Imperial College London, UK.

Outline

Introduction/Overview 

Managed vs. Responsive Mode 

Anonymity vs. Transparency 

Categories vs. Interdisciplinarity 

Fundamental vs. applied

Handling Risk

Costing models 

Impact and Public Engagement  

Concluding Remarks.

Outline

Introduction/Overview 

Managed vs. Responsive Mode 

Anonymity vs. Transparency 

Categories vs. Interdisciplinarity 

Fundamental vs. applied

Handling Risk

Costing models 

Impact and Public Engagement  

Concluding Remarks.

The 'Seven Principles of Public Life’

Selflessness Holders of public office should act solely in terms of the public interest. They should not do so in order to gain financial or other benefits for themselves, their family or their friends.

Integrity Holders of public office should not place themselves under any financial or other obligation to outside individuals or organisations that might seek to influence them in the performance of their official duties.

Objectivity In carrying out public business, including making public appointments, awarding contracts, or recommending individuals for rewards and benefits, holders of public office should make choices on merit.

Accountability Holders of public office are accountable for their decisions and actions to the public and must submit themselves to whatever scrutiny is appropriate to their office.

Openness /Transparency Holders of public office should be as open as possible about all the decisions and actions that they take. They should give reasons for their decisions and restrict information only when the wider public interest clearly demands.

Honesty Holders of public office have a duty to declare any private interests relating to their public duties and to take steps to resolve any conflicts arising in a way that protects the public interest.

Leadership Holders of public office should promote and support these principles by leadership and example.

Outline

Introduction/Overview 

Managed vs. Responsive Mode 

Anonymity vs. Transparency 

Categories vs. Interdisciplinarity 

Fundamental vs. applied

Handling Risk

Costing models 

Impact and Public Engagement  

Concluding Remarks.

Managed vs. Responsive Mode

’

“Responsive mode”

Agenda set by the community themselves, democratically

Responds to new trends quickly.

Allows meaningful collaborations to grow organically

Difficult to get “Critical Mass” for technology intensive programmes

Can end up with an incoherent research portfolio in smaller communities.

Managed

Imposes coherence on the portfolio

Can make scientists move with the times

Who chooses themes? (International opinion sought?)

Can become very political, and result in outdated portfolio and poor value for money.

Outline

Introduction/Overview 

Managed vs. Responsive Mode 

Anonymity vs. Transparency 

Categories vs. Interdisciplinarity 

Fundamental vs. applied

Handling Risk

Costing models 

Impact and Public Engagement  

Concluding Remarks.

Anonymity vs. Transparency

Any anonymity decreases transparency

Negative effects can be partly reduced by careful record keeping (aliases/ code numbers etc.) so that at least abuses are traceable.

“Fully Open” Applicants and referees known to each other and everyone else.Corrodes professional relationships Inhibits expression of honest opinionVery rare (US?)

“Single Blind”Applicants are known but referees known only to the administrationCan be open to misusePractical to administerProbably the most common(UK)

“Double blind”Applicants and referees both anonymousDifficult to implement in small communitiesUsed in EU, possibly to negate political forces

Outline

Introduction/Overview 

Managed vs. Responsive Mode 

Anonymity vs. Transparency 

Categories vs. Interdisciplinarity 

Fundamental vs. applied

Handling Risk

Costing models 

Impact and Public Engagement  

Concluding Remarks.

  Categories vs. Categories vs.

InterdisciplinarityInterdisciplinarity

We can’t live without categories!We can’t live without categories!

Advances often happen when ideas get transferred from one sphere Advances often happen when ideas get transferred from one sphere to anotherto another

Administration needs a carefully thought out way of handling bids Administration needs a carefully thought out way of handling bids that span disciplinesthat span disciplines

KshitiJ 2010, KolkataKshitiJ 2010, Kolkata

NCSTE Kazakhstan April 2012NCSTE Kazakhstan April 2012

Outline

Introduction/Overview 

Managed vs. Responsive Mode 

Anonymity vs. Transparency 

Categories vs. Interdisciplinarity 

Fundamental vs. applied

Handling Risk

Costing models 

Impact and Public Engagement  

Concluding Remarks.

NCSTE Kazakhstan April 2012NCSTE Kazakhstan April 2012

Fundamental vs. applied

Some thoughts.....

Perhaps 3% (?) of “fundamental” research will lead to immense benefits for society.

In the past, no-one has been able to forsee which 3% this will be (e.g. Lasers, Transistors, Nuclear Magnetic Resonance).

Translation times are long ( 20 yr).

To get the 3% you have to pay for the other 97%.

If you fund only research that will be applied in two years, then in two years you’ll have no new ideas coming through.

Politicians/Industrialists like applied research (because they’re not here for long).

Political changes (e.g. Towards greater democracy) can put pressure on fundamental research funding.

Outline

Introduction/Overview 

Managed vs. Responsive Mode 

Anonymity vs. Transparency 

Categories vs. Interdisciplinarity 

Fundamental vs. applied

Handling Risk

Costing models 

Impact and Public Engagement  

Concluding Remarks.

Risk Vs. Reward

•They are inseparable!

Good projects ALWAYS carry risk.

Risk.

Rew

ard

.

Risk Vs. Reward

•They are inseparable!

•Rewards come in many forms. (Commercial, intellectual, cultural).

•Risk SHOULD be there, and referees need to be educated carefully.

•Risk should be managed (contingency plans, review points, intermediate deliverables).

•Unless the funding system accepts and embraces risk, its research portfolio will never reach international standards.

Good projects ALWAYS carry risk.

Outline

Introduction/Overview 

Managed vs. Responsive Mode 

Anonymity vs. Transparency 

Categories vs. Interdisciplinarity 

Fundamental vs. applied

Handling Risk

Costing models 

Impact and Public Engagement  

Concluding Remarks.

Costing models

•“Full Economic Costing” (FEC) aims to capture the total cost of a programme including e.g. Coast of providing and maintaining buildings, covered in “Research Overheads”

•Many bodies (e.g. UK charities, EU programmes) don’t pay full FEC rates, and boast of budgets they’ve “leveraged” from other sources.

•FEC costing certainly helps referees judge a proposal properly. Especially overseas referees.

•“Leveraged” programmes are often under resourced sometime on a small but critical area (“spoiling the ship for a haporth of tar”).

•Resources need to be justified and appropriate.

Outline

Introduction/Overview 

Managed vs. Responsive Mode 

Anonymity vs. Transparency 

Categories vs. Interdisciplinarity 

Fundamental vs. applied

Handling Risk

Costing models 

Impact and Public Engagement  

Concluding Remarks.

Impact and Public Engagement

•Most science, and much technology is funded publicly.

•We probably, at the least, have a moral duty to give as much back to them as possible.

•In a democracy, funding agencies have the much more practical imperative to document their impact, or the money is likely to dry up.

•“Impact” comes in many forms...........

•Commercial Impact?•Trained staff/students.•Scientific Impact/glory?•Cultural value/advances in knowledge?•Ca the discoveries impact on different fields/disciplines?•Will the discoveries lead to progress in areas identified by the government/funding body?•How will the general population (and politicians!) hear about the work? (media, TV, books etc).

Outline

Introduction/Overview 

Managed vs. Responsive Mode 

Anonymity vs. Transparency 

Categories vs. Interdisciplinarity 

Fundamental vs. applied

Handling Risk

Costing models 

Impact and Public Engagement  

Concluding Remarks.