+ All Categories
Home > Documents > Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of...

Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of...

Date post: 31-Mar-2015
Category:
Upload: dale-holway
View: 213 times
Download: 1 times
Share this document with a friend
21
Research Assessment and Research Assessment and Funding: the experience Funding: the experience of other countries of other countries Dr Lisa Lucas Graduate School of Education University of Bristol
Transcript
Page 1: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

Research Assessment and Research Assessment and Funding: the experience of other Funding: the experience of other

countriescountries

Dr Lisa Lucas

Graduate School of Education

University of Bristol

Page 2: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

Change in RAE gradingsChange in RAE gradings% of top grades (5 & 5*) from 1992 to % of top grades (5 & 5*) from 1992 to

2001 by subject area2001 by subject area

22%(18)

54%(41)

9%11%(9)

46%(41)

10%15%(10)

38%(18)

9%

0%

10%

20%

30%

40%

50%

60%

RAE 1992 RAE 1996 RAE 2001

Biology

English

Sociology

Page 3: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

The Research Game in Academic Life The Research Game in Academic Life ((Lucas, 2006Lucas, 2006))

Intensification of the management and organisation of research activities

Differentiation of academics within departments over status/workloads

Struggles over classification of research active/research inactive

Lack of value perceived for teaching and associated work

Prioritising of research areas that can attract high levels of research funding and would be worthy of submission to high ranking journals

Page 4: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

UK University Core Funding for Research UK University Core Funding for Research and Teaching 2005-6and Teaching 2005-6

Source: Higher Education Funding Council (HEFCE)Source: Higher Education Funding Council (HEFCE)

Institution Core Funds

Teaching (£M)

Core Funds

Research (£M)

Ratio

(% research)

Cambridge 57.36 92.37 62%

UCL 58.05 92.98 62%

Oxford 57.72 90.16 61%

Manchester 83.38 68.93 45%

Bristol 53.22 37.86 42%

Kingston 45.44 0.81 2%

Coventry 36.54 0.53 1%

Page 5: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

Focus of PresentationFocus of Presentation

What can be learned from other national systems of funding and evaluating research in universities that utilise metrics?

Can other national systems provide alternative ideas to inform the development of the UK system of funding and evaluating research?

Page 6: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

University State Funding and the Higher University State Funding and the Higher Education Landscape in the 21Education Landscape in the 21stst Century Century

Reduction in State funding to universities– Resource Dependency Theory (Slaughter & Leslie,

1997)

Global Competition– Creating ‘World Class’ Universities (global

league tables)

Globalisation and Higher Education– Convergence of Higher Education Policies?

Page 7: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

Measuring the Scientific Impact of Measuring the Scientific Impact of Nations?Nations?

Rank Country Top 1% of highly cited publications

1997-2001

1 US 23,723

2 EU15 14,099

3 UK 4,831

7 Canada 2,195

10 Netherlands 1,435

11 Australia 1,049

20 China 375

Source: King, D.A. (2004) The Scientific Impact of Nations, Nature,

Vol 430, 15th July.

Page 8: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

National Case Studies of Research National Case Studies of Research Funding and EvaluationFunding and Evaluation

Hong KongThe NetherlandsAustralia and New Zealand

Page 9: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

Higher Education System in Hong KongHigher Education System in Hong Kong

Eight institutions Funded by the University Grants Committee

(UGC)“As far as the UGC is concerned, the UGC advocates

role differentiation among institutions, and the concentration of resources to reward performance

and encourage the growth of centres of excellence.” (UGC, Facts and Figures, 2002)

Page 10: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

Hong Kong RAE 2006Hong Kong RAE 2006

Based originally on the UK RAESingle quality threshold at attainable level

of excellence, no gradingCarnegie Classifications:

– scholarship of discovery – Scholarship of integration – Scholarship of application – Scholarship of teaching

Page 11: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

Hong Kong System: key issuesHong Kong System: key issues

Single quality threshold – no differentiation between research ‘active’ and ‘inactive’ staff.

Allowing for multiple dimensions of excellence by broadening the definition of ‘research’ to forms of scholarship.

BUT– Role differentiation amongst institutions mandated.– Greater compulsion for EVERYONE to do research.– Multiple dimensions of evaluation less easy in practice.– Pressure to publish ‘internationally’.

Page 12: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

Higher Education System in the Higher Education System in the NetherlandsNetherlands

“The universities prepare students for independent scientific work in an academic or professional setting and the hogescholen prepares students to practise a profession

and enable them to function self-consciously in society at large.”

13 universities and 50 hogescholen– Source: Boezerooy, P (2003) Higher Education in the

Netherlands: Country Report, CHEPS

Page 13: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

Research Funding and Evaluation in the Research Funding and Evaluation in the Netherlands Netherlands (VSNU)(VSNU)

Universities are funded for teaching and research as a block grant based primarily on historical circumstances

Development of Research Institute and Schools Structure (separate within Faculties)

Research (mainly self-evaluation) based on:– Quality– Productivity– Relevance– Vitality and Feasibility

Page 14: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

The Dutch System: key issuesThe Dutch System: key issues

Funding not linked to evaluation of research work Self-evaluation of research Research School/Institute structure means that

academics can be either research or teaching only BUT

– Binary system/small university sector– Institutions use evaluation to manage, control and direct

research activities– Possible separation of teaching and research staff within

faculties but also more collaboration of staff across institutions

Page 15: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

Research Funding and Evaluation in Research Funding and Evaluation in Australia (2008?)Australia (2008?)

40 universities Imminent change to the Research Quality

Framework (RQF) – combining quantitative and qualitative evaluation

On going debates on the formation of the RQF Concern with ‘basket of metric indicators’ –

metrics working group Concern with Impact of research (social, economic,

environmental) – impact working group

Page 16: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

Distribution of publication output by field, Distribution of publication output by field, Australian universities, 1999-2001 Australian universities, 1999-2001 (Butler, 2006)(Butler, 2006)

Field Books Book chapter

Journal article

Conf. paper

Band 1 Chem. Sc.

0.2 2.1 95.7 1.9

Band 2 Psych. 1.5 17.4 76.2 4.9

Band 3 Econ. 2.9 24.5 64.5 8.0

Band 4 Politics 5.8 37.3 46.1 10.8

Page 17: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

Research Funding and Evaluation in Research Funding and Evaluation in New Zealand (2006, 2012)New Zealand (2006, 2012)

Performance-based Research Funding (PBRF) – quantitative and qualitative indicators– Quality of researchers (60%)– Reflect research degree completions (25%)– Reflect external income (15%)

45 institutions Some Key Aims of the PBRF

– Ensure that research continues to support degree and postgraduate teaching

– Prevent undue concentration of funding that would undermine research support for all degrees

Page 18: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

What can be learned from other national What can be learned from other national systems that utilise metrics?systems that utilise metrics?

Quantitative measures should inform qualitative judgement not replace it.

A ‘basket of metrics’ needs to be utilised rather than simplistic single indicators.

Perverse research practices can be encouraged where simplistic measures are utilised (Butler, 2003).

Page 19: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

Can other national systems provide Can other national systems provide alternative ideas?alternative ideas?

Widening the definition of research and encouraging the integration of research and teaching

Encouraging greater collaboration across the sector to build capacity

Research for whom? Impact and communication of research Importance of combined quantitative and qualitative

indicators and variety of indicators to be utilised Indicator utilised that are appropriate for different subject

areas Holistic approach – concern to support the whole higher

education sector

Page 20: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

Dr Martin Dr Martin (Biology, Golden County University)(Biology, Golden County University)

“I would rather see all the vice chancellors lined up for a hundred yards dash and just assign money

on that basis, because that exercise would take at the most two minutes. Even the weakest vice chancellor could do a hundred yards in two

minutes and then get on with life. It is about as rational as that. At least you could train your

vice chancellor and pick a healthy one. At least you would have a use for a vice chancellor at long last, you’d be able to select on a rational basis. It might televise well and you might get money from the rights on watching it. And you

might get rid of a few each time.”

Page 21: Research Assessment and Funding: the experience of other countries Dr Lisa Lucas Graduate School of Education University of Bristol.

ReferencesReferences

Butler, L. (2003) Explaining Australia’s increased share of ISI publications – the effects of funding formula based on publication counts, Research Policy, 32: 143-155.

Butler, L. (2006) Research Assessment: moving beyond journal outputs, SPRU 40th Anniversary Conference – The Future of Science, Technology and Innovation Policy, University of Sussex, 11-13 September.

DEST, Research Quality Framework, Australia http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_issues/research_quality_framework/default.htm

King, D.A. (2004) The Scientific Impact of Nations, Nature, 430 (15th July): 311-6. Lucas, L. (2006) The Research Game in Academic Life, Maidenhead: SRHE/Open

University Press Slaughter, S. & Leslie, L. (1997) Academic Capitalism: politics, policies and the

entrepreneurial university, Baltimore: The John Hopkins University Press. Tertiary Education Comission, Performance-based Research Fund, New Zealand,

http://www.tec.govt.nz/funding/research/pbrf/pbrf.htm UGC (2002) Facts and Figures, Hong Kong SAR.


Recommended