International rankings of universities; An overview for managers M&C TU Delft

transcript

1Challenge the future

International rankings of universities

An overview for managers M&C TU DelftKim Huijpen, Strategic Policy Affairs | 17/10/11

International rankings

1. Context2. Criticism3. Overview4. The position of the TU Delft5. New initiatives to improve rankings6. How do we use international rankings?7. Do we influence international rankings?

Context

Rankings fill in a need• Stakeholders – students, parents, governments,

accreditation councils, industry (inter)national organizations – want to know the differences between HEI’s and how they perform

Rankings are more and more used (directly or indirectly via reputation)• By the media• By governmental institutions (reallocation of funds)• By students (Asia)• By HEI’s themselves! For marketing purposes or to

select partners for cooperation

International rankings, criticism and new developments

Most important international rankings in 2011• QS-, THE-, Shanghai-, HEEACT-, Leiden-ranking• not 5 rankings, but 66 (11+6+7+17+25)

Criticism• content: bias for big & old universities, focus on research,

bias for natural & medical sciences, language bias, comparison of whole HEI’s

• methodology: adding up all kind of indicators, numbering, dubious weighting, intransparency, institutions deliver data, methodological changes

Criticism

Conceptual1. some universities have an advantage: Anglo-Saxon,

beta- and medical disciplines, focus on research, big, old, general

2. you can’t compare whole universities3. you can’t add up all the indicators

Methodology4. underpinning of the weight factors5. sensitivity for outliers: best HEI=100 (z-scores are

better)6. methodological changes in time

Data7. limited or no insight in the raw data8. data provided by HEI’s themselves: mistakes,

manipulation

Overview: similarities and differences

Ranking

Focus Indicators

Data Time Type

QS ResearchEducationInternat.

SubjectiveObjective

Own ResearchDbase (Scopus)Data HEI’s

Present GeneralField

THE ResearchEducationInternat.Income

SubjectiveObjective

Own ResearchDbase (WoS)Data HEI’s

Present GeneralField

Shangha

Research Objective Dbases (e.g. WoS, Nobel-prize.org)

PastPresent

GeneralFieldSubject

Leiden Research Objective Dbase (WoS) Present General

HEEACT Research Objective Dbase (WoS/ESI) Present GeneralFieldSubject

QS World University Rankings®

& Times Higher Education World University Rankings

• In October 2009, QS and THE ended their collaboration

• Now there are two rankings:

1. QS World University rankings (QS-ranking) with the methodology of the old ranking• Published in September 2011

2. A new ranking by the Times Higher Education Supplement with Thomson Reuters: Times Higher Education World University Rankings (THE-ranking)• Published in October 2011

Indicators QS ranking (old THE)

Fields:• ‘natural science’• ‘life sciences &

biomedicine’ ‘engineering & IT’

• ‘social sciences’• ‘arts and

humanities

Ranking by field: solely based on ‘academic peer review’

QS-ranking 2005-2011 (general)TU Delft from 108 (2010) to 104 (2011)

QS-ranking 2005-2011 (engineering)TU Delft stayed at position 18

QS-ranking 2005-2011 (natural sciences)TU Delft from 84 (2010) to 79 (2011)

THE-ranking (with Thomson Reuters)

Fields:• ‘Engineering &

Technology’• ‘Life Sciences’• ‘Clinical, pre-

clinical & Health’• ‘Physical

Science’• ‘Social Sciences’• ‘Arts &

Humanities’

Ranking by field: based on same 13 indicators with slightly different weighting

Position of 3TU’s, LDE and IDEA League in THE-ranking 2011

University general ranking – top 400 (2010)(the lower, the better)

TU Delft 104 (151)

TU Eindhoven 115 (114)

Universiteit Twente 200 (185)

Universiteit Leiden 79 (124)

Erasmus Universiteit Rotterdam

157 (159)

Imperial College London 8 (9)

ETH Zurich 15 (15)

Ecole Polytechnique* 63 (39)

Aachen RWTH 168 (182)

* ParisTech exists of eleven ‘Grandes Ecoles Paris’ of which Ecole Polytechnique is the most well known.

THE Engineering and Technology Universities 2011-2012 (Top 50)

• TU Delft rose from 33 (2010) to 22 (2011)• TU Delft is the only Dutch university in this ranking Top 50 Engineering and Technology Universities 2011-

20121 California Institute of Technology, United States1 Massachusetts Institute of Technology, United States3 Princeton University, United States4 University of California, Berkeley, United States5 Stanford University, United States

9 ETH Zürich, Switzerland10 Imperial College London, United Kingdom

22 Delft University of Technology, Netherlands29 École Polytechnique, France

Citation impact THE (30%) vsnew Crown Indicator CWTS

Position of Dutch universities: THE-ranking ’11 (blue) & new Crown Indicator CWTS ’10 (red)

Indicators Shanghai-ranking (since '03)

Focus Indicators Weighting

Quality of

education

Alumni winning Nobel

prizes and fields medals

Quality of faculty Staff winning Nobel prizes

and fields medals

Award 20%

Highly cited researchers HiCI 20%

Research output Articles and papers in

Nature and Science

N&S 20%

Articles and papers in SCI

and SSCI

PUB 20%

Shanghai-ranking 2011 (general)TU Delft: 151-200

Universiteit Academic Ranking of World Universities - 2011 (2010)

National rank 2011 (2010)

TU Delft 151-200 (151-200) 7-9 (7-9)

Universiteit Twente 301-400 (301-400) 11-12 (10-11)

TU Eindhoven 301-400 (401-500) 11-12 (12)

Universiteit Leiden 65 (70) 2 (2)

Erasmus Universiteit Rotterdam

151-200 (151-200) 7-9 (7-9)

ETH Zurich 23 (23) 1 (1)

Imperial College London 24 (26) 4 (4)

Ecole Polytechnique* 301-400 (201-300) 14-17 (8-13)

RWTH Aachen University 201-300 (201-300) 15-23 (15-23)* ParisTech exists of eleven ‘Grandes Ecoles Paris’ of which Ecole Polytechnique is the most well known.

Shanghai-ranking 2011Field Rankings

• The TU Delft and University of Twente are in only one of five Field Rankings (top 100) between 76-100: • ‘Engineering/Technology and Computer Sciences’

(Engineering, Computer Science en Materials Science)• Last year TU Eindhoven was in none of the Field

rankings• Now TU/e is on position 52-75 of

‘Engineering/Technology and Computer Sciences’• TU/e has one full professor in ISIHighlyCited.com • But:

• Thomson Reuters seems to stop with ISIHighlyCited.com

New developments

improvement of existing rankings• more attention for education, finance and field (THE)• more representative survey on reputation (QS, THE)• rankings per field and subject (Shanghai, HEEACT)

new rankings and classifications (education and third mission, fields and subjects, ranking per indicator, no numbering)• CHE university ranking: BSc-students (EWI/LR, TBM)• CHE excellence ranking: MSc/PhD-students (EWI/TNW)

• 2012: EWI• U-Map (CHEPS): types/profiles• U-Multirank (CHERPA/CHE): institutional and field*

rankings* e.g. engineering

1. Teaching & learning profile2. Student profile3. Research involvement4. Knowledge exchange5. International orientation6. Regional engagement

U-Map dimensions

Presentation F. van Vught on the 13th DAIR seminar (4-11-09)

Example European Classification of HEI’s (U-Map)

How do we use international rankings?

Until now• Participation in QS-/THE-/CHE-ranking and U-map• Internal memos for the Executive Board • Annual report • Website‘facts and figures’• Marketing and PR

“Proof of the pudding…?”

Reserved way of commenting

TU Delft 22ste in Times Top 50 Technology Universities• Collegevoorzitter Dirk Jan van den Berg is blij te zien

dat de TU Delft in vergelijking met universiteiten wereldwijd goed beoordeeld wordt. De algemene THE World University Ranking kijkt naar 13 indicatoren en probeert zo de universiteit in de volle breedte te beoordelen. “Daarmee is onze positie in de ranking een mooi compliment voor de onderzoekers en docenten.”

• [...] “Hoe mooi de stijging van de TU Delft en andere Nederlandse universiteiten ook is, het laat ook zien dat het goed is om de waarde van dergelijke ranglijsten te relativeren. Kleine wijzigingen in de methode hebben een grote invloed op de uitkomst.”

How do we choose a ranking for marketing purposes?

Ideology versus pragmatism

Should we influence our position in international rankings? No

No We hold a good position in the rankings which are most

important to us (THE-Technology, Shanghai-Engineering, Leiden- “Crown Indicator”)

We useAn elaborate internal planning and evaluation cycle providing

all kind of data. These data are related to the way the government finances universities (dissertations, students, diploma’s)

There are national reviews of our educational- and research-programmes which we use to improve our performance. These reviews include peer reviews and data on publications and citations

Should we influence our position in international rankings? Yes

Ways to influence rankings• Neglect some rankings (marketing, annual report)• Do not participate:

- no choice: Shanghai, HEEACT, Leiden- choice: QS, THE, CHE, U-Map/U-Multirank

• Make agreements with other universities about data delivering

• Make well considered decision about interpretation of definitions

• Check and influence databases used by rankings:- WoS (THE/Shanghai/HEEACT) and Scopus (QS)- WoS/Highly cited (Shanghai)- Recruiters (QS): e.g. 180 HEI’s gave 45.000 names

• Include ranking-indicators in internal P&E-cycle (choose, monitor and benchmark indicators)

However, it is difficult to influence the rankings

Your position is dependent upon• Your own performances• The performances of other HEI’s• The way you and other HEI’s report data to the

ranking institutions• Methodological changes introduced by the ranking

institutions

Messages

• More and more international rankings (need)• Are used by several stakeholders and affect your

reputation• Are biased and have methodological drawbacks

• However, methodologies are improving• Nevertheless, important to be in the rankings• It is difficult for specialized universities to reach a

high position in general rankings (TU Delft: technology/engineering)• However, field normalization is improving

• New initiatives to improve international rankings:• CHE rankings• European Classification and Multidimensional ranking

project

Questions and discussion:

Do we need a ‘ranking strategy’?

Which rankings do we choose for marketing purposes?

• More information: • www.3tu.nl/uploads/media/Rankings_en_3TU.pdf

• Thanks to Johan Verweij• I elaborated on his presentations

Kim Huijpen, Policy Advisor, TU Delft / Corporate Policy AffairsT +31 (0)15 27 85296 | E K.Huijpen@tudelft.nl | @KimHuijpen

References

Sheets 3, 4, 5, 6, 8, 16, 19, 20, 21, 22, 26, 27, 28 and 29 are from earlier presentations by Johan Verweij

Sheets 2 and 23 are based on sheets from earlier presentations by Johan Verweij

Websites rankings:• http://www.shanghairanking.com/ARWU2011.html

• http://www.topuniversities.com/university-rankings/world-university-rankings/2011

• http://www.timeshighereducation.co.uk/world-university-rankings/2011-2012/top-40

0.html

• http://ranking.heeact.edu.tw/en-us/2010/TOP/100

• http://www.socialsciences.leiden.edu/cwts/products-services/leiden-ranking-2010-cw

ts.html