1Challenge the future

International rankings of universities

An overview for managers M&C TU DelftKim Huijpen, Strategic Policy Affairs | 17/10/11

2Challenge the future

International rankings

1. Context2. Criticism3. Overview4. The position of the TU Delft5. New initiatives to improve rankings6. How do we use international rankings?7. Do we influence international rankings?

3Challenge the future

Context

Rankings fill in a need• Stakeholders – students, parents, governments,

accreditation councils, industry (inter)national organizations – want to know the differences between HEI’s and how they perform

Rankings are more and more used (directly or indirectly via reputation)• By the media• By governmental institutions (reallocation of funds)• By students (Asia)• By HEI’s themselves! For marketing purposes or to

select partners for cooperation

4Challenge the future

International rankings, criticism and new developments

Most important international rankings in 2011• QS-, THE-, Shanghai-, HEEACT-, Leiden-ranking• not 5 rankings, but 66 (11+6+7+17+25)

Criticism• content: bias for big & old universities, focus on research,

bias for natural & medical sciences, language bias, comparison of whole HEI’s

• methodology: adding up all kind of indicators, numbering, dubious weighting, intransparency, institutions deliver data, methodological changes

5Challenge the future

Criticism

Conceptual1. some universities have an advantage: Anglo-Saxon,

beta- and medical disciplines, focus on research, big, old, general

2. you can’t compare whole universities3. you can’t add up all the indicators

Methodology4. underpinning of the weight factors5. sensitivity for outliers: best HEI=100 (z-scores are

better)6. methodological changes in time

Data7. limited or no insight in the raw data8. data provided by HEI’s themselves: mistakes,

manipulation

6Challenge the future

Overview: similarities and differences

Ranking

Focus Indicators

Data Time Type

QS ResearchEducationInternat.

SubjectiveObjective

Own ResearchDbase (Scopus)Data HEI’s

Present GeneralField

THE ResearchEducationInternat.Income

SubjectiveObjective

Own ResearchDbase (WoS)Data HEI’s

Present GeneralField

Shangha

i

Research Objective Dbases (e.g. WoS, Nobel-prize.org)

PastPresent

GeneralFieldSubject

Leiden Research Objective Dbase (WoS) Present General

HEEACT Research Objective Dbase (WoS/ESI) Present GeneralFieldSubject

7Challenge the future

QS World University Rankings®

& Times Higher Education World University Rankings

• In October 2009, QS and THE ended their collaboration

• Now there are two rankings:

1. QS World University rankings (QS-ranking) with the methodology of the old ranking• Published in September 2011

2. A new ranking by the Times Higher Education Supplement with Thomson Reuters: Times Higher Education World University Rankings (THE-ranking)• Published in October 2011

8Challenge the future

Indicators QS ranking (old THE)

Fields:• ‘natural science’• ‘life sciences &

biomedicine’ ‘engineering & IT’

• ‘social sciences’• ‘arts and

humanities

Ranking by field: solely based on ‘academic peer review’

9Challenge the future

QS-ranking 2005-2011 (general)TU Delft from 108 (2010) to 104 (2011)

10Challenge the future

QS-ranking 2005-2011 (engineering)TU Delft stayed at position 18

11Challenge the future

QS-ranking 2005-2011 (natural sciences)TU Delft from 84 (2010) to 79 (2011)

12Challenge the future

THE-ranking (with Thomson Reuters)

Fields:• ‘Engineering &

Technology’• ‘Life Sciences’• ‘Clinical, pre-

clinical & Health’• ‘Physical

Science’• ‘Social Sciences’• ‘Arts &

Humanities’

Ranking by field: based on same 13 indicators with slightly different weighting

13Challenge the future

Position of 3TU’s, LDE and IDEA League in THE-ranking 2011

University general ranking – top 400 (2010)(the lower, the better)

TU Delft 104 (151)

TU Eindhoven 115 (114)

Universiteit Twente 200 (185)

Universiteit Leiden 79 (124)

Erasmus Universiteit Rotterdam

157 (159)

Imperial College London 8 (9)

ETH Zurich 15 (15)

Ecole Polytechnique* 63 (39)

Aachen RWTH 168 (182)

* ParisTech exists of eleven ‘Grandes Ecoles Paris’ of which Ecole Polytechnique is the most well known.

14Challenge the future

THE Engineering and Technology Universities 2011-2012 (Top 50)

• TU Delft rose from 33 (2010) to 22 (2011)• TU Delft is the only Dutch university in this ranking  Top 50 Engineering and Technology Universities 2011-

20121 California Institute of Technology, United States1 Massachusetts Institute of Technology, United States3 Princeton University, United States4 University of California, Berkeley, United States5 Stanford University, United States

9 ETH Zürich, Switzerland10 Imperial College London, United Kingdom     

22 Delft University of Technology, Netherlands29 École Polytechnique, France

15Challenge the future

Citation impact THE (30%) vsnew Crown Indicator CWTS

Position of Dutch universities: THE-ranking ’11 (blue) & new Crown Indicator CWTS ’10 (red)

16Challenge the future

Indicators Shanghai-ranking (since '03)

Focus Indicators Weighting

Quality of

education

Alumni winning Nobel

prizes and fields medals

Alumn

i

10%

Quality of faculty Staff winning Nobel prizes

and fields medals

Award 20%

Highly cited researchers HiCI 20%

Research output Articles and papers in

Nature and Science

N&S 20%

Articles and papers in SCI

and SSCI

PUB 20%

17Challenge the future

Shanghai-ranking 2011 (general)TU Delft: 151-200

Universiteit Academic Ranking of World Universities - 2011 (2010)

National rank 2011 (2010)

TU Delft 151-200 (151-200) 7-9 (7-9)

Universiteit Twente 301-400 (301-400) 11-12 (10-11)

TU Eindhoven 301-400 (401-500) 11-12 (12)

Universiteit Leiden 65 (70) 2 (2)

Erasmus Universiteit Rotterdam

151-200 (151-200) 7-9 (7-9)

ETH Zurich 23 (23) 1 (1)

Imperial College London 24 (26) 4 (4)

Ecole Polytechnique* 301-400 (201-300) 14-17 (8-13)

RWTH Aachen University 201-300 (201-300) 15-23 (15-23)* ParisTech exists of eleven ‘Grandes Ecoles Paris’ of which Ecole Polytechnique is the most well known.

18Challenge the future

Shanghai-ranking 2011Field Rankings

• The TU Delft and University of Twente are in only one of five Field Rankings (top 100) between 76-100: • ‘Engineering/Technology and Computer Sciences’

(Engineering, Computer Science en Materials Science)• Last year TU Eindhoven was in none of the Field

rankings• Now TU/e is on position 52-75 of

‘Engineering/Technology and Computer Sciences’• TU/e has one full professor in ISIHighlyCited.com • But:

• Thomson Reuters seems to stop with ISIHighlyCited.com

19Challenge the future

New developments

improvement of existing rankings• more attention for education, finance and field (THE)• more representative survey on reputation (QS, THE)• rankings per field and subject (Shanghai, HEEACT)

new rankings and classifications (education and third mission, fields and subjects, ranking per indicator, no numbering)• CHE university ranking: BSc-students (EWI/LR, TBM)• CHE excellence ranking: MSc/PhD-students (EWI/TNW)

• 2012: EWI• U-Map (CHEPS): types/profiles• U-Multirank (CHERPA/CHE): institutional and field*

rankings* e.g. engineering

20Challenge the future

1. Teaching & learning profile2. Student profile3. Research involvement4. Knowledge exchange5. International orientation6. Regional engagement

U-Map dimensions

Presentation F. van Vught on the 13th DAIR seminar (4-11-09)

21Challenge the future

Example European Classification of HEI’s (U-Map)

22Challenge the future

How do we use international rankings?

Until now• Participation in QS-/THE-/CHE-ranking and U-map• Internal memos for the Executive Board • Annual report • Website‘facts and figures’• Marketing and PR

23Challenge the future

“Proof of the pudding…?”

24Challenge the future

Reserved way of commenting

TU Delft 22ste in Times Top 50 Technology Universities• Collegevoorzitter Dirk Jan van den Berg is blij te zien

dat de TU Delft in vergelijking met universiteiten wereldwijd goed beoordeeld wordt. De algemene THE World University Ranking kijkt naar 13 indicatoren en probeert zo de universiteit in de volle breedte te beoordelen. “Daarmee is onze positie in de ranking een mooi compliment voor de onderzoekers en docenten.”

• [...] “Hoe mooi de stijging van de TU Delft en andere Nederlandse universiteiten ook is, het laat ook zien dat het goed is om de waarde van dergelijke ranglijsten te relativeren. Kleine wijzigingen in de methode hebben een grote invloed op de uitkomst.”

25Challenge the future

How do we choose a ranking for marketing purposes?

Ideology versus pragmatism

26Challenge the future

Should we influence our position in international rankings? No

No We hold a good position in the rankings which are most

important to us (THE-Technology, Shanghai-Engineering, Leiden- “Crown Indicator”)

We useAn elaborate internal planning and evaluation cycle providing

all kind of data. These data are related to the way the government finances universities (dissertations, students, diploma’s)

There are national reviews of our educational- and research-programmes which we use to improve our performance. These reviews include peer reviews and data on publications and citations

27Challenge the future

Should we influence our position in international rankings? Yes

Ways to influence rankings• Neglect some rankings (marketing, annual report)• Do not participate:

- no choice: Shanghai, HEEACT, Leiden- choice: QS, THE, CHE, U-Map/U-Multirank

• Make agreements with other universities about data delivering

• Make well considered decision about interpretation of definitions

• Check and influence databases used by rankings:- WoS (THE/Shanghai/HEEACT) and Scopus (QS)- WoS/Highly cited (Shanghai)- Recruiters (QS): e.g. 180 HEI’s gave 45.000 names

• Include ranking-indicators in internal P&E-cycle (choose, monitor and benchmark indicators)

28Challenge the future

However, it is difficult to influence the rankings

Your position is dependent upon• Your own performances• The performances of other HEI’s• The way you and other HEI’s report data to the

ranking institutions• Methodological changes introduced by the ranking

institutions

29Challenge the future

Messages

• More and more international rankings (need)• Are used by several stakeholders and affect your

reputation• Are biased and have methodological drawbacks

• However, methodologies are improving• Nevertheless, important to be in the rankings• It is difficult for specialized universities to reach a

high position in general rankings (TU Delft: technology/engineering)• However, field normalization is improving

• New initiatives to improve international rankings:• CHE rankings• European Classification and Multidimensional ranking

project

30Challenge the future

Questions and discussion:

Do we need a ‘ranking strategy’?

Which rankings do we choose for marketing purposes?

• More information: • www.3tu.nl/uploads/media/Rankings_en_3TU.pdf

• Thanks to Johan Verweij• I elaborated on his presentations

Kim Huijpen, Policy Advisor, TU Delft / Corporate Policy AffairsT +31 (0)15 27 85296 | E K.Huijpen@tudelft.nl | @KimHuijpen

31Challenge the future

References

Sheets 3, 4, 5, 6, 8, 16, 19, 20, 21, 22, 26, 27, 28 and 29 are from earlier presentations by Johan Verweij

Sheets 2 and 23 are based on sheets from earlier presentations by Johan Verweij

Websites rankings:• http://www.shanghairanking.com/ARWU2011.html

• http://www.topuniversities.com/university-rankings/world-university-rankings/2011

• http://www.timeshighereducation.co.uk/world-university-rankings/2011-2012/top-40

0.html

• http://ranking.heeact.edu.tw/en-us/2010/TOP/100

• http://www.socialsciences.leiden.edu/cwts/products-services/leiden-ranking-2010-cw

ts.html