Date post: | 07-May-2015 |
Category: |
Education |
Upload: | giuseppe-de-nicolao |
View: | 1,459 times |
Download: | 3 times |
Quell’oscuro ogge,o del desiderio: le classifiche internazionali
degli atenei
Giuseppe De Nicolao Università di Pavia
SOMMARIO Prologo: “mamma li Turchi!”
1. Il grande balzo
2. Dall’EgiBo con furore
3. Should you believe in the Shanghai ranking?
4. Numb3rs!
5. Il potere dei numeri
6. Dove sNamo andando?
Prologo Mamma li Turchi!
«Siamo agli ultimi posti nelle classifiche mondiali. Per questo motivo presenteremo a novembre la riforma dell’Università, [...] Mi auguro di non dover più vedere in futuro - conclude - la prima università italiana al 174mo posto»
Capitolo 1 Il grande balzo
27 Aprile 2014
E’ stato pubblicato il World Reputation Rankings 2014, la classifica delle università mondiali elaborata dalla rivista inglese Times Higher Education.
L’Università di Pavia ottiene un ottimo posizionamento, classificandosi quinta tra le 15 università italiane presenti, guadagnando 76 posizioni rispetto al 2013.
La classifica è, come di consueto, dominata dalle università anglosassoni che monopolizzano anche la top ten con l’autorevole Harvard salda al primo posto forte di una percentuale del 100% dei consensi espressi dagli accademici interpellati.
... con l’autorevole Harvard salda al primo posto forte di una percentuale del 100% dei consensi espressi dagli accademici interpellati.
... con l’autorevole Harvard salda al primo posto forte di una percentuale del 100% dei consensi espressi dagli accademici interpellati.
Gli americani so’ forti… Ammazza gli americani, aoh! Non puoi mica combattere contro gli americani!
... con l’autorevole Harvard salda al primo posto forte di una percentuale del 100% dei consensi espressi dagli accademici interpellati.
... con l’autorevole Harvard salda al primo posto forte di una percentuale del 100% dei consensi espressi dagli accademici interpellati.
Gli americani so’ forti… ma non fino al 100% dei consensi (quelli sono i Bulgari)
The scores are based on the number of times an institution is cited by respondents as being the best in their field. The number one institution, Harvard University, was selected most often. The scores for all other institutions in the table are expressed as a percentage of Harvard’s, set at 100.
E’ stato pubblicato il World Reputation Rankings 2014, la classifica delle università mondiali elaborata dalla rivista inglese Times Higher Education.
L’Università di Pavia ottiene un ottimo posizionamento, classificandosi quinta tra le 15 università italiane presenti, guadagnando 76 posizioni rispetto al 2013.
March 6, 2014
100 università nessuna italiana
March 6, 2014
100 università nessuna italiana
?
... andiamo a guardare il video su YouTube ...
World University Rankings
sono due classifiche diverse
100 università nessuna italiana
400 università 15 italiane
Mistero spiegato?
Sì, ma ...
“Non capisco il miscuglio informa1vo fa3o dai vostri Top Managers”
October 3, 2013
March 6, 2014
sono due classifiche diverse
pubblicate in date diverse
October 3, 2013
March 6, 2014
sono due classifiche diverse
pubblicate in date diverse
E allora consoliamoci con qualche dolcetto che ci riserva, comunque, questa edizione. Abbiamo 15 Università nel “girone di serie B”, quello compreso fra il 200° e il 400° posto; l’anno scorso erano 14. Ed inoltre la maggioranza di queste 15 guadagna “bande” migliori rispetto al 2012 (ricordiamo che in questo settore la posizione è data per gruppi di 25 o di 50). Trento guida la pattuglia:
World Univ. Ranking
Home page Unipv
World ReputaXon Ranking
Capitolo 1 Dall’Egi,o con furore
Capitolo 2 Dall’Egitto con furore
rewind ...
16 se,embre 2010
New York Times, November 14, 2010
Alexandria’s surprising prominence was actually due to “the high output from one scholar in one journal” — soon idenNfied on various blogs as Mohamed El Naschie, an EgypNan academic who published over 320 of his own arXcles in a scienXfic journal of which he was also the editor.
rewind ...
26 novembre 2008
• ... of the 400 papers by El Naschie indexed in Web of Science, 307 were published in Chaos, Solitons and Fractals alone while he was editor-‐in-‐chief.
• El Naschie’s papers in CSF make 4992 citaXons, about 2000 of which are to papers published in CSF, largely his own.
never again?
THE ranking 2012: oops, I did it again!
• Only a few of the more than 100 co-‐authors of 2008 and 2010 reviews were from MEPhI
• The 2008 parNcle physics review received nearly 300 Nmes as many citaNons in the year a]er publicaNon as the mean for that journal
• Cites were averaged over the relaXvely small number of MEPhI’s publicaNons yielding a very high citaNon-‐rate
• Further, if citaNons are generally low in their countries, then insNtuNons get some more value added (regional modifica1on)
THE ranking 2012: oops, I did it again!
what about other rankings?
the top 10 most spectacular errors of …
reviewed by University Ranking Watch
QS greatest hits: internaXonal Students and Faculty in Malaysian UniversiXes
• In 2004 UniversiN Malaya (UM) in Malaysia. reached 89th place in the THES-‐QS world rankings.
• In 2005 came disaster. UM crashed 100 places
• PoliNcal opposiNon: shame on the university leadership!
• Real explanaNon: lot of Malaysian ciNzens of Indian and Chinese descent erroneously counted as “foreigners”.
QS greatest hits: 500 wrong student faculty raXos in 2007 QS Guide
• Someone slipped three rows when copying and pasNng student faculty raNos: Dublin InsNtute of Technology was given Duke’s raNo, Pretoria got Pune’s, Aachen RWT got Aberystwyth’s (Wales). And so on. Altogether over 500 errors.
Let’s go technical ...
Capitolo 3 Should you believe in the Shanghai ranking?
Shanghai: criteri e importanza %
The “normalizaXon trap” 1/2
The “normalizaXon trap” 2/2
Should you believe in the Shanghai ranking? An MCDM view J.-‐C. Billaut D. Bouyssou P. Vincke
• all criteria used are only loosely connected with what they intended to capture.
• several arbitrary parameters and many micro-‐decisions that are not documented.
• flawed and nonsensical aggregaNon method
• «the Shanghai ranking is a poorly conceived quick and dirty exercise»
«any of our MCDM student that would have proposed such a methodology in her Master’s Thesis would have surely failed according to our own standards»
HUNDREDS OF UNIVERSITIES WITH SIMILAR SCORES SC
OR
E SHANGHAI RANKING:
HOW MUCH RELIABLE?
RANK
THE RANKING
SHA
NG
HA
I RA
NK
ING
Twenty Ways to Rise in the Rankings (1/3) by Richard Holmes h,p://rankingwatch.blogspot.it/2013/12/twenty-‐ways-‐to-‐rise-‐in-‐rankings-‐quickly.html
1. Get rid of students. The university will therefore do beBer in the faculty student raNo indicators.
2. Kick out the old and bring in the young. Get rid of ageing professors, especially if unproducNve and expensive, and hire lots of temporary teachers and researchers.
5. Get a medical school. Medical research produces a disproporNonate number of papers and citaNons which is good for the QS citaNons per faculty indicator and the ARWU publicaNons indicator. Remember this strategy may not help with THE who use field normalisaNon.
7. Amalgamate. What about a new mega university formed by merging LSE, University College London and Imperial College? Or a tres grande ecole from all those liBle grandes ecoles around Paris?
9. The wisdom of crowds. Focus on research projects in those fields that have huge mulX -‐ “author” publicaXons, parNcle physics, astronomy and medicine for example. Such publicaNons o]en have very large numbers of citaNons.
10. Do not produce too much. If your researchers are producing five thousand papers a year, then those five hundred citaNons from a five hundred “author” report on the latest discovery in parNcle physics will not have much impact.
Twenty Ways to Rise in the Rankings (2/3) by Richard Holmes h,p://rankingwatch.blogspot.it/2013/12/twenty-‐ways-‐to-‐rise-‐in-‐rankings-‐quickly.html
13. The importance of names. Make sure that your researchers know which university they are affiliated to and that they know its correct name. Keep an eye on Scopus and ISI and make sure they know what you are called.
18. Support your local independence movement. Increasing the number of internaNonal students and faculty is good for both the THE and QS rankings. If it is difficult to move students across borders why not create new borders?
20. Get Thee to an Island. Leiden Ranking has a liBle known ranking that measures the distance between collaborators. At the moment the first place goes to the Australian NaNonal University.
Twenty Ways to Rise in the Rankings (3/3) by Richard Holmes h,p://rankingwatch.blogspot.it/2013/12/twenty-‐ways-‐to-‐rise-‐in-‐rankings-‐quickly.html
The Berlin Principles (1/2) • In 2004, the InsNtute for Higher EducaNon Policy in Washington and the UNESCO European Centre for Higher EducaNon (UNESCO-‐CEPES) in Bucharest founded the InternaXonal Ranking Expert Group (IREG) in 2004.
• In 2006, the group developed a set of principles of quality and good pracNce for higher educaNon rankings – referred to as the Berlin Principles on Ranking of Higher EducaNon InsNtuNons (IREG, 2006).
• Guidelines on four main areas: – purposes and goals – methodologies (design and weighNng of indicators), – collecNon and processing of data, and – presentaNon of ranking results.
The Berlin Principles (2/2) • Nel 2010, su una scala da 1 (nessuna congruenza) a 5 (congruenza
eccellente), su 25 rankings ben 13 non raggiungevano nemmeno il 3 (congruenza acceBabile), tra cui THE-‐QS, allora ancora unificata, che o,eneva 2.25, molto vicino al 2 (congruenza scarsa). STOLZ HENDEL HORN (2010): I. Stolz, D. Hendel, D., A. Horn, A, Ranking of rankings: benchmarking twenty-‐five higher educaNon ranking systems in Europe, Higher EducaNon, Online, p. 22. Springer, Netherlands.
• Nel dicembre 2011, l’IREG ha reso nota la possibilità di oBenere una cerNficazione “IREG approved” per le classifiche che superassero un audit basata sui Berlin Principles.
• Fino a Maggio 2013, nessun “IREG approved”.
• Un anno dopo, solo tre rankings sono “IREG approved” , tra cui QS è l’unico internazionale.
Capitolo 3 Numb3rs!
Capitolo 4
Let’s open the box
UNIVERSITY RANKING
RAW DATA
Fonte: “Malata e Denigrata” a cura di M. Regini, Donzelli 2009
INTERNAZIONALIZZAZIONE
Let’s open the box
1. “ScienNfic excellence” 2. Funding
1. “ScienXfic excellence”
Classifiche degli atenei: valore scien1fico assai dubbio.
Come si può misurare il peso di una nazione nel panorama scien1fico internazionale?
Contando gli arXcoli scienXfici che produce e
le citazioni che quesX o,engono
Italia: 8° per arXcoli scienXfici Fonte: SCImago su daN Scopus 1996-‐2012
0
10000
20000
30000
40000
50000
60000
70000
80000
90000
100000
1985 1990 1995 2000 2005 2010
Regno Unito
Germania
Giappone
Francia
Canada
Italia
Spagna
Olanda
Svizzera
Svezia
PUBBLICAZIONI (WoS)
0
1000000
2000000
3000000
4000000
5000000
6000000
PUBBLICAZIONI 2004-2010: NUMERO DI CITAZIONI
Fonte: VQR 2004-2010 – Rapporto Finale ANVUR, Giugno 2013 (Tab. 4.1) (dati ISI Web of Knowledge, Thomson-Reuters) http://www.anvur.org/rapporto/files/VQR2004-2010_RapportoFinale_parteterza_ConfrontiInternazionali.pdf
Efficienza: Italia ba,e Germania, Francia e Giappone
OCTOBER 2009
Ma come fanno questi italiani a produrre così tanta ricerca con così poche risorse?
2. Funding
Quanto è elitaria la “top 500”?
OTHER 16,500 UNIVERSITIES TOP 500
PERFORMANCE
NU
MB
ER O
F U
NIV
ERSI
TIES
... e cosa costa stare in cima?
OPERATING EXPENSES
FONDO FINANZIAMENTO ORDINARIO 2012
MILIARD
I DI EURO
LE “OPERATING EXPENSES” DI HARVARD AMMONTANO AL 44% DEL FONDO DI FINANZIAMENTO DELL’INTERO SISTEMA UNIVERSITARIO STATALE ITALIANO
E. Hazelkorn:
“EsFmated yearly budget of €1.5 billion to be ranked in the world’s top 100”
Spesa per università (% PIL): l’Italia è 30° su 33 (fonte: OCSE 2013)
e ciò nonostante ...
% di Atenei che entrano nei “top 500” (Leiden: top 250)
Fonte dei daF: “Malata e denigrata : l’universita italiana a confronto con l’Europa” (a cura di M. Regini, Roma, Donzelli 2009)
CLASSIFICA:
Capitolo 5 The power of numbers
Rankings
• Fragile scienNfic grounds • IncenNve to gaming
• Raw data are obscured • They are not necessary to manage funding (see RAE/REF)
Why, then?
Rankings are based on composite indicators
Science or pseudo-‐science?
Aggregators vs non-‐aggregators
• Aggregators: value in combining indicators: extremely useful in garnering media interest and hence the aVenFon of policy makers
• Non-‐aggregators: key objecFon to aggregaFon: the arbitrary nature of the weighFng process by which the variables are combined
Germany
• “We look back decades and people came to German universiFes; today they go to US universiFes.”
• The ExzellenziniXaXve (2005): from tradiNonal emphasis on egalitarianism towards compeNNon and hierarchical straXficaXon
France
• The Shanghai ranking
“generated considerable embarrassment among the French intelligentsia, academia and government: the first French higher educaFon insFtuFon in the ranking came only in 65th posi1on, mostly behind American universiFes and a few BriFsh ones”
Australia
• The SJT and QS: at least two Australian universiNes among the top 100.
• Opposing strategic opNons: – fund a small number of top-‐Ner compeNNve universiNes
– “creaNon of a diverse set of high performing, globally-‐focused insNtuNons, each with its own clear, disXncXve mission”.
Japan
• “The government wants a first class university for internaFonal presFge ”
• “in order for Japanese HEIs to compete globally, the government will close down some regional and private universiFes and direct money to the major universi1es”
• some insNtuNons will become teaching only.
Why obsessing about the “top 1%”?
OTHER 16,500 UNIVERSITIES TOP 1%
PERFORMANCE
NU
MB
ER O
F U
NIV
ERSI
TIES
Answer: trickle-‐down knowledge!
E. Hazelkorn on rankings
• 90 or 95% of our students do not aBend elite insNtuNons. Why are we spending so much on what people aren’t a,ending as opposed to what they are aBending?
• EsNmated yearly budget of €1.5 billion to be ranked in the world’s top 100. May detract resources from pensions, health, housing, ....
• Are “elite” insNtuNons really driving naNonal or regional economic and social development?
Does trickle-‐down work?
“Governments and universities must stop obsessing about global rankings and the top 1% of the world's 15,000 institutions. Instead of simply rewarding the achievements of elites and flagship institutions, policy needs to focus on the quality of the system-as-a-whole.”
There is little evidence that trickle-down works.
Capitolo 6
Where are we?
• (Even) Phil Baty (Times Higher EducaFon) admits that there are aspects of academic life where rankings are of liBle value
• Can we/you afford the ‘reputaNon race’? • We will have to live in a world in which extremely poor rankings are regularly published and used.
What can be done then?
“Stop being naive”
• There is no such thing as a ‘‘best university’’ in abstracto.
• Stop talking about these ‘‘all purpose rankings’’. They are meaningless.
• Lobby in our own insNtuNon so that these rankings are never menXoned in insNtuNonal communicaNon
• Produce many alternaXve rankings that produce vastly different results.
Grazie per l’a,enzione!