Post on 07-Jun-2018
transcript
Challenges in Ranking of Universities
First International Conference on World Class Universities, Jaio Tong
University, Shanghai, June 16-18, 2005
Anthony F.J. van Raan
Center for Science and technology Studies(CWTS)
Leiden University
6 Research Questions:
1. Research or Teaching?
2. How to Measure Performance?
3. For all universities in the world?
4. One numerical value?
5. Significance of positions?
6. How many?
Two most influential international
rankings:
Shanghai Jiao Tong University (60% bibliom.)
Times Higher Education Supplement (20% bibliom.)
VOLUME 88, Number 13 PHYSICAL REVIEW LETTERS 1 April 2002
Truncation of Power Law Behavior in “Scale-Free” Network Models
due to Information Filtering
Stefano Mossa,1,2 Marc Barthélémy,3 H. Eugene Stanley,1 and Luís A. Nunes Amaral1
1 Center for Polymer Studies and Department of Physics, Boston University, Boston, Massachusetts 02215
2 Dipartimento di Fisica, INFM UdR, and INFM Center for Statistical Mechanics and Complexity,
Universitàdi Roma “La Sapienza,” Piazzale Aldo Moro 2, I-00185, Roma, Italy
3 CEA-Service de Physique de la Matière Condensée, BP 12, 91680 Bruyères-le-Châtel, France
(Received 18 October 2001; published 14 March 2002)
We formulate a general model for the growth of scale-free networks under filtering information
conditions—that is, when the nodes can process information about only a subset of the existing nodes in the
network. We find that the distribution of the number of incoming links to a node follows a universal scaling
form, i.e., that it decays as a power law with an exponential truncation controlled not only by the system size
but also by a feature not previously considered, the subset of the network ―accessible‖ to the node. We test our
model with empirical data for the World Wide Web and find agreement.
DOI: 10.1103/PhysRevLett.88.138701 PACS numbers: 89.20.Hh, 84.35.+i, 89.75.Da, 89.75.Hc
There is a great deal of current interest in understanding the structure and growth mechanisms of global networks [1–3], such as the World Wide
Web (WWW) [4,5] and the Internet [6]. Network structure is critical in many contexts such as Internet attacks [2], spread of an Email virus [7], or
dynamics of human epidemics [8]. In all these problems, the nodes with the largest number of links play an important role on the dynamics of the
system. It is therefore important to know the global structure of the network as well as its precise distribution of the number of links.
Recent empirical studies report that both the Internet and the WWW have scale-free properties; that is, the number of incoming links and the
number of outgoing links at a given node have distributions that decay with power law tails [4–6]. It has been proposed [9] that the scale-free
structure of the Internet and the WWW may be explained by a mechanism referred to as ―preferential attachment‖ [10] in which new nodes link
to existing nodes with a probability proportional to the number of existing links to these nodes. Here we focus on the stochastic character of the
preferential attachment mechanism, which we understand in the following way: New nodes want to connect to the existing nodes with the largest
number of links—i.e., with the largest degree—because of the advantages offered by being linked to a well-connected node. For a large network it
is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with
based on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a
larger degree are more likely to become known.
VOLUME 88, Number 13 PHYSICAL REVIEW LETTERS 1 April 2002
Truncation of Power Law Behavior in “Scale-Free” Network Models
due to Information Filtering
Stefano Mossa,1,2 Marc Barthélémy,3 H. Eugene Stanley,1 and Luís A. Nunes Amaral1
1 Center for Polymer Studies and Department of Physics, Boston University, Boston, Massachusetts 02215
2 Dipartimento di Fisica, INFM UdR, and INFM Center for Statistical Mechanics and Complexity,
Universitàdi Roma “La Sapienza,” Piazzale Aldo Moro 2, I-00185, Roma, Italy
3 CEA-Service de Physique de la Matière Condensée, BP 12, 91680 Bruyères-le-Châtel, France
(Received 18 October 2001; published 14 March 2002)
We formulate a general model for the growth of scale-free networks under filtering information
conditions—that is, when the nodes can process information about only a subset of the existing nodes in the
network. We find that the distribution of the number of incoming links to a node follows a universal scaling
form, i.e., that it decays as a power law with an exponential truncation controlled not only by the system size
but also by a feature not previously considered, the subset of the network ―accessible‖ to the node. We test our
model with empirical data for the World Wide Web and find agreement.
DOI: 10.1103/PhysRevLett.88.138701 PACS numbers: 89.20.Hh, 84.35.+i, 89.75.Da, 89.75.Hc
There is a great deal of current interest in understanding the structure and growth mechanisms of global networks [1–3], such as the World Wide
Web (WWW) [4,5] and the Internet [6]. Network structure is critical in many contexts such as Internet attacks [2], spread of an Email virus [7], or
dynamics of human epidemics [8]. In all these problems, the nodes with the largest number of links play an important role on the dynamics of the
system. It is therefore important to know the global structure of the network as well as its precise distribution of the number of links.
Recent empirical studies report that both the Internet and the WWW have scale-free properties; that is, the number of incoming links and the
number of outgoing links at a given node have distributions that decay with power law tails [4–6]. It has been proposed [9] that the scale-free
structure of the Internet and the WWW may be explained by a mechanism referred to as ―preferential attachment‖ [10] in which new nodes link
to existing nodes with a probability proportional to the number of existing links to these nodes. Here we focus on the stochastic character of the
preferential attachment mechanism, which we understand in the following way: New nodes want to connect to the existing nodes with the largest
number of links—i.e., with the largest degree—because of the advantages offered by being linked to a well-connected node. For a large network it
is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with
based on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a
larger degree are more likely to become known.
VOLUME 88, Number 13 PHYSICAL REVIEW LETTERS 1 April 2002
Truncation of Power Law Behavior in “Scale-Free” Network Models
due to Information Filtering
Stefano Mossa,1,2 Marc Barthélémy,3 H. Eugene Stanley,1 and Luís A. Nunes Amaral1
1 Center for Polymer Studies and Department of Physics, Boston University, Boston, Massachusetts 02215
2 Dipartimento di Fisica, INFM UdR, and INFM Center for Statistical Mechanics and Complexity,
Universitàdi Roma “La Sapienza,” Piazzale Aldo Moro 2, I-00185, Roma, Italy
3 CEA-Service de Physique de la Matière Condensée, BP 12, 91680 Bruyères-le-Châtel, France
(Received 18 October 2001; published 14 March 2002)
We formulate a general model for the growth of scale-free networks under filtering information
conditions—that is, when the nodes can process information about only a subset of the existing nodes in the
network. We find that the distribution of the number of incoming links to a node follows a universal scaling
form, i.e., that it decays as a power law with an exponential truncation controlled not only by the system size
but also by a feature not previously considered, the subset of the network ―accessible‖ to the node. We test our
model with empirical data for the World Wide Web and find agreement.
DOI: 10.1103/PhysRevLett.88.138701 PACS numbers: 89.20.Hh, 84.35.+i, 89.75.Da, 89.75.Hc
There is a great deal of current interest in understanding the structure and growth mechanisms of global networks [1–3], such as the World Wide
Web (WWW) [4,5] and the Internet [6]. Network structure is critical in many contexts such as Internet attacks [2], spread of an Email virus [7], or
dynamics of human epidemics [8]. In all these problems, the nodes with the largest number of links play an important role on the dynamics of the
system. It is therefore important to know the global structure of the network as well as its precise distribution of the number of links.
Recent empirical studies report that both the Internet and the WWW have scale-free properties; that is, the number of incoming links and the
number of outgoing links at a given node have distributions that decay with power law tails [4–6]. It has been proposed [9] that the scale-free
structure of the Internet and the WWW may be explained by a mechanism referred to as ―preferential attachment‖ [10] in which new nodes link
to existing nodes with a probability proportional to the number of existing links to these nodes. Here we focus on the stochastic character of the
preferential attachment mechanism, which we understand in the following way: New nodes want to connect to the existing nodes with the largest
number of links—i.e., with the largest degree—because of the advantages offered by being linked to a well-connected node. For a large network it
is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with
based on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a
larger degree are more likely to become known.
VOLUME 88, Number 13 PHYSICAL REVIEW LETTERS 1 April 2002
Truncation of Power Law Behavior in “Scale-Free” Network Models
due to Information Filtering
Stefano Mossa,1,2 Marc Barthélémy,3 H. Eugene Stanley,1 and Luís A. Nunes Amaral1
1 Center for Polymer Studies and Department of Physics, Boston University, Boston, Massachusetts 02215
2 Dipartimento di Fisica, INFM UdR, and INFM Center for Statistical Mechanics and Complexity,
Universitàdi Roma “La Sapienza,” Piazzale Aldo Moro 2, I-00185, Roma, Italy
3 CEA-Service de Physique de la Matière Condensée, BP 12, 91680 Bruyères-le-Châtel, France
(Received 18 October 2001; published 14 March 2002)
We formulate a general model for the growth of scale-free networks under filtering information
conditions—that is, when the nodes can process information about only a subset of the existing nodes in the
network. We find that the distribution of the number of incoming links to a node follows a universal scaling
form, i.e., that it decays as a power law with an exponential truncation controlled not only by the system size
but also by a feature not previously considered, the subset of the network ―accessible‖ to the node. We test our
model with empirical data for the World Wide Web and find agreement.
DOI: 10.1103/PhysRevLett.88.138701 PACS numbers: 89.20.Hh, 84.35.+i, 89.75.Da, 89.75.Hc
There is a great deal of current interest in understanding the structure and growth mechanisms of global networks [1–3], such as the World Wide
Web (WWW) [4,5] and the Internet [6]. Network structure is critical in many contexts such as Internet attacks [2], spread of an Email virus [7], or
dynamics of human epidemics [8]. In all these problems, the nodes with the largest number of links play an important role on the dynamics of the
system. It is therefore important to know the global structure of the network as well as its precise distribution of the number of links.
Recent empirical studies report that both the Internet and the WWW have scale-free properties; that is, the number of incoming links and the
number of outgoing links at a given node have distributions that decay with power law tails [4–6]. It has been proposed [9] that the scale-free
structure of the Internet and the WWW may be explained by a mechanism referred to as ―preferential attachment‖ [10] in which new nodes link
to existing nodes with a probability proportional to the number of existing links to these nodes. Here we focus on the stochastic character of the
preferential attachment mechanism, which we understand in the following way: New nodes want to connect to the existing nodes with the largest
number of links—i.e., with the largest degree—because of the advantages offered by being linked to a well-connected node. For a large network it
is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with
based on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a
larger degree are more likely to become known.
VOLUME 88, Number 13 PHYSICAL REVIEW LETTERS 1 April 2002
Truncation of Power Law Behavior in “Scale-Free” Network Models
due to Information Filtering
Stefano Mossa,1,2 Marc Barthélémy,3 H. Eugene Stanley,1 and Luís A. Nunes Amaral1
1 Center for Polymer Studies and Department of Physics, Boston University, Boston, Massachusetts 02215
2 Dipartimento di Fisica, INFM UdR, and INFM Center for Statistical Mechanics and Complexity,
Universitàdi Roma “La Sapienza,” Piazzale Aldo Moro 2, I-00185, Roma, Italy
3 CEA-Service de Physique de la Matière Condensée, BP 12, 91680 Bruyères-le-Châtel, France
(Received 18 October 2001; published 14 March 2002)
We formulate a general model for the growth of scale-free networks under filtering information
conditions—that is, when the nodes can process information about only a subset of the existing nodes in the
network. We find that the distribution of the number of incoming links to a node follows a universal scaling
form, i.e., that it decays as a power law with an exponential truncation controlled not only by the system size
but also by a feature not previously considered, the subset of the network ―accessible‖ to the node. We test our
model with empirical data for the World Wide Web and find agreement.
DOI: 10.1103/PhysRevLett.88.138701 PACS numbers: 89.20.Hh, 84.35.+i, 89.75.Da, 89.75.Hc
There is a great deal of current interest in understanding the structure and growth mechanisms of global networks [1–3], such as the World Wide
Web (WWW) [4,5] and the Internet [6]. Network structure is critical in many contexts such as Internet attacks [2], spread of an Email virus [7], or
dynamics of human epidemics [8]. In all these problems, the nodes with the largest number of links play an important role on the dynamics of the
system. It is therefore important to know the global structure of the network as well as its precise distribution of the number of links.
Recent empirical studies report that both the Internet and the WWW have scale-free properties; that is, the number of incoming links and the
number of outgoing links at a given node have distributions that decay with power law tails [4–6]. It has been proposed [9] that the scale-free
structure of the Internet and the WWW may be explained by a mechanism referred to as ―preferential attachment‖ [10] in which new nodes link
to existing nodes with a probability proportional to the number of existing links to these nodes. Here we focus on the stochastic character of the
preferential attachment mechanism, which we understand in the following way: New nodes want to connect to the existing nodes with the largest
number of links—i.e., with the largest degree—because of the advantages offered by being linked to a well-connected node. For a large network it
is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with
based on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a
larger degree are more likely to become known.
VOLUME 88, Number 13 PHYSICAL REVIEW LETTERS 1 April 2002
Truncation of Power Law Behavior in “Scale-Free” Network Models
due to Information Filtering
Stefano Mossa,1,2 Marc Barthélémy,3 H. Eugene Stanley,1 and Luís A. Nunes Amaral1
1 Center for Polymer Studies and Department of Physics, Boston University, Boston, Massachusetts 02215
2 Dipartimento di Fisica, INFM UdR, and INFM Center for Statistical Mechanics and Complexity,
Universitàdi Roma “La Sapienza,” Piazzale Aldo Moro 2, I-00185, Roma, Italy
3 CEA-Service de Physique de la Matière Condensée, BP 12, 91680 Bruyères-le-Châtel, France
(Received 18 October 2001; published 14 March 2002)
We formulate a general model for the growth of scale-free networks under filtering information
conditions—that is, when the nodes can process information about only a subset of the existing nodes in the
network. We find that the distribution of the number of incoming links to a node follows a universal scaling
form, i.e., that it decays as a power law with an exponential truncation controlled not only by the system size
but also by a feature not previously considered, the subset of the network ―accessible‖ to the node. We test our
model with empirical data for the World Wide Web and find agreement.
DOI: 10.1103/PhysRevLett.88.138701 PACS numbers: 89.20.Hh, 84.35.+i, 89.75.Da, 89.75.Hc
There is a great deal of current interest in understanding the structure and growth mechanisms of global networks [1–3], such as the World Wide
Web (WWW) [4,5] and the Internet [6]. Network structure is critical in many contexts such as Internet attacks [2], spread of an Email virus [7], or
dynamics of human epidemics [8]. In all these problems, the nodes with the largest number of links play an important role on the dynamics of the
system. It is therefore important to know the global structure of the network as well as its precise distribution of the number of links.
Recent empirical studies report that both the Internet and the WWW have scale-free properties; that is, the number of incoming links and the
number of outgoing links at a given node have distributions that decay with power law tails [4–6]. It has been proposed [9] that the scale-free
structure of the Internet and the WWW may be explained by a mechanism referred to as ―preferential attachment‖ [10] in which new nodes link
to existing nodes with a probability proportional to the number of existing links to these nodes. Here we focus on the stochastic character of the
preferential attachment mechanism, which we understand in the following way: New nodes want to connect to the existing nodes with the largest
number of links—i.e., with the largest degree—because of the advantages offered by being linked to a well-connected node. For a large network it
is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with
based on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a
larger degree are more likely to become known.
…or let scientific output and its impact speak:
bibliometric analysis
Correlation between impact and rankingWorldwide top-universities in life/biomedical sciences
1.00
10.00
100.00
1 10 100 1000
r
CPP
P 2001-2004 C 2001-2004 CPP/FCSm
USA 994,650 0.270 3,747,932 0.380 1.38
JAPAN 278,420 0.076 605,876 0.062 0.86
GREAT BRITAIN 270,517 0.073 851,704 0.086 1.22
GERMANY 251,365 0.068 743,582 0.075 1.12
FRANCE 180,145 0.049 490,137 0.050 1.05
PEOPLES R CHINA 162,771 0.044 174,316 0.018 0.57
ITALY 132,091 0.036 320,773 0.033 0.95
CANADA 131,469 0.036 382,198 0.039 1.18
SPAIN 94,005 0.026 197,001 0.020 0.91
RUSSIA 91,749 0.025 83,335 0.008 0.43
AUSTRALIA 83,675 0.023 213,928 0.022 1.07
NETHERLANDS 75,903 0.021 251,170 0.025 1.27
SOUTH KOREA 70,878 0.019 101,548 0.010 0.78
INDIA 68,685 0.019 70,965 0.007 0.46
SWEDEN 59,144 0.016 187,035 0.019 1.15
SWITZERLAND 54,484 0.015 215,893 0.022 1.39
BRAZIL 46,005 0.012 59,758 0.006 0.6
TAIWAN 45,948 0.012 60,273 0.006 0.72
POLAND 42,490 0.012 54,078 0.005 0.6
BELGIUM 40,916 0.011 116,947 0.012 1.15
3,681,790 9,850,029
Expert Survey Problems:(methodological)
1. Biases: geographical, field-specific
2. Responding > Non-responding characteristics
3. Sample size > reliability of measurement
4. Nomination procedure
5. Scaling procedure
6. Controlling variables
7. Standard deviation scores
8. Statistical significance
Correlation between Expert Scores with Citation-Analysis Based Scores
THE Ranking 2004
y = 53.985x0.0397
R2 = 0.005
1
10
100
1000
0.1 1 10 100 1000
C
E
1. Technical problems:
- citing-cited mismatches
- definition & unification of institutions(specific responsibility)
2. Methodological Problems:
- Field definition
- Field-normalization of citation counts
- Black box indicators
- Highly cited scientists > highly cited article
- Article-type normalization of citation counts
- US bias
- Language bias (Germany: 25%!)
- Engineering, Social Sciences, Humanities
- Same data, same methodology, different rankings
New Approaches:
- Iteration of Expert Survey focused on top
- Output-specific analysis engineering fields, social
science and humanities
- Top-10% bibliometric analysis
Top-10% Approach:
1. Identify universities with P > 200/y (N ~ 250)
2. Collect all publication so these universities
3. Ranking by:
- entire oeuvre
- top-10% of the oeuvre
in both cases: CPP and CPP/FCSm
- CPP/FCSm(top) x P(top)
Outlook:
- Improved ranking procedures will further
de-equalize universities and reinforce a scientific elite
league
- Excessive evaluation hypes will les to science
destruction
- Balance has to be found by data-system improvement
and automation of advanced bibliometric assessment
procedures
Research profile
Output and impact per field2000 - 2003
Leiden University
0 1 2 3 4 5 6 7
ASTRON & ASTROPH (1.38)
BIOCH & MOL BIOL (0.96)
ONCOLOGY (1.05)
IMMUNOLOGY (1.22)
HEMATOLOGY (1.27)
GENETICS & HERED (1.48)
PHARMACOL & PHAR (1.11)
PHYSICS,MULTIDIS (1.84)
PHYSICS, COND MA (1.21)
ENDOCRIN & METAB (0.99)
MEDICINE,GENERAL (3.35)
RAD,NUCL MED IM (1.04)
CHEM, PHYSICAL (1.00)
CARD & CARD SYST (0.95)
RHEUMATOLOGY (1.75)
CLIN NEUROLOGY (1.72)
NEUROSCIENCES (0.86)
CHEM, INORG&NUC (1.82)
PHYSICS, AT,M,C (0.87)
PERIPHL VASC DIS (1.10)
CELL BIOLOGY (1.05)
MULTIDISCIPL SC (1.31)
CHEM, ORGANIC (1.02)
PLANT SCIENCES (1.04)
PATHOLOGY (1.56)
SURGERY (1.34)
CHEMISTRY (1.60)
COMPU SCI,THEORY (1.05)
PEDIATRICS (1.56)
FIELD
(CPP/FCSm)
Share of the output (%)
IMPACT: LOW AVERAGE HIGH
>50% above, and no field below
internat. average
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
45,000
50,000
1980 -
1983
1981 -
1984
1982 -
1985
1983 -
1986
1984 -
1987
1985 -
1988
1986 -
1989
1987 -
1990
1988 -
1991
1989 -
1992
1990 -
1993
1991 -
1994
1992 -
1995
1993 -
1996
1994 -
1997
1995 -
1998
1996 -
1999
1997 -
2000
1998 -
2001
1999 -
2002
2000 -
2003
2001 -
2004
P
C+sc
Research output from Shanghai (PRC)
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1980 -
1983
1981 -
1984
1982 -
1985
1983 -
1986
1984 -
1987
1985 -
1988
1986 -
1989
1987 -
1990
1988 -
1991
1989 -
1992
1990 -
1993
1991 -
1994
1992 -
1995
1993 -
1996
1994 -
1997
1995 -
1998
1996 -
1999
1997 -
2000
1998 -
2001
1999 -
2002
2000 -
2003
2001 -
2004
CPP/JCSm
CPP/FCSm
JCSm/FCSm
Normalized impact scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
1980 -
1983
1981 -
1984
1982 -
1985
1983 -
1986
1984 -
1987
1985 -
1988
1986 -
1989
1987 -
1990
1988 -
1991
1989 -
1992
1990 -
1993
1991 -
1994
1992 -
1995
1993 -
1996
1994 -
1997
1995 -
1998
1996 -
1999
1997 -
2000
1998 -
2001
1999 -
2002
2000 -
2003
2001 -
2004
% P not cited
% Self-citations
Percentages 'Publications not cited' and 'self-citations'
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1980 -
1983
1981 -
1984
1982 -
1985
1983 -
1986
1984 -
1987
1985 -
1988
1986 -
1989
1987 -
1990
1988 -
1991
1989 -
1992
1990 -
1993
1991 -
1994
1992 -
1995
1993 -
1996
1994 -
1997
1995 -
1998
1996 -
1999
1997 -
2000
1998 -
2001
1999 -
2002
2000 -
2003
2001 -
2004
Single address
National
International
Field normalized impact scores for scientific cooperation types
0% 20% 40% 60% 80% 100%
1980 - 1983
1981 - 1984
1982 - 1985
1983 - 1986
1984 - 1987
1985 - 1988
1986 - 1989
1987 - 1990
1988 - 1991
1989 - 1992
1990 - 1993
1991 - 1994
1992 - 1995
1993 - 1996
1994 - 1997
1995 - 1998
1996 - 1999
1997 - 2000
1998 - 2001
1999 - 2002
2000 - 2003
2001 - 2004
Single address
National
International
Growing (inter)national scientific cooperation