Scientific Research Measures

Marco Frittelli∗

Department of Mathematics

University of Milan

Loriano Mancini†

Swiss Finance Institute

and EPFL

Ilaria Peri‡

ESC Rennes

School of Business

This version: June 24, 2013

First version: May 2012

∗Corresponding author: Marco Frittelli, Department of Mathematics, University of Milan, Via Sal-dini 50, I-20133 Milan, Italy. E-mail: marco.frittelli@unimi.it.

†Loriano Mancini, Swiss Finance Institute at EPFL, Quartier UNIL-Dorigny, CH-1015 Lausanne, Switzer-land. E-mail: loriano.mancini@epfl.ch.

‡Ilaria Peri, Finance and Operations Department, ESC Rennes, 2 rue Robert d’Arbrissel CS 76522,F-35065 Rennes Cedex, France. E-mail: ilaria.peri@esc-rennes.fr.

Scientific Research Measures

Abstract

We introduce the novel class of Scientific Research Measures (SRMs) to rank scien-

tists’ research performance. In contrast to many bibliometric indices, SRMs take into

account the whole scientist’s citation curve, share sensible structural properties, allow

for a finer ranking of scientists, fit specific features of different disciplines, research

areas and seniorities, and include several bibliometric indices as special cases. We also

introduce the further general class of Dual SRMs that allows for a more informed rank-

ing of research performances based on “journals’ value.” An empirical application to

173 finance scholars’ citation curves shows that SRMs can be easily calibrated to actual

citation curves and produce different authors’ rankings than traditional bibliometric

indices.

Keywords: Bibliometric Indices, Citations, Risk Measures, Scientific Impact Measures,

Calibration, Duality

JEL Codes: C02, I20

1. Introduction

Over the last decades, the importance of evaluating scientific research performance has grown

exponentially. Crucial decisions such as funding research projects, faculty recruitment or

academic promotion depend to a large extent upon the scientific merits of the involved re-

searchers. Selecting inappropriate valuation criteria can have obvious and deleterious effects

on performance assessment of scientists and structures, such as departments or laboratories,

and distort funding allocation and recruiting process.

Two approaches have been proposed to assess research performance: content valuation,

based on internal committees and external panels of peer reviewers, and context valuation,

based on bibliometric indices, i.e., statistics derived from citations and characteristics of

journals where the research output appeared. Context valuation is far less expensive in

terms of time and resources involved than content valuation and can be easily carried out on

a systematic base, e.g., yearly base. In contrast, content valuation can only be carried out

on a multiple-year base and used to check or fine tune the evaluation based on bibliometric

indices. Moreover, content valuation is unfeasible when a large number of scientists or

research outputs need to be evaluated.

Because of the fast increase in availability and quality of online databases (e.g., Google

Scholar, ISI Web of Science, Scopus, MathSciNet), context valuation has become widely

popular and several bibliometric indices have been suggested to assess research performance.

The most popular citation-based metric is the h-index introduced by Hirsch (2005). The

h-index is the largest number such that h publications have at least h citations each. This

index has received a large attention from the scientific community and has been widely

used to assess research performance in many fields. Recently, various authors have proposed

several extensions of the h-index.1 Most of these indices have been introduced on an ad hoc

basis to improve on some of the shortcomings of the h-index and its subsequent extensions.

This paper provides three contributions to the fast growing literature on bibliometric

1For an overview of these proposals see Section 2 below and, e.g., Alonso, Cabrerizo, Herrera-Viedma, andHerrera (2009), Panaretos and Malesios (2009), and Schreiber (2010).

1

indices. The first contribution is to introduce a novel class of scientific performance measures

based on citation curves2 that we call Scientific Research Measures (SRMs). The main feature

of SRMs is to take into account the whole scientist’s citation curve and to assess her research

performance using predetermined performance curves. The performance curves determine

the minimum threshold of citations to reach a certain performance level. Importantly, such

performance curves can be chosen in a flexible way, for example reflecting the seniority of

the scientists and characteristics of the specific research field, such as typical number of

publications and citation rates. This flexibility is absent in virtually all existing bibliometric

indices. SRMs are derived from a “calibration” approach in the sense that SRMs are informed

by actual citation data. Under the premise that research performance is reflected by the

whole citation curve, rather than by only part of it, SRMs provide by construction a better

assessment of research performance than standard bibliometric indices.

SRMs have two desirable properties, namely monotonicity and quasi-concavity. The first

property simply implies that better scientists, as reflected by their citation curves, have

higher research performance measures. The second property, quasi-concavity, implies that

when a given scientist is combined with a better scientist (in the sense of a convex combina-

tion of their citation curves), the performance of the two is higher than the performance of

the initial scientist. Quasi-concavity is not only a sensible property of SRMs. It also allows

to rank scientists whose citation curves intersect each other (see Appendix A), as is often

the case in practice. In those cases, the monotonicity property alone would not lead to any

ranking.

SRMs subsume many existing bibliometric indices as special cases. Woeginger (2008)

provides an earlier attempt to collect bibliometric indices in a general class. However, there

are significant differences between the two approaches. SRMs allow for a granular ranking

of scientists’ research performances, share sensible structural properties, and appear to be

more flexible as SRMs reflect the seniority of scientists and specific features of the various

disciplines.

2The citation curve is the number of citations of each publication, in decreasing order of citations.

2

The second contribution of this paper is to introduce a further general class of research

measures called Dual SRMs. We provide a rigorous and axiomatic-based theoretical foun-

dation of the Dual SRMs. The novelty is to describe researchers’ performance using citation

records, rather than citation curves. A citation record associates to each journal the cita-

tions of the researcher’s publications in that journal. Then we draw an analogy between

citation records, viewed as random variables defined on journals, and risky financial payoffs.

This analogy allows us to formalize the mathematical properties of Dual SRMs by building

on the well-established theory of risk measures. The axiomatic approach developed in the

seminal paper by Artzner, Delbaen, Eber, and Heath (1999) has been very influential for

the theory of risk measures.3 Follmer and Schied (2002) and Frittelli and Rosazza Gianin

(2002) extended the theory of (coherent) risk measures to the class of convex risk measures.

The origin of our proposal of Dual SRMs can be traced back to the most recent devel-

opments of this theory, leading to the notion of quasi-convex risk measures introduced by

Cerreia-Vioglio, Maccheroni, Marinacci, and Montrucchio (2011) and further developed in

the dynamic framework by Frittelli and Maggis (2011).4

We establish a duality between the primal space of citation records and its dual space

given by the “value” of each journal. A journal’s value could be measured for example using

the impact factor of the journal or any other criterion deemed sensible by the evaluator who

is assessing the scientists’ research performances. Multiple criteria are possible which may

induce multiple values of a given journal.5 In general, there will be no agreement on the

3Rather than focusing on a specific measurement of risk carried by financial positions (e.g., variance ofasset returns), Artzner, Delbaen, Eber, and Heath (1999) proposed a class of measures satisfying sensibleproperties, so-called coherent axioms. Ideally, each financial institution could select its own risk measure,provided it obeyed the structural coherent properties. This approach added flexibility in the selection of therisk measure and, at the same time, established a unified framework. In the same spirit, we introduce DualSRMs to assess research performance, rather than financial risk.4Among other studies in this area, Cherny and Madan (2009) introduced the concept of an acceptabilityindex having the property of quasi-concavity, and Drapeau and Kupper (2013) analyzed the relation betweenquasi-convex risk measures and associated family of acceptance sets, already present in Cherny and Madan(2009).5To further elaborate on the analogy between Dual SRMs and risk measures, the journals are the statesof nature and each journal’s value is the price of the Arrow–Debreu security associated to the given state.Multiple values of the journal are multiple prices of the Arrow–Debreu security, which may occur whenfinancial markets are incomplete.

3

journals’ values. Notably, the Dual SRM provides a prudential or conservative assessment

of the scientist’s research performance, i.e., under the least favorable journal’s evaluation

criterion.

Dual SRMs are relevant for three reasons. First, if two scientists have the same citation

curve, any traditional bibliometric index would not be able to rank them. In contrast, Dual

SRMs can rank those scientists as soon as their publications appeared in different journals.

Second, additional information beyond the citation curve (such as journals’ values and the

evaluator’s assessment of the corresponding criteria) can be taken into account in the rank-

ings based on Dual SRMs. Consequently, Dual SRMs induce a more informed ranking than

existing bibliometric indices based on citation curves. Third, and perhaps most importantly,

Dual SRMs allow to achieve a meaningful aggregation of scientists’ research outputs, by

aggregating scientists’ citation records. Thus, Dual SRMs allow to rank the research per-

formances of teams, departments and academic institutions. The issue of ranking research

institutions on the basis of their research output has been a central theme in the scientific

community over the last decades, e.g., Borokhovich, Bricker, Brunarski, and Simkins (1995),

and Kalaitzidakis, Stengos, and Mamuneas (2003).

The third contribution of this paper is to present an extensive empirical application of

SRMs. Using Google Scholar, we construct a novel dataset of citation curves of 173 full

professors affiliated to 11 main U.S. business schools or finance departments. Our empirical

results show that the SRM characterized by hyperbolic-type performance curves describes

well actual citation curves. We compare authors’ and universities’ rankings based on the

SRM and other 7 existing bibliometric indices. It appears that rankings based on the SRM

and the other bibliometric indices are generally quite different. The largest discrepancies

in rankings concern a large portion of “average” performing scholars and universities (in

relative terms), which are arguably the most difficult to rank, rather than the best and least

performing scholars and universities. We recall that when research performance is reflected

by the whole citation curve, rather than by only part of it, our SRMs provide a more sensible

ranking than standard bibliometric indices. Finally, we provide clear operational guidance

4

on how to implement our SRMs and develop a simple two-step algorithm to compute SRMs

based on hyperbolic-type performance curves.

The remainder of the paper is organized as follows. Section 2 reviews the related litera-

ture. Section 3 introduces the SRMs. Section 4 presents the Dual SRMs. Section 5 discusses

the empirical analysis. Section 6 concludes. Appendices collect technical material.

2. Related Literature

At least since the early nineties, the use of bibliometric indices have encompassed many fields,

e.g., Chung and Cox (1990), May (1997), Hauser (1998), Wiberley (2003), and Kalaitzidakis,

Stengos, and Mamuneas (2003). Indeed, the effectiveness of citation-based metrics is vali-

dated by correlation analyses with peer-review ratings, e.g., Lovegrove and Johnson (2008).

Moreover, when a large number of scientists or research outputs need to be evaluated, bib-

liometric indices are essentially the only viable method.6

Bibliometric indices were initially introduced either to merely quantify the production of

researchers, e.g., total number of publications in a given period, or to assess the impact of

their publications, e.g., total number of citations. To obtain a more comprehensive evalua-

tion of the scientist’s research performance, Hirsch (2005) introduced the h-index, which is

nowadays the most popular citation-based metric.

The main achievement of the h-index is to combine scientific production (given by the

number of publications) and quality of research (given by the number of citations) in a single

bibliometric index. The index is also simple to compute.7 Moreover, Hirsch (2007) shows

that his index predicts future individual scientific achievements more than other bibliometric

indices. The main drawback of the h-index is the insufficient ability to discriminate between

6The Italian National Agency for the Evaluation of Universities and Research Institutes (ANVUR) is cur-rently undertaking a large evaluation process that requires the assessment of more than 200,000 researchitems (journal articles, monographs, book chapters, etc.) published by all Italian researchers between 2004and 2010 in 14 different fields. Each panel, one for each field, consists of 30 members and has about 15,000research items to evaluate in less than one year. All panels, except those in humanistic areas, decided to usebibliometric indices.7For example using ISI Web of Science, http://www.webofknowledge.com, the h-index can be easily com-puted by ordering the scientist’s publications by the field ‘Time Cited.’

5

scientists’ research performances. To illustrate this point, suppose an author has 10 pub-

lications with 10 citations each, and thus an h-index of 10. She has the same h-index as

a second author who has 100 publications but only 10 of them received 10 citations, and

a third author who has only 10 publications but with 100 citations each. In other words,

the h-index does not reflect publications with high and low number of citations, which may

well contain valuable information for assessing the scientific performance. The h-index is

fully determined by the Hirsch core, which is the set of the most cited publications with

at least h citations each. In the example above, the Hirsch core is 10 publications. The

report by the Joint Committee on Quantitative Assessment of Research8 (Adler, Ewing, and

Taylor, 2008) strengthened this point by a well-documented example and encouraged the use

of more complex measures. The report also emphasized the lack of mathematical analysis of

the properties of the h-index.9

To overcome the limitations of the h-index, a number of scientists from different fields

have proposed a wide range of indices to measure scientific performances. We classify the

numerous proposals in two broad categories: h-index variants, that suggest alternative meth-

ods to determine the core size of the most cited publications, and h-index complements, that

include additional information in the h-index especially concerning the further citations of

the most cited publications.

Among the h-index variants, one of the first proposals is the g-index by Egghe (2006).

The g-index is defined as the largest number g such that the most cited g articles received

together at least g2 citations. Following the same logic, Kosmulski (2006) suggests the h2-

index defined as the largest number h such that the h most cited publications receive at least

h2 citations each. Relative to the h-index, the h2-index is probably more appropriate in fields

where the number of citations per article is relatively high, like chemistry or physics, while

the h-index seems to be more suitable, say, for mathematics or astronomy. Many studies

8See http://www.mathunion.org/fileadmin/IMU/Report/CitationStatistics.pdf.9As any other bibliometric index, the h-index presents other limitations concerning for example the issue ofself citations, the number of co-authors and the accuracy of bibliometric databases. Recently, various indiceshave been proposed to alleviate these limitations. However, these indices are based on transformations of theoriginal citation data, raising new issues concerning the best practice of carrying out such transformations.

6

point out the arbitrariness of the definition of the citation core size. To alleviate this issue,

van Eck and Waltman (2008) propose the ha-index defined as the largest natural number

ha such that the ha most cited publications received at least a ha citations each. A further

variant is the w-index introduced by Woeginger (2008), and defined as follows: a scientist has

index w if w of her publications have at least w,w−1, . . . , 1 citations. Ruane and Tol (2008)

propose the rational h-index (hrat-index) whose advantage is to provide more granularity in

the evaluation process since it increases in smaller steps than the original h-index. Guns and

Rousseau (2009) review several real and rational variants of the h- and g-indices.

Rather than concentrating on the size of the citation core, the h-index complements seek

to measure the core citation intensity. For example, Jin (2006) proposes to compute the

average of the Hirsch core citations (A-index). Subsequently, to reduce the penalization of

the best scientists receiving many citations, Jin, Liang, Rousseau, and Egghe (2007) modify

the A-index by taking the square root of the Hirsch core citations (R-index). The same

approach is used by Egghe and Rousseau (2008) but they consider as citation core only a

subset of the Hirsch core (hw-index). Both the R- and hw-index can be very sensitive to

just a few publications with many citations. In an attempt to make these indices more

robust, Bornmann, Mutz, and Daniel (2008) propose to calculate the median, rather than

the mean, of the citations in the Hirsch core (m-index). The distribution of citations is often

positively skewed, and the suggestion of using the median appears to be a sensible proposal.

An interesting h-index complement is the tapered h-index, proposed by Anderson, Hankin,

and Killworth (2008), which attempts to take into account all citations, not just those in

the Hirsch core. Instead, Vinkler (2009) proposes the π-index defined as one hundredth

of the total citations obtained by the most influential publications, which are defined as

the square root of the total publications ranked in decreasing order of citations. Zhang

(2009) suggests a complementary index to the h-index to summarize the excess citations in

the Hirsch core by taking their square root (e-index). Further proposals to attenuate the

drawbacks of the h-index include the geometric mean of the h- and g-index (hg-index by

Alonso, Cabrerizo, Herrera-Viedma, and Herrera (2010)) and the geometric mean of the h-

7

and m-index (q2-index by Cabrerizo, Alonso, Herrera-Viedma, and Herrera (2010)).

The main consideration that emerges from this literature review is that most bibliometric

indices have been proposed on an ad hoc basis to remedy specific issues related either to the

core size of publications or to the impact of such core publications. Moreover, as pointed

out by Schreiber (2010), many of these indices are highly correlated and lead to very similar

rankings.

We depart from the literature by adopting a “calibration” approach to assess research

performances, rather than improving existing bibliometric indices. The main motivation

for our approach is that scientists of different fields and seniorities can have significantly

different citation curves. These differences are important and need to be reflected in the

research evaluation process. Our class of SRMs can take into account those specific features

using flexible performance curves that are calibrated on the data, as discussed in the next

section.

3. Scientific Research Measures

In this section we introduce our class of Scientific Research Measures based on citation curves.

A scientist’s citation curve is the number of citations of each publication, in decreasing order

of citations. Figure 1 shows a citation curve of a hypothetical scientist with 8 publications.

As discussed in the previous section, many existing bibliometric indices are based on citation

curves.

Let X denote the citation curve of an author. We model X as a function X : R+ → R+,

mapping each publication into the corresponding number of citations. By construction, X

is bounded, decreasing, and taking only a finite number of values on R+. As the number of

publications and citations are positive integers, the domain and range for X appear to be

the positive integers,10 rather than the positive reals. However, in the research evaluation

10The theory of SRMs developed in this section also holds when the domain and range of X are positiveintegers. To study the properties of SRMs, we only require X to be bounded and decreasing; for instance Xis never required to be continuous.

8

process, the domain and range of X need not be the positive integers. For example, if a

publication is coauthored by two or more scientists, the evaluator may choose to count that

publication as half or 1/(number of coauthors), when assessing the research performance

of one of the coauthors. The evaluator may also choose to give more weight to citations

of research papers, rather than review papers or books, inducing the range of X to be the

positive reals, rather than positive integers. Thus, we present the theory of SRMs embedding

X in the positive reals.

To assess a scientist’s research performance, we compare the whole citation curve X

with a theoretical performance curve fq. The performance curve represents the required

amount of citations to reach a performance level q. Intuitively, the higher the scientist’s

research performance, the higher the performance curve fq she can reach, and the higher the

corresponding level q is. Formally, the performance curve is a function fq : R+ → R that

associates to each publication x a theoretical number of citations fq(x). The performance

curve fq is assumed to be increasing in q, i.e., if q ≥ r then fq(x) ≥ fr(x) for all x. This

means that the higher the q the higher the research performance level associated to the

corresponding curve fq. Although not necessary for the theory, we assume that fq(x) is

decreasing in x, as any citation curve.

When q varies in a given set I ⊆ R+, we obtain a family of performance curves fqq∈I .

Typically, we will consider I = 1, 2, 3, . . . ⊂ N or I = [1,∞]. Figure 1 shows two families

of performance curves: square-type functions corresponding to the h-index and hyperbolic-

type functions corresponding to a specific SRM that we will use in Section 5. Recall that the

higher the scientist’s research performance, the higher her citation curve X is. Moving from

the origin (0, 0) of the graph to direction North-East the performance level increases, i.e., each

performance curve fq is associated to a higher and higher level q. The 4 performance curves

in Figure 1 associated to the h-index correspond to an h-index of 1, 2, 3 and 4, respectively.

Similarly, each hyperbolic-type performance curve is associated to an increasing level of

research performance.

9

We now introduce the novel class of SRMs, which is defined by the following ϕ-index:

ϕ(X) := sup q ∈ I | X(x) ≥ fq(x), ∀x ∈ [1,∞) (1)

where := denotes definition. The ϕ-index is obtained by comparing the actual citation curve

X and the family of performance curves fqq∈I . Specifically, the ϕ-index is the highest level

q of the performance curve still below the author’s citation curve. Consider the hyperbolic-

type performance curves in Figure 1. The ϕ-index of that citation curve is 24.75 and it

is associated to the performance curve passing through the point (3, 5). In general, the

numerical value of the ϕ-index (24.75 in the above example) has no interpretation. The

numerical values of the ϕ-index of various scientists are simply used to rank their research

performances. However, the ϕ-index can be normalized in order to attach some interpretation

to it.11

The family of performance curves fqq∈I plays a key role in the definition of SRMs.

Performance curves create the common ground or “level the playing field” to assess research

performances and rank scientists. Under the premise that scientist’s research performances

are reflected by the whole citation curves, performance curves should have a similar shape

as citation curves.

If performance curves are unrelated to citation curves, then it is unclear how to interpret

the resulting scientists’ rankings. Moreover, scientists of different research fields and seniori-

ties have obviously different citation curves. The performance curves in equation (1) can be

specified in a flexible way in order to reflect specific features of the scientific field, research

areas and seniority. Unfortunately, most existing bibliometric indices (Table 1 and Figure 2

provide some examples) do not reflect the shape of actual citation curves. Figure 1 suggests

that hyperbolic-type performance curves provide a much better description of the citation

11As discussed in Section 5, the performance curves in Figure 1 are given by fq(x) = q/xc−k. An equivalentrepresentation of these performance curves is fq(x) = αq/xc−k. The parameter α can be used, for example,to normalize to 100 the ϕ-index of the scientist with the largest number of publications or citations, or anyother scientist. This normalization is achieved replacing q by αq in the two-step algorithm developed inSection 5 and determining α using the citation curve of the scientist with the largest number of publicationsor citations.

10

curve X than square-type performance curves. As shown in our empirical application in

Section 5, actual citation curves exhibit hyperbolic-type shapes, calling for hyperbolic-type

performance curves.

An additional argument supporting performance curves that mimic actual citation curves

is the following. In Figure 1, the hypothetical author has an h-index of 4. Even if the 4 most

cited publications, i.e., the publications in the so-called Hirsch core, receive more citations,

by definition the h-index does not change. In contrast, the ϕ-index based on hyperbolic-type

curves may increase, depending on the specific shape of the performance curves and reflecting

the higher research performance of the scientist.12 A similar situation occurs when the least

cited publications have a modest increase in the number of citations. These additional

citations do not increase the h-index but may increase the ϕ-index.

The definition of the ϕ-index in equation (1) is based on performance curves. An equiv-

alent approach to define the ϕ-index is based on performance sets. To visualize these sets,

consider again Figure 1. Intuitively, a performance set is the area above a given perfor-

mance curve. Formally, given the performance curve fq, the associated performance set is

Aq := X : R+ → R+ | X(x) ≥ fq(x),∀x ∈ [1,∞). Since there exists a performance set

associated to each performance curve, when q varies in the set I ⊆ R+, we obtain a fam-

ily of performance sets Aqq∈I , which parallels the family of performance curves fqq∈I .

By definition, Aq is monotone13 and convex for any q and the family of performance sets

Aqq∈I is monotone decreasing. Thus, the ϕ-index in (1) can be equivalently defined as

ϕ(X) := supq ∈ I | X ∈ Aq.12Other bibliometric indices, such as A- and R-index reviewed in Section 2 and summarized in Table 1,increase when the number of citations in the Hirsch core increases. However, A- and R-index are not SRMsbecause they do not take into account the whole scientist’s citation curves and are not characterized byperformance curves, making their applications to scientists of different fields and seniorities non-trivial.13Monotonicity of Aq means that if X1 ∈ Aq and X2 ≥ X1 then X2 ∈ Aq. Convexity of Aq means that ifX1, X2 ∈ Aq then λX1 + (1− λ)X2 ∈ Aq,∀λ ∈ [0, 1]. The family of performance sets Aqq∈I is monotonedecreasing because if q ≥ r then Aq ⊆ Ar.

11

3.1. Existing Bibliometric Indices as Scientific Research Measures

The class of SRM defined in (1) includes many existing bibliometric indices. Different indices

can be recovered by suitably specifying the class of performance curves. For example, the

h-index is characterized by the performance curve fq(x) = q1(0,q](x), where 1(0,q](x) takes

value 1 when x ∈ (0, q] and zero otherwise. The h2-, hα- and w-index, reviewed in Section 2,

fall in the class of SRMs and the respective performance curves are reported in Table 1. The

maximum number of citations, as a bibliometric index, is a SRM whose performance curve is

given by fq(x) = q1(0,1](x). Similarly, the number of publications with at least one citation is

a SRM whose performance curve is given by fq(x) = 1(0,q](x). The rational and real h-index,

also reviewed in Section 2, fall in the class of SRMs. The corresponding performance curves

are the same as those of the h-index, but the q parameter varies in the rational and real

numbers, respectively, rather than in the natural numbers.

In general, any bibliometric index that can be represented using performance curves is a

SRM. Not all bibliometric indices proposed in the literature are SRMs. The ϕ-index in (1)

has three features: it takes into account the whole citation curve, it is defined in terms of

performance curves and, as discussed in the next section, it is monotone and quasi-concave.

Any bibliometric index that does not possess any of these three features is not a SRM. For

example, the g-, A-, R- and m-index, reviewed in Section 2, are not SRMs. The reason

is that these indices do not take into account the whole scientist’s citation curve and thus

cannot be represented in terms of performance curves as in (1).14 The advantage of using

flexible performance curves is to reflect in the research evaluation process specific features

of the scientific field and seniority of scientists, and to allow for a more granular ranking of

scientists. Without resorting to flexible performance curves this task appears to be quite

challenging to achieve.

Figure 2 shows six families of performance curves associated to six different bibliometric

indices, namely the h-, h2-, hα- and w-index, the maximum number of citations, and the

14The tapered h-index takes into account the whole scientist’s citation curve but it does so in an involvedmanner and it is not defined in terms of performance curves.

12

maximum number of publications with at least one citation; see Table 1 for a short description

of the indices. As citation curves have typically hyperbolic shapes, none of these performance

curves provides an adequate description of actual citation curves, making rankings of research

performances based on such bibliometric indices difficult to interpret.

3.2. Monotonicity and quasi-concavity of SRMs

The class of SRMs defined in (1) has two desirable properties, namely monotonicity and

quasi-concavity. The first property simply implies that better scientists, as reflected by

their citation curve, have higher ϕ-index. Specifically, if Scientist 1 has a lower research

performance than Scientist 2, i.e., X1 ≤ X2, then ϕ(X1) ≤ ϕ(X2). Thus, monotonicity of

the ϕ-index is well justified.15

If SRMs were characterized by monotonicity only, they would have been of limited use.

The reason is that the monotonicity property would only allow to rank scientists whose

citation curves satisfy a dominance order, like X1 ≤ X2 in the previous example. In many

cases, the evaluator would need to rank scientists whose citation curves are not ordered or

intersect each other. Quasi-concavity of the ϕ-index allows to rank scientists in such more

complex but realistic situations. Formally, quasi-concavity is expressed as follows: given two

citation curves X1 and X2, quasi-concavity of the ϕ-index implies that, for all λ ∈ [0, 1],

ϕ(λX1 + (1− λ)X2) ≥ min(ϕ(X1), ϕ(X2)). (2)

Quasi-concavity implies that a convex combination of two citation curves leads to a per-

formance level which is at least as large as the research performance of the less performing

scientist. In other words, when the citation curve of a less performing scientist is combined

with the citation curve of a better performing scientist, the resulting research performance

is at least as good as that of the less performing scientist. This appears to be a natural

15As pointed out by an anonymous referee, other criteria of monotonicity could be adopted. For example,one could adopt the following weak form of majorization: X1 ≼ X2 ⇔

∑ni=1 X1(xi) ≤

∑ni=1 X2(xi), for all

n ∈ N; see Marshall, Olkin, and Arnold (2011). This criterion would lead to a quite different theory of SRMsand we defer this topic to future research.

13

property of the SRM. Appendix A shows that the ϕ-index in (1) satisfies (2), i.e., it is

quasi-concave. Moreover, quasi-concavity ensures an internal consistency or coherency of

scientists’ rankings. Appendix A provides a numerical example to illustrate this point.

4. Dual Scientific Research Measures

In this section we introduce a further general class of research measures, called Dual SRMs.

To see the importance of this class, consider the following example. Two scientists have two

publications each and the same total number of citations. The first scientist has 10 citations

from her publication in journal A and zero citations from her publication in journal B. The

second scientist is in the opposite situation, i.e., she has zero citations from her publication

in journal A and 10 citations from her publication in journal B. As the distribution of

citations is the same for both scientists, any bibliometric index based on citation curves

would imply that the two scientists have the same research performance. If journal A is

more prestigious than journal B, then the first scientist has a higher research performance

than the second scientist. The Dual SRM allows to discriminate the research performance

of the two scientists and rank them correctly. The reason is that the Dual SRM evaluates

the scientist’s citation record. The citation record is the random variable that associates to

each journal the citations of the scientist’s publications in that journal.16 Citation records

contain significantly more information than citation curves. Consequently, the Dual SRM

allows for a more informed ranking of research performances. Of course, to implement this

approach a richer dataset for each scientist is needed, which includes citations, in which

journals publications appeared, and quality of the journals.

The Dual SRM also allows to aggregate scientists’ research outputs in a sensible way.

The reason is that the Dual SRM aggregates citation records, i.e., citations of publications

of different scientists appeared in the same journal, not citation curves. This in turn allows

to rank research teams, departments and academic institutions in a sensible way, which has

16Any scientist may have published more than one paper in the same journal and Dual SRMs account forthis situation, as discussed in Section 4.2.

14

been a central theme in the scientific community over the last decades.

We first provide a numerical example and then present the theory of Dual SRMs.

4.1. Numerical Example of Dual SRMs

To illustrate the derivation of the Dual SRM we use a simple numerical example. We consider

an evaluator who needs to rank scientists’ research performances.

The first step is to select the evaluation criteria of the journals. The evaluator needs to

quantify the “value” of the journals in which the publications appeared. This is obviously

a challenging task, but it is also a necessary step to achieve a sensible ranking of scientists

with the same or similar distributions of citations, and publications in different journals.

Consider the following three journals: Management Science (MS), Mathematical Finance

(MF) and Finance and Stochastics (FS). One possibility to assess a journal’s value is to use

the 1-year and 5-year journal’s impact factor.17 For MS, MF and FS, the 1-year impact

factors are 1.73, 1.25, and 1.15, and the 5-year impact factors are 3.30, 1.66, and 1.58,

respectively.18 The information in each impact factor is collected in a probability measure,

Q1 and Q5, respectively, on the journal space Ω = MS,MF,FS. For example, Q1(MS) =

1.73/(1.73 + 1.25 + 1.15), and similarly for the other journals and impact factor.

Consider a scientist with three publications, one in each journal, MS, MF, and FS, with

10, 6, and 8 citations each. These citations are collected in her citation record X(ω), where

ω = MS,MF,FS. Note that the citation record is not necessarily a citation curve, as values

of X(ω) need not be in decreasing order. In the current example these values are 10, 6, and

8, respectively.

The second step is to assess the scientist’s research performance under each criterion

or probability measure Q1 and Q5. Under Q1 the research performance of a scientist with

17Other criteria are obviously conceivable to assess journals’ quality, such as overall citations of the journalor acceptance rate of journal submissions. Such additional criteria can be easily accounted for in the DualSRM. For illustration purposes, we only consider two impact factors for each journal in this example.18The impact factors were obtained from the website of each journal in 2012. The impact factor was devisedin the mid 1950’s by Eugene Garfield. See Garfield (2005) for a recent account of the impact factor.

15

citation record X(ω) is defined as

β(Q1, X) := supq ∈ R | EQ1 [X] ≥ γ(Q1, q). (3)

Using citation record and impact factor, the value of EQ1 [X] can be easily computed. In the

example EQ1 [X] = 10 Q1(MS) + 6 Q1(MF) + 8 Q1(FS) = 8.2.

By definition, γ(Q1, q) represents the smallest Q1-average of citations to reach a perfor-

mance level of q. Intuitively, if the evaluator deems the criterion Q1 as highly important,

then γ(Q1, q) will be large, and it will be a challenging task for each scientist to do well

under that criterion. Moreover, if Q1 is more important than Q2, then γ(Q1, q) > γ(Q2, q).

Suppose the evaluator is indifferent between the criteria, Q1 and Q5. This ranking of the

journal evaluation criteria can be expressed by a constant γ(Q, q) function, i.e., γ(Q, q) := q,

for any Q. Thus, under Q1 the assessment of the scientist’s research performance is

β(Q1, X) := supq ∈ R | EQ1 [X] ≥ q = EQ1 [X] = 8.2.

If measure Q1 was the only measure to assess journal quality, then β(Q1, X) = 8.2 was the

scientist’s final research performance. Since measure Q5 is another journal criterion, the eval-

uator computes the scientist’s research performance also under Q5, obtaining a corresponding

index of β(Q5, X). In our example, EQ5 [X] = 10 Q5(MS)+ 6 Q5(MF)+ 8 Q5(FS) = 8.5. As

γ(Q5, q) := q constant, β(Q5, X) = EQ5 [X] = 8.5.

Finally, the evaluator takes the smallest value between β(Q1, X) and β(Q5, X). The Dual

SRM gives the scientist’s final research performance as

Φ(X) := min(β(Q1, X), β(Q5, X)) = min(EQ1 [X], EQ5 [X]) = 8.2.

The example shows that the Dual SRM leads to a conservative or “worst case scenario”

assessment of the research performance. In other words, the evaluator assess the scientist’s

research performance using the least favorable journal’s evaluation criterion. In general, there

16

will be no consensus on the journals’ evaluation criteria. Taking a conservative assessment

appears to be a natural approach. This is a key feature of the Dual SRM and will be

further discussed below. To assess the research performance of another scientist the evaluator

repeats the calculations above using her citation record and the same Q1 and Q5 measures.

Comparing the scientists’ Φ-indices gives the scientists’ ranking.

In the example above, the evaluator is indifferent between the criteria Q1 and Q5. We now

extend the example by considering an additional journal evaluation criterion and introducing

a new evaluator who assigns different weights to the three criteria. The additional criterion is

the number of years since the first journal’s edition and expressed by the probability measure

Q0.19 Under Q0, EQ5 [X] = 10 Q0(MS) + 6 Q0(MF) + 8 Q0(FS) = 8.8.

Suppose the evaluator assigns different weights to the three criteria. This decision can

be expressed by choosing the γ(Q, q) function as follows

γ(Q, q) := q α(Q) (4)

where α(Q) ≥ 0 represents the weight that the evaluator assigns to each criterion Q. A

natural range for the weight appears to be the interval [0, 1], but weights larger than 1 are

also admissible. The more important the journal’s evaluation criterion (according to the

evaluator), the higher the α(Q) weight. If a particular criterion is controversial or deemed

unimportant, it can receive a weight of zero.

Table 2 presents different cases. Evaluator 2 believes that criterion Q5 is more important

than criterion Q1 which in turn is more important than criterion Q0. Accordingly, this

evaluator assigns decreasing weights α(Q) to the three criteria. Evaluators 3 and 4 have

different rankings for the criteria, reflected in the respective weights. Evaluator 1 is indifferent

among all the three criteria, and assigns weight 1 to all of them. In the initial example,

the evaluator used only Q1 and Q5 and was indifferent between the two criteria, and thus

assigned α(Q1) = α(Q5) = 1 and α(Q0) = 0. Clearly, γ(Q, q) := q can be recovered by

19The first edition of MS, MF and FS appeared in 1954, 1991 and 1996, respectively, which implies Q0(MS) =(2012− 1954)/95 = 0.61, Q0(MF) = (2012− 1991)/95 = 0.22 and Q0(FS) = (2012− 1996)/95 = 0.17.

17

setting α(Q) = 1 in (4). The key aspect of the γ(Q, q) function is that it is selected by the

evaluator ex-ante and independently of any scientist’s citation record.

As in the previous example, the evaluator determines the scientist’s research performance

under the different criteria Q0, Q1 and Q5. Under Q0 the research performance of a scientist

with citation record X(ω) is now defined as

β(Q0, X) := supq ∈ R | EQ0 [X] ≥ q α(Q0) =EQ0 [X]

α(Q0)(5)

and similarly under Q1 and Q5.

The Dual SRM that provides the scientist’s final research performance is

Φ(X) := min(β(Q0, X), β(Q1, X), β(Q5, X)) = min

(EQ0 [X]

α(Q0),EQ1 [X]

α(Q1),EQ5 [X]

α(Q5)

). (6)

Different weights α(Q) lead to different Dual SRMs. Table 2 shows the research perfor-

mances of the hypothetical scientist when different evaluators use different Dual SRMs, i.e.,

assign different weights to the journal evaluation criteria.

As mentioned above, quantifying journals’ quality is a necessary step to rank scientists

with publications in different journals. A priori there will be no consensus on journals’ eval-

uation criteria. Taking the minimum in (6) naturally translates the evaluator’s prudential

assessment of the scientist’s research performance. Furthermore, there also exists a theoreti-

cal reason for taking the minimum in (6) that comes from the duality theory and is discussed

in the next section.

4.2. Theoretical Derivation of Dual SRMs

In this section we present the theory of the Dual SRM, which requires additional mathemat-

ical structures.

We consider the scientist’s citation record, rather than her citation curve. Formally, the

citation record is a random variable X(ω) defined on the events ω ∈ Ω, where each event ω

corresponds to the journal in which the publication appeared and Ω is the set of journals.

18

Thus, X(ω) is the number of citations of the papers published in journal ω.20 The primal

space is given by the set of all possible scientist’s citation records.

We now fix a family of probabilities P defined on Ω, where for each Q ∈ P , Q(ω)

represents the “value” of journal ω ∈ Ω. The dual space is given by all possible linear

valuation of the journals, i.e., the “Arrow–Debreu prices” of each journal. As mentioned

above, the journals’ evaluations, namely the selection of the family P , are determined a

priori and could be based on various criteria. For example, a specific measure Q can assign

high value to journals with high impact factor, while another measure Q can assign high value

to journals with low acceptance rate of submissions. A priori there will be no consensus on

how to evaluate a journal and this is reflected in the set of Q ∈ P rather than just a single

probability measure.

Then we select a family of functions γ(Q, q)q∈R. Each function γ(·, q) : P → R asso-

ciates to each measure Q the value γ(Q, q), representing the smallest Q-average of citations

which is necessary to reach a research performance level q. This research performance as-

sessment is under the measure Q and the function γ(Q, q) does not depend on the scientist’s

citation record X. The Dual SRM is given by the Φ-index

Φ(X) := infQ∈P

β(Q,X) (7)

where β(Q,X) := supq ∈ R | EQ[X] ≥ γ(Q, q) represents the research performance of

a scientist with citation record X, under the fixed measure Q. In general, the function

γ(Q, q) can be nonlinear in q, perhaps reflecting different scientist’s effort to reach a certain

performance level. The main feature of γ(Q, q) is to be non decreasing in q. In the previous

20Over the years any scientist may have published more than one paper in the same journal. A simpleapproach to account for this situation is to consider her total number of citations of all publications in thegiven journal. Indeed, if the journal evaluation criterion, say Qi, does not change over time, the total numberof citations per journal is the only relevant quantity when assessing the scientist’s research performance asEQi [X] is a linear operator. To simplify the exposition we implicitly adopted this approach in the currentsection. If instead the journal evaluation criterion changes over time (e.g., the impact factor of the journal),the citations of publications in a given year can be evaluated using the journal evaluation criteria for thatyear. Hence, citations of publications in different years can be evaluated using different journal evaluationcriteria.

19

section, the γ(Q, q) is either constant or specified as in (4). The Φ-index is a conservative or

prudential assessment of a scientist’s research performance, as it is obtained under the least

favorable journal’s evaluation criterion.

For each measure Q, the function β(Q,X) in (7) resembles equation (1) defining the ϕ-

index. Indeed, to derive the ϕ-index the citation curve X(x) is compared with performance

curves fq(x). In the dual setting, Q-averages of citation records X(ω), i.e., EQ[X], are

compared with the smallest Q-average of citations to reach a performance level q, i.e., the

function γ(Q, q). The key difference between the two approaches is that the set of functions

β(Q,X) are derived under “different scenarios” Q ∈ P characterized by different criteria to

assess journals’ quality. This feature characterizes the Dual SRM and it is not present in

the SRM or any other bibliometric index based on citation curves.

By construction any Dual SRM in (7) is monotone increasing and quasi-concave, as

proved in Proposition 2 in Appendix B. Both properties have a natural motivation. The

monotonicity property implies that better scientists have higher research performances. The

quasi-concavity induces sensible rankings of research centers. The citation records of two

scientists can be meaningfully aggregated by taking their linear convex combination λX1 +

(1 − λ)X2, where λ ∈ [0, 1].21 The aggregation is meaningful because it is at the level of

citation records, i.e., scientists’ citations of publications in the same journal. Notably, quasi-

concavity implies that the performance of the research team is at least equal to the scientific

performance of the less performing member, which appears to be a natural property.

An important question is the following: Does any monotone increasing and quasi-concave

map admit the representation in (7)? Essentially, the answer is positive and formalized in

Theorem 1. In order to prove the theorem, we need to assume the continuity from above of

Φ and to introduce some new notations.

Let (Ω,F , P ) be a probability space, where F is a σ-algebra and P is a probability

measure on F . Since the citation record of an author X is a bounded function, it appears

natural to take X ∈ L∞(Ω,F , P ), where L∞(Ω,F , P ) is the vector space of F -measurable

21If λ = 0.5, the convex combination is simply the arithmetic mean of the two citation records. To aggregaten scientists’ citation records one computes

∑ni=1 λiXi, where

∑ni=1 λi = 1 and λi ≥ 0 for all i.

20

functions that are P almost surely bounded. We also denote with ∆ := Q ≪ P the set of

all probabilities absolutely continuous with respect to P .

Theorem 1 Any map Φ : L∞(Ω,F , P ) → R that is quasi-concave, monotone increasing

and continuous from above can be represented as

Φ(X) := infQ∈∆

β(Q,X), ∀X ∈ L∞(Ω,F , P ) (8)

where β : ∆× L∞(Ω,F , P ) → R and γ : ∆× R → R are defined as

β(Q,X) := sup q ∈ R | EQ[X] ≥ γ(Q, q)

γ(Q, q) := infY ∈L∞

EQ[Y ] | Φ(Y ) ≥ q . (9)

The proof is in Appendix B.22

There are two technical differences between the ϕ-index in (1) and the Φ-index in (7).

The first difference is that the ϕ-index is law invariant while the Φ-index may not. If two

scientists have the same citation curve, then they have the same ϕ-index. If they published

in different journals, in general they have different Φ-indices. Thus, the Φ-index can rank

scientists with the same citation curves.23 The second difference is that the ϕ-index is not

necessarily quasi-concave on the citation records (even if it is always quasi-concave on the

citation curves). When the ϕ-index is quasi-concave on the citation records, the Φ-index

includes the ϕ-index as a special case.

22In equation (8) the infimum is taken with respect to all probabilities in ∆, and not in a prescribed subsetP ⊆ ∆. This must be the case, since in the theorem the map Φ is given a priory and only the objects thatcan be determined from Φ can appear in the representation (8).23Frittelli, Maggis, and Peri (2013) provide a theoretical analysis of quasi-convex maps defined on distribu-tions.

21

5. Empirical Analysis

This section provides an empirical application of our SRM, discusses its implementation,

and compares the empirical findings to 7 existing bibliometric indices.24

5.1. Data

We consider 173 full professors affiliated to 11 main U.S. business schools or finance depart-

ments. Thus, we consider a rather large and homogenous set of researchers, in terms of

seniority and field of research, making comparison meaningful. A Python script is run to

extract their citation curves from Google Scholar (http://scholar.google.com/), which are

publicly and freely available. The American Scientist Open Access Forum (2008) remarked

that “Google Scholar’s accuracy is growing daily, with growing content.”25 Our dataset is

created on September 6, 2012. Table 3 shows the list of business schools or finance depart-

ments that for brevity we call universities. The number of faculty members varies signifi-

cantly across universities, ranging from 8 members at Berkeley and Cornell to 34 members

at Columbia. To provide an overview of the data, Table 3 reports summary statistics for the

number of citations and publications, aggregated at the university level. Chicago has the

largest average number of citations, as well as median and standard deviation, and fourth

largest faculty by size. For all universities, the average number of citations is more than

twice the median number of citations, suggesting that the distribution of citations is highly

positive skewed. Princeton has the largest number of publications, and fifth largest faculty

by size. The average number of publications is often more than twice the median number of

publications.

24As discussed in the previous section, an empirical application of Dual SRMs requires a richer dataset,including in which journal each author’s publications appeared and values of those journals. We defer suchan application to future work.25http://openaccess.eprints.org/index.php?/archives/417-Citation-Statistics-International-Mathematical-Union-Report.html.

22

5.2. Estimates of the Scientific Research Measure

To properly rank research performances, performance curves should mimic actual citation

curves. We experimented different functional forms of performance curves. We emphasize

that the flexibility in the choice of performance curves stems from the fact that our class

of SRMs in (1) accommodates performance curves with largely different shapes. It turns

out that a sensible specification of performance curves is given by the hyperbolic function

fq(x) = q/xc − k, trading off fitting accuracy and parametric complexity of the curve.

Indeed, all citation curves in our dataset display hyperbolic-type shapes. Figure 3 shows

four typical citation curves. Superimposed to each citation curve is the fitted hyperbolic

function. The fitting appears to be accurate; indeed all adjusted R2’s are above 97.4%. We

fit the hyperbolic function fq(x) = q/xc−k using nonlinear least squares, i.e., by minimizing∑px=1(X(x)−fq(x))

2 with respect to q, c and k, where p is the author’s number of publications

with at least one citation each. Starting values for the nonlinear minimization are obtained

as follows. We note that fq(x) = q/xc is a restricted version of fq(x), imposing k = 0. As

in the log-log space fq(x) is linear in log(x), i.e., log(fq(x)) = log(q) − c log(x), the two

parameter estimates ˆlog(q) and c can be easily obtained by regressing the log number of

citations on a constant and the log number of publications, i.e., by running the least square

regression of log(X(x)) on 1 and log(x), where x = 1, 2, . . . , p. As starting values for q, c

and k we take exp( ˆlog(q)), c and 0, respectively.

For each author in our dataset, we fit the hyperbolic function to its citation curve, as

described above. Table 4 summarizes the estimation results. For all regressions, t-statistics

are significantly different from zero. All adjusted R2 are above 95%, suggesting that hy-

perbolic functions provide an accurate description of citation curves. We point out that

estimates of the q parameter in Table 4 are not research performance levels. The reason is

that the corresponding hyperbolic functions pass through the citation curves, and not below

the citation curves as required by (1).

23

Given the evidence above, we use the following ϕ-index to rank the authors in our dataset:

ϕ(X) := sup q ∈ R | X(x) ≥ q/xc − k, ∀x ∈ [1,∞) (10)

where we set c = 0.59 and k = 543.2, which are the weighted averages of all estimated c and

k coefficients in our sample, respectively, with weights given by the number of data points

used to estimate each coefficient. We also experimented other values of c and k, such as the

simple average of all estimated c and k coefficients. The empirical results are quite similar

and available from the authors upon request. The ϕ-index in (10) provides the common

ground to rank all authors in our dataset.

The following two-step algorithm can be used to compute the ϕ-index in (10). Given an

author with p publications with at least one citation each:

1. Compute qx := (X(x) + k)xc, for x = 1, 2, . . . , p, and q∗ := k pc.

2. Take ϕ(X) := min(q1, q2, . . . , qp, q∗).

By construction, the performance curve associated to qx passes through the point (x,X(x)).

This holds true for each x = 1, 2, . . . , p.26 The performance curve associated to q∗ crosses

the horizontal axis at (p, 0), i.e., fq∗(p) = 0. To better understand the two-step algorithm,

consider again Figure 1. The figure shows the p = 8 performance curves for the hypothetical

author with 8 publications associated to q1, . . . , qp. The performance curve associated to q∗,

not shown in Figure 1, would pass through the point (8, 0). Taking the minimum among all

the qx’s and q∗ (which is q3 = 24.75 in Figure 1) ensures that the associated performance

curve is below all the other performance curves (as performance curves are increasing in q)

and consequently it is also below the citation curve X(x). At the same time, it is also the

highest performance curve still below the citation curve, as it is touching the citation curve

in one point (which is (3, 5) in Figure 1). The performance curve associated to q∗ enters the

26For example, the performance curve associated to qp is fqp(x) = qp/xc − k. When evaluated at x = p, it

gives fqp(p) = qp/pc − k = (X(p) + k) pc/pc − k = X(p), showing that this performance curve is passing

through the point (p,X(p)).

24

calculation of the ϕ-index because the whole citation curve may lay above the performance

curve fq∗ , and in this case fq∗ determines the ϕ-index in (10). This may happen when the

citation curve is roughly linear and steep. Although in practice citation curves do not appear

to have this shape, the performance curve fq∗ needs to be considered when computing the

ϕ-index in (10).27

5.3. Empirical Results

For each of the 173 authors in our dataset, we compute the ϕ-index in (10) and the other

7 bibliometric indices listed in Table 1. To provide an overview of the results, we aggregate

estimates of the ϕ-index and bibliometric indices at the university level. Table 5 shows

mean and median for each index and university. Averages are often significantly larger than

medians (e.g., for Princeton and Stanford), implying that the distribution of the indices is

positively skewed. Thus, aggregated results should be interpreted cautiously.

The main message from Table 5 is that rankings of universities based on our ϕ-index and

the other bibliometric indices are generally different. Under the premise that the research

performance is reflected by the whole citation curve, rather than by only part of it, our

ϕ-index provides by construction a better assessment of research performances than the

other bibliometric indices, leading to a more sensible ranking. Differences in rankings are

relatively more concentrated on “average” performing universities, rather than in most and

least performing universities, in relative terms. For example, according to the average ϕ-

index, Chicago ranks first and Princeton ranks second. According to the average h-index,

Princeton ranks first and Chicago ranks second. Berkeley ranks tenth according to both

averages. For the remaining universities, the rankings provided by the ϕ- and h-index, as

well as the other indices, are quite different.

Empirical results at the level of individual full professors parallel the empirical results at

27More specifically when x = p, X(p) ≥ 1 > 0 = fq∗(p), which suggests that the citation curve X(x) may layabove fq∗(x) for all x when the citation curve is roughly linear and steep. As the citation curve X(x) = 0for all x > p and the performance curves are increasing in q, it is not necessary to consider citation curvesfor q > q∗, as all these performance curves lay above fq∗ and thus lay above the citation curve X(x) at leastfor all x > p.

25

the university level. Because of space constraints, we cannot report estimates of all indices

for all 173 full professors. Nonetheless, to give a sense of the empirical findings, Figure 4

shows the scatter plots of the ϕ-index versus the h-index and versus the g-index for each

scholar in our sample. Scatter plots of the ϕ-index versus the other bibliometric indices

(not reported) share similar patterns. The relation between the indices is clearly positive.

Best performing scholars tend to have the largest ϕ- and h-index (or g-index). Scholars in

the top-right corner of the scatter plots are typically Nobel laureates in economics. This

obviously induces a positive correlation between the indices. However, the relation is far

from monotone, meaning that different indices lead to different rankings. A closer look at

the scatter plots shows that in various cases, different scholars have the same (or almost

the same) h-index (plotted on the vertical axis of upper graphs), while their ϕ-indices are

different. This implies that the h-index cannot be used to rank them, while the ϕ-index

provides a more granular ranking of scholars. This situation occurs for example when two

authors have roughly the same number of publications (and h-index) but one author has

more citations per publication, especially in the Hirsch core, which implies a steeper citation

curve to the left and thus higher ϕ-index. As a consequence of this phenomenon, at the

university level the median h-index (rounded to the nearest integer) is 29 both for Cornell

and Yale, and 25 both for Berkeley and Penn. In contrast, the corresponding median ϕ-index

is quite different for these universities, as shown in Table 5.

To summarize, this empirical application shows that our ϕ-index can be easily imple-

mented and in general leads to different rankings than other bibliometric indices.

6. Conclusion

Funding research projects, faculty recruitment, and other key decisions for universities and

research institutions depend to a large extent upon the scientific merits of the involved

researchers. This paper introduces the novel class of Scientific Research Measures to rank

scientists’ research performances. The main feature of SRMs is to take into account the

26

whole scientists’ citation curves and to reflect specific features of the scientific field, research

areas, and seniority of the scientists. This task is achieved by using flexible performance

curves. Most existing bibliometric indices do not appear to have such a flexibility.

We also introduce the further general class of Dual SRMs that allows for a more informed

ranking of research performances using citation records. Dual SRMs allow to aggregate

scientists’ research outputs and induce sensible rankings of research teams, departments and

universities, which has been a central theme in the scientific community over the last decades.

Finally, we provide an empirical application of the SRM using on a novel dataset of 173

finance scholars’ citation curves. We develop a simple two-step algorithm to compute the

SRM. The empirical results show that hyperbolic performance curves describe well actual

citation curves. Authors’ ranking based on the SRM characterized by hyperbolic performance

curves and authors’ rankings based on other 7 existing bibliometric indices are generally quite

different. Under the premise that the research performance is reflected by the whole citation

curve, rather than by only part of it, the SRM provides by construction a more accurate

ranking of research performances.

A. Quasi-concavity of Scientific Research Measures

In this section we provide a formal discussion of the quasi-concavity property of SRMs. The

following proposition states that the SRM defined in (1) is quasi-concave by construction.

Proposition 1 Let fqq be a family of performance curves and ϕ be the associated SRM

defined in (1). Then ϕ is quasi-concave: for all citation curves X1 and X2

ϕ(λX1 + (1− λ)X2) ≥ min(ϕ(X1), ϕ(X2)), λ ∈ [0, 1].

Proof. Let m := min(ϕ(X1), ϕ(X2)), so ϕ(X1) ≥ m and ϕ(X2) ≥ m. By definition of ϕ,

∀ε > 0 ∃qi s.t. Xi ≥ fqi and qi > ϕ(Xi)− ε ≥ m− ε. Then Xi ≥ fqi ≥ fm−ε, as fqq is an

increasing family, and therefore λX1 + (1 − λ)X2 ≥ fm−ε. As this holds for any ε > 0, we

27

conclude that ϕ(λX1 + (1− λ)X2) ≥ m and ϕ is quasi-concave. As mentioned in Section 3.2, the reason to adopt quasi-concavity is an internal consistency

or coherency property that we now explain with an example.

Consider the citation curves of two scientists A and B, having 3 publications each and

the same total number of citations:

A =

30

20

10

, B =

20

20

20

.

The citations of A are more “concentrated” while the citations of B are less concentrated or

more “diversified,” with the obvious intuition of the meaning of these terms.

We evaluate A and B using a ϕ-index associated to the family of performance curves

fqq. Depending on the performance curves fqq that reflect the specific features of their

scientific field, research areas, seniorities, etc., one could obtain either

ϕ

30

20

10

≥ ϕ

20

20

20

(11)

which means that the SRM assigns more importance to concentration, or

ϕ

20

20

20

≥ ϕ

30

20

10

(12)

which on the contrary means that the SRM attributes more relevance to diversification.

These two different rankings do not depend on the property of quasi-concavity but on the

features of the performance curves fqq, that in turn reflect the characteristics of the cita-

28

tions of the particular field. Both rankings (11) or (12) are possible.28

Now consider the following three citation curves:

C1 =

22

20

18

; C2 =

25

20

15

; C3 =

28

20

12

.

These three citations curves have the same total number of citations as A and B, but the

citations of C1, C2 and C3 are more concentrated than B. Notice that none of the five

citation curves A, B, C1, C2 and C3 dominates any other. Quasi-concavity will induce a

conditional ranking among them that we now illustrate. Note that the h-index is 3 for all

scientists, so it would not induce any ranking.

Case I: Suppose that the SRM attributes more importance to concentration, meaning

that (11) is satisfied. Then we should expect that:

ϕ

22

20

18

≥ ϕ

20

20

20

, ϕ

25

20

15

≥ ϕ

20

20

20

, ϕ

28

20

12

≥ ϕ

20

20

20

. (13)

This natural and minimal coherency property (the coherency is between (11) and (13)) is

satisfied because of the quasi-concavity of ϕ. In other words, the quasi-concavity of ϕ implies

28For example, if fq(x) = −10x + 10 q, q ∈ N, then ϕ(A) = 4 > 3 = ϕ(B). As another example, iffq(x) = 5 q 1(0,q](x), q ∈ N, then ϕ(B) = 4 > 2 = ϕ(A).

29

that (13) holds true whenever (11) is satisfied.29 Indeed,

λ

30

20

10

+ (1− λ)

20

20

20

is equal to C1, C2, C3, respectively, for λ = 0.2; λ = 0.5; λ = 0.8; and so

ϕ

λ

30

20

10

+ (1− λ)

20

20

20

≥ min

ϕ

30

20

10

, ϕ

20

20

20

= ϕ

20

20

20

.

In the above expression the inequality is due to quasi-concavity and the equality to (11).

Case II: Suppose that the SRM attributes more importance to diversification, meaning

that (12) is satisfied. Note that the citations of C1, C2 and C3 are less concentrated than A.

Using the same argument as above, we see that quasi-concavity implies:

ϕ

22

20

18

≥ ϕ

30

20

10

, ϕ

25

20

15

≥ ϕ

30

20

10

, ϕ

28

20

12

≥ ϕ

30

20

10

. (14)

as we should expect from a SRM that emphasizes diversification.

29Quasi-convexity of ϕ would be that ϕ(λX1 + (1− λ)X2) ≤ max(ϕ(X1), ϕ(X2)). This property of ϕ wouldinduce quite different internal consistency conditions that are much less relevant than those induced byquasi-concavity. This is mainly due to the monotonicity property of the ϕ-index, which reflects the naturalrequest that a better scientist should have a higher ϕ-index. The monotonicity property provides a naturaldirection to compare citation curves. Specifically, we are interested in conditions guaranteeing that a citationcurve is “at least as good as” another citation curve, which are precisely the type of conditions in (13) or(14). In contrast, the quasi-convexity of ϕ would only imply that a citation curve is “at most as good as”another citation curve, that is:

(11) ⇒ ϕ (Ci) ≤ ϕ (A) , i = 1, 2, 3(12) ⇒ ϕ (Ci) ≤ ϕ (B) , i = 1, 2, 3.

30

The conclusion about the selection of the quasi-concavity property is the following: given

some citations curves and a ranking among them (ranking which depends on the specific

features of the performance curves of that particular field) quasi-concavity is a coherent

requirement – in terms of concentration or diversification – for the many other citation

curves that can be obtained by convex combination of the given ones.

B. Proof of Dual Representation of SRMs

As a direct consequence of its definition in (7), any Dual SRM is monotone increasing and

quasi-concave. We use the notation introduced before the statement of Theorem 1.

Proposition 2 Given a subset of probabilities P ⊆ ∆ and a map γ(Q, q) : P × R → R let

β(Q, t) := supq ∈ R | t ≥ γ(Q, q) (15)

Φ(X) := infQ∈P

β(Q,EQ[X]) = infQ∈P

β(Q,X). (16)

Then the map Φ : L∞(Ω,F , P ) → R is quasi-concave and monotone increasing.

Proof. Notice that Φ may be equivalently expressed using the map β or the map β, as it

is explicitly written in (16). One immediately also checks that the function β(Q, ·) : R → R

is monotone increasing. As any monotone function from R to R is also quasi-concave, we

obtain that, for each Q ∈ P and X1, X2 ∈ L∞(Ω,F , P ),

β(Q,EQ[λX1 + (1− λ)X2]) = β(Q, λEQ[X1] + (1− λ)EQ[X2])

≥ min(β(Q,EQ[X1]), β(Q,EQ[X2])

).

Taking the infimum with respect to Q ∈ P on both sides and exchanging the infimum and the

minimum, we deduce that also Φ : L∞(Ω,F , P ) → R is quasi-concave. The monotonicity

of β(Q, ·) and of the map EQ[·] : L∞(Ω,F , P ) → R guarantee the monotonicity of Φ :

L∞(Ω,F , P ) → R.

31

The goal of this section is the answer, anticipated in Theorem 1, to the reverse im-

plication of the above Proposition: does any quasi-concave monotone increasing map Φ :

L∞(Ω,F , P ) → R admit a representation in the form (16)? Before proving Theorem 1, we

need some topological structure. For simplicity we denote L∞ := L∞(Ω,F , P ) and with

L1 := L1(Ω,F , P ) the space of integrable random variables.

If we endow L∞ with the weak topology σ(L∞, L1) then L1 = (L∞, σ(L∞, L1))′ is its

topological dual. In the dual pairing (L∞, L1, ⟨·, ·⟩) the bilinear form ⟨·, ·⟩ : L∞ × L1 → R

is given by ⟨X,Z⟩ = E[ZX]. The linear function X 7→ E[ZX], with Z ∈ L1, is σ(L∞, L1)

continuous and (L∞, σ(L∞, L1)) is a locally convex topological vector space.

Definition 1 A map Φ : L∞ → R is σ(L∞, L1)-upper semicontinuous if the upper level sets

X ∈ L∞ | Φ(X) ≥ q are σ(L∞, L1)-closed for all q ∈ R.

For the class of maps that we are considering, σ(L∞, L1)-upper semicontinuity is equiv-

alent to the continuity from above. This fact can be proved in a similar way to the convex

case; see, e.g., Follmer and Schied (2004).

Lemma 1 Let Φ : L∞ → R be quasi-concave and monotone increasing. Then the following

are equivalent:

Φ is σ(L∞, L1)-upper semicontinuous;

Φ is continuous from above: Xn, X ∈ L∞ and Xn ↓ X imply Φ(Xn) ↓ Φ(X).

Proof. Let Φ be σ(L∞, L1)-upper semicontinuous and suppose that Xn ↓ X. The

monotonicity of Φ implies Φ(Xn) ≥ Φ(X) and Φ(Xn) is decreasing and therefore q :=

limn Φ(Xn) ≥ Φ(X). Hence Φ(Xn) ≥ q and Xn ∈ Bq := Y ∈ L∞ | Φ(Y ) ≥ q which is

σ(L∞, L1)-closed by assumption. As the elements in L1 are order continuous, from Xn ↓ X

we get Xnσ(L∞,L1)−→ X and therefore X ∈ Bq. This implies that Φ(X) = q and that Φ is

continuous from above.

Conversely, suppose that Φ is continuous from above. We have to show that the convex

set Bq is σ(L∞, L1)-closed for any q. By the Krein Smulian Theorem it is sufficient to

32

prove that C := Bq ∩ X ∈ L∞ | ∥ X ∥∞≤ r is σ(L∞, L1)-closed for any fixed r > 0. As

C ⊆ L∞ ⊆ L1 and as the embedding

(L∞, σ(L∞, L1)) → (L1, σ(L1, L∞))

is continuous it is sufficient to show that C is σ(L1, L∞)-closed. Since the σ(L1, L∞) topology

and the L1 norm topology are compatible, and C is convex, it is sufficient to prove that C

is closed in L1. Take Xn ∈ C such that Xn → X in L1. Then there exists a subsequence

Ynn ⊆ Xnn such that Yn → X a.s. and Φ(Yn) ≥ q for all n. Set Zm := supn≥m Yn ∨X.

Then Zm ∈ L∞, since Ynn is uniformly bounded, and Zm ≥ Ym, Φ(Zm) ≥ Φ(Ym) and Zm ↓

X. From the continuity from above we conclude: Φ(X) = limmΦ(Zm) ≥ lim supmΦ(Ym) ≥

q. Thus X ∈ Bq and consequently X ∈ C. Now we provide a dual representation of SRMs in the same spirit as Volle (1998), Cerreia-

Vioglio, Maccheroni, Marinacci, and Montrucchio (2011), and Drapeau and Kupper (2013).

We first provide the representation of Φ in Theorem 1 in terms of the dual function H

defined below in (17) and then we show that Φ can also be represented in terms of the right

continuous version H+ (defined in (20)) of H, which can be written in a different way as in

(15) or (21).

Proof of Theorem 1.

Step 1 We show that

Φ(X) = infZ∈L1

+

H(Z,E[ZX])

where H : L1 × R → R is defined by

H(Z, t) := supξ∈L∞

Φ(ξ) | E[Zξ] ≤ t . (17)

Fix X ∈ L∞. As X ∈ ξ ∈ L∞ | E[Zξ] ≤ E[ZX], by the definition of H(Z,E[ZX])

33

we deduce that, for all Z ∈ L1, H(Z,E[ZX]) ≥ Φ(X), hence

infZ∈L1

H(Z,E[ZX]) ≥ Φ(X). (18)

We prove the opposite inequality. Let ε > 0 and define the set

Cε := ξ ∈ L∞ | Φ(ξ) ≥ Φ(X) + ε .

As Φ is quasi-concave and σ(L∞, L1)-upper semicontinuous (by Lemma (1)), C is

convex and σ(L∞, L1)-closed. Since X /∈ Cε, (if Φ(X) = −∞, we may take CM :=

ξ ∈ L∞ | Φ(ξ) ≥ −M and the following argument would hold as well) the Hahn–

Banach theorem implies the existence of a continuous linear functional that strongly

separates X and Cε, that is there exist Zε ∈ L1 such that

E[Zεξ] > E[ZεX] ∀ξ ∈ Cε. (19)

Hence ξ ∈ L∞ | E[Zεξ] ≤ E[ZεX] ⊆ Ccε := ξ ∈ L∞ | Φ(ξ) < Φ(X) + ε and from (18)

Φ(X) ≤ infZ∈L1

H(Z,E[ZX]) ≤ H(Zε, E[ZεX])

= sup Φ(ξ) | ξ ∈ L∞ and E[Zεξ] ≤ E[ZεX]

≤ sup Φ(ξ) | ξ ∈ L∞ and Φ(ξ) < Φ(X) + ε ≤ Φ(X) + ε.

Therefore, Φ(X) = infZ∈L1 H(Z,E[ZX]). To show that the inf can be taken over the

positive cone L1+, it is sufficient to prove that Zε ⊆ L1

+. Let Y ∈ L∞+ and ξ ∈ Cε. Given

that Φ is monotone increasing, ξ + nY ∈ Cε for every n ∈ N and, from (19), we have:

E[Zε(ξ + nY )] > E[ZεX] ⇒ E[ZεY ] >E[Zε(X − ξ)]

n→ 0, as n → ∞.

As this holds for any Y ∈ L∞+ we deduce that Zε ⊆ L1

+. Therefore, Φ(X) =

34

infZ∈L1+H(Z,E[ZX]).

Step 2 We show that

infZ∈L1

+

H(Z,E[ZX]) = infZ∈L1

+

H+(Z,E[ZX])

where H+(Z, ·) is the right continuous version of H:

H+(Z, t) := infs>t

H(Z, s). (20)

Since H(Z, ·) is increasing and Z ∈ L1+ we obtain

H+(Z,E[ZX]) := infs>E[ZX]

H(Z, s) ≤ limXm↓X

H(Z,E[ZXm])

Φ(X) = infZ∈L1

+

H(Z,E[ZX]) ≤ infZ∈L1

+

H+(Z,E[ZX]) ≤ infZ∈L1

+

limXm↓X

H(Z,E[ZXm])

= limXm↓X

infZ∈L1

+

H(Z,E[ZXm]) = limXm↓X

Φ(Xm)(CFA)= Φ(X)

where in the last equality we used the continuity from above (CFA) of Φ.

Step 3 We prove that

H+(Z, t) = β(Z, t) (21)

where, for Z ∈ L1 and t, q ∈ R,

γ(Z, q) = infY ∈L∞

E[ZY ] | Φ(Y ) ≥ q

β(Z, t) = sup q ∈ R | γ(Z, q) ≤ t .

Clearly γ and β coincide with (9) and (15) when Q ∈ ∆ and Z = dQ/dP .

Note that β(Z, ·) is the right inverse of the increasing function γ(Z, ·) and therefore

β(Z, ·) is right continuous. To prove that H+(Z, t) ≤ β(Z, t) it is sufficient to show

35

that for all p > t we have:

H(Z, p) ≤ β(Z, p). (22)

Indeed, if (22) is true

H+(Z, t) = infp>t

H(Z, p) ≤ infp>t

β(Z, p) = β(Z, t)

as both H+ and β are right continuous in the second argument.

Writing explicitly the inequality (22)

supξ∈L∞

Φ(ξ) | E[Zξ] ≤ p ≤ sup q ∈ R | γ(Z, q) ≤ p

and letting ξ ∈ L∞ satisfying E[Zξ] ≤ p, we see that it is sufficient to show the

existence of q ∈ R such that γ(Z, q) ≤ p and q ≥ Φ(ξ). If Φ(ξ) = ∞ then γ(Z, q) ≤ p

for any q and therefore β(Z, p) = H(Z, p) = ∞.

Suppose now that ∞ > Φ(ξ) > −∞ and define q := Φ(ξ). As E[ξZ] ≤ p we have:

γ(Z, q) := inf E[Zξ] | Φ(ξ) ≥ q ≤ p.

Then q ∈ R satisfies the required conditions.

To obtain H+(Z, t) := infp>t H(Z, p) ≥ β(Z, t) it is sufficient to prove that, for all

p > t, H(Z, p) ≥ β(Z, t), that is:

supξ∈L∞

Φ(ξ) | E[Zξ] ≤ p ≥ sup q ∈ R | γ(Z, q) ≤ t . (23)

Fix any p > t and consider any q ∈ R such that γ(Z, q) ≤ t. By the definition of γ,

for all ε > 0 there exists ξε ∈ L∞ such that Φ(ξε) ≥ q and E[Zξε] ≤ t+ ε. Take ε such

that 0 < ε < p− t. Then E[Zξε] ≤ p and Φ(ξε) ≥ q and (23) follows.

Step 4 Normalization

36

From the above Steps 1, 2 and 3 we then deduce:

Φ(X) = infZ∈L1

+

H(Z,E[ZX]) = infZ∈L1

+

H+(Z,E[ZX]) = infZ∈L1

+

β(Z,E[ZX])

= infZ∈L1

+

β(Z,X).

To conclude the thesis we only need to normalize the elements Z ∈ L1+. This is possible

since, by definition of H(Z, t),

H(Z,E[ZX]) = H(λZ,E[(λZ)X]), ∀Z ∈ L1+, Z = 0, λ ∈ (0,∞)

and so, by setting dQ/dP = Z/E[Z], Q ∈ ∆, we obtain

Φ(X) = infZ∈L1

+(R)H(Z,E[ZX]) = inf

Q∈∆H(Q,EQ[X]) = inf

Q∈∆β(Q,X).

37

Index Description Author(s) fq(x)h A scientist has index h if h of her

papers have at least h citationsHirsch (2005) q1(0,q](x)

h2 A scientist has index h2 if h of herpapers have at least h2 citations

Kosmulski (2006) q21(0,q](x)

hα A scientist has index hα if h of herpapers have at least αh citations

van Eck and Waltman(2008)

αq1(0,q](x), α > 0

w A scientist has index w if w ofher papers have at least w,w −1, . . . , 1 citations

Woeginger (2008) (−x+ q + 1)1(0,q](x)

A A scientist has index A =∑hx=1X(x)/h, where h is the h-

index andX(x) the citation curve

Jin (2006) not SRM

R A scientist has index R =√∑hx=1 X(x), where h is the h-

index andX(x) the citation curve

Jin, Liang, Rousseau,and Egghe (2007)

not SRM

g A scientist has index g, where g isthe highest number of papers thattogether have at least g2 citations

Egghe (2006) not SRM

Table 1. Bibliometric indices. The table lists popular bibliometric indices, a short descriptionof each index, and the author(s) who introduced the index. The h-, h2-, hα-, and w-indexare Scientific Research Measures (SRMs) as defined in (1). The last column provides thefunctional form of the corresponding performance curves, fq(x). The A-, R-, and g-indexare not SRMs.

38

Evaluator 1 Evaluator 2 Evaluator 3 Evaluator 4α(Q) β(Q,X) α(Q) β(Q,X) α(Q) β(Q,X) α(Q) β(Q,X)

Q0 1 8.8 1/3 26.3 1/3 26.3 1 8.7Q1 1 8.2 2/3 12.3 1 8.2 2/3 12.3Q5 1 8.5 1 8.5 1 8.5 1/3 25.5

Φ(X) 8.2 8.5 8.2 8.7

Table 2. Numerical example of Dual SRMs. The journal space Ω consists of three journals,Ω = MS,MF,FS. The scientist has three publications, one in each journal, with 10, 6, and8 citations, respectively, i.e., her citation record X(ω) is given by X(MS) = 10, X(MF) = 6,and X(FS) = 8. To assess her research performance, three criteria are expressed by theprobability measures Q0, Q1, and Q5. Criterion Q0 expresses the number of years sincethe first journal’s edition (the first journal’s edition for MS, MF, and FS is 1954, 1991, and1996, respectively). Criterion Q1 expresses the 1-year impact factor of the journal (the 1-year impact factor for MS, MF, and FS is 1.73, 1.25, and 1.15, respectively). Criterion Q5

expresses the 5-year impact factor of the journal (the 5-year impact factor for MS, MF, andFS is 3.3, 1.66, and 1.58, respectively). Each evaluator assigns different weights α(Q) tothe three criteria and thus uses a different SRM. For each criterion Q, β(Q,X) := supq ∈R | EQ[X] ≥ γ(Q, q), where γ(Q, q) := q α(Q). The Dual SRM gives the final researchperformance as Φ(X) = min(β(Q0, X), β(Q1, X), β(Q5, X)).

39

Citations PublicationsFull name Faculty Avg Med Std Avg Med Std

Berkeley Haas School of Business 8 90 21 179 85 61 61Chicago Chicago Booth 17 279 68 562 104 85 97Columbia Columbia Business School 34 121 40 237 101 63 108Cornell Johnson 8 72 23 142 181 82 302Harvard Harvard Business School 15 152 45 283 59 44 61NYU Stern School of Business 26 110 27 285 81 73 44Penn. Wharton, Univ. of Pennsylvania 21 138 48 267 77 52 67Princeton Bendheim Center for Finance 13 127 37 268 207 82 271Stanford Graduate School of Business 11 131 52 226 43 31 32UCLA Anderson School of Management 9 145 39 281 81 74 36Yale Yale School of Management 11 161 55 333 75 54 45

Total 173

Table 3. Summary statistics of citations and publications. The dataset includes full pro-fessors affiliated to business schools or finance departments of 11 main U.S. universities.Faculty is the number of full professors for each university in our dataset. For each author,the dataset includes the number of publications with at least one citation each and thenumber of citations per publication, i.e., the citation curve. Avg, Med and Std are average,median and standard deviation, respectively, of citations and publications of all full profes-sors for each university. Entries are rounded to the nearest integer. The dataset of citationsand publications is from Google Scholar and extracted on September 6, 2012.

40

Estimates t-statistics R2

q c k q c kBerkeley 1,219.40 0.67 164.76 56.12 30.01 5.83 95.88Chicago 5,245.49 0.54 1,411.10 49.81 25.93 7.76 95.94Columbia 1,673.92 0.63 185.16 60.25 30.51 7.51 97.22Cornell 1,267.08 0.63 209.31 56.95 42.16 4.89 97.11Harvard 1,591.17 0.59 221.40 41.52 15.38 6.30 96.16NYU 2,656.87 0.64 321.18 45.40 23.47 4.77 96.81Penn. 1,975.27 0.61 672.08 55.40 27.21 7.05 97.13Princeton 2,742.20 0.58 204.09 61.19 38.69 8.83 96.33Stanford 5,156.73 0.35 4,159.60 26.01 11.19 4.81 96.56UCLA 2,377.92 0.44 577.17 31.42 14.09 4.83 95.56Yale 1,948.72 0.46 753.68 33.50 16.17 5.55 96.62

Table 4. Estimates of hyperbolic functions. For each author, the hyperbolic function fq(x) =q/xc−k is fitted to the citation curve using nonlinear least squares, i.e., parameter estimatesare obtained by minimizing

∑px=1(X(x)− fq(x))

2 with respect to q, c and k, where p is thenumber of publications with at least one citation each. R2 is the adjusted R2 in percentage.Table entries are obtained by averaging the corresponding values of all authors in eachuniversity. The dataset of citations includes 173 full professors affiliated to business schoolsor finance departments of 11 main U.S. universities. The dataset is extracted from GoogleScholar on September 6, 2012.

41

ϕh

h2

hα

wA

Rg

Avg

Med

Avg

Med

Avg

Med

Avg

Med

Avg

Med

Avg

Med

Avg

Med

Avg

Med

Berkeley

1,504

1,489

2725

1010

3532

4741

185

160

6965

5644

Chicago

3,007

2,338

4138

1615

4842

6660

479

331

135

117

8385

Columbia

1,778

1,415

3830

1211

4636

6350

229

186

8879

7357

Cornell

1,495

1,446

3229

1011

4336

5751

171

146

7371

6956

Harvard

1,705

1,626

2823

1110

3326

4534

247

239

7968

4844

NYU

1,768

1,428

3331

1212

4038

5752

238

186

8581

7167

Penn.

1,653

1,516

3125

1110

3932

5242

235

174

7964

5852

Princeton

2,404

1,853

5149

1413

6762

9182

296

278

119

94106

82Stanford

1,536

1,109

2319

119

2820

3728

219

218

7061

4331

UCLA

1,962

2,267

3734

1315

4540

6353

286

320

100

113

7661

Yale

1,726

1,706

3029

1211

3633

4944

274

244

8584

5954

Tab

le5.

TheScientificResearchmeasure

andbibliom

etricindices.For

each

scholar

inou

rdataset,ϕ-,h-,h2-,hα(w

ith

α=

0.5),w-,A-,R-,an

dg-index

arecomputed.Theϕ-index

isbased

onhyperbolic

perform

ance

curves

anddefined

inSection

5.For

each

index,Avgan

dMed

areaveragean

dmedianof

theindex

ofallscholarsin

agivenuniversity.Entries

arerounded

tothenearest

integer.

Thedataset

ofcitation

scovers

173fullprofessorsaffi

liated

tobusinessschoolsor

finan

cedepartm

ents

of11

mainU.S.universities.Thedataset

isextractedfrom

Google

Scholar

onSeptember

6,2012.

42

0 1 2 3 4 5 6 7 8 9 10 110

2

4

6

8

10

12

14

16

18

20

Publications, x

Cita

tions

Citation curve, X(x)

Figure 1. Citation curve. The graph shows the citation curve of a hypothetical scientist with8 publications, as well as performance curves based on h-index and ϕ-index. The citationcurve is obtained by ordering publications in decreasing order of citations. Performancecurves of the h-index are given by fq(x) = q 1(0,q](x), with q = 1, 2, 3, 4. Performance curvesof the ϕ-index are given by fq(x) = q/x0.55−8.49, with each curve passing through a differentpoint (x,X(x)), with x = 1, 2, . . . , 8. The hypothetical scientist has a h-index of 4 and ϕ-index of 24.75. The ϕ-index is determined by the performance curve passing through thepoint (3, 5). This performance curve is the highest curve still below the citation curve Xand indeed 24.75/30.55 − 8.49 = 5.

43

0 1 2 3 4 5 6 7 8 9 10 110

1

2

3

4

5

6

7

8

9

10

11

Publications, x

Cita

tions

, fq(x

)

h−index

0 1 2 3 4 5 6 7 8 9 10 110

1

2

3

4

5

6

7

8

9

10

11

Publications, x

Cita

tions

, fq(x

)

h2−index

0 1 2 3 4 5 6 7 8 9 10 110

1

2

3

4

5

6

7

8

9

10

11

Publications, x

Cita

tions

, fq(x

)

hα−index

0 1 2 3 4 5 6 7 8 9 10 110

1

2

3

4

5

6

7

8

9

10

11

Publications, x

Cita

tions

, fq(x

)

w−index

0 1 2 3 4 5 6 7 8 9 10 110

1

2

3

4

5

6

7

8

9

10

11

Publications, x

Cita

tions

, fq(x

)

Maximum number of citations

0 1 2 3 4 5 6 7 8 9 10 110

1

2

3

4

5

6

7

8

9

10

11

Publications, x

Cita

tions

, fq(x

)

Maximum number of publications

Figure 2. Performance curves. The graph shows six families of performance curves cor-responding to six different bibliometric indices: h-index with performance curve fq(x) =q1(0,q](x) and q = 1, 2, . . . , 7; h2-index with performance curve fq(x) = q21(0,q](x) andq = 1, 2, 3; hα-index with performance curve fq(x) = αq1(0,q](x), α = 0.5 and q = 1, 2, . . . , 7;w-index with performance curve fq(x) = (−x+ q + 1)1(0,q](x) and q = 1, 2, . . . , 7; maximumnumber of citations with performance curve fq(x) = q1(0,1](x) and q = 1, 2, . . . , 7; numberof publications with at least one citation each with performance curve fq(x) = 1(0,q](x) andq = 1, 2, . . . , 7. 44

0 20 40 60 80 1000

1000

2000

3000

4000

5000

6000

Cita

tions

, X(x

)

Publications, x0 10 20 30 40 50 60 70 80

0

500

1000

1500

2000

2500

Cita

tions

, X(x

)Publications, x

0 20 40 60 80 100 120 140 160 180

0

100

200

300

400

500

600

700

800

900

Cita

tions

, X(x

)

Publications, x0 10 20 30 40 50 60 70

0

100

200

300

400

500

600

700

800

900

Cita

tions

, X(x

)

Publications, x

Figure 3. Typical fit of citation curves. Each graph shows one citation curve X(x) (circles),i.e., the author’s citations in decreasing order of citations, where x = 1, . . . , p and p is thenumber of publications with at least one citation. Superimposed is the fitted hyperboliccurve fq(x) = q/xc − k. For each author, parameter estimates of the hyperbolic curve areobtained using nonlinear least squares, i.e., by minimizing

∑px=1(X(x)−fq(x))

2 with respectto q, c and k. All adjusted R2’s are above 97.4%. Table 4 provides an overview of the fittingof such hyperbolic functions to all scholars’ citation curves in our dataset.

45

0 1000 2000 3000 4000 5000 6000 7000 8000 90000

50

100

150

φ−index

h−in

dex

BerkeleyChicagoColumbiaCornellHarvardNYU

0 1000 2000 3000 4000 5000 6000 7000 8000 900010

20

30

40

50

60

70

80

90

100

110

φ−indexh−

inde

x

PennPrincetonStanfordUCLAYale

0 1000 2000 3000 4000 5000 6000 7000 8000 90000

50

100

150

200

250

300

350

φ−index

g−in

dex

BerkeleyChicagoColumbiaCornellHarvardNYU

0 1000 2000 3000 4000 5000 6000 7000 8000 90000

50

100

150

200

250

300

350

φ−index

g−in

dex

PennPrincetonStanfordUCLAYale

Figure 4. Scatter plots of ϕ-index versus h- and g-index. Upper graphs: scatter plot of ϕ-index (horizontal axis) versus h-index (vertical axis). Lower graphs: scatter plot of ϕ-index(horizontal axis) versus g-index (vertical axis). Those indexes are computed for each one ofthe 173 scholars in our dataset and displayed according to the university the scholar belongsto. Relations between ϕ-index and h-index (respectively g-index) are not monotone. Thusrankings of scholars given by the ϕ-index and h-index (respectively g-index) are different.The dataset of citations covers 173 full professors affiliated to business schools or financedepartments of 11 main U.S. universities. The dataset is extracted from Google Scholar onSeptember 6, 2012.

46

References

Adler, R., J. Ewing, and P. Taylor, 2008, “Citation Statistics,” Joint Committee on Quan-titative Assessment of Research, International Mathematical Union.

Alonso, S., F. Cabrerizo, E. Herrera-Viedma, and F. Herrera, 2009, “h-index: A ReviewFocused in its Variants, Computation and Standardization for Different Scientific Fields,”Journal of Informetrics, 3, 273–289.

, 2010, “hg-index: A New Index to Characterize the Scientific Output of ResearchersBased on the h- and g-indices,” Scientometrics, 82, 391–400.

American Scientist Open Access Forum, 2008, Comments to the Citation Statistics Report.Citation Statistics: International Mathematical Union Report, U.S.

Anderson, T., K. Hankin, and P. Killworth, 2008, “Beyond the Durfee Square: Enhancingthe h-index to Score Total Publication Output,” Scientometrics, 76, 577–588.

Artzner, P., F. Delbaen, J.-M. Eber, and D. Heath, 1999, “Coherent Measure of Risk,”Mathematical Finance, 9, 203–228.

Bornmann, L., R. Mutz, and H.-D. Daniel, 2008, “Are There Better Indices for EvaluationPurposes than the h-index? A Comparison of Nine Different Variants of the h-index UsingData from Biomedicine,” Journal of the American Society for Information Science andTechnology, 59, 830–837.

Borokhovich, K., R. Bricker, K. Brunarski, and B. Simkins, 1995, “Finance Research Pro-ductivity and Influence,” Journal of Finance, 50, 1691–1717.

Cabrerizo, F., S. Alonso, E. Herrera-Viedma, and F. Herrera, 2010, “q2-index: Quantitativeand Qualitative Evaluation Based on the Number and Impact of Papers in the HirschCore,” Journal of Informetrics, 4, 23–28.

Cerreia-Vioglio, S., F. Maccheroni, M. Marinacci, and L. Montrucchio, 2011, “Risk Measures:Rationality and Diversification,” Mathematical Finance, 21, 743–774.

Cherny, A., and D. Madan, 2009, “New Measures for Performance Evaluation,” Review ofFinancial Studies, 22, 2571–2606.

Chung, K., and R. Cox, 1990, “Patterns of Productivity in the Finance Literature: A Studyof the Bibliometric Distributions,” Journal of Finance, 45, 301–309.

Drapeau, S., and M. Kupper, 2013, “Risk Preferences and Their Robust Representation,”Mathematics of Operations Research, 38, 28–62.

Egghe, L., 2006, “Theory and Practice of the g-index,” Scientometrics, 69, 131–152.

47

Egghe, L., and R. Rousseau, 2008, “An h-index Weighted by Citation Impact,” InformationProcessing and Management, 44, 770–780.

Follmer, H., and A. Schied, 2002, “Convex Measures of Risk and Trading Constraints,”Finance and Stochastics, 6, 429–447.

, 2004, Stochastic Finance: An Introduction in Discrete Time. de Gruyter Studies inMathematics 27, Berlin, 2nd edn.

Frittelli, M., and M. Maggis, 2011, “Dual Representation of Quasiconvex Conditional Maps,”SIAM Journal on Financial Mathematics, 2, 357–382.

Frittelli, M., M. Maggis, and I. Peri, 2013, “Risk Measures on P(R) and Value at Risk withProbability/Loss Function,” Mathematical Finance, forthcoming.

Frittelli, M., and E. Rosazza Gianin, 2002, “Putting Order in Risk Measures,” Journal ofBanking and Finance, 26, 1473–1486.

Garfield, E., 2005, “The Agony and the Ecstasy – The History and Meaning of the Jour-nal Impact Factor,” International Congress on Peer Review and Biomedical Publication(Chicago).

Guns, R., and R. Rousseau, 2009, “Real and Rational Variants of the h-index and theg-index,” Journal of Informetrics, 3, 64–71.

Hauser, J., 1998, “Research, Development, and Engineering Metrics,” Management Science,44, 1670–1689.

Hirsch, J., 2005, “An Index to Quantify an Individual’s Scientific Research Output,” Pro-ceedings of the National Academy of Sciences, 102, 16569–16572.

, 2007, “Does the h Index Have Predictive Power?,” Proceedings of the NationalAcademy of Sciences, 104, 19193–19198.

Jin, B., 2006, “h-index: An Evaluation Indicator Proposed by Scientist,” Science Focus, 1,8–9.

Jin, B., L. Liang, R. Rousseau, and L. Egghe, 2007, “TheR- and AR-indices: Complementingthe h-index,” Chinese Science Bulletin, 52, 855–863.

Kalaitzidakis, P., T. Stengos, and T. Mamuneas, 2003, “Rankings of Academic Journals andInstitutions in Economics,” Journal of the European Economic Association, 1, 1346–1366.

Kosmulski, M., 2006, “A New Hirsch-type Index Saves Time and Works Equally Well as theOriginal h-index,” ISSI Newsletter, 2, 4–6.

Lovegrove, B., and S. Johnson, 2008, “Assessment of Research Performance in Biology: HowWell Do Peer Review and Bibliometry Correlate?,” BioScience, 58, 160–164.

48

Marshall, A. W., I. Olkin, and B. C. Arnold, 2011, Inequalities: Theory of Majorization andits Applications. Academic Press, New York, 2nd edn., Springer Series in Statistics.

May, R., 1997, “The Scientific Wealth of Nations,” Science, 275, 793–796.

Panaretos, J., and C. Malesios, 2009, “Assessing Scientific Research Performance and Impactwith Single Indices,” Scientometrics, 81, 635–670.

Ruane, F., and R. Tol, 2008, “Rational (Successive) h-indices: An Application to Economicsin the Republic of Ireland,” Scientometrics, 75, 395–405.

Schreiber, M., 2010, “Twenty Hirsch Index Variants and Other Indicators Giving More orLess Preference to Highly Cited Papers,” Annals of Physics, 522, 536–554.

van Eck, N., and L. Waltman, 2008, “Generalizing the h- and g-indices,” Journal of Infor-metrics, 2, 263–271.

Vinkler, P., 2009, “The π-index: A New Indicator for Assessing Scientific Impact,” Journalof Information Science, 35, 602–612.

Volle, M., 1998, “Duality for the Level Sum of Quasiconvex Functions and Applications,”Control, Optimisation and Calculus of Variations, 3, 329–343.

Wiberley, S., 2003, “A Methodological Approach to Developing Bibliometric Models of Typesof Humanities Scholarship,” Library Quarterly, 73, 121–159.

Woeginger, G., 2008, “An Axiomatic Characterization of the Hirsch-index,” MathematicalSocial Sciences, 56, 224–232.

Zhang, C.-T., 2009, “The e-Index, Complementing the h-Index for Excess Citations,” PLoSONE, 4, e5429.

49