A BIBLIOMETRIC ANALYSIS OF THE JOURNAL
ECONOMETRICA
A study submitted in partial fulfillment
of the requirements for the degree of
MSc Information Systems
at
THE UNIVERSITY OF SHEFFIELD
by
OLHA RODZEN
September 2013
2
Abstract
Background: Bibliometric analysis is used for assessment of contribution to the
particular journal by the scientists. It shows impact of each country or research
institution on the journal. In addition, bibliometric studies can explain publishing and
citing trends that is presented in the journal. Citation analysis provides information
about connection of the different disciplines with each other.
Aims: The study aimed to carry out bibliometric analysis of the journal Econometrica
and to investigate knowledge diffusion from this journal to others fields of sciences
and social sciences.
Methods: A data for the research was collected from Web of Science Core Collection
database. Based on that data bibliometric analysis was carried out and analysis for
main figures was provided. In addition, data about citation for specific publication
was retrieved from WoS and connection with other research areas was provided.
Results: The results of the study can be divided into two parts. First, bibliometric and
citation analysis of the journal Econometrica is provided. Bibliometric analysis
includes year wise distribution of publication with explanation of some critical values,
geographical distribution of contributors with outlining of recent trends and
organization distribution of publication. In addition, citation analysis of the journal
presents citation distribution. Second part of the study is focused on citation analysis
of specific highly cited publications in the journal Econometrica.
Conclusions: It is concluded that the journal Econometrica is international journal
with high impact on other research areas. Number of publications in the journal in
more than 20 years is stable, however some fluctuations were presented in 1965-
1974. The biggest contribution to the journal is made by authors from the USA and
UK. The biggest contributors to the journal is organizations which is situated in the
USA. Journal Econometrica is highly cited journal. It is cited by the journals from the
same research areas as well as from the different research areas.
3
Acknowledgements
I would like to thank my supervisor, Peter Willett, for his help and support throughout
the completion of this project.
I would like to thank my family for their encouragement and support.
4
Contents
Abstract ........................................................................................................................ 2
Acknowledgements ...................................................................................................... 3
Contents ....................................................................................................................... 4
List of Tables ................................................................................................................. 6
List of Figures ............................................................................................................... 7
1 Introduction .......................................................................................................... 9
1.1 Background context....................................................................................... 9
1.2 Journal description ...................................................................................... 10
1.3 Aim and objectives ...................................................................................... 11
1.4 Project overview .......................................................................................... 12
2 Literature review ................................................................................................ 13
2.1. Bibliometric studies of single journal .............................................................. 13
2.2 Citation analysis ................................................................................................ 15
2.3 Knowledge diffusion ......................................................................................... 18
3 Methodology ........................................................................................................... 20
3.1 Approach .......................................................................................................... 20
3.2 Justifying the use of WoS citation index .......................................................... 21
3.3 Data collection process .................................................................................... 22
3.4 Limitations of available data ............................................................................ 22
4 Results and Discussion: Publication and Citation analysis ................................. 24
4.1 Introduction ...................................................................................................... 24
4.2 Year wise distribution of publications in journal Econometrica ...................... 24
4.3 Geographical distribution of contributors in journal Econometrica ................ 27
4.4 Organisational distribution of contributors in journal Econometrica .............. 31
4.5 Citations distribution of publication in journal Econometrica ......................... 32
Contents
5
4.6 Citation distribution of source titles and research areas of journal
Econometrica .......................................................................................................... 36
5 Results and Discussion: Citation patterns of publications ...................................... 40
5.1 Introduction ...................................................................................................... 40
5.2 Citation pattern of most cited publication in journal Econometrica ............... 40
5.3 Citation pattern of oldest most cited publication in journal Econometrica .... 43
5.4 Citation pattern of newest most cited publication in journal Econometrica ... 46
5.4 Citation pattern of most cited publication in journal Econometrica that was
published in XXI century ......................................................................................... 48
5.5 Comparison of citation patterns ...................................................................... 50
6 Conclusions ............................................................................................................. 52
6.1 Key findings ...................................................................................................... 52
6.2 Meeting the aim and objectives ....................................................................... 54
6.3 Recommendation for further research ............................................................ 55
7 Bibliography ............................................................................................................ 57
6
List of Tables
Table 1: Types of published items in journal Econometrica
in 1964-1974
26
Table 2: Geographical distribution of top 10 main contributors
in journal Econometrica in 1933-2014
29
Table 3: Geographical distribution of contributors in journal
Econometrica in 1999-2013
30
Table 4: Most cited articles in journal Econometrica in 1933-
2014
35
Table 5: Journals most frequently citing journal Econometrica 37
Table 6: Research areas of the journals most frequently citing
journal Econometrica
38
7
List of Figures
Figure 1: Distribution of publications in journal Econometrica
in 1933-2014
25
Figure 2: Types of published items in journal Econometrica in
1964-1974
27
Figure 3: Geographical distribution of contributors in journal
Econometrica in 1933-2014
28
Figure 4: Zipf distribution of contributors in journal
Econometrica in 1933-2014
28
Figure 5: Organisational distribution of publications in journal
Econometrica in 1999-2013
32
Figure 6: Citations distribution of publications in journal
Econometrica in 1933-2014
33
Figure 7: Zipf distribution of citations of publications in journal
Econometrica in 1933-2014
34
Figure 8: Citations distribution of publications in journal
Econometrica in 1933-2014
36
Figure 9: Importers of knowledge from journal Econometrica 39
Figure 10: Citation over time to the article by Kahneman and
Tversky
41
Figure 11: Research areas most frequently cited Kahneman and
Tversky Prospect theory - analysis of decision under
risk
42
Figure 12: Citation over time to the article by Nash 44
Figure 13: Research areas most frequently cited Nash The
bargaining problem
45
Contents
8
Figure 14: Citation over time to the article by Johansen 47
Figure 15: Research areas most frequently cited Johansen
Estimation and hypothesis testing of cointegration
vectors in Gaussian vector autoregressive models
47
Figure 16: Citation over time to the article by Melitz 48
Figure 17: Research areas most frequently cited Melitz The
impact of trade on intra-industry reallocations and
aggregate industry productivity
49
Figure 18: Citation over time to the article by Kahneman and
Tversky, Nash, Johansen, Melitz
51
9
1 Introduction
1.1 Background context
Scholarly communication has a considerable impact on science and social science
development in the world. Exchange of ideas is mainly introduced by citation of
previous publication. This can be considered as connections of different opinions to
produce additional value for the scientific researches. Garfield as cited in Noruzi
(2005) identifies citations or “association-of-ideas” as “formal, explicit linkages
between papers that have particular points in common.”
Scientists use citation analysis, one of bibliometric techniques, for evaluation
productivity and importance of paper (Narin, 1976, p. 1). Therefore, it has a wider
area of implication, for example identification scientist of which country cited
particular journal (Borkenhagen, Decker, Brahler, Strauss, 2002), journals in which
particular paper is cited (Borkenhagen, Decker, Brahler, Strauss, 2002; Torricella-
Morales, Van Hooydonk, Araujo-Ruiz, 2000) and who is the author of cited
publication (Torricella-Morales, Van Hooydonk, Araujo-Ruiz, 2000).
Citation analysis is used by scientists to determine the impact of discipline to the
other research areas of science and social science (Lowry, Karuga, Richardson, 2007).
Scientists cites publication of other authors when it is important to their work and
has influence on it (Loebbecke, Leidner, 2012). In this study, an attempt to determine
influence of econometric studies on other research areas is made. This shows that
econometric studies is used by other fields of science and social science.
In addition, there are others methods, which helps to evaluate importance of the
research. Bibliometrics is used for evaluation of contribution to the particular field of
research by the authors of different nations (Damrosch, 2006; McElwee, Atherton,
1: Introduction
10
2005; Elster, Chen, Jenkins, 2004) and main topics which is relevant for this field
(McElwee, Atherton, 2005).
However, current bibliometric researches do not cover all field of science and social
science. Econometrics is one of the fields where few bibliometric researches were
carried out. Therefore, in this research will be analysed main trends in econometric
research and their influence on science and social science in general. This study will
be carried out based on journal Econometrica.
As was mentioned earlier, any bibliometric researches connected with econometrics
was not find. However, there are variety of bibliometric researches, which connected
with other disciplines. Methods from this scientific works can be implied on
econometric journal. Data for the research will be retrieved from Web of Science
databases.
1.2 Journal description
The journal Econometrica was created in 1933. Information about editors of the
journal Econometrica is displayed on the official web site. There have been twelve
editors of the journal since 1933.
The first editor of the paper was Norwegian economist Ragnar Frisch and he took
this post from the 1933 to 1954. He and Jan Tinbergen were first winners of Nobel
Prize in economics “for having developed and applied dynamic models for the
analysis of economic processes” (The Concise Encyclopedia of Economics, 2008). In
1953-1954, co-editor with Ragnar Frisch was American economist Robert H. Strotz.
Eventually, he become an editor of Econometrica in 1955-1968. Next editor of the
journal Econometrica became American economist Franklin M. Fisher. He took this
post in 1968-1977 and was replaced by another American economist Hugo
Sonnenschein who was an editor for 1977-1984. British economist Angus Deaton was
editor of journal Econometrica in 1984-1988. In 1988-1992 post of Econometrica
editor belongs to Spanish economist Andreu Mas-Colell. French economist Guy
Laroque was an editor in 1992-1996. He was replaced by American economist Drew
Fudenberg in 1996 and he took this position to 2000. Econometrica editor in 2000-
1: Introduction
11
2003 was Glenn Ellison and in 2003-2007 was Eddie Dekel. He was replaced by
Stephen Morris and he took this post to 2011. Turkish-American economist Daron
Acemoglu is the Econometrica editor since 2011.
Econometrica publishes theoretical and empirical articles in all economics fields. The
goal of the series is defined on the web site of journal: “to promote the publication
of high-quality research works in the fields of Economic Theory, Econometrics, and
Quantitative Economics more generally”. Econometrica is published bi-monthly and
there are more than 81 available volumes of journal nowadays. Econometrica is
published in English language, however first issues contains material on French
language.
Econometrica is international journal published in English language. Most of the
Econometrica editors were representatives of USA, however first editor was
Norwegian economist and nowadays it has international editor.
1.3 Aim and objectives
Aim
To make bibliometric analysis of journal Econometrica and investigate export
relationship between econometrics and other research areas based on this journal.
Objectives
• To carry out literature review presenting previous bibliometric analysis of
different journals and their impact on different scientific areas.
• To outline year wise distribution of publications in the journal Econometrica.
• To identify geographical distribution of contributors to the journal
Econometrica.
• To identify main organisations contributors to the journal Econometrica.
• To describe citation trends of the journal Econometrica.
1: Introduction
12
• To explore which disciplines and journals export knowledge from the journal
Econometrica.
• To identify most cited articles in the journal Econometrica.
• To explore export relationship of the journal Econometrica by the most cited
publications.
1.4 Project overview
The rest of dissertation is divided into five main sections. The next section presents a
literature review. This section includes presentation of researches that is connected
to bibliometric studies of single journal, citation analysis and knowledge diffusion
between disciplines.
Next part describes methodology that was used for conducting this study. It includes
description of the approaches for the research, justification of usage Web of Science
citation index, the process of data collection and main limitations of the research.
Than results of bibliometric analysis of the journal Econometrica is displayed. This
results include year wise distribution of publication, geographical and organization
distribution. In addition, citation distribution for the journal Econometrica is
presented. Next chapter includes results and discussion for the citation patterns of
most cited articles in the journal Econometrica. All presented data in tables and
graphs is discussed and analysed.
The final chapter presents main findings of the research. Meeting the aim and
objectives is discussed and recommendation for further research is presented.
13
2 Literature review
2.1. Bibliometric studies of single journal
A number of bibliometric studies was carried out on single journal. Scientist use this
technique for a different proposes. Anyi, Anuar and Zainab (2009) reviewed 82
bibliometric studies to reflect different approaches to bibliometric analysis, however
in general all bibliometric studies on single journal “creates a portrait of the journal”
and “indicate the quality, maturity and productivity of the journal”. Bibliometric
studies are carried out on different journals. These includes library and information
science journals as well as science and technology, medical and health sciences, arts,
humanities and social sciences journals (Anyi, Anuar and Zainab, 2009).
Anyi, Anuar and Zainab (2009) concluded that the major input into bibliometric
studies is done by Asian and African countries. However, the contribution of scientists
from the United States remain quite high. In addition, researchers who study journals
use following bibliometric measures article productivity, author characteristics,
author’s productivity, co-authorship pattern, content analysis, citation analysis and
characteristic of the editorial board.
For example, Jena (2006) demonstrated distribution of numbers of articles published
in Indian Journal of Fibre and Textile Research and their length from 1996 to 2004.
Author concluded that number of articles shows increasing trend from the year 1996
and average length of article is 6.45 pages. Similar study was produced based on
journal Economic botany by Biswas, Roy, Sen (2007). However, authors did not find
any trend in number of articles published in journal from 1994 to 2003 average length
of article is varying from 9 to 13.
In addition, study introduced by Jena (2006) include geographical distributions of
contributors of articles. Even though journal is published in English language it has
2: Literature Review
14
more than 86% contributors among Indian scientists because journal is from India.
Elster, Chen, Jenkins (2004) in the American Journal of Roentgenology, spots similar
trend in geographical distribution. Most articles, which is published in journal,
belongs to American scientist – more than 65% in 2000-2002. Input of American
scientist in Journal of Molecular Graphics and Modelling is lower than in previous
journal, but still significant – “no less than 45.8% of the articles” (Willett, 2007).
Similar trends reported by Jishi and Willett (2010) in Journal of Chemical Information
and Computer Science – significant part of the articles are contribution of United
States scientist.
However, trend of domination authors from one country is not present in all journals.
Borkenhagen, Decker, Brahler, Strauss (2002) shows that number of authors of the
journal Psychotherapy Research from North America and Europe are almost equal.
Same conclusion can be made based on Biswas, Roy, Sen (2007) bibliometric study of
the Economic botany journal.
Different trends in authorship pattern can be determined in journals. Ravichandra
Rao and Bhusan (2008) observed multiple authorship in the Journal of American
Society for Information Science & Technology (JASIST) and Scientometrics from 1992
to 2004. They show average number of authors in JASIST per paper increased from
1.45 in 1992 to 2.29 in 2004. Similar trend is present in Scientometrics average
number of authors rose from 1.43 in 1992 to 2.35 in 2004. Biswas, Roy, Sen (2007)
outlined that in the journal Economic Botany “the contributions by single author and
small teams comprising two or three authors are dominating the field”. However, in
study carried out by Jena (2006) three authored papers is dominant (more than 30%)
followed by two authored articles (28.6%) and multi-authored (22.88%). Single
authored papers were 14.4%.
In addition, Jishi and Willett (2010) studied most used subject area for indexing
Journal of Chemical Information and Computer Science articles. Researchers outlined
ten main subject areas and find out that most articles are indexed as computer
science and chemistry studies.
2: Literature Review
15
Literature review of single journal bibliometric studies shows that there are not one
trend for all journals. There are significant difference between journals from different
science and social science areas. Therefore, generalization cannot be made. This
shows that results of bibliometric analysis of journal Econometrica cannot be
extrapolated from previous researches.
2.2 Citation analysis
Science was compared by Bonzi, Snyder (1991) with social enterprise and citing other
papers “fills both a professional and a connectional role” in that enterprise to
maximize output of highly valued papers. MacRoberts and MacRoberts (1989)
suggest that usage of another work is an evidence that a different scientist has an
influence on particular paper. In addition, Urata (1990) suggest that citation is an
award and represents high prestige of a researcher. Citation analysis can be used for
“understanding of the ‘collective’ knowledge production within a field” (Hammarfelt,
2011). Citation analysis is an important tool to study scholarly communications,
therefore number of researches of citation behaviour is constantly growing
(Hellqvist, 2010).
Citation is used for different proposes. Garfield (1965) defines fifteen main reasons
of provision reference citations. Scientists pay homage to pioneers, identify original
publication in which idea was discussed and identify methodology as well as criticize
previous work, correct it and disclaim ideas of others. Citations is used by scientists
as “threads” to link distinct ideas or arguments, which appears in different sources
(Amsterdamska, Leydesdorff, 1989). In addition, citing behaviour of scholars is
varying in different field of science and social science (Hellqvist, 2010). Therefore, it
creates additional difficulties in citation analysis.
Distribution of citation counts for articles is highly skewed (Jishi, Willett, 2010) and
the majority of articles receive few, if any, citations from other researches. However,
Leydesdorff (1987) claims that citation analysis does not often show importance of
the publication for science and just reflects behaviour of researcher. Therefore,
scientist can refer to particular source because of institutional, behavioural, and
2: Literature Review
16
prestige reasons. In addition, even high quality articles with time become obsolescent
and obsolete, therefore, citation counts dropped off over time (Jishi, Willett, 2010).
Moreover, most scientist agreed that citation counts in general measure individual
contribution to the practice of science (Amsterdamska, Leydesdorff, 1989; Garfield,
1979). Nevertheless, there are a lot of criticism connected with citation analysis and
its ability to measure impact of publication. Garfield (1979) labelled these limitations
as “negative citations, self-citations, and methodological papers”. However, scientist
outlines that negative citations or criticism of the paper has important role in the
process of scientific publication, therefore it is a minor drawback of these technique
(Garfield, 1979).
Another limitation of citation analysis, which is outlined in literature is self-citations
(Bonzi, Snyder, 1991; Garfield, 1979; Lawani, 1982). The term self-citation is used
most commonly for characterization of links between authors of the cited and citing
article and it could be considered as egoism of the researcher in certain cases
(Lawani, 1982). However, Bonzi and Snyder (1991) argued that self-citation might
present “the best and most accurate information on a topic”. In addition, Tagliacozzo
(1977) pointed out that self-citation can be recognized as connection of different
works of one scientist for avoidance of replications and presenting historical
background of the research. Garfield (1979) support this statement and reports that
“a high self-citation count, more often than not, indicates nothing more ominous
than a narrow specialty”. Study conducted by Folly, Hajtman, Nagy and Ruff (1987)
represents that about 30% of all citations with no differences between disciplines are
direct or indirect self-citations, however according to future researches they
concluded “through not their number, but – the effect of self-citations is negligible.”
Final limitation of citation analysis that is proposed by Garfield (1979) connected with
citation counts of methodological papers and assumption that “citation counts
cannot be a valid measure because they favour those who develop research methods
over those who theorize about research findings.” This finding is confirmed by Peritz
(1983) and Folly, Hajtman, Nagy and Ruff (1987) in their studies. However, there is a
consensus among social scientists (Garfield, 1979; Folly, Hajtman, Nagy and Ruff,
2: Literature Review
17
1987; Peritz, 1983) that even though methodological papers are more cited some of
them are not cited at all.
In addition, Lowry, Karuga and Richardson (2007) outlines that one of the limitation
of citation analysis is biased because citation count depends on time and number of
articles in journal. However, this bias is important in the case when citation counts of
different journals is compared (Lowry, Karuga, Richardson, 2007).
MacRoberts and MacRoberts (1996) identify additional problems connected with
citation analysis such as some citations have no influence, citing is highly biased,
secondary sources is preferred by scientist, informal influences are not cited in
papers, citation rates is varying with disciplines, nationality, time period, size and type
of speciality, some areas of research do not get cited and technical problems
connected with citation count. In addition, Bornmann and Daniel (2008) have found
that citation data are not perfect because of time depend factors, journal depend
factors, article depend factors and availability of publication. This have influence on
citation counts, but citation analysis still used and improved “with advances in
computing capability and the growth of digital libraries and repositories” (Ding, Liu,
Guo, Cronin, 2013).
Literature review of citation analysis shows reasons for usage citation analysis for the
research. In addition, it determines main limitation connected with proposed
bibliometric method. However, this drawbacks have not significant influence on the
results of bibliometric analysis. This happens because negative citations is a part of
scientific process. Effect of self-citations can be neglected because of small influence
on the results and because it represents previous academic portfolio of the
researcher. Methodological paper have impact on future research in both
methodological and theoretical areas, therefore them are more cited. Other
disadvantages of this method more connected with citation behaviour. Some
limitations can be minimized because this study will be based on single journal.
2: Literature Review
18
2.3 Knowledge diffusion
Different subject areas is open systems. They can receive knowledge from one
discipline and give ideas to another (Urata, 1990). Citations creates “a network of
connections that effectively maps the spread of knowledge from one publication to
another” (Hessey, Willett, 2013). Knowledge diffusion between disciplines is defined
as export-import relationship. This economic term was introduced by Cronin and
Pearson (1990) to measure stability of discipline through import-export ratio of
knowledge. Larivière, Sugimoto and Cronin (2012) suggest that importation of
knowledge takes place when references from other discipline is made and
exportation is present when publication receive citation from other discipline.
Disciplines, which highly rely on import from others, are vulnerable, therefore
exporters of knowledge are robust (Cronin and Davenport, as cited in Cronin and
Pearson, 1990). The result of dividing the value of imports by value of exports can be
described with another economic term – the balance of trade in knowledge (Lockett,
McWilliams, 2005). Therefore, each discipline can have “a positive knowledge flow”
or “a negative knowledge flow” (Yan, Ding, Cronin, Leydesdorff, 2013). However, this
approach is criticized by Bedeian (2005) because high import of knowledge does not
show immaturity of discipline and can be considered as orientation inside the
discipline. Nonetheless, analysis of export-import relationship among the disciplines
can describe the structure of humanities and social sciences (Urata, 1990). In
addition, number of interdisciplinary researches is growing because of complexity of
research problems (Levitt, Thelwall, Oppenheim, 2011) this raise question of effective
evaluation of export-import relationship.
Several studies examined export-import relationship between disciplines based on
articles published in particular journal (Cronin and Pearson, 1990; Lockett,
McWilliams, 2005). However, Stigler (1994) highlight that export-import relationship
in that case also depend on other factors such as geographical distribution of journal,
number of subscribers, size of the journal, the number of issues a year and the age of
the journal.
2: Literature Review
19
Other concerns, which connected with knowledge diffusion between disciplines, are
outlined by Urata (1990). First, boundaries of academic disciplines are unclear and
this produce problem in classification both cited and citing literature. Second,
researcher can use knowledge of other discipline but does not cite it. This finding
supported by Cronin and Meho (2008) because “common subject interests do not
always or necessarily translate into the expected density of interdisciplinary citation.”
Therefore, clear and unambiguous relationship between disciplines cannot be
established.
Despite the problems associated with evaluation of export-import relationship,
journal analysis of the citation is one of the most commonly used method of their
evaluation. Cronin and Pearson (1990) investigate exportation of knowledge from
information science to other disciplines. They argued that information on available
ideas is transmitted in formal and informal way, but more than 90% of all generated
ideas are never acknowledged by other scientists. However, study by Cronin and
Pearson (1990) shows that high quality publications is cited by other authors. For
example, Cronin and Pearson (1990) find out that 105 analysed publication receives
a total of 1650 citations. However, the most of citations were done by publication in
the tame research area and analysed publications generates only 156 exports.
Stigler (1994) analyzed data about citations of 33 statistical journals including some
econometric journals (Econometrica, The Journal of Econometrics). Scientist
calculated the balance of trade in 1987-1989 of journals and highest exporter of the
ideas in the sample is journal Econometrica. However, according to Yan, Ding, Cronin,
Leydesdorff, (2013) because of acceleration of science and growth of human
knowledge the amount of exported-imported knowledge can increase significantly.
Disciplines establish strong relationship with other disciplines. This helps for them to
solve complex problems, which appears nowadays. However, this increases the
interest to determine flows of knowledge. Export-import relationship shows links
between different subject areas in science. But, evaluation of this links rises some
difficulties. Nonetheless, the most usable method to this evaluation is bibliometric
analysis of the citations in journals with different subject areas.
20
3 Methodology
3.1 Approach
To meet dissertation aim and objectives was required a methodology, which would
be suitable for single journal bibliometric review and evaluation of export-import of
knowledge.
There are wide range of papers, which provide methodological background for the
study. Some of the papers, which were used as methodological background, were
described in Literature review. The methodological background for bibliometric study
is presented by Jena (2006) and Biswas, Roy, Sen (2007). However, these two studies
were not connected with econometrics journal review, they conducted single journal
bibliometric study. Jena (2006) carries out study on nine volumes of Indian Journal of
Fibre and Textile Research published from 1996 to 2004. Author describes year wise
distribution of publication, citation pattern, bibliographical distribution of citations,
authorship pattern, geographical distribution and other bibliometric indicators.
Biswas, Roy, Sen (2007) produce study with similar indicators to describe
performance of journal Economic Botany. However, for this study were chosen
figures as year wise distribution of publication, geographical and organization
distribution of the authors and distribution of citations to the journal Econometrica.
In addition, an attempt for analysis of most cited publications and evaluation of their
contribution to other fields (export of knowledge) in this study.
Stigler (1994) produces the methodological background for evaluation of export-
import of knowledge. Researcher analyzed export-import and balance of trade in
sample of 33 statistic journals including Econometrica and The Journal of
Econometrics. In addition, methodology for knowledge diffusion analysis provided by
Cronin and Meho (2008). They “conducted a large-scale, longitudinal citation analysis
3: Methodology
21
of exports from, and imports to, the literature of IS based on an inclusive population
of journals and conference proceedings in the field.” Cronin and Meho (2008)
describes data collection process and further analysis of it. Previous bibliometric
studies (Cronin and Meho, 2008; Hessey and Willett, 2013) analysed knowledge
exported to other disciplines based on Journal Citation Reports (JCR) subject
categories. This maps information flow within different subjects, therefore provides
approach to a knowledge diffusion evaluation. In addition, Jishi and Willett (2010)
evaluates not only amount of exported knowledge, they produce main reasons for
citing each publication.
Based on this methodological approach research did not include human participants
and bibliometric study was not considered as sensitive topic. Therefore, this was no
risk study.
3.2 Justifying the use of WoS citation index
There are three main citation search services (Jacso, 2005) Web of Science (WoS),
Scopus and Google Scholar. Bergman (2012) states that WoS is the oldest citation
index among this three and it have existed since the 1970s. Scopus and Google
Scholar were launched in the fall of 2004 (Bar-Ilan, 2010).
Google Scholar is free access database that provides general information with Google
Scholar Metrics. However, according to information presented on web-site it covers
only publication published between 2008 and 2012. Therefore, it is not suitable for
bibliometric study of Econometrica, because this journal was published since 1933.
Both WoS and Scopus provide enough tools for retrieving and saving information
from their databases, however WoS was used by many scientist for their researches
(Jacso, 2005).
According to Jacso (2005) WoS is core of “the family of ISI citation index” and includes
“all references cited by papers in the primary (source) documents”. According to
Thomson Reuters web site WoS “provides researchers, administrators, faculty, and
3: Methodology
22
students with quick, powerful access to the bibliographic and citation information
they need to find research data, analyse trends, journals and researchers”.
In current research it was decided to use information retrieved from WoS.
3.3 Data collection process
As was mentioned before data for the research was collected from Web of Science
Core Collection database. Search was conducted for the publication with name
Econometrica. 7058 publications was retrieved. To obtain further data two tools from
the WoS were used.
First, Analyze Results this tool allows retrieve data about authors, editors, funding
agencies and other. In this project, data about publication years, document types,
countries, organizations, and research areas was gathered. In addition, this tool helps
to retrieve information about source titles of the journals that cites the journal
Econometrica.
Second, Create Citation Report tool from the WoS was used to retrieve data for
citation analysis. It allows gathering information about total citations, which was
received by each article. In addition, this tool was used for further analysis of
publications that cites specific article from the journal Econometrica.
3.4 Limitations of available data
Research had some limitations. Most of them connected with data. As was discussed
before, WoS is most appropriate source of data for this research, however, this
citation indexing service does not provide all necessary information.
First, it does not provide full list of information about countries that contribute
publications for the journal. As was mentioned before, there were 7058 publications,
however information about country were available just for 4010 publications.
Second, citation distribution over time for the journal cannot be analysed, because
WoS database provides information about citations as cumulative trend. Therefore,
number of citations is constantly growing.
3: Methodology
23
Finally, WoS databases does not cover all research areas and journals, therefore it
can cause some distortions in the results.
24
4 Results and Discussion: Publication and
Citation analysis
4.1 Introduction
This chapter describes main bibliometric characteristics of journal Econometrica.
It starts from presentation of data about year wise distribution of publication in
Econometrica. It is indicated main trends in number of published items. In addition,
reasons for deviations in publication count is explained.
Then geographical distribution of the authors of the publications in the journals from
1933 to 2014 is outlined. Dynamics for each country in 1999-2013 is discussed.
Then, organisational distribution of contributors is outlined. Next, distribution of
citations is presented. Finally, citation distribution of source titles and research areas
of journal Econometrica is discussed.
4.2 Year wise distribution of publications in journal
Econometrica
Journal Econometrica was firstly published in 1933. Since that time number of
articles, which are published in journal, fluctuated significantly, however, in the last
decade number of published articles did not show dramatic changes. Information
about year wise distribution of articles in journal Econometrica are illustrated on
Figure 1.
In total from the 1933 to beginning of 2014 in the journal Econometrica was issued
7058 publications. In average, it is 86 publications in a year. However, as can be seen
from Fig. 1, more than 8% of the publications were released in 1971 (585
4: Results and Discussion: Publication and Citation analysis
25
publications). For example, in 1970 this figure was 4.6% (327 publications) which is
also high number. Smallest number of articles were published in 1944 – only 16
(0.23% of total amount).
Figure 1: Distribution of publications in journal Econometrica in 1933-2014
There are several possible explanations for this result. For example, from 1941 to
1944 were published only three issues of the journal. However, there was four issues
of the Econometrica in 1933-1943. It can be presumed that reduction of number of
issues in this year is connected with entry of the US in World War II.
From 1944 to 1969, each volume of the journal Econometrica in general consists of
four issues. In addition, as can be seen from Figure 1, number of publications in 1944-
1963 was constantly growing. However, in 1964-1969 the number of publications
fluctuated significantly.
Sharp increase of number of publications started in 1970. It can be presumed that
this happened because each volume of the journal has six issues from the 1970.
However, the most interesting trend is that Econometrica issued 585 publications in
1971. It was more than 8% of total publications in journal. To explain this tendency
data about types of publication was retrieved and presented in Table 1.
0
100
200
300
400
500
600
19
33
19
36
19
39
19
42
19
45
19
48
19
51
19
54
19
57
19
60
19
63
19
66
19
69
19
72
19
75
19
78
19
81
19
84
19
87
19
90
19
93
19
96
19
99
20
02
20
05
20
08
20
11
20
14
4: Results and Discussion: Publication and Citation analysis
26
As can be seen from the table 1 high number of publications occurs when Meeting
abstracts of Book reviews is published in the journal Econometrica. This trend starts
in 1964.
Table 1: Types of published items in journal Econometrica in 1964 - 1974
DOCUMENT TYPES 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974
MEETING ABSTRACT 72 0 156 0 173 0 229 476 0 0 0 BOOK REVIEW 44 44 39 0 0 0 22 31 71 0 0
NOTE 8 15 7 3 4 14 15 20 35 14 13
ARTICLE 33 46 56 29 35 50 59 58 63 73 72 CORRECTION ADDITION 0 2 4 0 5 6 2 0 3 2 2
LETTER 0 0 0 0 0 0 0 0 5 0 0
REVIEW 0 2 0 0 0 0 0 0 0 0 0 ITEM ABOUT AN INDIVIDUAL 0 2 0 0 0 0 0 0 0 0 0
TOTAL 157 111 262 32 217 70 327 585 177 89 87
Meeting abstract in the journal Econometrica is a short publication approximately
two or three pages long. Therefore, this type of publication increases the overall
number of publications, however it does not affect the length of the issue. Influence
of this type of publication can be seen at 1964, 1966, 1968, 1970, and 1971. The
number of published articles increased from 1964 to 1966. After dramatic fall of the
number of published articles in 1966, it shows gradual growth to the 1974. However,
these tendencies did not affect the general number of published items in the journal
Econometrica (Fig. 2).
As can be seen from the Figure 2 there are significant difference in types of published
items in journal Econometrica. However, as was mentioned before, number of
published articles did not changed trends in number of published items significantly.
For example, the biggest amount of meeting abstracts were published in 1970 and
1971. This shows an outcome in number of publications. In 1971 the whole issue (Vol.
39 Issue 3) consisted of meeting abstracts from Second World Congress of the
4: Results and Discussion: Publication and Citation analysis
27
Econometric Society, Tenth Indian Econometric Conference and North American
Regional Conference.
Figure 2: Types of published items in journal Econometrica in 1964 - 1974
The number of publications in journal Econometrica in 1975-1980, however, it leveled
off in 1981 and did not show sharp decreases or increases until now.
4.3 Geographical distribution of contributors in journal
Econometrica
Most publication (6917) in journal Econometrica is in English language, however
some articles was published in French language (149). Last article in French was
published in 1972 and since then Econometrica is published on English. In total
scientist from 40 countries (Fig. 3).
Geographical distribution of contributors to the journal Econometrica is highly
skewed. It can be assumed that distribution follows Zipf law. Therefore, it will be
plotted log-log distribution of the contributors on Figure 4.
0
100
200
300
400
500
600
1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974
ITEM ABOUT AN INDIVIDUAL REVIEW LETTER
CORRECTION ADDITION ARTICLE NOTE
BOOK REVIEW MEETING ABSTRACT
4: Results and Discussion: Publication and Citation analysis
28
Figure 3: Geographical distribution of contributors in journal Econometrica
in 1933 - 2014
*Includes England, Scotland, Ireland and Wales.
**Includes Germany, Federal Republic of Germany and West Germany.
Figure 4: Zipf distribution of contributors in journal Econometrica in 1933 - 2014
0
500
1000
1500
2000
2500U
SA
UK
*
CA
NA
DA
FRA
NC
E
ISR
AEL
JAP
AN
AU
STR
ALI
A
NET
HER
LAN
DS
SPA
IN
BEL
GIU
M
SWIT
ZER
LAN
D
GER
MA
NY*
*
SWED
EN
ITA
LY
IND
IA
DEN
MA
RK
NO
RW
AY
NEW
ZEA
LAN
D
AU
STR
IA
SOU
TH K
OR
EA
PEO
PLE
S R
CH
INA
BR
AZI
L
MEX
ICO
TUR
KEY
SIN
GA
PO
RE
FIN
LAN
D
PO
LAN
D
HU
NG
AR
Y
RU
SSIA
TAIW
AN
PO
RTU
GA
L
CH
ILE
AR
GEN
TIN
A
USS
R
PH
ILIP
PIN
ES
EGYP
T
CZE
CH
OSL
OV
AK
IA
HO
NG
KO
NG
GR
EEC
E
BER
MU
DA
Number of published articles
0
0,2
0,4
0,6
0,8
1
1,2
1,4
1,6
0 1 2 3 4 5 6 7 8 9
LN(C
ou
ntr
y co
un
t)
LN(Number of published articles)
4: Results and Discussion: Publication and Citation analysis
29
As can be seen from the Figure 4 geographical distribution of contributors to the
journal Econometrica cannot be exactly defined as Zipf distribution. The main reason
for this is shortage of data. However, it can be assumed that for the bigger sample it
follows approximately Zipf distribution. Because there will be small amount of
countries with high contribution and higher amount of countries with smaller
contribution to the journal Econometrica.
Data obtained from WoS Core Collection database shows that highest number of
contributors are from the USA. As can be seen from Table 2 scientist from the USA
contributes more than 32% of total number of articles. Scientists from the United
Kingdom is on the second place and contribute nearly 6% of total publications.
Scientists from Canada and France is responsible for 3% and 2.5% respectively.
Table 2: Geographical distribution of top 10 main contributors in journal
Econometrica in 1933 - 2014
Country Number of published articles
% of 7058
USA 2267 32.1 %
UNITED KINGDOM* 403 5.7 %
CANADA 218 3.1 %
FRANCE 181 2.6 %
ISRAEL 130 1.8 %
JAPAN 84 1.2 %
AUSTRALIA 83 1.2 %
NETHERLANDS 64 0.9 %
SPAIN 59 0.8 %
BELGIUM 58 0.8 %
* Includes England, Scotland, Ireland and Wales.
To understand main trends in geographical distribution in recent years it was decided
to merge data and create intervals in 5 years each for each country contributions.
Grows rates for 1999 - 2003 were calculated with data from previous periods, which
is not displayed in a table. Results is presented in Table 3.
4: Results and Discussion: Publication and Citation analysis
30
Table 3: Geographical distribution of contributors in journal Econometrica
in 1999 - 2013
Country
1999-2003 2004-2008 2009-2013
Number Growth rate, %
Number Growth rate, %
Number Growth rate, %
USA 225 26 205 -9 295 44
UK* 46 77 39 -15 67 72
FRANCE 22 0 17 -23 32 88
ISRAEL 18 125 12 -33 11 -8
CANADA 13 -54 10 -23 29 190
NETHERLANDS 12 50 3 -75 6 100
SPAIN 9 -18 13 44 10 -23
JAPAN 8 33 9 13 8 -11
GERMANY 8 167 15 88 12 -20
DENMARK 7 0 8 14 11 38
BELGIUM 4 -33 4 0 3 -25
AUSTRALIA 4 -20 4 0 0 -100
AUSTRIA 4 33 0 -100 0 0
SOUTH KOREA 4 0 2 -50 7 250
NEW ZEALAND 4 0 7 75 3 -57
SWEDEN 3 50 3 0 6 100
SWITZERLAND 3 0 7 133 15 114
ITALY 2 -50 4 100 11 175
INDIA 2 -33 2 0 0 -100
MEXICO 0 -100 3 0 0 -100
NORWAY 0 -100 0 0 2 0
PEOPLES REPUBLIC CHINA 0 0 7 0 8 14
AUSTRALIA 0 0 0 0 11 0
SINGAPORE 0 0 0 0 5 0
BRAZIL 0 0 0 0 4 0 * Includes England, Scotland, Ireland and Wales.
It can be seen from Table 3 the biggest contribution to the journal Econometrica is
made by the USA scientists. Scientist from the United Kingdom are on the second
place. In addition, high impact on the journal Econometrica has such countries as
France, Israel, Canada, Netherlands and Spain. Therefore, non-English speaking
countries also have an impact on the journal.
4: Results and Discussion: Publication and Citation analysis
31
It is reasonable to look for the dynamics for ten main contributors, because dynamics
for smaller contributors is very volatile. The number of publications for main ten
contributors in 1999 - 2003 comparing for previous period (1994 - 1998) is increased
in general. However, decline shows Canada and Spain. The number of publications
from Canada’s scientists dropped more than a half. This lead to the loss of the
position of Canada’s scientists in journal. However, impact of scientists from
Germany, Israel and UK raised significantly.
In 2004 - 2008 contribution of top six countries is dropped. This leads to the growth
of contribution from other countries. Therefore, authors from Germany, Spain,
Denmark, and Japan increase the impact on journal Econometrica.
In contrast to the previous period, top contributors showed considerable growth of
publications, when scientists from Germany, Spain, and Japan contribute less to the
journal. However, the greatest growth of the publications was shown by the authors
from Canada in 2009 - 2013.
4.4 Organisational distribution of contributors in journal
Econometrica
WoS Core Collection database shows that there are 405 organisations, which
contribute to the journal Econometrica. However, WoS does not provide information
about organisation for all articles. Figure 5 presents organisational distribution of
publications.
Figure 5 shows all organizations which contribution for the journal Econometrica is
more than 100 publication. Organization with smaller contribution is combined into
OTHERS. Nearly 15% of the publications have not information about organization.
As can be seen from the Figure 5 first five organisations, which are biggest
contributors for the journal, is from the USA. Two organisations which contribution
is more than 100 publications are from the United Kingdom. However, the rest is from
the USA.
4: Results and Discussion: Publication and Citation analysis
32
Figure 5: Organisational distribution of publications in journal Econometrica
in 1999 - 2013
*Includes UC Berkeley; UC Davis; UC Irvine; UC Los Angeles; UC Merced; UC Riverside;
UC San Diego; UC San Francisco; UC Santa Barbara; UC Santa Cruz
**Includes 18 individual institutions, e. g. Birkbeck; King's College London; London
Business School; The London School of Economics and Political Science; Queen Mary
University of London; Royal Holloway; St George's; University College London.
4.5 Citations distribution of publication in journal
Econometrica
As was mentioned previously there are 7056 publication in journal Econometrica
from 1933 to July 2014. This publication received 339364 citations.
Distribution of citations is presented on Figure 6. The biggest number of citations
receive publication from Kahneman and Tversky (1979) Prospect theory: An analysis
of decision under risk – 9645. This is 2.8% of the total citations which journal
Econometrica receives since 1933. However, this number is unique for this journal.
UNIVERSITY OF CALIFORNIA SYSTEM*
NORTHWESTERN UNIVERSITY
YALE UNIVERSITY
STANFORD UNIVERSITY
MASSACHUSETTS INSTITUTE OF TECHNOLOGYMITUNIVERSITY OF LONDON**
HARVARD UNIVERSITY
UNIVERSITY OF CHICAGO
PRINCETON UNIVERSITY
UNIVERSITY OF CALIFORNIA BERKELEY
LONDON SCHOOL ECONOMICS POLITICALSCIENCEUNIVERSITY OF PENNSYLVANIA
OTHERS
NOT SPECIFED
4: Results and Discussion: Publication and Citation analysis
33
Next most cited publication receives more than 2000 less citation (7330). This shows
that distribution of citation is highly skewed. In addition, there are 2697 publications
that do not receive any citation. Distribution of the citations, which was received by
the journal Econometrica, is presented on the Figure 6.
Figure 6: Citations distribution of publications in journal Econometrica in 1933-2014
As can be seen from the Figure 6 citations in journal Econometrica follow
approximately Zipf distribution. Therefore, it can be proved by plotting data about
citations on a log-log graph (Fig. 7). Publications, which received zero citations, were
removed from the Figure 7.
Figure 7 proves the assumption that citations, which was received by the journal
Econometrica, can be approximated with Zipf law. In addition, as the figure below
illustrates, there is a small number of highly cited articles and high number of
publication receives few citation.
For example, more than 3% of the publication in Econometrica receive only 2
citations. The percentage of articles which receive 1 citation is 4,6. In addition, 38,2%
publications in journal Econometrica was never cited.
0
500
1000
1500
2000
2500
3000
0 2000 4000 6000 8000 10000
Total citations
Qu
anti
tyo
f ar
ticl
es
4: Results and Discussion: Publication and Citation analysis
34
Figure 7: Zipf distribution of citations of publications in journal Econometrica
in 1933 - 2014
On the other hand, 20 most cited articles is responsible for nearly 20% of all citation
in journal Econometrica.
Twenty most cited articles is presented in Table 4.
Most of the publications presented in Table 4 is methodological papers. Some of the
authors in this publication describes innovative approach to the evaluation of certain
processes. Therefore, they described some tests and methods, which were named
after them. For example, White test, Engle-Granger two-step method, Hausman test,
Dickey-Fuller test, and Chow test.
As can be seen from the table below some articles in journal Econometrica are highly
cited. Some authors repeats several times. For example White and Engle. The oldest
most cited publication is publication by Nash. It was published in 1950. The newest
most cited publication is publication by Johansen, which was published in 1991. There
is one more publication, which was published in 1991, by Nelson. However, it receive
less citation than previous one. Interestingly, the three most cited publication were
published within small interval of time in 1979 - 1980.
0
1
2
3
4
5
6
7
0 2 4 6 8 10
LN(Total Citation)
LN(Q
uan
tity
of
arti
cles
)
4: Results and Discussion: Publication and Citation analysis
35
Table 4: Most cited articles in journal Econometrica in 1933-2014
No. Publication Citations % of total citations
1 Kahneman, D., Tversky, A. (1979). Prospect theory - analysis of decision under risk
9645 2.8
2 White, H. (1980) A heteroskedasticity-consistent covariance-matrix estimator and a direct test for heteroskedasticity
7330 2.2
3 Heckman, J. J. (1979). Sample selection bias as a specification error.
6133 1.8
4 Engle, R. F., & Granger, C. W. (1987). Co-integration and error correction: representation, estimation, and testing.
5892 1.7
5 Engle, R. F. (1982). Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation.
4479 1.3
6 Newey, W. K., & West, K. D. (1986). A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix.
3597 1.1
7 Hausman, J. A. (1978). Specification tests in econometrics. 3592 1.1
8 Hansen, L. P. (1982). Large sample properties of generalized method of moments estimators.
2943 0.9
9 Dickey, D. A., & Fuller, W. A. (1981). Likelihood ratio statistics for autoregressive time series with a unit root.
2462 0.7
10 Nash Jr, J. F. (1950). The bargaining problem. 2111 0.6
11 Pratt, J. W. (1964). Risk aversion in the small and in the large.
2090 0.6
12 Sims, C. A. (1980). Macroeconomics and reality. 2065 0.6
13 Johansen, S. (1991). Estimation and hypothesis testing of cointegration vectors in Gaussian vector autoregressive models.
2008 0.6
14 Koenker, R., & Bassett Jr, G. (1978). Regression quantiles. 1966 0.6
15 Perron, P. (1989). The great crash, the oil price shock, and the unit root hypothesis.
1941 0.6
16 Chow, G. C. (1960). Tests of equality between sets of coefficients in two linear regressions.
1925 0.57
17 Hamilton, J. D. (1989). A new approach to the economic analysis of nonstationary time series and the business cycle.
1870 0.6
18 Cox, J. C., Ingersoll Jr, J. E., & Ross, S. A. (1985). A theory of the term structure of interest rates.
1780 0.5
19 White, H. (1982). Maximum likelihood estimation of misspecified models.
1710 0.5
20 Nelson, D. B. (1991). Conditional heteroskedasticity in asset returns: A new approach.
1635 0.5
4: Results and Discussion: Publication and Citation analysis
36
The total citations, which receive journal Econometrica each year, are different.
However, there are trend for growth as can be seen from Figure 8.
Figure 8: Citations distribution of publications in journal Econometrica
in 1933 - 2014
Highest number of citations journal received in 2013. However, this is cumulative
trend and growth of received citations is connected to the growth of the number of
publications.
4.6 Citation distribution of source titles and research areas of
journal Econometrica
Journal Econometrica is one of the most highly cited journal in the field. Journal
impact factor in 2013 was 3.5. Journal Econometrica research areas are Mathematics,
Mathematical Methods in Social Sciences, and Business Economics.
However, Econometrica is most cited by the journals that has connection either for
the econometrics or for the economic studies in general. Table 5 presents ten
journals, which are the largest users of information from journal Econometrica.
0
5000
10000
15000
20000
25000
19
33
19
36
19
39
19
42
19
45
19
48
19
51
19
54
19
57
19
60
19
63
19
66
19
69
19
72
19
75
19
78
19
81
19
84
19
87
19
90
19
93
19
96
19
99
20
02
20
05
20
08
20
11
20
14
4: Results and Discussion: Publication and Citation analysis
37
Table 5: Journals most frequently citing journal Econometrica
Source titles Records
Econometrica 3389
Economics Letters 3242
Applied Economics 2971
Journal of Economic Theory 2774
Journal of Econometrics 2685
American Economic Review 2633
Review of Economics and Statistics 1789
American Journal of Agricultural Economics 1554
Journal of Economic Dynamics Control 1476
Review of Economic Studies 1450
As can be seen from the table above journal Econometrica has high number of self-
citations. However, what is interesting in this data is that number of citations from
Economics Letters is very close to the self-citation level. In addition, field of research
of journal Economics Letters according to WoS is Business Economics. This is similar
to one of the Econometrica research areas. Therefore, strong connection between
these two journals is expected. Next journals presented in the Table 5 also have at
least one similar field of research with Econometrica. Research areas of Journal of
Econometrics and Econometica is similar. However, American Journal of Agricultural
Economics has one different research area – Agriculture. Therefore, it can be
assumed that journal Econometrica has cross-disciplinary connections.
To prove that assumption research areas of the journals that citing Econometrica
were retrieved and presented at the Table 6.
It is clear that most citations journal Econometrica receive from journals with similar
research areas. However, very high number of citation received from journals with
research area Business economics shows that Econometrica has strong influence in
this field. This means that econometrics methods and models are widely used in
economic studies.
4: Results and Discussion: Publication and Citation analysis
38
Citations from the journals, with research areas as mathematics and mathematical
methods in social sciences, shows that there are strong connections and diffusion of
knowledge in this fields.
Table 6: Research areas of the journals most frequently citing journal Econometrica
Research Areas records
Business economics 128482
Mathematics 28840
Mathematical methods in social sciences 20342
Computer science 8472
Operations research management science 8387
Environmental sciences ecology 7844
Engineering 6754
Government law 6333
Public administration 5391
Psychology 5079
However, as can be seen from the Table 6, Econometrica receives number of citations
from journals with different research areas. Data about biggest importers from the
journal is presented on Figure 9.
Data presented on the Figure 9 shows that Econometrica has strong cross-disciplinary
connections with others research areas.
The strongest relationship Econometrica established with journals connected to
Computer science. However, three main journals, which cited Econometrica from this
field, is also connected to Mathematics. Therefore, it can be assumed that
connections of this journals is stronger than data from Figure 9 shows, because it also
can be established through other journals connected to Mathematics.
Operations research management science is another research field from which
Econometrica receives high number of citations. In addition, three most connected
journals to Econometrica from this research area is also indexed as journals
connected to Business economics.
4: Results and Discussion: Publication and Citation analysis
39
Figure 9: Importers of knowledge from journal Econometrica
The most striking result to emerge from the data presented on Figure 9 is that
research area of Environmental sciences ecology is big importer of the knowledge
from Econometrica. However, second and third biggest importers of this field (Journal
of Environmental Economics and Management and Regional Science and Urban
Economics) is also connected to Business economics. Nonetheless, biggest importer
of this field journal Energy Policy has completely different research areas.
Authors from the journal Energy Policy uses methods of econometric modelling and
testing which was introduced in the journal Econometrica. Another interesting
findings is that authors of the journal Energy Policy implement different methods
from the journal to their researches. Some authors cites the journal Econometrica
more than once. Therefore, knowledge from the journal Econometrica can be used
in different research areas for different purposes.
0
2000
4000
6000
8000
Number of recordsCOMPUTER SCIENCE OPERATIONS RESEARCH MANAGEMENT SCIENCEENVIRONMENTAL SCIENCES ECOLOGY ENGINEERINGGOVERNMENT LAW PUBLIC ADMINISTRATIONPSYCHOLOGY AGRICULTURESOCIAL SCIENCES OTHER TOPICS SOCIOLOGYURBAN STUDIES HEALTH CARE SCIENCES SERVICESINTERNATIONAL RELATIONS TRANSPORTATIONPUBLIC ENVIRONMENTAL OCCUPATIONAL HEALTH GEOGRAPHY
40
5 Results and Discussion: Citation patterns of
publications
5.1 Introduction
Discussion of main patterns in citation will be based on the most cited publication of
the journal Econometrica.
It was chosen four articles that will be reviewed. First, this is most cited publication
in the journal. It is reviewed number of citations, which was received by the article
and research areas of journals, which cited this article.
Second, it is discussed main trends in citations of the oldest publication in Table 4.
Third, discussion about citation patterns of the newest article in the Table 4 is made.
Then, it is shown main tendencies in citations for the most cited article, which was
published in this century.
Finally, the attempt is made to compare citation patterns for these four articles.
5.2 Citation pattern of most cited publication in journal
Econometrica
The most cited article in the journal Econometrica is publication of two authors
Kahneman and Tversky. It was published in 1979 and named Prospect theory -
analysis of decision under risk. This article has received more than 9645 citation since
1979. This is 2.8% of total number of citations, which was received by the journal
Econometrica. In average, this article receives 271.1 citation per year.
Kahneman and Tversky (1979) in their research describe choices of the person in
different situations connected with risk. They criticise utility theory, because people
5: Results and Discussion: Citation patterns of publications
41
tend to change their attitudes to reward and loss because of level of risk and their
own value function. The decision is not based on maximal utility. It is individual choice
and this choice does not always reflect attitudes to money. In addition, Kahneman
and Tversky (1979) describes probabilistic insurance and differences in attitudes to
the risk when it connected with reward or loss. Differences in risk acceptance appears
because a value function is not a straight line.
Publication describes all this problems and present reasons for risk acceptance in
different situations. Therefore, high number of citations to this publication is
predictable. Data presented on the Figure 10 shows trends in citations of this article.
Year 2014 was removed from the figure, because data about this year is constantly
changing.
Figure 10: Citation over time to the article by Kahneman and Tversky
As can be seen from the Figure above the number of the citation for this publication
is constantly growing. In 1979, it received only four citations. However, in the next
year, this number quadrupled and publication receives 16 citations. The gradual
growth of the number of citations can be observed to the 2003. However, in 1991
and 1993 was present small decline in the number of citations, however, it did not
change the tendency. From the 2005, sharp increase in number of citations to this
publication can be observed. This number peaked in 2013 and was 837 citations.
0
100
200
300
400
500
600
700
800
900
1979 1981 1983 1985 1987 1989 1991 1993 1995 1997 1999 2001 2003 2005 2007 2009 2011 2013
Nu
mb
er o
f ci
tati
on
s
Year
5: Results and Discussion: Citation patterns of publications
42
However, the reasons of citing this article is changed from the 1979. For example, in
1979 some authors used methodology of the Kahneman and Tversky, gave credit for
related work and corrected their work. However, in 2013 most of the reviewed
publications, which cited Kahneman and Tversky, just mention their names, which
can be considered as paying homage to pioneers.
Publication by Kahneman and Tversky Prospect theory - analysis of decision under risk
can be considered as multidisciplinary. Because, on the one hand, it is stongly
connected to the economics, but, on the other hand, it describes psychological
behaviour of the person. To prove that data about research areas of the journals
which cited this article is displayed in the Figure 11.
Figure 11: Research areas most frequently cited Kahneman and Tversky Prospect
theory - analysis of decision under risk
As was expected the two research areas, which cited this publication the biggest
number of times, is Business economics and Psychology. The most striking result to
emerge from the figure above is that Government law is cited this article a large
4834
2458
653516 504 439 405 395 365 354 353 276 226 189 185
2356
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
BUSINESS ECONOMICS PSYCHOLOGY
GOVERNMENT LAW OPERATIONS RESEARCH MANAGEMENT SCIENCE
COMPUTER SCIENCE MATHEMATICAL METHODS IN SOCIAL SCIENCES
MATHEMATICS ENGINEERING
NEUROSCIENCES NEUROLOGY ENVIRONMENTAL SCIENCES ECOLOGY
SOCIAL SCIENCES OTHER TOPICS HEALTH CARE SCIENCES SERVICES
PUBLIC ENVIRONMENTAL OCCUPATIONAL HEALTH TRANSPORTATION
BEHAVIORAL SCIENCES OTHERS
5: Results and Discussion: Citation patterns of publications
43
number of times. However, after analysing articles from the Government law, it can
be concluded that these articles is closely connected to economy, psychology or
describes voting behaviour of the people.
5.3 Citation pattern of oldest most cited publication in journal
Econometrica
Oldest most cited publication in journals Econometrica is article produced by Nash in
1950. This publication has received more than 2111 citation since 1950. It is 0.6% of
total citations, which was received by the journal Econometrica.
Nash (1950) describes bargaining between two individuals. This is done by deploying
mathematical model. Therefore, author uses Theory of Games and basic concepts of
it in paper. However, Nash (1950) develops his own utility theory of the individual
and introduces main assumption for it in the paper. According to Nash (1950) all
anticipation between two individuals would be based on their utility functions. In
addition, author outlines that he takes into account “only those cases in which there
is a possibility that both individuals could gain from the situation” (Nash, 1950, p.
158). Finally, it is described in paper the solution for the fair bargaining between two
individuals. This is done by finding best solution for them based on their utility
functions and the set of possibilities. Nash (1950) proposes the solution when two
utility function is maximized, however this is not an ultimate bargaining for one
individual, this is fair bargaining.
It is oldest article from table 4, therefore it receives less citation per year in average
than others. This number is 32.6. However, average number does not describe
tendencies in citation patterns of this article. Therefore, data about trends in citations
of Nash is presented on Figure 12. Year 2014 was removed from the figure, because
data about this year is constantly changing.
5: Results and Discussion: Citation patterns of publications
44
Figure 12: Citation over time to the article by Nash
As can be seen from the Figure 12 the number of citation to this paper show gradual
growth. It was not cited in first 3 years after publication. The most striking results
from to emerge the figure above is that number of citation rise slowly. This increase
was not connected with either the award of the John von Neumann Theory Prize in
1978 or the Leroy P. Steele Prize in 1999. In addition, award of the Nobel Memorial
Prize in Economic Sciences in 1994 did not affect the number of citation. However,
sharp grows of the citation count in 2005 did not connected with any award. This was
an outcome of growing citations from Business economics journals. However, it can
be assumed that movie about John Nash “Beautiful Mind”, which was released in
2001 and received four Academy Awards in 2002, Golden Globe Award and several
BAFTA Awards is connected to grows of citation count.
However, high number of citation from Business economics research areas can be
explained because publication is connected to this area. In addition, publication by
Nash cited by other research areas, as can be seen from the Figure 13.
0
20
40
60
80
100
120
140
160
19
53
19
55
19
57
19
59
19
61
19
63
19
65
19
67
19
69
19
71
19
73
19
75
19
77
19
79
19
81
19
83
19
85
19
87
19
89
19
91
19
93
19
95
19
97
19
99
20
01
20
03
20
05
20
07
20
09
20
11
20
13
Nu
mb
er o
f ci
tati
on
s
Year
5: Results and Discussion: Citation patterns of publications
45
Figure 13: Research areas most frequently cited Nash The bargaining problem
Citations from research areas such as Business economics, Mathematical methods in
social sciences and Mathematics are citations from the same field. However, citations
from Computer science are considered as export of knowledge. Citations to this
publications from Computer science papers were mostly made when publication is
connected to the application, which use Nash approach in the Game Theory. This
shows effective import of knowledge from this publication to the publication in
different research areas.
The trend for citations has changed for this paper since it was published. In 1953 it
was cited by the two publications from Econometrica. In addition, one citation was
made by Nash. These citations were made to present previous researches in this field.
However, in 1958 this publication starts to import knowledge for other disciplines. It
was cited by the journal World Politics as well as with journals from Business
economics research area. In the next year it was cited by the journal from
Government law American Political Science Review this shows interdisciplinary
connections of publication by Nash. In 2013 this publication was cited 119 times. This
0
200
400
600
800
1000
1200
1400
BUSINESS ECONOMICS COMPUTER SCIENCE
MATHEMATICAL METHODS IN SOCIAL SCIENCES MATHEMATICS
ENGINEERING OPERATIONS RESEARCH MANAGEMENT SCIENCE
GOVERNMENT LAW SOCIAL SCIENCES OTHER TOPICS
PSYCHOLOGY TELECOMMUNICATIONS
INTERNATIONAL RELATIONS ENVIRONMENTAL SCIENCES ECOLOGY
AUTOMATION CONTROL SYSTEMS PHILOSOPHY
PUBLIC ADMINISTRATION OTHERS
5: Results and Discussion: Citation patterns of publications
46
was made by the authors from different research areas. It can be assumed that they
pay homage to the pioneers as well as provide background reading.
5.4 Citation pattern of newest most cited publication in journal
Econometrica
Newest most cited publication in journals Econometrica is article produced by
Johansen in 1991. Since 1991 Estimation and hypothesis testing of cointegration
vectors in Gaussian vector autoregressive models has received more than 2008
citations. In average it is 84.7 times in a year. This is 0.6% of total citations of the
journal Econometrica.
Johansen (1991) describes “maximum likelihood estimators and likelihood ratio tests
for cointegration in Gaussian vector autoregressive models” in his paper. Based on
data on money demand from two countries author outlines probability theory when
processes in one country can make asymptotic inference to the processes in another.
First, this paper describes cointegration model and the tests, which can measure,
cointegration rank.
As can be seen from the Figure 14 citations for this article has a tendency to growth.
However, in 1997 the number of citation for this paper started to fall. This tendency
has completely changed since 2003. It has shown gradual growth since 2003 and
peaked in 2011. In 2012 and 2013 it can be seen small decline in number of citation.
Nonetheless, small decrease in citation count to the publication by Johansen can be
considered as temporal.
Publication by Johansen is methodological paper with strong accent mathematical
methods and models of economic processes. This models mostly used only in
economic field therefore, this article has not strong connection with other disciplines.
This can be seen from the Figure 15, because highest number of citations this paper
receives from the papers in the same research areas. Scientists, who cited this article,
adopt this methodology to their researches. Therefore, this publication is strongly
oriented inside the discipline.
5: Results and Discussion: Citation patterns of publications
47
Figure 14: Citation over time to the article by Johansen
Figure 15: Research areas most frequently cited Johansen Estimation and hypothesis
testing of cointegration vectors in Gaussian vector autoregressive models
However, there are some disciplines, which cited this article and import knowledge.
For example, Environmental science ecology researchers adopt this methodology to
0
20
40
60
80
100
120
140
160N
um
ber
of
cita
tio
ns
Year
0
500
1000
1500
2000
BUSINESS ECONOMICS MATHEMATICS
MATHEMATICAL METHODS IN SOCIAL SCIENCES ENVIRONMENTAL SCIENCES ECOLOGY
COMPUTER SCIENCE ENERGY FUELS
AGRICULTURE INTERNATIONAL RELATIONS
OPERATIONS RESEARCH MANAGEMENT SCIENCE ENGINEERING
SOCIAL SCIENCES OTHER TOPICS PUBLIC ADMINISTRATION
GOVERNMENT LAW URBAN STUDIES
TRANSPORTATION OTHERS
5: Results and Discussion: Citation patterns of publications
48
their own publications. Some publications in Computer science research area has
strong connection to Business economy. Publication from Energy fuels research area
also adopted methodology proposed by the Johansen in their researches.
5.4 Citation pattern of most cited publication in journal
Econometrica that was published in XXI century
The most highly cited article which was published in this century is article by Melitz.
It was published in 2003 and named The impact of trade on intra-industry
reallocations and aggregate industry productivity. It has been cited more than 1293
times. It is 107.8 times each year in average.
Article, published by Melitz (2003), develops “dynamic industry model with
heterogeneous firms to analyse the role of international trade as a catalyst for these
inter-firm reallocations within an industry”.
Data about citations of this article from 2004 to 2013 is presented on Figure 16. Year
2014 was removed from the figure, because data about this year is constantly
changing.
Figure 16: Citation over time to the article by Melitz
0
50
100
150
200
250
300
2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
Nu
mb
er o
f ci
tati
on
s
Year
5: Results and Discussion: Citation patterns of publications
49
Article published by Melitz was firstly cited in 2004. In that year it received 13
citations. Since 2004, it has shown rapid growth in number of citations. In 2013 it was
cited by 252 other publications.
Figure 17: Research areas most frequently cited Melitz The impact of trade on intra-
industry reallocations and aggregate industry productivity
This is methodological paper, which focus on economic problem, therefore, it can be
suggested that publication does not receive high number of citation from different
research areas. Data from Figure 17 proves this assumption. It is highly cited by the
journal from the same research area. Therefore, this publication orients inside the
discipline.
However, publication by Melitz receives citations from publications that is connected
to International relations. This publication also have connection to economy and uses
to provide background reading and pay homage to pioneers. Similar to this pattern is
presented in papers from Public administration research area.
0
200
400
600
800
1000
1200
1400
BUSINESS ECONOMICS INTERNATIONAL RELATIONS
PUBLIC ADMINISTRATION MATHEMATICAL METHODS IN SOCIAL SCIENCES
ENVIRONMENTAL SCIENCES ECOLOGY GOVERNMENT LAW
GEOGRAPHY MATHEMATICS
AGRICULTURE URBAN STUDIES
OPERATIONS RESEARCH MANAGEMENT SCIENCE ENGINEERING
AREA STUDIES NUTRITION DIETETICS
FOOD SCIENCE TECHNOLOGY TRANSPORTATION
SOCIAL SCIENCES OTHER TOPICS
5: Results and Discussion: Citation patterns of publications
50
5.5 Comparison of citation patterns
The publications described earlier are the most cited publication in journal
Econometrica. However, this publications are cited by other scientists in different
ways. For example, publication by Kahneman and Tversky or Nash is big exporters of
the knowledge to the other disciplines. Publications by Johansen or Melitz also export
knowledge to other research areas, however, this publications is more oriented
inside the discipline. It is hard to determine reasons for citing these articles, however
it can be assumed that scientists from other filed use these publications as
methodological base for their researches.
Article published by Melitz in first year after publication received biggest number of
the citations. It can be assumed that articles became more available for the citations
because of the Internet. In addition, this article shows the fastest growth of the
citations count. In 2010, it has higher number of the citations than publications by
Nash or Johansen.
In addition, article published by Melitz gets higher number of citations than article
published by Nash or Johansen. However, paper by Kahneman and Tversky is still
most cited article in the journal Econometrica and has much higher number of
citations each year (Figure 18). Figure 18 contains information about citation count
to the year 2013 because data about 2014 is constantly changing.
As can be seen from the Figure 18 number of citations received by the articles is
growing. Most rapid growth shows the article published by Kahneman and Tversky.
Therefore, it can be assumed that this article will be the most cited publication in the
journal Econometrica for several more years.
5: Results and Discussion: Citation patterns of publications
51
Figure 18: Citation over time to the article by Kahneman and Tversky, Nash,
Johansen, Melitz
This shows that articles published in journal Econometrica can be interested for
scientists from different research areas. This fact has great impact on citations that is
received by each publication. It was shown that some articles in a journal
Econometrica is oriented inside the discipline and others are importer of knowledge.
The most cited papers in journal is methodological papers, which propose new
approaches to solution of different problems. However, citation count is not highly
related to the time of the publication. Publications, which were produced in recent
years, can receive high number of citations.
0
100
200
300
400
500
600
700
800
900
19
53
19
55
19
57
19
59
19
61
19
63
19
65
19
67
19
69
19
71
19
73
19
75
19
77
19
79
19
81
19
83
19
85
19
87
19
89
19
91
19
93
19
95
19
97
19
99
20
01
20
03
20
05
20
07
20
09
20
11
20
13
Kahneman and Tversky Kahneman and Tversky average
Nash Nash average
Johansen Johansen average
Melitz Melitz average
52
6 Conclusions
6.1 Key findings
From the first part of the research it can be concluded, that there were few
fluctuations in number of publications in journal Econometrica. This fluctuations can
be explained either general situation in the world (minimal level of publication in
1944) or policy of the editorial board. For example, increase from 1945 to 1964 can
be explained as development of the journal Econometrica. Maximal number of
publication in 1971 is the results of edition of big amount of meeting abstracts from
Second World Congress of the Econometric Society, Tenth Indian Econometric
Conference and North American Regional Conference.
Than it was find out that, there are significant differences in contribution for the
journal Econometrica from the scientists of different countries. Greatest contribution
is made by USA’s scientists. Second place is contributions from the UK’s scientists and
third is Canada. However, in recent years greatest contributors to the journal are
scientists from USA, UK and France, which shows the international impact of the
journal. In addition, the internationalization of the journal can be observed because
of growth in number of publications from authors from Israel, Canada, Netherlands
and Spain.
University of California System consists of ten individual universities and makes the
biggest contribution to the journal. The biggest contribution to the journal
Econometrica is made by institution that situated in USA. However, universities from
the United Kingdom also make significant contribution to the journal. For example,
University of London, which consists of 18 individual institutions, is on the sixth place
for its contribution.
6: Conclusions
53
Journal Econometrica is highly cited journal. However, different publications received
different number of citations. For example, twenty most cited articles receives almost
twenty percent of total citations. However, 2697 publications in the journal
Econometrica were never cited by others. This shows that distribution of citations is
highly skewed. In addition, in the Chapter 4 was proposed that distribution of
citations could be approximated with Zipf law
From the data analysed in chapter four can be concluded that journal Econometrica
has influence on journals from other fields. It is obvious that strongest relations
journal has with journals from similar research areas. However, knowledge from
Econometrica is imported by Computer science, Operations research management
science, Environmental sciences ecology and other sciences and social sciences.
In the Chapter Five citation patterns of four articles were analysed. It were
publications by Kahneman and Tversky (1979), Nash (1950), Johansen (1991) and
Melitz (2003).
Publication by Kahneman and Tversky is by far the most highly cited publication in
journal Econometrica. In 1979 it introduces new ideas and approaches of evaluation
decision making processes under risk. This publication was highly cited. Because of
this, it becomes popular among wide range of the scientist. However, nowadays
some citations is made because scientists pay homage to pioneers. In addition, high
number of citation, which receives this article, is connected to the fact that it is
interesting for different research areas.
Article published by Nash in 1950 was not cited in first years after publication.
However, it shows gradual growth of citation till nowadays. It was cited by the wide
range of scientists from different research areas. This shows that this article is
exporter of knowledge to other disciplines.
Publication by Johansen is newest one of the most highly cited article in journal
Econometrica. It can describes econometric methods and it can be suggested that it
is oriented inside the discipline. However, other disciplines use methodology
developed by Johansen as well as Business economic. For example, scientists from
6: Conclusions
54
Environmental science ecology and Energy fuels adopt this methodology to their own
researches.
Melitz’s publication The impact of trade on intra-industry reallocations and aggregate
industry productivity is one of highly cited article which was produced in recent years.
This article become highly cited in a short period. However, this publication describes
economic problem and receives most citations from the journals with the same
research areas. This shows that publication orients inside the discipline. In addition,
some citations from other research areas is mostly made because topic of the
research is highly connected to economy.
6.2 Meeting the aim and objectives
The overall aim of the study was ‘to make bibliometric analysis of journal
Econometrica and investigate export relationship between econometrics and other
research areas based on this journal’. To reach that aim next objectives were set up:
Objective 1: To carry out literature review presenting previous bibliometric analysis
of different journals and their impact on different scientific areas.
Chapter 2 of the study presents literature review. This part includes review of the
bibliometric studies of single journal, citation analysis and knowledge diffusion.
Therefore, impact of the journal on different scientific areas was assessed with
citation analysis and diffusion of knowledge from one research area to others.
Objective 2: To outline year wise distribution of publications in the journal
Econometrica.
The year wise distribution of publication in the journal Econometrica was presented
and analysed in second part of Chapter 4. It includes discussion of existed patterns
and explanation of unexpected results.
Objective 3: To identify geographical distribution of contributors to the journal
Econometrica.
6: Conclusions
55
Geographical distribution of contributors to the journal Econometrica is presented in
third part of Chapter 4. Distribution of contributors from different countries to the
journal Econometrica in 1933 - 2014 was discussed. In addition, recent trend in
contributions from different countries was presented and analysed.
Objective 4: To identify main organisations contributors to the journal
Econometrica.
Main organizations that contributes to the journal Econometrica is presented in forth
part of Chapter 4.
Objective 5: To describe citation trends of the journal Econometrica.
Citation trends in to the journal Econometrica is presented and discussed in fifth part
of Chapter 4. Analysis of total number of citation, which were received by to the
journal Econometrica. In addition, twenty most cited articles were presented. Finally,
total number of citation, which was received by the journal Econometrica, is
presented.
Objective 6: To explore which disciplines and journals export knowledge from the
journal Econometrica.
Export of knowledge from the journal Econometrica was discussed in final part of
Chapter 4 and in Chapter 5. Sixth part of Chapter 4 presents data about export of
knowledge by the journals and to the different research areas from the journal
Econometrica in general. Chapter 5 is focused on four most cited articles from the
journal. In this part was described export of knowledge from specific articles.
6.3 Recommendation for further research
Carry out similar study to the other econometric journals and compare
received results with findings of this study.
Analyse more specific publications from the journal Econometrica and add
received results to findings of this study.
6: Conclusions
56
Make deeper analysis of reasons for citing specific publication by the authors
from the other research areas.
Total word count: 10,131
57
7 Bibliography
Amsterdamska, O., Leydesdorff, L. (1989). Citations: indicators of significance?.
Scientometrics, 15(5-6), 449-471.
Anyi, W. U. K., Anuar, N. B., Zainab, A. N. (2009). Bibliometric studies on single
journals: a review. Malaysian Journal of Library & Information Science, 14(1),
17-55.
Bar-Ilan, J. (2010). Citations to the “Introduction to informetrics” indexed by WOS,
Scopus and Google Scholar. Scientometrics, 82(3), 495-506.
Bergman, E. M. L. (2012). Finding Citations to Social Work Literature: The Relative
Benefits of Using Web of Science, Scopus, or Google Scholar. The Journal of
Academic Librarianship, 38(6), 370-379.
Biswas, B. C., Roy, A., Sen, B. K. (2007). Economic botany: a bibliometric study.
Malaysian Journal of Library & Information Science, 12 (1), 23-33.
Bedeian, A. G. (2005). Crossing disciplinary boundaries a epilegomenon for Lockett
and McWilliams. Journal of Management Inquiry, 14(2), 151-155.
Bonzi, S., Snyder, H. W. (1991). Motivations for citation: A comparison of self citation
and citation to others. Scientometrics, 21(2), 245-254.
Borkenhagen, A., Decker, O., Brahler, E., Strauss, B. (2002). The Journal of the Society
for Psychotherapy Research and the diversity of Psychotherapy Research: a
compilation and comparison of North American and European contributions.
Psychotherapy Research, 12(4), 491-514.
Bornmann, L., Daniel, H. D. (2008). What do citation counts measure? A review of
studies on citing behavior. Journal of Documentation, 64(1), 45-80.
7: Bibliography
58
Cronin, B., Meho, L. I. (2008). The shifting balance of intellectual trade in information
studies. Journal of the American Society for Information Science and
Technology, 59(4), 551-564.
Cronin, B., Pearson, S. (1990). The export of ideas from information science. Journal
of Information Science, 16(6), 381-391.
Elster, A. D., Chen, M. Y., Jenkins, C. B. (1994). The internationalization of the
American Journal of Roentgenology: 1980-1992. AJR. American Journal of
Roentgenology, 162(3), 519-522.
Damrosch, L.F. (2006). The "American" and the "international" in the American
Journal of International Law. American Journal of International Law, 100, 2-
16.
Ding, Y., Liu, X., Guo, C., & Cronin, B. (2013). The distribution of references across
texts: Some implications for citation analysis. Journal of Informetrics, 7(3),
583-592.
Folly, G., Hajtman, B., Nagy, J. I., & Ruff, I. (1981). Some methodological problems in
ranking scientists by citation analysis. Scientometrics, 3(2), 135-147.
Garfield, E. (1965, December). Can citation indexing be automated? In Statistical
association methods for mechanized documentation, symposium
proceedings, 189-192.
Garfield, E. (1979). Is citation analysis a legitimate evaluation tool?. Scientometrics,
1(4), 359-375.
Hammarfelt, B. (2011). Interdisciplinarity and the intellectual base of literature
studies: Citation analysis of highly cited monographs. Scientometrics, 86(3),
705-725.
Hellqvist, B. (2010). Referencing in the humanities and its implications for citation
analysis. Journal of the American Society for Information Science and
Technology, 61(2), 310-318.
7: Bibliography
59
Hessey, R., Willett, P. (2013). Quantifying the value of knowledge exports from
librarianship and information science research. Journal of Information
Science, 39(1), 141-150.
Jacso, P. (2005). As we may search: Comparison of major features of the Web of
Science, Scopus, and Google Scholar citation-based and citation-enhanced
databases. Current Science, 89(9), 1537-1547.
Jena, K. L. (2006). A bibliometric analysis of the journal 'Indian Journal of Fibre and
Textile Research', 1996–2004. Annals of Library and Information Studies,
53(1), 22-30.
Jishi, R. A., Willett, P. (2010). The Journal of Chemical Documentation and the Journal
of Chemical Information and Computer Sciences: publication and citation
statistics. Journal of Chemical Information and Modelling, 50(11), 1915-1923.
Johansen, S. (1991). Estimation and hypothesis testing of cointegration vectors in
Gaussian vector autoregressive models. Econometrica: Journal of the
Econometric Society, 59(6), 1551-1580.
Kahneman, D., Tversky, A. (1979). Prospect theory: An analysis of decision under risk.
Econometrica: Journal of the Econometric Society, 47(2), 263-291.
Lawani, S. M. (1982). On the heterogeneity and classification of author self‐citations.
Journal of the American Society for Information Science, 33(5), 281-284.
Larivière, V., Sugimoto, C. R., Cronin, B. (2012). A bibliometric chronicling of library
and information science's first hundred years. Journal of the American Society
for Information Science and Technology, 63(5), 997-1016.
Levitt, J. M., Thelwall, M., & Oppenheim, C. (2011). Variations between subjects in
the extent to which the social sciences have become more interdisciplinary.
Journal of the American Society for Information Science and Technology, 62(6),
1118-1129.
Leydesdorff, L. (1987). Towards a theory of citation?. Scientometrics, 12(5), 305-309.
7: Bibliography
60
Lockett, A., McWilliams, A. (2005). The balance of trade between disciplines: Do we
effectively manage knowledge?. Journal of Management Inquiry, 14(2), 139-
150.
Loebbecke, C., Leidner, D. (2012). The contribution of top IS publications to
subsequent research: A citation analysis. Communications of the Association
for Information Systems, 30, 423-439.
Lowry, P. B., Karuga, G. G., Richardson, V. J. (2007). Assessing leading institutions,
faculty, and articles in premier information systems research journals.
Communications of the Association for Information Systems, 20(1), 142-203.
MacRoberts, M. H., MacRoberts, B. R. (1989). Problems of citation analysis: A critical
review. Journal of the American Society for Information Science, 40(5), 342-
349.
MacRoberts, M. H., MacRoberts, B. R. (1996). Problems of citation analysis.
Scientometrics, 36(3), 435-444.
McElwee, G., Atherton, A. (2005). Publication trends and patterns in
entrepreneurship: The case of the International Journal of Entrepreneurship
and Innovation. Journal of Small Business and Enterprise Development, 12(1),
92-103.
Melitz, M. J. (2003). The impact of trade on intra‐industry reallocations and aggregate
industry productivity. Econometrica: : Journal of the Econometric Society,
71(6), 1695-1725.
Narin, F. (1976). Evaluative bibliometrics: the use of publication and citation analysis
in the evaluation of scientific activity. Washington, USA: Computer Horizons.
Nash Jr, J. F. (1950). The bargaining problem. Econometrica: Journal of the
Econometric Society, 18(2), 155-162.
Noruzi, A. (2005). Google Scholar: The new generation of citation indexes. Libri, 55,
170-180.
7: Bibliography
61
Peritz, B. C. (1983). Are methodological papers more cited than theoretical or
empirical ones? The case of sociology. Scientometrics, 5(4), 211-218.
Ravichandra Rao, I. K., Bhusan, S. B. (2008). Distributions of Multiple Authors in Two
Journals (JASIST and Scientometrics) and in the subject Software Studies: A
Case Study. Retrieved from: http://www.collnet.de/Berlin-
2008/RaoWIS2008dma.pdf.
Stigler, S. M. (1994). Citation patterns in the journals of statistics and probability.
Statistical Science, 94-108.
Tagliacozzo, R. (1977). Self-citations in scientific literature. Journal of Documentation,
33(4), 251-265.
The Concise Encyclopedia of Economics. 2008. "Ragnar Frisch." Library of Economics
and Liberty. Retrieved June 23, 2014 from the World Wide Web:
http://www.econlib.org/library/Enc/bios/Frisch.html
Torricella-Morales, R. G., Van Hooydonk, G., Araujo-Ruiz, J. A. (2000). Citation analysis
of cuban research. Part 1. A case study: the Cuban Journal of Agricultural
Science. Scientometrics, 47(2), 413-426.
Urata, H. (1990). Information flows among academic disciplines in Japan.
Scientometrics, 18(3), 309-319.
Willett, P. (2007). A bibliometric analysis of the Journal of Molecular Graphics and
Modelling. Journal of Molecular Graphics and Modelling, 26(3), 602-606.
Yan, E., Ding, Y., Cronin, B., & Leydesdorff, L. (2013). A bird's-eye view of scientific
trading: Dependency relations among fields of science. Journal of
Informetrics, 7(2), 249-264.
Information School Research Ethics Panel
Confirmation of a ‘no-risk’ application
Date: 12th
June 2014
TO: Olha Rodzen
The Information School Research Ethics Panel has examined the following
application:
Title: A bibliometric analysis of the journal ‘Econometrica’
Submitted by: Olha Rodzen
The Panel has concluded that the proposed research is classed as ‘no-risk’, and as
such does not require ethics approval. No further action needs to be taken.
This letter is the official record of ethics approval by the School, and should
accompany any formal requests for evidence of research ethics status.
Effective Date: 12th
June 2014
Dr Angela Lin
Research Ethics Coordinator
Information
School.
Access to Dissertation
A Dissertation submitted to the University may be held by the Department (or School) within which the
Dissertation was undertaken and made available for borrowing or consultation in accordance with
University Regulations.
Requests for the loan of dissertations may be received from libraries in the UK and overseas. The
Department may also receive requests from other organisations, as well as individuals. The conservation
of the original dissertation is better assured if the Department and/or Library can fulfill such requests by
sending a copy. The Department may also make your dissertation available via its web pages.
In certain cases where confidentiality of information is concerned, if either the author or the supervisor so
requests, the Department will withhold the dissertation from loan or consultation for the period specified
below. Where no such restriction is in force, the Department may also deposit the Dissertation in the
University of Sheffield Library.
To be completed by the Author – Select (a) or (b) by placing a tick in the appropriate box
If you are willing to give permission for the Information School to make your dissertation available in
these ways, please complete the following:
✔ (a) Subject to the General Regulation on Intellectual Property, I, the author, agree to this dissertation
being made immediately available through the Department and/or University Library for
consultation, and for the Department and/or Library to reproduce this dissertation in whole or
part in order to supply single copies for the purpose of research or private study
(b) Subject to the General Regulation on Intellectual Property, I, the author, request that this
dissertation be withheld from loan, consultation or reproduction for a period of [ ] years from
the date of its submission. Subsequent to this period, I agree to this dissertation being made
available through the Department and/or University Library for consultation, and for the
Department and/or Library to reproduce this dissertation in whole or part in order to supply
single copies for the purpose of research or private study
Name Olha Rodzen
Department Information School
Signed
Date 29.08.14
To be completed by the Supervisor – Select (a) or (b) by placing a tick in the appropriate
box
(a) I, the supervisor, agree to this dissertation being made immediately available through the
Department and/or University Library for loan or consultation, subject to any special restrictions
(*) agreed with external organisations as part of a collaborative project.
*Special
restrictions
(b) I, the supervisor, request that this dissertation be withheld from loan, consultation or
reproduction for a period of [ ] years from the date of its submission. Subsequent to this period,
I, agree to this dissertation being made available through the Department and/or University
Library for loan or consultation, subject to any special restrictions (*) agreed with external
organisations as part of a collaborative project
Name
Department
Signed Date
THIS SHEET MUST BE SUBMITTED WITH DISSERTATIONS BY DEPARTMENTAL REQUIREMENTS.