Maynooth University

What is Bibliometrics and how does it work?Ciaran Quinn, Research Support Librarian, Maynooth University6th May 2015

17/04/2015

Maynooth University

What is Bibliometrics?Statistical analysis of bibliographic data by

counting citations ◦ Measures patterns of authorship , publication and

the use of the literature. ◦ Usually Citation analysis of research outputs &

publications◦ Assess Research impact of individuals, groups,

institutions◦ Measuring by Author (H Index), Article (Plos), or

Publication (Journal Impact Factor)◦ Measure of Output not Quality (Quantitative Not

Qualitative !) Other measures could include funding received, number

of patents, awards granted, or qualitative measures such as peer review

17/04/2015

Maynooth University

Why use Bibliometrics? Measure of the Impact of your Research

◦ Your Publishing activity !◦ CV, Promotion, Tenure, Grants, Feedback to funding

bodies/Industry/Public◦ Showcase Individual/Group/Institutional Research

Identify Areas of Research Strengths/Weaknesses ◦ Inform Research Priorities

Identify highest impact or top performing Journals in a Subject Area◦ Where to Publish, learning about a particular subject

area, identify emerging areas of research Identify the top researchers in a subject area

◦ Collaborations/Competitors◦ Recruitment◦ Learning about a subject area

17/04/2015

Maynooth University

Examples of Metrics Scholarly Output: How many articles by an individual, Research

Group, University Citation counts (to measure the impact of an article or author)

No one comprehensive source for all articles !

Field-weighted cited impact (SciVal) Outputs in top percentiles (SciVal)

Journals in top 10% Good for early career researchers to be in the top journals !

Collaboration & Potential Collaboration metrics (SciVal) H-Index:

available from a number of sources

Journal Impact Factor (Web of Science) Compare Journals: SJR, SNIP, IPP (Scopus) ScImago Journal & Country Rank: SJR (uses Scopus Data)

Weighted by the Prestige of a Journal

Google Scholar Citations (Citations/H Index) Publish or Perish (Author Impact, Journal Impact) Good for unpublished material & books

17/04/2015

Maynooth University

Citation Tracking Tools Key Tools

Web of Science/Incites: ◦ Arts & Humanities, Social Sciences & Science

Citation Indexes (Thomson Reuters)Scopus/SciVal:

◦ Citation Overview/ Tracker, Analyse Results, Author Evaluator (Elsevier)

◦ Also uses Mendeley Readership Stats !SciMago: Journal & Country Rank (Elsevier)Publish or Perish (Harzing)Google Scholar Citations

17/04/2015

Maynooth University

Other Tools (More Limited !)

PubMed (Cited in PubMed Central) EBSCO: Find Citing Articles (Cited by EBSCO) IEEE Xplore (Electrical Eng & Computer Science)Cited by

IEEE ACM Digital Library (Cited by) JSTOR: (Items citing this item) Anthro Source (Citation alerts) SciFinder (Get References & Get Citing Options) Cite Seer citation statistics

◦ Computer & Information Science MathsSciNet: Author & Publications Citations Altmetics tools: Look at non traditional metrics at article

level◦ Impact Story, Altmetrics.com, Mendeley (Elsevier) Plum Analytics

(EBSCO)

17/04/2015

Maynooth University

Other Journal Rankings listing

Washington & Lee University School of Law: Law Library Law Journals: Submissions and Ranking, 2006 - 2013 :Journal Citations

http://lawlib.wlu.edu/LJ/Association of Business Schools

http://www.bizschooljournals.com/SciMago http://

www.scimagojr.com/journalrank.php?area=1400

Journal Quality List (Harzing) http://www.harzing.com/jql.htm

17/04/2015

Maynooth University

Issues & Limitations (1) Sciences (quantitative) Vs Humanities (qualitative) :

Do Bibliometrics suit sciences better? Citation rates in Humanities much lower (Social Sciences fare better)

Citation patterns can differ greatly between disciplines. Need to compare like with like !

Some disciplines don’t publish as much in Journals Computer Science, Electrical Engineering, Arts, Humanities, Social

Sciences

Only published Literature included Policy papers, Professional Journals, Working papers, Monographs,

Reports ,Dissertations

Citation does not have to be positive! Citation tools don’t index all research areas or publications .

Different tools/different datasets = different results

17/04/2015

Maynooth University

Issues & Limitations (2) Manipulation of the system!

◦ Self citing, citing colleagues, splitting outputs into many articles, participation in multi authored publications

Editors can manipulate their journal’s ranking by asking authors to include more citations of other articles from that journal (Coercive citation!)

Some Publishers overly concerned with increased Impact Factor ◦ Rejecting quality articles !

Difficult for early career researchers to show their output, metrics take time◦ Reflect poorly against more experienced researchers.

Measures Impact (number of citations) not the quality of the work !◦ Quantitative not qualitative

Academic freedom? Where you publish, What you publish, How often you publish◦ Value of research vs getting cited !

◦ Shorter research cycles !

◦ Is curiosity driven research too narrow for journals to be interested?

◦ Increase in shorter articles to “Letters” Journals (Biology, Physics)17/04/2015

Maynooth University

Recent Articles“Bibliometrics: The Leiden Manifesto for research

metrics”: Diana Hicks et al,(2015)http://www.nature.com/news/bibliometrics-the-leiden-manifesto-for-research-metrics-1.17351

“The focus on bibliometrics makes papers less useful”

Forcing research to fit the mould of high-impact journals weakens it. Hiring decisions should be based on merit, not impact factor, says Reinhard Werner. (2015)

http://www.nature.com/news/the-focus-on-bibliometrics-makes-papers-less-useful-1.16706

17/04/2015

Maynooth University

Key points Compare like with like

◦ Similar Research Areas◦ Stage of academic career◦ Similar Journals (Discipline)◦ Similar size institutions

Don’t just use one tool ! ◦ Coverage varies in content, depth, discipline

Does your discipline publish in journals and are those journals included in the Datasets being used to generate metrics?

Data needs to be looked at in context ! Use a variety of metrics and other qualitative information where appropriate

17/04/2015

Maynooth University

What Metrics Should I use?

What question am I trying to achieve/answer?

What am I evaluating/showcasing?

How will I recognise good performance?

Which metrics will help me?

17/04/2015

Maynooth University

Practical Session: Creating a Researcher Profile; Researcher Impact; Citation

Tracking; Journal Impact

Ciaran Quinn, Research Support Librarian, Maynooth University.6th May 2015

17/04/2015

Maynooth University

Researcher Identity & Publication Profiles

Why do it? Avoid Author & Institutional Disambiguation

◦ Be consistent in the variation of your name (& Institutional name) that you use

◦ Makes sure your work is correctly attributed to you◦ Groups your name variants and different institutions you may

have published in together Improves Visibility = Being Found & Read=More Citations

◦ Helps Researchers, Students, Funding Agencies, Potential Collaborators find your work

Helps you track and measure the impact of your research Easier for you to create a list of your publications and

sort them by impact Monitor your Researcher Identity and keep it up to date !

17/04/2015

Maynooth University

What Tools are available? Institutional

◦ RIS (Research Information System) Maynooth◦ RPD (Researcher Profile Directory)

http://apps.maynoothuniversity.ie/rpd/Researcher ID (Thomson Reuters)Scopus Author Identifier (Elsevier)Open Researcher & Contributor ID (ORCID)Google Scholar Citations Peer Networks (Academia.edu, ResearchGate.net

, Mendeley.com, Social Science Research Network (SSRN.com), LinkedIn.com

Utilise social bookmarking with Mendeley, Zotero or CiteULike.org

17/04/2015

Maynooth University17/04/2015

Maynooth University17/04/2015

Maynooth University

Researcher ID

17/04/2015

Maynooth University17/04/2015

Maynooth University

Scopus: Author Identifier

17/04/2015

Maynooth University17/04/2015

Maynooth University17/04/2015

Maynooth University

Google Scholar Citations

17/04/2015

Maynooth University17/04/2015

Maynooth University

Researcher ImpactKeeping track of your research

◦Check your H Index◦Set up Citation Alerts: WOS, Scopus,

Google Citations & others ◦Research/Social Networking sites &

Altmetrics to see who is view/downloading or discussing your work

17/04/2015

Maynooth University

Improving your Research ImpactIncrease your visibility

◦Set up a Researcher ID (WOS), Check & update Your Scopus ID

Export them to Orcid

◦Google Scholar Citations◦Social Media/Social Bookmarking/Blogging◦ Institutional Repositories◦Make yourself findable by linking up you

Research title, Article title and Keywords◦Posters/Presentations at Conferences◦Collaborate with other academics◦Submit articles to the highest ranking journals

17/04/2015

Maynooth University

Citation Tracking using Alerts

ScopusWOS: Citation Indexes

◦ (Humanities, Social Sciences, Sciences, Conferences, Books)

Google Scholar

17/04/2015

Maynooth University

WOS Citation Indexes

17/04/2015

Maynooth University

Scopus Alerts

17/04/2015

Maynooth University

Other Citation SourcesERIC (Cited by)High Wire (Cited by High Wire article)HeinOnline (Cited by) JSTOR (Items citing this item)ProQuest (Cited by other articles in the

database)Science Direct (Citing articles from Scopus)Springer Link (Cited by)SSRN (Citations tab)Wiley (Cited by) Ingenta Connect Institute of Physics: IOP (All citing articles)

17/04/2015

Maynooth University

Why are Journal Rankings Important? Journals are ranked by how frequently recently published

papers from a particular journal are cited Prestige of publishing in a particular Journal !

◦ Bur does it measure the academic value/research quality of an article!

However it does have an impact on the reputation of a researcher and therefore its important for careers◦ Used by funding agencies and academic institutions◦ 36% of top cited articles in top 5% of Journals

Gradually declining

Metrics can be gamed◦ Journals may decide to publish on certain topics or certain article

types (such as reviews) to maximise their JIF San Francisco Declaration on Research Assessment 2012

IF creates biases and inaccuracies when appraising scientific research. It also states that the impact factor is not to be used as a substitute "measure of the quality of individual research articles, or in hiring, promotion, or funding decisions”

17/04/2015

Maynooth University

Journal Rankings/MetricsJournal Citation Reports (includes

Journal Impact Factor): Thomson Reuters (2 & 5 years)

Journal Compare/Analyzer: Scopus (Elsevier)(3years)

SciMago Journal & Country Ranking (Elsevier)(3 years)

Google Scholar Metrics (Top Publications, 5 years)

Publish Or Perish17/04/2015

Maynooth University

Journal Citation Reports Web of Science

17/04/2015

Maynooth University

Compare Journals Scopus

17/04/2015

Maynooth University

ScImago

17/04/2015

Maynooth University17/04/2015

Maynooth University

Other Journal RankingsCreated Citations e.g. Legal & BusinessWashington & Lee University School of

Law: Law Library Law Journals: Submissions and Ranking, 2006 - 2013 :Journal Citations

http://lawlib.wlu.edu/LJ/Association of Business Schools

http://www.bizschooljournals.com/SciMago http://

www.scimagojr.com/journalrank.php?area=1400

Journal Quality List (Harzing) http://www.harzing.com/jql.htm

17/04/2015

Maynooth University

Eigenfactor.Org◦http://eigenfactor.org/index.php

Centre for advanced studies in Management and Economics (List of Journal Ranking Tools)◦http://

www.cefage.uevora.pt/en/links/revistas_cientificas_rankings

Journal Metrics.com (SciMago)◦http://www.journalmetrics.com/about

-journal-metrics.php17/04/2015

Maynooth University

Benchmarking individual/institutional ResearchTools

◦Scival (Elsevier) SciVal offers quick, easy access to the

research performance of 4,600 research institutions and 220 countries worldwide. Overviews, Benchmarking, Collaborations

◦Incites (Thomson Reuters) Incites is a tool that allows you to analyze

institutional productivity and benchmark your output against peers worldwide.

17/04/2015

Maynooth University17/04/2015

Maynooth University17/04/2015

Maynooth University17/04/2015

Maynooth University

Time to give it a go !

Practical Session !

17/04/2015

Maynooth University

Author Name Look up your own author name (or someone you

know) on Scopus, Web of Science and Google Scholar◦ Are they there? Does the data adequately represent

them?◦ Are there many differences between the data sources?

Number of documents, H Index

◦ How many name variants are there?◦ Are there many errors?

Set up a profile on Researcher ID (Wos) and Google Citations. Check your profile on Scopus. Export your profile to Orcid.

Are there corrections that will need to be made?◦ Name variants/Correct Documents assigned

17/04/2015

Maynooth University

Citation AlertsSearch for your articles and set

up a citation alert ◦Do this in WOS, Scopus and Google

Scholar

17/04/2015

Maynooth University

Journal ImpactLook at the impact of an

important journal in you field (use WOS & Scopus)◦Is it included in the datasets?◦If not does that surprise you?

Identify the top Journal in your research area

17/04/2015

Maynooth University

The EndQuestions?

Contact: ciaran.quinn@nuim.ie

Research Support Blog: http://ciarnthelibrarian.blogspot.ie/

17/04/2015