Monitoring and assessing research performance€¦ · Dr. Walid HASSAN Research Consultancy...

transcript

November 2017

Monitoring and assessing research performance

November 2017

On ResearchBenchmarking and Valuation (Translation)

Dr. Walid HASSANResearch Consultancy Services Manager

The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man”..George Bernard Shaw, Man and Superman (1903) "Maxims for Revolutionists"

CLARIVATE ANALYTICS: A GLOBAL FOOTPRINT

1. With more than 4,200 staff worldwide, Clarivatesupports customers through curated editorialcontent, leading professional services, customproduct and service development and muchmore.

2. Over 7,000 leading government and academicinstitutions, and research intensive corporationsuse the Web of Science for its 100 years worth ofresearch.

3. Headquartered in the US, we are a global,information-led company operating in over 100countries around the world.

BIG DATA CAPABILITIES

THOMSON INNOVATION

• DWPI – world’s most comprehensive patent data set

• 4.5 Release and Smart Search – find what your are looking for quickly, easily

• ThemeScape – easily visualize how competitor portfolios overlap

OUR PEOPLE: CONNECTING ACROSS THE GLOBE

Content

� Top 2016 Discoveries

� Situation Analysis- Brazil

� Back to basics, Challenges

� On R&D Strategies

� Benchmarking

� Case Studies

Top 2016 DiscoveriesSlight Genetic Mutation 800 Million Years Ago Leads To Multicellular Life

Researchers found that an ancientmolecule, GK-PID, is the reason single-celled organisms started to evolve intomulticellular organisms approximately 800million years ago . The molecule hasbeen found to be like a molecularcarabiner able to pull chromosomestogether to latch them onto the inner wallof a cell membrane when division occurs.This allows cells to copy properly andavoid becoming cancerous.

Photo credit: TenOfAllTrades /Wikimedia

A Ninth Planet Was Discovered In The Solar System

California Institute of Technologypresented evidence that a ninth planettruly does exist with an orbital period of15,000 years.The astronomers who published theirdiscovery have calculated that there is“only a 0.007 percent chance, or aboutone in 15,000, that the clustering could bea coincidence.” Presently, Planet Nineremains hypothetical, but astronomershave calculated its orbit to be quitemassive. If it does exist, the planet wouldlikely be approximately 2–15 times themass of Earth and orbit between 200 and1,600 Astronomical Units (AU) from theSun. An AU is 150,000,000 kilometers,which means that the planet could orbit asfar from the Sun as 240,000,000,000kilometers.

Photo credit: Wikimedia

Top 2016 Discoveries

An Eternal 5D optical data Storage

University of Southampton: Scientists havesuccessfully used nano-structured glass to create aprocess for recording and retrieving data. The storagedevice is a small glass disk about the size of anAmerican quarter that can hold 360TB of data andremain intact up to 1,000°C. This means that itsaverage shelf life when held at room temperaturewould be approximately 13.8 billion years (Roughlythe same amount of time the universe has existed).

Photo credit: southampton.ac.uk

Stem Cells Injected Into Stroke Patients Re-Enable Patient To Walk

A clinical trial held at StanfordUniversity School of Medicineinjected modified human stem cellsdirectly into the brains of severalchronic stroke patients. Theprocedures were all successful withno negative effects described fromthe injection and only mildheadaches as a result of theprocedure, which was performedon mildly anesthetized patients. All18 showed significant healing longafter any healing is expectedfollowing a stroke (a period of sixmonths)Photo credit: Nissim Benvenisty

Dr. Walid Hassan

11/13/2017

Private Company SpaceX Successfully Lands A Rocket Vertically

13Where We Are

14How We Look

15Who?

16How? (% of Documents Cited)

17How? (% of Hot Papers)

19Who is concerned?

Research Process

PhD Students/Res

earchers

Research Managers`

Decision Makers

� Researchers: PhD Students and Researchers� Research Managers: Research unit managers, department managers, etc.. � Decision makers: Research Laboratory Managers, Institution Management, etc..

20Challenges and Repartition

The Start Point

Researchers are challenged at the first moment, when defining their research subject area

Topics Strategy

35 USD billions has been wasted developingthings that are already documented in a patentspecification .(Source: British Patent Office,2000)

23Patents Versus Publications

Publications

Patents

R&D STRATEGY

—TYPES OF RESEARCH

—PATHWAY FROM RESEARCH TO IP, STRATEGY

Fundamental

Publications

Citations, Visibility

Research Community (Academia )

Applied

Patents

Licensing

Industry

Theories, methodologies

Applications

IPKnow HowV

Positioning of needs: University

University: Opportunities and Challenges

PUBLISH, READ AND CITE

Publish or Perish

Publish ………. or Perish

They Say:

Something is Missing !!!!!!!!

Publishing: Problem Positioning

Today: What to Read

Contradictions?

Need a Benchmark

Research

papers?

Need a Model

EUGENE GARFIELD’S “ASSOCIATION OF IDEAS INDEX”

A benchmark..

A Model

Challenges

Why to publish?

Where ?

When ?

With Whom?

Why we Publish

Diffusion of Knowledge

Promotion

Sharing Ideas

Community belonging

Research Valorization

Getting Feedback

Professional Profile

Where to Publish

• Which journal? Indexed?

• How journals are classified? Which is better?

• Should I look at the Impact factor?

• Would my publications be accepted in high-IF journals?

• Open Access?

BENCHMARKING

The main Pillar of Strategic Planning

BENCHMARKINGBENCHMARKING

Strategizing the Step from A to B

40BENCHMARKING IN RESEARCH

41GOALS AND OBJECTIVESIn

ts Human Resources

Funding

Facilities

Administration

Social Capital

ies Research

Education & Training

Partnership

Recruitment

Thought Leadership

ts Publications

Graduates

Collaborations

Innovation

New Resources

s Scientific Impact

Societal Impact

International Reputation

Increased Funding

42KPI’s

uts Human

Resources

Funding

Facilities

Administration

Social Capital

ies Research

Partnership

Recruitment

Thought Leadership

ts Publications

Graduates

Collaborations

Innovation

New Resources

s Scientific Impact

Societal Impact

Increased Funding

Where is our

funding coming

How much

funding do we

receive from

various

sources?

Who are the Who are the leaders in interested

fields?

What connections can we take

advantage of to recruit top

talent?

How much

research do we

disseminate?

publications

channels do we

use most

often?

How do we

compare to

similar

institutions?

How are we

perceived by

the research

community?

43PLANNING

ESTOU FAZENDO ÀS COISAS CERTAS?

• Theory Evaluations

• Mission statement development

• Needs assessments

• Strategic planning

ESTOU FAZENDO ÀS COISAS CERTAS CERTO?

• Process Evaluations

• Regular progress reports

• Monitoring databases and user interface systems

• Resource usage tracking

ESTOU FAZENDO O SUFICIENTE DAS COISAS CERTAS PARA FAZER A DIFERENÇA?

• Outcome Evaluations

• Output tracking

• Performance benchmarking

• Longitudinal attribution

Who are the stakeholders?

Executives

Researchers

Funders

Partners

Public

What domains are being addressed?

Design

Implementation

Impact and outcome

Efficiency

How is the information presented?

Summary

Detailed report

Presentation

Infographic

Campaign

What are the expected returns?

Awareness

Advocacy

Accountability

Action

Assets

CAPITALISE

46Where We Are

47Organizations

Não é possível exibir esta imagem no momento.

Organizations (Documents in Top 1%)

Dr. Eng. Walid HassanResearch Consultancy Services Manager

Email: walid.hassan@clarivate.com

Twitter: @Walid_Hassan_TR

Linked In: Walid Hassan, PhD

Case Study: University X (Optional)

Overview of a Brazilian University’s

research performance.

Evaluation of

University X’s

Research Activity,

2000-2016

Use case: exploring how bibliometric data can indicate institutional performance

across multiple components of a university research activity framework.

uts Human

Resources

Funding

Facilities

Administration

Social Capital

ies Research

Partnership

Recruitment

Thought Leadership

ts Publications

Graduates

Collaborations

Innovation

New Resources

s Scientific Impact

Societal Impact

Increased Funding

Evaluation of

University X’s

Research Activity

uts Human

Resources

Funding

Facilities

Administration

Social Capital

ies Research

Partnership

Recruitment

Thought Leadership

ts Publications

Graduates

Collaborations

Innovation

New Resources

s Scientific Impact

Societal Impact

Increased Funding

Funding

Acknowledged

Funding

(2008-2016)

Acknowledged

Funding

(2008-2016)

CNPq and CAPES excluded

Evaluation of

University X’s

Research Activity

uts Human

Resources

Funding

Facilities

Administration

Social Capital

ies Research

Partnership

Recruitment

Thought Leadership

ts Publications

Graduates

Collaborations

Innovation

New Resources

s Scientific Impact

Societal Impact

Increased Funding

Partnerships

Evaluation of

University X’s

Research Activity

uts Human

Resources

Funding

Facilities

Administration

Social Capital

ies Research

Partnership

Recruitment

Thought Leadership

ts Publications

Graduates

Collaborations

Innovation

New Resources

s Scientific Impact

Societal Impact

Increased Funding

Publication output

by year

Total (2000-2016): 142,077 publications

Publication output

by CAPES area

6 - Health Sciences, 41.0%

3 - Biological Sciences, 27.2%

5 - Exact and Earth Sciences, 25.8%

4 - Engineering, 12.2%

1 - Agricultural Sciences, 9.1%

9 - Multidisciplinary, 9.1%

2 - Applied Social Sciences, 3.7%

7 - Humanities and Social Sciences, 2.4%

8 - Linguistics, Literature and Arts, 0.2%

Note: Does not sum to 100% due to

publications in multiple categories

Evaluation of

University X’s

Research Activity

uts Human

Resources

Funding

Facilities

Administration

Social Capital

ies Research

Partnership

Recruitment

Thought Leadership

ts Publications

Graduates

Collaborations

Innovation

New Resources

s Scientific Impact

Societal Impact

Increased Funding

Collaborations

Brazilian

Collaborations

International

Collaborations

Evaluation of

University X’s

Research Activity

uts Human

Resources

Funding

Facilities

Administration

Social Capital

ies Research

Partnership

Recruitment

Thought Leadership

ts Publications

Graduates

Collaborations

Innovation

New Resources

s Scientific Impact

Societal Impact

Increased Funding

Impact as

Citation counts

Times Cited

Document count

Times Cited

Normalized Citation

Impact

Scientific ImpactUsing normalized citation measures

to indicate the impact of inputs, activities, and outputs

Example: Category Normalized Citation Impact

Where:

e = the expected citation rate or baseline,

c = Times Cited,

p = the number of papers,

f = the field or subject area,

t = year,

d = document type,

n = the number of subjects a paper is assigned to and

i = the entity being evaluated (institution, country, person, etc).

The Category Normalized Citation Impact (CNCI) of a document is calculated by dividing the actual

count of citing items by the expected citation rate for documents with the same document type, year

of publication and subject area.

World Avg, 1.0

University X, 0.88

Acknowledged

Funding

(2008-2016)

CNPq and CAPES excluded

US DOE

US NIH US NSF

Acknowledged

Funding

(2008-2016)

CAPESNCPq

Output and Impact

by CAPES category6 – Health Sciences

5 – Exact and Earth Sciences

7 – Humanities and Social Sciences

Sub-analysis of

CAPES category

“Exact and Earth

Science”

Sub-analysis of

CAPES category

“Humanities

Social Sciences”

Collaborations,

Country-level

Netherlands

Percentage of

papers with

international

collaborations by

Total percentage

2000-2016: 29.72%

Journal analysis

using citation

impact normalized

to the journal

Evaluation of

University X’s

Research Activity

uts Human

Resources

Funding

Facilities

Administration

Social Capital

ies Research

Partnership

Recruitment

Thought Leadership

ts Publications

Graduates

Collaborations

Innovation

New Resources

s Scientific Impact

Societal Impact

Increased Funding

Evaluation of

University X’s

Research Activity

uts Human

Resources

Funding

Facilities

Administration

Social Capital

ies Research

Partnership

Recruitment

Thought Leadership

ts Publications

Graduates

Collaborations

Innovation

New Resources

s Scientific Impact

Societal Impact

Increased Funding

Case Study: The Impact Factor and it’s

(Optional)

The Impact FactorThe impact factor is a measure of the frequency with which the average

article in a journal has been cited in a particular year. The JCR also lists

journals and their impact factors and ranking in the context of their

specific field(s).

Citations and IF variations

Dr. Walid Hassan

11/13/2017

Categories/Quartiles

- J1 IF1

- J2 IF2

- J3 IF3

- J4 IF4

- J5 IF5

- J6 IF6

- J7 IF7

- ------------

25%- Q1

25%- Q2

25%- Q3

25%- Q4

Case Study: Building a PhD Subject

(Optional)

—SELECTING THE RIGHT RESEARCH TOPIC

Example: Image Reconstruction

—TREND (NUMBER OF DOCUMENTS PER YEAR IN WOS)

—NORMALIZED CITATION IMPACT

—PATENT ANALYSIS: IMAGE RECONSTRUCTION

—THEMESCAPE (DERWENT INNOVATION)

Red Dots: Alive Patents

WEB OF SCIENCE (WOS) DATABASES

SCIENCE CITATION INDEX EXPANDED

•41.2 million records

•1900-present

SOCIAL SCIENCES CITATION INDEX

•1900-present

ARTS & HUMANITIES CITATION INDEX

•1975-present

CONFERENCE PROCEEDINGS CITATION INDEX

•1990-present

BOOK CITATION INDEX

•977,199 records

•2005-present

CURRENT CHEMICAL REACTIONS

•212,622 records

•1985-present

INDEX CHEMICUS

•337,305 records

•1993-present

DATA CITATION INDEX

•1900-present

CURRENT CONTENTS CONNECT

•1998-present

BIOSIS CITATION INDEX

•1926-present

ZOOLOGICAL RECORD

•4 million records

•1864-present

MEDLINE

•22 million records

•1950-present

DERWENT INNOVATION INDEX

•22.9 million basic inventions

•1963-present

A comprehensive gateway to the most relevant research information: reliable, integrated, multidisciplinary