Measuring ImpactEd Baker & The Scratchpads Team

ViBRANT and eMonocot

ViBRANT and eMonocot

• Projects that focus on data collection, curation and reuse

ViBRANT and eMonocot

• Projects that focus on data collection, curation and reuse

• Would like metrics to know how well we are doing

ViBRANT and eMonocot

ViBRANT and eMonocot

Projects based around communities

ViBRANT and eMonocot

Projects based around communities

Provide tools for communities

ViBRANT and eMonocot

Projects based around communities

Provide tools for communities• Quantify user contribution to multi-user

projects

ViBRANT and eMonocot

Projects based around communities

Provide tools for communities• Quantify user contribution to multi-user

projects• Quantify how content generated by the

community is used/reused

Citizen Science

Citizen Science

• COMBER

Citizen Science

• COMBER

• anymals+plants

Citizen Science

• COMBER

• anymals+plants

• Citizen Science profile

Citizen Science

• COMBER

• anymals+plants

• Citizen Science profile

Metrics allow for more competitive activities and for users to quantify their involvement

Who needs impact metrics?

Who needs impact metrics?

• The authors of data

Who needs impact metrics?

• The authors of data

• People who employ data authors

Who needs impact metrics?

• The authors of data

• People who employ data authors

• People who reuse the data

Who needs impact metrics?

• The authors of data

• People who employ data authors

• People who reuse the data

Am I useful?

Who needs impact metrics?

• The authors of data

• People who employ data authors

• People who reuse the data

Am I useful?

Do we have our priorities right?

Who needs impact metrics?

• The authors of data

• People who employ data authors

• People who reuse the data

Am I useful?

Do we have our priorities right?

Who needs impact metrics?

• The authors of data

• People who employ data authors

• People who reuse the data

Am I useful?

Do we have our priorities right?

Hidden gem?

Who needs impact metrics?

• The authors of data

• People who employ data authors

• People who reuse the data

Am I useful?

Do we have our priorities right?

Hidden gem?

Avoid like the plague?

Our premise

We want people to share data…

…people want to please the “powers that be” …

… the powers that be like statistics.

So we made some

But the people weren’t happy.

But the people weren’t happy.

Of all the things I do you only care about papers?

But the people weren’t happy.

Of all the things I do you only care about papers?

The only way you value my papers is by citation in other

papers?

But the people weren’t happy.

Of all the things I do you only care about papers?

The only way you value my papers is by citation in other

papers?

The impact of my career is measured by one number?

So some people made their own

But there are still problems

But there are still problems

This STILL only deals with my publications!

But there are still problems

This STILL only deals with my publications!

How do I get credit for data?

But there are still problems

This STILL only deals with my publications!

How do I get credit for data?

I wrote some useful biodiversity software – what

about that?

Are alt metrics useful?

Are alt metrics useful?

Are alt metrics useful?

These are fairly scholarly

Are alt metrics useful?

These are fairly scholarly

These measure broader public impact

Are alt metrics useful?

These are fairly scholarly

These measure broader public impact

• A good start

Are alt metrics useful?

These are fairly scholarly

These measure broader public impact

• A good start

• Much broader than just scholarly citations

Are alt metrics useful?

These are fairly scholarly

These measure broader public impact

• A good start

• Much broader than just scholarly citations

• Actually ignores traditional citations

Are alt metrics useful?

These are fairly scholarly

These measure broader public impact

• A good start

• Much broader than just scholarly citations

• Actually ignores traditional citations

• Still only applied to papers

What are we doing?

What are we doing?

What are we doing?

What are we doing?

Biodiversity Data Journal

Data papers (descriptions of data)

Biodiversity Data Journal

Data papers (descriptions of data)

• Allows data to become citable

Biodiversity Data Journal

Data papers (descriptions of data)

• Allows data to become citable

• Allows data to participate in standard credit/metric systems

Biodiversity Data Journal

Data papers (descriptions of data)

• Allows data to become citable

• Allows data to participate in standard credit/metric systems

• … allows data to participate in alt metrics

Scratchpads Statistics Module

What content exists on a Scratchpad site?

Scratchpads Statistics Module

What content exists on a Scratchpad site?

• Filter by user or taxonomic term

Scratchpads Statistics Module

What content exists on a Scratchpad site?

• Filter by user or taxonomic term

• How much content is there?

Scratchpads Statistics Module

What content exists on a Scratchpad site?

• Filter by user or taxonomic term

• How much content is there?

• What kind of content is it?

Scratchpads Statistics Module

What content exists on a Scratchpad site?

• Filter by user or taxonomic term

• How much content is there?

• What kind of content is it?

• How many registered users?

Scratchpads Statistics Module

What content exists on a Scratchpad site?

• Filter by user or taxonomic term

• How much content is there?

• What kind of content is it?

• How many registered users?

• How often is the content viewed?

Scratchpads Statistics ModuleAvailable now:

http://antkey.org/stats

[scratchpad.url]/stats

Scratchpads Metrics Module

Puts the Scratchpad content in a broader context

Scratchpads Metrics Module

Puts the Scratchpad content in a broader context

• Users and content

Scratchpads Metrics Module

Puts the Scratchpad content in a broader context

• Users and content

• Opt-in (not enabled by default)

Scratchpads Metrics Module

Puts the Scratchpad content in a broader context

• Users and content

• Opt-in (not enabled by default)

• Modular (pick and choose what to include)

Scratchpads Metrics Module

Puts the Scratchpad content in a broader context

• Users and content

• Opt-in (not enabled by default)

• Modular (pick and choose what to include)

• Available by end of the year

Scratchpads User Metrics

A partial implementation of the Scholar Factor proposed by Bourne and Fink

Scratchpads User Metrics

A partial (modified) implementation of the Scholar Factor proposed by Bourne and Fink

Citations

Scratchpads User Metrics

A partial (modified) implementation of the Scholar Factor proposed by Bourne and Fink

Citations

Scholar Factor

Scratchpads User Metrics

A partial (modified) implementation of the Scholar Factor proposed by Bourne and Fink

Citations

Scholar Factor

Software / Data

Scratchpads User Metrics

A partial (modified) implementation of the Scholar Factor proposed by Bourne and Fink

Citations

Scholar Factor

Software / Data Grant / manuscript reviews

Scratchpads User Metrics

A partial (modified) implementation of the Scholar Factor proposed by Bourne and Fink

Citations

Scholar Factor

Software / Data Grant / manuscript reviews

Scholar Factor

Scratchpads User Metrics

Scholar Factor

Scratchpads User Metrics

Scratchpads User Metrics Module

Scholar Factor

Scratchpads User Metrics

Scratchpads User Metrics Module

Google Scholar (h-index)

Scholar Factor

Scratchpads User Metrics

Scratchpads User Metrics Module

Google Scholar (h-index) Scratchpad

Scholar Factor

Scratchpads User Metrics

Scratchpads User Metrics Module

Google Scholar (h-index) Scratchpad GitHub

Scholar Factor

Scratchpads User Metrics

Scratchpads User Metrics Module

Google Scholar (h-index) Scratchpad GitHub ?

Scholar Factor

Scratchpads User Metrics

Scratchpads User Metrics Module

Google Scholar (h-index) Scratchpad GitHub ?

Modular

Scholar Factor

Scratchpads User Metrics

Scratchpads User Metrics Module

Google Scholar (h-index) Scratchpad GitHub ?

Modular

Scholar Factor

Scratchpads User Metrics

Scratchpads User Metrics Module

Google Scholar (h-index) Scratchpad GitHub ?

Modular

Same modular structure for content

Same modular structure for content

• Number of web links to page

Same modular structure for content

• Number of web links to page

• Scholarly links (Google Scholar)

Same modular structure for content

• Number of web links to page

• Scholarly links (Google Scholar)

• Social links

Summary

We’re not there yet.

Summary

We’re not there yet, but we have …

Summary

We’re not there yet, but we have…

• Modular framework to add services as they become available

Summary

We’re not there yet, but we have…

• Modular framework to add services as they become available

• multi-dimensional – not just papers and citations

Summary

We’re not there yet, but we have…

• Modular framework to add services as they become available

• multi-dimensional – not just papers and citations

• Not a one size fits all approach

Summary

We’re not there yet, but we have…

• Modular framework to add services as they become available

• multi-dimensional – not just papers and citations

• Not a one size fits all approach