Date post: | 15-Apr-2017 |
Category: |
Science |
Upload: | anita-bandrowski |
View: | 48 times |
Download: | 1 times |
Breaking the phenotyping barrier
and the dire need for identification
Anita Bandrowski, SciCrunch / NIF / RRID
Phenotyping data comes from authors Curators read
the articlesAlign free text to community
ontologiesDeposit
aligned data to databases
Step 1 is identification
But, quality of published research is in question
Solving quality: mandates and solutions
NIH NOT-OD-16-011Journals RRIDs
Societies FASEB GuidelinesNon-Profits TOP Guidelines
*Reproducibility of science is key*
What is reproducible science?
Step 1: Material
s
Step 2: Method
s
Step 1: In
Practice
How common is this?
Papers are currently poor at identifying the
simplest part of the paper, the materials used
Vasilevsky 2013
But the author knows what was used! This author got back to me within 2 hours with the stock number of this mouse
Left open annotation with this stock number from JAX, but will others find this?
How can we get better data?• 2009: LAMHDI meeting – project hatched • 2011: Meeting with Society for Neuroscience, Journal of Neuroscience full editorial board presenting the
problem and results of text mining study• 2012: Society for Neuroscience – defined the problem for Editors of top Neuroscience journals; sponsored
by INCF• 2013: NIH Meeting - brought the editors back to define the solution; 2 day workshop sponsored by NIDA
and INCF, several IC directors in attendance• 2013: Society for Neuroscience – mainly publishers, defined the timeline of starting the project• 2014: Neuroscience Information Framework – built scicrunch.org/resources based on NIF technologies and
members of the OHSU team populated web pages / instructions etc.• 2014: Project starts with Journal of Neuroscience, Neuroinformatics, F1000, Brain and Behavior and
Journal of Comparative Neurology taking a strong lead• 2015: Paper describing how RRIDs are used by authors of the first 100 papers is co-published in 4 journals• 2016: integration with Hypothes.is tool gives curators an easy way to verify RRIDs, sci-score gives
authors an easier way to detect what is a resource
Journals key solution
Journals will not change
Journals in aggregate
Funders role
Project Management Key
ID based tracking
Technology innovation
Inclusion of stock center data Repository Name StatusAmbystoma Genetic Stock Center IncludedA Resource Center for Tetrahymena thermophila IncludedDevelopment of Validated Drosophila in vivo RNAi Models of Human Diseases IncludedWormBase IncludedJAX IncludedRGD IncludedMGI IncludedZebrafish International Resource Center IncludedBloomington Drosophila Stock Center at Indiana University IncludedMutant Mouse Resource and Research Center Included
MMRRC at University of California, Davis IncludedZFIN IncludedCaenorhabditis Genetics Center IncludedThe Mouse Mutant Resource (MMR) IncludedCre Driver Strain Resources IncludedDrosophila Genomics Resource Center IncludedMutant Mouse Regional Resource Center at University of North Carolina IncludedMutant Mouse Resource and Research Center at the University of Missouri IncludedNational Swine Resource and Research Center IncludedNational Xenopus Resource Center IncludedRat Resource and Research Center IncludedSperm Stem Cell Libraries for Biological Research. IncludedThe Special Mouse Strain Resource (SMSR) at The Jackson Laboratory IncludedXiphophorus Genetic Stock Center IncludedAnimal Model Resources for Cystic Fibrosis No repsonseNational Gnotobiotic Rodent Resource Center No repsonseAdult Mesenchymal Stem Cell Resource Stocks not availableGene Library Resource for the Sea Urchin S. purpuratus Stocks not availablePrimate Embryo Gene Expression Resource Stocks not availableResearch Resources for Model Amphibians Stocks not availableViper Resource Center (VRC) at Texas A&M University-Kingsville (TAMUK) Stocks not availableWormGuides Stocks not available
Data in SciCrunch is used by authors to identify organisms, authors publish papers and curators and scientists then find clear simple citations to resources
Copy/PastePublish
Resources are identifiable!
Bandrowski et al, 2015
Which repositories are being cited?• This data set is from 980
papers, 499 organisms noted • Curators filled in resources that
are missing RRIDs in a small number of cases (eg., registered reports for cancer reproducibility studies in eLife), most are RRIDs asserted by authors
• Pie chart shows total number of organism annotations by species
• Most citations are to repositories that have been included for the longest time (eg JAX, RGD, BDSC)
• NXR – frog is a recently joined repository, 1st paper is out
Data can be found herehttps://docs.google.com/spreadsheets/d/1VOUml95YoxGQnG0hjTwOKHf6vbVb6hmNb53L2bixLGM/edit?usp=sharing
JAX MGI BDSC RGD ZFIN MMRRC CGC ZIRC RBRC EM KOMP NCIMR NXR0
50
100
150
200
250
223
104
55 48
18 179 9 6 3 2 1 1
Fly; 55; 11% Fish; 27; 5%
Worm; 9; 2%
Rat; 48; 10%Mouse
; 359; 72%
Frog; 1; 0%
Most popularRRID:IMSR_JAX:000664 27RRID:RGD_70508 9RRID:ZIRC_ZL1 7RRID:RGD_737903 5RRID:RGD_734476 5RRID:IMSR_JAX:012569 5RRID:RGD_737929 3RRID:RGD_737891 3RRID:IMSR_JAX:007677 4RRID:IMSR_JAX:006410 4RRID:IMSR_JAX:005628 4RRID:IMSR_JAX:000671 4RRID:ZFIN_ZDB-GENO-030619-2 3RRID:IMSR_MMRRC:000230 3
Recent Journals/PublisherJuly 2016
Aug 2016 Sept/Oct 2016
11 papers
Why is this working?How: Make it easy for authors to
include a unique identifier for each research resource that is used
At publicationInstructions to authors
*Instructions to reviewers*Direct contact from editors
We get 97% accuracy from authors (~1000 papers verified, ~9000
RRIDs)
Bandrowski et al 2015F1000 doi: 10.12688/f1000research.6555.2, Journal of Comparative Neurology (doi: 10.1002/cne.23913),
Brain and Behavior (doi: 10.1002/brb3.417) and NeuroInformatics (doi: 10.1007/s12021-015-9284-3).
Make compliance easy and traceable for journals
Attention from NIH, societiesJournals listen
201425 journals signed on5 executed effectively
2016139 journals represented
10 execute effectively*many more coming*
Next Steps ….we are at 1%, how to get to 100?
Fly Fish Worm Rat Mouse Frog0
0.2
0.4
0.6
0.8
1
1.2
1.41.31233595800525
0.913705583756345
0.627615062761506
0.122019421424577
1.22042425890672
0.909090909090909
RRIDs as a Percent of total papers 2015/2016 (Organism search term) Outreach: We need more journals / authors; we
need support from the stock centers
Peer pressure: We need to stress the new NIH guidelines, thank you NIH!
Tools: We need to have tools that make the process easier for journals
SciBot – finds RRIDs in papers (curation tool)Hypothes.is – display annotations on papers – W3C compliant annotationsSci-Score.com – finds sentences that should have RRIDs
*estimate of the number of papers per organism in 2015/16 was based on key words in PubMed representing the main model organism (mouse, rat etc)
To put the proper citation format onto your repository!To help increase awareness of RRIDs (blogs, webinar, newsletters, twitter #RRID)
To add / ask for RRIDs (authors / reviewers)To bring your journal (editors)
Comments / Complaints: [email protected]
6 papers since late April 2016