Date post: | 22-Jan-2016 |
Category: |
Documents |
Upload: | johana-kelley |
View: | 216 times |
Download: | 0 times |
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Best Practices in Software Development:Users and Developers Unite!
Bennett Landman, Ph.D.
David N. Kennedy, Ph.D.
Xiaodong Tao, Ph.D.
Paul A. Yushkevich, Ph.D.
Gary Hui Zhang, Ph.D.
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Schedule13:30 - 15:00 Session 1: Presentations
13:30 - 13:40 Introduction to the Tutorial13:40 - 14:00 Neuroimaging Informatics Tools and Resources Clearinghouse: NITRC
David N. Kennedy14:00 - 14:20 3DSlicer3 Extension Modules: Concept and Implementation
Xiaodong Tao14:20 - 14:40 Challenges of Collaborative Open-Source Development: Lessons learned from the
early evolution of the Java Image Science ToolkitBennett Landman
14:40 - 15:00 Making others use your software: Lessons learned from ITK-SNAPPaul A. Yushkevich
15:00 - 15:30 Tea/Coffee Break
15:30 - 17:00 Session 2: Panel Discussion & Demonstrations15:30 - 16:15 Panel Discussion16:15 - 17:00 Demos
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Academic Objectives
All participants should gain knowledge regarding:• Methods for open source software development• Methods for software distribution, including licensing
issues• Techniques for software validation• Techniques for user interactions (documentation,
tutorials, feedback/communication, etc.)• Resources for finding existing relevant software
projects.• See http://www.nitrc.org/projects/best_practices
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
FacultyBennett Landman, Ph.D.Assistant ProfessorDepartment of Electrical EngineeringVanderbilt University Institute of Image [email protected]
David N. Kennedy, Ph.D.Professor, Department of PsychiatryUniversity of Massachusetts Medical SchoolWorcester, MA [email protected]
Xiaodong Tao, Ph.D.Computer ScientistGE Global Research CenterImaging [email protected]
Gary Hui Zhang, Ph.D.PICSL, Department of RadiologyUniversity of Pennsylvania School of Medicine3600 Market Street, Suite 320Philadelphia, PA [email protected]
Paul A. Yushkevich, Ph.D.Assistant ProfessorPenn Image Computing and Science LaboratoryDepartment of RadiologyUniversity of [email protected]
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Neuroimaging Informatics Tools and Resources Clearinghouse - NITRC
Software Tools - A Users Perspective
David N. Kennedy, Ph.D.
Director, Division of Neuroinformatics
Department of Psychiatry, UMass Medical School
Child and Adolescent NeuroDevelopment Initiative
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
With apologies and credit to Gary Larson - The Far Side
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
users developers
How do I change the contrast? How do I add layers? How do I invert the transform?
blah blah blah blah blah blah RTFM blah blah
With apologies and credit to Gary Larson - The Far Side
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
usersdevelopers
v
Well, this is really a very complex piece of code. Have you read my paper from MICCAI? It’s probably too complex for you…
With apologies and credit to Gary Larson - The Far Side
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
• Data Acquisition & Analysis - Normal Development and Studies in Child Psychiatry (Autism, Bipolar Disorder,
etc.)
– Structure
– Diffusion
– Resting State
User Scope:
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Finding the Right Tool
“MRI segmentation software brain”
• Google?– About 157,000 results– Top Hit: Rescue
• Source Forge?
• NITRC?
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Google Top Hit
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Finding the Right Tool“MRI segmentation software brain”• Google?
– About 157,000 results– Top Hit: Rescue– Results include: papers, pdfs, ppts, programs,
pictures, videos, etc.
• Source Forge?• NITRC?
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Finding the Right Tool“MRI segmentation software brain”• Google?
– About 157,000 results
• Source Forge?– 15,539 results– Top Hit: PortableApps.com: Portable Software/USB – Results include: you name it…
• NITRC?
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Source Forge Results
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Finding the Right Tool“MRI segmentation software brain”• Google?
– About 157,000 results
• Source Forge?– 15,539 results
• NITRC?– 39 Software Packages– Top Hit: ITK-SNAP– Results include: All the ‘usual suspect’ tools, and
more!
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
NITRC Result
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Refining the Search
• Operating System: MacOS
• Supported Image Type: DICOM
• Results in: 16 Resources
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
NITRC ResultRefined
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Best Practices Take-Home Points:
1. Findability Description/Promotion
2.
3.
4.
5.
6.
7.
8.
User Requirements Developer Requirements
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Installation
• Findability is just first step. Can I install it, easily?
• Examples:– FSL– Mango
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
FSL Installation Steps1. Click download link
2. Agree to license
3. Provide some personal information
4. Download appropriate release
5. Run FSL installer script (from command line)
6. Test the installation
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Mango Installation Steps
1. Click download link
2. Open the download w/ ‘Archive Utility’
3. Run Installer (Double click from UI) - Answer usual questions
4. Run Program (Double click from ‘Applications’ directory)
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Best Practices Take-Home Points:
1. Findability Description/Promotion
2. Easy to Install Easy to Install/Verify
3.
4.
5.
6.
7.
8.
User Requirements Developer Requirements
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Ease of Use• Now what? You’ve found and installed
your program, how do you use it?
• From the UI, is it clear what to do?
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
User Interface
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Documentation• Of course the software is very powerful, and
it’s impossible for everything to be intuitively obvious from the UI, who do you think we are, Microsoft?
• Users are obligated to RTM (read the manual) prior to bugging the developers with questions.
• But, where is the manual? Is it easy to use? Are there examples? Tutorials? FAQ?
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
FSL Documentation
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Best Practices Take-Home Points:
1. Findability Description/Promotion
2. Easy to Install Easy to Install/Verify
3. Easy to Use Intuitive UI/Good Documentation
4.
5.
6.
7.
8.
User Requirements Developer Requirements
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Help!
• OK, I REALLY did read the manual, and I just can’t figure out how to [insert problem here]…
• I checked the FAQ, and searched on the mail list archive…
• Oh, wait a minute, is there a FAQ? Is there a mail list? Is there a users ‘forum’? Where are they?
• OK, I checked those too, how do I get more help?– User Support Forums, Mailing Lists– Email an administrator?– What’s the preferred method for this project?
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Best Practices Take-Home Points:
1. Findability Description/Promotion
2. Easy to Install Easy to Install/Verify
3. Easy to Use Intuitive/Good Documentation
4. Get Help Bug Tracker/Forums/Mailing List
5.
6.
7.
8.
User Requirements Developer Requirements
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Latest Version?• Once you succeed at contacting the
developers, how often does the following happen to you?– Oh, you can’t do [blah]? ‘That’s been added in
the ‘most recent’ version’ Or worse yet, ‘that’s only available in the development version’. Or, ‘that’s planned for release 3.X’.
• Users: Know your versions!• Developers: Keep your public releases up
to date! (yes, ALL of them…)
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Best Practices Take-Home Points:
1. Findability Description/Promotion
2. Easy to Install Easy to Install/Verify
3. Easy to Use Intuitive/Good Documentation
4. Get Help Bug Tracker/Forums/Mailing List
5. Up-to-Date Versioning/Synchronization/Dashboard
6.
7.
8.
User Requirements Developer Requirements
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
I want the best!• Of course, I only want to use the best software know to
human kind for my research.• Unfortunately, ‘best’ is exceedingly domain dependent, and
many problems (and software packages) cross many domains in a specific problem (skull striping, registration, pre-filtering, statistical test, etc.).
• Also, ‘best’ is data dependent. The ‘best’ surface extraction in post mortem tissue may not be the best in patients w/ AD.
• Users: Know what best means in your context!• Developers: Provide quality metrics (on standard cases)
when possible!
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Best Practices Take-Home Points:
1. Findability Description/Promotion
2. Easy to Install Easy to Install/Verify
3. Easy to Use Intuitive/Good Documentation
4. Get Help Bug Tracker/Forums/Mailing List
5. Up-to-Date Versioning/Synchronization/Dashboard
6. High Quality Validation/Comparison Metrics
7.
8.
User Requirements Developer Requirements
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Say I’m not the only one…• Even if using the ‘best’ tool, the user will run into
publications problems if no one else has ever used that tool before.
• Have you gotten reviewers who say: “You should have used the Talairach coordinate system”, or “You should have used SPM…”?
• Even if it’s not fair or optimal, users need to know how popular a resource is. # of downloads # of users # of publications, etc.
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Best Practices Take-Home Points:
1. Findability Description/Promotion
2. Easy to Install Easy to Install/Verify
3. Easy to Use Intuitive/Good Documentation
4. Get Help Bug Tracker/Forums/Mailing List
5. Up-to-Date Versioning/Synchronization/Dashboard
6. High Quality Validation/Comparison Metrics
7. Use Metrics Downloads/Users/Publication Counts
8.
User Requirements Developer Requirements
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
I can make it better…• I’ve found a great tool, it works really well for me,
and does everything I need… almost.• I submitted a ‘Feature Request’, and I completely
understand that your funding and time is limited, and you will not get to my request until 2021.
• But hey, I’m a wicked good [insert programming language here] programmer myself. Let me at your source code…
• Open Source is not the solution for everything, but with proper management and policy, it’s an important tool for extensibility, transparency, and public relations. Besides, it ought to be the law…
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Best Practices Take-Home Points:
1. Findability Description/Promotion
2. Easy to Install Easy to Install/Verify
3. Easy to Use Intuitive/Good Documentation
4. Get Help Bug Tracker/Forums/Mailing List
5. Up-to-Date Versioning/Synchronization/Dashboard
6. High Quality Validation/Comparison Metrics
7. Use Metrics Downloads/Users/Publication Counts
8. Contribute/Extend Source Code Availability/Policy
User Requirements Developer Requirements
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Summary• Top 10 reasons users use a specific
software package:
10. It’s what my lab uses…
9. Has the most publications…
8. Is easiest to use…
7. Is part of the LONI pipeline…
6. I wrote it… when I was a graduate student… 20 years ago…
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Summary• Top 10 reasons users use a specific
software package:
5. My graduate student wrote it… who is a botany major… with no formal programming training…
4. It works with 3D Slicer…
3. It’s written in ITK…
2. Everyone uses SPM…
1. At least it’s not SPM…
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Summary, continued• Top 1 reason NOT listed as a reason to use
a specific software package:
1. It is the best…
This is where developers and users have to continue to work together to:
• Insist on the best
• Define the parameters of best
• Develop standards for performance
• Document the performance
• And use the guidelines of Best Practices for software development!
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Acknowledgements
• All NITRC Users and Developers• Today’s Presenters & Audience• MICCAI Organizers• NITRC Development Team
(Neuromorphometrics Inc, TCG Inc, UCSD)• NITRC Sponsors (NIH Blueprint, NIBIB)• The ‘tutorial’ lives on at
http://www.nitrc.org/projects/best_practices
www.nitrc.orgnitrc.nih.gov
F-T4: Best Practices in Software DevelopmentMICCAI 2010 - September 24, 2010
Best Practices Take-Home Points:
1. Findability Description/Promotion
2. Easy to Install Easy to Install/Verify
3. Easy to Use Intuitive/Good Documentation
4. Get Help Bug Tracker/Forums/Mailing List
5. Up-to-Date Versioning/Synchronization/Dashboard
6. High Quality Validation/Comparison Metrics
7. Use Metrics Downloads/Users/Publication Counts
8. Contribute/Extend Source Code Availability/Policy
User Requirements Developer Requirements