Enabling Science Through Campus Bridging

A case study with mlRhoScott MichaelJuly 24, 2013

Outline

• What is Campus Bridging?• Why is Campus Bridging important?• The gap between Campus Champion and

Extended Collaborative Support• What does Campus Bridging look like in

practice?– Use case example with mlRho

What is Campus Bridging?

• NSF Task Force on Campus Bridging Report– Campus bridging is the seamlessly integrated use of cyberinfrastructure

operated by a scientist or engineer with other cyberinfrastructure on the scientist’s campus, at other campuses, and at the regional, national, and international levels as if they were proximate to the scientist, and when working within the context of a Virtual Organization (VO) make the ‘virtual’ aspect of the organization irrelevant (or helpful) to the work of the VO.

• Campus Bridging makes a supercomputer as easy to use as your desktop!

What is Campus Bridging?

• Stewart et. al. 2012 defined a series of goals based on use cases– Authentication– Training and information dissemination– Interactive computing– Data transfer– Distributed workflows– Resource sharing– User support

What is Campus Bridging?

• Campus Bridging efforts are yielding results from a number of tools– Globus Online– Global Federated File System (GFFS)– Cluster installation distributions

• What is missing from many descriptions of Campus Bridging is explicit support for users to transition their applications to increasingly complex systems

Why is Campus Bridging Important?• Researchers in fields relatively new to HPC do

not always have the necessary expertise scale their workflow to massive systems

Why is Campus Bridging Important?• What is driving new areas of research and fields

of science to use HPC resources?– Data, not tightly coupled computation

• Researchers with massive amountsof data are prime candidates forcampus bridging

• Some example fields– Biology– Humanities– Social media

Campus Champions and ECSS

• Campus Champions are primarily providers of information and points of communication– Source of local, regional and national high-performance computing and

cyberinfrastructure information on your campus– Source of information regarding XSEDE resources and services that will benefit

research and education on your campus– Source of start-up accounts on your campus to quickly get researchers and

educators using their allocations of time on XSEDE resources– Conduit for the campus high-performance computing needs, requirements and

challenges, with direct access to XSEDE staff

• Champions may not have the necessary domain or technical knowledge to effectively bridge between campus and XSEDE resources

Campus Champions and ECSS

• XSEDE Extended Collaborative Support allocations provide – “expert staff members for an extended period to work

together to solve challenging science and engineering problems through the application of cyberinfrastructure”

• Require an allocation request• Are targeted towards a well defined and novel

problem

The Gap Between Campus Champions and Extended Collaborative Support

ECSS• What’s missing?

– Assistance in preparing allocation request– Scaling up to large numbers of processors– Managing massive data– Improving performance at scale

The mlRho Use Case

• Work at IU with the Michael Lynch lab nicely demonstrates potential of investing in Campus Bridging

• mlRho is a serial application developed jointly between IU researchers and researchers at the Max Planck Institute for Evolutionary Biology

• It uses a maximum likelihood approach to estimate linkage disequilibrium rates for a variety of species

mlRho from Desktop to XSEDE

• Lynch lab researchers wererunning 100s of serial mlRho jobs Quarry

• They needed larger resource to speed up theirresearch

• We helped them preparean XSEDE allocation request

Organism Size of Profile (GB) Distance/sec

F. cylindrus (diatom) 0.72 0.323

P. ornithorhynchus (platypus)

11 0.020

C. familiaris (dog) 31 0.005

Preparing a Successful Allocation Proposal

• We bundled serial jobs using the BigJob framework• We assisted the researchers in estimating the SUs

the would require• The most appropriate

XSEDE machine to target

• Assisted in performingscalability studies

• Helped with file systemissues

Application Optimization

• Once the application was running at scale we assisted with performance optimization

• We traced theapplication using Vampirand gave theresults to thedeveloper

Application Optimization

• The developer improved the runtime by morethan 50x

• We probably didless work than what would be considered anECSS collaboration

Future Explorations• We have begun exploring running multiple serial

instances of mlRho on the Xeon Phi acclerators• BigJob is not available for the Phi so some low

level scripting wasrequired

• Not overly difficultbut too much formost biologists

• Lynch lab is working to developa mlRho gateway

Conclusions and Questions

• Campus Bridging is taking researchers from desktop computing, through campus computing, to computing with XSEDE resources

• Researchers new to HPC need help at all stages of an XSEDE allocation

• Campus Bridging tools will help with many pain points in this transition, with a modest investment of human resources there is huge potential ROI

• Campus Bridging activities can be a stepping stone for gateways

Conclusions and Questions

• What is the best way to provide these resources?

• How should Campus Bridging Specialists fit into the ecosystem of Campus Champions and ECSS?

• Comments? Questions?• scamicha@iu.edu

Acknowledgements and Citations

• Sen Xu• Abhinav Thota• Robert Henschel• Thomas Doak

• BigJob development team• TACC support staff• Rich Knepper• Craig Stewart

• Craig A. Stewart, Richard Knepper, James Ferguson, Felix Bachmann, Ian Foster, Andrew Grimshaw, Victor Hazlewood, and David Lifka. 2012. What is campus bridging and what is XSEDE doing about it?. In Proceedings of the 1st Conference of the Extreme Science and Engineering Discovery Environment: Bridging from the eXtreme to the campus and beyond(XSEDE '12). ACM, New York, NY, USA, , Article 47 , 8 pages.

• NSF Advisory Committee for Cyberinfrastructure Task Force on Campus Bridging. Final Report. March 2011. Available from: http://www.nsf.gov/od/oci/taskforces/TaskForceReport_CampusBridging.pdf