7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 1/17
Massachusetts Green
High Performance Computing Center
Educause 2013
Research Computing CG
October 16, 2013
www.mghpcc.org
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 2/17
The MGHPCC: A Data Center and a Consortium2
A partnership between 5 universities….
The Commonwealth, and industrial
sponsors
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 3/17
Origins (circa 2008)
Recognition that…
Computing is fundamental to the research mission
Incremental on-campus expansion will not end well
Interest in working together Financial pressure
Opportunities that single institutions cannot address
State government interest in supporting research competitiveness
Getting out ahead of disruptive change in how computing gets done
Plan Outline to Ribbon Cutting in ~3 years
Project is on-time and under-budget
3
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 4/17
MGHPCC: Three Works in Progress
Regional Data Center for Research Computing
University/Industry/Government Research Collaboration
Regional economic development
4
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 5/17
Physical Infrastructure
8 Acres for expansion
~1 acre of computer room = ~20,000 servers = ~400PB data
15MW for compute and cooling
Green, Low Cost Power
5
LEED Platinumcertified
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 6/17
Network Access
Commercial Providers
Northern CrossroadsInternet 2
NEREN /NYSERNET
Research and Education Networks
NLR
Others
Others
PublicInternet
PrivateLinks
MIT Fiber Ring(10Gbps dedicated links)
25+Research
Institutions
Dark Fiber
MA Broadband Institute
GENI
6
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 7/17
Computer Room Total Capacity
Main Distribution AreaUniversity Fiber FeedsUniversity backbone switchesMeet me switch
Compute Pod24 Racks14kW per rack average~25KW per rack max
N x 10 GigE to the MDA
Space33,500 sq ft680 racks
Power and Cooling
10 MW for compute5 MW for cooling
Target PUE: 1.3
Network 10 Gb/s feeds toevery campus
UPS limitedto 20% of
the Podsin aphysically consolidated area.
7
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 8/17
How the MGHPCC is Organized
Structure
Non-profit service corporation owned by five universities
Owns the facility and employs facility operations staff
Oversight
Research mission and direction – Research Vice Presidents
Operational direction and goals – CIOs
Day to day issues and opportunities Research – Faculty working group
Operational – Business and IT working groups
8
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 9/17
Consortium Operational Collaboration
Dedicated MGHPCC Staff focuses on facility operation
User support comes from existing research computing groups
Evolving Collaboration
Staying out of each others’ way
Learning from each other
Unified networking plan
Coordinated vendor management Acquisition and operation of jointly owned resources
Larger scale solutions to common problems
9
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 10/17
10
Collaboration: R esearch, Education, Outreach
• Seed Grant Program (promising cross-university projects)• Jointly owned computing infrastructure• Cross-university research initiatives
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 11/17
For UMass, at the ground level….
Lower operating cost
Flexible access to larger resource pool
Greater funding leverage
Starting with a great diversity of readiness and capabilities
11
More of this...Less of this...
11
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 12/17
Computational Research5 UMass campuses, 5 examples
Analyzing urgent DNA samplesto help find treatmentstrategies. Big data. (UMass Amherst, MIT,
Harvard Medical School)
12
Predicting the propertiesof new materials(UMass Lowell)
Understanding thedynamics of the ocean floor(UMass Dartmouth, WHOI,MIT)
Algorithms analyzing thefunctional genome, geneexpression in response toenvironmental stimulus.
(UMass Med.)
Analyzing forestcover using satelliteand LIDAR imagery (UMass Boston)
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 13/17
UMass Governance
13
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 14/17
UMass Service Model
Shared Operations Staff leveraging campus “centers of excellence”
Goal of developing distributed application support
14
Shared ServicesFair Share Pool
5 CampusPurchase
Fair Share
PI Priority 1
PI Priority 2
Condo Pool
Fair Share
Shared Scheduler, Network and IdM
Co-Lo
Pool
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 15/17
Summary
HPC: long term strategic importance to research enterprise
computation: “third leg of science and engineering” along with theory and experimentation but there is also a growing need in social sciences,humanities, and others.
“data”: growing importance
15
MGHPCC:
many “modes of use”
many opportunities service-oriented
amortized cost,
inexpensive,
green energy
contributing
my clusters leverages
access to more
HPC as
an easy to use
service
research
collaboration
strategic
investment
place to put
my clusters!
a
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 16/17
Closing thoughts
What it means for the Commonwealth:
What it Means for UMass Research Infrastructure: timely, strategically important investment in the 21st century
research enterprise Vehicle for Collaboration: within the UMass system, with other universities, industry and
the state/region
A More Efficient and Effective IT Model: for developing high quality researchcomputing in an efficient, cost-effective manner
16
“This landmark project is a testament to what is possible whengovernment, academia and business work together, and it willserve as an economic development model for the state and the
nation for generations to come”remarks by Governor Deval Patrick
7/27/2019 Research Computing Constituent Group (176610591)
http://slidepdf.com/reader/full/research-computing-constituent-group-176610591 17/17
Discussion…
"If we knew what we
were doing, it wouldn't be called research would it ?“
Albert Einstein
17