NICS UpdateBruce Loftis

16 December 2009

National Institute for Computational Sciences

University of Tennessee and ORNL partnership· NICS is the 2nd NSF Track 2 center

– Builds on strengths of UT and ORNL

· NICS operates the first academic petascale supercomputer in the world

· Staff of 25

2 Managed by UT-Battellefor the Department of Energy

NICS Timeline

3

Track-2B Award announcedfor University of Tennessee

September 2007

Cray XT3 Production 7K cores, 40 TF June 2008

Cray XT4 Production 18K cores,166 TF August 2008

Cray XT5 Production 65K cores, 600 TF2.3 GHz, quad-core Barcelona

February 2009

Cray XT5 Istanbul upgrade, Production ~100,000 cores, 2.6 GHz, 1 PF

November 2009(Actual: Oct 5, 2009)

Delivered over 305M hours to researchers!

Upgrade to Istanbul ProcessorsSeptember 2009

Barcelona Istanbul Impact

Clock Speed 2.3 GHz 2.6 GHz

Core Counts 8 per node 12 per node 70% more FLOPS

Memory Bandwidth

18.1 GB/s/node

> 21.2 GB/s/node 17% more memory bandwidth

Level 3 Cache 4 MB/node 12 MB/node 3x more cache

Memory Upgrade mixed 16 GB / compute node

COLANICS

JAMSTECU of Tokyo

ECMWF

COLA Center for Ocean-Land-Atmosphere Studies, Calverton, MD, USA

ECMWF European Center for Medium-Range Weather Forecasts, Reading, England

JAMSTEC Japan Agency for Marine-Earth Science and Technology, Tokyo, Japan

UTokyo University of Tokyo, Japan

NICS National Institute for Computational Sciences, University of Tennessee, Knoxville, TN USA

An International, Dedicated High-End Computing Project to Revolutionize Climate Modeling

NICAM

Satelite Observation Model Simulation

• The only global atmospheric model capable of resolving clouds

User Survey – Big Results

– Requirements for computing capability and archival storage will continue to grow · 15 respondents expect to store as much as 1 Pbyte annually in the next 5

years– Significant interest in exploring GPGPU’s to increase

computing capability– Concern about visualization and analysis of output data

· moving large datasets home or elsewhere is a challenge· remote visualization is appealing

9

Keeneland – An NSF-Funded Partnership to Enable Large-Scale Computational Science on Heterogeneous Architectures· Track-2D: Experimental HPC System of

Innovative Design – Large GPU cluster– Initial delivery Fermi system – Spring 2010– Full scale system – Spring 2012

· Partners are: Georgia Tech, NICS, U of Tennessee, Oak Ridge, NVIDIA, HP

· Software tools, application development

· TeraGrid Resource Provider

· Operations, User Support at NICS

· Education, Outreach, Training for scientists, students, industry

• Sean Ahern, PI

• Partnership between UT, ORNL, LBNL, U of Wisconsin and NCSA

• Purpose: provide TeraGrid and XD users with remote and shared memory resources for exploring, analyzing, and visualizing large-scale data

• Central hardware: Nautilus, a shared memory SGI UltraViolet system with:

‣ 1024 Intel Nehalem EX cores

‣ 4 TB of shared memory

‣ 16 GPUs

‣ ~1 PB filesystem

‣ TeraGrid and Kraken connectivity

• Provide wide range of software tools for data analysis, visualization, and workflow management

Timeline for Deployment

NICS Education, Outreach, and Training

· “Introduction to Petascale Computing” workshop for the 2009 Richard Tapia Celebration of Diversity in Computing

· Co-sponsored and organized workshop for NIMBioS Center at UT-Knoxville.

· Outreach to student chapter of the Society of Women Engineers at UT-Knoxville.

· Two million CPU hours committed to Institute for Computing in the Humanities, Arts and Social Sciences.

· Participated in student and teacher programs at TG ‘09 and SC09.

· Virtual School for Computational Science and Engineering: two Summer Schools hosted during summer of 2009; hosting three week-long schools in 2010.

· Co-sponsored Hex-Core Cray XT5 workshops in December 2009 (ORNL) and February 2010 (UC-Berkeley)

· TeraGrid/Blue Waters technical workshop, Austin, March 2010

· TeraGrid Introductory training via ReadyTalk, quarterly, online.

· Introductory workshops for UTK users and undergraduate classes.

Thank You !!