David A. LifkaChief Technical OfficerCornell Theory Centerlifka@tc.cornell.edu

Cornell Theory Center

Windows High Performance Computing Solutions Center

A Center of Excellence

in High Performance Computing

and Interdisciplinary Research

located at Cornell University

CTC Manahattan

A Globally Accessible Center Providing

High Performance Computing Solutions to

Today’s Business and Finance Problems

located in the heart of the Financial District.

Supporting World Class Computational Sciencehttp://www.tc.cornell.edu/WhoWeAre/AboutCTC/index.asp

Worlds First, and Largest, All Windows Production HPC Facilityhttp://www.tc.cornell.edu/Services/Cluster/index.asp

Corporate Programhttp://www.tc.cornell.edu/Corporate/index.asp

David A. LifkaChief Technical OfficerCornell Theory Centerlifka@tc.cornell.edu

Microsoft Windows High Performance ComputingSupporting over 100 Diverse Areas of Research at Cornell

Computational Materials Institutehttp://www.tc.cornell.edu/Research/CMI/index.asp

Cornell Institute for Social and Economic Researchhttp://www.ciser.cornell.edu/

Computational Biology Service Unithttp://cbsu.tc.cornell.edu/index.htm

Computational Genomicshttp://www.genomics.cornell.edu/focus_areas/computational/

Computational Financehttp://www.ctc-manhattan.com/Research/index.asp

David A. LifkaChief Technical OfficerCornell Theory Centerlifka@tc.cornell.edu

David A. LifkaChief Technical OfficerCornell Theory Centerlifka@tc.cornell.edu

CTC Clusters

Cluster Nodes Procs Server Processor Memory Disk Net

CTCDN 32 64 Dell 2450 PIII 900 MHz 2GB 50GB-RAID5 100BT

MANDN 4 8 Dell 1650 PIII 1.4GHz 1GB 30GB-RAID5 100BT

Total 36 72 72GB 1.73TB

Cluster Nodes Procs Server Processor Memory Disk Net

Velocity 1 64 256 Dell 6350 PIII Xeon 500 MHz

4GB 50GB-RAID0 Giganet + 100BT

Velocity 1 Plus 64 128 Dell 2450 PIII 733 MHz 2GB 50GB-RAID0 Giganet + 100BT

CMI 64 128 Dell 1550 PIII 1GHz 2GB 50GB-RAID0 Giganet + 100BT

Development 8 16 Dell 1550 PIII 800MHz 2GB 50GB-RAID0 Giganet + 100BT

Serial 18 18 Dell 2450 PIII 800MHz 1GB 50GB-RAID0 100BT

Long 17 17 Dell 2450 PIII 600MHz 1GB 50GB-RAID0 100BT

CBSU 192 384 Dell 2650 Xeon 2.4GHz 2GB 50GB-RAID0 100BT

CBWeb 64 128 Dell 1550 PIII 1GHz 2GB 50GB-RAID0 Giganet + 100BT

Velocity 2 128 256 Dell 2650 Xeon 2.4GHz 2GB 50GB-RAID0 1000BT

Manhattan 16 32 Dell 2650 Xeon 2.4GHz 2GB 50GB-RAID0 1000BT

Total 575 1,363 1.363TB 28.75TB

David A. LifkaChief Technical OfficerCornell Theory Centerlifka@tc.cornell.edu

David LifkaCTO

Resa AlvordSystems Manager

Susan MehringerConsulting Manager

Lynn BairdBrenda Lapp

Will LawLaurie McEvoyRuth MitchellMike PadulaDon Skillings

Linda ButtelPat ColasurdoSusan Martin

Daniel SverdlikJohn ZollwegSlava Zaloj

Lucy WalleR&D Manager

Lee GrantDrew DolgertKen Ridley

Hardik Shukla

David A. LifkaChief Technical OfficerCornell Theory Centerlifka@tc.cornell.edu

Data Intensive ComputingSeamless Open Interfaces to Data Sources

Arecibo - World’s largest Radiotelescope Johannes Gehrke, Jim Cordes, David Lifka, Jim GrayServing Astronomy Data via SQL Server and Web Services

http://arecibo.tc.cornell.edu/arecibo/index.aspxhttp://www.cs.cornell.edu/johannes/

Computational Materials InstituteTony Ingraffea, Keshav PingaliServing Finite Element Models via SQL Server and Web Services

http://www.tc.cornell.edu/Research/CMI/Multiscale/index.asp

Physically Accurate ImagerySteve Marschner

http://www.cs.cornell.edu/~srm/

The Structure and Evolution of the WebWilliam Arms

http://www.cs.cornell.edu/wya/

David A. LifkaChief Technical OfficerCornell Theory Centerlifka@tc.cornell.edu

What makes one Job Scheduler better than another?• Job Schedulers match resource requests to resources• Job Schedulers provide mechanisms to implement policies• Job Schedulers should be integrated with Resource Management• Ease of Use & Administration

Let the Database do the work• Designed to perform tasks like resource matching• Implementing Scheduling Algorithms and Policies is straight forward • Ideal tool for Scalable Resource Management

Don’t Reinvent what the Platform Provides

• Windows Server 2003 – Delegation & Impersonation

• Active Directory

• SQL Server

• Windows Management & Instrumentation

• Microsoft Message Queues

• Performance Alerts• Identify “poison nodes”• Node changes

The Velocity Scheduler Motivation

David A. LifkaChief Technical OfficerCornell Theory Centerlifka@tc.cornell.edu

Connecting to the Grid with Standard Windows Components

CertificateServer

Web Server(IIS 6.0)

Security TokenServer

SQL CertificateDatabase

Domain Controller

Clusters WorkstationsSQL HPC

Data Sources

Local Job Scheduling &Resource Management

RequestManager

submitcancelrestartsubmitsubmitJobScheduler

RequestManager

submitcancelrestartsubmitsubmitJobScheduler

RequestManager

submitcancelrestartsubmitsubmitJobScheduler

RequestManager

submitcancelrestartsubmitsubmitJobScheduler

RequestManager

submitcancelrestartsubmitsubmitJobScheduler

David A. LifkaChief Technical OfficerCornell Theory Centerlifka@tc.cornell.edu

Gordon Bell, senior researcher at Microsoft's Bay Area Research Center, concurs. "This project demonstrates the potential of a new way to build applications and the potential for a new software industry structure based on delivering results," he said. "Users don't have to buy apps programs and maintain a more complex software environment; they simply call a program or database. This is one of the few projects that I would call a Web Service, and it is well beyond what is running on today's experimental Grid.“

Frederica Darema, Senior Science and Technology Advisor in the CISE Directorate at NSF, is the cognizant NSF official for the ASP project. "It is to the credit of the scientists working on this project that they have developed such a cohesive collaboration," she said. "I am pushing for a new paradigm in application simulation and measurement capabilities called Dynamic Data Driven Application Systems, and the ASP model of multidisciplinary collaboration, together with the technology advances made by the project, are essential for enabling this new paradigm. I am very pleased with the outcomes of this project and its broader impact."

David A. LifkaChief Technical OfficerCornell Theory Centerlifka@tc.cornell.edu

Visualization

CTC Visualization Informationhttp://www.tc.cornell.edu/Services/Vis

Innovative use of Windows based Virtual Reality and VisualizationUsing the CAVE to explore the three-dimensional structures of proteins

OpenDXhttp://www.tc.cornell.edu/Services/Vis

VTK for CAVE Animationshttp://www.tc.cornell.edu/Services/vis/Cave

David A. LifkaChief Technical OfficerCornell Theory Centerlifka@tc.cornell.edu

Summary

The CTC Approach Windows HPC

Community Served Cornell Research Community and Collaborators

Why TCO, Usability, Integrated Security, Development Environment, Innovation

Costs Hardware, Software, People

Allocation Issues Pier Reviewed based on Academic Merit

Cost RecoveryIndustry Clients & Partnerships, State & Federal Funding Agencies

Proud of CTC making the Windows HPC vision a reality.

Gotchas Never get too Comfortable with a Solution. - expect more - always look to the future