About the EnLightened Computing ProjectPrepared for Grid Computing Classat UNCW and UNCC
Carla S. HuntSenior Solutions ArchitectMCNCApril 12, 2007
Your Presenter
Education: B.S. Computer Science, University of Wilmington M.S. Information Sciences, North Carolina Central
University Industry Experience
IBM, Network and Systems Engineer Robert Taylor Associates, Chief Internet Services
Engineer Sentinel Services, Information Technology
Consultant MCNC, Senior Solutions Architect
MCNC & NCREN
Who we are: MCNC (Microelectronics Center of NC)
We own and operate NCREN (NC Research and Education Network) One of the most advanced statewide research and education networks in
the country Provide Internet, video and related services to University of North
Carolina’s 16 campus system, Duke, Wake Forest, etc. Non-profit organization, about 50 employees
History: Commissioned by the General Assembly in 1980 to further economic
development in the state of North Carolina What we do:
Provide non-commodity Internet and related services to more than half a million users daily!
Network has an optical ring based architecture with speeds up to 2.5 Gbps
EnLIGHTened Computing Project
International recognition and collaboration NSF seed funded 10 Gbps national research testbed Grid Middleware developed to co-schedule and co-allocate
network and compute resources
Major Accomplishments: First ever demonstration of trans-pacific, inter-domain co-
scheduling of network and compute resources across two national grid computing research testbeds
First ever demonstration of External Network to Network Interface (E-NNI) capability to establish on-demand end-to-end light paths via Calient optical switches
First ever high definition video broadcast over the Internet of classroom instruction—LSU high performance computing class
EnLightened and G-Lambda GLIF Demonstration
Who: EnLIGHTened (US) and G-Lambda (Japan)
Where: 11th floor of AIST building in Tokyo, Japan
When: September 11th-13th, 2006 What: Researchers from US and Japan
demonstrated trans-pacific, inter-domain advance reservation of coordinated network and compute resources across two national research testbeds during GLIF’s 6th Annual Global Lambdagrid Workshop
Organization of Research Teams
Testbed Network and Compute Resources
ApplicationMiddlewarePerformance and monitoring
EnLIGHTened Testbed
Testbed Components
Compute Resources:ClustersStorageClient machines/displays
Network Resources:Core Optical SwitchesOther Network Devices
US Domain: Compute Resources
X1U
RA1
BT2
CH1
VC1
LA1
BT1
Caltech—LA1
4 nodes (LA1-LA4)2 nodes---4x2.2GHz Sun Opteron2 nodes---4x2.4GHz Tyan S2891 Opteron
RHELv4, 64 bitTorque and MauiGlobus Toolkit10 Gig Nic cardsShared resource sometimes
CCT at LSU: BT1
Pelican.cct.lsu.edu14 nodes8 x 1.9 GHz IBM POWER5 processors16 GB RAM per nodeAIX 5.2 w/ LoadlevelerShared resource
CCT at LSU: BT2
Santaka.cct.lsu.eduSGI shared memory32 – 1.5 GHz Intel Itanium2 processors128 GB RAM per systemSUSE Enterprise 9/ PBSPro10 Gig Nic card (I think)Shared resource
MCNC: RA1
Kite1.enlightenedcomputing.orgApproximately 40 nodes available2x2.8GHz Intel Xeon4 GB RAMRHELv4Torque/MauiGlobus Toolkit v 4.0.2Shared resource
VCL at NCSU: VC1
vclb5-1.hpc.ncsu.eduIBM Blade CenterRHELv4Torque/MauiGlobus Toolkit v 4.0.2Dedicated Resource
Coordinating Compute Resources
Policy for shared & borrowed resources User accounts Security
Installation and configuration of Operating systems Cluster management software (scheduler, resource mgr) Grid middleware (globus toolkit and grid security)
IP address management Other
Application requirements (software libraries or pkgs) Performance tuning (tcp parameters for network card)
KDDINRM
CRM
Cluster
CRM
Cluster
CRM
Cluster
NTTNRM
CRM
Cluster
CRM
Cluster
ELNRM
CRM
Cluster
CRM
Cluster
CRM
Cluster
JAP
AN
CRM CRM CRMCRM CRM CRM
USApplication
(MPI)Application
(Visualization)
Request Networkbandwidthand Computers
ReservationFrom xx:xx to yy:yy
Request Networkbandwidthand Computers
ReservationFrom xx:xx to yy:yy
JapanApplication
GLGrid Resource
Scheduler
EL→GLwrapper
KDDINRM
CRM
Cluster
CRM
Cluster
CRM
Cluster
NTTNRM
CRM
Cluster
CRM
Cluster
ELNRM
CRM
Cluster
CRM
Cluster
CRM
Cluster
USApplication
ELGrid ResourceCoordinator/
App. Launcher
EL→GLGNS-WSIwrapper
GNS-WSI
JAP
AN
US
GL: G-lambdaEL: Enlightened
GL request
EL request
G-lambda/Enlightened middleware coordination diagram
EL→GLCRM
wrapperHARC
Acceptor
Resource map
X2S
Japan South
Japan North US
X2N
X1N X1U
X1S
KMFFUK
KHN
AKB
RA1
BT2
KAN
TKB
OSA
CH1