10/18/06 J. P. Gary 1G O D D A R D S P A C E F L I G H T C E N T E R
Inputs on Advanced R&E Networks-- Limited to Pat’s Perspective --
Outline•NASA Science Mission, esp. Earth Science•Current GSFC Advanced R&E Networks•On-Going and Future Applications
J. Patrick (Pat) GaryNetwork Projects Leader
Networks and Information Technology Security Group/Computational and Information Sciences and Technology Office
NASA Goddard Space Flight Center
October 18, 2006For MPLS2006 Advanced Networks for Research & Education Panel
10/18/06 J. P. Gary 2G O D D A R D S P A C E F L I G H T C E N T E R
Pat’s Limited PerspectiveNASA
• Science Mission Directorate (HQ)
– Earth Science Division (HQ)
• Goddard Space Flight Center (GSFC)
– Science and Exploration Directorate
» Computational and InformationSciences and Technology Office
10/18/06 J. P. Gary 4G O D D A R D S P A C E F L I G H T C E N T E R
NASA’S VISION
To improve life here, To extend life to there, To find life beyond.
NASA’S MISSION
To understand and protect our home planetTo explore the Universe and search for life
To inspire the next generation ofexplorers
… as only NASA can.
10/18/06 J. P. Gary 6G O D D A R D S P A C E F L I G H T C E N T E R
• How is the global Earth system changing?
• What are the primary forcings of the Earthsystem?
• How does the Earth system respond tonatural and human-induced changes?
• What are the consequences of changes in theEarth system for human civilization?
• How well can we predict future changes in theEarth system?
How is the Earth changing and what are theconsequences of life on Earth?
ESE Fundamental Science Questions
ERBS
Terra
Aqua
GRACE
QuikScatSAGE III
SeaWinds
TRMM
TOMS-EP
UARS
Jason Landsat 7
SORCE
ACRIMSAT
EO-1
TOPEX/Poseidon
SeaWiFS
ICESat
NASA Earth Science Research Satellites
10/18/06J. P. Gary
7
Earth System Enterprise-Data Lives inDistributed Active Archive Centers (DAAC)
SEDAC (0.1 TB)Human Interactions in
Global Change
GES DAAC-GSFC (1334 TB)
Upper AtmosphereAtmospheric Dynamics, Ocean
Color, Global Biosphere, Hydrology,Radiance Data
ASDC-LaRC (340 TB)Radiation Budget,CloudsAerosols, Tropospheric
Chemistry
ORNL (1 TB)Biogeochemical
DynamicsEOS Land Validation
NSIDC (67 TB)Cryosphere
Polar Processes
LPDAAC-EDC (1143 TB)Land Processes
& Features
PODAAC-JPL (6 TB)Ocean Circulation
Air-Sea Interactions
ASF (256 TB)SAR Products
Sea IcePolar Processes
GHRC (4TB)Global
Hydrology
EOS Aura Satellite Will Be Launched SoonChallenge is How to Evolve to New Technologies 10/18/06
J. P. Gary
8
GSFC Managed Networks
Administrative
MissionOperations
Science &Engineering
CNE
IONet
SEN
HECN
HST/JWSTNet
EOS
CNE: Center Network EnvironmentEOS: Earth Observing SystemHECN: High End Computing NetworkHST/JWSTNet:: Hubble Space Telescope/James Webb Space Telescope NetworkIONet: IP Operational NetworkSEN: Science & Engineering Network
10/18/06J. P. Gary
9
xxxxxxxOctober
Updated9/13/06
10/18/06 J. P. Gary 11G O D D A R D S P A C E F L I G H T C E N T E R
The DRAGON TestbedWashington, DC metro region
CLPK
ARLG
DCGW
MCLN
MIT Haystack Observatory(HAYS)
U. S. Naval Observatory(pending)
University of Maryland Baltimore County(UMBC)
Goddard Space Flight Center(GSFC)
National Computational Science Aliance (NCSA)
Univ of Southern California/Information Sciences Institute
(ISIE)
DCNE
MAX
GIG-EF
HOPI
NREN
AbileneCaveWave
Northrop Grumman
Venter
Source: Jerry Sobieski (UMCP/MAX)
10/18/06 J. P. Gary 12G O D D A R D S P A C E F L I G H T C E N T E R
Source: Mark Foster (ARC)
10/18/06 J. P. Gary 13G O D D A R D S P A C E F L I G H T C E N T E R
10/18/06 J. P. Gary 14G O D D A R D S P A C E F L I G H T C E N T E R
10/18/06 J. P. Gary 15G O D D A R D S P A C E F L I G H T C E N T E R
Previous and/or On-Going Applications
• Using ARC/NAS/Columbia Supercomputer (w/NREN)
• Distributed ESMF R&D
• eVLBI (w/MIT-Haystack, …)
• OptIPuter & Multi-channel Collaboration/VideoStreaming Technologies(w/UCSD & UIC)
• 3D HDTV-over-IP R&D (w/Physical Optics Corporation)
• SAN-over-IP (w/UMIACS & NGC)
Inputs on Advanced R&E Networks-- Limited to Pat’s Perspective --
10/18/06 J. P. Gary 16G O D D A R D S P A C E F L I G H T C E N T E R
10/18/06 J. P. Gary 17G O D D A R D S P A C E F L I G H T C E N T E R
10/18/06 J. P. Gary 18G O D D A R D S P A C E F L I G H T C E N T E R
Coupled Atmosphere-Ocean Models
Atmosphere
Ocean
Different grid type, resolution
10/18/06 J. P. Gary 19G O D D A R D S P A C E F L I G H T C E N T E R
Cross-Organization Coupling ofClimate Models through ESMF(A Prototype Over High-Speed Networks)
Shujia Zhou (Lead), C. Cruz, R. Burns, B. Womack, G. Higgins NASASIVO/Northrop Grumman TASC
Collaborators: High-speed network: P. Gary, B. Fink, P. Lang (NASA GSFC/ADNET) Cluster system admin: K. Fisher (NASA GSFC) XCAT/Proteus: M. Govindaraju, K. Chiu, M. Head (SUNY, Binghampton) Models: J. Spahr, C. Mechoso (UCLA), C. Hill (MIT), P. Jones (LANL)
Presented at NASA Exhibit (booth 1810) at SC|05, November 14-18, 2005
10/18/06 J. P. Gary 20G O D D A R D S P A C E F L I G H T C E N T E R
ESMF-Enabled CoupledModels
Atmosphere(e.g., NASA
GEOS)
Ocean(e.g., NASAPoseidon)
AtmosphereData Analysis
(e.g., NOAA GSI)
OceanData Analysis(e.g., NASA)
Land(e.g., NCAR CLM)
Sea Ice(e.g., LANL
CICE)
ESMF Coupler ESMF Component
10/18/06 J. P. Gary 21G O D D A R D S P A C E F L I G H T C E N T E R
NASA AmesMt. View, California
National Lambda Rail (10 - 40 Gb/s)
All modelOutputs
(500 GB perexecution,~15TB for
entire season
00:00 GMT
NCEPInputs (1 GB)
INTEL CorpPortland, Oregon
Northrop GrummanMcLean, Virginia
NASA GoddardGreenbelt, Maryland
Conventional Network(600 Mb/s)
DISTRIBUTEDCOMPUTING
NODES
NEXT-GENNETWORKS
Main Server
NASA GoddardGreenbelt, Maryland
DATASERVERS /
LONG TERMSTORAGE
Tape Backup
NASA GoddardGreenbelt, Maryland
WEBSERVICES /ADVANCED
VISUALIZATIONS
2006 Hurricane Season - Global Modeling
Source: Mike Seablom (GSFC/610.3)
DRAGON eVLBI Experiment Configuration
10/18/06J. P. Gary
22
10/18/06 J. P. Gary 23G O D D A R D S P A C E F L I G H T C E N T E R
Combining Telepresence withRemote Interactive Analysis of Data Over NLR
HDTV Over Lambda
OptIPuter Visualized
Data
SIO/UCSD
NASA Goddard
www.calit2.net/articles/article.php?id=660
August 8, 2005
iGrid 2005 Workshop, 26-29Sep05, UCSD/CalIT2Accelerating the Use of Multi-10Gigabit per Second International and
National Networks: www.igrid2005.org
GSFC’s Ben Kobler (left) and POC’s Sookwang Ro and KirillKolesnikov (right) work to set up POC’s 35” x 35” holographic3D HDTV video display system (center) prior to the start ofiGrid 2005.
Only a non-stereo image of the True-3D display is captured in thisphoto of the real-time stereo-HDTV images transmitted fromGSFC.
Stereoscoptically-aligned Sony HDV 1080i HDR-FX1HDTVcameras and the viewed targets at GSFC.
US130: Real-Time True-3D/HDTV (No Goggles) Visualization Over theNational LambdaRailNASA and Physical Optics Corporation demonstrate a holographic 3D HDTV video display system that doesnot require goggles or other special head gear, using a live cross-country video feed from NASA Goddard SpaceFlight Center to the iGrid 2005 site in San Diego. POC is a NASA SBIR Phase 1 awardee, and worked withNASA GSFC on this project.www.poc.com/emerging_products/3d_display/default.asp
2410/18/06
J. P. Gary
3D HDTV Over Lambda
10/18/06 J. P. Gary 25G O D D A R D S P A C E F L I G H T C E N T E R
Current SAN-over-IP Test-bed
Source: Fritz McCall (UMIACS)
10/18/06 J. P. Gary 26G O D D A R D S P A C E F L I G H T C E N T E R
Wide Area Storage Configuration
RedHat Linux Windows RedHat Linux Windows
McData San Switch
YottaYotta
McData San Switch
McData Channel Extender
EMC DMX EMC ClariionApple XRAID
3000mi/50msSimulated
60mi/1msActual
Northrop McLean VA Northrop McLean VA NASA Greenbelt MD
ADIC StorNext Distributed 3 way Mirror
Brocade SanSwitch
Source: Bob Bramow (YottaYotta)
10/18/06 J. P. Gary 27G O D D A R D S P A C E F L I G H T C E N T E R
Upcoming and/or Future Applications
• SOA+Brokering for ECHO (w/SIO, JPL & UAH)
• Dynamic Linking (w/ORNL, CUNY)
• Grid Computing (w/TBD: SURAGrid, UMBC, …)
• InterPlaNetary Internet
Inputs on Advanced R&E Networks-- Limited to Pat’s Perspective --
10/18/06 J. P. Gary 28G O D D A R D S P A C E F L I G H T C E N T E R
"Brokering and Chaining Distributed Services and Data UsingOptIPuter and the National Lambda Rail" by Ramapriyan (GSFC) et al
to NASA's ROSES NRA
10/18/06 J. P. Gary 29G O D D A R D S P A C E F L I G H T C E N T E R
“Enabling NASA Applications Across Heterogeneous High PerformanceNetworks” by Habib (CUNY) et al to NASA NNH05ZDA001N-Applied
Information Systems Research (a.k.a. ROSES:D3)
InterPlaNetary InternetDefining a New NASA Space Communications Architecture
Source: JPL, Vint Cerf, MCI
MarsNet
10/18/06J. P. Gary
30
10/18/06 J. P. Gary 31G O D D A R D S P A C E F L I G H T C E N T E R
10/18/06 J. P. Gary 32G O D D A R D S P A C E F L I G H T C E N T E R
Backup Slides
Inputs on Advanced R&E Networks-- Limited to Pat’s Perspective --
10/18/06 J. P. Gary 33G O D D A R D S P A C E F L I G H T C E N T E R
10/18/06 J. P. Gary 35G O D D A R D S P A C E F L I G H T C E N T E R
10/18/06 J. P. Gary 36G O D D A R D S P A C E F L I G H T C E N T E R
Previous and/or On-Going Applications• Multi-channel Collaboration/Video Streaming Technologies
– Scalable Adaptive Graphics Environment (SAGE)(http://www.evl.uic.edu/cavern/sage)
– HDTV-over-IP– Demonstrations of 21st Century National-Scale Team Science
(http://www.calit2.net/newsroom/release.php?id=660)• 3D HDTV-over-IP
– 3D Multichannel Networked System via NASA SBIR FY06 Phase2awardee Physical Optics Corporation
– Live 3D HDTV multi-Gbps real-time data streaming from GSFC toholographic display at iGrid2005 as the US130/Real-Time_True-3D_Visualization exhibitor(http://www.igrid2005.org/program/applications/vizservices_3dviz.html)
Inputs on Advanced R&E Networks-- Limited to Pat’s Perspective --
10/18/06 J. P. Gary 37G O D D A R D S P A C E F L I G H T C E N T E R
Previous and/or On-Going Applications• Enabling e-VLBI real-time data flows from GGAO to
MIT/Haystack (http://web.haystack.mit.edu/e-vlbi/evlbi.html)
• Prototyping of Earth System Modeling Framework (ESMF)-based cross-organization coupling of climate models overa high speed network (http://cisto.gsfc.nasa.gov/L-Netpdfs/sc05_esmf_demo_v5.pdf)
• Evaluating SAN-over-IP and distributed shared file systemapplicability to enhancing science data flows– NCCS’ participation in the Data Intensive Computing Environment (DICE)
Project (http://www.avetec.org/dice)– NCCS data portal environment– McCall et al, “A framework for Managing Inter-site Storage Area Networks
using Grid Technologies”(http://romulus.gsfc.nasa.gov/msst/conf2006/Papers/2006-025-McCall.pdf)
Inputs on Advanced R&E Networks-- Limited to Pat’s Perspective --
10/18/06 J. P. Gary 38G O D D A R D S P A C E F L I G H T C E N T E R
Future Plans (partial list)• New NGC(Colshire) and UMBC DWDM connections to DRAGON
• Leverage existing DRAGON-provided 10-Gbps connection with Internet2’sNLR/HOPI lambda
• Support plans identified in NASA NRA Proposals– "MAP Core Integration LambdaGrid Infrastructure" by Smarr (UCSD) et al to NASA's MAP
NRA– "Brokering and Chaining Distributed Services and Data Using OptIPuter and the National
Lambda Rail" by Ramapriyan (GSFC) et al to NASA's ROSES NRA– “Enabling NASA Applications Across Heterogeneous High Performance Networks” by
Habib (CUNY) et al to NASA NNH05ZDA001N-Applied Information Systems Research(a.k.a. ROSES:D3)
• Extend GSFC’s existing10 Gbps L-Net to additional GSFC buildings, computers, andusers; increase the number and type of GSFC science/exploration research projectsthat benefit from the increased throughput performance that multi-wavelength opticalnetworking can provide
• Expand SAN-over-IP testing: intra-GSFC, between GSFC-UMCP & GSFC-ARC & …
Inputs on Advanced R&E Networks-- Limited to Pat’s Perspective --
10/18/06 J. P. Gary 39G O D D A R D S P A C E F L I G H T C E N T E R
NLR/GSFC Applications: Hurricane Prediction
The NASA Finite-Volume GeneralCirculation Model (fvGCM) has beenproducing real-time, high-resolution(~25 km) weather forecasts focused onimproving hurricane track and intensityforecasts. During the active 2004 Atlantic hurricaneseason, the fvGCM provided landfallforecasts with an accuracy of ~100 km up to5 days in advance. The 50–100 Mbps throughput availablebetween fvGCM users at GSFC and theColumbia supercomputer at ARC greatlyhindered carrying out time-criticalsimulations of the hurricanes thatdevastated Florida. The 10 Gbps NLR access will enableremote, 3D visualization analysis as soon asforecast variables become available. Key Contacts: Ricky Rood, Bob Atlas,Horace Mitchell, GSFC; Chris Henze, ARC.
http://fvnwp.gsfc.nasa.gov
In an fvGCM forecast, Hurricane Frances makes landfall onthe Gulf Coast of Florida while Hurricane Ivan intensifies inthe tropical Atlantic. Visualization by J. Williams, GST.
10/18/06 J. P. Gary 40G O D D A R D S P A C E F L I G H T C E N T E R
NLR/GSFC Applications: Global Aerosols
http://www-abc-asia.ucsd.edu
Project Atmospheric Brown Clouds (ABC) isan international effort to discover and analyzeareas of brown colored atmosphere to learn howdust and pollution particles are transported andwhat impacts they have on the environment,climate, agricultural cycles, and quality of life. GSFC and the Scripps Institution ofOceanography (SIO) are planning acollaboration to predict the flow of aerosols fromAsia across the Pacific to the U.S. on timescalesof days to a week. GSFC will provide an aerosol chemical tracermodel (GOCAR) embedded in a high-resolutionregional model (MM5) that can assimilate datafrom Indo-Asian and Pacific ground stations,satellites, and aircraft. Remote computing and analysis tools runningover the NLR will enable acquisition andassimilation of the Project ABC data. Key Contacts: Yoram Kaufman, William Lau,GSFC; V. Ramanathan, Chul Chung, SIO.
The global nature of brown clouds is apparent inanalysis of NASA MODIS Data. Research by V.Ramanathan, C. Corrigan, and M. Ramana, SIO.
Strategically located ground stations in the Indo-Asianand Pacific regions monitor atmospheric pollution.
10/18/06 J. P. Gary 41G O D D A R D S P A C E F L I G H T C E N T E R
NLR/GSFC Applications: Remote Viewing andManipulation of Large Earth Science Data Sets
Remote viewing and manipulation of datasets at GSFC and JPL is needed to supportEOSDIS and Earth system modeling. GSFC’s EOSDIS Clearing House (ECHO)and JPL’s GENESIS prototype scienceanalysis system (iEarth) will becomeconnected over the NLR. The link will enablecomparison of hundreds of terabytes of data,generating large, multi-year climate records. Initial work will focus on the Estimating theCirculation and Climate of the Ocean(ECCO) modeling team. Besides readyaccess to the NLR, the team will needversatile subsetting and other datamanipulation functions to reduce computeand bandwidth requirements as well as a setof Grid-accessible statistical analysis andmodeling operators to refine and validate theECCO models. Key Contacts: ECHO metadata gatewayteam, GSFC; GENESIS team, led by TomYunck, JPL.
http://www.ecco-group.org
Near-surface (15-m) ocean current speed from an eddy-permittingintegration of the cubed-sphere ECCO ocean circulation model.Research by JPL and MIT. Visualization by C. Henze, Ames.
10/18/06 J. P. Gary 42G O D D A R D S P A C E F L I G H T C E N T E R
NLR/GSFC Applications: Integration of Laser andRadar Topographic Data with Land Cover Data
NASA has executed two advancedmissions to create an accurate high-resolutiontopographic model of the Earth: the ShuttleRadar Topography Mission (SRTM) andICESat, with its Geoscience Laser AltimeterSystem (GLAS). The agency now has the opportunity tomerge the two data sets, using SRTM toachieve good coverage and GLAS togenerate calibrated profiles. Properinterpretation requires extracting land coverinformation from Landsat, MODIS, ASTER,and other data archived in multiple DAACs. Use of the NLR and local data mining andsubsetting tools will permit systematic fusionof global data sets, which are not possiblewith current bandwidth. Key Contacts: Bernard Minster, SIO; TomYunck, JPL; Dave Harding, ClaudiaCarabajal, GSFC.
SRTM Topography
ICESat – SRTM Elevations (m)
WUS L2B - MODIS (500m) VCF %Tree Cover vs. ICESat-SRTM Differences
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
-100 -80 -60 -40 -20 0 20 40 60 80 100
ICESat Centroid - 30m SRTM (m)
No
rm.
# o
f O
ccu
rren
ces 0-20% (11490)
20-40% (6294)40-60% (3657)60-80% (12503)80-100% (126)
WUS L2B - MODIS (500m) VCF %Tree Cover vs. ICESat-SRTM Differences
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
-100 -80 -60 -40 -20 0 20 40 60 80 100
ICESat Centroid - 30m SRTM (m)
Norm
. # o
f O
ccurr
ences 0-20% (11490)
20-40% (6294)40-60% (3657)60-80% (12503)80-100% (126)
WUS L2B - MODIS (500m) VCF %Tree Cover vs. Waveform Extent
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150
Waveform Extent (m)
No
rm.
# o
f O
ccu
rren
ces
0-20% (11490)
20-40% (6294)
40-60% (3657)
60-80% (12503)
80-100% (126)
% Tree Cover Classes
MODIS Vegetation Continuous Fields(Hansen et al., 2003)
% Tree Cover% Herbaceous Cover
% Bare Cover
ICESatElevation Profiles
0
3000
meters
Elevation DifferenceHistograms as Function
of % Tree Cover
http://icesat.gsfc.nasa.govhttp://www2.jpl.nasa.gov/srtm
http://glcf.umiacs.umd.edu/data/modis/vcf
Now large-scale simulation has been wide-spread in many disciplines of solid Earth scienceresearch. A typical numerical test in the simulation can easily reach 1013 flops and beyond.
One such research problem that we are working on now is to establish a framework onpredicting geomagnetic secular variation on decadal and longer time scales, utilizing surfacegeomagnetic/paleomagnetic records and our MoSST core dynamics model (Figure 1). In thisapproach, model forecast results and observations are weighted to provide initial state forassimilation (Figure 2). Typically 30 independent numerical tests are necessary for areasonable ensemble size. This could easily require a computing cycle on orders of petaflopsand larger.
A single super-computing facility for such studies is not an optimal choice, due to manylimitations, in particular those on user management and administration. But it is relativelyeasy for users (researchers) to manage because of a unified system environment.
Grid computing can be a much better choice so that independent numerical tests can becarried out independently on different systems. However, researchers (users) have to dealwith heterogeneous systems and other problems, such as those on network communication.
In this poster, we discuss our activities in GSFC on application of grid computation togeodynamics modeling.
Introduction
High speed networking and Grid computing for large-scale simulation in geodynamicsHigh speed networking and Grid computing for large-scale simulation in geodynamics
W. Kuang1, W. Jiang2, S. Zhou3, P. Gary1, M. Seablom1, W. Truszkowski1, J. Odubiyi4, D. Liu2, J. Palencia5, G. Gardner6
1 NASA Goddard Space Flight Center, 2 JCET, UMBC, 3 Northrop Grumman IT/TASC, 4 Bowie State University, 5 Raytheon ITSS, 6 INDUSCORP
Prototype on MoSST simulation with independent systems
Discussions
There are parallel, but related research going on in GSFC on networking and softwaredevelopment. These research activities are updated inhttp://esdcd.gsfc.nasa.gov/LNetImplement.html. Recent overview of GSFC research activitiesis given by Dr. M. Halem and can be found in http://esdcd.gsfc.nasa.gov/L-Netpdfs/ESSAAC_MHpres9904.pdf. Some of the activities listed in the report are shown inFigures, 5 and 6. These activities work towards establishing 21st century cyber infrastructurefor large-scale scientific teamwork based on fast network.
Numerical results from our MoSST coredynamics model
Figure 1. Radial component of themagnetic field at the CMB invertedfrom surface geomagnetic observation(left panel) and from numericalmodeling (top).
( )fofaHxxKxx !+=
Xa : Assimilation solutionXf : Forecast solutionXo : Observation data
Geomagnetic data assimilation
Figure 2. Mathematical foundation of dataassimilation. The common gain K depends onknowledge of error statistics of observations and ofmodels. If ensemble Kalman-filter approach isapplied. An ensemble size of at least 30 (i.e.independent tests) is required.
The objective of this prototype work is to test operability of executing our MoSST coredynamics model on independent computing systems. Individual computing units are slatedout from selected components of our beowulf system to mimic independent computingenvironment. The prototype program for grid computing is built upon xcat3 framework(based on java/python). See Figure 3 for conceptual layout of our prototype experiment.
The sample script and the execution process are shown in Figure 4.
Our prototype experiment is very successful. With this experiment, we can proceed furtherour test on real remote systems. Also with this experiment, we can identify the needs fromthe user’s considerations on supporting environment and other middleware that makes gridcomputing “friendly” .
cw-03
cw-04
cw-02
cw-01
Local
Remote 1
Remote 2
cw-00
Driver
Dispatcher
Architecture
dispatcher Geo1
useCMD useCMD
provideCMDprovideCMD
driver
Geo2
useCMD
provideCMD
1 2
2
3
4
3
Workflow
System configuration
OS: Fedora core 2; MPICH-1.2.5.2; Intel Fortran Compiler;Java 2PE: Dual Intel Xeon, 2.4 Ghz, 1 GB, 1 GigEthernet
Figure 3. Prototypelayout
import sysimport cca
from jarray import zeros
from java.lang import Systemfrom java.lang import String, Object# get the absolute location for XCAT.# create the TypeMap for the user component# create component wrappersgeo1Provides = cca.createComponentWrapper("geo1Provider",geo1ProviderMap)geo2Provides = cca.createComponentWrapper("geo2Provider",geo2ProviderMap)#uses = cca.createComponentWrapper("user", userMap)# assign a machine namecca.setMachineName(uses, "cw-00")cca.setMachineName(provides, "cw-00")cca.setMachineName(geo1Provides, "cw-03")cca.setMachineName(geo2Provides, "cw-04")
# set a creation mechanism to in-processcca.setCreationMechanism(uses, "local")cca.setCreationMechanism(provides, "local")cca.setCreationMechanism(geo1Provides, "ssh")cca.setCreationMechanism(geo2Provides, "ssh").# connect their portscca.connectPorts(uses, "dispatchUsesPort", provides,"dispatchProvidesPort")cca.connectPorts(uses, "geo1UsesPort", geo1Provides,"geo1ProvidesPort")cca.connectPorts(uses, "geo2UsesPort", geo2Provides,"geo2ProvidesPort")# invoke the methodcca.invokeMethodOnComponent(uses,
portClassName, portType, providesPortName, methodName, methodParams)
wjiang@corewulf:~/XCAT_HOME[wjiang@corewulf XCAT_HOME]$ xcat.jython.RunScriptsrc/jython/samples/usesdispatchgeo2ssh.py. . . . . .Executing jython script : src/jython/samples/usesdispatchgeo2ssh.py[ 20:00:11.004 main: xcat.ccacore.XCATServicesImpl.java:118 <init> ] called with name: CompositionTool[ 20:00:12.531 main: xcat.jython.CompositionTool.java:353 createTypeMap ] called[ 20:00:12.554 main: xcat.jython.ComponentWrapper.java:338 getInstanceName ] called. . . . . . .[ 20:00:12.555 main: xcat.jython.CompositionTool.java:233 instantiate ] called for component: user. . . . . .[ 20:00:12.591 main: xcat.ccacore.XCATComponentIDServerImpl.java:110 <init> ] exporting ComponentID as aGrid service. . . . . .[ 20:00:12.597 main: samples.esmfcca2.src.usesgeo2.UsesDispatchGeo2.java:62 setServices ] addingdispatchUsesPort[ 20:00:19.228 Thread-0: samples.esmfcca2.src.usesgeo2.UsesDispatchGeo2.java:152 startExecuting ] Command/home/wjiang/research/app/test1/run_main_mpi is sent to cluster cw-03[ 20:00:19.250 Thread-0: samples.esmfcca2.src.usesgeo2.UsesDispatchGeo2.java:154 startExecuting ] Command/home/wjiang/research/app/test1/run_main_mpi is received![ 20:00:19.358 Thread-0: samples.esmfcca2.src.usesgeo2.UsesDispatchGeo2.java:158 startExecuting ] Postinfo: Application is running on Cluster cw-03. Result will be stored in ~/data/test1[ 20:00:19.359 Thread-0: samples.esmfcca2.src.usesgeo2.UsesDispatchGeo2.java:161 startExecuting ] Command/home/wjiang/research/app/test1/run_main_mpi is sent to cluster cw-04[ 20:00:19.380 Thread-0: samples.esmfcca2.src.usesgeo2.UsesDispatchGeo2.java:163 startExecuting ] Command/home/wjiang/research/app/test1/run_main_mpi is received![ 20:00:19.491 Thread-0: samples.esmfcca2.src.usesgeo2.UsesDispatchGeo2.java:166 startExecuting ] Postinfo: Application is running on Cluster cw-04. Result will be stored in ~/data/test2. . . . . . .[ 20:00:19.491 Thread-0: samples.esmfcca2.src.dispatch.CMDPortDispatch.java:95 useCMD ]CMDPortDispatch.java: useCMD(...)!CMDPortDispatch: useCMD(), geoCMD is Application is running on Cluster cw-04. Result is stored in~/data/test2[ 20:00:19.493 Thread-0: xcat.ports.BasicPortImpl.java:95 destroyImpl ] called; going down in 5 seconds. . . . . .[ 20:00:19.579 Thread-0: samples.esmfcca2.src.usesgeo2.UsesDispatchGeo2.java:189 startExecuting ] YourYoursimulation is successful!!*!!!simulation is successful!!*!!!
Figure 4. Prototype Operation Script (left) and ScreenCaption (right)
Related work at GSFC
High Performance Networking and Remote Data AccessGSFC L-Net for NCCS and Science Buildings
JPG 8/05/04
ISI-E
GSFC at Greenbelt
N
L
R
CMU
JPL
SIO
UCSD
UIC
GWU
UMCP
BosSNet
ATDneto
o
o
MIT/
Haystack
UMBC
D
R
A
G
O
N
ORNL
A512p
A512p
A512p
A512p
A512p
A512p
A512p
A512p
A512p
A512p
A512p
A512p
T512p
T512p
T512p
T512p
T512p
T512p
T512p
T512p
FE64/64
SATA
35TB
Fibre Channel
20TB
SATA
35TB
SATA
35TB
SATA
35TBSATA35TB
SATA35TB
SATA35TB
SATA35TB
FC Switch 128pFC Switch 128p
Fibre Channel
20TB
Fibre Channel
20TB
Fibre Channel
20TB
Fibre
Channel
20TB
Fibre
Channel
20TB
Fibre
Channel
20TB
Fibre
Channel
20TB
InfiniBand10GigE/1GigE
GR
AP
HIC
S
o
o
o
o
o
o
High Performance
Remote Data
Cache Facility
(creating
Inter-Facility virtual
SANs using
SAN-over-IP
technologies)
ARC/
Project
Columbia
A512p
A512p
A512p
A512p
A512p
A512p
A512p
A512p
A512p
A512p
A512p
A512p
T512p
T512p
T512p
T512p
T512p
T512p
T512p
T512p
FE64/64
SATA
35TB
Fibre Channel
20TB
SATA
35TB
SATA
35TB
SATA
35TBSATA35TB
SATA35TB
SATA35TB
SATA35TB
FC Switch 128pFC Switch 128p
Fibre Channel
20TB
Fibre Channel
20TB
Fibre Channel
20TB
Fibre
Channel
20TB
Fibre
Channel
20TB
Fibre
Channel
20TB
Fibre
Channel
20TB
InfiniBand10GigE/1GigE
GR
AP
HIC
S
GISS
o o o
FC
C
P
U
(s)
C
P
U
(s)
VIS
GE Sw/Rtr
o o o
FC
C
P
U
(s)
C
P
U
(s)
VIS
GE Sw/Rtr
o o o
o
o
o
CPU(s)
CPU(s)
CPU(s)
CPU(s)
F
C
o o o
FC
C
P
U
(s)
C
P
U
(s)
VIS VIS
GE Sw/Rtr
iFCP
Gateway
FCIP
Gateway
iSCSI
Gateway
FC
o o o
NCCS
“Classic”
L
e
v
e
l
3
P
O
P
a
t
M
c
L
e
a
n
VIS
Op. Sw/
OADM
o o o
FC
C
P
U
(s)
C
P
U
(s)
VIS VIS
GE Sw/Rtr
o o o
FC
C
P
U
(s)
C
P
U
(s)
VIS VIS
GE Sw/Rtr
Other GSFC Science Data Facilities 10-GE
SW/RTR
10-GE
SW/RTRB
G
P
F
W
10-GE Sw/Rtr
w/OSPF
10 Gbps GE
1 Gbps GE
Dark Fiber
Legend
2 Gbps FC
Figure 5. NASA GSFC IRAD work on regional fastnetwork
Figure 6.
1. Our research on geomagnetic data assimilation can greatly benefit from grid computing.
2. Our prototype experiment is successful and can be readily expanded to systems withidentical settings and SSH communication protocol.
3. Our prototype experiment is limited in many areas, such as handling networkcommunication between independent systems (e.g. instant feedback of remote systems tohost systems), heterogeneous environment (e.g. prior knowledge on participating systemsis necessary), authentication (e.g. prototype cannot handle high level access securityrequirement). Therefore, further experiment is needed to improve our work, such asintegrating our work with other (developed and developing) middleware handling theproblems.
10/18/06J. P. Gary
43
10/18/06 J. P. Gary 44G O D D A R D S P A C E F L I G H T C E N T E R
GSFC High End Computer Network (HECN) Project’sResearch Partners and Collaborators
•DRAGON Project: http://dragon.maxgigapop.net/twiki/bin/view/DRAGON/WebHome•PI: Jerry Sobieski (UMCP)•GSFC L-Net on DRAGON network diagram: http://dragon.maxgigapop.net/twiki/bin/view/DRAGON/Network
•e-VLBI Project: http://web.haystack.mit.edu/e-vlbi/evlbi.html•PI: Alan Whitney (MIT/Haystack)•GSFC L-Net on e-VLBI network diagram: http://cisto.gsfc.nasa.gov/L-Netpdfs/SC04_eVLBI_network.pdf
•GLIF: http://www.glif.is/•Chair: Kees Neggers (SURFnet)•GLIF network diagrams: http://www.glif.is/publications/#maps
•NGC IT Sector: http://www.it.northropgrumman.com/index.html•PI: Brice Womack (NGC)•GSFC L-Net on NGC IT Sector Colshire network diagram: http://cisto.gsfc.nasa.gov/L-Netpdfs/DRAGON_NGC_030606.pdf
•NLR: http://www.nlr.net/•CEO: Tom West (NLR)•NLR network diagram: http://www.nlr.net/infrastructure/
•NREN Project: http://www.nren.nasa.gov/•PM: Ken Freeman (ARC)•GSFC L-Net/SEN on NREN network diagram: http://cisto.gsfc.nasa.gov/L-Netpdfs/CENIC2006_13_mfoster_excerpts.pdf
•OptIPuter Project: http://www.optiputer.net/•PI: Larry Smarr (UCSD)•GSFC L-Net on OptIPuter network diagram: http://cisto.gsfc.nasa.gov/L-Netpdfs/SMARR-OptIPuter-AHM-gold.pdf
•TeraFlow Testbed Project: http://www.teraflowtestbed.net/•PI: Robert Grossman (UIC)•GSFC L-Net on TeraFlow Testbed network diagram: http://www.ncdm.uic.edu/maps/index.jpeg
10/18/06 J. P. Gary 45G O D D A R D S P A C E F L I G H T C E N T E R
GSFC Internal
• High End Computer Network Team– Bill Fink/606.1– Kevin Kranacs/585– Paul Lang/ADNET/606.1– Aruna Muppalla/ADNET/606.1– Jeff Martz/CSC/606.2– Mike Steffenelli/CSC/606.2– Kevin Fisher/586/UMBC coop
• ESDIS Network Prototyping Lab– George Uhl/SWALES/423
• ESTC Computing Technology Project– PM: Jim Fischer/606
• IT Pathfinder Working Group– Chair: Dr. Milton Halem/Emeritus &
UMBC
• Thunderhead Cluster– John Dorband/696
Special AcknowledgementsGSFC External
• National LambdaRail– CEO: Tom West– Net Eng Lead: Debbie Montano
• OptIPuter Project (NSF-funded)– PI: Dr. Larry Smarr/UCSD– Co-PI: Dr. Tom DeFanti/UIC– PM: Maxine Brown/UIC– UCSD Net Eng: Greg Hidley, Arron Chin,
Phil Papodopolos– U IC Net Eng: Alan Verlo, Linda Winkler
• DRAGON Project (NSF-funded)– PI: Jerry Sobieski/UMCP– Co-I: Tom Lehman/USC-ISI/E– Net Eng: Chris Tracy/UMCP
• NASA Research and Education Network– DPM: Kevin Jones/ARC
Inputs on Advanced R&E Networks-- Limited to Pat’s Perspective --
10/18/06 J. P. Gary 46G O D D A R D S P A C E F L I G H T C E N T E R
GSFC Lambda Network Project Website
• http://cisto.gsfc.nasa.gov/IRAD_Lambda.html
– Designs• GSFC Local Network Part (i.e., within GSFC)• Regional Network Part (i.e., between GSFC in Greenbelt, MD, & Level3
POP in McLean, VA, typically involving the DRAGON optical network)• Transcontinental Network Part (i.e., use of NLR, GSFC 10-GE switch &
workstations in the Level3 POP in McLean, VA, & remote end users/sites)– Implementation Status
• GSFC Local Network Part• Regional Network Part• Transcontinental Network Part
– Presentations/Events in the News• Eg: P. Gary’s 18Feb05 presentation at GSFC’s FY04 IRAD Colloquium
<http://cisto.gsfc.nasa.gov/L-Netpdfs/FY04IRADGARY.pdf>• Live Demonstration of 21st Century National-Scale Team Science
<http://www.calit2.net/articles/article.php?id=660>– Related Links (e.g., DRAGON, HOPI, NLR, OptIPuter, …)
Inputs on Advanced R&E Networks-- Limited to Pat’s Perspective --