The Quarterly Publication of the NASA Advanced Supercomputing Division New algorithms have enabled NAS researchers to visualize extremely large datasets using personal workstations, cutting computation time and costs, while increasing resolution. See page 16 Center for Nanotechnology Earns High Marks From Peer Reviewers — 4 Reducing Jet Noise Through Computational Aeroacoustics — 6 Public Key Infrastructure: Get A Passport To Grid Country — 10 The Power of Unsteady Flow Visualization — 16 IPG Supports National Airspace Demonstrations — 14 Summer 2001
Transcript
The Quarterly Publication of the NASA Advanced Supercomputing Division
New algorithms have enabled NAS researchers tovisualize extremely large datasets using personal
workstations, cutting computation time andcosts, while increasing resolution. See page 16
Center for Nanotechnology Earns High Marks From Peer Reviewers — 4
Reducing Jet Noise Through Computational Aeroacoustics — 6
Public Key Infrastructure: Get A Passport To Grid Country — 10
The Power of Unsteady Flow Visualization — 16
IPG Supports National Airspace Demonstrations — 14
Summer 2001
Vol. 2, No. 2 • Summer 2001
Executive Editors: Bill FeiereisenJohn Ziebarth
Editor: Nicholas A. Veronico
Writers: Holly A. AmundsonNeal ChaderjianJill A. DunbarDavid EllsworthTimothy A. Sandstrom
Graphic Designer: Sue KimIllustrator: Gail Felchle
Contact the Editor:Nicholas A. VeronicoGridpointsM/S 258-6Moffett Field, CA 94035-1000Tel. (650) 604-3046Fax (650) [email protected]
Gridpoints is a quarterly publication of the NASAdvanced Supercomputing Division at NASAAmes Research Center. The division’s staff includesgovernment employees and contractors fromAdvanced Management Technology Inc., Compu-ter Sciences Corp., Cray Inc., Eloret, Raytheon Co.,and SGI.Gridpoints acknowledges that some words, modelnames and designations mentioned herein are theproperty of the trademark holder. Gridpoints usesthem for identification purposes only.
The staff of Gridpoints would like tothank the following people for theirassistance in preparing this publication:David Alfano, Dick Anderson, CharlesBauschlicher, Michael Boswell, JohnnyChang, Jill Dunbar, Chris Gong, T.R.Govindan, Daniel Herr, Mary Hultquist,Bill Johnston, Randy Kaemmerer, ChrisKleiber, MiYoung Koo, Marco Librero,Y.K. Liu, Bill McDermott, Nagi Mansour,Meyya Meyappan, Pat Moran, GeorgeMyers, Terry Nelson, Jana Nguyen,Marcia Redmond, Ryan Spaulding,Deepak Srivastava, Sumit Talwar, LeighAnn Tanner, Bill Thigpen, Judith Utley,Arsi Vaziri, Cliff Williams
On The Cover:
Subscribe to GridpointsSubscriptions to Gridpoints are complementary.Please send e-mail with your name, companyname, title, address, and zip code to:
Note: Subscribers outside the continental UnitedStates can access Gridpoints on the web in PDFformat at: http://www.nas.nasa.gov/gridpoints
The Quarterly Publication of the Numerical Aerospace Simulation Systems Division4
6
10
16
FeaturesNASA Ames Center forNanotechnology Earns HighMarks from ReviewersA biannual peer review of the Center for Nanotechnologyprovides guidance to researchers while showcasing thework of Ames scientists.Holly A. Amundson
Public Key Infrastructure:Get a Passport to Grid CountryMembers of the Information Power Grid development team recently established a system toissue certificates (passports) giving users access togrid computer resources.Holly A. Amundson
GlobusSystemGlobusSystem
IPGCertificate
Scripts
IPGCertificate
Scripts
NetscapeCertificate
System
X.500DirectoryDatabase
NetscapeCertificate
System
X.500DirectoryDatabase
Reducing Jet Noise ThroughComputational AeroacousticsScientists at the joint NASA/Stanford UniversityCenter for Turbulence Research are using NAS com-putational resources to predict the radiated jet noise ofaircraft engines in an effort to reduce noise emissions.Nicholas A. Veronico
IPG Supports National AirspaceSimulation DemonstrationsThe Computational Sciences Division and mem-bers of the IPG team collaborated to simulatecommercial air traffic crossing the United States.Holly A. Amundson
The Power of Unsteady Flow VisualizationNew algorithms have enabled NAS dataanalysis group researchers to visualize extreme-ly large datasets using personal workstations. Timothy A. Sandstrom and Neal Chaderjian
Newly developed algorithms have enabled NAS data analysis group researchers tovisualize the ground vortex and jet exhaust from the Harrier aircraft from differentangles. For more information, please turn to page 16.
14
Gridpoints 1
It’s official – the NAS division has modified its nameto more accurately reflect our expanded role in thesupercomputing community within NASA and ex-
ternally in the high-end computing community. Nowknown as the NASA Advanced Supercomputing Divis-ion, we were formed in the early 1980s to simultaneous-ly develop supercomputers and advance the science ofcomputational fluid dynamics. In the last five years the division has broadenedits focus to support other areas of NASA computational sciences by bringingnew developments in high-performance computing to bear in Earth-, space-,life-sciences, and nanotechnology, as well as our traditional area of aeronautics.
Distributed high-performance computing is the future model by which wewill deliver computational horsepower to our scientists, allowing remotely lo-cated researchers to access distant computational resources through a commoninterface. The Information Power Grid is NASA’s contribution to the world-wide community developing this technology.
We continue to co-develop the “power generators” on this grid and apply themto NASA’s scientific research efforts. As an example, the computer scientists inour terascale applications group have worked during the last year with Earthscientists at NASA Goddard’s Data Assimilation Office to improve the speedat which we are able to simulate the Earth’s climate. The computational toolsof global climate modeling are indispensable for answering questions relatedto our changing environment. But these questions demand computers andsoftware tools that are among the largest and most intricate that have everbeen produced. Working with Goddard and our partners in industry, we havebeen able to apply new programming techniques to the Data AssimilationSystem (DAS) that allow us to speed up our calculations by a factor of ten.
Running on the new 512-processor SGI Origin 3800 supercomputer, Chap-man, using 384 processors, the DAS system can now process more than 200days of weather information per day. Our terascale applications group research-ers believe they can further increase the code’s performance by a factor of two– delivering more than 400 days per day.
Using multi-level parallelism (MLP) coding techniques developed at NAS,researchers anticipate increasing the size of the Data Assimilation System’scomputations by adding more weather observations into the calculation whilereducing the memory requirements needed to complete those calculations.Better utilization of our computational resources requires less capital invest-ment in computer hardware, which translates into the NASA mantra of“faster, better, cheaper.” The combination of MLP coding techniques and thesingle-system image architecture of the Origin 3800 supercomputers promisesa new level of performance in global climate modeling and, therefore, inNASA’s ability to address the important questions of global climate change.
NAS MissionTo lead the country in the research,development, and delivery of revolu-tionary, high-end computing servicesand technologies, such as applicationsand algorithms, tools, system software,and hardware to facilitate NASA mis-sion success.
NewsFromNAS
NAS Name ChangeReflects High-EndComputing Focus
A fter nearly 20 years of sustainedgrowth, the Numerical Aerospace
Simulation (NAS) Systems Division,part of the Information Sciences andTechnology Directorate at Ames, ischanging its name to the NASA Ad-vanced Supercomputing (NAS) Divis-ion.The change more accurately reflectsthe division’s leadership role within thehigh-performance computing and in-formation technology communities.
The NAS Division was formed in theearly 1980s to manage NASA’s compu-tational fluid dynamics efforts and themachines used in CFD. The divisionhas since evolved as a world leader inthe development of NASA’s Infor-mation Power Grid (IPG), whichallows remotely located researchers toaccess distant computational resources;and through its NASA Research andEngineering Network (NREN) affili-ate, engineers within the NAS Divisionare working to build the Next Gen-eration Internet.
The NAS Division will continue to fo-cus in areas of research that are key todeveloping new, integrated high-perfor-mance computing environments suchas the the IPG: applications, networks,problem-solving environments, high-performance computing, mass storage,
Continued on Page 2
2 Gridpoints
News From NASgrid common services, and other research areas, such as nan-otechnology and device modeling.
NAS Engineer Honored at AnnualFlemming Awards
NAS Division aerospace engineer Stuart Rogers wasawarded the prestigious Arthur S. Flemming Award for
his contributions to the field of computational fluid dynam-ics (CFD) during a ceremony at George Washington Uni-versity, Washington, D.C., on June 5. Rogers’ developmentand application of CFD tools has resulted in significantstrides in solving real-world fluid dynamics problems, in-cluding analysis and design of complex aircraft, spacecraft,and submarines.
Rogers’ work has had far-reaching benefits to NASA and theaerospace industry by providing tools that reduce the costand design-cycle time of aerospace vehicles and components.
“Stuart has expertise not only in algorithms for computa-tional flow simulation and in computer science for develop-ing software, but he also has a deep understanding of flowphysics,” says Dochan Kwak, chief of NAS’s ApplicationsBranch. “On top of his multifaceted talents, Stuart is anextremely hard-working researcher. It’s not surprising thathe’s made many critical contributions to Ames, NASA, andthe nation’s technical well-being.”
Among Rogers’ many important accomplishments in his 12years at NASA Ames Research Center are the development ofa software tools package that allows first-of-a-kind flow com-
putation about an entire subsonic transport aircraft, includ-ing simulation of a complete Boeing 777 aircraft in a high-lift landing configuration. The Chimera Grid Tools softwarepackage, together with the OVERFLOW flow solver,received honorable mention in the 1998 NASA Software ofthe Year Award.
Rogers also co-developed the INS3D code resulting in world-class production software for the solution of incompressibleNavier-Stokes equations. The code has had a broad impacton a wide range of applications, including advanced rocketpump development and the DeBakey Ventricular Assist De-vice, a miniaturized heart pump that prolongs the life ofpatients awaiting a transplant.
Rogers holds a bachelor’s degree and a master’s degree inAerospace Engineering from the University of Colorado,and a doctorate in Aeronautics and Astronautics Engineeringfrom Stanford University. His recent research and develop-ment work has been funded by NASA’s High PerformanceComputing and Communications Program. The Arthur S.Flemming Awards, established in 1948, honors outstand-ing federal employees.
NASA Ames Gets New Cray SV1Vector Supercomputer
NASA’s Consolidated Supercomputing ManagementOffice (CoSMO) has leased a Cray SV1e vector super-
computer from Cray Inc., Seattle, through GovernmentMicro Resources Inc. The system, installed at the NAS Facil-ity on May 25, will be used for scientific applications in aero-space, earth science, and space science.
The new SV1e, which will replace CoSMO’s current CRAYC90, has 32 processors (CPUs) and four gigawords (GW) ofmain memory, compared to the C90’s 16 CPUs and oneGW of memory. Another four GW of solid-state memorywill be installed later this year to create a Cray SV1ex. The air-cooled SV1e is a fourth-generation complementary metal-oxide semiconductor vector system designed to handle abroad range of application workloads.
“After installation and acceptance of the ‘x’ solid state mem-ory, we hope to transition all users from the C90 to the CraySV1ex,” says CoSMO Deputy Director Ken Stevens. TheSV1ex enhancement will include improved clock speed to 450megahertz and improved cache over the Cray SV1e.
NAS staff will install additional applications (such as OVER-FLOW and Laura) and math libraries on the SV1e to matchthe current C90 environment. After benchmarks are run, sev-eral experienced users will test-run their own applications.After running both systems in parallel for about a month, theC90 will be retired. Once testing is complete, users from gov-ernment organizations – as well as industry and university
Stephen J. Trachtenberg, right, George Washington Universitypresident, presents NAS researcher Stuart Rogers with theArthur S. Flemming Award during the June 5 ceremony inWashington, D.C. (Washington Photography Productions Inc.)
www.nas.nasa.gov/gridpoints Gridpoints 3
users with government contracts, grants, or cooperative agree-ments – will be able to use the SV1e for their large-scale, sci-entific applications.
NAS Researcher Tackles PuzzlingNanotechnology Problem
NAS researcher Toshishige Yamada recently presented anew interpretation of a problem that has puzzled sci-
entists for several years. In the past, researchers have observedstrange current-voltage (I-V) patterns in a scanning tunnel-ing microscope tip-carbon nanotube system, and attributedthem to the intrinsic nanotube properties.
Now, Yamada has proposed a new model for the observedpatterns. His work shows that a measurement electrode-nanotube junction is responsible for the observed behavior,and not the nanotube itself. Yamada’s paper, “Modeling ofElectronic Transport in Scanning Tunneling Microscope Tip– Carbon Nanotube System,” was presented at the AmericanPhysical Society Meeting in Seattle, March 12-16, and pub-lished in Applied Physical Letters, Vol. 78 (12) pp. 1739-1741(2001). Applied Physical Letters is on the web at:http://ojps.aip.org/aplo
Report Compares Parameter Study Tools
A recent NAS technical report “A Comparison of Para-meter Study Creation and Job Submission Tools,”
compares the differences among available general-purposeparameter study and job submission tools. NAS authorsAdrian DeVivo, Maurice Yarrow, and Karen McCann focuson comparing a new NAS-developed software package,called ILab, with other tools. Results show that ILab is easi-er to use, completes jobs significantly faster, and is bettersuited to research and engineering environments.
The team compared the functionality of ILab, Nimrod/G,Nimrod/Clustor, Condor, and AppLeS/APST. For back-ground on ILab, see the NAS feature story, “ILab: New IPGTool Improves Parameter Study Implementation” (Grid-points, Spring 2001, page 12). A postscript or PDF version ofthe technical report can be downloaded at:www.nas.nasa.gov/Research/Reports/Techreports/2001/nas-01-002-abstract.html
Mansour Appointed Deputy Director for CTR
Nagi Nicolas Mansour, lead scientist for the NASDivision’s Physics Simulation and Modeling Office, has
been appointed Deputy Director for the Center for Turbu-lence Research (CTR) at Ames Research Center. Mansourhas extensive experience in turbulence research, and has serv-ed as Ames coordinator to the CTR for the past six years. He
will continue his duties in the NAS Division. Mansour holdsa bachelor’s degree in mechanical engineering, master’s de-grees in mechanical engineering and mathematics, and a doc-torate in mechanical engineering. In recognition of his work,Mansour was elected as an Associate Fellow of the AmericanInstitute of Aeronautics and Astronautics, and Fellow of theAmerican Physical Society for his work on turbulence, anddrops and bubbles.
The CTR is a research consortium for fundamental study ofturbulent flows, jointly operated by NASA Ames ResearchCenter and Stanford University. The CTR’s principal objec-tive is to stimulate significant advances in physical under-standing of turbulence, leading to improved capabilities forcontrol of turbulence and turbulence modeling for engineer-ing analysis. Particular emphasis is placed on probing turbu-lent flow fields developed by direct numerical simulationsand/or laboratory experiments using new diagnostic tech-niques and mathematical methods, and on concepts for tur-bulence control and modeling. The CTR is directing its at-tention to the application of fluid dynamics to biology andthe origin of life, protoplanetary disks, and atmospheric andgeophysical flow phenomena.
Understanding How Electrons Spin
A study to determine how electrons spin has been pub-lished in the journal, Physica Status Solidi-B, Vol. 222,
p. 523 (2000). The study, “Dyakonov-Perel Effect on SpinDephasing in n-Type GaAs,” conducted by Cun-Zheng Ning,NAS Research Branch, and M. Wu, University of Californiaat Santa Barbara, explores the possibility of using electronspin coherence for application to faster, more energy-efficientinformation processing in the future.
Using a new quantum kinetic theory, Ning and Wu haveconducted a large-scale computer simulation to determinethe electron spin coherence lifetime in Gallium Arsenide(GaAs), a compound semiconductor material typically usedfor optoelectronics applications. The quantum kineticapproach and the results obtained will help in the under-standing of spin coherence for future information technolo-gies, and for quantum information processing and transport.A copy of the study is available online at: www.wiley-vch.de/contents/jc_2232/2222.html
Gridpoints Readers SurveyThe staff of Gridpoints would like your input about thepublication. A short survey has been posted on the publi-cation’s home page at:
http://www.nas.nasa.gov/GridpointsYour opinion matters and is greatly appreciated. —Eds.
4 Gridpoints
A tremendous amount of progress has been made inthe area of nanotechnology research at NASA Ames’Center for Nanotechnology since its inception in
early 1996. The nanotechnology effort began with a handfulof researchers from the NAS Division running computationson nanotubes, and has since grown into the robustCenter for Nanotechnology. Encompassingboth computational and experimental aspectsof nanotechnology, the center is directed byMeyya Meyyappan.
Progress on research at the Center for Nanotech-nology is peer-reviewed every two years – mostrecently on April 24 and 25. Ames Center DirectorHenry McDonald kicked off the April review ses-sion with a few words of encouragement, “I think itis important to have peer reviews. This grouphas made a significant impact, and I ampleased with the development in the areaof nanotechnology.”
Speaking before a panel of reviewersfrom industry and academia were mem-bers from Ames’ nanotechnology re-search and development, computa-tional nanotechnology, and compu-tational electronics and optoelec-tronics groups, which make up theCenter for Nanotechnology. Pre-sentation topics ranged from con-trolled growth of carbon nanotubesto computer codes for modeling elec-tron transport through carbon nan-otubes, and quantum computing.
NAS researchers Deepak Srivastava, Charles Bauschlicher,Toshishige Yamada (see “The Smallest Nanoelectronics:Atomic Devices with Precise Structures,” Gridpoints, Sum-mer 2000, page 10), and Cun-Zheng Ning (see “VCSELLasers: Tiny Lasers, Huge Potential,” Gridpoints, Spring2000, page 12) were among the scientists to report on theprogress of their work in computational nanotechnology.
Showcased research at the review included an update fromMeyyappan about the ongoing collaboration between Amesand the National Cancer Institute (NCI). The NASA/NCIcollaborators are developing a miniature probe, based on car-bon nanotube technology, for early detection of cancerouscells. Nanotube probe experiments have reached an in vitro
testing stage using biologicalsamples (see “NAS’sNanotube Technology Usedto Develop BiomedicalDevices,” Gridpoints, Spring2000, page 3).
At the conclusion of thereview session, the panel ofevaluators reported to AmesDeputy Director Bill Berry:“The quality of the work isexcellent. The energy andenthusiasm of the groupwas very visible which wasan indicator of how excitedthey are about their work.”In the past four years,researchers from the Centerfor Nanotechnology have
NASA Ames Center forNanotechnology Earns High Marks from Reviewers
A biannual peer-review of the Center for Nanotechnology was held last spring toprovide guidance to researchers while showcasing the work of Ames scientists.
Figure 1: A Buckytube probe (comprised of a pyridinemolecule attached to, or functionalized with a carbon nan-otube) serves as the tip of an atomic force microscope. Thepyridine molecule was selected as the probe moleculebecause of its ability to identify atoms, or data on the sur-face of a sample (in this case, diamond with some of itshydrogen surface atoms replaced with fluorene). Developedby Charles Bauschlicher and Ralph Merkle; visualizationby Al Globus.
www.nas.nasa.gov/gridpoints Gridpoints 5
published more than 170 papers and contributed more than350 seminar presentations and invited talks to publicize thework accomplished in the area of nanotechnology at Ames.
For additional information on the Center for Nanotechnol-ogy, visit: www.ipt.arc.nasa.gov
— Holly A. Amundson
Figure 2 (above): These imagesrepresent carbon nanotube junc-tions – the main figure is anacute angle y-junction, while theinset image is an obtuse angle y-junction. NAS researcher Deepak Srivastava and colleagues werethe first to propose the use of three-terminal carbon nanotubejunctions for nanoscale molecular electronic device applications.The switching behavior of the junctions is affected by structuralsymmetry – there is much less destructive interference betweenthe electronic wave function in the two branches of a symmetri-cal y-junction. The configurations of these model junctions weretested using the NAS facility’s Origin 2000 computers Stegerand Turing. These figures were generated using a visualizationprogram created by Srivastava and collaborator Madhu Menonfrom the University of Kentucky.
Figure 3a–e (right): In theory, nanotubes are capable of stor-ing hydrogen in a very dense form and could potentially beused to fuel rocket engines to Mars. The figures at right demon-strate hydrogen storage in carbon nanotubes with different con-figurations of hydrogen molecule placement. Green moleculesrepresent carbon nanotubes, white molecules represent hydrogencoverage on the nanotubes. There is an increase in bondstrength with the addition of hydrogen molecules, as well asgreater stability in structures with ordered hydrogen placement.The configuration in Figure 3a demonstrates the most stablestructure and the highest binding energy, due to the carefulplacement, or symmetry of hydrogen atoms in the carbon nan-otube. Research conducted by Charles Bauschlicher.
Scientists at the joint NASA/Stanford University Center for Turbulence Research areusing NAS computational resources to predict the radiated jet noise of aircraft enginesin an effort to reduce noise emissions.
In 1975, it was estimated that 7.5 million Americans livedin areas where aircraft noise exceeded 65 decibels – equalto having a TV set playing loudly in the background 24-
hours a day. The Airport Noise and Capacity Act of 1990 setdown what became known as “Stage 3” regulations thatrequired more than 7,500 jet airliners to be modified withquieter engines or to be retired from the fleet by December31, 1999. The direct result of Stage 3 regulations saw thenumber of people affected by jet noise reduced to about600,000 according to the Federal Aviation Administration.
Currently, the International Civil Aviation Organization, inconjunction with aviation regulatory agencies in the UnitedStates and the European Union, are working to address noiseand environmental issues from jet aircraft. The results ofthese meetings will establish noise reduc-tion and emission guidelines for futureStage 4 regulations.
Two sources of noise from jet engines canbe identified – one mechanical due to ro-tating components moving at highspeeds, the other due to aerodynamicaleffects. The latter is also called aeroacou-stic noise, which is the whistle that onehears when a high-pressure valve is open-ed, or the noise that the air makes whena car window is opened while moving athigh speed on the highway. While theaviation community successfully reducednoise emissions emanating from a jetengine’s fan and turbine section, reduc-tion of noise caused by the mechanical components has near-ly reached its limit. New methods to quiet an engine arebeing pursued actively by the jet engine manufacturers,NASA, and the academic community.
Scientists George S. Constantinescu and Sanjiva K. Lele ofthe Center for Turbulence Research (CTR) are continuingwork on a computational model for predicting jet exhaustbehavior using the NAS Facility’s high-end computing re-sources. Their goal is to reduce the intensity of acoustic noisesources in a jet’s exhaust that in turn will reduce engine noise.
Investigating Sound Attenuation Through Numerical Computations Before researchers can offer solutions for reducing the soundemanating from a jet’s exhaust, they must first understand themechanisms that are responsible for noise generation. To ac-complish this, researchers calculated the spatial and temporaldistribution of the acoustic sources, a task that is “practically
impossible to perform using experimentalmeasurements,” says Lele. These acousticsources emanate from the interactions ofthe turbulent eddies in the flow of the jet.
The flow of fluids is characterized by anon-dimensional number, called the Rey-nolds number, which is the ratio of theinertia forces over the viscous forces.When the velocity of a fluid flow is small,the viscous forces are large compared tothe inertia forces, thus the Reynoldsnumber is small. A fluid flow in thisregime can be observed to be regular andis characterized as laminar (or smooth).At high speeds, the Reynolds number islarge and the observed fluid flow consists
of eddying motions that are chaotic. The fluid flow in thisregime is called turbulent. As the Reynolds number increas-es, the range of the size of the eddies also increases. Two sim-ulation methods that are actively being pursued by NAS and
“The LES methodallows Constantinescuand Lele to accurately
capture the distributionof acoustic sources and
radiated sound waves inthe jet near field. Thesewaves are a direct resultof the sound emitted bythe acoustic sources in
the jet.”
www.nas.nasa.gov/gridpoints Gridpoints 7
CTR scientists are the Direct Numerical Simulation (DNS)technique and the Large Eddy Simulation (LES) technique.In DNS all the scales of motion are resolved, which limits thesimulations to moderate/low Reynolds numbers. Using LES,the energy containing large eddies are resolved and the effectsof small eddies on the large eddies are modeled, which enablehigh Reynolds number simulations.
To determine where and how exhaust noise is generated, theacoustic group at CTR, led by professors Lele and ParvizMoin of Stanford University, have used both LES and DNStechniques to simulate noise generation from turbulentflows. Constantinescu and Lele have been employing theLES technique to accurately simulate turbulent eddies in jetexhaust. Future use of DNS for high-speed jets will be ideal,but, in the foreseeable future, will be limited to moderateReynolds numbers.
Working with Moin and Lele in 1999, Jonathan B. Freund,currently a professor at the University of Illinois Urbana-Champaign, used DNS calculations to develop an under-standing of the sound generation in exhaust jets. In his sim-ulations, Freund was only able to simulate jet exhaust turbu-lence at a Reynolds number of 3,600 by using 25 milliongridpoints – a very expensive simulation. DNS simulationsat Reynolds numbers close to one million, which correspondto the actual operating conditions of jet engines, will be out
of reach for supercomputers in the foreseeable future. Con-stantinescu and Lele’s computational modeling may be seenas an extension of Freund’s work for high-speed (high Rey-nolds numbers) jets.
In Constantinescu and Lele’s computational model, theyomitted the jet engine’s exhaust nozzle. This was done pri-marily because modeling the nozzle would be too expensive.To compensate, the researchers imposed a velocity profile intothe calculations similar to that measured at the jet’s nozzle.
The Center for Turbulence Research (CTR) is a jointNASA/Stanford University center that sponsors thework of post-doctoral fellows studying turbulenceresearch. This includes studies of turbulence modeling,combustion modeling, turbulence control, aero-acoustics and, in general, research on flows where tur-bulence plays a dominant role. The center has a facili-ty at Stanford and one at Ames Research Center (ARC).Co-located in the same building at ARC are civil serv-ice scientists from the Physics Simulation and Model-ing Office of the NAS Division who collaborate withCTR post-doctoral fellows and visitors.
Center for Turbulence Research
Figure 1: Visualization of the turbulent jet at a Reynolds number of 72,000, and its radiated noise in the computational domain.Figure 1a: Instantaneous contours of vorticity magnitude, which are used to visualize the turbulent jet; Figure 1b: Instantaneouscontours of dilatation contours show how the sound radiates from the centerline away from the jet; and Figure 1c: Instantaneousdistribution of the Lighthill sound sources, which visualize the intensity of the sound sources. (Constantinescu)
Figure 1a Figure 1b Figure 1c
8 Gridpoints
In addition, most of the noise is produced at the end of theexhaust flow – away from the nozzle, where the exhaust tran-sitions from laminar, or smooth flow, into a turbulent flow.
“Because we are using a subgrid scale model we want to showthat the model’s contribution to the noise generation is verysmall and doesn’t contaminate the sound field,” Constan-tinescu says. “The purpose of doing these calculations is notto calculate the aerodynamic field. What we really want to
do is capture the sound that is emitted by a propulsive jet.In order to do that, we must employ a very accurate numer-ical method.
“Sound is emitted through very small pressure fluctuations,which are two orders of magnitude less than the pressurefluctuations associated with the jet turbulence. If the goal isonly to compute the average velocity field and the meanquantities that characterize turbulence in the jet, it can beaccomplished using a second-order method. But for aero-acoustic applications, we really need a very accurate method.In our case, we chose to employ a six-order accurate methodbased on Padé schemes in the streamwise and radial direc-tions, and Fast Fourier Transform methods in the azimuthaldirection,” Constantinescu explains. The higher fidelity ofthe six-order accurate method more efficiently represents thejet’s turbulent eddies with a minimum of artificial or numer-ical viscosity.
Solving The Singularity ProblemThe LES method allows Constantinescu and Lele to accu-rately capture the distribution of acoustic sources and radiat-ed sound waves in the jet near field. These waves are a directresult of the sound emitted by the acoustic sources in the jet.However, before successfully performing these simulations,an accurate treatment of the jet centerline had to be devel-oped. When the researchers attempted to perform LES athigh Reynolds numbers on relatively coarse meshes com-pared to DNS, they ran into convergence problems, whichcaused the code to diverge. The origin of the problem is adifficulty encountered by almost all numerical methods usedto solve the governing equations in cylindrical coordinates –the flow equations are discretized in cylindrical coordinates,which is the natural coordinate system chosen to simulateflows that are nearly axisymmetric (the jet axis, or the cen-terline, plays this role in the jet simulations). However, in thecylindrical coordinate system, the governing flow equationsare multi-valued at the centerline (known as the singularityproblem). A special mathematical treatment had to be usedon the centerline and in its vicinity.
Although Constantinescu and Lele tried to employ severalmethods previously proposed by other researchers to solvethe singularity problem, they were not entirely satisfied withthe results. “Many people have tried different approaches todeal with coordinate singularity, and what George and I havedone is develop a rather nice way of dealing with the coordi-nate singularity in the context of high-order numerical meth-ods,” Lele says. “We are dealing with a flow that has a nearlycircular symmetry, at least in an average sense. When a jetcomes out of a circular nozzle, it has that statistical symmetry,but the eddies in the instantaneous flow are three-dimensional.”
To resolve the singularity problem, the researchers developeda more accurate algorithm to solve the governing flow equa-tions at the centerline. In this method, a new set of equations,valid at the centerline, was derived using the most general
More Information About LESResearch papers describing the Large Eddy Simulationtechniques employed by Constantinescu and Lele can beviewed at the Center for Turbulence Research website:http://ctr.stanford.edu/publications.html
Figure 2a
Figure 2b
Figure 2: Dilatation contours in a forced laminar jet. In theoriginal method, Figure 2a, the equations at the centerlinewere solved in a Cartesian Coordinate system. In the presentmethod, Figure 2b, a new set of equations at the centerline wasobtained using series expansions. Calculations in Figure 2bmaintain the same order of accuracy of the solution at the cen-terline as in the rest of the computational domain.
(Constantinescu)
www.nas.nasa.gov/gridpoints Gridpoints 9
series expansions of the flow variables near the centerline.This maintains the same order of accuracy of the solution atthe centerline as in the rest of the computational domain.Running a parallel code on the NAS Facility’s Origin 2800computers, employing 32 processors, Constantinescu andLele can now run LES simulations at much higher Reynoldsnumbers (100,000) on coarser grids using only four milliongridpoints. This has enabled the scientists to more accuratelyestimate the amount of sound generated by high-speed engineexhaust jets.
Continuing ResearchUsing the data provided by Constantinescu and Lele’s simu-lations, engineers will be able to introduce disturbances witha specific frequency spectrum into the exhaust, thereby atten-uating the intensity of the noise. This will result in a signifi-cant reduction of the sound radiated by the jets. Finding
ways to force the jet to attenuate noise will be a challengingtask. Constantinescu predicts it will take at least two moreyears of research to arrive at a point where scientists can phys-ically control noise at Reynolds numbers corresponding toactual operating conditions. A parallel effort will be directedtoward developing LES subgrid models that are suitable fornoise calculations in high speed flows. These models shouldnot introduce spurious noise that can destroy the directlysimulated noise field from LES.
As computer modeling capabilities expand during the nextfew years, an additional goal of the researchers will be to sim-ulate the presence of the jet nozzle . This will allow a more com-plete understanding of the flow physics and enable research-ers to propose more efficient ways of controlling the noise.
—Nicholas A. Veronico
Sanjiva K. Lele is an associate professor with joint appoint-ments in the Departments of Aeronautics and Astronau-tics and Mechanical Engineering at Stanford University.He received his bachelor of science degree from the IndianInstitute of Technology, Kanpur, India,and his doctorate from Cornell Uni-versity, both in Mechanical Engineering.Lele worked at NASA Ames from 1986to 1990, then he joined the staff atStanford. He has received the FrancoisN. Frenkiel award from the AmericanPhysical Society, Division of FluidDynamics, and the National Science Foundation’s Presi-dential Young Investigator award. Last year he receivedbest paper awards from both the American Society ofMechanical Engineers and the American Institute of Aero-nautics and Astronautics.
About The ResearchersGeorge S. Constantinescu received his master of sciencedegree in environmental engineering at the Civil En-gineering Institute, Bucharest, Romania,and his doctorate from the IowaInstitute of Hydraulic Research (Uni-versity of Iowa). He served as a postdoc-toral associate in both the Departmentof Mechanical and Aerospace Engineer-ing at Arizona State University and atthe Center of Turbulence Research, Stan-ford University. Constantinescu has recently moved toStanford’s Center for Integrated Turbulence Simulationswhere he works on numerical simulations of gas turbineengines.
Figure 3: Sound power spectrum in the acoustic near field of the jet at a Reynolds number of 72,000. Figure 3a presents the dilata-tion power spectrum with the arrow indicating the frequency at which the spectrum peaks. Experimental measurements show thatthe peak in the power spectrum is obtained for a non-dimensional frequency (Strouhal number) of 0.2 to 0.4. Figure 3b shows thesame information represented in decibels. (Constantinescu)
Figure 3a Figure 3b
10 Gridpoints
Every time you fly on a plane, drive a car, or cross theborder into another country, you are required, by law,to carry a valid form of identification. Whether it’s a
driver’s license to drive, or a passport to cross the border,these forms of identification can only be obtained by follow-ing a predefined set of instructions. Like the traveler whomust fill out a passport application, users in the InformationPower Grid (IPG) community must ob-tain a valid certificate to use the grid’scomputer resources. Members of the IPGteam in the NAS Division have recentlyestablished their own trusted CertificateAuthority (CA), enabling users to obtaincertificates through a web browser or the“command line process,” a new and im-proved procedure for requesting, process-ing, and issuing IPG certificates.
“Getting an IPG certificate is now veryeasy, fast, and efficient. An IPG certificatescript has been modified to make the certificate process user-friendly and reliable,” explains IPG certificate authoritydeveloper Jana Nguyen of the NAS Division. The new cer-tificate request process takes less than 24 hours and requiresusers to follow only a few easy steps, thanks, in part, to anautomated e-mail notification system and a customizablecertificate authority management system (see Getting aCertificate Using the Command Line Process, page 11). Onceusers have an IPG account on the systems they plan to usefor their job, a public and private key (a pair of keys that areused to encrypt and decrypt a message), and a valid certifi-cate, they can then access the IPG.
Managing the ProcessTo help run an in-house certificate authority, the IPG teamchose to use Netscape’s Certificate Management System
(CMS). They adapted the CMS with additional scripts fromthe National Computational Science Alliance (NCSA),which enabled the team to customize the certificate requestand issuing process, and integrate it into the IPG environ-ment. The Netscape software, built to assist with the certifi-cate authority duties, is invisible to the user. “There is no realstandard for designing a certificate authority, but members of
the grid community are currently devel-oping a standard policy for the certificateitself that will apply to all grids,” explainsSumit Talwar, IPG security team lead.
Building BlocksThe first authority to issue certificates togrid users was the Globus CA at ArgonneNational Laboratory in Illinois. BecauseArgonne is a research organization, not aproduction facility for issuing grid cer-tificates, other grid organizations havebeen encouraged to design their own cer-
tificate authority processes. “Developing our own certificateauthority helps us build up our own in-house expertise in thesecurity area of public key infrastructure – which includes acertificate management system to support the public keyalgorithm. This is good for Argonne and for us – it is a win-win situation,” says NAS computer scientist, Y.K. Liu.NCSA and San Diego Supercomputing Center have also setup their own certificate authorities using the Netscape CMS.
Initially, the IPG team created a web browser interface,which would enable IPG users on all platform types – Unix,Mac, or PC, to request and receive certificates. The Globusinterface developed at Argonne did not offer a web browsermethod, making it more difficult for PC and Mac users.“With the IPG certificate authority, you can use any plat-form – that was the team’s goal,” says Talwar. “Granted, it is
Public Key Infrastructure:Get a Passport to Grid Country
Members of the Information Power Grid team recently established a system toissue certificates (passports) giving users access to grid computer resources.
“...Members of the gridcommunity are current-ly developing a standardpolicy for the certificateitself that will apply toall grids.”
— Sumit TalwarIPG security team lead
www.nas.nasa.gov/gridpoints Gridpoints 11
a little more difficult to go through the web browser than itis to go through a command line, but this way users runningon any platform type can request a certificate.” The IPG cer-tificate authority requires no additional software to request acertificate using the web method (only a web browser such as
Internet Explorer version 5.5 or higher, or Netscape version4.7 or later).
Although the web method is compatible with any platformtype, it requires the user to navigate through nearly 20 steps,
A simplified illustration of the public-key algorithm: When Bob wants to send an encrypted message to Alice, he will use Alice’s pub-lic key to encrypt the message. After Alice receives the encrypted message from Bob, she will use her private key to decrypt his mes-sage. Conversely, if Alice wants to send an encrypted message to Bob she will use his public key.
All users of a Public Key Infrastructure must have a regis-tered identity. These identities are stored in a digital formatknown as a public key certificate. Certificate Authorities(CAs) represent the people, processes, and tools to createdigital certificates that securely bind the names of users totheir public keys. Receiving a certificate from the Infor-mation Power Grid (IPG) CA is similar to being issued adriver’s license from the Department of Motor Vehicles. Toapply for a certificate, users must first be authenticated (theprocess of verifying a person’s registered identification) andhave an account on an IPG machine. Once users have metthe criteria, they can request a certificate from the IPG CAusing either the web browser method, or the new commandline process. Below is an outline of the steps users must fol-low to receive a certificate using the command line:
1. The user requests a certificate by executing a scriptthrough the command line on one of the IPG systems.
2. This script sends the user’s request to the CA system,which in turn generates an e-mail that is sent to the Certifi-cate Authority administrator, notifying him or her of therequest.
3. The CA administrator then decides whether or not toapprove the request based on a positive identification (au-thentication) of the user.
4. If approved, the Certificate Authority software issues acertificate that includes both the encrypted digital signatureof the CA and an assigned certificate identification number.“Signing” the certificate automatically generates and sendsan e-mail notifying the user that their certificate is availablefor retrieval.
5. The user then executes a second command line script,“ipg-cert-retrieve” with the assigned identification number.This script automatically stores the “signed” certificate andsends e-mail to the IPG staff at four NASA ResearchCenters (Ames, Glenn, Jet Propulsion Laboratory, andLangley) requesting them to make an entry in the grid-mapfile, which identifies them as a grid user.
For security measures, if a certificate has not been retrievedwithin a certain amount of time, the CA administrator willcheck with the user to see if they received the notification,or why they have not retrieved their certificate.
Getting a Certificate Using the Command Line Process
12 Gridpoints
whereas the new command line method is completed in justfive. “The original reason for going the web route was thatour team thought it would be easier for the user,” explainsGeorge Myers, NAS scientific consultant and portal devel-opment group lead. “Everyone has the idea that the webmakes everything easier. In practice, we quickly realized thatin this particular case, it was not true.”
Command Line FeaturesCustomized security features of the IPG CA include a scriptthat searches the NASA X.500 directory database (an inter-
national standard thatdefines the format tostore user’s informa-tion) to confirm a user’sidentification. “We tryto do as much securitywork in-house as possi-ble – we want to limitthe amount of securityoutsourcing,” explainsLiu. The team also de-veloped a script to runan automated e-mailnotification system be-tween the IPG certifi-cate authority, the Glo-
bus support staff, and the user. “The script streamlines thecertificate request and retrieval process,” says Liu.
For added security, certificates are only good for one year,and users must obtain a proxy certificate each time they usea grid resource (see “Infrastructure For NASA’s InformationPower Grid Nears Completion,” Gridpoints, Summer 2000,page 18). The researcher has the freedom to set the number
of hours the proxy certificate is valid for – the default beingeight hours. Each user’s IPG access session automaticallycloses at the end of the number of hours stated in the proxycertificate. “The proxy certificate is like a temporary pass orvisa – if your paperwork or certificate is not on file, you willbe denied access to the IPG resources,” explains Liu.
Certificates for Everyone In addition to users at Ames, NASA IPG partners at bothNASA Langley (Virginia), and NASA Glenn (Ohio)Research Centers have successfully obtained certificatesthrough the IPG Certificate Authority. Eventually, otherNASA centers, grid organizations, and universities could beauthorized to obtain certificates from the IPG CertificateAuthority as well.
Since the end of February, about 50 users have followed thecommand line procedures to obtain their IPG certificate.According to one user, Scott D. Thomas, a scientific pro-grammer at Ames, “It was very easy to get the certificateinformation into the files through the command line. Thisprocess was done automatically by script – much easier thanthrough the web browser which requires you to manuallymanipulate files.”
The team will continue to make software enhancements andadd more features to increase robustness of the IPG CA.Says Talwar, “We encourage users to go through the IPGwebsite or command line interfaces to get a passport (cer-tificate) for accessing valuable IPG resources to facilitatetheir research.”
— Holly A. Amundson
GlobusSystemGlobusSystem
IPGCertificate
Scripts
IPGCertificate
Scripts
NetscapeCertificate
System
X.500DirectoryDatabase
NetscapeCertificate
System
X.500DirectoryDatabase
Right: The integration of IPG certificate scripts with threeimportant systems (X.500 Directory Database, NetscapeCertificate System, and Globus System) makes it easy for theuser to request and retrieve a certificate. The scripts retrieve theuser’s information from the NASA X.500 directory database,connect to the Netscape Certificate System to request a certificateand, once the certificate request has been approved, sets up thecertificate and the private key in the Globus system.
For additional information on NASA’s Information PowerGrid Certificate Authority, visit the “New User’s” sectionon the website at: www.ipg.nasa.gov/
Information on obtaining an account can be found at:www.ipg.nasa.gov/usersupport/newusers/accounts.htm
Netscape offers a website detailing its Certificate Manage-ment System: http://home.netscape.com/cms/v4.0
Certificate Authority Resources
“It was very easy to get thecertificate information intothe files through the com-mand line. This processwas done automatically byscript – much easier thanthrough the web browserwhich requires you to man-ually manipulate files.”
— Scott D. ThomasNASA Ames programer
www.nas.nasa.gov/gridpoints Gridpoints 13
NAS Technical Training SeminarsA number of scientists have presented their research to
audiences at the NAS Facility during the first half of2001. Many of this year’s presentations have been
videotaped, and can be borrowed by sending e-mail to theNAS Documentation Center ([email protected]). Fiveof these seminars, all of which were videotaped, are high-lighted below:
• Terry Holst and Tom Pulliam of the NAS Division’sApplications Branch presented “Genetic Algorithms Appliedto Aerodynamic Shape Optimization” on May 15. They dis-cussed a method for aerodynamic shape optimization usinga genetic algorithm with real number encoding. This algo-rithm was demonstrated on three different problems: a sim-ple hill-climbing problem; a two-dimensional airfoil prob-lem using an Euler/ Navier-Stokes equation solver; and athree-dimensional trans-sonic wing problem using a nonlin-ear potential solver. Holst and Pulliam noted that althoughthe emphasis is on aerodynamic shape optimization, thegenetic algorithm presented isquite flexible and could beapplied with minimal imple-mentation effort to many single-or multi-objective problems inother fields, providing themeans of evaluating the objec-tive function exists.
• Stanford University’s CharlesPierce discussed his research onthe “Large Eddy Simulation of aCoaxial Jet Combustor” onMarch 29. Pierce reviewed thelarge eddy simulations that wereconducted to test the perform-ance of a new chemistry modelin a methane-fueled coaxial jetcombustor for which experi-mental data is available. Pierce’schemistry model utilizes trans-port equations for a mixture fraction variable, which tracksthe mixing of fuel and oxidizer, and a progress variable thattracks the degree-of-reaction at each point in the flowfield.
• Frank Werblin, Department of Molecular and Cell Biologyat the University of California, Berkeley, presented “A VisualLanguage of the Retina: A Dozen Simultaneous Movies” onMarch 27. Werblin discussed how the retina generates a par-allel stack of at least a dozen different representations of thevisual world, and how these representations, surprisinglysparse in both space and time – differ in their space-time fil-tering properties. They are formed through an intricate inter-
action between numerous excitatory and inhibitory patterns,and each selects for a different set of visual features.
• San Jose State University’s James Wayman spoke on “Bio-metric Identification: Teaching Computers to RecognizePeople” on March 20. Biometric identification is the auto-matic identification or identity verification of living, humanindividuals based on behavioral and physiological character-istics. Technologies include fingerprint, hand, and voicerecognition, as well as face and iris recognition. Customersfor this technology have ranged from Disney World to theSuper Bowl. Wayman discussed a scientific approach to ana-lyzing applications and matching them to technologies,along with algorithms and mathematical models for predict-ing system performance based on experimental estimationsof model parameters.
• On March 15, Suhrit Dey, from Eastern Illinois Universityand recently a visiting scientist at Ames, presented “Hemo-
dynamics of T-cells with Appli-cations to Breast Cancer.” Dey dis-cussed the role of white blood cellsin fighting cancerous cells, notingthat they are contained in less thanone percent of blood’s volume. T-cells originate from bone marrow,circulate through blood to the thy-mus (a lymphoid organ locatedinside the chest cavity above thebreast and in the space betweenthe lungs), and exit the thymus as“fighter” cells. Dey is studying thehemodynamics of breast cancer,and has found that a strong circu-lation of T-cells should help pre-vent its occurrence. In his talk,Dey discussed some things womencan do to stimulate the thymus,increase the circulation of T-cells,and lessen the chance of contract-
ing breast cancer. The next phase of Dey’s research willinvolve studying the biomechanics of T-cells by applyingcomputational models to explain surveillance, mobilization,and defense strategy of these lymphocytes.
For information on these and other NAS technical trainingevents, contact Marcia Redmond at: [email protected]. For information on future training sessions, see theGridpoints Calendar of Events (page 21), or visit the NASTechnical Training Event Web site: www.nas.nasa.gov/User/Training/training.html
A visualization from Suhrit Dey's research (see March15) shows a large, cancerous white blood cell being over-powered by relatively small (purple) T-cells.
(Visualization by Cliff Williams)
14 Gridpoints
Since the outset of its development in fall 1998, NASA’sInformation Power Grid (IPG) has been designed toenable interdisciplinary collaborations. The IPG’s dis-
tributed network of scientific instruments, data archives, andcomputational resources was recently applied to the area ofaviation safety. With the help and collaboration of NAS Div-ision IPG team members, the Aerospace Extranet group inthe Computational Sciences Division at Ames Research Cen-ter employed grid resources to simulate commercial air traf-fic flow across the United States.
“The Aerospace Extranet Program demo was a major successfor the IPG – it demonstrates the true capabilities of thegrid,” says Johnny Chang, NAS scientificconsultant. By helping the AerospaceExtranet group, the IPG team gainedvaluable insights about the process ofintegrating new grid users. “We learned agood deal about the requirements IPGusers are likely to need as a result of thisdemonstration,” says Judith Utley, NASGlobus administrator. “We will now beable to respond faster and more smoothlyto new IPG users and bring in new sys-tems as well as more NASA centers.”
On April 9 and 12, the Aerospace Extra-net group (led by David Maluf, withWilliam J. McDermott, principal investigator ) used the IPGto demonstrate near-real-time, distributed modeling andsimulation capabilities of the National Airspace SimulationSystem. The demonstrations were accomplished using a Sun
Ultra Enterprise 450 computer, known as Washington, locat-ed within the Computational Sciences Division. Washingtonwas connected to the IPG as a fully integrated client, allow-ing the Aerospace Extranet team to use Washington, as well asthree IPG resources, Evelyn and Turing (both SGI Origin2000s) at the NAS Facility, and Sharp (also an Origin 2000)at NASA Glenn Research Center, Cleveland, Ohio.
To enable the Aerospace Extranet group’s application to sendand receive data between Washington and the IPG, theGlobus toolkit (software designed to enable different types ofgrid resources to interface with each other) was installed.Globus administrators Judith Utley and Mary Hultquist
assisted Matt Linton (system administratorfor Washington) with the installation, config-uration, and testing of the Globus softwarepackage. Using Java applications, the Aero-space Extranet group created their own webinterface on Washington to initiate Globuscommands. Globus enables the team tointeract with their simulation as it runs onthe IPG resources.
To obtain user access to the grid, the IPGcertificate authority software (see Getting aPassport to Grid Country, page 10) was alsoinstalled on Washington. Researchers reliedheavily on the security of the IPG because of
the sensitive nature of the simulation data. The group’s mas-sive datasets required resources that could simultaneouslyrun multiple jobs using a high number of computationalcycles – exactly what the IPG was built to provide.
IPG Supports NationalAirspace Simulation System Demonstrations
In a recent collaboration between the NASA Ames’ Computational Sciences Divisionand members of the IPG team in the NAS Division, grid resources were employed tosupport simulations of commercial air traffic crossing the United States.
“Once all installationsand modificationswere complete, a
seamless environmentwas provided betweenWashington, IPG sys-tems at NAS, and an
IPG system at Glenn.”— Judith Utley
Globus Administrator
www.nas.nasa.gov/gridpoints Gridpoints 15
During the demonstration of the prototype National Air-space Simulation System, jobs were submitted from Wash-ington, where the database files originated, and distributed toEvelyn, Turing, and Sharp. The output files, or engine simu-lation parameters, were generated using the Numerical Pro-pulsion System Simulation (NPSS) software installed onSharp. The engine parameters were then sent back to Wash-ington where users had the ability to select, manipulate, and
graph the output. The data used for the simulations wastaken from radar tracks of flights arriving and departing fromAtlanta, Georgia’s Hartsfield International Airport.
“Once all installations and modifications were complete, aseamless environment was provided between Washington,IPG systems at NAS, and an IPG system at Glenn,” explainsUtley. “The demos went off without a hitch,” adds Chang.Additional IPG team members involved with assisting theAerospace Extranet group with the demonstrations included:IPG Project Manager Bill Johnston, Deployment and Inte-gration Manager Leigh Ann Tanner, and NAS Scientific Con-sultants Terry Nelson and Chuck Niggley.
—Holly A. Amundson
Editor’s note: The Aerospace Extranet group’s simulations arepart of the NASA-funded Information Technologies Base Pro-gram, and are conducted on behalf of the National AirspaceSimulation System to ensure safe and efficient movement of air-craft through the nation’s airspace.
Additional information about NASA’s InformationPower Grid (IPG) can be found at: www.ipg.nasa.gov
The Computational Sciences Division at NASA AmesResearch Center is on the web at:
http://infotech.arc.nasa.gov:80/compsci.html
Details about the National Airspace System are avail-able at: www.faa.gov/education/resource/National%20Airspace%20System%20(NAS).htm
Learn More About the IPG
16 Gridpoints
T he enormous computational power of the Infor-mation Power Grid (IPG – a distributed network ofscientific instruments, data archives, and computa-
tional resources) is enabling scientists in the NASA AdvancedSupercomputing (NAS) Division to carry out flow simula-tions that were previously unattainable. The dataset size for atypical simulation consists of hundreds of gigabytes, which,if printed out, would create a stack of paper more than oneand one-half miles high. The sheer scale of these datasets callsfor a new breed of visualization tools that will allow the userto interact with the data in real-time. Commercial off-the-shelf software has difficulty performing real-time interactionwith datasets of this size. This situation is further exacerbat-ed by the need to visualize dozens, or even hundreds, of data-sets. However, work being done by the NAS ResearchBranch’s data analysis group specifically targets tool devel-opment that enables high levels of user interaction with verylarge datasets.
NAS Applications Branch scientists Neal Chaderjian, JasimAhmad, Shishir Pandya, and Scott Murman serve as a recentexample of the need for visualization tools that enable usersto interactively explore very large, unsteady datasets. Theresearchers computed 45 time-accurate Navier-Stokes flowsimulations of a YAV-8B Harrier jet, which is capable of ver-tical take-off and landing, hovering above a tarmac with a33-knot headwind (see figures 1a and 1b, page 17). This Ad-vanced Technology Application (ATA) is part of the Com-putational Aero Sciences High Performance Computing andCommunications Program.
Computational Visualization For Mission SafetyThe purpose of the Harrier flow simulations is to demon-strate a technology to generate a stability and control data-
base for an aircraft with complex flow physics, while reduc-ing computational costs. Controlling an aircraft hoveringnear the ground can be a tricky proposition, as a fatal January2001 crash of a Harrier demonstrates.
During take-off and landing, the Harrier’s thrust vectoringnozzles are directed toward the ground to provide lift need-ed to support the vehicle in low-speed flight. The vectorednozzle exhaust impacts the ground and interacts with theambient flow to form a ground vortex. A jet fountain canalso form when the jet flow impacting the ground turns upto strike the underside of the fuselage. The overall concernis that these flow features can cause vehicle safety and con-trol problems. For example, hot gas ingestion by the inletmay cause the Harrier to crash due to a rapid loss of enginethrust, which in turn causes a loss of lift. Rapid flow accel-erations on the underside of the vehicle can also cause a“suck-down effect” where the vehicle is pulled toward theground. In addition, there is also a concern with ingestionof ground debris, and for the safety of ground personnel inthe vicinity of the flying aircraft. Visualization of the flowfield is crucial to understanding how the jet exhaust flow be-haves, and for improving the safety of flight operations atlow altitudes. Sixteen hundred files consisting of more than100 Gigabytes of data were used to visualize the unsteadyflow for a single Harrier simulation hovering at a height of30 feet above the tarmac.
Using both Gel and Batchvis, software developed by the dataanalysis group, NAS researchers were able to discern flow fea-tures not readily attainable with traditional software packagessuch as FAST and Plot3D (see sidebar: How a New Algo-rithm Powers Unsteady Flow Visualization, page 18). The newsoftware enabled the team to choose locations to release vir-tual smoke particles into the flow by grabbing “rakes” anddragging them around with the mouse. Gel then tracked the
The Power of Unsteady Flow VisualizationNew algorithms have enabled NAS data analysis group researchers tovisualize extremely large datasets using personal workstations, cuttingcomputation time and costs while increasing resolution.
By Timothy A. Sandstrom and Neal M. Chaderjian
www.nas.nasa.gov/gridpoints Gridpoints 17
locations of thousands of particles as theymoved about the aircraft. A vortex core detec-tion algorithm was also used as a feature detec-tion tool to automatically extract the locationof vortical flows. A side view of the Harrier’sjet exhausts interacting with the ambient flowto form a ground vortex is shown in Figure 2.The ground vortex is unsteady, periodicallychanging its size and position with time. Thejet exhausts also oscillate back and forth asthey interact with the time-varying groundvortex. The frequency of this oscillation wasfound to correspond precisely with the varia-tion of lift with time. Thus, a direct correla-tion between the time-varying lift and un-steady ground vortex was established.
Figure 3 depicts an oblique view, where thevirtual flow particles are rendered as smallspheres rather than screen pixels to highlightthe ground vortex. Notice that a fountain vortex
Figure 1a: The U.S. Marine Corps operates the Harrier fighterjet, capable of vertical take-off and landing. Computationalmodels were used to visualize the air flow about the jet whilehovering. (Boeing)
Figure 1b: A Harrier grid system was used to compute 45 time-accurate Navier-Stokes flow simulations of a YAV-8B Harrierhovering above a tarmac with a 33-knot headwind.
(Timothy A. Sandstrom)
Figure 2 (left) and Figure 3 (bottom left):Using Gel and Batchvis software, NAS dataanalysis group researchers are able to visualizethe ground vortex and jet exhaust of the Harrierfrom different angles (Figure 2, side view, andFigure 3, oblique view). The jet’s exhaust canform a fountain that strikes the underside of theaircraft. Hot gasses from the jet fountain, ifingested, can cause a rapid loss of engine thrust.The streaklines depicted in these figures are col-ored by temperature (blue depicts cool, red is hot).
(Timothy A. Sandstrom)
Continued on page 20
Figure 2
Figure 3
18 Gridpoints
The 100 gigabyte Harrier simulation is an example ofthe large datasets that can be produced by the 512-and 1,024-processor supercomputers at the NAS
Facility. Once the datasets have been computed, they are ana-lyzed using visualization software such as Gel and Batchvisrunning on a researcher’s personal workstation. These data-sets, however, are difficult to visualize on workstations withtraditional visualization applications (such as FAST andPlot3D) because they assume that the entire dataset (or atleast a pair of time steps for time-varying data) can be loadedinto main memory. Unfortunately, many personal worksta-tions do not have sufficient memory capacity to handle thisworkload. In addition, most workstations do not have suffi-cient disk space to hold large datasets, which means the in-formation must be stored on a remote file server.
During the past four years, researchers in the NAS dataanalysis group have been developing new algorithms thatenable the visualization of large datasets on personal work-stations. These algorithms are called “out-of-core” visualiza-tion techniques because they leave most of the data on diskinstead of loading all of it into main memory (previouslycalled “core”). Since many visualizations only examine asmall fraction of the dataset at any one time, the currentlyneeded data is often small enough to fit into a workstation’smain memory. Reducing the amount of data required isimportant because it allows the visualization to be computedat the workstation’s maximum performance (see chart atright). If the data did not fit in main memory, it would haveto be repeatedly retrieved from the disk drives, which aremany thousands of times slower. Figures A and B (page 19)show how out-of-core algorithms reduce the data that mustbe read.
Remote visualization, where data is visualized from a remoteserver, allows numerous researchers to share a dataset withoutmaking multiple copies of it. It also allows data to be visual-ized directly from the machine where it was computed,which makes it much easier to verify that the computationwas completed correctly. Finally, remote visualization is prac-tical because Gigabit Ethernet is fast enough to not be abottleneck, and is now inexpensive enough to be deployed tothe desktop.
The original out-of-core visualization application developedat NAS did not perform both the computation and diskaccess at the same time. Instead, when the program detectedthat data must be read, the calculation was stopped while thedata was read from disk. If the dataset was read from disk ona remote server, the computation was further delayed whilethe data was transferred over the network. The data analysisgroup’s new algorithm simultaneously overlaps the computa-tion, disk access, and network transfer functions. This is ac-complished by dividing the visualization into a number oftasks – when one task must wait for data to be retrieved fromdisk, a scheduling algorithm runs another task to keep theprocessor busy. Data retrieval from a local or remote disk
How a New Algorithm PowersUnsteady Flow Visualization
By David Ellsworth
Animation on the webImages and animation from the NAS data analysis group’sresearch can be downloaded from the web at:
www.nas.nasa.gov/Groups/VisTechClick on the Images and Movies button for a selectionof more than 50 aerospace, biological, physical, and earthand space science visualizations.
When compared to the existing Network File System (NFS), thenew “out-of-core” algorithm dramatically improves the timeneeded to visualize the 100-gigabyte Harrier simulation.
(David Ellsworth)
Remote Out-of-Core Performance
www.nas.nasa.gov/gridpoints Gridpoints 19
proceeds independently while the requesting task waits. Thenew algorithm is general enough to support a variety of visu-alization techniques, which must be modified so that thework can be divided into a number of smaller tasks. How-ever, such modifications are similar to those made to enablea visualization to run in parallel.
The chart on page 18 depicts the improved performance ofthe new out-of-core algorithm when computing a 1,600-frame animation of the Harrier dataset using parameters sim-ilar to the ones used in Figure 2 (see page 17). The datasetwas read from a remote system over an 800-megabits-per sec-ond HIPPI (HIgh Performance Parallel Interface) networkconnection, and used SGI Onyx workstations for both thelocal and remote systems. The chart compares the perform-ance results using the standard Network File System protocolto retrieve the data versus performance using the new algo-rithm. When using one processor, the time decreased from207 minutes to 146 minutes, or 30 percent. The improvementin performance is attributed to the increased overlap of com-putation, disk access, and network data transfer. When fourlocal processors were used, compute time decreased by morethan half, from 159 to 77 minutes, demonstrating that thealgorithm can make use of multiple processors. The compu-tation time did not decrease by a factor of four when thenumber of processors was increased because the disk and net-work speed were the same for all visualization runs.
The new out-of-core algorithm will make it easier for re-searchers to visualize the output of simulations run on IPGsystems. In the future, NAS researchers will test the per-formance of the algorithm using a wide area network toaccess data from remote IPG systems.
Editor’s note: Michael Cox, formerly of the NAS Division,began work on the out-of-core algorithm in 1997. In theensuing four years, the data analysis group has worked todevelop the algorithm into the robust tool that it is today.The author and Tim Sandstrom wrote a number of the appli-cations that use out-of-core technology, while Pat Moran andChris Henze worked on the Field Encapsulation Library,which provided the framework for the current out-of-coreimplementation. For more details about the new algorithm,see: www.nas.nasa.gov/Research/Reports/Techreports/2001/nas-01-004-abstract.html
David Ellsworth is a research scientistfor Advanced Management Technology,Inc. and is a member of the NAS dataanalysis group. He received a doctoratein Computer Science from the Univer-sity of North Carolina at Chapel Hill,and has been researching methods forvisualizing very large data sets sincejoining NAS in 1997.
Figure A visualizes the airflow around theexternal tank of the Space Shuttle launchvehicle (data courtesy Ray Gomez, NASAJohnson Space Center). Figure B displaysthe outlines of the solution data blocks (inwhite) that were read by the out-of-corealgorithm to compute the visualization.Since the out-of-core library only reads thedata needed by the streamline calculation,only eight percent of the 573-megabytedataset was read in this case. Note that thedata blocks are only around and aft of theexternal tank. This demonstrates that theblocks around the orbiter and solid rocketboosters, as well as those blocks furtheraway, were not read. (David Ellsworth)
Figure A Figure B
20 Gridpoints
forms in front of the primary ground vortex and isthen swallowed up into the primary vortex.
When the Harrier descends to within 10 feet above thetarmac, the fountain vortex is ingested into the inlet(see Figure 4.) This results in the hot gas ingestion phe-nomenon that leads to a loss of thrust and subsequent-ly lift, which in turn may cause the Harrier to crash.
In Figure 5, a vortical flow structure on the horizontaltail, as viewed from behind the aircraft, is displayed.This unusual vortex system is the result of very low-speed ambient flow being entrained by a complexinteraction between the jet fountain coming up fromthe underside of the fuselage, and a wing flap vortexthat convects back over the top of the horizontal tail.This vortex system would not occur under normalhigh-speed flight conditions.
Expanding Visualization Tools For the FutureInteractive unsteady flow images help scientists andengineers better understand what is happening to theHarrier when close to the ground, and why. Research-ers are tempted to use instantaneous streamline toolsbecause this method only requires data at one time step(rather than 1,600 in the present case). However, theresulting static images do not properly capture thedynamic behavior of the unsteady flow, and can givemisleading or even wrong impressions of what is actu-ally happening.
NAS data analysis group researchers are working todevelop emerging interactive tools, such as Gel, to pro-vide the engineer and scientist with a powerful labora-tory to explore flow features and improve the safety offlight operations as well as the working environment ofground personnel.
Timothy A. Sandstrom is a sen-ior systems analyst for AdvancedManagement Technology, Inc.and a member of the NAS dataanalysis group. He has beenwriting visualization softwarefor 10 years and was a keymember of the team that devel-oped the Flow Analysis SoftwareToolkit (FAST), which wonNASA’s Software of the Yearaward in 1995.
Figure 4: The fountain vortex has been isolated in this view to showhow it travels up from the tarmac into the engine inlet (spheres col-ored by temperature). (Timothy A. Sandstrom)
Continued from page 17
Figure 5: Tail view of the Harrier showing a vortex (white lines)impacting the horizontal tail and causing an upward swirling flow(particles colored by temperature). This vortex is formed through acomplex interaction between the jet fountain and wing flap vortex.
(Timothy A. Sandstrom)
Neal M. Chaderjian is a research scientist inthe NAS Applications Branch, and grouplead for the Computational AerosciencesPowered-Lift Advanced Technology Appli-cation project. He joined NASA after receiv-ing his doctorate from Stanford University.He is currently interested in integrating CFDand IT technologies to enable first-of-a-kindunsteady flow simulations that impact NASAprograms. He has received NASA and DODawards, and is an Associate Fellow of the Amer-ican Institute of Aeronautics and Astronautics.
www.nas.nasa.gov/gridpoints Gridpoints 21
Calendar ofEvents
10th IEEE International Symposium on HighPerformance Distributed ComputingSan Francisco, California • August 7–9The 10th International Symposium on High PerformanceDistributed Computing is a forum for presenting the latestresearch findings on the use of networked systems for high-performance computing. All aspects of high-performancedistributed computing will be presented, including, visuali-zation, collaboration, hardware technologies, network pro-tocols, the middleware that ties distributed resources togeth-er into “computational, data, and collaboration grids,” mid-dleware that enables application use of grids, and tools andlanguages that support application development. Details areavailable at: www-itg.lbl.gov/HPDC-10
14th International Conference on Parallel and Distributed Computing SystemsDallas, Texas • August 8–10The 2001 International Conference on Parallel and Distri-buted Computing Systems is a forum for the exchange ofknowledge and experience among researchers, engineers,and practitioners working with parallel architectures andsystems, as well as distributed computing and informationsystems. On the web, visit: www.isca-hq.org/confr.htm
SIGGRAPH 2001Los Angeles, California • August 12–17SIGGRAPH 2001 showcases applications of computergraphics and interactive techniques developed by computergraphics scientists, artists, engineers, and educators. Foradditional information, visit: www.siggraph.org/s2001
2001 NATO Research and Technology Office –Short Course: Error Estimation and SolutionAdaptive Discretization in CFDNASA Ames Research Center • September 10–14The NATO Research and Technology Office will host afour-day workshop on Error Estimation and SolutionAdaptive Discretization in CFD at the NAS Facility.Discussions will cover: Delaunay triangulation, computa-tional geometry; Introduction to A-posteriori error estima-tion, Giles/Pierce theory; Adaptive Finite Element Methodsfor fluid flow, model adaptivity; Implicit A-posteriori com-putation of bounds, “Energy” norms and outputs of inter-est; and A-posteriori error estimation, stability and errorcontrol. Visit www.nas.nasa.gov/User/Training/training.html for registration information.
Ninth Foresight Conference on Molecular NanotechnologySanta Clara, California • November 9–11The Ninth Foresight Conference on Molecular Nanotech-nology will review the advances in nanotechnology andcover topics such as molecular electronics and machines,scanning probes, self-assembly, nano-materials and struc-tures, as well as nanoelectronics and nanodevices. Jie Han ofthe NAS Division’s computational nanotechnology group isone of the featured speakers. Visit: www.foresight.org
SC2001Denver, Colorado • November 10–16 SC2001 will bring together scientists, engineers, designers,and managers from all areas of high-performance network-ing and computing, and showcase the latest in systems,applications, and services. The conference website is:www.sc2001.org
2001 Information Power Grid WorkshopNASA Ames Research Center • December 5–6The NASA Information Power Grid Workshop will be heldat the NAS Facility on December 5 and 6. Planned topicsinclude: IPG infrastructure and deployment; grid interoper-ability middleware; IPG security infrastructure; grid utiliza-tion tools; data access and grid computing; grid user inter-faces; IPG applications and tools; as well as other relatedtopics of interest to grid computing. For the latest informa-tion, visit: www.ipg.nasa.gov or NAS’s training site:www.nas.nasa.gov/User/Training/training.html
This visualization reveals the arrangement of electrons in a mol-ecule of vitamin B12. The vitamin is used by many enzymes asa cofactor in reactions involving structural rearrangements ofvarious molecules – in other words, the enzyme-vitamin com-plex serves as a kind of molecular reassembler or "nanomanipu-lator.” This important functionality of vitamin B12 relies on theelectronic behavior of a metallic cobalt atom in its core. To see a3-D animation, visit the NAS website’s video collection at:www.nas.nasa.gov/About/Media/videos.html
