The Quarterly Publication of the NASA Advanced ...The Quarterly Publication of the NASA Advanced...

The Quarterly Publication of the NASA Advanced Supercomputing Division

NAS researchers have filed a patent for usingcoupled lasers to achieve ultrafast modulationand switching in optical networks, one of the

many accomplishments highlighted in the division’s year-end review. See page 8

Parallelization – The Key to Faster Codes,Higher Fidelity Simulations – 18

Winter 2003

www.nas.nasa.gov

Completing Large Parameter Studies with Time to Spare – 4

News From NAS – 1

Vol. 4, No. 1 • Winter 2003

Executive Editor: John Ziebarth

Editor: Nicholas A. Veronico

Writers: Holly A. AmundsonJill A. DunbarJulie Jervis

Graphic Designer: Sue Kim

Illustrator: Cliff Williams

Photographer: Michael Boswell

Contact the Editor:Nicholas A. Veronico

GridpointsM/S 258-6

Moffett Field, CA 94035-1000Tel. (650) 604-3046 • Fax (650) 604-4377

[email protected]

Gridpoints is a quarterly publication of the NASAAdvanced Supercomputing Division at NASAAmes Research Center. The division’s staff includesgovernment employees and contractors fromAdvanced Management Technology Inc., Compu-ter Sciences Corp., Cray Inc., Eloret, Raytheon Co.,and SGI.Gridpoints acknowledges that some words, modelnames and designations mentioned herein are theproperty of the trademark holder. Gridpoints usesthem for identification purposes only.

The staff of Gridpoints would like to thankthe following people for their assistance inpreparing this publication: Jan Aikins, Michael Aftosmis, Dick Anderson,Robin Amundson, M.P. Anantram, BryanBiegel, Chris Buchanan, Galina Chaban,Christopher Dateo, Suhrit Dey, HeinzErzberger, Graham Fletcher, Robert Frampton,Chris Gong, Peter Goorjian, Chris Henze,Tom Hinke, Winifred Huo, Dochan Kwak,Henry Jin, Gabriele Jost, Randy Kaemmerer,John Kitowski, Timothy Lee, Tony Lisotta,Nateri Madavan, Nagi Mansour, Cun-ZhengNing, James Ramage, Marcia Redmond,Michael Rogers, Stuart Rogers, Subash Saini,Tim Sandstrom, David Schwenke, Chung-LinShie, Deepak Srivastava, Astrid Terlep,William Thigpen, Joseph Totah, DunyouWang, Charles Wiseman, Alan Wray, JerryYan, Maurice Yarrow

On The Cover:

Subscribe to GridpointsSubscriptions to Gridpoints are complimentary.Please send e-mail with your name, companyname, title, address, and zip code to:

[email protected] subscribe online at:

www.nas.nasa.gov/gridpointsNote: We do not mail outside the United States.Foreign subscribers can access Gridpoints on the

web in PDF format at:http://www.nas.nasa.gov/gridpoints


4

8

Features

NAS Division in 2002:A Trip Down Shared-Memory LaneThe NAS Division looks back on a suc-cessful year of research in diverse areasand recognition for outstanding scientif-ic achievement.Julie Jervis

NAS scientists have proven the effectiveness of using coupled lasers to achieve ultrafastmodulation and switching in optical networks. Their method uses two coupled vertical-cavity surface-emitting lasers (VCSEL), each with a diameter of about ten microns, one-tenth the thickness of a human hair. (NASA/Cun-Zheng Ning)

18

Also in this issue:1 News From NAS1 From the Division Chief

21 Calendar

Completing LargeParameter Studieswith Time to SpareNAS Division researchers createan automated CFD system to run large parameter studies onIPG resources. Holly A. Amundson

Parallelization – The Keyto Faster Codes, HigherFidelity SimulationsNAS Division researchers create a computer-aided tool to help automate the tedious processof parallelizing serial code. Holly A. Amundson

This past year has been an exciting time in the NASDivision. Our team made tremendous strides in com-putational science and computer technology in sup-

port of NASA missions, science, and engineering milestones.Researchers within our division received both the NASASoftware of the Year award and the NASA Commercial Invention of the YearAward, and a number of patents were issued (see page 8). In addition, research-ers in our Applications Branch received the first annual Ames Research Center In-formation Sciences and Technology Directorate “Technology Infusion Award.”

Our computational scientists have demonstrated the highest performance onany U.S. computer architecture – by any vendor – on the 1,024 processor SGIOrigin 3800 system, Chapman, for Earth science and aerospace codes. Thisillustrates that when the goal is high-performance, or what is referred to as“capability computing,” the common shared-memory architecture is a tremen-dous tool for engineers and and the scientists using those architectures.

In November, NAS scientists showcased the division’s latest supercomputingresearch and technologies at the SC2002 Conference of High PerformanceNetworking and Computing in Baltimore. The division’s new “hyperwall,” ascientific visualization system, attracted large crowds (see Gridpoints, Fall 2002,page 11). Also in the NASA exhibit were researchers from the University ofUtah who teamed with NAS scientists to demonstrate state-of-the-art remote,interactive ray-tracing. Computations were performed using more than 1,000of Chapman’s processors, and the images were displayed on the show floor inBaltimore. Ray-tracing is a powerful but extremely compute-intensive visuali-zation technique. Due to excellent performance scaling of the ray-tracing codeon Chapman, the Utah researchers were able to produce and display interactive(that is, multiple frames per second) visualizations of multi-gigabyte datasets,including an eight-gigabyte high-resolution scan of a human male thorax (datafrom the Visible Human Project, National Library of Medicine), and a 64-giga-byte terrain dataset, with a resolution of more than 17 billion polygons.

One of the biggest challenges of this demonstration was overcoming the inher-ent problems associated with TCP/IP when moving large amounts of real-timedata across the country. Working closely with the Utah scientists, NAS networkengineers tuned a variety of parameters at the kernel, system interface, and net-work layers to maximize throughput and minimize delay and delay variance(jitter). Once tuned, throughput was increased six-fold, delay was reduced by20 percent, and jitter was reduced by 50 percent, enabling scientists to manip-ulate extremely large datasets in real time from 2,800 miles away.

On a final note, I have enjoyed more than five years with NASA Ames, bothas Director of the Consolidated Supercomputing Management Office andas Deputy Division Chief and acting Chief of the NAS Division. I will soonbe joining Los Alamos National Laboratory as head of the AdvancedComputing Laboratory.

I would like to take this opportunity to thank the NAS Division and NASA forthe opportunity to meet and work with many of the finest scientists and staff,as well as collaborators both in industry and academia, I have ever had the priv-ilege to know and serve. Thank you.

From The Division Chief

NAS MissionTo lead the country in the research,development, and delivery of revolu-tionary, high-end computing servicesand technologies, such as applicationsand algorithms, tools, system soft-ware, and hardware to facilitate NASAmission success.

NewsFromNAS


NAS researchers filed a patent for using coupled

lasers to achieve ultrafast modulation and

switching in optical networks, one of the many

accomplishments highlighted in the

division’s year in review. - See page 11

Parallelization – Faster Codes and Higher Fidelity – 18

Winter 2003

www.nas.nasa.gov

Completing Large Parameter Studies with Time to Spare – 4

Gridpoints 1

NATO Lecture SeriesProvides Guidance forFuture UAV Design

Nearly 100 researchers from aroundthe world gathered in Palo Alto,

Calif., for the NATO (North AtlanticTreaty Organization) lecture series,hosted by NASA Ames ResearchCenter on Oct. 29–30. Sponsored byNASA’s Computing, Information, andCommunication Technologies Pro-gram, the event was a unique opportu-nity for researchers to discuss their cur-rent work in the area of uninhabitedair vehicles (UAVs).

UAVs are remotely or self-piloted air-craft that can carry cameras, sensors,communications equipment, or otherpayloads. Both NATO and NASAhave applications in UAVs and have astrong interest in their design and de-velopment to meet national securityrequirements and NASA mission needs.

This year’s series, “Applications, Con-cepts and Technologies for FutureTactical UAVs,” highlighted topicsincluding integrated mission systemsdesign concepts and alternatives, keyenabling technologies (sensors, situa-tion awareness, communication), andautonomous operations. “Many of theissues that we have addressed in thepast and that we continue to explorefor the future here at NASA AmesResearch Center are the same issues

Continued on page 2

John Ziebarth

being discussed by those of you attending this NATO event,”said Jan Aikins, acting deputy director of Ames’ InformationSciences and Technology Directorate. “Our researchers in theInformation Sciences and Technology, Astrobiology and

Space Research, and Aerospace directorates are dealing withboth development and applications of autonomous controltechnologies for science, with human-machine interfaces forground control systems, and with policies and regulationsnecessary for operating in the national airspace, among

News From NASContinued from page 1

2 Gridpoints

On November 21, 2002, Eastern IllinoisUniversity professor of mathematics Suhrit

Dey returned to NAS to present his latest breastcancer research findings (see Gridpoints, Fall2001, page 4). Dey is developing a system ofmathematical equations to model the effects ofchemotherapy, radiation therapy, and immuno-therapy on the growth and spread of breast can-cer cells in the human body. Breast cancer cur-rently affects more than two million women in theUnited States. Dey believes the effects of breastcancer can be managed by understanding howthe disease behaves. His predictions on the dis-ease’s interactions within the body are based onmathematics and computer simulations.

Among Dey’s recent work is a series of equationsaccounting for the growth and spread of cancer,which are used to model how these factors affectnon-cancerous cells in the body. The new equa-tions were integrated into Dey’s 3-D dynamicanimation, developed in part by his son CharlieDey, also at the University of Illinois. Dey’s ani-mation enables him to rotate the model in anydirection and observe movement of individual cancer cellsin the body from all angles, especially in the breast area.Working with John Koontz of SkyNetworks in Champaign,Illinois, Dey has improved the code for generating his ani-mations. The code requires massively parallel computationsin the message-passing interface (MPI) environment.

Strength in NumbersAn important part of Dey’s work has been traveling theglobe – from Scotland, to Italy, to India, to form collabora-tions with doctors and deliver invited talks. Discussing hisresearch findings with doctors provides Dey with feedbackabout his mathematical models. Most recently, Dey hasbeen working closely with Charles Wiseman, chief oncolo-gist at Los Angeles’ St. Vincent Hospital. Wiseman has sup-plied the Eastern Illinois University professor with data thatsupports the predictions of his three-dimensional model.

Dey has focused his research on the cause of recurringbreast cancer cases. “The mathematical model reveals thatwhen the main tumor is gone, that does not mean that thecancer is gone. The cancer cells could still be alive and gain-

ing strength, especially if the immune system gets weaker,”explains Dey. To determine what is needed to ward offrecurring cases of cancer, and to increase the number offighter T-cells, B-cells, and Natural Killer cells in the body,Dey is experimenting with variables in his mathematicalmodels such as stress, depression, and side effects of med-ications that weaken the immune system.

To reach his long-term goal of putting most, if not all, ofthe human body’s cells into a single configuration, Dey willrequire more computer power. Using a Pentium 4 machine,it currently takes two days to complete his calculations.Dey is confident that he could complete them in under 20minutes on a multi-processor, shared-memory supercom-puter such as those housed at the NAS facility.

Editor’s Note: Suhrit Dey’s breast cancer research was initiat-ed after receiving grants from Bob Augustine, dean of the East-ern Illinois University graduate school, Mary Anne Hanner,dean of the university’s College of Sciences, and Bud May,director of the university’s Grants and Research Department.

Computational Modeling to Understand Breast Cancer’s Behavior

During the November 21, NAS New Technology seminar, Suhrit Dey,mathematics professor at Eastern Illinois University, presented several newequations for his 3-D dynamic animation used to model breast cancer inthe human body. (NASA/Tom Trower)

www.nas.nasa.gov/gridpoints Gridpoints 3

others. We are pleased to be hosting this event, and aredelighted to be working with you to further our commonresearch goals for UAVs.”

Among the panel of speakers were three members of theNATO lecture series team: John Kitowski, Robert Frampton,and James Ramage. Kitowski spoke on the economical andtechnological challenges associated with UAVs, while Framp-ton discussed their increasing role in support of combat mis-sions. During his talk on the considerations for future UAVresearch and development, Ramage explained that furtherautomation research is required to reach the design and per-formance goals of the UAV. In addition, Ramage believesthat stringent safety regulations and technology maturity ofuninhabited aircraft have, and will, present a huge challengefor UAV designers and developers. Concluding his presenta-tion, Ramage said, “Future R&D directions should empha-size demonstration and validation of key enabling technolo-gies in realistic integrated system operation environments,using mission relevant metrics.”

Researchers have many challenges ahead before UAVs canfunction as fully integrated systems that can execute taskssuch as automatically adjusting to the changing tactical re-quirements of combat. Essentially, UAVs must be able to“think” on their own, be self-directing, or autonomous. Auto-nomy, a system’s ability to perform a task without directionfrom an external source, reduces and automates the work-load, enabling the execution of tasks ranging from vehiclemanagement, based on terrain and weather conditions, tothreat avoidance.

In addition to advancing our nation’s security, UAVs play arole in meeting NASA missions. Acting Chief of the Ames’Computational Sciences Division Daniel Clancy delivered atalk, “Autonomy for Deep Space Exploration as Relevant toUAVs,” in which he described clear autonomy needs of theupcoming Europa mission, scheduled for launch in March2008. This mission entails the robotic exploration of Europato seek out evidence of life. Clancy emphasized: “Autonomyis a capability, not some flashy technology – it increases safe-ty and capability.”

Heinz Erzberger, senior scientist in Ames’ Aviation SystemsDivision, also delivered a talk highlighting some of Ames’newest air traffic management technologies and decisionsupport tools, which have been fielded with great success.Said Erzberger: “I saw several parallels in UAV automationand in air traffic management automation, especially with re-spect to human interface and situational awareness issues.”

Next year’s NATO event on this topic will take place in June2003. For more information on the lecture series, visitNATO’s Research and technology website at:www.rta.nato.int/Meetings.asp.

NAS Division Tech Reports

Technical reports are one of the ways NAS Division re-searchers share their work with others in the scientific

community. A large number of reports are available elec-tronically in PDF format on the division’s website, includingan archive dating back to 1989. The most recent reports canbe found at:www.nas.nasa.gov/Research/Reports/techreports.html

NAS Technical Reports for 2003

“Performance Evaluation of Remote Memory Access (RMA)Programming on Shared Memory Parallel Computers,” byHaoqiang Jin and Gabriele Jost • NAS-03-001

“NAS Parallel Benchmarks I/O Version 2.4,” by Rob F. VanDer Wijngaart and Parkson Wong • NAS-03-002

NAS Technical Reports for 2002

“A Posteriori Error Estimates for Higher Order GodunovFinite Volume Methods on Unstructured Meshes,” byTimothy Barth and Mats G. Larson • NAS-02-001

“Hybrid MPI+OpenMP Programming of an Overset CFDSolver and Performance Investigations,” by Jahed Djomehriand Haoqiang Jin • NAS-02-002

“Resource Selection Using Execution and Queue Wait TimePredictions,” by Warren Smith and Parkson Wong • NAS-02-003

“High-Order Central WENO Schemes for Multi-Dimensional Hamilton-Jacobi Equations,” by Steve Bryson(NASA Ames) and Doron Levy (Stanford University) •NAS-02-004

“NAS Grid Benchmarks Version 1.0,” by Rob F. Van DerWijngaart and Michael Frumkin • NAS-02-005

“High-Order Semi-Discrete Central-Upwind Schemes forMulti-Dimensional Hamilton-Jacobi Equations,” by SteveBryson and Doron Levy • NAS-02-006

“NAS Parallel Benchmarks, Version 2.4,” by Rob F. Van DerWijngaart NAS-02-007

“The Efficiency and the Scalability of an Explicit Operatoron IBM POWER4 System,” by Michael Frumkin • NAS-02-008

“Implementation of the NAS Parallel Benchmarks in Java,”by Michael Frumkin, Matthew Schultz, Haoqiang Jin, andJerry Yan • NAS-02-009

4 Gridpoints

Completing Large ParameterStudies — with Time to Spare

When designing an aircraft, every possible flightcondition is scrutinized before it leaves the draw-ing board. Engineers spend countless hours test-

ing thousands of parameters and vehicle configurations.With the evolution of computational fluid dynamics (CFD),a sophisticated computational analysis technique used topredict fluid flows, the time and expense involved in design-ing air- and spacecraft has been reduced substantially. Still,the CFD method is a labor-intensive activity, using hundreds

of hours to complete the calculation of just one geometryunder a single flight condition.

Researchers in the NASA Advanced Supercomputing (NAS)Division have automated the CFD process by creating theAeroDB system. AeroDB boils down the testing process ofaerospace vehicle design, saving both time and money.The NAS AeroDB team of Stuart Rogers, Edward Tejnil,Michael Aftosmis, Shishir Pandya, Jasim Ahmad, and Neal

Figure 1: This diagram representsseveral main components, or mod-ules (written in the Perl program-ming language), that make up theAeroDB system. The database is theheart of the system — all communi-cation between components takesplace here. The other componentsinclude: the job submission script,which automatically creates aunique identification number foreach job and provides the user withthe interface to specify the case’sinput files and parameter values;the job launcher module, which isresponsible for monitoring the data-base and launching each job sub-mitted; a remote execution module,which draws on information fromthe database to run the flow solver;a run manager library, which mon-itors solver output to determinewhen the job converges or runs outof time; and a user portal, whichserves as the interface for users toview the status of their jobs.

NAS Division researchers create an automated CFD system to run large parameter studies on IPG resources.


Chaderjian combined their knowledge and experience withCFD programming, the Perl scripting language, and theOverflow and Cart3D flow solvers to create this new soft-ware system. The project is supported by the NASAComputing, Information, and Communication Technologies(CICT) Program’s Computing, Networking, and Informa-tion Systems (CNIS) Project.

Automating the CFD process leaves designers and engineersfree to focus on important aspects of vehicle analysis, such asdesign optimization. Part of this analysis includes the studyof flow parameters, including angle of attack, sideslip angle,and Mach number. The process of setting up parameter stud-ies (a collection of computer jobs with a slightly differentinput for each job) is very labor intensive and error prone. Byreducing dependency on the user, AeroDB decreases theamount of error in the analysis. “The AeroDB system signif-icantly simplifies the process of executing many CFD jobs. Itis a big step toward automating the process to take the userout of the loop of having to monitor every single job,”explains Rogers, senior scientist on the AeroDB project.

Automating the CFD ProcessAeroDB is a collection of Perl scripts, a database, and a Webportal (see Figure 1, page 4). The system utilizes resourcesand tools provided by the Information Power Grid (IPG),NASA’s geographically distributed network of computing

and data resources, to support its automated system (see Grid-points, Fall 2002, page 4).

The Perl scripts, developed largely by Tejnil, interact with thesoftware tools provided by the IPG to launch, run, and mon-itor jobs on the grid’s network of computers. A job submis-sion script is used to enter basic information about new jobs(such as location of the input files, job size, type of flow sol-ver, and flow parameters), into the database. A job launcherscript sends all jobs for execution to the most suitable com-puter resources. The job launcher uses information providedby the IPG resource broker, which, in turn, obtains data froman information service called the Monitoring and DiscoveryService (MDS). The MDS contains information about eachIPG resource, including the number of jobs currently run-ning, in order to suggest the optimal place to run a job.

To store information about CFD jobs, AeroDB has its owndatabase. “The database is the heart of the system – the com-munication hub and repository for all information about thejob,” explains Rogers. The database contains both static anddynamic job information – unchanging computer host anduser account information, and data that is constantly updat-ed as a job is running. “AeroDB’s database-centric approachprovides the user with a convenient central warehouse forinvestigating the results of simulations,” adds AeroDB teammember Michael Aftosmis.

Figure 2: Surface grids on one configuration of the Liquid-Glide-Back Booster (LGBB) calculated using the Overflow flow solver.

This grid system contained more than eight million points. Atypical case with this grid requires 200 CPU hours on an

SGI Origin 3000 supercomputer. (Stuart Rogers)

6 Gridpoints

Controlling jobs and accessing information about them ismade easy through a Web portal, the interface for theAeroDB system. Through this password-protected interface,users can view the status of each job in the database, and eas-ily rerun and restart jobs as necessary using globusrun (acommand used to manage job submission to grid resources).“AeroDB’s Web portal really adds to the convenience of thetool,” explains Aftosmis. It taps the system’s database, allow-ing the user to check the progress or manipulate jobs in aparameter study, from any Web browser.” To view detailedinformation about a particular job, the user just clicks on theautomatically generated job ID number.

Working with the FlowTo demonstrate AeroDB’s capabilities, two flow solverswere used: Overflow and Cart3D. Overflow is used to solveNavier-Stokes equations, which mathematically describe

how air or water flows around an object. When compiling,Overflow expends many CPU hours to complete calcula-tions for even a simple geometry (see Figure 2, page 5).Cart3D solves Euler equations, which ignore viscosity (theresistant forces of a fluid’s flow), so flow can be representedwith a lot fewer points; however, accuracy is somewhat lostwhen analyzing more extreme angles of attack (see Figure 3).

The sample dataset used to demonstrate AeroDB’s ability toautomate the CFD process was the Liquid-Glide-BackBooster (LGBB), a conceptual design for a new, reusablelaunch vehicle. This vehicle is currently being studied underNASA’s Space Launch Initiative (SLI) Program, establishedto develop technologies to significantly reduce the cost ofspace transportation and enable future space missions. Someexperimental wind tunnel data for the LGBB has alreadybeen generated at NASA’s Langley Research Center, Virgina,

Figure 3: This figure represents a Cartesian unstructured mesh generated with the Cart3D flow solver, for the sameconfiguration of the Liquid-Glide-Back Booster as the Overflow grids in Figure 2 (see page 5). A Cart3D generatedflow solution requires approximately one-tenth of the time; however, accuracy suffers somewhat due to the nature ofthe equations used in the flow solver, which overlook viscosity. (Michael Aftosmis)


and has been used to validate CFD results generated usingthe AeroDB system. “Preliminary comparison betweenexperimental and computed results look encouraging,” saysRogers. One configuration of the LGBB geometry includedthe wing and fuselage, vertical tail, and canards. The teamanalyzed this flight vehicle over a wide range of flight condi-tions (see Figure 4).

Testing the ToolTo measure the success of the automated CFD system, sev-eral metrics were established: Within seven days, the systemmust successfully execute 1,000 Cart3D cases and 100Overflow cases. This metric was surpassed in just three days,and at the end of the seventh consecutive day, 2,863 Cart3Dcases and 211 Overflow cases had been completed success-fully. These cases were run on 13 different IPG computersdistributed over four different geographical locations:NASA’s Ames and Glenn Research Centers, the InformationSciences Institute (ISI) at the University of Southern Califor-nia in Marina Del Rey, and the National Center for Super-computing Applications (NCSA) in Illinois.

At the same time the AeroDB system was utilizing the IPG’sdistributed resources, the application helped point out someareas for improvement within the grid environment. “Wewere helping improve the IPG environment by running ourapplication. I think we’ve really tested the IPG to the extentwhere it has never been tested before – it was something thatreally needed to be done,” says Rogers. “Working with theAeroDB team and their application was a valuable exercise.Many lessons were learned that will ultimately improve theIPG software. AeroDB was the first application we [the IPG

team] have seen that stressed every aspect of the grid – fromits technology to human factor issues such as communicationand coordination. The AeroDB team has made a significantand welcome contribution to the NASA grid effort,” addsNAS IPG lead Tony Lisotta.

Pushing the LimitsEven though the AeroDB system exceeded the metrics of thedemonstration, the group plans to make several improve-ments, such as making the system accessible through theWeb to any NASA user with an IPG certificate. The teamalso plans to add an automatic error recovery capability forinstances when a job has to be restarted. This restart functionwill prevent the loss of work and valuable computer time.Since AeroDB is highly specialized, it can only accommodateOverflow and Cart3D, but with some additions to the Perlscripts, the system will be able to accommodate other flowsolvers, enabling access to other CFD software packages.

In the future, the AeroDB team plans to integrate more ofthe grid’s tools into their system, making it more versatilewithin the grid environment and a more seamless process forusers. Specifically, the IPG job manager will be incorporatedinto the AeroDB system, shifting the responsibility for filetransfer, staging, and interfacing with globusrun to the IPGteam. This modification will provide a higher-level interfacewith these tools. “Incorporating the IPG job manager intoAeroDB will not require a lot of changes,” explains Rogers.“It should be fairly straightforward to do – our system wascreated in a modular fashion.”

—Holly A. Amundson

Figure 4: This plot represents therelationship between several parame-ters for the Cart3D Liquid-Glide-BackBooster cases run with no sideslip flow:lift coefficient (CL), Mach number, andangle of attack. About 20 percent of thecases run within a seven-day time-periodare represented by this plot, each dot rep-resenting the CL from one case.

(Aftosmis/Rogers)

8 Gridpoints

The year 2002 was marked by many innovative devel-opments from the NASA Advanced Supercom-puting (NAS) Division, and major accomplishments

in the areas of computational and computer science con-tributed to the success of NASA missions and science andengineering milestones throughout the agency.

In addition to receiving the NASA Software of the YearAward and the NASA Commercial Invention of the YearAward, the expertise and achievements of NAS scientistswere recognized by a number of NASA Space Act Awards,patents, and invitations to sit on national and internationaloversight panels.

In the area of capability computing, NAS demonstrated thehighest performance on any U.S. architecture, by any ven-dor, on the 1,024 processor SGI system for Earth science andaerospace codes. This accomplishment means that the com-mon shared-memory architecture is proving increasinglyvaluable for scientists and engineers, and represents a majorstep forward in innovative and high-performance computingfor science and missions throughout the agency.

Here are some of the highlights gathered from the divisionin 2002:

DeBakey Ventricular Assist Device NamedNASA’s Invention of the Year 2001 To improve the performance of the DeBakey VentricularAssist Device (VAD), NAS scientists Cetin Kiris andDochan Kwak used computational fluid dynamics (CFD)technology developed for the Space Shuttle’s main enginefuel turbopump to recommend several design modificationsfor solving blood flow problems.

The DeBakey VAD is a life-saving miniature heart assistdevice for human implantation, and the result of a collabo-

rative project between researchers at NASA Ames andJohnson Space Centers and MicroMed Technology Inc., ofHouston. Thanks to the improvements suggested by theNAS scientists, the device has now been implanted success-fully in more than 170 people, and in April the DeBakeyVentricular Assist Device was named NASA’s Invention ofthe Year for 2001.

“Our involvement with the development and improvementof the DeBakey VAD makes me extremely proud of what wedo at NASA,” says Kiris. “This work will improve manylives.”

NASA Software of the Year Award: Cart3DCart3D, an aerodynamic simulation tool developed byNASA scientists Michael Aftosmis and John Melton, in col-laboration with Marsha Berger of the Courant Institute inNew York, won NASA’s prestigious Software of the Yearaward for 2002. The Cart3D software package provides

NAS Division in 2002: A Trip DownShared-Memory Lane

The NAS Division looks back on a successful year of research in diverse areas, andrecognition for outstanding scientific achievement.

Using computational fluid dynamics analysis, NAS researchersCetin Kiris and Dochan Kwak found that major design mod-ifications to the DeBakey Ventricular Assist Device were neces-sary. The result of these changes increased overall efficiency ofthe device by 22 percent. (MicroMed Technology)


aerospace designers and engineers with afully automated geometry processing andcomputational fluid dynamics simulationsuite. This powerful tool streamlines theconceptual and preliminary analysis of newand existing aerospace vehicles.

“Cart3D reduces simulation time by a fac-tor of at least 250,” says Aftosmis. “Thisaward recognizes the utility and impact thatCart3D is having throughout the engineer-ing simulation community worldwide.”

The software is already being used by anumber of leading universities, and by anexpanding list of more than 100 commer-cial users, including the Boeing Companyand Northrop Grumman. The Ames Com-mercial Technology Office has granted alicense for the software to ICEM CFDEngineering of Berkeley, California, a sub-sidiary of ANSYS, Inc., which will extendthe use of Cart3D into other industries,including automotive and electronics.

Chapman Begins Operations Chapman – the first and only 1,024 processor cache-coher-ent, shared-memory computer in the world – began opera-tion in 2002, the result of collaboration between the NASDivision and SGI. The NASA/industry collaboration contin-ues to build successively larger single-system-image comput-

ers and to advance the research and under-standing in parallel supercomputing.

“Parallel shared-memory systems havemany properties that provide significantadvantages over clustered parallel systems,”says NAS scientist Bob Ciotti. “For exam-ple, using the Multi-Level Parallel (MLP)library with other features and enhance-ments to the operating system, a simple yetpowerful parallel programming environ-ment allows users to more easily adapt engi-neering problems to parallel processing sys-tems. This results in better utilization ofscarce human programming resources,enabling the solution of NASA’s most com-plex computational problems.”

UFAT Software ReceivesSpace Act Award The Unsteady Flow Analysis Toolkit(UFAT) software program, a pioneeringtool for visualizing very large time-depend-ent or “unsteady” flow datasets from CFDsimulations, won a NASA Space Act awardin 2002. NAS scientist David Kao was rec-ognized for his contribution to UFAT,which has been used to process and analyzehigh-fidelity simulation results for theSpace Shuttle, military and commercial air-craft, artificial heart devices, and manyother projects.

The software effectively reduces the analysis time of multi-gigabyte datasets from weeks to hours using state-of-the-artparticle tracing and feature extraction algorithms developedby NAS scientists. For more information on UFAT, visit:www.nas.nasa.gov/Software/UFAT/

Other NAS recipients of the NASA Space Act Award includeNateri Madavan for his Aerodynamic Design using NeuralNetworks technology, and Guru Guruswamy for the HighFidelity Multidisciplinary Analysis Process (HiMAP) soft-ware system.

IPG Power ToolsLast summer, a team of developers working on NASA’s Infor-mation Power Grid (IPG) deployed several new grid tools tomake it easier for scientists throughout the United States touse the IPG’s distributed resources. The new tools all use theGlobus Toolkit, which provides a uniform interface to IPGresources, and were created to simplify the process of sub-mitting jobs to the grid’s distributed resources. New toolsinclude a job manager to run jobs and track their progress,an abstraction tool for handling job scheduler informationon multiple machines, and a resource broker for selectinggrid resources (see Gridpoints, Fall 2002, page 4).

Cart3D is used to automate the grid genera-tion process, shown at right, on the SpaceShuttle orbiter. The software suite has signif-icantly contributed to the NASA mission,and was awarded the agency’s Software ofthe Year award in June. (Michael Aftosmis)

Chapman, an SGI Origin 3800 parallel high-performancecomputer is now available for use by NASA researchers andtheir collaborators. (NASA/Tom Trower)

10 Gridpoints

The collaborative project drew on the expertise of IPG teammembers at NASA Ames, Glenn, and Langley Research Cen-ters, together with researchers at the National Center forSupercomputing Applications (NCSA) in Illinois, theInformation Sciences Institute at the University of SouthernCalifornia, and Argonne National Laboratory in Illinois.

Delivery of the grid programming environment helpedNASA’s Computing, Information, and CommunicationsTechnology (CICT) Program’s Computing, Networking,and Information Systems (CNIS) Project meet one if itsmajor milestones. “This milestone provided us with a greatopportunity to study the interoperability of grids,” said NAS

IPG task Tony Lisotta. “This will lead to future work in mak-ing grids more compatible with one another.”

ILab meets IPG challengeAnother step forward for NAS’s Information Power Grid(IPG) was accomplished in March, when ILab successfullycompleted 1,024 jobs using ten systems at four NASA sites(Ames, Glenn, and Langley Research Centers, as well as theJet Propulsion Laboratory). The software package was devel-oped by NAS scientists Maurice Yarrow and Karen McCannfor automating the process of creating, submitting, and mon-itoring parameter studies on the IPG.

Following that accomplishment, NAS scientists again usedILab to run the Reusable Launch Vehicle (RLV) GrandChallenge computation as a five-by-five parameter study,using seven systems at three sites (Ames, Glenn, andLangley). Each computation ran through three separaterestart phases, required 16 processors and 3.2 gigabytes ofsystem memory, and generated output flow-data files of near-ly 500 megabytes.

Largest Dataset Ever Visualized at NAS Arguably the largest dataset visualized by NAS researcherswas a simulation of the next generation of the Space Shuttle’smain engine fuel pump, force-feeding liquid hydrogen in-to the main engine at 10,000 gallons per minute. The visu-alization focused on showing temperatures, pressure, andparticle tracing, using 1,600 time steps – each with a giga-byte of data.

“We convert the dataset for each time step into a proprietaryformat that allows us to pull in paged chunks of data,” ex-plains NAS visualization specialist Timothy Sandstrom. “This

This diagram represents a conceptual view of the IPG softwareenvironment, including software components developed by theIPG team used to meet the requirements of a Computing,Information, and Communications Technology (CICT)Program milestone. The tools include the IPG resource brokerand the Monitoring and Discovery Service (MDS).

The image shows light intensity varying with angle of laser out-put from two coupled VCSELs. (Cun-Zheng Ning)

Patent Filed on Ultrafast Optoelectronics DeviceUsing semiconductor laser simulations, NAS scientistsCun-Zheng Ning and Peter Goorjian proved the effec-tiveness of using coupled lasers to achieve ultrafast modu-lation and switching in optical networks, resulting in apatent filed in June. Their method couples two vertical-cavity surface-emitting lasers (VCSEL), each with a diam-eter of about ten microns – one-tenth the thickness of ahuman hair, and carrying light at speeds of up to 40 giga-hertz. (For more information on VCSELs, see Gridpoints,Spring 2000, page 12.)

“Coupling introduces a new frequency with a muchgreater limit than the intrinsic frequency of a singledevice,” explains Ning. “Our method drives the higherfrequency of the joint devices.” Results from the projectwere published in the Journal of Modern Optics, OpticsLetters and Proceedings of The International Society forOptical Engineering.


enables us to pull the exact data we need and focus on vis-ualizing that one part.”

In the meantime, NAS scientists continue to develop toolsthat will enable them to scale up flow-solver simulations foruse on the 1,024-processor supercomputer, Chapman, sig-nificantly shortening design time.

NAS Nanotechnology Leading the WayExperiments conducted at the Rensselaer Polytechnic Insti-tute in New York, the Science and Nanotechnology Center in

Sussex, England, and Ulm University in Germany, producedactual carbon nanotube t-junctions for the first time, workoriginally proposed possible in 1997 by NAS division scien-tists based on simulations.

“Now we are seeing the theory we predicted in action,” saysNAS scientist Deepak Srivastava. Since then, Srivastava hasbeen investigating more complex structures, and is currentlytesting thermal signals, the instantaneous heat flow on abranched nanotube structure, to verify the amount of infor-mation processing contained in this structure.

This Space Shuttle fuel pump simulation was the largestdataset computed at the NAS Facility during 2002. (Dataset:Cetin Kiris, Visualization: Timothy Sandstrom)

Shocking ChemistryBy simulating comet and meteor impacts on primitiveatmospheres, NAS scientists are hoping to help re-searchers understand how life may have been formedon Earth, and whether the same processes could hap-pen elsewhere.

At the American Chemical Society’s National Meetingin Boston last August, NAS scientist ChristopherDateo gave a presentation of his group’s findings,“Prebiotic Processing Induced by Comet and MeteorImpact.” The study, funded by the Office of SpaceScience’s exobiology program, simulated high energyshocks on four different atmospheres to determine theeffect on the formation of molecules, and whether thismight be a means of supplying organic material to aplanetary atmosphere.

Dateo is developing the model and benchmarking it toexperiments conducted by scientists in the SpaceScience Division at NASA Ames. He is optimistic thatthese studies could lead to the identification of bio-sig-natures or organic molecules.

In an effort to determine how life began on Earth, scientists areresearching the possibility that comet and meteor impacts broughtorganic material that could have eventually formed a planet’s atmos-phere. Mars’ Newton Crater, formed by an asteroid impact thatprobably occurred more than three billion years ago, shows manynarrow gullies, believed to have been formed by flowing water.(NASA/JPL/GRIN GPN-2000-001430)

Biological neural systems were the inspiration for this simula-tion of a 3-D treelike structure of carbon nanotubes. NAS sci-entists are currently investigating the transmission properties ofthis type of architecture as a biomimetic approach to futurecomputing paradigms. (Deepak Srivastava)

12 Gridpoints

Innovation in nanotechnology earned Srivastava and scien-tist Chris Henze a grant from the Ames Research CenterDirector’s Discretionary Fund in 2002. Their proposal for the“biomimetic simulation of signal transmission in nanotube-based dendritic networks” was inspired by the structure andoperations of biological neural systems. The scientists areinvestigating the properties of a 3-D, tree-like architecture ofcarbon nanotubes, and hope to prove its viability for appli-cations involving the transmission and processing of signals.

Modeling Neural SystemsResearchers studying neural encoding and information theo-ry analysis in the sensory system of a cricket’s tail are usingvisualization and modeling techniques developed by NASscientist Chris Henze to gain a better understanding of neu-ral structures and how they operate.

Using a specially designed wind tunnel at the Center forComputational Biology at Montana State University,researchers Gwen Jacobs and John Miller record the crickets’responses to various wind stimuli, analyze the data, and fig-ure out how the cricket’s neural system encodes the inputand output. In addition to visualizing a 3-D neural map foreach stimulus, Henze has written software that allows theresearchers to model what the cricket is doing, and has cre-ated a computational analogue of the laboratory set-up thatharnesses the power of NAS’ supercomputers. Researchershope this work will provide a better understanding of theneural basis of perception and cognition.

Results from the project were presented at the internationalconference on Cooperative Dynamics of Neurosystems inChile, the Computational Neuroscience Meetings inChicago, and Neuroscience 2002, the annual meeting of theSociety for Neuroscience.

Cataloging the Building Blocks of LifeComputational chemists in the NAS Division have devel-oped a theoretical model that is already generating highlyaccurate data for a catalog of amino acids (see Gridpoints,Spring 2002, page 6). This catalog will help astronomersidentify the molecules they observe in space, potentially pro-viding more clues to solving the mystery of life’s origin.

The model combines quantum and molecular mechanics ina unique vibrational infrared method. NAS scientist Galina

Chaban presented her findings to the second biennial Astro-biology Science Conference at NASA Ames Research Centerin April, and since then has added glycine, alanine, andvaline to the database.

NAS Scientist Receives International AwardFor his distinguished work in computational nanosciencesand carbon nanotubes, senior scientist Deepak Srivastavareceived the Eric Reissner Medal in August 2002 (see Grid-

The ‘hyperwall’ – From Concept to DebutA team of NAS researchers has developed an innovativevisualization system that allows scientists to view com-plex datasets and multiple parameters. The hyperwall, aseven-by-seven cluster of flat panel screens, (right), eachdriven by its own dual-processor computer with high-end graphics card, made its debut at the annual super-computing conference, SC 2002, in Baltimore, and hasproved popular among scientists throughout NASA (seeGridpoints, Fall 2002, page 11).

“The hyperwall allows scientists to look at data fromdifferent viewpoints,” explains the system’s chief archi-tect, Chris Henze. “It can tile a single image like a pow-erwall can, but its main strength lies in its ability to sup-port spreadsheet-style approaches to visualizing data.”

H

H

H H

H

C

CO

O

N

0.973

106

1.362 1.524

123

1.221

126

115

1.093

1.455

1.016

Illustration showing the complex of glycine, the simplest aminoacid that occurs in many proteins and has already been observedin space. (Galina Chaban)


points, Fall 2002, page 1). The prestigious medal, namedafter the noted researcher and academic, is awarded everytwo years by the International Conference on Computa-tional and Experimental Engineering and Sciences.

NAS Scalable Algorithms used in Materials ScienceIn a collaborative effort that joined two universities in theUnited States, four universities in Japan, and NASA AmesResearch Center, a multiscale simulation was successfullyconducted to study the environmental effects of water mole-cules on fractures in silicon. The simulation was performedon a grid of 25 PCs distributed over three clusters in theUnited States and Japan. NAS senior scientist Subhash Sainidesigns algorithms for large-scale applications and participat-ed in the project. Saini co-authored several papers on scala-ble algorithms in 2002, including “ARMS: an agent-basedresource management system for grid computing,” whichappeared in Scientific Programming 10, and “CollaborativeSimulation Grid: Multiscale Quantum-Mechanical/ClassicalAtomistic Simulations on Distributed PC Clusters in theU.S. and Japan,” published by the Institute of Electrical andElectronics Engineers.

NAS Research Advances Theory in DNA Electronics Research in the conductive properties of DNA continuesto challenge researchers around the world with contradic-tory experimental results. In 2002, NAS scientists M.P.Anantram, T.R. Govindan, and Alexei Svizhenko producedtwo papers from their theoretical and modeling work thatthey hope will help explain some of the experiments. The

papers, “Environment and structure influence on DNA con-duction” and “Influence of disorder on DNA conductance”are due to appear this spring. The scientists are also interest-ed in learning how arbitrary sequences of DNA might beexploited in nanotechnology.

Parallelizing Quantum ChemistryAs large supercomputers and parallelization offer researchersgreater amounts of processing power, the theoretical ap-proach to quantum chemistry is becoming increasing rele-vant. Simulations using the laws of quantum mechanicscan provide researchers with detailed information on thesubtle interactions between electrons, without the high costof experiments.

NAS scientist Graham Fletcher has been designing scalablealgorithms to distribute data-intensive processing over agroup of processors. In 2002, he developed a scalable, paral-lel, distributed-data Multi-Configurational Self-ConsistentField (MCSCF) program that allows researchers to study the-oretical situations in which electrons are being separated.Fletcher believes this is particularly important for materialsscientists designing lighter and stronger materials for spacehardware. Fletcher’s program is available on the GeneralAtomic and Molecular Electronic Structure System, and main-tained by members of the Gordon Research Group at IowaState University.

NAS Releases Grid Benchmarks for Global Community The Global Grid Forum (GGF) called on NAS scientistsRob Van Der Wijngaart and Michael Frumkin for guidance

Marsoweb Usage Doubles…Then Triples

Following an article in the Summer 2002 edition ofGridpoints and a subsequent press release, Marsoweb, aNAS-developed website for the Mars ExplorationProgram Landing Sites data, saw its user base soar to70,000 – more than double the number of visitors in twomonths, compared with the past three years.

Marsoweb, developed by NAS visualization specialistGlenn Deardorff, is designed to provide a one-stop-shopfor scientists involved in landing site selection for up-coming Mars Rover missions. In addition, it hosts a vari-ety of images and an interactive data map that has alsoproven popular with Mars enthusiasts and the educa-tional community.

Mars Orbiter Camera (MOC) images for MER 2003landing site studies can be displayed on regional navigationpages for closer scrutiny of the planet’s surface. This userinterface is used to select and view images in the MOCimage archive. (Glenn Deardorff )

14 Gridpoints

in measuring grid efficiency and user-friendliness. Followingapproval for the formation of an official research group ongrid benchmarks in July, NAS promptly responded with therelease of GridNPB3.0, a pencil-and-paper specification offour families of problems representing important applica-tions on computational grids (see Gridpoints, Summer 2002,page 5, and Gridpoints, Fall 2002, page 15).

The release of GridNPB3.0 coincided with the release of amajor new version of the NAS Parallel Benchmarks, designedto incorporate several important aspects of today’s comput-ing needs.

Conference Hosts Biologists, Nano-technologists, and Information TechnologistsIn the first workshop of its kind, nearly 100 experts gatheredat NASA Ames Research Center last October to explorecross-disciplinary approaches to solving the science and tech-nology challenges facing future NASA missions. NAS seniorscientist T.R. Govindan co-chaired the Biology, Information

Science, Nanotechnology Fusion and NASA MissionsInvitational Workshop, which was organized by NASA Amesand the University Space Research Association.

“Our goal was to start a conversation and gather new ideas,”explains Govindan. “Based on the level of enthusiasm andinterest in the fusion of these three fields, it’s likely that thiskind of conference might become a regular event.” For moreinformation, visit http://binfusion.arc.nasa.gov

Analyzing Spectral CongestionCalculating the vibrational frequencies and infrared opacitiesof water and methane is important in studying planetaryatmospheres, and in the astrobiological search for life. NASscientists developed computational methods to helpastronomers analyze their observations and understand thecomposition of planetary atmospheres, with an error rate ofless than 0.00012 electron volt. This level of accuracy pro-

vides researchers with a computational method to ascertainmethane presence, where spectral congestion has until nowmade this an impossible task.

NASA’s Office of Space Science funded the project, and NASscientist David Schwenke presented the research, “Ab InitioTheory of Water and Methane Frequencies and Opacities,”to the American Geophysical Union Fall Meeting in Dec-ember. Astronomers worldwide are now using this data.

Calculating Planetary Entry ReactionsWhen a spacecraft enters a planetary atmosphere at very highspeed, it causes a reaction that excites, dissociates, and ionizesthe ambient atoms and molecules, generating temperatures

in excess of 20,000 degrees Kelvin. To help heat-shielddesigners understand how molecules like oxygen and nitro-gen react in such high temperatures, NAS scientist DunyouWang has completed the first accurate theoretical calculationof the rate reaction, providing important data for flow-fieldmodeling and determining heat load. The paper, “QuantalStudy of the Exchange Reaction for N+N2 Using an Ab InitioPotential Energy Surface,” will be published by the Journalof Chemical Physics.

Studying Diffuse Interstellar BandsFor more than 50 years, scientists have been trying to deter-mine the origin of diffuse interstellar bands (DIBs) in theastronomical spectrum. More recently, speculation has focus-ed on classes of polycyclic aromatic hydrocarbons (PAHs),and two collaborative studies between NAS scientists and theAstrochemistry Lab at NASA Ames Research Center haveresulted in significant findings in the way PAHs behave. “Welooked at the neutral, positive, and negative ions of a class ofcompounds called perylene, terrylene, and quaterrylene,”

Meyya Meyyappan, director of the Center for Nanotechnology atAmes, presents an overview of the nanotechnology session of theworkshop. (NASA/Tom Trower)

This photograph of Apollo 8 reentering the Earth’s atmospherewas taken by a U.S Airforce Airborne Lightweight OpticalTracking System camera mounted on a KC-135A aircraft fly-ing at 40,000 feet. NAS researchers are studying how atomsand molecules interact during vehicle reentry. Results from thesestudies will be used in future space vehicle designs.

(NASA/JSC/GRIN GPN-2000-001357)


explains NAS scientist Timothy Lee. “Our theoretical resultsexplained all of the Astrochemistry Lab’s observations.”

Lee’s collaborative paper, “Electronic Absorption Spectra ofNeural Perylene, Terrylene and Quarterrylene and TheirPositive and Negative Ions: Neon Matrix-Isolation Spec-troscopy and Time Dependent Density Functional TheoryCalculations,” is due to appear in the Journal of PhysicalChemistry this year. Results of a similar study, “Time Depen-dent Density Functional Theory Calculation of Large Com-pact PAH Catons: Implications for the Diffuse InterstellarBands,” is due to appear in the Astrophysical Journal.

Modeling the Probability of IonizationIonization – the process by which molecules kick out elec-trons – is important for the survival of mankind, as it pro-vides a protective shield that deflects cosmic rays. Ionizationalso has a variety of other purposes, and in 2002 NAS scien-tists completed a project to develop an accurate model forcalculating the probability of ionization. This model willhelp in the design of silicon chips and in studying how thehuman body reacts when exposed to radiation in space. Theresult, “Total Electron-Impact Ionization Cross-Sections ofCFx and NFx (x = 1-3)” by Winifred Huo, VladimirTranovsky, and Kurt Becker, was published in ChemicalPhysics Letters by Elsevier, May 31, 2002. “The next step is tocalculate the probability of ionization breaking up mole-

cules,” says NAS scientist Huo. “This is important in study-ing the health effects of ionization and how carcinogensdamage DNA.”

NAS Researcher Awarded for New Mesh Refinement Strategy NAS researcher Michael Aftosmis and colleague MarshaBerger from the Courant Institute in New York received theAmerican Institute of Aeronautics and Astronautics (AIAA)best paper award for 2002. The award was presented in Juneat a special luncheon during the 32nd AIAA Fluid DynamicsConference for their joint paper, “Multilevel Error Esti-mation and Adaptive h-Refinement for Cartesian Mesheswith Embedded Boundaries.” This paper describes the team’sdevelopment of new techniques for error estimation andadaptive refinement for CFD solutions, and formed thebasis of a planned adaptation module for NASA’s Cart3Danalysis software.

NAS Scientist Appointed to Springer-VerlagEditorial BoardNAS scientist Timothy Barth was appointed to the editoralboard for the book series Lecture Notes in Computational Sci-ence and Engineering, published by international scientificpublisher Springer-Verlag. Previously, Barth was a contribut-ing author to two editions of the lecture series, which wasestablished in 1997 and is part of the publisher’s mathematics

Chimera Grid Toolsand Pegasus5 NAS scientists William Chan and StuartRogers continued to improve grid gener-ation processes in 2002, with the releaseof version 1.7 of the Chimera Grid Tools(CGT) suite, and Pegasus5 software.

The CGT package contains a variety ofprograms and scripts used in the oversetCFD process for grid generation, diag-nostics, and solution post-processing.The new release contains a module forautomatic boundary conditions selec-tion and a components module, allow-ing a fast and user-friendly way to createthe inputs for simulations involvingmultiple components in relative motion.

Pegasus5 software is used to perform thepreprocessing task of linking together alarge number of randomly overset grids.During 2002, major enhancements weremade to significantly reduce the CFDcycle-time.

For more information on CGT and Pegasus5, visit: http://www.nas.nasa.gov/~rogers/cgt/doc/man.html

This screenshot from the latest version of the Chimera Grid Tools software packageshows an example of a fictitious Space Shuttle Launch Vehicle separation procedure.Each body is shown in dynamic relative motion and contains multiple grids, depict-ed in different colors. (NASA/William Chan, Stuart Rogers)

16 Gridpoints

division. “Springer is embracing computational science andengineering as a new subject,” says Barth, whose duties onthe editorial board include approving new books for theseries and providing input to the senior editor about poten-tial authors and new topics.

Patent for Non-linear Optics SimulationIn addition to awarding NAS scientist Peter Goorjian a dis-tinction in the 2002 NASA Space Act Awards for his patenton photonic switching devices using light bullets, NASAheadquarters granted an exclusive license under the patent toCyrospace, Inc., a technology mining, development, andcommercialization company.

Goorjian’s work studies light pulses, commonly known aslight bullets because they measure approximately ten fem-toseconds, or 1/100,000,000,000,000 of a second. Resultsfrom simulations of light bullets propagating through certainmaterials, like glass or fused silica, can help researchers deter-

mine how these short pulses may be used to make opticalswitches. The next step is for Cyrospace, Inc. to begin creat-ing experiments with light bullets in the laboratory, usingtechniques from Goorjian’s patent.

Center for Turbulence Research’s Summer Program 2002The biennial summer program at the Center for TurbulenceResearch (CTR) welcomed 46 participants from 11 coun-tries, nine states, and 40 institutions worldwide in a collabo-rative effort that brought theorists and experimentaliststogether in a number of turbulence research projects. CTR isa cooperative program between Stanford University andNASA Ames Research Center, and the NAS Division sup-ports the program with supercomputer access and projectadvice and management.

Each of the 32 projects conducted in the summer were basedon researchers’ own proposals that matched with NASA’s

Modeling the SunThe NAS Division’s expertise in turbulence research, com-bined with the power of the division’s supercomputers, isthe driving force behind a collaborative project withStanford University scientists interested in modeling thephysical properties of the sun.

The main goal of the project is to understand the physicsof a region inside the sun called the “convection zone,”where energy produced in the core is transported to thesurface in much the same way as a pot of boiling waterbubbles and rolls. “The Earth’s weather is strongly depend-ent on variations in solar radiation,” explains NAS scien-

tist Alan Wray. “We want to know how these variationsoccur, why, and in what timeframe.” Results of thisresearch may also help NASA protect its astronauts fromspurious solar rays, and ensure communication satellitesstay in optimum orbital positions. This work may alsoinfluence NASA’s search for extra-solar life.

Above: This sequence of images, captured by the NASA/ESASolar and Heliospheric Observatory (SOHO), shows an erup-tive prominence or gaseous emission from the sun measuringmore than 80,000 miles long and traveling at more than15,000 mph. Such events can disrupt power and communi-cations in space. (NASA/GSFC/GPN-2002-000120)


interests. Research focused on nine areas: tur-bulence acoustics, Reynolds-Averaged Navier-Stokes (RANS) modeling, large eddy simula-tion (LES), numerical methods for LES, tur-bulence fundamentals, stratified shear flowturbulence, flow optimization, nanofluidicsand biology, turbulent combustion and sprays.

A summary of the findings will appear in thejournal Physics of Fluids, followed by a full vol-ume of final papers due to be published byCTR this year.

Strained WakesThe ability to predict the flow over a high-liftairfoil using standard CFD codes depends onunderstanding how turbulence behaves in thewake of multicomponents (flaps, spoilers, andleading edge slats) that separate to increase liftfor take-off and landing. NAS scientistMichael Rogers has been simulating strainedwakes – regions of reduced velocity behind anobstacle – and mixing layers where two streamsof different speeds come together, then analyz-ing their effect on the types of turbulence that are found inhigh-lift airfoils. Rogers’ paper, “The Evolution of StrainedTurbulent Plane Wakes,” was published in 2002 in theJournal of Fluid Mechanics, vol. 463, pp. 53-120.

Stratified Turbulent FlowsAnother area where Michael Rogers employs his turbulencemodeling expertise is in helping researchers understand strat-ified turbulence for making global climate models. Stratifiedturbulence is found in the atmosphere around the Earth, par-ticularly relating to cloud physics and formation. DuringCTR’s Summer Program, Rogers participated in several proj-ects analyzing data from numerical simulations, including oneeffort that attempted to distinguish between fluctuatingmotions caused by internal gravity waves and classical strati-fied turbulence.

“Scientists at CTR have been developing new statistics thatwill help with turbulence modeling,” explains Rogers.“Researchers from all over the world are interested in becom-ing more familiar with CTR’s modeling ideas.”

Contrails and Cloud Formation NAS scientists studying aircraft trailing vortices focused onaircraft contrails, the lines that trail across the sky like smokebehind an airplane, in one of CTR’s summer program exper-iments that will ultimately be useful to Earth scientists mod-eling the global climate.

NAS scientist Karim Shariff collaborated with researcherRoberto Paoli, a post-doctorate scientist with the Compu-tational Fluid Group at CERFACS, the European Centre forResearch and Advanced Training in Scientific Computation

in France. Paoli’s research project simulated the interactionof jet exhaust with the vortex of the aircraft, and drewupon prior knowledge gained by researchers in the EarthScience Division at Ames to determine how contrails arti-ficially increase cloudiness and trigger the formation of cir-rus clouds.

Wireless Firewall Gateway GainsNational RecognitionDevelopment of a system that ensures secure interoperabilityof a wireless network, using the 802.11b standard, gainednational recognition when the magazine Government Com-puter News featured an article on the project on the frontcover of their October 8 issue. The Wireless Firewall Gate-way (WFG) promotes easy and secure wireless networkingusing techniques that could be applied inexpensively any-where. NAS network and security team members NicholeBoscia and Derek Shaw worked with Dave Tweten, comput-er security official at NAS, and were interviewed for the arti-cle. Their paper, “Wireless Firewall Gateway,” was also pub-lished in a book titled Wireless Security Essentials.

The NAS Division’s achievements in computational sci-ence and information technology cover a broad spectrumof research and development throughout NASA, andreflect the diversity, drive and determination of each mem-ber of the NAS team. By expanding its high-performancecomputing power and forming collaborative partnershipswith industry, academia, and other NASA centers, NAShas affirmed its leadership position in the future of capa-bility computing.

— Julie Jervis

NASA-operated Boeing 727 with wing tip smoke generators graphically illus-trates the aircraft’s wing tip trailing vorticies. NAS scientists are collaboratingwith partners in France to determine how trailing vorticies affect the cloud for-mation. (NASA/Dryden ECN-3831)

18 Gridpoints

Parallelization — The Key to FasterCodes, Higher Fidelity Simulations

NAS Division researchers create a computer-aided tool to helpautomate the tedious process of parallelizing serial code.

To create better cloud models, researchers are using acomputer code recently parallelized by NASAAdvanced Supercomputing (NAS) Division re-

searchers Henry Jin and Gabriele Jost. Parallelizing the codeenabled scientists at NASA Goddard Space Flight Center,Greenbelt, Md., to run larger cloud simulations in shorterperiods of time, helping to shed light on phenomena such asair-sea interactions and global climate changes.

Jin and Jost worked with scientists Wei-Kuo Tao, DanJohnson, and Chung-Lin Shie of NASA Goddard, toimprove the three-dimensional Goddard Cumulus Ensemble(GCEM3D) Code. To accomplish this work, NAS re-searchers applied their tool, Computer-Aided Parallelizationand Optimizer (CAPO), designed to automate the tediousand error-prone steps of parallelizing serial code, and to takeadvantage of shared-memory parallel computers. “The pur-pose of parallelizing code is to make it run faster. The idea is,if it runs faster on one processor, it should run ‘n’ times as faston ‘n’ number of processors,” explains Jost. Parallelizing codeenables researchers to solve bigger problems – problems thatmay not have been previously solvable due to a lack of timeor available resources.

Building Tools with ToolsDevelopment of the CAPO tool in 1998 was inspired by acollaboration with the developers of CAPTools at theUniversity of Greenwich in the United Kingdom. CAPToolsis software that generates code to run on distributed memo-ry machines. Jin and Jost believed that they could extendCAPTools to produce a parallel program for shared-memorymachines to take advantage of the unique computing envi-ronment at the NAS Facility (see Gridpoints, Spring 2002,“The Evolution of High-Performance Computers in theNASA Advanced Supercomputing Division,” page 1A). “Ithink the CAPO tool is very useful, especially for model runsdemanding lots of CPUs and massive amounts of memory,”says Goddard’s Shie.

The CAPO tool traverses through several steps in order toparallelize a serial code (see Figure 1). First, the tool performsa data dependency analysis (to determine how different vari-ables depend on one another), then a loop level analysis(looking for repeated sequences of instructions in the code),followed by a series of graphical user interfaces that illustratethe parallelization process to the user. “The tool takes awaythe tedious and error-prone work of parallelizing code,allowing the user to focus on optimization of critical parts ofthe code, all through a single interface,” explains Jost.

Dependency analysis, the core part of the CAPO software,determines the relationships between variables within a seri-al code. This process is time consuming in most cases becausecomplex codes have many routines and subroutines (sets ofprogramming instructions designed to perform specifictasks). “Dependency analysis is an important step in theprocess, because you have to decide whether operations can

Figure 1: The process of parallelizing serial code using theCAPO tool begins by importing serial code (top left of figure).The tool then analyses data dependencies to determine the rela-tionship among variables in the code. After the data dependen-cy analysis step, a loop level analysis helps determine how todivide the code to run in parallel. Then, a series of graphicaluser interfaces guides the user through the process of addressingsections of code that cannot be parallelized in original form.


be performed in parallel or not. If memory access is depend-ent on other operations, the operations have to be performedin a certain order [versus running operations in parallel],”explains Jin.

All data generated during the dependency analysis process isstored in a database. If the CAPO system runs into an obsta-cle while conducting the analysis, information about thenature of and reasons for the obstacle are stored in a database.A user can later examine the database and potentially removeobstacles based on his or her knowledge about the applica-tion and its input parameters. “The user guides the tool, butthe tool also guides the user – it’s an interaction,” explainsJost.

Following the dependency analysis, the loop level analysisexamines the code. “CAPO examines the loops for potentialdata dependencies that might prohibit parallelization. If youhave a loop which iterates over some repeated sections of thecode, you can actually break this loop into individual pieces,such that you can run them concurrently on the processors,”explains Jin. “That’s how you get the speed-up in code per-formance.” Speed-up is the ratio of the rate at which work isdone when a job is run on ‘n’ processors to the rate at whichit is done by just one processor.

Once the loop level analysis is complete, users are guidedthrough a series of the tool’s built-in graphical user interfaces.CAPO’s interface enables users to view all instances wherethe code did not parallelize. The more errors or obstacles auser is able to remove, the higher level of parallelization thatcan be achieved (See Figure 4, page 20).

Proof Is in the NumbersBefore applying CAPO to GCEM3D, the cloud modelingcode was only able to run very small cases, scaling up to fourprocessors on a PC. After applying the CAPO tool and mak-ing a few adjustments, Jin and Jost achieved a speed-up ofa factor of 12 when running on 16 processors of an SGIOrigin 3000 for a 130-by-66-by-34 test case (see Figure 3).Using larger cases, the code was able to scale up to 64 CPUs.Test cases were also run on an SGI Origin 2000, a SunEnterprise, and a Dell PC, demonstrating the portability ofthe code CAPO generates. Wherever the OpenMP program-ming model is supported, CAPO-generated code can be run.OpenMP is a portable, scalable model that provides parallelprogrammers a simple and flexible interface for developingshared-memory parallel applications.

The newly parallelized GCEM3D code enabled researchersat Goddard to increase the resolution of their case studies.They successfully ran a large test case of 1,026-by-1,026-by-34, using more than seven gigabytes of memory – a new featusing this application. “Our goal of cloud modeling not onlyaims to better understand the microphysical and dynamicalprocesses of the cloud system itself, but also to improve theirrepresentation for large-scale applications, such as studies onthe precipitating convective system, air-sea interactions andcloud-aerosol interactions, as well as the global change inclimate and hydrology,” explains Shie. Having the ability to

Figure 3: This bar chart represents the contrast in performancebetween the original GCEM3D code (in red), and the parallelCAPO version (in green). This specific example is a 130-by-66-by-34 case run on an SGI Origin 3000. While the original didnot scale beyond four processors (CPUs), the CAPO versionachieved a speedup of 12.4 on 16 CPUs, improving perform-ance of the code by a factor of 8.2. (NASA/IPG)

Figure 2: The different levels of ease-of-use and performance forvarious parallelization approaches are illustrated in this dia-gram. Hand-generated parallel code usually shows very highperformance, however, this method requires a lot of user timeand effort. High-performance Fortran (HPF) or OpenMP areprogramming paradigms to bridge this gap. Compiler-basedautomatic parallelization is easy to use; however, performance ofthe generated code is often limited. Using a parallelization toolsuch as CAPO, considerably eases the workload for users, whilemaintaining acceptable performance of the generated code.

20 Gridpoints

run much larger cases enables NASA Goddard scientists toachieve these research goals faster.

Sticking to What WorksAlthough there are other methods for parallelizing code,CAPO is optimal for achieving performance, ease of use, andautomation with a single tool. Other methods include gen-erating the modified code by hand, using an automatic com-piler, and employing other parallelization tools such as high-performance Fortran and OpenMP (see Figure 2, page 19).The CAPO tool is superior in its ability to pinpoint errorsversus an automatic compiler. “Unlike a compiler, CAPOallows the user to provide the knowledge about code struc-tures or input parameters in order to remove unnecessarydata dependencies,” explains Jin. Using an automatic com-piler to parallelize code will not result in the kind of speed-up the CAPO tool can achieve, he adds.

After successfully parallelizing their cloud modeling code,researchers at NASA Goddard are interested in applying theCAPO tool to some of their other serial codes. “I will applyCAPO to other codes in the future because of the substantialimprovement in model performance due to computationalefficiency and memory extension,” says Shie.

Tao and Shie visited NASA Ames in September 2002 tolearn more about the parallelization tool. And recently, Jinand Jost visited NASA Goddard to demonstrate the tool to agroup of researchers. The CAPO tool team’s eventual goal isto transfer the knowledge of the tool so that each user canapply the tool to codes individually. Decades of effort havebeen consumed generating serial code that now needs to beparallelized so it will run more efficiently on the shared-memory parallel systems at the NAS Facility.

—Holly A. Amundson

Figure 4: These dia-grams represent time-lines of the first few iter-ations of a loop in aroutine for eight threads.(Threads are portions ofa program that can berun independently of,and concurrently with,other portions of theprogram.) Dark bluecoloring indicates a run-ning thread, light blueindicates an idle thread,and red represents inac-tive threads.

Case No. 1, top: Inthe original GCEM3Dcode, only one thread (themaster thread) is activemost of the time, whilethe remaining threadsare idle.

Case No. 2, right: Inthis case, the CAPO-parallelized version ofthe cloud modeling code,all threads are activemost of the time (repre-sented by the dark bluecoloring).

Calendar ofEvents

17th Annual HPCC ConferenceNewport, Rhode Island • March 25–27, 2003The National High Performance Computing and Com-munications Council holds an annual High PerformanceComputing and Communications Conference each spring.This is one of the few conferences that emphasizes commu-nication between manufacturers and users, as well as aca-demics and the government agencies which establish policyand regulate the use of advanced technologies. Topics cov-ered include: wireless computing, e-government, Internetsecurity, grid computing, mass storage, homeland cyber secu-rity, and bioterrorism. For more information, visit:www.hpcc-usa.org/genconf.html

Linux Clusters: The HPC Revolution 2003 Las Vegas, Nevada • June 17–20The Linux Clusters: the HPC Revolution 2003 Conferenceis the premier international forum for Linux cluster users andsystem administrators to share information on applicationsand tools development, scientific computing techniques, andsystems administration of Linux clusters. More informationis available at:www.linuxclustersinstitute.org/Linux-HPC-Revolution

High Performance Distributed ComputingConference (HPDC)Seattle, Washington • June 22–24The Twelfth IEEE International Symposium on High-Performance Distributed Computing will be a forum for pre-senting the latest research findings on the design and use ofhighly networked systems for computing, collaboration, dataanalysis, and other innovative tasks. HPDC-12 will be heldin Seattle, Washington immediately preceding the 8th GlobalGrid Forum. Details are available at:www-csag.ucsd.edu/HPDC-12

International Workshop on Active Middleware ServicesSeattle, Washington • June 25Held in conjunction with the 12th International Symposiumon Grid Computing (HPDC 12), the fifth annual Inter-national Active Middleware Workshop will focus on Auton-omic Computing and will bring together leading researchersand ideas in this emerging discipline. Submission deadlinefor abstracts is February 28. More information can be foundat: www.caip.rutgers.edu/ams2003

SIGGRAPH 2003San Diego, California • July 27–31SIGGRAPH 2003 hosts the largest, most comprehensiveexhibition of products and services for the computer graph-ics and interactive marketplace. Details at:www.siggraph.org/s2003/conference


The IPG Tutorial and Workshop will be held February 4,providing an introduction to the NASA grid and its vari-ous tools. Tutorials will review the general capabilities ofthe IPG, the basic functions available to its users and thosedeveloping IPG applications. The tutorial requires no pre-vious IPG experience.

In contrast to other grid workshops and events, the IPGworkshop is focused specifically on NASA’s InformationPower Grid. The objectives of the IPG workshop are toprovide a forum in which: IPG system developers andimplementors can share their experiences; IPG applicationdevelopers and users can share their experiences; potentialIPG users can gain a broad understanding of its capabili-ties; and IPG research and development personnel and theNSF PACI partners can report on the progress of their gridresearch and development.

Presentations will include talks from research and develop-ment teams from each of the major IPG-related organiza-tions including: NASA centers; Argonne NationalLaboratory; National Computational Science Alliance; San

Diego Supercomputer Center; University of Southern Cali-fornia’s Information Sciences Institute, and other partici-pating universities.

For additional information on this event, visit the website:www.nas.nasa.gov/2003ipg or contact Marcia Redmond at:[email protected].

2003 NASA Information Power Grid (IPG) Tutorial and WorkshopPalo Alto, California • February 4-6

FIRST CLASS MAILPOSTAGE & FEES PAID

NASAPermit No. G-27

National Aeronautics and Space AdministrationNASA Advanced Supercomputing DivisionAmes Research CenterMS IN258-6Moffett Field, CA 94035-1000

Official BusinessPenalty for Private Use $300

Address CorrectionIf the above address or mail stop is incorrect, please e-mail a current address to [email protected], or write to:Gridpoints Editor, MS 258-6, Moffett Field, CA 94035

PLEASE RECYCLEPrinted on recycled and recyclable paper with vegetable-based inks

Parallelization — The Key to FasterCodes, Higher Fidelity SimulationsScientists within the NAS Division have designed a computer-aided tool to help automate the tedious process of parallelizingserial code. See page 18.

Completing Large Parameter Studies –with Time to SpareNAS Division researchers create an automated CFD system to run large parameter studies on IPG resources. See page 4.

www.nas.nasa.gov/gridpoints

Date post:	12-Mar-2021
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

The Quarterly Publication of the NASA Advanced ...The Quarterly Publication of the NASA Advanced...

Documents