APPLIED MATHEMATICS · Research has made significant, enduring advances in applied mathematics that...

APPLIED MATHEMATICS

A Report by an Independent Panel from theApplied Mathematics Research Community

Prepared by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.

DisclaimerThis document was prepared as an account of work sponsored by an agency of the United States government.Neither the United States government nor Lawrence Livermore National Security, LLC, nor any of their employeesmakes any warranty, expressed or implied, or assumes any legal liability or responsibility for the accuracy, com-pleteness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its usewould not infringe privately owned rights. Reference herein to any specific commercial product, process, orservice by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply itsendorsement, recommendation, or favoring by the United States government or Lawrence Livermore NationalSecurity, LLC. The views and opinions of authors expressed herein do not necessarily state or reflect those ofthe United States government or Lawrence Livermore National Security, LLC, and shall not be used for advertis-ing or product endorsement purposes.

LLNL-TR-401536

About the Cover

A cylindrical shock wave produces this intricate flow pattern as it converges from outside sixteencircular obstacles and diffracts. Researchers are interested in the structure of the solution near the focus(green region). This high-resolution numerical solution was computed using adaptive overlapping meshes.Source: Center for Applied Scientific Computing, Lawrence Livermore National Laboratory.

APPLIED MATHEMATICS

AT THE U.S. DEPARTMENT OF ENERGY:

Past, Present and a View to the Future

A Report by an Independent Panel from the

Applied Mathematics Research Community

May 2008

Panel Members:

David L. Brown (chair), Lawrence Livermore National Laboratory

John Bell, Lawrence Berkeley National Laboratory

Donald Estep, Colorado State University

William Gropp, University of Illinois Urbana-Champaign

Bruce Hendrickson, Sandia National Laboratories

Sallie Keller-McNulty, Rice University

David Keyes, Columbia University

J. Tinsley Oden, The University of Texas at Austin

Linda Petzold, University of California, Santa Barbara

Margaret Wright, New York University

Table of Contents

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1. Applied Mathematics for the Department of Energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.1 DOE Mission Drivers and Mathematical Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.2 Historical Successes of the DOE Applied Mathematics Program . . . . . . . . . . . . . . . . . . . . . . . . . . .7

1.3 Future Success of the DOE Applied Mathematics Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2. Advancing Mathematics for Modeling, Simulation, Analysis and Understanding of Complex Systems . . . . . . . . . . . . . . 9

2.1 Predictive Modeling and Simulation of Complex Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.1.1 Multiscale, Multiphysics, and Complex Hybrid Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.1.2 The Role of Data-Model Fusion in Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.1.3 Modeling Stochastic Effects in Complex Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.1.4 Networks, Systems and Systems of Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.2 Analyzing the Behavior of Complex Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.2.1 The Role of Data in the Analysis and Understanding of Complex Systems . . . . . . . . . . . . . 162.2.2 Sensitivity Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.2.3 Uncertainty Quantification and Mitigation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.3 Using Complex Systems to Inform Policy-Making . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.3.1 Risk Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.3.2 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.3.3 Inverse Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3. Driving Innovation and Discovery in Applied Mathematics Through Effective Program Leadership . . . . . . . . . . . . . . . 25

3.1. Encouraging and Rewarding Risk-Taking in Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.2. Building Effective Connections with Science and Engineering. . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.3. Leveraging the Expertise in National Laboratories and Universities . . . . . . . . . . . . . . . . . . . . . . 26

3.4. Connecting Applied Mathematics and Advanced Computing. . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Epilogue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Applied Mathematics at the U.S. Department of Energy: Past, Present and a View to the Future

III

the mathematical developments required to meetthe future science and engineering needs of theDOE. It is important to emphasize that the pan-elists were not asked to speculate only on advancesthat might be made in their own research special-ties. Instead, the guidance this panel was given wasto consider the broad science and engineering chal-lenges that the DOE faces and identify the corre-sponding advances that must occur across the fieldof mathematics for these challenges to be success-fully addressed. As preparation for the meeting,each panelist was asked to review strategic planningand other informational documents available forone or more of the DOE Program Offices, includingthe Offices of Science, Nuclear Energy, FossilEnergy, Environmental Management, LegacyManagement, Energy Efficiency & RenewableEnergy, Electricity Delivery & Energy Reliabilityand Civilian Radioactive Waste Management as wellas the National Nuclear Security Administration.The panelists reported on science and engineeringneeds for each of these offices, and then discussedand identified mathematical advances that will berequired if these challenges are to be met.

A review of DOE challenges in energy, theenvironment and national security brings to light abroad and varied array of questions that the DOEmust answer in the coming years. A representativesubset of such questions includes:

• Can we predict the operating characteristics ofa clean coal power plant?

• How stable is the plasma containment in atokamak?

• How quickly is climate change occurring andwhat are the uncertainties in the predicted timescales?

• How quickly can an introduced bio-weaponcontaminate the agricultural environment inthe US?

• How do we modify models of the atmosphereand clouds to incorporate newly collected dataof possibly of new types?

• How quickly can the United States recover ifpart of the power grid became inoperable?

Over the past half-century, the AppliedMathematics program in the U.S. Department ofEnergy’s Office of Advanced Scientific ComputingResearch has made significant, enduring advancesin applied mathematics that have been essentialenablers of modern computational science.Motivated by the scientific needs of the Departmentof Energy and its predecessors, advances have beenmade in mathematical modeling, numerical analysisof differential equations, optimization theory, meshgeneration for complex geometries, adaptive algo-rithms and other important mathematical areas.High-performance mathematical software librariesdeveloped through this program have contributed asmuch or more to the performance of modern scien-tific computer codes as the high-performance com-puters on which these codes run. Funding from thisprogram has assisted generations of graduate andpostdoctoral students who have continued on topopulate and contribute productively to researchorganizations in industry, universities and federallaboratories. The combination of these mathemati-cal advances and the resulting software has enabledhigh-performance computers to be used for scientif-ic discovery in ways that could only be imagined atthe program’s inception.

Our nation, and indeed our world, face greatchallenges that must be addressed in coming years,and many of these will be addressed through thedevelopment of scientific understanding and engi-neering advances yet to be discovered. The U.S.Department of Energy (DOE) will play an essentialrole in providing science-based solutions to manyof these problems, particularly those that involvethe energy, environmental and national securityneeds of the country. As the capability of high-performance computers continues to increase, thetypes of questions that can be answered by apply-ing this huge computational power become morevaried and more complex. It will be essential thatwe find new ways to develop and apply the mathe-matics necessary to enable the new scientific andengineering discoveries that are needed.

In August 2007, a panel of experts in applied,computational and statistical mathematics met for aday and a half in Berkeley, California to understand

Executive Summary

EXECUTIVE SUMMARY 1

• What are optimal locations and communicationprotocols for sensing devices in a remote-sens-ing network?

• How can new materials be designed with aspecified desirable set of properties?

In comparing and contrasting these and otherquestions of importance to DOE, the panel foundthat while the scientific breadth of the require-ments is enormous, a central theme emerges:Scientists are being asked to identify or providetechnology, or to give expert analysis to informpolicy-makers that requires the scientific under-standing of increasingly complex physical andengineered systems. In addition, as the complexityof the systems of interest increases, neither experi-mental observation nor mathematical and computa-tional modeling alone can access all components ofthe system over the entire range of scales or condi-tions needed to provide the required scientificunderstanding.

These observations motivated the panel to artic-ulate a new framework for describing the appliedmathematical developments that will be required toaddress future scientific and engineering challengesfor DOE. Two significant reasons make this newframework valuable. The first is that it aligns thethinking about future mathematics research forDOE around the most difficult scientific and engi-neering problems that the DOE faces in comingyears. The second is that it identifies needed math-ematical advances in a way that clearly articulateshow current strengths in the DOE AppliedMathematics program can be leveraged and, per-haps more importantly, where significant gaps existin the mathematics that must be developed tounderstand these complex systems. Expressed interms of applied mathematics research topics, thesefindings can be summarized as follows:

• DOE must leverage the current strengths of theApplied Mathematics program in predictivesimulation and modeling, expanding thesecapabilities to more fully address simulationand modeling for complex systems. Areas thatare not adequately developed include thedevelopment and analysis of methods to modellarge stochastic systems, and techniques fordecomposing complex systems into systems ofcanonical subsystems. The mathematical

underpinnings of sensitivity analysis, uncertain-ty quantification, risk analysis, optimizationand inversion must also be significantlyexpanded to address the challenges presentedby complex systems.

• It is essential that the program increasinglyfocus its perspective on the end goal of devel-oping mathematical approaches for understand-ing the complex systems themselves. Thisrecognizes that breaking problems down intosimpler components cannot be the only mathe-matical approach, but must be complementedwith the development of modeling, simulationand analysis tools that deal with the full com-plex systems.

• The mathematics for analyzing very largeamounts of data, whether produced by simula-tions or through experimental observations,requires serious development. A particularchallenge is to enhance the theory and tools fordata-model fusion for complex systems, whereobservational or experimental data are incorpo-rated in an essential way with simulation andmodeling.

The full panel report describes these challengesin terms of three encompassing themes togetherwith high-level strategies for addressing the gaps inour understanding. These are summarized below:

1. Predictive modeling and simulation ofcomplex systems

Advance the fidelity, predictability and sophisticationof modeling and simulation methodologies for complexsystems:

• Develop analytical and computationalapproaches needed to understand and modelthe behavior of complex multiphysics, and mul-tiscale phenomena.

• Enhance the theory and tools for complex mul-tiscale, multicomponent models when observa-tional or experimental data are incorporated inan essential way.

• Develop new approaches for efficient modelingof large stochastic systems.

• Develop mathematical techniques for decom-posing complex systems into systems of canoni-cal subsystems and modeling their behavior.

2 EXECUTIVE SUMMARY

2. Mathematical analysis of the behaviorof complex systems

Address the challenges of analyzing and understandingthe behavior of mathematical models for complexscientific and engineering systems:

• Develop sound, computationally feasible strate-gies and methods for the collection, organiza-tion, statistical analysis and use of dataassociated with complex systems.

• Advance the theory and tools for sensitivityanalysis to address the challenges posed bycomplex multiscale, multicomponent models.

• Significantly advance the theory and tools forquantifying the effects of uncertainty andnumerical simulation error on predictionsusing complex models and when fitting com-plex models to observations.

3. Using models of complex systems toinform policy makers

Develop the mathematics needed to inform policy mak-ers based on the prediction, optimization and under-standing of complex systems:

• Significantly advance the mathematics that sup-ports risk analysis techniques for policy-mak-ing involving complex systems that includenatural and engineered components, and eco-nomic, security and policy consequences.

• Develop techniques for formulating, analyzingand solving challenging optimization problemsarising in complex natural and engineered sys-tems.

• Develop techniques for addressing the mathe-matical and computational difficulties ofinverse problems associated with complexsystems.

In addition to developments in fundamentalmathematics in these areas, the program mustcontinue to develop the high-performance softwarethat translates mathematical advances for complexsystems into enabling computational tools that canbe used by computational scientists. This will alsodepend on complementary investments focused onunderstanding and exploiting the latest high-per-formance hardware developments as they are antici-pated and become available.

Finally, the panel paid special attention to theopportunities that exist for innovative approaches inprogram leadership that can promote innovation anddiscovery in applied mathematics. The panel reportrecommends that the program encourage risk-takingand innovation in research by taking steps thatreduce the perceived risk to investigators while atthe same time ensuring accountability for theirresearch activities. Rewarding investigators whoconsistently produce innovative and mission-relevantresearch results with continued long-term supportencourages risk-taking and recognizes that “transfor-mational” science is often the result of the concen-trated long-term development of understanding.Introducing new ideas into research efforts is alsoan essential element of a successful program. Thiscan be enhanced by providing opportunities foryoung researchers to participate in the program,and for researchers to interact and collaborate inintense environments such as workshops and sum-mer schools. Finally, the effective application ofmathematics requires deep understanding of theunderlying scientific and engineering applications.By encouraging and supporting early interaction andcollaboration of mathematicians with their colleaguesin science and engineering fields, the program willaccelerate the development of successful applicationsof mathematics to the scientific and engineering chal-lenges that the DOE must address for the future.

EXECUTIVE SUMMARY 3

4 EXECUTIVE SUMMARY

1. Applied Mathematics for the Department of Energy

nuclear waste storage and environmental cleanupproblems all require high-fidelity models for sub-surface flow that integrate the inherent uncertain-ties in subsurface characterization with anintegrated capability for risk management10.Climate modeling involves a number of similarconsiderations and introduces additional elementsof data assimilation to integrate observational datadirectly into the simulation process.

Other mission-critical applications require sub-stantial improvements in the ability to mathemati-cally model behavior at microscopic or quantummechanical scales. The design of systems to har-ness biochemical processes for the production ofbiofuels requires new approaches for modelingexcited states of chemical systems11. The design ofnew materials at the nanoscale, such as high tem-perature superconductors, requires new, more fun-damental approaches to computing the propertiesof materials from first principles12. Understandingthe behavior of biological systems at microscopicscales requires methodologies for capturing theeffect of fluctuations in nonequilibrium thermody-namic systems.

A class of applications of emerging importanceto DOE focuses on the behavior of complex net-works. Typical of this type of problem are issues ofreliability and security of the nation’s electricalpower grid. In this case, the system is a hierarchi-cal network with independent entities controllinglocal subnets. System-level models at scales rangingfrom individual power plants to the internationalenergy distribution systems raise problems in poli-cy-making and risk analysis. Another exampleinvolves the logistical issues arising in the trans-portation of nuclear waste, which are characterizedby discrete optimization problems13.

Data are no longer simple and researchers findthemselves grappling with ever-increasing amountsof data14. Today, DOE applications such as climatemodeling can bring massive amounts of heteroge-neous data together that must be understood bothalone and in the context of complex simulations.High-energy physics experiments will soon produceenormous amounts of data that must be managed,transmitted and analyzed. Biology is becoming an

1.1 DOE mission drivers and mathematicalchallenges

As the U.S. Department of Energy (DOE)works to meet its energy, environmental andnational security missions, increasingly complexscientific and technological challenges must beaddressed. New energy technologies will berequired both for tapping the potential of newsources of energy and for effectively utilizing exist-ing energy resources. Policy makers will need tounderstand quantitatively the impact of energypolicies on the environment and be able to evaluatethe risks associated with different strategies forwaste storage and environmental cleanup. DOEmust also ensure the safety and reliability of thenuclear stockpile while preventing the proliferationof nuclear materials1.

Applied mathematics has an increasinglyimportant role to play in the support of theseexpanding scientific and technological challenges.Where simple models, simple approximations andone-off solutions once sufficed, the need now existsto develop, simulate, analyze and understand mod-els for complex combinations of processes withmultiple scales in space and time2,3,4,5. Policy mak-ers need science-based analyses of these complexmodels, supported by clearly defined margins ofuncertainty and statistical characterizations. Whenobservational or experimental measurements areavailable, these must be incorporated with mathe-matical rigor into the scientific analyses that sup-port the creation of critical national policy. Thepurpose of this report is to provide a description ofthe applied mathematics advances that will berequired to help answer the many mission-criticalquestions that DOE faces in the coming years.

Although the issues associated with meetingthe DOE mission span an enormous range of appli-cations, from a mathematical perspective a numberof specific themes emerge. Next-generation nuclearpower plants6, fusion reactors7,8, clean coal tech-nologies and new engine designs all require an abil-ity to model systems that incorporate a variety ofphysical processes across a broad range of scales.9

Carbon sequestration, enhanced oil recovery,

APPLIED MATHEMATICS FOR THE DEPARTMENT OF ENERGY 5

increasingly quantitative science with the adventof unprecedented high-throughput experimentalmeasurement techniques, and correspondingly theability to digest and analyze huge amounts of het-erogeneous data has become essential for scientificdiscovery in the biological arena. It is thereforecrucial that sound strategies and methods be devel-oped for the collection, organization, analysis, andrepresentation of the results for extremely hetero-geneous and diverse sources of information. Thesemethodologies must integrate the diverse sources ofdata and information and their associated uncer-tainties to develop full distributions for perform-ance metrics that can aid DOE policy-making inthe face of uncertainty.

Each of the problem areas discussed above iscomplex and multifaceted and will require collabo-rations that encompass a broad range of expertise.Each solution is a multi-step process embodying anumber of elements. In many cases new and appro-priate mathematical models need to be formulated.These models must incorporate the appropriatecoupling between disparate length and time scalesand between fundamentally different types ofprocesses, and may also need to incorporate rele-vant experimental or observational data. Some typeof solution or approximation procedure needs to bedeveloped to quantify the behavior of the model.

This quantification should include a characterizationof inherent uncertainties in the model. The behaviorof the model must be analyzed in the context ofavailable data that characterizes the observedbehavior of the system. This type of analysis canrange from statistical behavior of simulation data tothe solution of inverse problems to dynamic dataassimilation that integrates simulation with observa-tion. Finally, the models that have been developedcan be used for design optimization, risk analysisand policy-making.

Applied mathematics has a substantial role toplay in all aspects of the solution process. The DOEApplied Mathematics Program has traditionallyplayed a strong role in the development of numeri-cal methods for differential equations, numericallinear algebra and optimization. Developing method-ologies for solving the classes of problems discussedabove will certainly build on that expertise; how-ever, applied mathematics must be involved morebroadly in the overall solution strategy. Appliedmathematicians must work collaboratively withdomain scientists and engineers in the entireprocess from problem formulation through theanalysis and integration of data to the design/ poli-cy-making process. New mathematical approacheswill be required at each step in the solution processto adequately address DOE mission requirements.

FIGURE 1. Accelerating a thermonuclear flameto a large fraction of the speed of sound (possi-bly supersonic) is one of the main difficulties inmodeling Type Ia supernova explosions, whichlikely begin as a nuclear runaway near the cen-ter of a carbon-oxygen white dwarf. Algorithmsdeveloped under the auspices of the DOEApplied Mathematics program were used toperform this simulation showing an unstableoutward propagating flame. Source: Center forComputational Science and Engineering,Lawrence Berkeley National Laboratory

6 APPLIED MATHEMATICS FOR THE DEPARTMENT OF ENERGY

1.2 Historical successes of the DOE AppliedMathematics program

The use of mathematical and computationalmodels to simulate physical events or the behaviorof engineered systems is arguably one of the mostimportant developments in science and technologyof the past century. Today, computational modelsbased on mathematical characterizations of theoryenable scientists and engineers to predict the behav-ior of extremely complex natural and human-madesystems and provide a basis for creating policy criti-cal to the competitiveness and well being of thenation. The DOE Applied Mathematics program hasmade substantial contributions to this development.

The growth over the past decades in the rawprocessing speed of computer processing units hasbeen staggering. However, as impressive as thisgrowth in computing power has been, our modernscientific computing capability would not havedeveloped without an equally important investmentin the underlying enabling mathematics. John vonNeumann recognized this essential factor, and inthe early 1950s, having become a Commissioner forthe U. S. Atomic Energy Commission, asked LosAlamos computer scientist John R. Pasta to create acontract research program for applied mathematicsand computer science. This AEC program, whichfunded research mathematicians and computerscientists at the AEC Laboratories and at U.S. uni-versities, was the beginning of the present-day DOE

Applied Mathematics program. During its 50 yearsof existence, the Applied Mathematics program hasbeen responsible for supporting fundamental math-ematical developments that have substantiallyadvanced research in many scientific fields. Thestrength of these mathematical results derives notonly from the development of algorithms that trans-late scientific theory into discrete equations that acomputer can solve, but in the mathematical andnumerical analysis that provides the basic under-standing of both the scientific theories and theirnumerical counterparts.

The impact of the Applied Mathematics pro-gram on the field of fluid dynamics is one signifi-cant example of these contributions. In the early1950s, researchers were interested in fluid behaviorin regimes that allowed the formation of shocks—very rapid transitions in fluid velocity and pressure.Motivated by the need to understand and simulatethese shock physics problems, Peter Lax and his col-leagues at the Courant Institute of MathematicalSciences began to look in more detail at the theoryof hyperbolic systems of conservation laws, thecategory of nonlinear partial differential equationsthat describes the behavior of shocks and otherwave-like phenomena in physics. A substantialbody of work, much of it funded by the AppliedMathematics program, was summarized by Lax inhis 1973 SIAM monograph entitled “HyperbolicSystems of Conservation Laws and the MathematicalTheory of Shock Waves.15” Lax provided us with anelegant description of the theory for generalizedsolutions to the initial value problem for nonlinearhyperbolic systems that answers many of the basicstructural questions about solutions to shock prob-lems. This theory provided the foundation neededfor applied mathematicians in the 1970s and 1980sto develop sophisticated new methods for solvingsystems of conservation laws that form the core ofmost modern compressible flow simulation codesused at universities and at DOE and other govern-ment research institutions. Additional mathematicaldevelopments, also funded in part by the AppliedMathematics program, established the framework toextend these types of approaches to a broader rangeof problems, including subsurface flow, combustionand atmospheric modeling.

The DOE Applied Mathematics program hasalso had significant impact on the field of mathe-matical optimization. For more than 30 years, DOE

FIGURE 2. Moore’s original 1965 graph showing the cost per transistor asa function of the number of transistors on a chip. Moore predicted that thenumber of transistors at the curve minimum, which represents the mostefficient chip capacity, would double every two years.

APPLIED MATHEMATICS FOR THE DEPARTMENT OF ENERGY 7

has supported research on a broad spectrum of top-ics in optimization, including theory and softwaredevelopment, with emphasis on their fruitful inter-play. Researchers at Argonne National Laboratory,in collaboration with university colleagues, devel-oped a substantial body of theory that was immedi-ately, and still is, regarded as the foundation ofnonlinear optimization. This theory in turn led tothe development of the unconstrained optimizationmethods that form the mainstay of modern opti-mization. An integral part of the Argonne group’saccomplishments was software based on soundmathematical theory. The MINPACK-1 package wasamong the first to include reliable routines forunconstrained optimization, nonlinear equationsand nonlinear least squares. Another group atStanford University developed both theory andsoftware for constrained optimization problemswith an emphasis on explaining the behavior inpractice of the implemented methods. All of thissoftware has been used to solve problems from avariety of disparate applications, including the opti-mal power flow problem, real-time optimization ofa hydroelectric plant, optimal control in networks,calculation of beam emittances in a particle accel-erator, and trajectory optimization.

1.3 Future success of the DOE AppliedMathematics program

Fluid dynamics and optimization are just twoexamples of areas on which this program has hadenormous impact. Similar stories can be told aboutthe advances in numerical linear algebra, dynami-cal systems, bifurcation theory, and many otherareas. Future success of the program will be tiedto our ability to make substantial investments toreach the vision of using high-performance-com-puting-based predictive simulation and analysis ofcomplex natural and engineered systems as a basisfor policy-making involving critical energy, envi-ronmental and national security issues. Thesecompelling opportunities for new mathematicaland statistical approaches to DOE’s research chal-lenges are predicated upon continued strength infundamental areas of applied and computationalmathematics. Areas such as the solution of ordi-nary differential equations, computational geome-try, adaptive meshing, harmonic analysis, graphalgorithms, and linear algebraic systems have mademajor progress in past decades in both theory andhigh-quality software. Nonetheless, there is a clearneed for continued development of these and othermore mature mathematical areas at their frontiersto meet the vision just articulated.

The next chapter examines DOE’s futurerequirements in detail and identifies some of theprototypical problem areas where new research inapplied mathematics can significantly advance ourability to find scientific and engineering solutionsto pressing problems of national interest. From thissurvey of problems we extract a number of funda-mental mathematical areas where new mathemati-cal research is needed and identify specificresearch opportunities that would significantlyadvance our capabilities in these areas. Theseadvancements can, in turn, help meet mission-criti-cal needs throughout DOE. In the final chapter weidentify some of the guiding principles for ensuringthe effectiveness of a research program in appliedmathematics and maximizing its impact on DOEmission problems.

1001970 1980 1990 2000 2010

101

102

103

Effe

ctive

sust

aine

dsp

eed

ineq

uiv.

giga

flops

104

105

106

Full EarthSimulator(Japan) improved

electronmodels

improvedlinearsolvershigh-orderelementssemi-

implicit

partially-implicit

delta-f,magneticcoordinates

gyro-kinetics

Micro-turbulenceeffective speed

Global MHDeffective speed

Effective speedfrom hardwareimprovementsalone

1000 NERSCSP3 processors(typical)16 processorCray C90

Cray YMP

FIGURE 3. Magnetic fusion energy “effective speed” increases came fromfaster hardware, better mathematical models and improved computationalalgorithms. Source: A Science-Based Case for Large-Scale Simulation(SCaLeS) Volume 2

8 APPLIED MATHEMATICS FOR THE DEPARTMENT OF ENERGY

For centuries, mathematics has provided thelanguage for expressing scientific theories thatdescribe how nature behaves. Now, mathematically-based computational models enable scientists tomove beyond using mathematics for its descriptivecapabilities alone. Modern applied mathematicsenables the prediction and understanding of thebehavior of extremely complex natural and human-made systems and provides a basis for creating pol-icy critical to the competitiveness and well being ofthe nation.

For the discussion below, it will be useful tohave a common understanding of what we mean bya “complex system.” A complex system is a collectionof multiple processes, entities or nested subsystemswhere the overall system is difficult to understandand analyze because of the following properties:

• The system components do not necessarilyhave mathematically similar structures andmay involve different scales in time or space;

• The number of components may be large,sometimes enormous;

• Components can be connected in a variety ofdifferent ways, most often nonlinearly and/orvia a network. Furthermore, local and system-wide phenomena may depend on each other incomplicated ways;

• The behavior of the overall system can be diffi-cult to predict from the behavior of individualcomponents. Moreover, the overall systembehavior may evolve along qualitatively differ-ent pathways that may display great sensitivityto small perturbations at any stage.

Such systems are often described as “multicom-ponent systems,” or when the components arephysics based, “multiphysics systems.” When thecomponents involve multiple spatial or temporalscales, the adjective “multiscale” can be used as well.

We also mean to discuss complex systems inthe broadest possible sense. Examples of systemsthat we view to be complex include:

• Problems that involve a single physical system,which becomes complex when modeled using amultiscale approach. An example occurs in the

composite design of materials when a hybriddiscrete-continuum model is used to describeatomistic-macroscopic phenomena.

• Problems that involve the coupling of multiplephysical processes described with different models.An example occurs in the modeling of carbonsequestration, where a quantitative study mayrequire the simulation of multiple fluid phases,geomechanics, and a complex set of biogeo-chemical reactions over a wide range of scales.

• Problems that describe complex engineered sys-tems. An example is the electric power grid,where models may involve inequality andother types of constraints, severe nonlinearitiesand discontinuities, a mixture of continuousand integer variables, a large number of vari-ables, a huge range of scales, and non-uniquesolutions that may make it difficult to charac-terize the most physically reasonable result.

Careful observation has always been the main-stay of scientific and engineering discovery. With

2. Advancing Mathematics for Modeling, Simulation,Analysis and Understanding of Complex Systems

FIGURE 4. Understanding complex systems will require a combination ofexperimental and computational techniques. Left: experimental images ofthe diffraction pattern from a cylindrically converging shock that hitsseven obstacles. Right: images from an adaptive numerical simulation,showing excellent agreement with the experiment. Source: Center forApplied Scientific Computing, Lawrence Livermore National Laboratory

Experiment Computation

ADVANCING MATHEMATICS FOR MODELING, SIMULATION, ANALYSIS AND UNDERSTANDING OF COMPLEX SYSTEMS 9

the development of modern mathematical and com-putational models for complex systems, computa-tional prediction now complements detailedexperiment and observation as a tool for developingscientific understanding. However, as the systemswe are interested in become ever more complex,neither experimental observation nor mathematicaland computational modeling alone can access allcomponents of a complex system over the entirerange of relevant scales. Reductionist approachesthat seek to obtain understanding by breaking com-plex systems into simpler components may missemergent, and often significant behavior that arisesdue to the interaction of these components in non-linear ways. Researchers are now recognizing thatthrough the fusion of observational data and predic-tive model-based simulation, a much fuller pictureof the behavior of complex systems can beobtained. The “glue” that can provide this “data-model fusion” is based on applied, statistical andcomputational mathematics.

The scientific and technical issues that DOEmust address over the coming decades pose signifi-cant challenges. The remainder of this chapter dis-cusses the advanced predictive tools needed tosupport effective policy-making by our nation’sleaders. These tools will require new, rigorouslyjustified mathematical developments in predictivemodeling, simulation, analysis and understandingof complex systems.

2.1 Predictive modeling and simulation ofcomplex systems

Advance the fidelity, predictability and sophistica-tion of modeling and simulation methodologies bydeveloping the mathematical tools needed for theanalysis and simulation of complex systems char-acterized by combinations of multiple length andtime scales, multiple processes or components.

The basis for policy-making based on predict-ing and understanding complex systems depends inan essential way on the development of mathemati-cally rigorous, scientifically based models for thosesystems. This section discusses the mathematicaldevelopments that will be required to build multi-scale, multiphysics and complex hybrid models. Italso considers the role of data-model fusion and sto-chastic effects in these models. Finally, it discussesthe decomposition of complex systems into systems

of canonical subsystems and the modeling of theirbehavior.

Advances in modeling complex systems willenable DOE to answer questions such as:

• Can we predict the operating characteristics ofa clean coal power plant?

• Can we modify chemical pathways in a plantto produce biodiesel?

• How do impurities affect the performance of amembrane in a hydrogen fuel cell?

• What are the performance characteristics ofpossible nuclear fuel sources for the next-gen-eration nuclear reactor?

• What is the predicted land contact and force ofan Atlantic hurricane?

While modeling and simulation of complex sys-tems has provided enormous benefits throughoutscience and engineering within DOE, there is grow-ing evidence that new developments in this disci-pline could have profound effects on scientificdiscovery and on technologies in diverse fields ofapplied science. Indeed, with mathematicaladvances in the areas discussed below, new model-ing and simulation methods could revolutionize theway science and engineering is done, complementtraditional observational science, enrich or displacetraditional experiments, and expand the vistas of

FIGURE 5. In the wake of Hurricane Katrina, U.S. Army Corps of Engineersused supercomputers at DOE’s National Energy Research ScientificComputing Center to simulate storm surges as a step toward improvingprotective levees. The Overview simulation shows elevated storm surgesalong the Gulf Coast, while the simulation detail shows highest surge ele-vation (in red) striking Biloxi, Miss. New Orleans is the dark blue crescentto the lower left of Biloxi. Source: U.S. Army Corps of Engineers.

10 ADVANCING MATHEMATICS FOR MODELING, SIMULATION, ANALYSIS AND UNDERSTANDING OF COMPLEX SYSTEMS

scientific discovery beyond that otherwise achiev-able. We shall describe several areas in which fun-damental research in applied mathematics can beleveraged to lift modeling and simulation to a newlevel and to equip many ongoing DOE programswith powerful tools critical to achieving their mis-sion.

2.1.1 Multiscale, multiphysics, and complexhybrid models

Develop analytical and computational approachesneeded to understand and model the behavior ofcomplex, multiphysics and multiscale phenomena.

In traditional science, our knowledge of thephysical universe is generally partitioned into spe-cific scales, spatial and temporal, from parsecs andmillennia representing the cosmos to angstromsand femtoseconds measuring events at atomic lev-els. The science of today and the future, however,must cope with events that transcend many scales.In most cases, we are lacking the mathematicalfoundations to make this transition. This “tyrannyof scales,” as some call it, is arguably the mostimportant and difficult area of research facingadvances in modeling and simulation. This relative-ly new field of research is referred to as multiscalemodeling16,17,18. It attempts to develop broad theo-ries of physical behavior that connect events atmany levels, and has become a key issue in manyimportant applications, including the design andanalysis of advanced materials, nano-manufactur-ing, biological systems, drug design and delivery,and environmental models.

A parallel area of research of increasing impor-tance is multiphysics modeling and simulation.Today’s problems, unlike traditional science andengineering, do not involve physical processes cov-ered by a single traditional discipline of physics orthe associated mathematics. Complex systemsencountered in virtually all applications of interestto DOE involve many distinct physical processes.For example, a complete computational model of alarge-scale fusion device is a complex systeminvolving issues of fluid dynamics, deformation ofsolid materials, thermal effects, ablation, fracture,corrosion and aging of materials, radiation andmany other phenomena. All of these multiscaleevents must be properly modeled and interconnect-ed for a viable predictive model of the behavior ofthe entire system. This requires new mathematical

methods, new algorithms, and perhaps most impor-tantly, significant new reformulations of appropri-ate models.

The issue of coupling models of differentevents at different scales and governed by differentphysical laws is largely wide open and representsan enormously challenging area for futureresearch. Many of these problems involve theintermingling of discrete models with continuummodels. For example, discrete molecular and atom-istic models may be developed to study phenomenain nanodevices, material defects, or particulateflows, but these must be coupled in some appropri-ate sense with macroscale models that depict themedium as a continuum. These hybrid discrete-continuum models, the multiscale events governingtheir behavior, and the different physical theoriesunderlying the coupled components representchallenging but vital areas of research in appliedmathematics.

FIGURE 6. The Tramonto code was used to calculate the structure of afluid bilayer as well as the surrounding water in the vicinity of an assemblyof antimicrobial peptides based on a coarse-grained molecular model. Thefigure, based on 3-dimensional fluids-density functional theory calcula-tions, is remarkable because it captures four distinctly different types offluid structure simultaneously. These Fluids-DFTs can be characterized asintegral equations of finite range that are neither sparse nor dense matrixproblems. The theories are solved with a specialized segregated Schurcomplement approach developed at Sandia National Laboratories (SNL) forthis class of problems. The particular solution shown in the figure wasobtained as a result of coupling the Fluids-DFT algorithms with LOCA (theLibrary of Continuation Algorithms – also developed at SNL). The calcula-tions were performed on 256 processors of the Red Squall platform.Source: Sandia National Laboratories


Another source of complexity arises not frommultiplicity of physical process or nonlinearities,but from the dimensionality of the problem.Prototypical of this source of complexity are prob-lems requiring the accurate modeling of quantummechanical effects. Quantum mechanical systemshave three spatial dimensions for each atomic par-ticle in the system. Traditionally, assumptions areintroduced at the level of the formulation to reducethis dimensionality. Mathematical techniques areneeded to derive computational approximationsthat are appropriate when traditional assumptionsare not applicable.

Specific strategies for meeting these challengesare to:

• Develop analytical tools for decomposing com-plex, multiphysics systems into their compo-nent processes and for elucidating the couplingbetween these component pieces;

• Develop methodologies for representing behav-ior at fine scales in models for the system atlarger scales. Develop the corresponding ana-lytical tools and computational approachesneeded to quantify the impact of the fidelity offiner-scale models on large-scale dynamics;

• Develop algorithmic techniques appropriate foremerging computer architectures for simulating

multiphysics and multiscale processes withquantifiable fidelity;

• Develop and analyze numerical methods forhybrid models that couple continuum and dis-crete processes. How do changes in the discretevariables affect the accuracy of the continuumpart of the model?

• Develop approaches for deriving computation-ally tractable approximations to systems thatare formulated in very high dimensionalspaces, such as those arising in quantummechanics.

2.1.2 The role of data-model fusion in predictionEnhance the theory and tools for complex multi-scale, multicomponent predictive models whenobservational or experimental data are incorpo-rated in an essential way.

The fusion of observational and experimentaldata with advanced simulation promises to providegreater understanding of complex multicomponentmultiscale systems than either approach alone canachieve. For example, experimental measurementsare limited both in terms of the kind and frequencyof observations that can be taken and may onlyprovide access to limited aspects of a complexsystem. Predicting the long-term safety of storing

FIGURE 7. A multi-physics multi-domaincomputation of thermal-hydraulics conju-gate-heat-transfer involving fluid flow pastnine fuel rods in a simulated nuclear reactor.The interior of each fuel rod is modeled as asolid with heat transfer and is coupled to anincompressible fluid that flows from bottomto top in the region around the rods. Source:Center for Applied Scientific Computing,Lawrence Livermore National Laboratory


of the current and past states of a system are incor-porated into the simulation regularly with theobjective of producing a data-informed analysisthat can predict states outside the observationrange. This approach has been used successfully toimprove weather forecasts, and has potential broadapplicability to many DOE applications.

The general state of the art in theory and com-putational methodology for data-model fusion iswoefully underdeveloped, despite significant suc-cesses in particular applications. Although thereare many approaches, as yet there is no standardsoftware. What is needed are rigorous mathemati-cal frameworks and efficient, robust implementa-tions for data assimilation, including thedevelopment of suitable metrics for comparingmodel output to system observation. Newapproaches that move past the standard methodolo-gy, e.g., based on Bayesian formulations usingGaussian process models, are required. Given theexpense associated with model solution, approach-es that minimize the number of simulation calcula-tions must be developed. There are seriousimplementation issues that must be addressed,such as storage requirements for variationalapproaches, advanced code generation based onautomatic differentiation, and optimal control ofcomputational errors in data-model fusion.

Specific strategies for meeting the challengespresented by data-model fusion are to:

• Develop systematic mathematical approachesfor constructing nonlinear empirical modelsinformed by physics principles, possiblyincluding physically imposed constraints;

radioactive waste at the Yucca Mountain repositoryprovides an example of such a situation. We canperform detailed experiments to test the safety ofwaste containers, but can obtain only crude andindirect information about other components, e.g.,porosity of the earth substrata in the region andfrequency and duration of volcanic and seismicactivity. Moreover, we can make direct measure-ments only over a relatively short time span incomparison to the time span for desired prediction,which is measured in tens of thousands of years.

There are three types of data-model fusionbased on the relative dependence on data versusprocess model. First are applications where empir-ical predictive models of system behavior are basedentirely on experimental data. These problems arefrequently high-dimensional, complex and spanboth space and time, thus raising serious computa-tional hurdles. Examples of these applications canbe found in waste management and environmentalremediation. The second are predictions based onsimulations of physics-based mathematical models.While the predictions of system behavior are main-ly based on knowledge of the physics, experimentaldata plays a key role in determining constitutiverelations, in data and parameter values, and in thevalidation of prediction results. An example is pro-vided by the challenges associated with the designof next-generation nuclear reactors, where bothengineering design of the reactor and performanceevaluation of potential nuclear fuels is required.The third are applications that require a combina-tion of experimental and model simulation knownas data assimilation. In this approach, observations

FIGURE 8. Increased computingpower and improved algorithmsenable scientists to see finerdetails in climate simulations.From the atmospheric compo-nent of the Community ClimateSimulation Model, this visualiza-tion shows precipitate waterfrom a high-resolution experi-mental simulation (T170 resolu-tion, about 70 km).Source: SciDAC Earth SystemGrid


models of complex systems must correctly take intoaccount the uncertainty that results from experi-mental errors and the random nature of the datadefining the system. For example, in biology, evencells of the same type have differences—they maybe bigger or smaller, younger or older, etc., andsome system parameters can be very difficult tomeasure and hence are subject to large uncertain-ties. Uncertainties in data must be quantified andpropagated throughout the system to produceresults with quantifiable levels of uncertainty. Theuncertainties can take the form of errors and/orvariations in the data, or of stochastic perturbationsto the model due to external noise. We will saymore about this huge and complex problem later inour discussion of the role of data and of the quan-tification of uncertainty. The fundamental impor-tance of accounting for the variability of data incomputer predictions is transforming how simula-tion is done and presents many open issues in themathematics of stochastic systems.

Among these questions are those concernedwith adding spatial dependency to the simulationof discrete stochastic and multiscale systems. Theadditional compleixty is analogous to that involvedin the transition from solving ordinary differentialequations to solving partial differential equationsincontinuous deterministic simulation—the computa-tional complexity increases by several orders ofmagnitude, in addition to some very challengingphysical and algorithmic issues. Both new mathe-matics and the efficient use of advanced computerarchitectures will be required to tackle these prob-lems. The field of mathematical modeling of large-scale stochastic systems is, in many respects, in itsinfancy. This area is, therefore, a very importantone for applied mathematics research and willimpact many problems of interest to DOE.

Specific strategies in this area are to:• Develop fast methods for discrete stochastic

simulation that can effectively utilize next-gen-eration computer architectures;

• Develop adaptive multiscale discrete stochasticsimulation methods that are justified by theoryand which can automatically partition the sys-tem into components at different scales;

• Develop new algorithmic approaches for largestochastic systems, particularly spatiallydependent systems;

• Develop systematic methodologies for the esti-mation of system parameters, constitutive rela-tions and uncertainties based on data;

• Develop mathematically rigorous frameworksand efficient, robust numerical methods fordata assimilation into models of complex sys-tems that are informed by numerical analysis-based error estimates for simulations andstatistics-based error estimates for the assimilat-ed data.

2.1.3 Modeling stochastic effects in complexsystems

Develop new approaches for efficient modeling oflarge stochastic systems.

The random nature of events that occur in thephysical world and throughout our everyday experi-ences have been well recognized for more than acentury and a rich mathematical foundation hasbeen developed for studying classical stochastic sys-tems. Recent advances in engineering and scienceenabling manipulations at the microscopic scale todrive processes at the macroscale have raised anumber of problems in which modeling of discretestochastic and multiscale systems is a central issue.For example, probabilistic or stochastic approachesmust be employed in physical situations where thenumber of molecules involved is too small for thecontinuum hypothesis to hold, yet full deterministicinformation is also not available or is inappropriateto describe individual molecular trajectories andcollisions. Examples of stochastic behavior arise inthe study of physics of rarified gases such as thosein fusion experiments where long mean free pathsimply that fluctuation effects become important,biological reactions where there are only a fewmolecules available to interact so that traditional(e.g. Arrhenius) reaction kinetics cannot beemployed, and in general for models that describethe coupling of molecular physics into meso-scaleor continuum scale equations.The increasing speedand power of computers has enabled stochasticsimulation of some relatively simple models, forexample, in molecular biology. However, the simu-lation of more complex and spatially dependent sto-chastic and multiscale systems will require newmathematics to justify the necessary approximations.

Another recent development has arisen withthe realization that the predictability of large-scale


ª Develop efficient strategies for estimating theprobabilities of rare events; particularly MonteCarlo approaches for sampling the tails of dis-tributions;

• Develop the mathematical tools for sensitivityanalysis, model development and optimizationof large stochastic systems.

2.1.4 Networks, systems and systems ofsystems

Develop mathematical techniques for decompos-ing complex systems into systems of canonicalsubsystems and modeling their behavior.

A large number of complex systems that oper-ate within the purview of DOE involve networksand heterogeneous assemblies of different systemtypes, which collectively operate as a single system.Advances in simulation and modeling have enabledthe solution of complex system-wide models, aswell as models of different system components.The development of systems of systems (SOS) mod-els, which integrate these disparate models into asingle analytic framework, is a natural extensionof this capability that promises to provide moredetailed predictions of system-wide dynamics andinterdependencies.

Motivating examples of SOS applicationsinclude modeling the national power grid, planningfor investments in energy infrastructures, charac-terizing supply chain management in the nuclearpower industry, and planning for the impact of newtechnologies on national security. These applica-tions address large-scale inter-disciplinary problemsinvolving multiple heterogeneous, distributed sys-tems that are embedded in networks at multiplelevels and in multiple domains. SOS models oftenneed to be developed by integrating a variety ofexisting models, and these models may need to bedeveloped using expertise from different disci-plines. Thus, SOS models often represent complexdynamics that reflect a lack of centralized policy-making and control.

The effective size and complexity of SOS mod-els will require a new paradigm in modeling andsimulation. Simple integration of existing modelingcapabilities is not likely to enable the modeling oflarge-scale systems. Integration of models with dif-ferent temporal and geographic scales poses manyof the same challenges seen in multiscale physicalsimulation models. Further, integrating different

modeling and simulation techniques remains anoutstanding challenge in many applications. Forexample, discrete network models can effectivelymodel interactions amongst subsystems, but thesemodels need to be effectively integrated withdetailed models of subsystem dynamics. Finally,the computational cost across SOS sub-modelsneeds to be managed to enable effective scalabilityto national-scale applications.

Another key challenge for SOS models is char-acterizing their dynamics and optimizing theirperformance. Validation of SOS models is a criticalchallenge for their use in practice, and this ishampered by the fact that it is difficult to predictthe dynamics of complex SOS models containingheterogeneous subsystems. In practice, the analy-sis of so-called emergent behavior in these modelswill be critical for model validation, and for theassessment of the robustness of SOS dynamics.Further, there remain many challenges when con-sidering the optimization of SOS models toimprove performance, as well as the analysis ofSOS dynamics in extreme conditions. Such analy-ses are particularly challenging due to the compu-tational cost of SOS simulation models, and thusparallelization of SOS frameworks is an importanttechnical goal.

Specific strategies in this area are to:

• Develop mathematical approaches for modelingthe behavior of hierarchical networks in whichsubnets are controlled autonomously. For exam-ple, develop mathematical approaches foranalysis of networks of intelligent agents;

• Develop techniques for decomposing complexsystems into canonical subsystems.Characterize canonical system interfaces thatsupport scalable SOS models;

• Develop parallelization schemes for SOS mod-els. What parallel architectures can effectivelysimulate these models? How can SOS modelsexploit emerging computer architectures?

• Developing modeling approaches for usinglarge-scale network system models withdetailed models of subsystems;

• Develop analysis methodologies for discover-ing and characterizing emergent SOS behavior.How can visualization and data analysis beused to aid the modeler in the discovery ofunexpected dynamics?


• Develop Monte Carlo methodologies for rapid-ly exploring worst-case scenarios in complexsystems.

2.2 Analyzing the behavior of complex systems

Address the challenges of analyzing and under-standing the behavior of mathematical modelsfor complex scientific and engineering systems.

The development of models for complex sys-tems is just one step in the process of developingscientific understanding. Techniques are also need-ed for analyzing the behavior of these systems. Theanalysis and understanding of complex systemswill enable questions important for the DOE mis-sion to be addressed, such as

• How stable is the plasma containment in aTokomak?

• Can we safely sequester CO2 underground?

• How will changes in the demand for electricpower affect the stability of the power grid?

• How quickly is climate change in the terrestri-al system occurring and what is the uncertain-ty in the predicted time scales? What are thegreatest sources of climate change and how cansociety most effectively reduce the rate ofchange?

• What is the level of safety of storing radioac-tive waste at Yucca Mountain? What are thepossible consequences of failures in any com-ponent of the storage facility?

• How quickly can an introduced bio-weaponcontaminate the agricultural environment inthe United States?

Research in sensitivity analysis, uncertaintyquantification and in the role of data in the analysisand understanding of complex systems is essentialfor addressing these and a host of similarly com-plex questions for DOE.

2.2.1 The role of data in the analysis andunderstanding of complex systems

Develop sound, computationally feasible strate-gies and methods for the collection, organization,statistical analysis and use of data associatedwith complex systems.

The ability to understand the behavior and pre-dictive capability of models for a complex systemdepends in an essential way on the data associatedwith that system. Such data comes in manyforms—observational data of varying quality, multi-ple types of experimental data and the results ofcomputational simulations. The researcher whouses and analyzes this data also faces significantchallenges:

• The data may be voluminous, yet provide onlya sparse representation of a complex system,e.g. in the common case of a high dimensionalproblem;

• Data is frequently both spatially and temporal-ly heterogeneous;

• Data from experimental measurement or sim-ple observation is rarely presented in a form

FIGURE 9. Multiphysics simulations will be required to develop a complete understanding of future magnetic fusion devices.This figure shows potential contours of microturbulence for a magnetically confined plasma. The finger-like perturbations(streamers) stretch along the weak field side of the poloidal plane as they follow the magnetic field lines around the torus.Source: Princeton Plasma Physics Laboratory.


that is immediately practical for mathematicalmodeling;

• The collection, storage and processing of dataoften entails tremendous cost, Therefore, care-ful consideration must be given to what data tocollect, particularly given the vast choices ofdata collection to support the development anduse of complex systems.

Mathematics will play an essential role inaddressing these challenges, providing rigorouslyjustified techniques for data representation andtransformation, the combining of different types ofdata, and in the development of effective data col-lection approaches, particularly in the face of con-strained resources.

Different statistical representations of data areuseful for different kinds of analyses and mathe-matically well-founded techniques for transformingdata are crucial. For example, transforming infor-mation between different spatial or temporal scalesmay need to be accomplished when only sparsedata are available to support a high-resolution simu-lation, such as in simulations of subsurface porousmedia flow. Statistical and mathematical analysis oflarge, heterogeneous data sets is a daunting chal-lenge, raising the need to develop new mathemati-cal approaches for dimensional reduction in orderto discover the essential features represented in thedata. Non-linear principal components analysis,topological data analysis techniques, support vectormachine approaches and functional data analysisare examples of dimensional reduction approachesthat might be developed further.

In many cases, researchers are faced with manydifferent data sets of incongruent size, quality andtype. For example, understanding the performanceof a nuclear weapon requires the melding of datafrom historical full systems tests, small-scale experi-ments and integrated experiments with complexphysics simulations. Similar challenges abound inclimate analysis, material design, accelerator designand combustion modeling. Bayesian, network,graphical and hierarchical models are beginning toemerge to address the challenges in combining het-erogeneous data types in space and time.

Faced with an explosion of observational andsimulation technology that offers many options fordata collection and generation, often in huge quan-tities, researchers must answer the question of how

to design experiments, both physical and computa-tional, to optimally collect data. The traditionaldesign of experiments is concerned with allocatingtrials within a single, typically physical, experi-ment. As computation has emerged as an experi-mental tool, new methods have been developed tooptimize data collected from computational experi-ments as well. To tackle the complex multiscale,multicomponent systems that DOE needs to under-stand in the future, mathematics must be devel-oped that will enable researchers to move wellbeyond the design of physical and computationalexperiments to the design of hybrid methods thatcan simultaneously optimize data collection fromacross a wide range of information types, includingmultiple types of physical experiments, computa-tional experiments, historical records and expertjudgment.

Specific strategies for meeting these challengesare to:

• Develop efficient methods for the statisticalanalysis of large, heterogeneous data sets;

• Develop rigorous mathematical and computa-tionally feasible approaches for combining dataof different types and of different quality that

FIGURE 10. The STAR Experiment at Brookhaven National Laboratory gen-erates petabytes of data resulting from the collisions of millions of parti-cles. Data management tools make it easier for scientists to transfer,search and analyze the data from such large-scale experiments.


can also quantify the various forms of uncer-tainties in the data;

• Develop rigorous but computationally feasiblemethods for dimensional reduction of data;

• Develop hybrid experimental design methodsto optimize the collection of observational,experimental, computational and historicaldata for complex systems.

2.2.2 Sensitivity analysis.Advance the theory and tools for sensitivityanalysis to address the challenges posed bycomplex multiscale, multicomponent models.

Sensitivity analysis is the organized study ofthe way in which the output of a model respondsto variations in model inputs (parameters, initialand boundary conditions) and in the model itself.Inputs into a model are subject to many sources ofuncertainty, variability and measurement error.The model itself may be subject to uncertainty aris-ing from incomplete information or poor under-standing of the physical processes and drivingforces. The coupling of different physical processesin a model through a range of scales, where vari-ability and uncertainty in one physical componentaffects the other components, also affects the out-put of the model. True predictive simulationrequires an understanding of how all of theseissues contribute to the uncertainty in observedmodel outputs.

Sensitivity analysis provides a methodology forquantifying the stability of model output withrespect to given changes in parameters and initialand boundary conditions. This information can becompared to how the physical system depends onthe conditions represented by physical data.Sensitivity analysis is also a powerful tool forinvestigating which components of a model con-tribute the most to variability in model output andhence must be better represented computationally.Likewise, sensitivity analysis can be used to deter-mine regions of the parameter space and initial andboundary conditions for which the model outputvariation is most significant and to find regions inparameter space that lead to significant changes inthe physical behavior of the model, e.g., bifurca-tion points.

Inverse sensitivity analysis reverses the pointof view, and is used to determine the allowable

uncertainty in inputs to a model given a desireddegree of uncertainty in the model output. This canprovide a powerful tool for linking model results toexperimental data. Coupled with experimentaldesign, this information can then be used to deter-mine which parameter and data values need to bemeasured most accurately in an experiment. Whenpredicting the behavior of a complex system, e.g., aclimate model that depends on hundreds of param-eters that are difficult to determine and expensiveto evaluate, any information on the relative impor-tance of parameters and the requirements for theiraccurate determination can lead to effective priori-tization of experiments and data collection.

When considering sensitivity analysis of com-plex multiscale, multicomponent systems, addition-al theoretical hurdles arise. There are no generalmethodologies for treating the effects of variationand uncertainty in high dimensional parameterspaces that include strong interdependenciesbetween parameters and large-scale ranges inparameter values. A large parameter set often leadsto a wide range of model behavior, particularly ifbifurcation point behavior occurs, making the inter-pretation of sensitivity analysis results difficult.There are few good methods for understanding thecovariance structures of observational data versusthat of model predictions. Sensitivity analysis of acomplex system typically benefits from a fusion ofmathematical and statistical techniques, yet mathe-matical frameworks for combining such approachesin a unified fashion do not yet exist.

These characteristics also mean that sensitivityanalysis of a complex system is very computation-ally intensive. Well-established experimental designand analysis techniques for studying the input-out-put spaces for computational models typically failas models become more complex. Methods for sta-tistical design of efficient and accurate samplingprocedures typically do not take full advantage ofthe implicit mathematical structure of complexmodels. The large computational overhead associat-ed with existing methods of sensitivity analysis is asignificant impediment to their use. Approachesthat exploit mathematical structure or that effec-tively take advantage of characteristics of high-endcomputers can typically require invasive changes insimulation codes, which can be undesirable whendealing with large legacy codes in production envi-ronments.


There are a number of significant challengesassociated with uncertainty quantification andmitigation for complex multiphysics systems.Uncertainty comes from many sources, e.g., fromexperiment and computation; it can be representedin different ways mathematically, e.g., statistical,probabilistic, and deterministic; and is even associ-ated with situations where it is unclear how tomathematically represent that uncertainty. In spiteof the many approaches for representing uncertain-ty, it is important that mathematically consistentapproaches for relating various types of uncertaintybe developed.

Not only can uncertainty enter into a complexmodel through a number of avenues, the subse-quent propagation of uncertainty through themodel is also a complex process. New analytic andcomputational methods must be developed forquantifying the effects of uncertainty and numeri-cal error on model predictions, model calibration,and data assimilation analysis based on what isknown about the uncertainty in inputs and theaccuracy of numerical methods. Because multi-scale, multiphysics simulations are often computa-tionally resource-limited, uncertainty analysismethods that are robust with respect to limitednumerical accuracy must be developed.

Uncertainty quantification is also costly incomputational terms and often requires intrusiveadditions to existing software. There is a strongneed for both theoretical and computationalapproaches for adaptive sampling and numericalsolution that lead to robust and efficient control ofuncertainty when possible.

Specific strategies to support uncertainty quan-tification and mitigation are to:

• Develop mathematical, statistical and hybridapproaches for describing and combininguncertainty and error from multiple sourcespresented with multiple representations;

• Develop mathematical, statistical and hybridapproaches for analyzing and quantifying theeffects of uncertainty and error propagatedthrough a complex model on model predictions,model calibration and data assimilation analysis;

• Develop memory-access-efficient algorithmsthat match current and emerging computerarchitectures, reducing or eliminating the needfor multiple runs in sampling-based approaches.

Not least of the difficulties associated with sensi-tivity analysis of complex systems is a need for effi-cient methods to organize, visualize and understandsensitivity analysis results. Large-scale ranges, high-dimensional parameter spaces, and high-dimensionaloutput from analysis all conspire to make the inter-pretation of analysis results very difficult.

Specific strategies for meeting the challenges ofsensitivity analysis for complex systems are to:

• Develop accurate, efficient computational toolsfor sensitivity analysis and inverse sensitivityanalysis of complex systems characterized byhigh dimensional parameter and data spacesand wide ranges of model behavior;

• Develop mathematical and computationalframeworks for fusing a variety of statisticaland deterministic analysis approaches for sensi-tivity analysis, for inverse sensitivity analysisand for model calibration;

• Develop efficient and effective methodologiesfor presenting and interpreting the results ofsensitivity analysis of complex systems.

2.2.3 Uncertainty quantification and mitigation.Significantly advance the theory and tools forquantifying the effects of uncertainty and numeri-cal simulation error on predictions using complexmultiscale, multicomponent models and when fit-ting complex models to observations.

While sensitivity analysis seeks to relate theoutputs of a model to its inputs, uncertainty quan-tification casts a much broader net in terms ofassessing confidence of predictions based on allavailable information. Predictive uncertainty isassociated with the combined effects of limitationsin sensitivity and accuracy of physical measure-ments, incomplete understanding of the underlyingphysical processes, the complexity of coupling dif-ferent physical processes across large-scale differ-ences, and the numerical errors associated withsimulations of complex models.

The quantitative understanding of predictiveuncertainty is essential when predictions are to beused to inform policy making or mitigation solutionswhere significant resources are at stake. For example,an understanding of predictive uncertainty played anessential role in the acceptance of the need to designpolicies to address global warming where the cost ofdifferent choices varies by trillions of dollars.


2.3 Using complex systems to informpolicy-making

Develop the mathematics needed to inform poli-cy-makers based on the prediction, optimizationand understanding of complex natural and engi-neered systems

Advances in mathematical modeling provideever-expanding capabilities for the prediction andunderstanding of the behavior of complex systems,both natural and engineered. The next step isanswering the myriad questions that arise in policy-making involving these systems, such as:

• How can new materials be designed to displaya specified list of desirable properties, andwhat are the tradeoffs among these properties?

• What is the optimal design of a MEMS-scaleelectromechanical device?

• Which characterization of equilibrium con-straints is most suitable for designing a com-pact stellarator experiment?

• What are the optimal locations and communi-cation protocols for sensing devices in aremote-sensing network?

• What changes should be made in models of theatmosphere and clouds to most effectivelyincorporate newly collected data for improvedclimate change predictions?

• Which statutory constraints on operating proce-dures have the greatest influence on powerplant efficiency?

• What are the most harmful health-related out-comes of exposure to nuclear waste in theshort term?

• How quickly could the United States recover ifpart of the power grid became inoperable?

• What are the most likely effects if a futureenergy technology fails to live up to its currentexpectations and timeline?

Research in risk analysis, optimization andinverse problems is crucial for addressing these anda host of similarly complicated questions for DOE.

2.3.1 Risk analysisSignificantly advance the mathematics that sup-ports risk analysis techniques for policy-makingabout complex systems that include natural and

engineered components, and economic, securityand policy consequences.

Risk analysis—broadly defined as the provisionof rigorous procedures to evaluate sources of riskand their consequences—is an integral part of poli-cy-making in activities with potentially negativeoutcomes. Risk analysis is ubiquitous in businessand finance, where the negative consequences tendto be financial, and in gauging the health risks ofdrug development and medical trials. DOE hasresponsibilities that if not executed safely and cor-rectly bear awesome potential for harm, potentiallyresulting in nuclear accidents, toxic and radioactivewaste releases, power grid failures, oil spills andgroundwater contamination. In addition, DOE mustundertake high-stakes policy-making with unavoid-ably incomplete information about related issues ofscience, engineering, logistics and security.Examples of this are the uncertainties associatedwith safe transportation and storage of nuclearwaste at Yucca Mountain, or in the simultaneousevaluation of the safety, surety and reliability of thenuclear weapons stockpile.

Traditional risk analysis is based on mostlywell-defined static objectives and constraints, butthese are woefully inadequate for most DOE prob-lems, especially those that depend on incorporatingscientific insights from complex nonlinear models.Significant mathematical advances will be requiredto more effectively address the challenges of policymaking supported by models of complex systems.

To address DOE’s needs, mathematics researchsupporting risk analysis is required in several areas:

• Combining multiple sources of information,including theoretical models, test and observa-tional data, computer simulations, and expertknowledge from scientists, field personnel andpolicy makers;

• Developing formal methods to integrate dis-parate information sources relative to thecontents of the evaluation (for example, simul-taneous assessment of performance, reliability,sustainability, dependability and safety);

• Addressing the varied needs of multiple stake-holders in policy making involving, for example,complex resource allocation (e.g., in mounting afull system test or building a new experimentalfacility) or continuous evaluation (e.g., support-


ing decisions that need to be made at multipletimes and at unpredictable intervals);

• Incorporating explicit uncertainty and dynamicchanges in the evolution of knowledge aboutthe system being studied.

Research in support of analysis and assessmentof risk for highly complex systems, in the face ofuncertainty, requires a unified approach foundedon several disciplines, including statistics, probabili-ty, computer science, decision theory, graph theory,knowledge elicitation and representation, not tomention modeling and simulation. Only throughthe language and theory of mathematics will repre-sentations and analyses for risk assessment bedeveloped that can bridge these disciplines.

Effective analysis of the complex risks thatDOE faces in the future demand a much broaderapproach than is taken today for knowledge elicita-tion, representation and transformation. Ontologies,which are the models that represent concepts andrelationships from differing “cultures’’ such as sci-ence, economics, security and public policy, mustbe formalized to so that problems in overlappingdomains can be addressed using methods that areboth expressed in the various “native languages” ofthe collaborating experts and are mathematicallytractable. This process involves iterative cycles ofrepresentational refinement and quantification,resulting in predictive statistical models that makeintuitive sense to all parties. The need for better

ways to represent knowledge and assess risk isespecially urgent because of DOE’s increasingreliance on multidisciplinary teams in which math-ematicians are expected to develop predictive mod-els integrating multiple types of data, informationand knowledge.

The way in which information is organized alsohas a major influence on how that information isused. With the wide range of information (andknowledge) needed for DOE’s continuous policy-making responsibilities, it is critical that this infor-mation is formally organized. This will require avariety of tools to capture information, organize itand make the results available to different commu-nities, including policymakers and the public.

Incorporating the results of complex mathemati-cal and computational models into risk analysis isextremely challenging because of unresolved issuesabout calibration and validation of models based onlimited experimental and simulation data. This isespecially true when the model outputs are highdimensional. Additionally, the need to systematicallyinclude uncertainty and risk when developing themodels initially takes mathematician/statisticiansoutside the “comfort zone” of traditional probabilitytheory and into uncharted areas of fuzzy measure-ment, belief functions and possibility theory.Approaches that expand Bayesian inference,Bayesian hierarchical models and techniques fromspatial statistics to risk assessment grounded in

FIGURE 11. Computation is an importantelement in developing an understanding ofglobal climate change. This image shows asnapshot of the simulated time evolution ofatmospheric carbon dioxide (the red plumes)concentration originating from the land sur-face at the beginning of the industrial carboncycle (around 1900). This CO2 is a productof the net ecosystem exchange, the CO2 fluxdue to respiration of vegetation and soilmicrobes (green areas on land) minus thattaken up for ecosystem production (orangeareas on land). The underlying simulation isone of a number of runs performed forPhase 1 of the Coupled Climate/CarbonCycle Model Intercomparison Project.Source: National Center for AtmosphericResearch; Oak Ridge National Laboratory


complex models will be required to address thesechallenges.

Strategies for meeting these challenges are to:

• Develop mathematically rigorous conceptualgraphs and statistical graphical models for trans-lating from qualitative to quantitative represen-tations;

• Develop mathematically based risk assessmenttechniques whose reliability can be estimatedaccurately—even when minimal data is avail-able for the system being studied;

• Expand Bayesian inference, Bayesian hierarchi-cal models and techniques from spatial statisticsto risk assessment grounded in complex and fre-quently computational models;

• Develop reliable and mathematically soundtechniques for deducing information from dataand knowledge sources that are high dimension-al and heterogeneous in nature and quality;

• Perform research in knowledge management tosupport risk analysis with a focus on the devel-opment of analysis tools that can be accessedand used by a wide range of interested parties.

2.3.2 OptimizationDevelop techniques for formulating, analyzing andsolving challenging optimization problems arisingin complex natural and engineered systems.

Optimization is a broad and pervasive paradigmfor determining and characterizing the “best”results. The field has made great progress for morethan 40 years, and a wide variety of optimizationproblems once viewed as unsolvable are regardedtoday as routine. Nonetheless, further research inoptimization is essential because many of DOE’simportant policy-making decisions involve optimiza-tion problems that are either beyond the capabilitiesof today’s state of the art or else provablyintractable in their most general form.

DOE applications increasingly result in nonlin-ear optimization problems with a mixture of vari-able types. For example, the design of fossil energypower generation systems involves both continuousquantities such as length and width, and integervariables such as the number of processing units.Categorical variables (those that lack a naturalordering, such as alternative energy technologies)are also common in policy-based applications, andoccur, for example, in determining the optimal loca-

tions for placing sensors or designing nanomateri-als. Since there are no guaranteed solution tech-niques for general nonlinear mixed-variableproblems, progress depends on developing theoryand algorithmic strategies that exploit special struc-ture. These might include fast algorithms thatapproximate the solution within a guaranteed factorof optimality, or techniques that use randomizationto produce an accurate solution with very highprobability.

The most-studied objective functions in opti-mization problems are assumed to display somedegree of smoothness, yet non-smooth behavior iscommon in the real world—for example, the col-lapse of an aging material or the splitting of a singleflow path into two. Similarly, constraints are oftendescribed as equalities or inequalities involving acontinuous function of the variables, but non-smooth constraints, including equilibrium, comple-mentarity and disjunctive constraints, also appearprominently in DOE applications such as comput-ing electric power market equilibria. Several sophis-ticated techniques have been developed forhandling problems whose constraints are all of asingle type, but theory and algorithms are lackingfor problems with a mixture of constraint forms.

The basic formulation of an optimization prob-lem can also significantly affect its mathematicaland computational tractability. Two problem classesthat remain challenging for today’s state of the artare multilevel and multi-objective optimization. Inmultilevel or hierarchical optimization, there is anexplicit stratification from highest to lowest inwhich each level solves its own optimization prob-lem, dependent on the solution of problems fromhigher levels. This multilevel optimization occurs,for example, in DOE energy models that includefederal and state governments, where federal regu-lations impose fundamental constraints on thelocally determined objectives of each state. Multi-objective optimization occurs when several objec-tives, typically conflicting, are to be optimizedsimultaneously—for example, in finding an energysource that is both inexpensive and non-polluting.Multilevel and multi-objective optimization prob-lems are extremely difficult even for the simplestlinear models, becoming “impossible” as the dimen-sion increases and nonlinearities become moreprominent. Research is therefore necessary toformulate and solve specially structured versions of


these problems in which the number of variablesand the degree of nonlinearity are realistic.

Some DOE problems, notably in biology andphysics, are posed as the global minimization of anonlinear function subject to constraints, and aresolved using computationally intensive techniquesbased on physical intuition. Because of widelyacknowledged inefficiencies in these approaches,even small advances in global optimization wouldbe extremely valuable, including new results thatmathematically characterize the efficiency and reli-ability of popular heuristics.

Very large-scale optimization problems canusually be solved efficiently only by taking advan-tage of known special structure of the underlyingproblem, e.g. by recognizing that the power grid isa sparse network of heterogeneous elements. Suchproblem characteristics often translate into identifi-able matrix properties and can also be reflected inthe symbolic representations deduced by a model-ing language. Research on the distinctive matrixcomputation needs of optimization would greatlyenhance solution speed for DOE’s portfolio ofextremely large problems with few, mild, or separa-ble nonlinearities.

In science- and engineering-based contexts,especially in longstanding physics applications, sci-entists typically use techniques that are highlyproblem-specific, e.g., adjusting selected acceleratordesign parameters based on expert knowledge.Although often useful, this kind of approach canlead to inefficiency when hardwired into algo-rithms and software that are used later for otherproblems. Research in modeling languages gearedto optimization would increase both flexibility andadaptability by allowing a cleaner separationbetween formulation and solution techniques.

The models appearing in constrained optimiza-tion problems involve uncertainty from varioussources, and it is essential to characterize the quan-titative and qualitative effects of such uncertaintyon an “optimal’’ solution. When the associated per-turbations are small, strategies are known for ana-lyzing worst- or average-case effects; when aprobability distribution representing the behaviorof the uncertainty is available, stochastic optimiza-tion can be applied to specialized problem classessuch as linear programming. However, techniquesare needed for applications in which the uncertain-ties are large, nonlinear, and possibly discontinu-

ous, such as in achieving mandated safety levels ina nuclear power plant.


• Develop analysis and algorithms for optimiza-tion problems with continuous, discrete andcategorical variables, and with non-smoothand/or nonstandard objective and constraintfunctions;

• Perform theoretical and computational investi-gations of general-purpose and specialized algo-rithms for important subclasses of optimizationproblems, including problems that areintractable in general but solvable in particularcases of interest to DOE;

• Develop methods for finding approximate solu-tions of global optimization problems and forfinding global solutions of problems with spe-cial structure;

• Develop algorithms for solving specially struc-tured versions of multilevel and multi-objectiveoptimization problems;

• Develop algorithms for fast solution of the high-ly structured, not necessarily sparse, linear alge-braic sub-problems that arise in optimization;

• Create sophisticated modeling languages thatallow large structured optimization problems tobe expressed in the natural language of the user;

• Invest in analysis and algorithms for stochasticoptimization, addressing the effects of nonlin-earities, special structures and nonstandardprobability distributions.

• Develop new optimization methods and algo-rithms that match the needs for increased con-currency and tolerance of memory latency incurrent and emerging computer architectures.

2.3.3 Inverse problemsDevelop techniques for addressing the mathemati-cal and computational difficulties of inverse prob-lems associated with complex systems.

Inverse problems arise in applications whereessential elements of a mathematical model arepoorly known or understood. In a typical inverseproblem, these model parameters, which mayinclude inaccessible material properties orunknown initial or boundary conditions, aredefined by minimizing the discrepancy betweenexperimental observations and the model’s predic-



tions. Inverse problems occur in numerous DOEapplications, including studies of contaminanttransport, crude oil recovery, climate prediction,astrophysics and diagnostic and detection proce-dures such as non-destructive evaluation, wherewaves are transmitted through an unknown mediain order to identify inhomogeneities.

Unfortunately, inverse problems are typicallyill-posed—that is, a small perturbation in theobserved data can lead to a large, even discontinu-ous, change in the estimated parameters. As aresult, sophisticated techniques (often called “regu-larization”) are used to prioritize otherwise indistin-guishable solutions. Insights for regularizinginverse problems are often derived from a detailedphysical understanding or theoretical analysis ofspecific properties of the underlying mathematicaloperators. Since inverse problems from differentdomains vary significantly in mathematical struc-ture, their successful solution is likely to requireclose collaborations between mathematicians andapplication experts. In addition, since high-qualityobserved data is critical in solving inverse prob-lems, mathematical research would benefit fromties with the collection of experimental data.

Research is also needed on reliable and com-putable metrics that complement regularization byproviding essential information such as sensitivitiesand adjoints. A serious limitation of several stan-dard regularization schemes is their assumption ofcloseness to linearity, which is not valid for thecomplex systems associated with the most chal-lenging DOE applications. Numerical linear algebrais a key ingredient because matrix decompositionshelp to reveal the underlying structure of ill-condi-tioning; however, the most powerful linear algebra-ic tools available today, such as the full singularvalue decomposition, become prohibitively expen-sive computationally for complex problems of theincreasing sizes that are interesting to DOE.Computationally tractable alternatives are neededto address very large problems of this type.

The solutions of most inverse problems willinevitably be uncertain to some degree. Dependingon mathematical structure, such uncertainties—even when tiny—can cause inverse problems to beso ill-posed that available mathematical techniquescannot yield meaningful results. The causes ofthese unfavorable conditions may not be obviousin advance. They may reflect an inappropriatemodel formulation, a less-than-optimal choice ofregularization, or (most ominously) an inherentmathematical difficulty. Research is needed ontechniques for understanding the multiple rolesand implications of uncertainty in inverse prob-lems.


• Develop and extend analysis tools for inverseproblems, especially approaches that do notdepend on near-linearity;

• Develop large-scale linear algebraic techniquesthat reliably capture the structure of ill-posed-ness, allow its analysis and guide subsequentmodel reformulation;

• Enhance connections between mathematicalresearch in inverse problems and expertise/experiments in important application domains;

• Develop techniques for understanding the con-nections between uncertainty in mathematicalstructure and in the chosen model formulation;

• Develop mathematical techniques for reducinguncertainty and/or dimensionality by incorpo-ration of prior conditions.

The Department of Energy’s AppliedMathematics program supports research that pro-duces the mathematical advances needed to addressour nation’s energy, environmental, and nationalsecurity challenges. This report has already dis-cussed technical areas in which mathematicsresearch could make major contributions to theseproblems, and we now turn to strategies for effectiveprogram leadership.

DRIVING INNOVATION AND DISCOVERY IN APPLIED MATHEMATICS THROUGH EFFECTIVE PROGRAM LEADERSHIP 25

3.1.Encouraging and rewarding risk-taking inresearch

A fundamental practical difficulty in buildingand sustaining a strong research program is the uni-versally acknowledged fact that basic research doesnot lend itself to detailed or accurate predictions ofwhat its results will be, or when they will occur.Even “transformational” discoveries seldom takeplace in a vacuum, but rather depend on a longsequence of individual advances, some of which aresmall. Since “one cannot cultivate only the fruit ofthe tree”, a balanced portfolio is necessary for ahealthy research enterprise.19 The optimal form ofbalance cannot be specified precisely, but an essen-tial ingredient is that researchers should be encour-aged to take risks—which, to be genuine, include thepossibility of failure.

From a researcher’s perspective, the researchendeavor involves two forms of risk. Significantadvances of the kind desired by both the researcherand the sponsoring program are more likely if theresearcher feels free to take risks in devising newscientific approaches. However, if researchersbelieve that their funding requires the productionof measurable results within a strict timetable, thedanger of losing funding acts as a strong disincen-tive to risk-taking in research. The obvious conun-drum that faces any research program supported bypublic funds, therefore, is fostering risk-taking inresearch while requiring accountability throughmeasured results. An example of the associated dif-ficulties arises in designing a process for regularreviews of research progress. A significant andimportant challenge is to balance the review neces-sary for quality control with the need to encouragerisk-taking.

For the past sixty years, national and interna-tional discussions of research policy have empha-sized that the timeframe for transformationaladvances may involve ten to fifteen years of con-centrated effort. Hence imposing a much shortertime limit (for example, three years) on all projectsupport will inherently discourage risk-taking. Onestrategy for improving the likelihood of break-

throughs while retaining accountability is to rewardinvestigators and programs with a proven trackrecord of achieving the program’s goals with con-tinued long-term support. At the same time, it isalso important to invigorate the flow of ideas—forexample, through mechanisms to bring newresearchers, especially those early in their careers,into the research program.

Finally, the program can support activitieswhere a diverse group of researchers meet infor-mally to discuss, argue, and brainstorm, thus pro-viding a stimulus for “out of the box” ideas insettings conducive to “risky” thoughts. At work-shops and summer schools, perhaps focused on asingle challenge area, participants can explore newideas without the pressure of producing immediateresults. Such events often serve as the catalyst fornew ideas and collaborations.

3.2.Building effective connections withscience and engineering.

A key metric for success of the AppliedMathematics program is contributing mathematicalanalyses and algorithms that advance DOE’s mis-sion. Progress in this direction is accelerated whenmathematicians work closely with application sci-entists and engineers who are connected to missionneeds, since new insights frequently arise from afortuitous combination of multiple perspectives.The strategy of forming and funding teams of col-laborators from different disciplines has been high-ly successful in addressing several complex DOEproblems. However, depending on context, differ-ent organizational modes are needed.

In some cases, the scientific issues are suffi-ciently well understood so that the mathematicalresearch is limited to development of models thatare both physically accurate and computationallytractable. The most appropriate operational strategyis then likely to be the formation of partnershipsbetween mathematicians and scientists, jointly fund-ed by their respective sponsors, to produce a mathe-matical model tailored to that application area.

3. Driving Innovation and Discovery in Applied MathematicsThrough Effective Program Leadership

In other situations, however, the researchissues are primarily mathematical, such as whenthe science is less well-developed or the mathemati-cal approaches are likely to be relevant to a broadset of application areas. For such problems, mathe-maticians should ideally participate in every step ofthe problem solution process, including the initialformulation of a model as well as design of algo-rithms and software. When the creation of newmathematics is the most essential element inresearch involving mathematicians and applicationscientists, it may be most effective for the mathe-matics sponsor (in this case, the Office of AdvancedScientific Computing Research) to provide all of theresearch support.

3.3.Leveraging the expertise in NationalLaboratories and universities

The Applied Mathematics program should takemaximum advantage of mathematics research tal-ent, keeping in mind the differences between theNational Laboratories and academia. With full-timeprofessional research staff who often spend signifi-cant fractions of their career at a single institution,Laboratories are well positioned to pursue large-scale research efforts, especially those involvingcollaboration among a group of researchers over along period of time. The Laboratories are also ideal

environments to develop and maintain high-qualitymathematical software libraries, which require con-sistent long-term attention.

The institutional mobility of researchers in aca-demia, including PhD students and postdoctoralresearchers, tends to be much larger. While a facul-ty research advisor can focus on a single researcharea for many years, much of the research “prod-uct” in universities consists of self-contained effortscompleted by a single student or postdoctoralresearcher. The research produced by these latterjunior researchers often includes significant newideas.

Because an increase in multidisciplinaryresearch will be essential for driving futureadvancements in applied mathematics for the DOE,mechanisms that encourage multidisciplinary activ-ities, both at Laboratories and at universities, willbe important. To enhance the ties betweenLaboratories and academia, mechanisms could bedevised to provide incentives for academic investi-gators to join multidisciplinary collaborative teamsand Laboratory partnerships. Since the NationalLaboratories are responsible for executing much ofDOE’s “big science”, the Laboratories could organ-ize multi-institutional activities that bring mathe-maticians together with scientists and engineers towork together on scientific areas of particularimportance to DOE.

3.4.Connecting applied mathematics andadvanced computing.

Although the primary focus of the AppliedMathematics program is foundational mathematics,an element differentiating DOE’s program fromothers is its longstanding commitment to the math-ematical and computational underpinnings of large-scale scientific discovery through high-performancecomputing. A crucial link between mathematicsand applications is high-quality mathematical soft-ware that embodies the latest models and algo-rithms and is also designed to run efficiently onleading-edge computing platforms. Effective mech-anisms for producing, maintaining and enhancingthis software are an essential part of applied math-ematics within DOE.

FIGURE 12. DOE’s Computational Science Graduate Fellowship Programprovides support to help train next generation of computational scientists

26 DRIVING INNOVATION AND DISCOVERY IN APPLIED MATHEMATICS THROUGH EFFECTIVE PROGRAM LEADERSHIP

DRIVING INNOVATION AND DISCOVERY IN APPLIED MATHEMATICS THROUGH EFFECTIVE PROGRAM LEADERSHIP 27

Epilogue

This report has provided an analysis of theDOE’s needs for new applied, computational andstatistical mathematical developments in order tosupport the science and engineering-based solu-tions to the problems of critical national impor-tance that DOE must address in the future.Through effective management and thoughtfuldirection, the DOE Applied Mathematics programcan continue its half-century legacy in making sig-nificant, enduring advances in applied mathematics

that will enable future scientific discovery throughcomputational science. This program will makesignificant advances in the predictive modeling,simulation and analysis of complex natural andengineered systems in support of the DOE’s energy,environmental and national security missions.Effective policy-making on issues of critical nation-al importance will be supported by innovative,often transformational mathematical advancesproduced by this program.

28 REFERENCES

References

1. U. S. Department of Energy Strategic Plan,2006. http://www.energy.gov/about/strategic-plan.htm

2. Keyes, D., Colella, P., Dunning Jr., T., andGropp, W., (Eds.): A Science-Based Case forLarge-Scale Simulation – Volume 1: Departmentof Energy – Office of Science WorkshopReport, July 2003. http://www.pnl.gov/scales/

3. Keyes, D., Colella, P., Dunning, Jr., T., andGropp, W. (Eds.); A Science-Based Case for Large-Scale Simulation – Volume 2: Department ofEnergy – Office of Science Workshop Report,September 2004. http://www.pnl.gov/scales/

4. Oden, J. T. (Ed.); Simulation-Based EngineeringScience: Revolutionizing Engineering Sciencethrough Simulation – Report of the NationalScience Foundation Blue Ribbon Panel onSimulation-Based Engineering Science, February2006. http://www.nsf.gov/pubs/reports/sbes_final_report.pdf

5. Bashor, J. and Hules, J. (Eds.); ScientificDiscovery: A progress report on the U.S.Department of Energy’s Scientific Discoverythrough Advanced Computing (SciDAC) Program,Office of Science ,Office of Advanced ScientificComputing Research of the U.S. Department ofEnergy, 2006. http://www.scidac.gov/img_results/FinalSciDACreport.pdf

6. Finck, P., Keyes, D., and Stevens, R. (Eds.);Workshop on Simulation and Modeling forAdvanced Nuclear Energy Systems, Departmentof Energy – Office of Science WorkshopReport, August 2006.www.er.doe.gov/ascr/Misc/gnep06-final.pdf

7. Dalhburg, J. (Ed.); Fusion Simulation Project:Integrated Simulation & Optimization of FusionSystems, U.S. Department of Energy – Office ofScience, Final report of the Fusion EnergyScience Advisory Committee, IntegratedSimulation and Optimization of MagneticFusion Systems Subcommittee, DOE/SC-0073,November 2002. www.ofes.fusion.doe.gov/More_HTML/ FESAC/FESAC1102/Dahlburg.pdf

8. Kritz, A., and Keyes, D. (Eds.); Fusion SimulationProject Workshop Report: U. S. Department ofEnergy – Office of Science Workshop Report,July 2007, http://www.sc.doe.gov/ofes/More_HTML/FESAC/FESAC07/FSP_report_070702d.pdf

9. Office of Clean Coal Strategic Plan, Office ofthe Assistant Secretary for Fossil Energy – U.S.Department of Energy, September 2006

10. Report of the Computational Subsurface SciencesWorkshop, U.S. Department of Energy – Officeof Science Workshop report, January 2007,http://subsurface2007.labworks.org/report/

11. Breaking the Biological Barriers to CellulosicEthanol: A Joint Research Agenda, U.S.Department of Energy – Office of ScienceWorkshop report, DOE/SC-0095, June 2006,http://genomicsgtl.energy.gov/biofuels/b2bwork-shop.shtml

12. McCurdy, C. W., Stechel, E., Cummings,P., Hendrickson, B., and Keyes, D. (Eds.);Theory and Modeling in Nanoscience, U.S.Department of Energy – Office of ScienceWorkshop Report, 2002, http://repositories.cdlib.org/lbnl/LBNL-50954/

13. Hendrickson, B. A., and Wright, M. H., (Eds.);Mathematical Research Challenges inOptimization of Complex Systems, U. S.Department of Energy – Office of ScienceWorkshop Report, December 2006,http://www.sc.doe.gov/ascr/Research/AM/ComplexSystemsWorkshopReport.pdf

14. Mount, R. P. (Ed.); The Office of Science Data-Management Challenge, U.S. Department ofEnergy – Office of Science Workshop Report,November 2004, http://www.science.doe.gov/ascr/ProgramDocuments/Final-report-v26.pdf

15. Lax, P. D., Hyperbolic Systems of ConservationLaws and the Mathematical Theory of ShockWaves, Society for Industrial and AppliedMathematics, 1973.

16. Colella, P., Hou, T. and Petzold, L. (Eds.); Reportof the First Multiscale Mathematics Workshop:First Steps toward a Roadmap, U. S. Department

REFERENCES 29

of Energy – Office of Science Workshop Report,August 2004, http://www.sc.doe.gov/ascr/Research/AM/MultiscaleMathWorkshop1.pdf

17. Estep, D., Shadid, J. N., and Tavener, S. J.,(Eds.); Final Report: Second DOE Workshop onMultiscale Problems, Broomsfield, Colorado, U. S.Department of Energy – Office of ScienceWorkshop Report, October 2004, http://www.sc.doe.gov/ascr/Research/AM/MultiscaleMathWorkshop2.pdf

18. Dolbow, J., Khaleel, M. A., Mitchell, J., (Eds.);Multiscale Mathematics Initiative: A Roadmap,U.S. Department of Energy – Office of ScienceWorkshop Report, December 2004. http://www.sc.doe.gov/ascr/Research/AM/MultiscaleMathWorkshop3.pdf

19. Experiments in International Benchmarking ofU.S. Research Fields, the National AcademiesPress, 2000, page 54.

Date post:	20-May-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times