American Institute of Aeronautics and Astronautics
1 Approved for public release; distribution is unlimited.
Integrating Computational Science and Engineering with
Testing To Re-engineer the Aeronautical Development
Process
Dr. Edward M. Kraft 1
United States Air Force
The nation is at a strategic crossroads relative to the capabilities required to develop
aeronautical systems. One the one hand, the majority of the major wind tunnels and turbine
engine tests cells used today in aeronautical research, development, and test and evaluation
(RDT&E) were designed and commissioned in the 1950s and 60s. These facilities remain the
backbone of the aeronautical development process although they are becoming more
challenging to maintain. On the other hand, rapid advances in computer hardware and
software offer the potential to dramatically alter the design and development process for
flight systems through the application of computational science and engineering (CSE).
However, after 30 years of promises to eliminate the need for test facilities, advanced CSE
has still not diminished significantly the need for test facilities or reduced the overall cycle
time for development of flight systems.
Even with peta-flop (1015
floating point operations) computing on the immediate horizon,
CSE will not replace the need for ground-test facilities in the foreseeable future. Advances
in computers to peta-flop performance and beyond are necessary but not sufficient to
transform the aeronautical development process. It will take holistic advances in the
integration of people, processes, and tools to enable the kind of revolution envisioned for the
last three decades. Even more than the tools, the people and processes need to be better
understood and integrated with the advanced computer hardware and software systems to
increase the effectiveness of CSE in the aeronautical development process.
In this paper, the current way in which people, processes, and tools are used is explored
to identify inhibitors to better application of CSE to the design and development of flight
systems. In addition, visionary concepts for fully integrating CSE with testing to increase the
effectiveness of system development are presented.
I. Introduction
Peta-flops are coming, peta-flops are coming. The profound increase in high-performance computing
capabilities in recent years has reenergized a debate over when computational tools will eliminate the need for test
facilities. Adding fuel to the fire is the fact that the majority of test facilities in the nation are approaching +50 years
of age, requiring more and more investment to maintain capabilities. With declining budgets and fewer major
aeronautical systems projected for development, it will be increasingly challenging to maintain a national
experimental capability. Hence it is reasonable to examine whether computational modeling can replace test
capabilities, and if so, when.
First, for clarity, we need to define what we mean by CSE relative to the ubiquitous phrase modeling and
simulation (M&S). For this paper, we will focus our attention on high-fidelity, physics-based modeling and
simulation as opposed to engagement or theater war gaming models. The former, which we will refer to as CSE, is
more directly reflective of the aeronautical design process and the way test facilities are used to develop systems.
The latter, that we will refer to as M&S, is more associated with doctrine, tactics and techniques, and training for the
Department of Defense (DoD). Also we will use the terminology CSE to connote that we are talking about the
entire spectrum of physics-based modeling such as computational fluid dynamics (CFD), computational structural
mechanics/computational structural dynamics (CSM/CSD), computational electromagnetics (CEM), or Computer
1 Chief Technologist, AEDC/CZ, 100 Kindel Dr, Arnold AFB, TN 37389-1327, AIAA Fellow.
48th AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition4 - 7 January 2010, Orlando, Florida
AIAA 2010-139
This material is declared a work of the U.S. Government and is not subject to copyright protection in the United States.
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
2 Approved for public release; distribution is unlimited.
Aided Engineering (CAE). Embodied in these modeling activities are also computational heat transfer and chemical
kinetics.
It is important for our discussions to maintain this distinction between M&S and CSE. M&S and CSE are both
computational simulations, but within the upper levels of the DoD M&S is ascribed to any and all computational
simulations. In reality, the DoD focuses its management structure and funding on war gaming and training M&S.
Aside from the activities of the Office of the Secretary of Defense (OSD) High Performance Computing
Modernization Office (HPCMO), CSE has largely been left in the hands of Science and Technology (S&T), Test
and Evaluation, and Engineering functionals within the government as well as academia and industry. Consequently
it has evolved in an ad-hoc fashion, albeit with great success in select areas.
It is also important to understand the difference in the use of CSE in support of S&T and engineering. In the
former, great strides are being made in the use of peta-flop computing in research approaching the molecular level.
Designer materials are being developed by using computer simulations to manipulate molecules to achieve desired
properties. While the S&T community has been the principal driver for advances in CSE, it has only been recently
that the engineering community has become a key component of the CSE community. In the past, the engineering
design function has relied on legacy capabilities and relatively small hardware systems. What has become clear in
recent years is that a major justification for developing even larger scale computer systems is tied to the engineering
process more than to the S&T community, i.e., the economic justification is based on the output of the product
development process for commercial and military systems. The integration of multidisciplinary simulations and
design optimization of aerospace vehicles throughout their mission profile is primarily an engineering function that
can be enhanced by the next generation CSE capabilities. It is also the engineering application of CSE that directly
competes with the engineering use of wind tunnels in the development process. In this paper we will focus
primarily on the engineering applications of CSE.
In this paper, we will also focus on military systems which provide the most technically challenging environment
for large-scale computing. Military systems fly over larger speed ranges and operating envelopes, have more
intense integration issues such as low observables, weapons integration, buried inlet/engine configurations, etc., and
have a much more challenging multiorganizational environment in which to implement CSE.
So what is the potential of advanced CSE to improve engineering processes? To understand the potential impact
peta-flop computing should have on aeronautical system development, a projection made by the author in 1993 of
the impact of computer speed on aeronautical system modeling is shown in Fig. 1. The author recently revisited this
figure with some colleagues with the conclusion that the figure is still generally correct. In the late 1980s and early
1990s serious engineering calculations were being performed even though the state of the art in computer capability
was barely at the giga-flop level. An inviscid, Euler solution for an F-15E aircraft complete with stores, pylons,
pods, etc., using about 1 million grid points could be computed in less than 8 CPU (central processor units) hr,
making it a useful engineering tool. The simulation, augmented and validated by wind tunnel data, was sufficiently
accurate to predict the release of weapons from the conformal fuel tanks on the F-15E well enough to safely guide
flight testing. An unsteady, viscous simulation of the flow in the internal weapons bay of the B-1B again using
approximately 1 million grid points could be obtained in about 100 CPU hr, making it a demonstration of the state of
the art. It also was a sufficiently accurate simulation to effect changes in the design of weapons bay leading-edge
treatments to reduce acoustic effects in the bay.
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
3 Approved for public release; distribution is unlimited.
Figure 1. Circa 1993 projected impacts of computer trends on CFD advances.
In less than two decades, computing speed has increase by 106. Another decade of technical advances could
increase this capability another 103 to 106. Based on the 1993 predictions shown in Figure 1, with peta-flop
computing now available, the CSE community should already be performing practical engineering solutions in
minutes and demonstrating multidiscipline design in tens of CPU hours. Surely the CSE revolution should be
overtaking testing by now.
So what happened? To a large part, the added speed and storage of the hardware has been consumed by
increasing the fidelity and complexity of the class of problems in hand. This increase in fidelity is required to
improve the accuracy of the solutions to put them on par with wind tunnel and flight testing data. While a million
grid point inviscid solution for the F-15E was a breakthrough engineering calculation in 1988, very few calculations
of that simplicity are performed today. This is illustrated in Fig. 2 which provides a history of increased complexity
of CFD calculations performed at the Air Forces Arnold Engineering Development Center (AEDC). With the 1988
calculation of the F-15E referred to above as a baseline, subsequent advances in CSE enabled more physics
(unsteadiness and viscous effects) as well as refined grids to produce more accurate engineering solutions in less
time. In the chart, complexity is defined as the product of the number of processors used times the number of grid
points times the output in number of solutions per week. It is not uncommon to use 30-50 million grid points for a
time-accurate, unsteady, viscous simulation of a complete aircraft and produce a solution in less than 8 CPU hours.
These advances were made at AEDC, which has aggressively applied CSE to engineering problems for over 20
years, but has relatively modest computing horsepower.1 Many of the solutions illustrated in Fig. 2 used only 32
processors.
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
4 Approved for public release; distribution is unlimited.
Figure 2. Increasing complexity of CFD simulations.
On the other hand, as will be discussed in this paper, advances in computer speed alone are not sufficient to alter
the aeronautical development process. Advances in the software engineering tools have not kept up with the
promises of peta-scale computing. More importantly, the people and processes required to truly revolutionize the
use of CSE in the development process have not been considered as part of the approach to CSE applications.
II. Why Hasnt CSE Already Replaced Testing?
The less-filling, tastes great debate between CSE and wind tunnels has been ongoing for over 30 years. The
classic AIAA Dryden Lecture delivered by Dean Chapman2 in 1979 was the first serious salvo in the debate.
Chapmans visionary paper clearly identified the rapid growth in CFD hardware, software, and modeling
capabilities that could transform the aerodynamic design process. Many of his CFD projections have been exceeded
over the last 30 years (He was forecasting breakthroughs only through the 1990s and did not extend his vision to the
scale of CSE today.). On the other hand, the average number of wind tunnel hours used in development of
commercial and military aircraft has continued to grow3 over that same 30-year period even though the efficiency of
wind tunnels has increased by at least a factor of four.4 At the same time, more and more DoD programs (and some
commercial programs) have overrun their original cost and schedule estimates. So what gives? Obviously CSE has
not effectively changed the aeronautical development process to the degree envisioned by Chapman.
Very simply, advances in computers even to peta-flop performance and beyond are necessary but not sufficient
to transform the aeronautical development process. It takes a holistic advance in the integration of people,
processes, and tools to enable the kind of revolution people have envisioned for the last three decades. Even more
than the tools, the people and processes need to be better understood and integrated with the advanced computer
hardware and software to increase the effectiveness of CSE in the aeronautical development process. In this section,
we will explore challenges to the technologies, intellectual capital, and processes that will have to be overcome to
achieve the full promise of CSE in the development process.
A. Technological Impediments Indeed, large-scale computing power is at our doorsteps. Although access to peta-flop computers is necessary
for a revolution in applying CSE to aeronautical system development, it is definitely not sufficient. Having
software that can efficiently and effectively use the massive parallel computing power, having robust algorithms for
complex and multidisciplined applications, improving modeling of essential physical phenomena, and systematically
verifying and validating that the tools will robustly work in the engineering environment are equally important. In
this section we will highlight some of these technical challenges.
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
5 Approved for public release; distribution is unlimited.
Software Scalability The trend in high-performance computing architecture is toward massive parallel
processing to upwards of 100,000 Central Processor Units (CPU) or cores. These trends are being driven by the
rapidly growing cost of further increases in processor clock speed and the emergence of power density and cooling
requirements as dominant considerations. High-performance computing centers are now requiring megawatts of
power for operation. Although peta-flop computers are already available in select Federal computing centers and
exa-flop (1018 flops) machines are on the horizon, the legacy CSE tools routinely applied to design and development
problems have not been scaled to maximize the use and gain the efficiencies afforded by clusters with tens of
thousands of processors. Most codes have been optimized to run on less than 100 processors. CFD solution
algorithms tend to scale reasonably well, but many algorithms have topped out around 512 CPUs, and only a few
have operated with a few thousand cores. The very best CFD codes scale linearly to 5,000 cores, which is three
orders of magnitude smaller than the potential million-core machines envisioned for the future. Even the highest performing CFD algorithms quickly lose ground when significant input/output (I/O) is required to grab and store
solutions every few time steps for a graphical representation of an unsteady flow solution. Other CSE algorithms do
not scale as well as CFD codes. Hence, even though peta-flop machines are becoming available, current software
scalability limitations do not enable the solvers to use all of the hardware capability. One of the strategies to offset
this near-term lack of scalability is to use a large number of available cores to simultaneously solve a number of
parallel cases. This strategy will be useful in either rapidly reducing the design space in the early phases of concept
development or building building a significant CSE database in later stages of development.
Complexity At the same time computer systems have advanced, so has the complexity of aeronautical systems.
Over the last 30 years, expanded flight envelopes, super-maneuverability, super-cruise, low observables, and
advances in materials technology have made it more challenging to model the physics of military flight systems.
A significant challenge to developing a full flight system is the integration of the major subsystems, i.e.,
airframe/propulsion integration, airframe/structure integration, electromagnetic interference, control systems, and
airframe/weapon systems. The major defects frequently found late in the development cycle for a flight system
usually occur at the interface of major subsystems, e.g., aerodynamically induced structural failures. For example,
on average 10 structural flaws are found in the flight test phase for military aircraft even after a comprehensive
ground-test campaign and massive application of CSE. The fixes for these structural flaws can range from simple to
significant costs of as much as $1B and delaying a program by a year or more.
Although significant advances in multidiscipline dynamical simulations for maneuvering vehicles have been
made, the fidelity of current capabilities in terms of grid resolution, model complexity, and inter-disciplinary
coupling is still only a fraction of what is needed in the long run.
Performance Predictions vs. Reliability It is frequently overlooked by the CSE community desiring to
replace testing that test facilities are used not only to predict performance, but also to predict the operability,
reliability, and maintainability of an aeronautical system. The majority of the ad hoc success stories in applications
of CSE have to do with performance predictions only. CSE is not capable, for example, of simulating the
aeromechanical performance of a turbine engine over its mission life to ensure that it will be reliable enough to field.
It is also not capable of simulating the dynamic stressing of its structure to ensure its reliability to stay on wing for
hundreds of hours. Similarly, CSE is not robust enough to decide whether an engine can be restarted at altitude or
survive a bird strike or debris ingestion. Comparable limitations exist for modeling dynamic fatigue cycles for the
aircraft structure. Consequently, in spite of advances in computer horsepower and applications of CSE, test facilities
will be essential to ensure that a system is operable, reliable, and maintainable.
Physics Modeling The list of physics modeling challenges that inhibit the robust application of CSE is legend.
The classical problems are turbulence modeling, boundary-layer transition, and flow separation. For relatively
benign attached or mildly separated flow, the use of Reynolds averaged Navier Stokes (RANS) codes with the
addition of large eddy simulations (LES) has advanced to a very good engineering capability but still has enough
inaccuracy to preclude total reliance on the computed results. For vortex dominated or massively separated flows
typical of advanced tactical aircraft at the corners of the flight envelope, the CSE tools are not nearly as capable.
The dynamics of separated flow have a large impact on structural dynamics, stability and control, as well as control
surface response.
Turbulence modeling may be one of those intractable engineering problems that cannot be solved with higher
performance computing. Turbulence modeling in todays CFD codes is a semiempirical approximation of the
physics of turbulence to support practical calculations. To enhance predictions using turbulence modeling requires
decreasing the size of the numerical grids. To double the resolution of a three-dimensional flow problem requires an
increase in computer horsepower by a factor of 16. Although the promise of the revolution in computer hardware
will enable this increase, the scalability of the software will make it challenging to fully utilize the increase for
realistic geometries. The step beyond turbulence modeling enabled by high-performance computing is direct Navier
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
6 Approved for public release; distribution is unlimited.
Stokes (DNS) simulations. DNS does not make approximations to the equations of motion, but will require billion-
plus mesh point grids. Although research in this area is progressing for relatively benign geometries, it will be
decades before DNS will be useful for relevant geometries of flight systems.
High-speed and hypersonic flight bring in another range of physics modeling challenges. At hypersonic
conditions, additional physical phenomena such as real-gas chemistry, conjugate heat transfer, wall catalicity,
shock/shock interactions, etc., create significant problems for CSE. Compounding the physics modeling issue is the
dearth of qualified experiments and test facilities to explore the physics and provide sufficient high-quality data to
validate and verify the models. For example, to fully benchmark hypersonic boundary-layer transition phenomena
would require experiments that encompass a wide range of Reynolds numbers, Mach numbers, angles of attack,
bluntness, favorable and adverse pressure gradients, roughness, waviness, wall temperatures, cross-flow phenomena,
surface catalicity, and a range of gas chemistries. Not to be overlooked is the requirement for advanced flow
diagnostic tools that can be applied in the high-temperature, high-pressure hypersonic flight regime. A critical
review of CSE and testing for hypersonics, presented in Ref. 5, suggests an incremental approach to CSE and testing
to overcome the challenges to each.
Validation & Verification (V&V) The aeronautics community has given itself one huge head fake. There
are numerous (and growing) conference papers showing good comparisons between CSE solutions and select
experiments. These comparisons have been the basis for many marketing efforts to try to make the argument that
CSE can duplicate test facilities. However, an accumulation of anecdotal comparisons does not result in a robust
tool. Tinoco6 probably expressed it best: CFD validation cannot consist of the comparison of the results of one code to those of one experiment. Rather, it is the
agglomeration of comparisons at multiple conditions, code-to-code comparisons, an understanding of the wind tunnel
corrections, etc., that leads to the understanding of the CFD uncertainty and validation of its use as an engineering tool.
Examples include comparisons of predictive CFD to subsequently acquired test data. The question is not can CFD give a
great answer for one or two test cases, but can the CFD processes give good answers for a range of cases when run by a
competent engineer? This is what validation for an intended purpose is all about.
The recent AIAA drag prediction workshop7 compared results from a number of state-of-the-art codes applied by
experienced CSE practitioners to the prediction of drag on transonic transport aircraft configurations. The workshop
provided a very broad view of the state of the art of CFD applications within the industry, much more so than that
which can be garnered by an isolated study. In fact, by reviewing in isolation any one of the individual data blocks,
one may arrive at different conclusions than those determined from the complete data set comparison. For example,
a typical publication may show how successful a CFD solution matches test data. By combining a large set of
solutions from many sources around the world, this workshop clearly showed that there remains much room for
improvement.
The need for robust V&V also underscores the requirements to put error bars on the computational results as
well as the experimental results. One has to exercise caution in doing so. A CSE solution, since it is deterministic
for a given computation, will have zero precision errors, but it could have excessive bias errors driven by grid
resolution, time steps, numerical dissipation, boundary conditions, and physics modeling. On the other hand,
experimental data can have both precision and bias errors. Precision errors at the 95% confidence level are usually
well documented in the experiment, but attention needs to be paid to bias errors driven by geometric modifications
of a scaled model, Reynolds number scaling, wall interference, support interference, etc. Experimental validation
data for CSE V&V needs to be well documented for precision and bias errors. Furthermore, comprehensive V&V
of CSE needs clear identification of all boundary conditions which will require off-body flow measurements for
completeness of the experimental database.
B. Experience and Intellectual Capital Increasing the use of CSE versus testing is a two-edged sword relative to the technical talent involved in
aeronautical system development. On the one hand, visual output from high-fidelity models provides unprecedented
insight into flow features that cannot be obtained in any other way. Being able to see streamlines and vortex
patterns on flow over a vehicle brings new understanding in the causative relations between aerodynamic shapes and
vehicle performance. The tools also allow relatively rapid evaluation of changes to the design which in its own way
allows for more insight.
On the other hand, having a generation of engineers experienced only in the zeros and ones of advanced
modeling has the downside of limiting real understanding of the physics of the problem especially when extending
into realms beyond the physical fidelity of the model. The experiential insight gained from physically measuring
phenomena is important in two ways it provides more depth in understanding and is absolutely essential to guide
development of models to capture the physics. There seems to be a circular argument that we can better model the
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
7 Approved for public release; distribution is unlimited.
physics than the experiments when the models are only as good as our physical understanding gained from
experiments. If we no longer have experimental facilities, how do we advance the physical representation in the
models?
It is painfully apparent in the aerospace industry that there has been a significant decline in the experience base
of aeronautical designers and developers. As shown in a RAND study,8 the experience base for post-WWII
engineers was approximately 6-12 new design aircraft per career. The number of new military aircraft program
starts per decade is shown in Fig. 3. In the 1950s there were 60 aircraft programs in various stages of development.
In contrast, an aerospace engineer starting his career today may experience only one, maybe two new system designs
during his or her career. The decadal decline of career opportunities in other aerospace technical areas is also shown
in Fig. 3. For most aerospace systems such as rocket engines, turbine engines, high-speed X-vehicles, and ground-
test facilities, engineers today have far fewer opportunities to hone their skills than their predecessors. Anecdotal
evidence has linked this trend to problems experienced in many recent aerospace development programs.
Counterarguments point out that rapid advances in design, manufacturing, and information technologies used in the
design and development process of todays new design aircraft have compensated for some or all the declining
experience base.
Figure 3. Experience trends in aerospace systems development.
A study was performed at the Massachusetts Institute of Technology9 to understand whether the application of
large-scale computer simulation to the design process would offset the inexperience of aircraft designers. The study
explored results from multiple aircraft programs covering four decades from the 1960s through the 1990s. Aircraft
weight management through the development cycle is a critical and well-documented parameter that can be
compared from program to program and decade to decade. In the study, there was clear evidence that weight
management degraded from decade to decade and was clearly linked to the level of experience of the designers. The
key findings from the study were as follows:
There is a strong linkage between experience and performance
70s-era design efforts outperformed 90s-era in weight management
Test phase is an important downstream indicator of design performance test personnel understood design flaws through exposure to recurring problems
Modern design tools are graphically compelling, but reduced experimental experience led to deficiencies. While simulation and automation of the design process certainly helped, it did not substitute for the intuition and
inspiration that contribute to successful new and innovative designs. Also, such automation was only marginally
effective when dealing with new and untried technologies because the basic information needed for the
computational algorithms was missing or of low fidelity. Furthermore, it should be clear that one cannot really
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
8 Approved for public release; distribution is unlimited.
assess a design only on a computer. One has to build the prototype and test it; otherwise, design flaws will flow
downstream into manufacturing and operations. The earlier design flaws are discovered through prototype testing,
the better.
In the MIT study, some negative impact was found to be associated with todays computational tools, not so
much the tools themselves, but with regard to the tacit knowledge derived when interacting with them. Todays
tools are much less effective at developing the tacit knowledge of the users. Sophisticated simulation models of all
types, some with realistic graphic presentations, seem to command a greater level of creditability than they deserve
in many cases. In digging for a root cause to some design issues, it was clear that there were significant shortcuts
taken with respect to supporting wind tunnel testing and modeling efforts needed to develop a model worthy of the
level of confidence with which it was being applied.
It is hard to envision that the late Richard T. Whitcomb of the NASA Langley Research Center would have made
his breakthrough contributions to aerodynamics if the only tool he had was a computer. His insights into transonic
area ruling, supercritical airfoil sections, and winglets came about by persistent experimental research and sound
physical understanding of the flow phenomena.10 The three-legged stool of theory, experiments, and computations
is necessary to make real advances in the aeronautical sciences.
C. Processes CSE is just a single tool in the systems engineering process required to design, develop, and field an aeronautical
system. Consequently, if the CSE community and its practitioners are not equally fluent in understanding the
overall processes, CSE will generally not have the desired effect on overall development. Ensuring the process
environment is conducive to integration of CSE may be the single most important consideration for advancing CSE.
Cultural Acceptance The application of CFD to aeronautics over the past 40 years has seen some interesting
dynamics in acceptance by the community. In the early 1970s when CFD was just emerging as a viable tool for
augmenting aeronautical development, the young turks engaged in its development were enthusiastic about its
potential. However, the managers making decisions at that time had not grown up in an environment of CSE and
were not prone to support a large-scale application of CSE. In reality the tools were not quite mature enough to
have a major impact either.
After a generation of CSE fledgling applications, the original young turks became the mid-level decision
makers in the 1990s and were influential in increasing the applications of CSE. They were the zealous advocates for
CSE being able to replace or reduce the need for traditional development tools like testing. However, by the early
2000s, it was painfully obvious that their predictions were not coming true, leading to a backlash in credibility for
M&S.
Why did they not succeed? Simply, they oversold the capabilities of CSE, not so much on the technical
potential, but because they did not fully comprehend the people and process issues we are trying to identify in this
paper. Also, in the 1990s, the DoD, recognizing the potential ascendance of M&S, decided to organize their efforts.
The DoD, however, largely organized and funded M&S for wargaming and training, leaving CSE (as we use the
terminology in this paper) to ad hoc development by the research and engineering communities. Although highly
successful in support of wargaming and training, M&S was not appropriate for eliminating testing or reducing cycle
time and obviously did not succeed in those realms. Unfortunately, the shortcomings of M&S in supplanting the
need for testing has produced a negative inference on CSE as well.
Another cultural dynamic that impedes the successful application of CSE to major programs is the lack of
understanding that one needs to invest in a capability before taking the promised gains. It is not uncommon in the
DoD to take the forecasted savings from M&S up front usually by diminishing the resources for testing. The need to
invest in and implement the CSE tools to support the projected savings is usually not budgeted. As a consequence,
the modeling and the testing efforts in support of system development both come up short, leading to further
skepticism about M&S in general.
Finally, it needs to be recognized that the aeronautical development community is very conservative. Their
design and development processes have been refined over generations of applications and are intended to reduce
risks. Coupled with the forecast for fewer major aeronautical system developments, it will be challenging to have
the industry perform a significant overhaul to their processes, no matter how attractive CSE appears. To further
advance CSE into the development process will require a clear advantage to the program manager relative to better
quality of data, lower costs, reduced risk, or reduce cycle time. Currently shifting from testing to CSE is viewed as a
risk without clear changes in quality, costs, or cycle time.
Concept of Operations (CONOPS) - Application of CSE in an S&T environment to a few predictions of the
performance of a vehicle is woefully short of the operational needs required for the development of a system. The
current operational model for large-scale computers in the DoD and Department of Energy are suited best for S&T.
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
9 Approved for public release; distribution is unlimited.
To obtain access to the large number of processors needed to supply peta-flop computing for S&T, it is acceptable to
queue up a number of very large batch problems and take days, weeks, or even months of wall clock time. The
engineering design and development process will require a significantly different CONOPS to succeed. In the early
phases of design, literally thousands of configurations need to be evaluated quickly, albeit with simpler engineering
models. As the design matures, a handful of parametric factors need to be evaluated with higher fidelity. As the
design matures further, the data requirements rise exponentially. More data are required with very high accuracy on
shorter cycles, and large databases need to be obtained for loads, stability and control, subsystem integration, flight
simulation, etc. Quick turn-around computing to support interactive design is essential. This intensity of schedule,
accuracy, and volume cannot be supported by competing in a queue with S&T projects, i.e., a dedicated facility will
be required.
The dedicated use of engineering models in the very early phases of the design process will almost certainly be
performed on proprietary systems within the aerospace industry. However, it is not envisioned that industry will
invest in peta-scale computing resources even though the unit cost of computing is dropping dramatically. Industry
chose to stay at modest levels of computing capability in the 1990s and rely on government investments to have
access to larger systems.11 Consequently, high-fidelity peta-flop computer systems will be limited to a few federal
sites for the foreseeable future. This could limit their application to developmental engineering without a better
CONOPS. National capabilities may need to be scheduled for dedicated applications to major systems in
development much as government national wind tunnels are scheduled. There may not be enough peta-scale
computing capacity at the national level to simultaneously support the S&T and aeronautical engineering
community during a major DoD development. Also these same large-scale computers will need to support other
government acquisition programs such as naval ship design. Clearly, a strategy for providing sufficient capacity as
well as a CONOPS to support design and development of systems will be required to enable any potential success
for large-scale application of CSE to the development process.
Managing the Process Requires a Monopsony - When trying to understand the reasons why high-fidelity CSE
has not had a larger impact on aeronautical system development, it is worthwhile to identify the common attributes
of those areas where significant inroads have been made. It is the authors observation that CSE has had a
significant impact on aeronautical system development when:
The process is controlled by a single organization which can ensure the use of CSE in design and development.
The organization has a substantial and sustained organic capability dedicated to building and applying CSE tools in a rigorous development process.
The organization has at least de facto V&V of their tools as well as a sustained knowledge base of the lessons learned from the application of CSE across multiple systems.
Two pockets of success that meet these criteria stand out design/development of commercial aircraft and the
certification of air armaments on military aircraft. The first case is obvious a commercial aircraft company owns
the entire design and development process, maintains its own databases and tools as a competitive edge, and sustains
a critical mass of experienced practitioners. Since the CSE tools are used consistently from program to program
internally, there exists within the aircraft company a knowledge base on their use and their validity.
In the second case, the Air Force Seek Eagle process for certifying the safe carriage and release of air armaments
is the primary example. The AF Seek Eagle Office (AFSEO) owns the process for air armament certification
recommendations. Consequently, AFSEO has complete control of the use of modeling, ground testing, and flight
testing in the certification process. In conjunction with AEDC, AFSEO has aggressively developed and applied
advanced CFD modeling to simulate the carriage and release of weapons from aircraft for over 20 years.12-15 The
advanced CFD tools have been fully integrated with ground and flight testing to provide an effective approach to
weapon separation.16 The community, now including the Navy,17 has developed a common set of tools, a library of
grid models for important DoD aircraft and air armaments, and a body of knowledge of CSE applications including
validation and verification. This has culminated in the HPCMO funded Institute for High Performance Computing
to Air Armament Applications (IHAAA), which has built the tools, refined the applications process, documented a
common models library, and created a critical mass of experts.
In the general development of military aircraft, there is not a single process owner. Although the original
equipment manufacturers (OEMs) have their own internal design capabilities used to support development, the
development community at large does not have an integrated set of CSE tools. The OEM tools and databases are
considered proprietary; hence, they cannot be used by the broader community, particularly on different programs. In
addition, the OEM tools have a wide range of levels of fidelity, different providers with different interface standards,
a lack of rigor of recognized V&V, and an unwillingness to compromise. DoD acquisition policies introduced in the
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
10 Approved for public release; distribution is unlimited.
1990s relinquishing total system performance responsibility to the OEM have been a major detriment to fully
integrating CSE into the design and development processes for military systems.
Hence, to fully implement CSE into the design and development of military flight systems will require the
government to create a monopsony (a single customer vs. a single supplier as in a monopoly). The monopsony for
design and development of flight systems will require:
Government guidance on the systems engineering approach to design and development, fully integrating testing and CSE.
A common architecture for applied CSE, enabling optimization for large-scale computing, multidisciplinary dynamic simulations, standard libraries, and databases for DoD systems,
A modular plug and play environment permitting OEMs to use their own proprietary CSE tools, but in the common development process.
A critical mass of government CSE applications experts to ensure development and sustainment of the common architecture as well as to provide the government the ability to perform independent assessment of
OEM designs during the acquisition process.
III. Reengineering the Aeronautical System Development Process to Increase Effectiveness
So we now come full circle. The proper national debate that needs to be held is not CSE versus test facilities.
The aeronautics community would be better served putting their energy into creating a vision for how CSE can be
integrated with physical testing processes to increase the effectiveness of both during the development of systems.
Effectiveness in the context of this article means the ability to reduce the overall cycle time for development while
minimizing the need for rework of late defect discoveries. The elements that need to be advanced to re-engineer the
aeronautical development process include CSE as well as test facilities. In addition, a vision needs to be created for
innovative ways to bring CSE and testing together to have the maximum impact on the effectiveness of the
development process.
The CSE tools that will enable a monopsony for aeronautical development are being developed under the OSD
HPCMO Computational Research & Engineering Acquisition Tools and Environment (CREATE) program.
CREATE is developing advanced modeling capabilities to support aeronautical, naval, and radio frequency design.
CREATE-AV (air vehicle) is the aeronautical program under CREATE and is focused on the use of CSE tools
across the entire spectrum of development and sustainment of aeronautical systems.18 By analyzing common
computational needs across more than 20 acquisition program engineering activities from concept evaluation and
system development to implementation and sustainment, the CREATE-AV team was able to determine a compact
set of advances in CSE required. The CREATE-AV team determined there were four key software products needed
by the acquisition engineering workforce that fit within the available budget and that were accomplishable in the
CREATE Program timeline. The four software products are Helios, a virtual helicopter simulation tool, Kestrel, a
virtual fixed-wing aircraft simulation tool, Firebolt, an airframe-propulsion integration simulation tool, and DaVinci,
a conceptual design tool. All four tools are currently under development.
An important CREATE software design philosophy that will support use by the community is modularity. A
common architecture in CREATE-AV is a Python-based infrastructure and executive and either C or Fortran 90/95
components. This allows a build-up approach to adding capability and multidisciplinary physics. It also allows a
factored approach to the software, aiding in code maintenance and supportability. This approach also allows all of
CREATE to share components among software products to reduce the cost of development. The modular
architecture for CREATE-AV is illustrated in Fig. 4. Particularly noteworthy is the Additional Executables interface
that would permit any proprietary computational module used by the OEMs to stay proprietary within their
application, but make the output available to the government evaluation of the system performance.
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
11 Approved for public release; distribution is unlimited.
Figure 4. CREATE-AV architecture design.
18
Implementing new technologies to maximize effectiveness will require changes to test facilities as well.
Furthermore, older facilities will eventually reach a point where they become too costly to sustain and upgrade, and
building new construction is more cost effective. However, when such thresholds are reached, these moments
become opportunities to design from the outset facilities whose functionality reflects comprehensively our vision for
how to conduct aeronautical ground testing. Kraft and Huber4 have created a vision for what future aeronautical
ground-test facilities need to look like to support better integration of CSE and to increase their effectiveness. Some
of the attributes required for upgrades to current facilities or for future test facilities include:
Ability to install and de-install test articles in minutes to support high-frequency, short-duration tests focused in areas where primary uncertainties exist and to optimize use of design of experiments (DOE).
Ability to rapidly prototype and manufacture models reflecting design changes that are instantly transmitted by customers of ground-test facilities to their test partners using the latest in compatible CAD/CAM and
model shop tools and materials.
Ability to efficiently modify test conditions or proceed through a test point matrix to minimize energy usage while reflecting to a maximum extent DOE considerations.
Convenient and thorough optical accessibility for flow diagnostics tools
Connectivity to high-performance computing capabilities to integrate and merge CSE simulations and test data.
Advances in data mining and data merging software as an integral part of the facility data systems to enable rapid analyses of the variances along response surfaces.
Virtual presence, networking, and connectivity to achieve a fully integrated developmental and operational test (DT/OT) approach in an interoperable environment.
To bring CSE and test facilities into a unified toolset for streamlining the aeronautical development process
requires a focus on the effectiveness of the process, not just on the efficiency of the tools. CSE has to fully integrate
with ground and flight testing to reduce the overall cycle time for development. Kraft19 introduced a holistic
approach to integrating CSE with testing using a systems approach. Concepts evolved from the application of CSE
to weapons integration led to a broader approach for acquisition programs by recognizing CSE as the potential
unifying backbone for system knowledge management across the development cycle. The integrated approach in
Ref. 18 reinforces the need to have a monopsony for managing the tools and knowledge across the entire
development process to impact acquisition programs.
A primary objective measure for determining the effectiveness of the aeronautical development process is
acquisition cycle time. Using CSE to reduce cycle time will have a greater overall influence on decreasing program
costs and justifying CSE applications than any other cost-cutting strategy. Trying to justify CSE only as an offset to
testing misses the best business case since testing is only a small fraction of development costs. Reducing cycle
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
12 Approved for public release; distribution is unlimited.
time for major programs that can expend $1-3M per day is much more cost effective than reducing testing.
Continued emphasis on the efficiency of producing data has marginal return on investment. For example, the cost of
a wind tunnel campaign for development of a twin engine fighter is about 5% of the overall cost of T&E. In turn, the
total cost of T&E for a development program is generally just a few percent of the total development cost. Hence, a
50% reduction in the unit cost of a wind tunnel campaign equates to just a few tenths of a percent reduction in
program costs. Reducing cycle time by months can easily save a major development program on the order of $1B.
Cycle time can be estimated by the following relationship:
~
In this expression, Cycle Time is the total time required for system development. Workload is the total amount
of work to be accomplished (e.g., manhours, test unit occupancy hours, data points, computed cases, etc.); q is a
quality measure that indicates the fraction of the total work that is done right the first time (i.e., the inverse of late defects and rework); and Capacity (measured in units of work per unit of time) depends on the availability of the
development infrastructure (testing and CSE), the staffing to use the capabilities, and the throughput. The three
primary levers to decrease cycle time are reducing the workload required, minimizing rework, and increasing
capacity.
The total workload involved in aeronautical system development is primarily process driven. For example, if a
wind tunnel campaign for a major fixed-wing aircraft requires about 22,000 hours of wind tunnel testing, then given
todays national capacity of about 6,000 hr/y, such a campaign requires 3 to 4 years to conduct. Surprisingly, wind
tunnel campaigns are traditionally designed around test hours, not test points. That is why a fourfold increase in
productivity generated by the wind tunnel community in the 1990s had essentially no impact on reducing the
number of wind tunnel hours for the F-35 program as compared to the F-22 program performed a decade earlier.4
Given more efficient throughput, the users of wind tunnels take more data, rather than reduce test hours. Anecdotal
discussions with several aircraft companies over the years strongly suggest that a large fraction of the data acquired
in the wind tunnel is not used, but is retained as a security blanket in case an anomaly arises. Re-engineering the
way wind tunnel data are obtained and used has the potential to be a major driver for increasing the effectiveness of
ground testing. Although CSE has perennially offered the ability to reduce overall workload, it has been offered as a
replacement for testing. Currently, CSE as a direct replacement for testing cannot come anywhere near efficiently
replacing the total wind tunnel and flight test hours.
Similarly, the inverse of q, the amount of rework normally performed, is also process driven. For most aerospace
systems in development, q is approximately 0.25, resulting in 4 to 10 rework cycles. The incremental increase in
program costs is proportional to (1/q)-1, indicating the potential to easily double development costs through late
defects and rework. The best way to minimize the impact of rework on cycle time is early discovery of defects. This
will entail improvements in design methodologies employed by aircraft companies coupled with improvements in
wind tunnel testing and modeling techniques. These latter improvements minimize any defects in design being
passed downstream to flight testing, where the cost of fixing the defect increases an order of magnitude. Also,
feedback loops from discrepancies found in flight testing back to ground testing and back to design methodology
need to be institutionalized to make further improvements. A primary target for decreasing rework is improving the
early determination of the impact of steady and unsteady flow effects on the vehicle structure. Historically, most
aircraft development programs have discovered 10 structural flaws in flight with varying degrees of cost and
schedule impacts that can reach a billion dollars and a year to overcome. As can be seen from this example,
increasing q (decreasing late discoveries) will have a profound impact on development cycle time and cost. The
early reduction of defects may be the single most important area for the use of CSE. However, multidisciplined
approaches will have to be improved to realize the potential gains in defect reduction.
In contrast to process-driven parameters, capacity is primarily budget driven. Capacity equals the availability of
the capability times the staffing available to use the capability times the throughput. For testing, the availability of
the equipment depends on investments in maintenance and reliability. Also, the budget determines whether a facility
is staffed for one, two, or three shifts. Staffing is the most dynamic variable for increasing or decreasing capacity.
Throughput (e.g., test points per hour, solutions per day, etc.) is also budget driven. Increasing capacity of existing
test facilities is the least effective of the three parameters for significantly decreasing acquisition cycle time. The
capacity of CSE is also budget driven. The availability of large-scale computers, the critical mass of intellectual
capital to use the capability, and the throughput of the computations will similarly drive cycle time. Developing and
funding integrated test facilities and CSE with capability and capacity optimized to maximize throughput using the
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
13 Approved for public release; distribution is unlimited.
reduced workload and defect avoidance and discovery approaches will be a powerful adjunct to process
reengineering.
The discussion on cycle time focuses on the cycle time for testing. To aggressively attack the cycle time for
development of a new flight system, one also needs to address the contributions to cycle time from design,
prototyping, analysis of results, and other development and manufacturing maturation activities. There is potential
interplay between these processes and processes from test that can further help reduce overall cycle time. In this
article we are focused on reducing the equivalent cycle time for testing through better integration of CSE.
The elements that make up a wind tunnel campaign are illustrated in Fig. 5 (from Ref. 20) which is a summary of
the total force accounting for a typical fighter aircraft. As seen in the figure, wind tunnel tests are performed to
determine loads, performance, stability and control, propulsion integration (inlet and afterbody effects), and
weapons integration. Also, baseline wind tunnel studies are performed to assess impacts of test methodologies such
as support interference, boundary-layer transition, or wall interference effects. Figure 5 reinforces the fact that using
CSE applications for performance predictions impacts only a small fraction of the utility of the wind tunnel.
CSE does, however, offer significant potential to impact the overall wind tunnel campaign in three significant
areas. First, and most importantly, CSE can be used to reduce the overall workload. Second, CSE, if applied
appropriately, can reduce downstream effects of late defect discovery on total development cycle time. Third, CSE
can reduce the amount of testing required to determine support interference or wind tunnel wall effects or to better
understand scaling issues from wind tunnel to flight conditions. This not only supports a workload reduction, but
also can reduce late defect discovery if defects passed downstream are an artifact of testing issues in the wind
tunnel. CSE is an excellent tool for calculating incremental effects of tunnel conditions on flight performance.
Figure 5. Typical force accounting system for military aircraft.
20
The primary target for re-engineering aeronautical development to increase effectiveness is to reduce the overall
workload without increasing risk. A major contributor to the number of wind tunnel test hours is the need to
generate about 2.5 million data points to determine the stability and control (S&C) of the vehicle. This is
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
14 Approved for public release; distribution is unlimited.
traditionally done in the one-factor-at-a-time (OFAT) mode where data are obtained for each model configuration,
orientation, speed, and simulated altitude over the entire operating envelope. This ponderous number of data points
also has been the primary reason that CFD has not made greater inroads into developmental wind tunnel testing.
Estimates to compute the equivalent 2.5 million OFAT points range from approximately 100 to 1,200 years using
existing computer tools.
Recently, the CFD community21 introduced an innovative and efficient computational method for accurately
determining the static and dynamic S&C characteristics of high-performance aircraft. In contrast to the brute
force approach to filling an entire S&C database for an aircraft, an alternate approach is to reduce the number of
simulations required to generate a complete aerodynamic model of a particular vehicle configuration at selected
flight conditions by using one or a few complex dynamic motions (e.g., varying frequency and amplitude over a
dynamic trajectory) and nonlinear system identification (SID) techniques. This approach now makes CFD a
reasonable source of S&C data for an aircraft. Interestingly, there is a comparable experimental technique using the
prefiltered dynamic output from the force/moment balance used in the wind tunnel, SID techniques, and a fly the
mission profile in the wind tunnel.
As indicated in Fig. 6, using these advanced fly the mission modeling and testing methodologies combined
with design of experiments (DOE) offers an innovative, aggressive approach to reducing the overall test workload.
Attempts to apply DOE to streamline a traditional individual wind tunnel test have been only marginally successful
because current wind tunnels are not conducive to rapidly changing parameters to optimize randomness of the data
set. However, if one shifts to thinking about DOE at the campaign level there may be a more productive approach
to using DOE.
Instead of the OFAT approach to building the colossal database characteristic of todays aeronautical
development processes, an approach using DOE response surface techniques could be more effective. A response
surface is a mathematical construct that represents the parameter space along which the characteristics of the vehicle
are captured. An example of the use of response surface modeling for aerodynamic configurations is given in Ref.
22.
In contrast to traditional OFAT approaches that basically fill up the entire parameter space and try to interpolate
to determine the characteristics of the vehicle, an initial response surface could be built using simple engineering
models. Of course the uncertainty over the response surface would be high, but more refined high-fidelity physics
modeling could then be efficiently applied to reduce the uncertainties over the response surface using the fly the
mission approach mentioned above. Those areas on the response surface which still exhibit a high degree of
uncertainty then become the primary focus for the wind tunnel test campaign, i.e., the focus is put on key areas for
risk reduction versus defining the entire parameter space. DOE coupled with estimation theory could help determine
the minimum number of computations or test points to reduce uncertainties in areas of interest on the response
surface. Finally, the areas of residual uncertainty become the primary interest for focused flight testing which serves
to reduce the overall workload for that phase of testing. In this manner, the overall amount of testing could be
dramatically reduced with a commensurate impact on total cycle time.
The mathematics of the DOE methodology helps assure the optimum data set is taken. The alpha and beta (or
power coefficients) of the DOE process can be used to address how much further variance can be reduced on the
response surface by an additional calculation, wind tunnel test, or flight test. There is a point at which doing another
CFD solution will not reduce uncertainty further; hence, one needs to move on to wind tunnel testing. Likewise,
there is a point of diminishing return for doing another wind tunnel test and the program needs to move on to flight
testing. Thus, unnecessary modeling and/or testing can be minimized. The beta coefficient also provides some
insight into the probability that a defect is being passed downstream to the next development step.
The response surface method also provides an invaluable approach to supporting integrated developmental
testing (DT) and operational testing (OT) as well as addressing networking and interoperability issues. The
characteristics of the vehicle captured in the response surface can be translated directly into the performance math
engine for a manned flight simulator as suggested in Fig. 6. Even at the earliest phases of development, this manned
flight simulator can start to address some of the operational integration issues thereby allowing integrated DT/OT
earlier in the program. If early brass-board or digital models of the avionics and communications packages are
brought into the manned flight simulator, the evolving performance of the system can be evaluated as a node in a
distributed mission simulation. Feedback from this integrated approach can be used in the very early stages to
improve the design for maximum performance as an interoperable system. Today, most of the OT interface issues
as well as interoperability are not addressed until very late in the development process. The overall impact on
reducing development cycle time using such an innovative approach could be immense.
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
15 Approved for public release; distribution is unlimited.
Figure 6. Streamlining the aeronautical development process.
A key to increasing the quality, q, or decreasing the amount of rework, is earlier and better integration of major
subsystems such as the airframe/structure, the airframe/propulsion systems, or the airframe/weapon systems. Most
defects occur at the interface of major subsystems. Current practices generally address system integration issues
later in the development process which maximizes the amount of rework required (and increases associated costs) if
a defect is discovered. Key enablers required to get earlier insights into integration issues include high-fidelity
multi-disciplined modeling capabilities and advanced on-body and off-body flow diagnostic techniques such as
pressure sensitive paint (PSP) and planar Doppler velocimetry (PDV).
An example of the interplay of these enablers to reduce late defect discoveries is suggested in Fig. 7.
Multidiscipline, high-fidelity CFD/CSD can be used earlier in the design cycle to examine interactions between
major subsystems such the airframe and structure or airframe and propulsion system. Traditionally, pressure loads
data are obtained on a very early (and expensive) wind tunnel model specifically designed with hundreds to a few
thousand pressure taps on the surface of the model. These pressure loads are provided to the structural engineers to
perform a structural analysis and design of the vehicle. While the structural engineers are doing their analyses, the
aerodynamicists are usually continuing to refine the outer mold lines of the vehicle to improve performance.
Because of the cost and complexity of wind tunnel pressure models, effects on pressure loads due to changes in
outer mold lines are usually not updated. When the airframe and underlying structure are integrated into the first set
of flight vehicles, it is not uncommon to find structural flaws. (Remember that on average 10 structural flaws are
found on each major aircraft development during flight testing.) Contributing to these late discoveries are
inadequate characterization of the dynamic interactions between fluids and structures as well as a lack of integration
of aerodynamic and structural analysis tools.
The application of peta-scale computing in the near future will enable integrated modeling of aerodynamics,
structures, and propulsion systems during the design process. The ability to integrate these multiple disciplines will
address many of the subsystem issues early on. Having advanced diagnostic tools such as pressure-sensitive paint23
(PSP) in ground-test facilities will not only enable model validation, but also will better help characterize the
dynamic flowfield effects on flight vehicle structures. PSP will also permit rapid updating of flowfield loads as part
of structural analyses without having to build or update pressure models.
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
16 Approved for public release; distribution is unlimited.
Minimizing potential weight growth of the airframe structure to account for defects discovered in flight can also
have an important effect on the development of the propulsion system. Frequently, when weight growth occurs late
in the development cycle because of structural changes, the propulsion system developers are tasked to produce
more thrust to ensure meeting vehicle performance parameters. It is not uncommon for the engine developer to have
to significantly improve the performance of the engine fairly late in the development cycle. All of these interactive
weight issues also impact control surface effectiveness and control system gains. This vicious interplay between the
various subsystems is a contributor to late cycle churn and program delays.
Figure 7. Early integration to avoid late defect discovery.
Also suggested in Fig. 7 is the potential for sharing some of the same modeling methodologies between the
structural analysts and the propulsion system designers. The fluid-structure interactions that drive structural design
exhibit the same fundamental physics as the fluid-structure interactions on the aeromechanics of fan and compressor
blades. Advances in integrated CFD/CSD tools will help better understand and avoid potential high cycle fatigue
issues earlier in the design cycle.
Finally, CSE can be an invaluable tool to ensure better use of ground-test facilities to preclude design defects
from finding their way into the flight-test program. Use of CSE to account for Reynolds number scaling effects and
potential bias errors such as wind tunnel wall interference is well understood and effectively applied. An area where
scaling effects are not well understood and CSE may have the potential for producing new insights is simulation of
military tactical aircraft at high-angle maneuvering conditions. In these conditions, the flow is dominated by vortex
structures and flow separation. Surprisingly, a large number of tactical military aircraft have required a significant
modification to control surface size or structure even after a comprehensive wind tunnel campaign. Changes of this
magnitude during the flight-test program can have a profound impact on program cost and schedule. Coupled
effects on manufacturing costs can also become significant during this phase.
There exists a strong potential that a root cause for these late defect discoveries may be the lack of understanding
of scaling principles for vortex dominated or separated flow phenomena. The general Reynolds number scaling
principles used today were developed in the mid-1970s from attached flow data taken on commercial transport
aircraft configurations. At the time, computational tools as well as flow diagnostics were not capable of supporting
more in-depth understanding of separated flow phenomena.
At high angles of attack, flow separation from the leading edge can create vortex structures that impinge on
vertical tails. The appearance and interaction of these vortices with the vehicle can strongly influence control
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
17 Approved for public release; distribution is unlimited.
authority or cause structural failures. The classical wing-drop roll-control problem for the F-18 was caused by
vortex-shock interactions. Vertical tail structural flaws caused by vortex impingement and breakdown have been
discovered on a number of twin-tail flight vehicles, including the F-22.
In the wind tunnel, the model is generally geometrically scaled. If one examines the leading-edge vortex
formation and separation for a typical tactical fighter at high angle of attack, there are at least five characteristic
lengths involved in the problem chord length, leading-edge radius, boundary-layer displacement thickness, vortex
core diameter, and vortex breakdown length. It is not clear whether these are dependent or independent length
scales, which begs the question of whether geometric scaling is sufficient to model vortex-dominated or massively
separated flow phenomena. Current CSE tools including large eddy simulations (LES) have been used to model
vortex effects on aircraft at high angles of attack.24 Coupling such tools with advanced off-body flow laser
diagnostic tools such as planar laser induced fluorescence25 (PLIV) could provide an integrated
computational/experimental approach to better understanding the causative effects of the various length scales and
better predicting flight conditions from wind tunnel data.
Figures 6 and 7 present an aggressive use and integration of modeling and testing simulation methodologies to
change the future effectiveness of aeronautical development. It is clear that various test capabilities cannot be
addressed and judged in isolation but have to be treated as an integral combination with technical expertise,
improved processes, and better test methods to achieve the desired state of effectiveness. In addition, they will have
to be applied in a common environment to ensure that gains in effectiveness can be replicated from program to
program.
IV. Conclusions
High-performance computing has advanced to a state that should support more applications of CSE in the
aeronautical system development process. With such advances, a national debate has re-emerged on using CSE to
replace testing. The author argues that a national discussion of replacing testing with CSE is misguided. The nation
would be better served by putting its energy into determining approaches to fully integrate CSE with testing to
reduce the cycle time for aeronautical system development. To successfully integrate CSE and testing will require
advances not only in high performance computing but in intellectual capital and process management as well. Key
recommendations for advancing the use of CSE are as follows:
Most importantly, the government has to adopt a monopsony for the application of CSE to the development process for military flight systems.
A common architecture for the application of multidisciplinary computational tools in a high-performance computing environment needs to be adopted by the industry. This architecture should not preclude use of
proprietary physical models from industry, but should enable CSE and testing to be optimized for use across
any aeronautical development process.
In spite of computer hardware systems advances, there is still much work to be done in building software tools to best use advanced computer systems, notably, better physics modeling, scalability of solvers to tens
of thousands of processors, and better multidisciplinary modeling to enable dynamic simulation of complete
maneuvering aircraft.
CSE alone will not provide maximum impact to cycle time reduction but must be integrated with other tools such as design of experiments, streamlined test methodologies, advanced diagnostic tools, networking, and
knowledge management.
In addition, a concept of operations and the necessary computing capacity need to be developed to support the aeronautical systems engineering process.
References 1Kraft, E. M. and Matty, Jere J. Application of High Performance Computing at the Arnold Engineering Development
Center (AEDC), ITEA Journal, June/July 2005, pp. 55-60. 2Chapman, Dean R. Computational Aerodynamics Development and Outlook Dryden Lectureship in Research, AIAA
Journal, Vol. 17, No. 12, December 1979. 3Melanson, Mark R. An Assessment of the Increase in Wind Tunnel Testing Requirements for Air Vehicles Over the Last
Fifty Years, AIAA Paper 2008-830, January 2008. 4Kraft, Edward M. and Huber, Arthur F II A Vision for the Future of Aeronautical Ground Testing, The ITEA Journal of
Test and Evaluation, Vol. 30, No. 2, June 2009. 5Kraft, E. and Chapman, G. A Critical Review of the Integration of Computations, Ground Tests, and Flight Test for the
Development of Hypersonic Vehicles, AIAA Paper 93-5101, Presented at the AIAA/DGLR 5th International Aerospace Planes
and Hypersonics Technologies Conference, Munich Germany, 30 Nov-3 December 1993.
Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139
American Institute of Aeronautics and Astronautics
18 Approved for public release; distribution is unlimited.
6Tinoco, Edward N. Validation and Minimizing CFD Uncertainty for Commercial Aircraft Applications, AIAA Paper
2008-6902, 26th AIAA Applied Aerodynamics Conference, 18 - 21 August 2008, Honolulu, Hawaii. 7Vassberg, J. C., et al. Abridged Summary of the Third AIAA Computational Fluid Dynamics Drag Prediction Workshop,
Journal of Aircraft, Vol 45, No. 3, May-June, 2008. 8Drezner, J. A. Maintaining Future Military Aircraft Design Capability, RAND Report R-4199-AF, 1992. 9Andrew, W. Geoffrey Do Modern Tools Utilized in the Design and Development of Modern Aircraft Counteract the
Impact of Lost Intellectual Capital Within the Aerospace Industry, Masters Thesis, Massachusetts Institute of Technology, May
2001. 10Hansen, James R., Engineer in Charge A History of the Langley Aeronautical Laboratory, 1917-1958, NASA History
Series, NASA SP 4305, 1986. 11Mavriplis, D. J., Darmofal, D., Keyes, D., and Turner, M. Petaflops Opportunities for the NASA Fundamental Aeronautics
Program, AIAA Paper 2007-4084, 18th AIAA Computation Fluid Dynamics Conference, Miami, FL, 25-28 June 2007. 12Kraft, E. M., "Combining Ground Test, Computational Fluid Dynamics, and Flight Tests to Improve Aircraft/Weapons
Integration," AIAA 94-2660, London, United Kingdom, 31 May 2 June 1994. 13Carlson, D. R., King, M. O., and Patterson, R. C. Integrated Store Separation And Ballistic Accuracy - Applying Enabling
Technologies To Improve Aircraft Store Separation, AIAA Paper 95-3431-CP, AIAA Atmospheric Flight Mechanics
Conference, Baltimore, MD, 7-10 Aug. 1995. 14Benek, John A. and Kraft, Edward M. The Changing Role of Wind Tunnel Testing and Computational Fluid Dynamics in
the Development of Flight Vehicles, AIAA 96-3349, Presented at the Second Test and Evaluation International Aerospace
Forum, 25-27 June 1996, London, UK. 15Dean, J., Morton, S., McDaniel, D., and Goetz, S. Efficient High Resolution Modeling of Fighter Aircraft with Stores for
Stability and Control Clearance, AIAA Paper-2007-1652, 2007 USAF T&E Days, Destin, FL, 13-15 Feb. 2007. 16Keen, K. S., Morgret, C. H., Langham, T. F. , and Baker, W. B. Jr. Trajectory Simulations Should Match Flight Tests and
Other Lessons Learned in 30 Years of Store-Separation Analysis, AIAA Paper 2009-99, 47th AIAA Aerospace Sciences
Meeting, Orlando, FL, 5-8 January 2009. 17Cenko, A. Lessons Learned in 30 Years of Store Separation Testing, AIAA Paper 2009-98, 47th AIAA Aerospace
Sciences Meeting, Orlando, FL, 5-8 January 2009. 18Morton, Scott A., McDaniel, David R., Sears, David R., Tillman, Brett, and Tuckey, Todd R. Kestrel A Fixed Wing
Virtual Aircraft Product of the CREATE Program, AIAA Paper-2009-0338, 47th AIAA Aerospace Sciences Meeting, Orlando,
FL, 5-8 January 2009. 19Kraft, E. M. Integrated Test and Evaluation -- A Knowledge-Based Approach to System Development, AIAA 95-3982,
1st AIAA Aircraft Engineering, Technology, and Operations Congress, Los Angeles, CA, 19-21 September 1995. 20Skelley, M. L., Langham, T.F., Peters, W. L., and Frantz, B. G. Lessons Learned During Joint Strike Fighter Ground
Testing and Evaluation at AEDC, AIAA Paper 2007-1635, U.S. Air Force T&E Days Conference, Destin, FL, 13-15 February
2007. 21Dean, John P., Scott A. Morton, David R. McDaniel, James D. Clifton, and David J. Bodkin. Aircraft Stability and
Control Characteristics Determined by System Identification of CFD Simulations, AIAA Paper 2008-6378, August 2008. 22Landman, Drew, Jim Simpson, Dan Vicroy, and Peter Parker. 2007. Response Surface Methods For Efficient Aircraft
Configuration Aerodynamic Characterization. Journal of Aircraft 44 (4): 11891195. 23Sellers, Marvin E. December 2005. Advances in AEDCs Lifetime Pressure-Sensitive Paint Program, AIAA Paper
2005-7638. 24Morton, Scott Detached-Eddy Simulations of Vortex Breakdown over a 70-Degree Delta Wing, Journal of Aircraft 2009
Vol. 46 No. 3 (746-755). 25Ruyten, W. M., Williams, D. O., and Heltsley, F. L., June 1994. Computational Flow Imaging for Planar Laser-Induced
Fluorescence Applications, AIAA Paper 1994-2621. Dow
nloa
ded
by S
teve
n D
unn
on A
ugus
t 30,
201
3 | h
ttp://
arc.
aiaa
.org
| D
OI:
10.
2514
/6.2
010-
139