1 American Institute of Aeronautics and Astronautics

Low-Speed Wind Tunnel Testing via Designed Experiments: Challenges and Ways Forward

Jim Simpson1

Florida State University, Department of Industrial Engineering, Tallahassee, FL, 32310

Drew Landman2 Old Dominion University, Department of Aerospace Engineering, Norfolk, VA, 23529

Engineers with experience in ground test can attest that, although the confines of wind tunnels permit low noise testing, what is lacking many times are efficient test plans and the full use of the massive amount of data collected. From the perspective of industrial statisticians and engineers specializing in efficient design of experiments via statistical methods, the wind tunnel environment offers unique challenges for determining a best-practices approach to test design. Both variants of practitioners have realized the benefit of adapting classical statistically-based experimental design techniques to wind tunnel testing. With this general approach to test, the number of tests required is significantly reduced, the true underlying cause/effect relationship between aircraft configuration and aerodynamic performance is realized and uncertainty can be precisely determined in the presence of testing condition alterations. This paper makes use of case studies to discuss and illustrate the conditions common in low-speed wind tunnel testing which require adaptations to standard experimental design application. Remedies for each of these conditions based on methods successfully used elsewhere are provided.

Nomenclature CA = axial force coefficient Cl = rolling moment coefficient CN = normal force coefficient Cy = side force coefficient Cm = pitching moment coefficient Cn = yawing moment coefficient α = angle of attack in degrees β = sideslip angle in degrees δR25 = BWB right wing 2-5 ganged elevon deflection in degrees, positive trailing edge down δ1 = BWB ganged left and right elevon 1 deflection in degrees, positive trailing edge down δL25 = BWB left wing 2-5 ganged elevon deflection in degrees, positive trailing edge down δL67 = BWB left wing 6-7 ganged lower elevon deflection in degrees, positive trailing edge down δL89 = BWB left wing 8-9 ganged upper elevon deflection in degrees, positive trailing edge down δLrud = BWB left winglet rudder deflection in degrees, positive trailing edge left δa = X-31 aileron deflection in degrees, (right aileron trailing edge up positive – left aileron down),

1 Associate Professor, Florida State University, Department of Industrial Engineering, 2525 Pottsdamer St., Tallahassee, FL 32310, and AIAA Member. 2 Associate Professor, Old Dominion University Department of Aerospace Engineering, 3750 Elkhorn Ave., Suite 1300, Norfolk, VA 23529, and AIAA Senior Member

U.S. Air Force T&E Days 5 - 7 February 2008, Los Angeles, California

AIAA 2008-1664

Copyright © 2008 by Florida State University and Old Dominion University. Published by the American Institute of Aeronautics and Astronautics, Inc., with permission.

= X-31canard deflection in degrees, (trailing edge up positive) δc

δ = X-31 rudder deflection in degrees, (trailing edge left positive) r

I. Introduction ERODYNAMIC testing can take place in a variety of environments, depending on the purpose of the test. Design and analysis of experiments is a standard method available for most any type of test, and the purpose is to discern which of the predetermined factors varied during test have an effect on system performance as

measured by quantifiable responses. Following test, data is reduced in preparation for plotting or performing more formal statistical analyses. In statistical evaluations it is the signal to noise ratio that is important for discovery, and controlling the level of noise is frequently the challenge. Systems often have an abundance of noise. For example, in flight test, noise variability can accumulate due to differences due to aircraft, pilot, sortie, weather, situational factors, maneuvers, data collection scheme, control surface settings, g-loads, or many other influences. These types of tests are typically costly and time consuming. In addition, the objective in most flight testing is to characterize the nature of the relationships between input factors varied and the responses measured. Flight testing by its nature is more representative of traditional design of experiments (DOE) challenges. In contrast, low-speed wind tunnels and many ground testing platforms offer pristine environments for test and individual tests are relatively quick and cheap. Wind tunnel test factors can be difficult to randomize and the objective of the tests is often to extract the precise relationships between attitude/control surface deflections and aerodynamic coefficients. This mapping often involves fairly complex modeling of highly nonlinear functions. The unique characteristics of low-speed wind tunnel testing relative to the application of experimental design methods are the focus of this paper.

A

II. Principles of DOE Applied to Wind Tunnels The experimental design approach to test in low-speed wind tunnels has been hugely successful regardless of the

purpose of the test, the type of model being tested, and independent of the sophistication of tunnel used1-8. In fact, in our experience there has not been a test entry when a superior alternative to experimental design existed. On every occasion, the design of experiments approach tailored to the needs of the test and the customer has been the right choice. The purpose of this paper is to highlight some of the features common to testing in low-speed wind tunnels that occur less frequently when applying designed experiments in other venues. For example, one of the fundamental principles of experiment design is the need for test point replication in order to adequately quantify the level of uncertainty associated with completely restarting, then conducting a test. With an experienced wind tunnel test crew, this variability can be orders of magnitude smaller than most other testing environments, with the possible exception of stochastic simulations. Test points are also often relatively inexpensive in low-speed wind tunnel testing, but in the case of static stability and control tests, the time required between runs to change aircraft control surface deflections can be time consuming without remote control actuation. Configuring an aircraft for remote actuation can significantly reduce overall test time, and is almost required if a second fundamental principle of experimental design is maintained, that of randomizing the test sequence. Test run order randomization has the benefit of averaging out the influences of all variables (known and unknown) changing over time during test. Randomization of run order can be a challenge, so departures from this fundamental procedure are sometimes necessary. An alternative approach using split plot design and analysis is a viable alternative, but in general this method requires advanced statistical methods and is less desirable than using a completely randomized approach. The final principle of DOE, blocking, allows nuisance variables such as changes in tunnel operator, or day-to-day variability to be extracted from the uncertainty estimate, realizing a larger signal-to-noise ratio for statistical testing. Blocking can also be effectively used in wind tunnel tests. The discussion in this paper will proceed by addressing aspects of wind tunnel testing that require the analyst competent in design of experiments to deviate from common practice to suit the needs of the low-speed wind tunnel testing environment A case study approach will enable discussion of needed departures from standard practices typical of industrial experimentation .

III. Constrained Designs and Low-Noise Systems Testing

2 American Institute of Aeronautics and Astronautics

Experiments conducted in relatively low-noise systems are typically testers’ dreams come true. Anticipation of clearly revealed relationships between parameter changes and response variation creates optimism and expectations for few challenges with the analytical computations. The test engineer also assumes that once the test is complete, summary measures and plots of the findings will clearly reveal the underlying system influences. It turns out that all is not so straight-forward when noise variation is small. In fact, increased attention needs to be paid to certain

3 American Institute of Aeronautics and Astronautics

aspects of planning, the experimental design, and to the analysis that would otherwise be part of the standard procedures in a higher level noise settings. In terms of planning, it is important that all potential outside influences on system noise be identified and appropriately addressed. For example, if the operators in the control room do not have and follow a set of detailed test execution procedures, proper settings can easily be missed. If control room operation personnel changes during testing, care must be taken to assure consistent, high-level performance is maintained. Any unintended but influential changes in the system during test will inflate the otherwise nominal noise levels, making it difficult to detect true affects due to intended parameter changes.

One of the benefits of testing in low-noise environments is the relative ease of identifying influential parameters affecting aerodynamic performance. In fact, the standard approach to analysis is often to plot the data and fit a curve using some sort of smoothing function. Typically the residuals (error magnitudes) are so small that the data falls close to the fitted line. The drawback of such a technique though is that the system uncertainty is not effectively quantified, and estimation of aerodynamic forces and moments cannot be made with any degree of confidence. An alternative strategy is to use statistical empirical modeling that characterizes the functional relationship regarding influential parameters, provides an estimate of uncertainty, and allows for predicting forces and moments with a specified level of confidence. Part of the protocol involves collecting a few additional data points to confirm model validity. These empirical models are typically built using least squares regression analysis. Before proceeding further in this discussion, it is important that we attempt to quantify the magnitude of noise present in wind tunnel testing relative to more typical industrial or engineering-related testing. System noise can be estimated in a regression or analysis of variance (ANOVA) context by partitioning the total response variability into a component due to parameter changes (model) and a component due to error. The error variability estimate is often referred to as the mean squared error. A related statistic is commonly used that reflects the level of system noise. The measure of total response variability accounted for by the model, R2, is a ratio between 0 and 1, where higher numbers indicate better model fits, and correspondingly less system noise. R2 values vary greatly depending on the environment, but values greater than 0.7 are deemed acceptable in most cases, while values greater than 0.90 are often considered excellent. In most wind tunnel applications R2 values routinely exceed 0.99 and values of 0.995 are not uncommon. Because R2 is essentially a signal to noise ratio, the high values obtained in wind tunnels also reflect the relative ease in identifying the signal, the system influences (e.g. control surface deflections) driving changes in the responses.

Wind tunnel experiments often involve changing a number of factors and the functional relationship between factor changes and responses such as lift and drag is at least quadratic. As such, model fitting using a polynomial expansion can easily involve dozens of model terms. Even so, the data analysis for regression models with such high signal to noise ratios would appear to be somewhat trivial. A few considerations should accompany such a study. The potential model terms should be assessed for significance and only be included in the final model if statistically significant. In low noise studies even barely influential signals could be deemed statistically significant, so the analyst is often confronted with the decision of whether or not to include a fairly substantial subset of terms even though they are statistically significant. The decision to include model terms ultimately weighs the desire to fit the response well versus the need to properly interpret the factor-response relationships. For example, consider a static stability and control experiment conducted at the Langley Full Scale Tunnel using the tri-jet Blended-Wing-Body (BWB) concept developed by the Boeing Company6. The model has three pylon-mounted nacelles located on the upper surface of the aft center-body. The control surfaces consist of 18 elevons distributed along the trailing edge, rudders on each winglet and leading edge slats, as shown in figure 1. The two outboard elevons (labeled as “8 Upper / 6 Lower” and “9 Upper / 7 Lower”) split to serve as both elevons and drag rudders. Due to weight constraints the model was limited to eleven actuators. This required several of the control surfaces to be ganged to a single actuator so they move in unison as if they were a single combined surface. This was the case with elevons 2 thru 5, the upper elevons 8 and 9 and the lower elevons 6 and 7.

Resources for this exploratory project were limited so that a subset of representative factors was chosen. Angle of attack and sideslip angle were of course required factors for longitudinal and lateral aerodynamic characterization. Interest focused on possible interactions between control surfaces with particular regard to those located adjacent to each other. It was decided to choose the left (port) trailing edge elevons and winglet rudder and to include right influences using the right 2-5 ganged elevons. A mechanical constraint is present due to the collocation of lower surfaces 6-7 and upper surfaces 8-9. These can be deployed as drag rudders with 8-9 deflected up and 6-7 deflected down or as elevons with both deflected in the same direction. The deflection constraint which arises from their actuation limits and interference is given as: δL67 - δL89 > 5°. The eight factors, their nominal design levels, labels, and constraints are summarized in table 1.

4 American Institute of Aeronautics and Astronautics

To accommodate the particular mechanical constraints due to the test article configuration, a fractional factorial design was modified manually resulting in an asymmetric design space. The constraint and subsequent model modification are shown in two dimensional space in figure 2. Here the two effected factors are first shown in an unconstrained FCD. Applying the constraint results in a clipping of the top left corner of the square so as to move the top left factorial point and the upper face centered axial to the right. As a result, the orthogonality of the design was disrupted producing overlapping information when estimating the factor effects. Collinearity refers to the amount of overlap and can be quantified by computing the variance inflation factor (VIF) for each model term. In general, a VIF of less than ten is desirable, which was achieved in the modified design9. In this case, modifying a classical design was successful since the constraints were not too severe. Consequently, many of the desirable properties of the classical design were retained. Prior to the test the constrained FCD model was evaluated using a BWB flight simulation model developed from previous testing10.

The experimental design to study the eight constrained factors is based on a 128 run fractional factorial combined with 16 face-centered axial points and features eight replicated center points, requiring a total of 152 runs. The 128-run fractional factorial is only a ½ fraction, allowing for clear estimation of all two- and three-factor interactions. This design approach is not typical of traditional experimental strategies, in that the resolution of the initial design here provides the opportunity to completely and clear discern all potential model terms. The reason this approach is routine in wind tunnel experiments is that historical evidence and system knowledge provide sufficient justification that these interactions often play a role in characterizing the underlying aerodynamic performance. So the higher resolution (less fractionation and more experimental runs) designs with less of a sequential approach to test are commonly employed in wind tunnels. The upside of such a strategy is fewer interruptions between testing, less confusion in terms of discerning the truly significant model terms, and overall more confidence in the experimental design methodology. The downside is the increased chance of wasting runs, and performing more than is absolutely necessary to build the best models. In general though, the waste from such an approach is typically on the order of 10-20% over and above a test design that is usually 100-300% more efficient than the standard one-factor-at-a-time (OFAT) default strategy used in most wind tunnel tests.

The tests were conducted in two blocks, the first with the 128 factorial combinations and four center point runs, and the second with the 16 axial points and four centers. In terms of the anticipated regression polynomial, this design is capable of estimating all linear effects, interactions (two-factor and three-factor) and pure quadratic terms, totaling 101 regression parameters for each response model. The experiment was conducted with the intent of measuring all six aerodynamic forces and moments for each of the 152 runs. The tests were successfully conducted and the data was compiled. The analysis of the data resulted in the building of an empirical polynomial regression model for each aerodynamic coefficient. A technique for culling model terms called stepwise regression (using the recommended type I error thresholds) was employed. Each of the six models had different subsets of model terms, so consider the axial force, CA as an example. The chosen regression model using the default cutoff type I error values contained the terms shown in figure 3a. This model contains 26 model terms involving linear, two-factor interaction and pure quadratic terms. The model summary statistics reveal unusually high signal-to-noise ratios given by the 0.9985 R2

adj value. Some of the model terms (e.g. AG, BC) are 3 orders of magnitude less significant (in terms of F-values) than the most important model terms. So a revised model was constructed using more restrictive stepwise thresholds. The resulting revised model (figure 3b) contains only 16 terms, which is more manageable and easier to interpret. The associated loss in fit (0.9981 R2

adj) is acceptable. It is possible that further reductions in model terms may be necessary. An effective strategy for further trimming

model terms is to attempt to interpret some of the least significant model terms. For example, consider the AD interaction plot in figure 4. An interaction occurs when one factor produces an effect on the response that is dependent on the setting of another factor. Departures from parallel lines indicate an interaction is present. In this plot, the divergence of the lines is difficult to detect, making it more challenging to explain such an effect. So although the AD interaction is indeed statistically significant with a 0.005 p-value, the analyst must also determine practical significance prior to building the final model. In general, once system noise becomes appreciably small, challenges arise in developing a parsimonious model that contains the important terms for characterizing the system and for predicting future observations while not including too many terms.

In this study, interest focused on the probable synergistic behavior of adjoining trailing edge actuators. An example is seen in the normal force response to δR25 and δ1 shown in figure 5. This interaction plot shows that the normal force due to deflecting of both surfaces is greater than the sum of either surface deflected alone. While this is not surprising perhaps, it is identified directly and quantified with the DOE approach.

5 American Institute of Aeronautics and Astronautics

IV. Higher Order Modeling and Response Surface Methods For the majority of industrial systems, experimental designs adequate for first order, first order plus interaction,

or second order polynomials characterizing the factor-response model are all that is necessary. Polynomial terms beyond second order are rare, especially at a level of significance that rivals the lower order model terms. Wind tunnel testing for aerodynamic coefficients often require polynomial terms of third order or higher. So experimental designs must accommodate these added terms. Strategies for augmenting existing second order designs are available, but off-the-shelf third order designs are nonexistent. Still another aspect of wind tunnel studies not typically encountered in industrial settings is the desire to effectively model response performance over a broad range of factor settings, usually in high dimension (many factors) space. Accordingly, new experimental strategies must be developed for efficiently mapping the factor space, adapting to the need for varying density of data depending on the complexity of the underlying function at different locations in the input space. For example, consider the lift curve in figure 6, showing lift coefficient for the X-31 aircraft. For a fairly wide band of α, the lift coefficient is essentially linear, requiring only a few points for a valid fit. Over a fairly short α range (30 to 37 degrees, depending on β), the function is cubic, requiring a tighter density of points for a reliable fit. This field of study is a specialty within response surface methods called surface mapping. Again, a case study is used here to discuss some of the challenges particular to wind tunnel testing.

The intent of the X-31 wind tunnel test entry was to apply experimental design techniques to low-speed stability and control assessment of a high-performance aircraft, over a fairly broad range of attitude settings8. The aircraft model (figure 7) used in this study is a modified version of the 19 percent scale X-31 used by NASA Langley Research Center11. The model was unpowered and the vectored thrust paddles were removed; the canard, rudder, and outboard trailing edge elevons (shaded grey in figure 7) were actuated by servo motors. The outboard elevons were moved differentially to act as ailerons; that is to say that when the left surface was deflected upward the right surface was deflected downward an equal amount. The control surface position tolerance was approximately ±3°, suitable only for non-precision testing, like this exploratory project. The set up was fully automated to allow for randomized ordering of the test sequence; the attitude changes and control surface settings were commanded from the control room of the wind tunnel.

As is the nature of experimental design and analysis for high dimension, complex response surfaces, the challenges reside in the choice of design points for data collection and the model fitting method to be used in characterizing this underlying surface. On the design side of the issue, the approach used for this case study and recommended for further applications, is to begin investigation with a third order variant of a classic second order design. The classic design structures employed were the fractional factorial and the face centered (central composite) designs (FCD). The goal was to build a third order design capable of fitting and predicting over a cuboidal region of interest (common to most wind tunnel tests). Experience with stability and control testing of aircraft models suggested the need for at least five levels of control surface deflection. The approach involved inscribing an FCD within another FCD, allowing for data collection at 5 levels of each factor. One alternative is to not fractionate the corner points resulting in a full factorial FCD within another full FCD (as shown in two factors in figure 8). If further reduction in test runs is needed, one could consider fractionating the inner or outer FCD, or both. For the control surface deflections, each surface would be deflected to ±50% and ±100% of the available throw in addition to the neutral setting. After running the nested FCD, an analysis is performed prior to performing the next set of tests. The statistical analysis will assist in identifying factor space locations requiring additional test points to increase fitting precision. A number of optimality criteria are available for determining the best locations of additional points. These criteria attempt to either minimize the variance of the regression model coefficients or minimize aspects of prediction variance over the factor space. Computer software packages (such as JMP® or Design Expert®) are available for augmenting these initial designs.

There are also choices for methods to best fit these often complex response surfaces12. Least squares polynomial regression is the option most familiar to statistical analysts trained in experimental design. Two other options are worth considering. Kriging is a computational modeling technique which arose in geostatistics and is used routinely in the design and analysis of computer experiments13. The model contains both a global component (a function of the factors similar to regression) and a local Gaussian random process component. An example Kriging model is provided for the X-31 using the baseline data to fit the roll response (figure 9). Another promising method for fitting and prediction high dimension functions with limited data is multivariate adaptive regression splines (MARS). MARS essentially builds flexible models by fitting piecewise linear regressions, i.e. flexible splines14. An example MARS fit to the pitch response is provided in figure 10. There are distinct advantages and disadvantages to the various fitting techniques associated with the ability to conform to complex surfaces, adaptability to higher

6 American Institute of Aeronautics and Astronautics

dimension (increasing number of factors), interpretability, and tendency to overfitting such that the models predict poorly. Tradeoffs typically exist among these methods between ease of conformance to complex surfaces and a tendency to overfit. Likewise methods that fit complex surfaces easily also tend to have complex model structures such that the transparency between factor influence and the response is clouded. Figure 11 shows a comparison of the three modeling techniques for a bimodal surface consisting of a major and minor peak. The known surface is shown in the top left plot (figure 11a).

Specific comparisons to baseline data are used to “validate” the models and show that they well represent the data. In addition, a representative plot derived from the regression models is presented to illustrate their utility at providing traditional aircraft stability and control data. Aileron control power was evaluated using the regression models for rolling and yawing moment at five deflections for two angles of attack and is shown in figure 12. Here it is clearly shown that roll control power is more limited at 35° than at 10° while the resulting yawing moment response is relatively constant. Confidence is gained in this result after reviewing the uncertainty bounds. This representative control power plot is one example of classical test data results available using the RSM methodology. Interactions represent potential new results afforded by employing the RSM method. Interactions are much more difficult and time consuming to obtain using traditional OFAT methods yet the DOE/RSM approach will inherently identify the interactions as part of the test methodology.

V. Designed Experiments for Aircraft Design, Segmenting Designs and Restrictions on Randomization

Wind tunnel testing is also used to perform aircraft configuration design and trade studies, allowing for changes in fuselage, wing, or control surface orientation, size or special features. Designed experiments are well suited to this type of testing, but adaptations to standard thought processes are required. In this discussion we will be illustrating the principles using a previous test from an area receiving increased attention due to the need for military applications, Micro Unmanned Aerial Vehicles (MAVs). MAVs are small scale, remotely controlled aerial vehicles used for reconnaissance, intelligence gathering, and battle damage assessment. MAVs carry payloads such as sensors, cameras, and communications equipment. The case being studied involves a tandem wing MAV (figure 13) that is designed to have retractable wings for transport, and has control surfaces on the aft wing and two different vertical tail configurations15.

Stability and control characteristics are essential data to gather from wind tunnel testing. This issue is addressed primarily in the aircraft design phase and verified in wind tunnel and flight testing. One challenge pertaining to the MAVs’ stability and control evaluation is that the vehicles are flying at low speeds with relatively small mass and size, resulting in a very low Reynolds number. Because predictions for stability and control are difficult to validate using computational simulations, high quality data from wind tunnel testing are essential. Using traditional OFAT approaches to evaluating aircraft design variations for stability and control enhancements, discovery is often limited or results in a suboptimal decision, because the approach is not capable of identifying factor interactions nor is a response surface predictive model typically built. The advantage of a designed experiment and response surface modeling approach lies in the ability to quantify significant interactions, nonlinear effects, and estimate uncertainty, while requiring less test runs. One hurdle that must be overcome in this venue is the desire to completely randomize the run order. Proper analyses and valid results depend on randomization. However, complete randomization is often not practical when aircraft model configuration changes are an integral part of test. The departures from classic design of experiments needed for this environment are segmented testing (partitioning the entire set of tests into smaller tests with unique objectives) and experiments with restricted randomization. Restricted randomization testing has become prevalent in recent years, as has the discovery of tools to effectively design test matrices and build verifiable response surface models.

Split plot designs are appropriate when the experiment involves some factors that are difficult, time-consuming, or costly to change. The strategy involves partitioning these two types of factors to create the test matrix. A split plot design separates the hard-to-change (HTC) factors from the easy-to-change (ETC) factors to accommodate the randomization restrictions; the HTC factors will be referred to as whole-plot (WP) factors and the ETC factors as subplot factors (SP). Each group of WP runs is referred to as a WP. The WP factors within each WP will remain constant for different randomized run combinations of SP factors, yet these WPs will be run in a random order. Therefore, there are two distinct randomization schedules; and two different components of experimental error. Myers and Montgomery9 provide a detailed background and introduction to split plot design and analysis.

To gain an appreciation for the differences between standard factorial design/analysis for a completely randomized design and that required for a split plot design, consider an experiment involving four factors, two are

7 American Institute of Aeronautics and Astronautics

ETC (factors A, B), and two are HTC (factors C, D). Suppose a full factorial is desired subject to the necessary randomization restrictions for the HTC factors. One possible representation of such a test is shown in figure 14. Here the approach specifies that the experiment will be conducted such that one of the corners of the larger AB square will be run, so the A and B factor levels will be fixed, while a completely randomized design (smaller square) is run in factors C and D. The fifth point (grey shade) is a CD center point with A and B still set at their corner point levels and is used to test for nonlinear effects in C and D. Then another corner of the AB square is selected (randomly) and run. This procedure is followed until all tests are completed. This split plot design reduces the number of HTC factor changes from a maximum of the number of total runs, to only four. Huge time savings due to setup between runs are enjoyed. The analysis of split plot designs requires an analytical technique capable of estimating the two error components, one at the WP level and one at the SP level. General linear models are appropriate for this situation and fortunately several software packages offer this estimation method (see Myers and Montgomery9).

The experimental design tunnel entry for the MAV case study consisted of several sequential experiments involving different subsets of aircraft design (location of tandem wings, vertical wingtip height, and various propulsion options), control surface deflections, and attitude changes15. These separate experiments had distinct research objectives and the experiment design strategy allowed for hard-to-change factors as well as second order model fitting. The classic strategy in experimental design is to combine all factors together in a single test or series of increasing resolution sequential tests. The purpose in combining all factors is to enable estimation of all factor interaction combinations, a worthy objective. Sometimes, as happens in aircraft design, many aircraft configurations, involving certain combinations of factors, are nonsensical or physically impossible. Therefore, it is often appropriate to subdivide the testing into segments or groups of tests, changing only a subset of the factors while holding the remaining factors constant16. For the MAV design study, the testing involved four segments. The first segment consisted of a baseline study involving α, β, left and right control surface deflection. A completely randomized nested FCD was implemented due to the relative ease in changing factor settings. Segment 2 focused directional stability involving the height of the vertical tail located at the wing tips, α, and β. The vertical tail height represented a HTC, so a second order split plot design17-19 was necessary. The second order design used was a minimum whole plot FCD from a catalog created by Parker20. The third set of tests considered the vertical tail being placed at the side of the fuselage with rudder deflection, α, and β as factors. Another second order split plot design was necessary to accommodate the HTC rudder deflection. The final segment evaluated propulsion options, in which the current to the motor was a HTC factor. The ETC factors for this second order split plot design were α, left and right control surface deflections.

In terms of the experimental findings, the objective was to assess the effects of design changes on stability and control. For example, response surfaces and plots can be used to determine the preferred vertical tail size and location. These design configurations are expected to have an effect on L/D and Cn, and the magnitude of the difference of these effects for each of the vertical tail locations is desired. Additionally, examining the difference in magnitude and effects for each of the other five responses will be useful. The response surface (figure 15) of α versus yaw angle interaction for the L/D response with vertical tails located at the wingtips shows the nonlinear behavior in both factors, particularly α. The target value of Cn at α = 6 and β = 10 is 0.02 for directional stability. From figure 16 it is observed that only one configuration may be capable of achieving this target value under these conditions; wingtip height = 4 inches. Therefore, the recommended design for vertical tails due to the goals of directional stability is wingtips with a height of 4 inches.

VI. Conclusion A direct result of the experiment design protocol is that systematic and simultaneous changes in aircraft attitude,

configuration changes, and control surface deflections allow for estimation of independent influences. These influences can be due to multiple factors interacting, and highly nonlinear influences that heretofore are not customarily extracted using OFAT testing. Determining the proper testing configurations and ensuring timely collection of aerodynamic coefficient data can be best accomplished with a working knowledge of statistically-based experimentation methods. Experimental design techniques have been successfully applied for more than 75 years in the widest variety of test applications. The history of these techniques applied to wind tunnel test is brief in comparison, but those experiences have demonstrated the perfect match between the statistical tool and the testing requirements. This paper provides by example how classic experimental design and analysis can be tailored to the typical demands of low-speed wind tunnel testing. More details regarding the theory behind these adaptations are

8 American Institute of Aeronautics and Astronautics

available in the references21-22. The wind tunnel test engineer should not shy away from this discipline, as they can be trained in less than two weeks to confidently apply these methods in each and every tunnel entry.

References 1Landman, D., Simpson J. R., Hall, B., and Sumner, T., “Use of Designed Experiments in Wind Tunnel Testing of

Performance Automobiles,” SAE 2002-01-3313, SAE 2002 Transactions Journal of Passenger Cars - Mechanical Systems, Vol. 111-6, 2003, pp. 2339-2346.

2DeLoach, R., “The Modern Design of Experiments for Configuration Aerodynamics: A Case Study,” AIAA-2006-923, 44th AIAA Aerospace Sciences Meeting and Exhibit, Reno, Nevada, Jan. 9-12, 2006.

3DeLoach, R., and Berrier, B., Productivity and Quality Enhancements in a Configuration Aerodynamics Test Using the Modern Design of Experiments, AIAA-2004-1145, 42nd AIAA Aerospace Sciences Meeting and Exhibit, Reno, Nevada, Jan. 5-8, 2004.

4DeLoach, R., and Erickson, G., Low-Order Response Surface Modeling of Wind Tunnel Data Over Truncated Inference Subspaces, AIAA-2003-456, 41st Aerospace Sciences Meeting and Exhibit, Reno, Nevada, Jan. 6-9, 2003.

5Simpson, J. R., Kowalski, S. M., and Landman, D., “Experimentation with Randomization Restrictions: Targeting Practical Implementation,” Quality and Reliability Engineering International, Vol. 20, No. 5, 2004, pp. 481-495.

6Landman, D., Simpson J. R., .Vicroy D., and Parker P., “Response Surface Methods for Efficient Complex Aircraft Configuration Aerodynamic Characterization,” Journal of Aircraft, Vol. 44, No. 4, 2007, pp. 1189-1195.

7Landman, D. and Simpson J., “A Wind Tunnel External Balance Calibration using Design of Experiments,” AIAA paper 2007-1604, Air Force T&E Days Conference

8Landman, D., Simpson, J. R., Mariani, R. Ortiz, F. and Britcher, C., “Hybrid Design for Aircraft Wind-Tunnel Testing Using Response Surface Methodologies,” Journal of Aircraft, Vol. 44, No. 4, 2007, pp. 1214-1221.

9Myers, R. H., and Montgomery, D. C., Response Surface Methodology., Second Edition. John Wiley and Sons, 2002. 10Vicroy, D., Murri, D. and Grafton, S., “Low-speed, Large Angle Wind Tunnel Investigation of a Subsonic Blended-Wing-

Body Tri-jet Configuration,” NASA CDTM-10044, 2004. 11Banks, D. M. et al., “Low-Speed Longitudinal and Lateral-Directional Aerodynamic Characteristics of the X-31

Configuration,” NASA TM-4351, 1992. 12Ortiz, Francisco, “Development of a Multi-Criteria Designs for Global Optimization and Global Mapping of Systems and

Processes,” PhD Dissertation, Florida A&M University, 2006. 13Sacks, J., Welch, W.J., Mitchell, T.J., and Wynn, H.P (1989). “Design and Analysis of Computer Experiments”. Statistical

Science. 4(4), pp. 409-435. 14Friedman, J. H., (1991). “Multivariate Adaptive Regression Splines” The Annals of Statistics, Vol. 19, No. 1 pp. 1-67. 15English, T., “Application of Experimental Design for Efficient Wind Tunnel Testing: the Tandem Wing MAV Case,”

Master’s Thesis, Florida State University, 2007. 16Simpson, J. R., and Wisnowski, J. W., “Streamlining Flight Test Using Design and Analysis of Experiments,” AIAA

Journal of Aircraft, Vol. 38, No. 6, 2001, pp. 1110-1116. 17Parker, P. A., Kowalski, S. M. , and Vining, G. G., “Unbalanced and Minimal Point Equivalent Estimation Second-Order

Split-Plot Designs," Journal of Quality Technology, 39, 2007, pp. 376-388. 18Parker, P. A. , Anderson-Cook, C. M. , Robinson, T. J. , and Liang, L., “Robust Split-Plot Designs," to appear in Quality

and Reliability Engineering International, 2007. 19Kowalski, S. M. , Parker, P. A. , and Vining, G. G., “Tutorial on Split-Plot Experiments," Quality Engineering, 19, 2007,

pp. 1-15. 20Parker, P. A. (2005). Response surface design and analysis in the presence of restricted randomization: equivalent split-plot

designs (dissertation etd-03302005-194026). Blacksburg, VA: Virginia Polytechnic Institute and State University. 21Montgomery, D.C., Design and Analysis of Experiments, 6th ed., John Wiley and Sons, 2004. 22Box, G. E. P., Hunter, J. S., and Hunter, W. G. ., Statistics for Experimenters: Design, Innovation and Discovery, 2nd

Edition, John Wiley and Sons, 2005.

Figure 1. Blended Wing Body (BWB) Configuration Control Surfaces.

Table1. Blended Wing Body Factor Levels and Constraints

Factor Factor ID Low Center High Constraintsα A 4 7 10 noneβ B -5 0 5 none

δR25 C -30 -5 20 noneδ1 D -30 -5 20 none

δL25 E -30 -5 20 noneδL67 F -30 10 50 F - G > 5δL89 G -50 -15 20 F - G > 5δLrud H -20 5 30 none

9 American Institute of Aeronautics and Astronautics

Figure 2. Face Centered Design Modifications.

Figure 3. Regression model ANOVA and summary statistics for CA response in BWB test showing a) a model using the default stepwise settings and b) a trimmed model using more restrictive cutoffs.

ANOVA - Axial Force Coefficient ResponseStepwise cutoff = 0.100

Sum of Mean F p-valueSource Squares df Square Value Prob > F

Block 0.00694 1 0.006936Model 0.23075 26 0.008875 4062.76 < 0.0001 A-Alpha new 0.17426 1 0.174256 79769.44 < 0.0001 B-Psi 0.00002 1 0.000017 7.89 0.0057 C-R25 0.02000 1 0.020004 9157.03 < 0.0001 D-EL12 0.00413 1 0.004132 1891.70 < 0.0001 ANOVA - Axial Force Coefficient Response E-L25 0.02048 1 0.020480 9375.14 < 0.0001 Stepwise cutoff = 0.005 G-L89Up new 0.00148 1 0.001477 676.02 < 0.0001 H-LWRUD 0.00005 1 0.000050 22.89 < 0.0001 Sum of Mean F p-value AC 0.00072 1 0.000722 330.60 < 0.0001 Source Squares df Square Value Prob > F AD 0.00004 1 0.000039 17.85 < 0.0001 Block 0.00694 1 0.006936 AE 0.00058 1 0.000583 266.85 < 0.0001 Model 0.23064 16 0.014415 5065.08 < 0.0001 AG 0.00001 1 0.000009 4.11 0.0448 A-Alpha new 0.17681 1 0.176812 62128.33 < 0.0001 BC 0.00001 1 0.000011 4.85 0.0295 C-R25 0.02009 1 0.020091 7059.57 < 0.0001 BD 0.00001 1 0.000009 3.95 0.0490 D-EL12 0.00423 1 0.004226 1484.91 < 0.0001 BE 0.00001 1 0.000011 4.88 0.0289 E-L25 0.02067 1 0.020667 7262.11 < 0.0001 BG 0.00001 1 0.000010 4.37 0.0384 G-L89Up new 0.00177 1 0.001768 621.35 < 0.0001 CD 0.00002 1 0.000023 10.69 0.0014 H-LWRUD 0.00006 1 0.000056 19.85 < 0.0001 CE 0.00020 1 0.000196 89.55 < 0.0001 AC 0.00072 1 0.000720 252.97 < 0.0001 CG 0.00002 1 0.000017 7.64 0.0065 AD 0.00004 1 0.000037 12.87 0.0005 DE 0.00004 1 0.000043 19.75 < 0.0001 AE 0.00060 1 0.000595 209.10 < 0.0001 EG 0.00001 1 0.000006 2.79 0.0973 CD 0.00003 1 0.000026 9.17 0.0029 A^2 0.00011 1 0.000109 49.95 < 0.0001 CE 0.00019 1 0.000195 68.36 < 0.0001 C^2 0.00008 1 0.000084 38.41 < 0.0001 DE 0.00005 1 0.000049 17.12 < 0.0001 D^2 0.00016 1 0.000159 72.92 < 0.0001 A^2 0.00009 1 0.000094 33.06 < 0.0001 E^2 0.00015 1 0.000148 67.75 < 0.0001 C^2 0.00011 1 0.000115 40.37 < 0.0001 G^2 0.00001 1 0.000010 4.65 0.0329 D^2 0.00022 1 0.000224 78.84 < 0.0001 H^2 0.00001 1 0.000008 3.86 0.0514 E^2 0.00021 1 0.000209 73.35 < 0.0001Residual 0.00029 131 0.000002 Residual 0.00040 141 0.000003Cor Total 0.23797 158 Cor Total 0.23797 158

Model Summary Statistics Model Summary StatisticsAxial Force Coefficient Response Axial Force Coefficient Response

Std. Dev. 0.0015 R-Squared 0.9988 Std. Dev. 0.0017 R-Squared 0.9983Mean -0.0008 Adj R-Squared 0.9985 Mean -0.0008 Adj R-Squared 0.9981C.V. % 188.9920 Pred R-Squared 0.9982 C.V. % 215.7143 Pred R-Squared 0.9978PRESS 0.0004 Adeq Precision 236.5336 PRESS 0.0005 Adeq Precision 257.8594

10 American Institute of Aeronautics and Astronautics

11 American Institute of Aeronautics and Astronautics

Figure 5. Control Surface Interaction: CN as a function of δR25 and δ1.

0

+

-30-17.5

-57.5

20

-30 -17.5-5 7.5

20

-

CN

δ1 δR25

α = 6β = 0δL25 = 0δL67 = 0δL89 = 0δLrud = 0

0

+

-30-17.5

-57.5

20

-30 -17.5-5 7.5

20

-

CN

δ1 δR25

α = 6β = 0δL25 = 0δL67 = 0δL89 = 0δLrud = 0

0

+

-30-17.5

-57.5

20

-30 -17.5-5 7.5

20

-

CN

δ1 δR25

α = 6β = 0δL25 = 0δL67 = 0δL89 = 0δLrud = 0

Design-Expert® Softw are

Ca

C- -30.000C+ 20.000

X1 = A: AlphaX2 = C: R25

Actual FactorsB: Psi = 0.000000D: EL12 = -5.00000E: L25 = -5.00000F: L67Low new = 5.00000G: L89Up new = -8.32500H: LWRUD = 5.00000

C: R25

5.10000 6.67500 8.25000 9.82500 11.4000

Interaction

A: Alpha

Ca

- 0.08 00000

- 0.04 25000

- 0.00 500000

0.0325000

0.0700000

δR25= -30.0

δ = +20.0R25

Figure 4. Statistical versus practical significance with low noise models.

Baseline CL vs. α

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

-10 -5 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75α

CLβ = 0β = 4β = 8β = 12β = 16

RSM Low RSM High

Baseline CL vs. α

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

-10 -5 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75α

CLβ = 0β = 4β = 8β = 12β = 16

RSM Low RSM High

igure 6. Baseline Lift Curves at Sideslip and Identification of Model Subspaces. F

12 American Institute of Aeronautics and Astronautics

igure 7. Modified X-31 Model with Actuated Control Surfaces Shown Shaded in Gray. F

A +A -

B+

B-

A +A -

B+

B-

Figure 8. Nested FCD in Two Factors, A and B.

-200

2040

6080-20

-10

0

10

20

-0.04

-0.02

0

0.02

0.04

0.06

YawAngle of attackα

β

Cl

-

-

-200

2040

6080-20

-10

0

10

20

-0.04

-0.02

0

0.02

0.04

0.06

YawAngle of attackα

β

Cl

-

-

Figure 9. Kriging Fitted Surface for Baseline Cl.

13 American Institute of Aeronautics and Astronautics

Figure 10. MARS fit to pitch response CM as a function of angle of attack and yaw.

-20 0 20 40 60 80-20

0

20

-0.15

-0.1

-0.05

0

0.05

Yaw

CM response plot

Angle of attack

CM

14 American Institute of Aeronautics and Astronautics

-1

-0.5

0

0.5

1 -1

-0.5

0

0.5

1

0

0.5

1

1.5

X2

Response Surface of Major Minor Function

X1

-1

-0.5

0

0.5

1 -1-0.5

00.5

1

-0.5

0

0.5

1

1.5

X2

RSM

X1

-1

-0.5

0

0.5

1 -1-0.5

00.5

1

-0.5

0

0.5

1

1.5

X2

Kriging

X1

-1

-0.5

0

0.5

1 -1

-0.5

00.5

1

-0.5

0

0.5

1

1.5

X2

MARS

X1

-1

-0.5

0

0.5

1 -1

-0.5

0

0.5

1

0

0.5

1

1.5

X2

Response Surface of Major Minor Function

X1

-1

-0.5

0

0.5

1 -1-0.5

00.5

1

-0.5

0

0.5

1

1.5

X2

RSM

X1

-1

-0.5

0

0.5

1 -1-0.5

00.5

1

-0.5

0

0.5

1

1.5

X2

Kriging

X1

-1

-0.5

0

0.5

1 -1

-0.5

00.5

1

-0.5

0

0.5

1

1.5

X2

MARS

X1

Figure 11. Comparison of model fitting techniques using a) a bimodal surface with major and minor peaks, b) a polynomial regression RSM fit, c) a fit using Kriging, and d) and fit using MARS.

a) b)

c) d)

Cn , Cl vs. δa

-0.05

-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.04

0.05

-35 -30 -25 -20 -15 -10 -5 0 5 10 15 20 25 30 35

δa

Cn,

Cl

Cn a = 35Cl a = 35Cn a = 10Cl a = 10

(β = 0 , δc = 0°, δr = 0°)

Error Bars shown for RSM Low model

Cn α = 35Cl α = 35Cn α = 10Cl α = 10

Cn , Cl vs. δa

-0.05

-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.04

0.05

-35 -30 -25 -20 -15 -10 -5 0 5 10 15 20 25 30 35

δa

Cn,

Cl

Cn a = 35Cl a = 35Cn a = 10Cl a = 10

(β = 0 , δc = 0°, δr = 0°)

Error Bars shown for RSM Low model

Cn α = 35Cl α = 35Cn α = 10Cl α = 10

Figure 12. Aileron Control Power as a Function of Angle of Attack.

15 American Institute of Aeronautics and Astronautics

Figure 13. Tandem wing Micro UAV a) configured with wingtip rudders and propulsion and b) in the wind tunnel without wingtips or propulsion system.

a) b)

− D ++C−

− A +

+

B

−

− D ++C−

− A +

+

B

−

Figure 14. Split plot design structure for four factor factorial design with 2 HTC factors (A and B) and 2 ETC factors (C and D).

16 American Institute of Aeronautics and Astronautics

Primary Test Points Experimental Design Plans

Config AOA Beta L eft surface

Right surface 1 BCWT1 AOA1 0 0 0 Segment 1--Baseline Performance 2 BCWT1 AOA1 10 0 0 3 BCWT1 AOA1 0 -10 -10 1a: Central Composite Design in 4 factors 4 BCWT1 AOA1 10 -10 -10 1b: FCD in 4 factors Augmented with CCD axials 5 BCWT1 AOA1 10 -10 10 to build sequentially a nested 6 BCWT1 AOA1 0 -10 10 FCD in two blocks if necessary 7 BCWT1 AOA1 - 10 -10 10 10 BCWT1 AOA2 10 0 0 Segment 2--Wingtips 11 BCWT2 AOA2 10 0 0 Second-order Split-plot design in 3 factors 12 BCWT3 AOA2 10 0 0 1 WP (Wingtip Height) and 2 SP (AoA and Yaw Angle) 13 BCWT4 AOA2 10 0 0 Segment 3--Vertical Tail at Side of Fuselage with Rudder 14 BCWT4R10 AOA2 10 0 0 Second-order Split-plot design in 3 factors 15 BCWT4R10 AOA2 0 0 0 1 WP (Rudder Deflection) and 16 BCWT4R10 AOA2 - 10 0 0 2 SP (AoA and Yaw Angle) 17 BCWT1P1 AOA2 0 0 0 Segment 4--Propulsion Testing 18 BCWT1P1 AOA2 0 0 0 Second-order Split-plot design in 4 factors 19 BCWT1P1 AOA2 0 0 0 1 WP (Current to Motor) and 20 BCWT1P1 AOA2 0 0 0 3 SP (AoA, Lt and Rt Control Surfaces)

Table 1 Experimental Design Plans for MAV Primary Test Segments with Configuration Description

Config DescriptionB BodyC Canard baseline, 3 deg incidence, mounted fuselage fwd topW Wing, mounted lower aft fuselage, 1.5 deg incidence, 1" elevator chordT1 vertical tail, wing tips, 4" highT2 vertical tail, wing tips, 3" highT3 vertical tail, wing tips, 2" highT4 vertical tail, fuselage side, with rudder, 4" highT4Rx T4 with rudder deflected x deg (positive for trailing edge right)P1 Propulsion system configuration 1

17 American Institute of Aeronautics and Astronautics

Figure 15. Yaw Angle and AoA Interaction for L/D with Vertical Tails at Wingtips.

Design-Expert® Software

L/D6.7354

0.7862

X1 = B: AoAX2 = C: Yaw Angle

Actual FactorA: Wingtip Height = 3.00

-2.00

2.00

6.00

10.00

14.00

-10.00

-5.00

0.00

5.00

10.00

0.7

2.225

3.75

5.275

6.8

L/D

B: AoA C: Yaw Angle

Figure 16. Yaw Angle and wingtip height interaction for Yaw Moment.

18 American Institute of Aeronautics and Astronautics