Analyzing Experiments with Correlated Multiple Responses CHIH-HUA CHIAO Soochow University, Taipei, Taiwan MICHAEL HAMADA Los Alamos National Laboratory, Los Alamos NM 87545 Statistically designed experiments have been employed extensively to improve product or process quality and to make products and processes robust. In this paper, we consider experiments with correlated multiple responses whose means, variances, and correlations depend on experimental factors. Analysis of these experiments consists of modeling distributional parameters in terms of the experimental factors and finding factor settings which maximize the probability of being in a specification region, i.e., all responses are simultaneously meeting their respective specifications. The proposed procedure is illustrated with three experiments from the literature. Introduction A proactive method for improving quality and making products and processes robust is to use statistically designed experiments. Such experiments allow one to assess a variable’s impact on a product or process’s quality. For most products, quality is multidimensional, so it is common to observe multi- ple responses on the experimental units. There are numerous examples of experiments with multiple re- sponses that appear in the literature—see, for exam- ple, Symposia on T aguchi M ethods (American Sup- plier Institute (1984-1993)). When analyzing multi- ple response data, analyzing each response separately is not very satisfactory, especially when the multiple responses are correlated. For example, separate anal- yses may lead to conflicting recommendations regard- ing the levels of important factors, because a factor level may improve one response’s quality but degrade another’s. Moreover, when correlations between the responses are ignored, one may miss finding settings which simultaneously improve the quality of all the responses. Dr. Chiao is an Associate Professor in the Department of Business Mathematics. His email address is chchiao@bmath. scu.edu.tw. Dr. Hamada is a Technical Staff Member in Statistical Sciences. Derringer and Suich (1980) propose using desir- ability functions which turn the multiple response problem into a single response problem; that is, the response that is analyzed is the total desirability D =(d 1 (Y 1 ) ··· d m (Y m )) 1/m , where (Y 1 ,...,Y m ) are m responses and d 1 ,...,d m are individual desirabil- ities. There are two drawbacks to this approach: first, D will likely be harder to model because it is a complex function of the m responses; second, it is hard to say what the difference between expected values of D means except that the higher one is bet- ter. Other papers account for a common variance- covariance matrix of the multiple responses in a cri- terion to determine optimal settings (Khuri and Con- lon (1993) Khuri (1996) and Vining (1998)). In this paper, we consider the variance-covariance matrix as dependant on experimental factors as suggested by Pignatiello (1993). This might be viewed as an ex- tension of modeling variances (Bartlett and Kendall (1946)), which has been promoted by Taguchi (1986). We propose a simple method to properly handle mul- tiple responses. First, the parameters of the multi- ple response distribution are modeled in terms of the experimental factors. Then, factor settings that op- timize a suitable criterion are found. The criterion considered in this paper is the probability that all responses simultaneously meet their respective spec- ifications, a criterion that is easily interpreted. The paper is organized as follows. First, we con- Vol. 33, No. 4, October 2001 451 www.asq.org
Transcript
mss # ms305.tex; AP art. # 5; 33(4)
Analyzing Experiments with
Correlated Multiple Responses
CHIH-HUA CHIAO
Soochow University, Taipei, Taiwan
MICHAEL HAMADA
Los Alamos National Laboratory, Los Alamos NM 87545
Statistically designed experiments have been employed extensively to improve product or process quality
and to make products and processes robust. In this paper, we consider experiments with correlated multiple
responses whose means, variances, and correlations depend on experimental factors. Analysis of these
experiments consists of modeling distributional parameters in terms of the experimental factors and finding
factor settings which maximize the probability of being in a specification region, i.e., all responses are
simultaneously meeting their respective specifications. The proposed procedure is illustrated with three
experiments from the literature.
Introduction
Aproactive method for improving quality andmaking products and processes robust is to use
statistically designed experiments. Such experimentsallow one to assess a variable’s impact on a productor process’s quality. For most products, quality ismultidimensional, so it is common to observe multi-ple responses on the experimental units. There arenumerous examples of experiments with multiple re-sponses that appear in the literature—see, for exam-ple, Symposia on Taguchi Methods (American Sup-plier Institute (1984-1993)). When analyzing multi-ple response data, analyzing each response separatelyis not very satisfactory, especially when the multipleresponses are correlated. For example, separate anal-yses may lead to conflicting recommendations regard-ing the levels of important factors, because a factorlevel may improve one response’s quality but degradeanother’s. Moreover, when correlations between theresponses are ignored, one may miss finding settingswhich simultaneously improve the quality of all theresponses.
Dr. Chiao is an Associate Professor in the Department of
Business Mathematics. His email address is chchiao@bmath.
scu.edu.tw.
Dr. Hamada is a Technical Staff Member in Statistical
Sciences.
Derringer and Suich (1980) propose using desir-ability functions which turn the multiple responseproblem into a single response problem; that is, theresponse that is analyzed is the total desirabilityD = (d1(Y1) · · · dm(Ym))1/m, where (Y1, . . . , Ym) arem responses and d1, . . . , dm are individual desirabil-ities. There are two drawbacks to this approach:first, D will likely be harder to model because it isa complex function of the m responses; second, itis hard to say what the difference between expectedvalues of D means except that the higher one is bet-ter. Other papers account for a common variance-covariance matrix of the multiple responses in a cri-terion to determine optimal settings (Khuri and Con-lon (1993) Khuri (1996) and Vining (1998)). In thispaper, we consider the variance-covariance matrix asdependant on experimental factors as suggested byPignatiello (1993). This might be viewed as an ex-tension of modeling variances (Bartlett and Kendall(1946)), which has been promoted by Taguchi (1986).We propose a simple method to properly handle mul-tiple responses. First, the parameters of the multi-ple response distribution are modeled in terms of theexperimental factors. Then, factor settings that op-timize a suitable criterion are found. The criterionconsidered in this paper is the probability that allresponses simultaneously meet their respective spec-ifications, a criterion that is easily interpreted.
The paper is organized as follows. First, we con-
Vol. 33, No. 4, October 2001 451 www.asq.org
mss # ms305.tex; AP art. # 5; 33(4)
452 CHIH-HUA CHIAO AND MICHAEL HAMADA
FIGURE 1. Specification Regions for a Bivariate Response.
sider a specification region determined by each re-sponse’s own specifications, and then we considera multivariate normal distribution for the multipleresponses. Next, two classes of experimental de-signs are discussed that are used to improve qualityand to make quality robust. The multiple responsedata from such experiments can then be analyzed byestimating the parameters of the multivariate nor-mal distribution and then modeling them in termsof the experimental factors. This proposed analysismethodology is illustrated with data from three ex-periments for which the goals were quality improve-ment and robust quality improvement. The paperconcludes with a discussion.
A Specification Region and Modelfor Multiple Responses
First we consider a specification region for multi-ple responses or, in other words, a multivariate re-
sponse. For m responses, let Y = (Y1, Y2, · · · , Ym)T
denote the m × 1 vector of multiple responses or anm dimensional multivariate response. Each compo-nent response Yi has a specification (li, ui) consistingof lower and upper limits within which it is desirablefor Yi to be. Using the component specifications,we define a specification region S for the multivari-ate response as the m dimensional hypercube whosesides are the m component specifications. Figure 1displays the nine possible specification regions fora bivariate response (m = 2) as indicated by theshaded areas. Given a specification region, a mea-sure of quality is the probability that the m com-ponent responses are simultaneously meeting theirrespective specifications or the proportion of confor-mance (which Wang and Lam (1996) introduced fora univariate response)
P (Y ∈ S). (1)
In order to calculate the value in Equation (1) we
Journal of Quality Technology Vol. 33, No. 4, October 2001
mss # ms305.tex; AP art. # 5; 33(4)
ANALYZING EXPERIMENTS WITH CORRELATED MULTIPLE RESPONSES 453
FIGURE 2. Some 95% Contours and a Specification Region.
need a model for the multivariate response Y. Weassume that Y follows an m dimensional multivariatenormal distribution with mean µ = (µ1, µ2, · · ·,µm)T and variance-covariance matrix Σ (the diago-nal elements of which are the variances σ2
1 , σ22 , · · · , σ2
m
and the off-diagonal elements of which are the covari-ances ρk�σkσ�, 1 ≤ k < l ≤ m, where ρk� is the cor-relation between Yk and Yl). The joint probabilitydensity function for Y, f(Y;µ,Σ), is given by
f(Y;µ,Σ) = (2π)−(m/2)|Σ|−1/2
× exp[−1
2(Y − µ)
′Σ−1(Y − µ)
].
(2)
Assuming that the multivariate normal distributionfor Y in Equation (2) holds, the proportion of confor-
mance in Equation (1) can be evaluated for a given µand Σ. Figure 2 presents comparisons of two bivari-ate response distributions with different µ and Σ.The 95% contours for Y (i.e., the probability con-tent of a contour which is centered at its µ is 0.95.)indicate the shape of the distribution. Figure 2 il-lustrates three situations in which the specificationregion is a rectangle. In Figure 2a, only the means ofboth Y1 and Y2 are different, and case (a1) is better;the two cases, (a1) and (a2), have proportions of con-formance of 0.336 and 0.119, respectively. In Figure2b, only the variances of Y2 are different. Case (b2)is preferred because the Y2 variance for this case issmaller. The two cases (b1) and (b2) have propor-tions of conformance of 0.366 and 0.911, respectively.In Figure 2c, the correlations between Y1 and Y2 aredifferent; there is no correlation in (c1) and nega-
Vol. 33, No. 4, October 2001 www.asq.org
mss # ms305.tex; AP art. # 5; 33(4)
454 CHIH-HUA CHIAO AND MICHAEL HAMADA
FIGURE 3. The Inverse Hyperbolic Tangent Transformation of the Correlation (Rho).
tive correlation (ρ = −0.9) in case (c2). Case (c2)is preferred because of its proportion of conformanceof 0.501, which is comparatively better than 0.366for case (c1). Note that cases (a1), (b1), and (c1)are the same for purposes of comparison. Figures 2band 2c show that different variances and correlationscan impact the proportion of conformance.
As can be seen in Figure 2, the proportion ofconformance is determined by the means, variancesand correlations of the multivariate response distri-bution. In order to improve quality or make qualityrobust, one needs to first identify what variables im-pact these distributional parameters and then findthe values of these variables which make these goalsachievable. In the next section, we consider modelsfor these distributional parameters in terms of thesevariables which will be used to analyze experimentswith multiple responses.
Distributional Parameter Models
Let x denote a row vector of covariates associatedwith a set of factors which may include an intercept,main effects, two-factor interactions, etc. Considerthe following models for the distribution parameters
of the m dimensional multivariate response:
µk = xαk, k = 1, · · · , m, (3)
log(σ2k) = xβk, k = 1, · · · , m, (4)
and
tanh−1(ρk�) = xγk�, 1 ≤ k < l ≤ m. (5)
Note that αk, βk, and γk� are column vectors. Thenonzero components of αk, βk, and γk� determinethe impact of the associated variables. We refer toΘ as the vector of all the coefficients from Equations(3) - (5). Note that the standard log-linear model forvariances in Equation (4) ensures positive values forthe variances. All real values are allowed on the logscale so that the additive model given above is welldefined. A similar transformation is needed for cor-relations (ρ) which range between −1 and +1; herewe use the inverse hyperbolic tangent transformationin Equation (5) (Rao (1973)), which is defined as
tanh−1(ρ) =12
log(1 + ρ)(1 − ρ)
and is displayed in Figure 3. Perhaps a better expla-nation for using tanh−1(ρ) is as follows: note that0 ≤ (ρ + 1)/2 ≤ 1 is a proportion, that taking the
Journal of Quality Technology Vol. 33, No. 4, October 2001
mss # ms305.tex; AP art. # 5; 33(4)
ANALYZING EXPERIMENTS WITH CORRELATED MULTIPLE RESPONSES 455
logit transformation of a proportion is common prac-tice (i.e., the basis of logistic regression), and thatone can show that tanh−1(ρ) = 0.5logit((ρ + 1)/2).
Experiments and Their Goals
Statistically designed experiments provide proac-tive means for identifying the important factors thatimpact quality and for determining factor levels thatlead to quality gains. Fractional factorial designs areused when a large number of factors need to be stud-ied in a relatively small number of runs. For example,to study factors F1, . . . , Fr, each at two levels, a 2r−p
fractional factorial design might be used. In order toachieve robust quality in Taguchi’s robust parame-ter design paradigm, one needs to find the levels ofthe control factors (which are controllable) so thatthe quality is insensitive to the variation of so-callednoise factors (which are hard or impossible to con-trol). Crossed array designs are often used in robustquality experiments. A crossed array consists of acontrol array, the array of settings for control factorsF1, · · ·, Fr, and a noise array, the array of settingsfor noise factors N1, · · ·, Ns. In the experiment, eachnoise factor setting is performed with each controlfactor setting.
The form of the covariate vector x depends onthe type of experiment. For fractional factorial de-signs, the x consists of covariates corresponding to anintercept, main effects, and two-factor interactionsinvolving the factors F1, . . . , Fr. The crossed arraydesign allows an intercept, all Control main effects(and possibly some Control×Control interactions),all Control×Noise interactions, and all Noise maineffects to be estimated; Control and Noise refer tothe control factors F1, · · ·, Fr and noise factors N1,· · ·, Ns, respectively. For quality improvement, theexperimental goal is to maximize the proportion ofconformance in Equation (1), which is the proportionof products which simultaneously meet all specifica-tions. For robust quality improvement, the experi-mental goal is to maximize the proportion of confor-mance in Equation (1) while accounting for variationin the noise factors. To do this, one must first iden-tify the distributional parameter models in Equa-tions (3)–(5) for the multivariate normal responseand use them to find the factor settings that accom-plish the goal. Thus, the proportion of conformancedepends on the factor levels x, which is expressed as
P (Y ∈ S | x). (6)
The control and noise factors in a robust quality ex-periment are denoted by x = (xcontrol,xnoise), so the
goal for robust quality experiments is to maximize
where Pnoise indicates that the variation in the noisefactors is accounted for.
Analysis Methodology forMultiple Responses
Consider experiments in which there are n repli-cates at each experimental run, i.e., each factor set-ting in the experimental design. We propose a sim-ple method in which one first estimates the distri-butional parameters of the multivariate normal re-sponse, µ and Σ, at each run and then identifies theexperimental factor models (3)-(5). We use samplemeans, variances, and correlations to estimate µ andΣ as follows:
µ̂k =
n∑i=1
Yki
n,
σ̂2k =
n∑i=1
(Yki − µ̂k)2
n − 1,
and
ρ̂k� =
n∑i=1
n∑j=1
(Yki − µ̂k)(Ylj − µ̂l)√
n∑i=1
(Yki − µ̂k)2n∑
j=1
(Yli − µ̂l)2,
1 ≤ k < l ≤ m,
where Yki is the ith replicate of the kth response fora given run, i =1, · · ·, n. The experimental factormodels (3)-(5) can then be fit using the (µ̂k, σ̂2
k, ρ̂k�)to obtain least square estimates (LSE’s) for the fac-torial effects Θ given in these models.
Standard methods for analyzing unreplicated ex-periments apply here, such as normal or half-normalplots to determine the significance of the effects(Daniel (1959)). Since estimating variances and cor-relations is much harder than estimating means, con-cern arises when the number of replicates in an ex-periment is small. Hamada and Balakrishnan (1998)study formal tests of effect significance. It turns outthat log σ̂2 and tanh−1ρ̂ are approximately normalwith constant variance: 2/(n−1) for log σ̂2 (Bartlettand Kendall (1946)) and 1/(n − 3) for tanh−1ρ̂
Vol. 33, No. 4, October 2001 www.asq.org
mss # ms305.tex; AP art. # 5; 33(4)
456 CHIH-HUA CHIAO AND MICHAEL HAMADA
TABLE 1. Factors and Levels for Wheel
Cover Component Experiment
LevelFactor −1 +1
F1: mold temperature 80 110F2: close time 16 21F3: booster time 1.88 1.7F4: plunger time 4 8F5: pack pressure 1300 1425F6: hold pressure 1100 700F7: barrel 490 505
(Fisher (1970)). For a two-level, four-factor, full fac-torial design (24), the number of runs m is 16. TheLSE’s of the effects then have a standard deviationof
√2/[m(n − 1)] for log σ̂2 and
√1/[m(n − 3)] for
tanh−1ρ̂. The results of Hamada and Balakrishnan(1998) for the Lenth method (Lenth (1989)) indicatethat the power of detecting an effect one and a halftimes the standard deviation is about 0.70 and thatthe power is below 0.40 for an effect equal to thestandard deviation. The fact that the transformedsample variances and correlations are approximatelynormal with constant and known variances allows asimple formal test: since the LSE’s are normal, theLSE’s divided by their standard deviations are nor-mal and can be compared with ±1.96 for a 5% test.The power of such tests is easily computed: 0.99 foran effect one and a half times the standard deviationand 0.81 for an effect equal to the standard devia-tion. Thus, these simple tests are much more power-ful than the Lenth method, and the power increasesfor larger designs with run size m, say 32 runs. Also,smaller effects can be detected for more replicates nsince the standard deviation of log σ̂2 is
√2/(n − 1)
and that of tanh−1ρ̂ is√
1/(n − 3).
Based on the estimates of the significant effects,factor settings can then be found that maximize theproportions of conformance in Equations (6) and(7) in quality improvement and robust quality im-provement experiments, respectively. When thereare three or more responses, the predicted variance-covariance matrix needs to be checked for positivedefiniteness, i.e. all its eigenvalues are positive soEquations (6) and (7) can be evaluated. As withany data analysis, it is worth checking model as-sumptions. Andrews et al. (1971, 1973) providesa comprehensive review of tests for multivariate nor-mality. The limitation for the experiments discussed
here is that the number of replicates may be toosmall. Checking for multivariate normality is illus-trated, however, in the third example in the nextsection.
Examples
The proposed methodology is illustrated withthree experiments from the literature. The first twofocus on quality improvement. The third experimentconsiders robust quality improvement.
Example 1: Wheel Cover Component Exper-iment
Harper, Kosbe, and Peyton (1987) report on anexperiment to find the optimum combination of in-jection molding parameters to minimize the imbal-ance of a plastic wheel cover component. Seven fac-tors thought to be potentially important to the com-ponent’s balance are listed in Table 1 as well as thelevels for each factor denoted by −1 and +1. The rel-evant quality characteristics of the wheel cover com-ponent are the total weight (Y1 in grams) and thebalance (Y2 in inch-ounces) of the component. Thetwo-sided specification region for Y1 and Y2 is definedby (710, 715) and (0.3, 0.4), respectively. The datafor eight runs of a 27−4 fractional factorial design areshown in Table 2 in which five components were mea-sured in each run (Box, Hunter, and Hunter (1978)).Only main effects of the seven factors can be studiedin this experiment.
Estimates for Θ, which includes all main effects{αp, βp, γp} for p = 1, · · ·, 7 and their intercepts,respectively, were obtained using the least squaresmethod. The least square estimates (LSE’s) of the ef-fects have standard deviations of
√2/[m(n − 1)] for
log σ̂2 and√
1/[m(n − 3)] for tanh−1ρ̂, where m = 8and n = 5. Using these standard deviations for theLSE’s of the effects for σ̂k (k=1,2) and tanh−1ρ̂ andLenth’s approach for µ̂k (k=1,2), we identify the fol-lowing experimental factor models:
µ̂1 = 720.763 + 1.873x1 + 5.318x5
− 3.408x7,
µ̂2 = 0.967 + 0.113x1 + 0.328x5
− 0.174x7,
log σ̂21 = 0.944 − 0.509x2 + 1.189x4
+ 1.196x5 − 0.487x7, (8)log σ̂2
2 = −4.797 − 0.692x2,
tanh−1(ρ̂12) = −0.116 + 0.490x1 − 0.505x5,where x1, x2, x4, x5 and x7 are the covariates corre-sponding the five significant factors.
Journal of Quality Technology Vol. 33, No. 4, October 2001
mss # ms305.tex; AP art. # 5; 33(4)
ANALYZING EXPERIMENTS WITH CORRELATED MULTIPLE RESPONSES 457
TABLE 2. Experimental Design and Bivariate Responses for Wheel Cover Component Experiment
Using the experimental factor models, the propor-tion of conformance in Equation (6) can be evaluatedfor all 128 (= 27) possible factor settings, with eachfactor taking on levels −1 and +1. We use the IMSLFORTRAN subroutine bnrdf to evaluate the propor-tion of conformance (IMSL 1999). Based on valuesof the proportion of conformance, the optimal set-ting is xopt = (−1,+1,−,−1,−1,−, +1), where aninsignificant factor is denoted by −. That is, theoptimal setting is level −1 for F1, F4, and F5 andlevel +1 for F2 and F7, for which (µ̂1, µ̂2, σ̂1, σ̂2, ρ̂) =(710.16, 0.35, 0.30, 0.06,−0.10). The proportion ofconformance at the optimal setting is 0.515. Thisproportion of comformance seems low and suggests
that exploration outside the current experimental re-gion be considered for further improvement.
To illustrate the potential benefits of the proposedprocedure, consider what would result from usingtwo existing approaches which evaluate (i) only themeans and (ii) only the means and variances. Thebest three settings are presented in Table 3. Therank vector orders the settings by proportion of con-formance for approaches (i) and (ii).
(i) Identify significant factors using the meanmodel (3), and find the optimal settings whosemeans are the nearest to the center of the speci-fication region. We use the experimental factor
TABLE 3. Best Settings for Wheel Cover Component Experiment
Setting F1 F2 F4 F5 F7 P (Y ∈ S|x) µ̂1 µ̂2 σ̂1 σ̂2 ρ̂ Rank
models for the means in Equation (8) to evalu-ate (µ̂1, µ̂2) for all possible factor settings. Thesetting x = (+1,−1,−1,−1,−1,−1, +1) pro-duces the minimum squared distance between(µ̂1, µ̂2) and the center of the specification re-gion ((710+715)/2, (0.3+0.4)/2), which is 2.04;however, its proportion of conformance is only0.068 based on all of the experimental factormodels in Equation (8), which is much smallerthan the 0.515 for the optimal setting found bythe proposed procedure.
(ii) Identify significant factors using the mean andvariance models ((3) and (4)), and find the op-timal settings whose means are the nearest tothe center of the specification region with mini-mum variance. We use the experimental factormodels for the means and variances in Equa-tion (8) to evaluate (µ̂1, µ̂2, σ̂
21 , σ̂2
2) for all pos-sible factor settings. The criterion used to eval-uate the settings is the sum of the squareddistance given in (i) and the two variances(σ̂2
k, k = 1, 2). The optimal setting using thiscriterion is x = (+1, +1, +1,−1,−1, +1,+1)whose value is 2.13. The corresponding propor-tion of conformance is only 0.003 based on allof the experimental factor models in Equation(8), which is again much smaller than the 0.515for the optimal setting found by the proposedprocedure.
Example 2: Plasma Enhanced ChemicalVapor Deposition Experiment
Tong and Su (1997) report on an experiment toimprove a plasma enhanced chemical vapor depo-sition (PECVD) process in the fabrication of inte-grated circuits. In this experiment, eight factors areselected, the names and corresponding levels of whichare given in Table 4.
The two quality characteristics are the deposition
thickness (Y1) in Angstroms (◦A) and a refractive in-
dex (Y2). The specifications are (950, 1050) for Y1
and (1.90, 2.10) for Y2. An L18 orthogonal array isused to accommodate the one two-level factor andseven three-level factors studied in the experiment,and each of the 18 runs was replicated five times(Taguchi (1986)). The experimental design and bi-variate response data are displayed in Table 5. Theexperimental factor models in Equations (3)–(5) con-sisting of the main effects are fit, and their estimatesappear in Table 6. Note that there are two esti-mates for each of the three-level factors correspond-ing to linear and quadratic effects which appear inthis order. The covariates in models (3)-(5) for three-level factors corresponding to levels 1-3 are (−1, 0, 1)for the linear effect and (1,−2, 1) for the quadraticeffect. Here, the standard deviations of the LSE’sare
√2/[m(n − 1)] for log σ̂2 and
√1/[m(n − 3)] for
tanh−1ρ̂, where m = 18 for the two-level factor ef-fect, m = 12 for the three-level factor linear effect,and m = 36 for the three-level factor quadratic ef-fect. The significant effects are identified by asterisksin Table 6.
Using the experimental factor models in Equa-tions (3)–(5) with the estimates of significant effectsgiven in Table 6, the proportion of conformance inEquation (6) can be evaluated at all 4,374(= 2× 37)factor settings. The five best settings appear in Ta-ble 7, and xopt = (2, 1, 3, 1, 2, 2, 2, 3) is the optimumsetting; i.e., level 1 for F2 and F4, level 2 for F1, F5,F6, and F7, and level 3 for F3 and F8. At this optimalsetting the proportion of conformance is 0.719. Theoriginal setting, xoriginal = (2, 1, 2, 2, 2, 2, 2, 2), has aproportion of conformance of 0.530, so the proposedmethod can produce an improvement of more than36%.
TABLE 4. Factors and Levels for Chemical Vapor Deposition Experiment
LevelFactor 1 2 3
F1: cleaning method No YesF2: chamber temperature 100◦C 200◦C 300◦CF3: batch after cleaned chamber 1st 2nd 3rdF4: flow rate of SiH4 6% 7% 8%F5: flow rate of N2 30% 35% 40%F6: chamber pressure 160 mtorr 190 mtorr 220 mtorrF7: R. F. power 30 watt 35 watt 40 wattF8: deposition time 11.5 min 12.5 min 13.5 min
Journal of Quality Technology Vol. 33, No. 4, October 2001
mss # ms305.tex; AP art. # 5; 33(4)
ANALYZING EXPERIMENTS WITH CORRELATED MULTIPLE RESPONSES 459
TABLE 5. Experimental Design and Bivariate Responses for Chemical Vapor Deposition Experiment
To illustrate why it is necessary to model dis-tributional parameters in terms of the experimentalfactors, observe what happens when one assumes aconstant variance-covariance matrix for this exam-ple. Using the models for the means in (3) and opti-mizing the proportion of conformance for a constant
variance-covariance matrix, the optimal setting is(1, 1, 1, 1, 3, 2, 2, 3). The actual proportion of confor-mance for this setting (evaluated under all of the ex-perimental factor models (3)–(5)) is only 0.502, eventhough the location (µ̂1 = 975.93 and µ̂2 = 2.01) isclose to the middle of the specification region. Com-
TABLE 7. Best Settings for Chemical Vapor Deposition Experiment
Rank F1 F2 F3 F4 F5 F6 F7 F8 P (Y ∈ S|x) µ̂1 µ̂2 σ̂1 σ̂2 ρ̂
pare this with the value 0.719 for the optimal settingfound with the proposed method.
Example 3: LED Experiment
An experiment is performed to improve the relia-bility of light emitting diodes (LEDs) and make re-liability robust. The luminosity of an LED degradesover time, and the slower its degradation, the higherthe reliability. An LED is composed of a die, a solid-state component, which is attached by silver epoxyto the lead frame which carries the electric current.The control factors chosen for the experiment are dievendor (F1), type of epoxy material (F2), and leadframe design (F3). In the manufacturing process, thelocation on the lead frame where the die is attachedvaries. One goal considered in Chiao and Hamada(1996) for this experiment is how to make the LEDreliability robust to the die location variation; hence,the die location (Factor N) is a noise factor. Theexperimental design used is a crossed array consist-ing of an eight-run control array and a two-run noisearray as displayed in Table 8 (Taguchi (1986)). Notethat all factors are studied at two levels (denoted by−1 and +1), giving eight different LED designs (asspecified by the eight-run control factor array), eachof which uses two different die locations (as specifiedby the two levels in the noise factor array); for the ex-periment, the die location needs to be specially con-trolled so that information about die location can beobtained. This yields 16 different control and noisefactor level combinations performed in the experi-ment in which a sample of thirty LEDs is tested.
In the experiment, the luminosities of the LEDsare inspected ten times. The observed relative degra-dation rate R(t) is defined in terms of the luminosityat the third inspection (at t3 = 192 hours) because
the degradation paths stabilize by then:
R(t) =L(t3) − L(t)
L(t3), (9)
where L(t) denotes the luminosity in lumens at timet.
To illustrate the proposed methodology for mak-ing quality robust, consider a bivariate response con-sisting of Y1 = R(500) and Y2 = R(1000) for R(·),which is defined in Equation (9). Scatter plots of(Y1, Y2) for the thirty LEDs at each of the sixteenfactor level combinations in the experiment are dis-played in Figure 4. For purposes of illustration, thespecifications for (Y1, Y2) are taken to be (0.0, 0.05)and (0.0, 0.15), although the ideal R(t) would be zerofor all t. The specifications for (Y1, Y2) are based onthe expectation that the degradation rates should bebelow 5% and 15% at the first 500 and 1,000 hoursof operation, respectively.
Before fitting the experimental factor models inEquations (3)–(5), consider a simple procedure tocheck the bivariate normality assumption. For eachLED in a run, define the Mahalanobis distance as
d2 = (Y − µ̂)T Σ̂(Y − µ̂), (10)
where µ̂ and Σ̂ are estimated from the replicatesfrom the run. Then d2 follows approximately a χ2
distribution with m degrees of freedom for an m di-mensional multivariate normal response. Thus, achi-squared probability plot of the Mahalanobis dis-tances should be a straight line if the multivariatenormal assumption holds. Figure 5 displays the chi-squared probability plot (with 2 degrees of freedom)of the Mahalanobis distances for the LED experi-ment and suggests that the bivariate normal assump-tion is reasonable. Note that a number of repli-cates are required for each run in order to estimate(µ, Σ) needed in Equation (10). For a bivariateresponse there are five parameters, so at least fivereplicates are required. Next, we fit the experimen-tal factor models in Equations (3)–(5) for the LEDexperiment. The covariates used correspond to threeControl main effects, three Control×Control inter-actions, one Noise main effect, and three Control×Noise interactions. The LSE’s of the effects forlog σ̂2 have a standard deviation of
√2/[m(n − 1)]
and those for tanh−1ρ̂ have a standard deviation of√1/[m(n − 3)], where m = 16 and n = 30. The half-
normal plots (Daniel, 1959) given in Figure 6 are usedto identify the following experimental factor models(i.e., those effects whose absolute estimates fall abovethe line through the smallest absolute estimates
Vol. 33, No. 4, October 2001 www.asq.org
mss # ms305.tex; AP art. # 5; 33(4)
462 CHIH-HUA CHIAO AND MICHAEL HAMADA
FIGURE 4. Scatter Plots of Bivariate Responses For LED Experiment.
FIGURE 5. Mahalanobis Distance Plot For LED Experiment.
Journal of Quality Technology Vol. 33, No. 4, October 2001
mss # ms305.tex; AP art. # 5; 33(4)
ANALYZING EXPERIMENTS WITH CORRELATED MULTIPLE RESPONSES 463
where x1, x2, x3 are the covariates correspondingto the three control factors and xN is the covariatecorresponding to the noise factor. Based on theseidentified models, the proportion of conformance inEquation (7) for each setting specified in the controlarray is computed. Here, for purposes of illustra-tion, we assume that the distribution of the noisefactor is N(0, (1/3)2) so that the bulk of the dis-tribution is within the experimental region [−1, 1].Table 9 presents the proportions of conformance foreach control array setting, and it shows that the op-timum settings are (−1,−1, +1) for factors F1, F2,and F3, respectively; that is, run 5 has the largestperformance of conformance with a value of 0.952.
Conclusions
In this paper, we have proposed a simple methodfor improving quality and for achieving robust qual-ity using statistically designed experiments with mul-tiple responses. While the experimental plans arethe same ones used for improving any quality char-acteristic, it is the simultaneous analysis of multipleresponses that requires new consideration. Both themean vector µ and the variance-covariance matrix Σ
of the multivariate response distribution are modeledin terms of experimental factors. Once such mod-els are identified, optimal settings can then be de-termined using a particular criterion; in this paper,we have used the proportion of conformance, or theprobability that all responses simultaneously meettheir respective specifications. Other criteria mightbe considered.
It is important that practitioners be aware ofthe method’s limitations. This method depends onthe normality assumption which can be checked bythe chi-squared probability plot of Mahalanobis dis-tances. Given these concerns, it is important to runconfirmation experiments at the recommended set-tings. With the fitting of multiple models and po-tentially poor parameter estimation, it is prudent tocheck whether or not the predictions at those settingsare valid.
Acknowledgments
C. H. Chiao was a Visiting Fellow at the Univer-sity of Michigan when research for this paper wasperformed and is grateful for financial support fromthe Ministry of Education of Taiwan. M. Hamada’sresearch was supported by NSF grant DMS-9704649.The authors thank the referees whose insightful com-ments helped to improve a previous version of thispaper.
References
Andrews, D. F.; Gnanadesikan, R.; and Warner, J. �L.
(1971). “Transformations of Multivariate Data”. Biometrics
27, pp. 825–840.
Andrews, D. F.; Gnanadesikan, R.; and Warner, J. L.
(1973). “Methods for Assessing Multivariate Normality” in
Multivariate Analysis 3 edited by P. R. Krishnaiah. John
Wiley and Sons, New York, NY, pp. 825–840.
Bartlett, M. S. and Kendall, D. G. (1946). “The Statisti-
cal Analysis of Variance-Heterogeneity and the Logarithmic
Transformation”. Journal of the Royal Statistical Society B
8, pp. 128–138.
Box, G. E. P.; Hunter, W. G.; and Hunter, J. S. (1978).
Statistics for Experimenters. John Wiley and Sons, New
York, NY.
Chiao, C. H. and Hamada, M. (1996). “Using Degradation
Data from an Experiment to Achieve Robust Reliability for
Light Emitting Diodes”. Quality and Reliability Engineering
International 12, pp. 89–94.
Daniel, C. (1959). “Use of Half-Normal Plots in Interpret-
ing Factorial Two-Level Experiments”. Technometrics 1,
pp. 311–340.
Derringer, G. and Suich, R. (1980). “Simultaneous Opti-
mization of Several Response Variables”. Journal of Quality
Technology 12, pp. 214–219.
Journal of Quality Technology Vol. 33, No. 4, October 2001
mss # ms305.tex; AP art. # 5; 33(4)
ANALYZING EXPERIMENTS WITH CORRELATED MULTIPLE RESPONSES 465
Fisher, R. A. (1970). Statistical Methods for Research Work-
ers. Hafner Press, New York, NY.
Hamada, M. and Balakrishnan, N. (1998). “Analyzing Un-
replicated Factorial Experiments: A Review with Some New
Proposals”. Statistica Sinica 8, pp. 1–41.
Harper, D.; Kosbe, M.; and Peyton, L. (1987). “Opti-
mization of Ford Taurus Wheel Cover Balance (by Design
of Experiments—Taguchi’s Method)”. Fifth Symposium on
Taguchi Methods, Romulus, MI: American Supplier Insti-
tute, pp. 527–539.
IMSL (1999). IMSL Fortran Subroutines for Statistical Ap-
plications Manual, Stat/Library, Visual Numerics.
Khuri, A. I. (1996). “Multiresponse Surface Methodology”. in
Handbook of Statistics: Design and Analysis of Experiments
13. edited by S. Ghosh and C. R. Rao, Elsevier Science B.
V., New York, NY, pp. 377–406.
Khuri, A. I. and Conlon, M. (1981). “Simultaneous Opti-
mization of Multiple Responses Represented by Polynomial
Regression Functions”. Technometrics 23, pp. 363–375.
Lenth, R. V. (1989). “Quick and Easy Analysis of Unrepli-
cated Factorials”. Technometrics 31, pp. 469–473.
Pignatiello, J. J., Jr. (1993). “Strategies for Robust Mul-
tiresponse Quality Engineering”. IIE Transactions 25, pp. 5–
15.
Rao, C. R. (1973). Linear Statistical Inference and Its Appli-
cation. John Wiley and Sons, New York, NY.
Symposia on Taguchi Methods (1984-1993). Romulus, MI:
American Supplier Institute,
Taguchi, G. (1986). Introduction to Quality Engineering.
Asian Productivity Organization, Tokyo.
Tong, L. and Su, C. (1997). “Optimizing Multiresponse
Problems in the Taguchi Method by Fuzzy Multiple At-
tribute Decision Making”. Quality and Reliability Engineer-
ing International 13, pp. 25–34.
Vining, G. G. (1998). “A Compromise Approach to Multire-
sponse Optimization”. Journal of Quality Technology 30,
pp. 309–313.
Wang, C. M. and Lam, C. T. (1996). “Confidence Limits for
Proportion of Conformance”. Journal of Quality Technology
