2 Introduction to Response Surface Methodology

2.1 Goals of Response Surface Methods

• The experimenter is often interested in

1. Finding a suitable approximating function for the purpose of predicting a future response.

2. Determining what values of the x1, x2, . . . , xk’s are “optimum” with respect to the re-sponse y. “Optimum” is used in the sense of finding the x1, x2, . . . , xk’s which wouldyield one of the three experimental goals: a maximum, minimum, or a specific (target oraim) value of the response.

• Response surface procedures are primarily used to either (i) determine what are the optimumoperating conditions which minimize or maximize a response or (ii) determine an operatingregion in the design variable space for which certain operating specifications are met.

• Although the eventual goal is usually to answer certain questions regarding operating con-ditions, it is extremely important that the decision be made, at the outset, regarding whatexperimental design be used for data collection, that is, what factor levels should be consideredin the experimental process.

• Because model coefficients are estimated from experimental data, it is obvious that this es-timation can be accomplished effectively if proper thought is given to the question of whatexperimental design to use.

• It is assumed that the experimenter is concerned with a system involving a response y whichdepends on the input variables ξ1, ξ2, . . . , ξk.

• It is assumed that there exists an exact functional relationship E(y) = η(ξ) where E(y) isthe expected response at ξ = (ξ1, ξ2, . . . , ξk). This relationship, however, is usually unknown.

• The ξi’s, also called the natural or uncoded variables. It is assumed that the ξi’s can becontrolled by the experimenter with negligible error.

• The ξi’s are usually transformed using a linear transformation which center and scale each ofthe ξi’s. The transformed variables x1, x2, . . . , xk are called the design or coded variables.

ξi 100 150 200xi −1 0 1

where xi =

• Because the true form of η is unknown and is perhaps extremely complicated, the success ofRSM depends on the approximation of η by a simpler function restricted to some region ofthe independent ξi variables.

• Low order polynomials are most often used because of their local smoothness properties.Mathematically, we are assuming that over a limited region of interest R(ξ), the main char-acteristics of the underlying true function η(ξ) are adequately represented by the low orderterms in a Taylor series approximation of η(ξ).

1. If the approximating model is a linear model with only first order design variable terms

(first-order model), then

f(x) = b0 +k∑

i=1

bixi = β̂0 +k∑

i=1

β̂ixi.

15

This model may be useful when the experimenter is interested in studying η in narrowregions of x = (x1, x2, . . . , xk) where little or no curvature or interactions are present.

– This model is also used in situations where small pilot experiments are used as pre-liminary experiments leading to a more extensive experimental exploration. Theseexperiments are often sequential in nature, where the procedure “leads” the experi-menter toward the general region containing optimal values of (x1, x2, . . . , xk). Onesuch procedure is the path of steepest ascent (descent).

2. If the approximating model is a linear model with first order design variable terms and

their pairwise interactions (interaction model), then

f(x) = b0 +k∑

i=1

bixi +

= β̂0 +k∑

i=1

β̂ixi +

This model may be useful when the experimenter is interested in studying η in narrowregions of x1, x2, . . . , xk, that is, where little or no curvature is present.

3. If the approximating model is a linear model with all first and second order design

variable terms (second-order or quadratic model), then

f(x) = b0 +k∑

i=1

bixi +∑ k∑

i<j

bijxixj +

= β̂0 +k∑

i=1

β̂ixi +∑ k∑

i<j

β̂ijxixj +

This model may be useful when the experimenter is interested in studying η in widerregions of x1, x2, . . . , xk where curvature is expected to be present.

4. Occasionally, models of order greater than 2 (such as, cubic models) are used.

• It should be understood that the region of interest R(ξ) lies within a larger region calledthe region of operability O(ξ) which is defined as the region over which experiments couldbe conducted.

– Example: You are interested in determining the water temperature ξ which extractsthe maximum amount of caffeine η from a particular brand of coffee. The region ofoperability O(ξ) is between 0◦C and 100◦C. The region of interest R(ξ) is limited totemperatures near boiling, say between 95◦C and 100◦C.

• The fact that the polynomial is an approximation does not necessarily detract from its useful-ness. If a model provides an adequate representation in the region of interest, then analysisof a model fitted from data will approximate an analysis of the physical system.

16

2.2 Approximating Function Example

A two-factor (3× 3 or 32) factorial experiment with n = 2 replicates was run. Factor x1 representstemperature and x2 represents process time. The factor levels for x1 and x2 were coded as −1, 0,and 1. The process yield is labelled y.

• The true functional relationship between the response y and the variables x1 and x2 is

y = η(x1, x2) =

• An experiment was run yielding the following experimental data:

η(x1, x2) η(x1, x2) + εx1 x2 Rep (True y) (Observed y)-1 -1 1 7.7183 7.7917-1 -1 2 7.7183 7.8480-1 0 1 5.6065 5.6715-1 0 2 5.6065 5.3444-1 1 1 5.1353 4.9865-1 1 2 5.1353 5.12110 -1 1 9.4817 9.07430 -1 2 9.4817 9.71550 0 1 6.0000 6.33900 0 2 6.0000 5.80110 1 1 5.2231 5.12710 1 2 5.2231 4.93381 -1 1 12.3891 12.43401 -1 2 12.3891 12.32401 0 1 6.6487 6.55661 0 2 6.6487 6.74301 1 1 5.3679 5.17301 1 2 5.3679 5.3430

• A second-order regression model f(x1, x2) was fit to the data. The fitted model was

f(x1, x2) = b0 + b1x1 + b2x2 + b12x1x2 + b11x21 + b22x

22

= 5.89 + 0.98x1 − 2.38x2 − 1.09x1x2 + 0.28x21 + 1.41x22

• The plots show the relationship between the fitted regression model f(x1, x2) and the truefunctional form η(x1, x2).

• The first set of plots contain the three-dimensional surface plots of the true function

η(x1, x2) =

and the approximating function

f(x1, x2) = 5.89 + 0.98x1 − 2.38x2 − 1.09x1x2 + 0.28x21 + 1.41x22

• The second set of plots are contour plots of the same two surfaces.

17

1818

19 19

20

20

Example: Taylor Series Approximation

• For function η(x, y) = 5 + e.5x−1.5y and p0 = (0, 0) in some open ball B containing p0,define p = (x, y) and

P2(p,p0) = η(0, 0) +x

1!

∂η

∂x

∣∣∣∣(0,0)

+y

1!

∂η

∂y

∣∣∣∣(0,0)

+x2

2!

∂2η

∂x2

∣∣∣∣(0,0)

+xy

1!1!

∂2η

∂x∂y

∣∣∣∣(0,0)

+y2

2!

∂2η

∂y2

∣∣∣∣(0,0)

R2(p,p∗) =

3∑k=0

xky(3−k)

k!(3− k)!

∂kη

∂kx∂(3−k)y

∣∣∣∣p∗

where p∗ is a point on the line segment joining p and (0, 0).

• Taking the partial derivatives η(x, y) = 5 + e.5x−1.5y , and evaluating them at (0, 0) yields

∂η

∂x

∣∣∣∣(0,0)

= = .5e0 = .5

∂η

∂y

∣∣∣∣(0,0)

= = −1.5e0 = −1.5

∂2η

∂x2

∣∣∣∣(0,0)

= = .25e0 = .25

∂2η

∂y2

∣∣∣∣(0,0)

= = 2.25e0 = 2.25

∂2η

∂x∂y

∣∣∣∣(0,0)

= = −.75e0 = −.75

Thus, the second-order Taylor series approximation of η(x, y) = 5 + e.5x−1.5y about thepoint (0, 0) is

f(x, y) = 6 +0.5

1x +

−1.5

1y +

−.75

(1)(1)xy +

.25

2x2 +

2.25

2y2

= 6 + .5x − 1.5y − .75xy + .125x2 + 1.125y2

• But, this function, too, is unknown. So what to we do as statisticians? Although f(x, y)is unknown, we know a second-order Taylor series approximation exists. We then use ourdata to estimate the intercept and coefficients of the terms in a full second-order (quadratic)regression model above.

21

• Recall: For our data example, the fitted model using least-squares was

f̂(x, y) = 5.89 + .98x − 2.38y − 1.09xy + .28x2 + 1.41y2

• To find the critical point, take the first partial derivatives, equate to zero, and solve:

∂f̂

∂x=

∂f̂

∂y=

• Setting∂f̂

∂x= 0 and

∂f̂

∂y= 0 produces two equations and two unknowns.

• The solution is the critical point (x0, y0) = (−.50, .65).

• Next, perform the second-derivative. We need to evaluate the derivatives at critical point(x0, y0) = (−.50, .65): (

∂2f̂

∂x∂y

)2∣∣∣∣∣∣(−.50,.65)

= −1.09

(∂2f̂

∂x2

)∣∣∣∣∣(−.50,.65)

= .56

(∂2f̂

∂y2

)∣∣∣∣∣(−.50,.65)

= 2.82

Evaluating ∆ yields

∆|(−.50,.65) =

(∂2f̂

∂x∂y

)2

−

(∂2f̂

∂x2

) (∂2f̂

∂y2

)∣∣∣∣∣∣(−.50,.65)

= (−1.09)2 − (.56)(2.82) = −.39

But∂2f̂

∂x2> 0, so (−.50, .65) is a minimum for the fitted model (response surface).

22

2.3 Bias

• We assume the polynomial response surface model is an inadequate approximation of the truemodel. As a result, the model coefficients are biased by terms that are of order higher than theorder of the assumed model. For example, if a first-order model is fit when curvature exists,the coefficients are seriously affected by the exclusion of important interaction and squaredterms.

• Let y = (y1, y2, . . . , yn)′ be a vector of n responses.

• Let ε = (ε1, ε2, . . . , εn)′ be a vector of n random errors having zero vector mean.

• Let f(ξ) be the approximating function of η(ξ). The true functional relationship can berepresented by

y =

or, using the approximating model, by

y =

where δ(ξ) = η(ξ)− f(ξ) is the difference between the actual and approximating models. Wewould like this to be very small over the region of interest R(ξ).

• Thus, there are two types of errors which must be taken into account: (i) systematic, or biaserrors δ(ξ) = η(ξ)− f(ξ) and (ii) random errors ε.

• Although this implies that systematic errors δ(ξ) are unavoidable, there has been a tendencysince the time of Gauss (1777-1855) to ignore them and to concentrate only on the randomerrors ε. This has been done because nice mathematical and statistical results are possible,in particular, in hypothesis testing results that rely on normality of the random errors.

• When choosing an experimental design, ignoring systematic errors can be a serious problembecause the design point selection is based on an inadequate approximating model which, inturn, can lead to misleading results.

2.4 Fitted Model Example Using SAS

The following experimental data was used to fit the model y = b0 + b1x+ b2x2.

x 180 200 220 240 260 280 300y 82.0 89.2 94.5 99.6 102.5 103.2 103.8

• We will review the SAS output and what it represents.

• Note that σ̂2 = MSE = .15417 which is our estimate of σ2. Then

σ2(X′X)−1 ≈

The square root of the diagonal entries are the standard errors of the model parameter esti-mates:

s.e.(b0) =√

(.15417)(238.62) = 6.065

s.e.(b1) =√

(.15417)(.01723) = .0515

s.e.(b2) =√

(.15417)(.0000000744047) = .000107

23

22

24

23

25

24

26

25

27