Hae-Jin ChoiSchool of Mechanical Engineering,

Chung-Ang University

7. Kriging – Gaussian Process Model(Ch.11. 5 Experiments with Computer Models)

1

Introduction to Metamodel— Metamodel is ‘model of a model’— Computer simulation model (such as FEM) is a model of a physical

system.— Metamodeling is modeling the computer simulation model with a

mathematical construct. (i.e., model of a simulation model)— Why? Metamodel

— Computer simulation tends to require a large amount of computing resources (i.e., computationally expensive)

— Design optimization, which is an iterative process to find an optimum, cannot be performed with computationally expensive simulation model

— A metamodel reduces the computing expenses significantly, since it is an mathematical equation.

2

Metamodeling of computer experiments— Two types of computer simulation models

— Stochastic simulation model : output (response) is random variable— Deterministic simulation model : output (response) is deterministic by

mathematic behind the simulation. (i.e., NO CHANGE in INPUT then NO CHANGE in OUTPUT)

— For the optimization with Stochastic simulation model, traditional DOE techniques and RSM methodology may be applied.

— For the optimization with Deterministic simulation model, those are not available since it does not include randomness in response.

3

Metamodeling of computer experiments— Design of computer experiments: Space-filling methods

— Latin Hypercube Design— Sphere-Packing Design— Uniform Design— Maximum Entropy Design

— Modeling of computer experiments— Kriging – Gaussian Process Model

4

Kriging – a metamodeling method— Kriging (Gaussian Process Model) is a representative metamodeling

technique originated from Geostatistics. (D.G. Krige,South African geologist)

Original elevation data and sample points: 300 randomly placed points where elevation data was sampled.

Predicted elevation with 300 randomly placed points where elevation data was sampled.

(Source: http://casoilresource.lawr.ucdavis.edu/drupal/node/442) 5

Kriging Theory— Assumption:

— Response function is composed of a regression model and stochastic process.

1 2

1 2

[ ( ), ( ),..., ( )] : ( 1) vector of regression functions

[ , ,..., ] : ( 1) vector of unknown coefficients

Tp

Tp

where f f f p

pb b b

= ´

= ´

f(x) x x x

β

Kriging model

Z(s1)

Z(s2)

Z(s3)

Z(s4)Regression model

s1 s2 s3 s4

( ) ( ) Y Z= +Tx f(x) β x

6

Kriging Theory – continued..— In many case, is assumed to be a constant (simple or

ordinary kriging), or it may be first or second order polynomial.— Z(x) is assumed to be a Gaussian process with zero mean, E[Z(x)]=0,

and covariance, Cov [Z(xi), Z(xj)].— While globally approximates the design space, Z(x) creates

“localized” deviations so that the kriging model interpolate nsampled data.

— The covariance matrix of the samples is

Tf(x) β

Tf(x) β

2

s s

[ ( ), ( )] ([ ( , )])

is an (n n ) correlation matix with ones along the diagonal

[ ( , )] is correlation function

i j i j

i j

Cov Z Z R s swhere

R s s

s=

´

s s R

R

7

Kriging Theory – correlation function— Correlation function, , in the correlation matrix, R, is user

defined function and a variety of correlation function exist.— Gaussian correlation function (the mostly preferred correlation

function) is

( , )i jR s s

2

1( , ) exp

: unknown correlation parameters : the number of design variables and : components of the samples, and

k k

k k

ni j i j

kk

k

i j th i j

R

where

nk

q

q

=

é ù= - -ê ú

ê úë ûås s s s

s s s s

8

Kriging Theory – correlation function— Various correlation functions

( )e, k kk i jijwher d = -s s

9

Kriging Theory – correlation function

— Various correlation functions for0 ≤ ≤ 2

— Dash, full, and dashed-dotted line for

kijd

0.2, 1, 5kq = kijd k

ijd

kijd

kijd

0.5

0.2

1

10

Kriging for Prediction

— Predicted estimates at the untried point x is

Here,

ˆ ˆˆ( )

ˆ

Ywhere

=

=

T T -1

T -1 -1 T -1

x f(x) β + r(x) R (Y -Fβ)

β (F R F) F R Y

1 2

2

2

: correlation matrix[ , ,..., ]

[ ( , ), ( , ),..., ( , )][ , ,..., ]

s

s

ns

nT T T T

n

Ts s s

R R Ry y y

=

=

=

1

1

RF f(s ) f(s ) f(s )r(x) x s x s x sY

11

Kriging for Prediction

— Predicted estimates at the untried point x is

ˆ ˆˆ( )Y = T T -1x f(x) β + r(x) R (Y -Fβ)

Z(s1)

Z(s2)

Z(s3)

Z(s4)

s1 s2 s3 s4x12

Kriging Theory – correlation parameter

— How to determine the correlation parameters, ?— Maximizing Maximum Likelihood Estimator (MLE) to find the

optimum correlation parameters.

2

2

ˆln lnmaximize

2

ˆ ˆˆ (estimated process variance) /

s z

Tz s

n

where

n

s

s

é ù+-ê úë û

=

θ

-1

R

(Y -Fβ) R (Y -Fβ)

1 2[ , ,..., ]nq q q=θ

13

Kriging – different types of Kriging— Simple kriging assumes a known constant trend: .— Ordinary kriging assumes an unknown constant trend: .— Universal kriging assumes a general polynomial trend model, such as

linear trend model .

ˆ ˆˆ( )Y = T T -1x f(x) β + r(x) R (Y -Fβ)

14

Kriging Example

Ordinary Kriging Example

15

Kriging Example

16

Kriging Example

17

Kriging Example

18

Kriging Example

19

Kriging Example

20

Kriging Example

21

Latin Hypercube Design— A space-filling sampling technique with random permutation

(McKay, Conover, and Beckman,1979)— X1=[2,5,1,4,3,6], X2=[4,5,6,3,2,1]

Random field with two variables (uniform distribution)

Random field with two variables (normal distribution)

x1

x2

22

Maximum Entropy Design— Popular space-filling sampling technique (Shewry and Wynn 1987)— Entropy : a measure of the amount of information contained in the

distribution of a data set— The maximum entropy is achieve by maximizing determinant

of()

23

References— Kriging

— Sacks J., W. J. Welch, T. J. Mitchell, and H.P. Wynn, (1989) “Design and Analysis of Computer Experiments,” Statistical Science, Vol. 4, No. 4, pp. 409-435, Simpson, T., T.M, Mauery, J. J. Korte, and F. Mistree, “Kriging Models for Global Approximation in Simulation-Based Multidisciplinary Design Optimization,” AIAA Journal, Vol. 39, No. 12, 2001

— Latin Hypercube Design— McKay, M. D., Beckman, R. J. and Conover, W. J. (1979), “A comparison of

three methods for selecting values of input variables in the analysis of output from a computer code,” TechnometricsVol. 21, pp. 239-245.

— http://prod.sandia.gov/techlib/access-control.cgi/1998/980210.pdf— Maximum Entropy Design

— Shewry, M. C. and Wynn, H. P. (1987) “Maximum entropy sampling,” J. Appl. Statist., Vol. 14, pp. 165-170.

24