KE-90.5100
Process Monitoring (4cr)
Course information
• The course consists of– Lectures:
• Tue 14 – 16, Ke3• Thu 12 – 14, Ke3
– Exercises: • Fri 10:00 – 12:00, computer class room
– Exam:• Oct 31 2008• Jan 8 2009
Course information
• The course consists of– 5 obligatory homeworks (presented during exercises):
• Assistant: M.Sc. Cheng Hui • submit report by email within 2 weeks ([email protected])• All exercises have to be OK before exam
– Assignments: Group work to be presented at the end of the course
• The grade consists of – Assignment (30%)– Exam (70%)
Course material
• Course web pages
• Slides
• Handouts
• Exercises/Homework
• Material from assignment
Course Staff
• Hui Cheng, [email protected], reception Thu 10 –11, F302
• Alexey Zakharov
• Fernando Dorado
• Di Zhang
Scope of the course
Tools of the process control engineer
ClassicalControl-PID-Bode, -Nyqvist-…
ModernControl- Multivariable control- MPC- IMC-…
Intelligentmethods- Neural Networks- Fuzzy logic- GA
Multivariate data analysis-PCA-PLS-SOM
KE-90.5100 Process Monitoring
KE-90.2100 Basics of processautomation
KE-90.4510 Control applications in process industries
Modeling andsimulation- First principles modeling- Identification- Simulation of dynamic systems- …
KE-90.3100 Process Modelling and Simulation
Scope of the course
• After the course you will know– How and when to use some statistical process
monitoring methods– The basics of neural networks and fuzzy
systems and how to utilize them in monitoring and control
– The basics of genetic algorithms
Introduction
• The idea of process monitoring• Goals of process monitoring• The process monitoring loop• Process monitoring methods• Data selection • Data pretreatment• Univariate vs. multivariate statistics
The idea
To monitor or monitoring generally means to be aware of the state of a system
The idea
• Multivariate data in industrial processes: impossible for human operator to monitor hundreds of measurements for possible faults
• Costs and safety issues with equipment malfunctions / process disturbances – Shutdowns expensive– Amount of maintenance breaks– New equipment: delivery time– Safe working environment for plant staff
The idea
• Process equipment malfunctions and process disturbances– E.g. contamination of sensors, faults of
analyzers, clogging of filters, degradation of catalyst, changing properties of feed stock, leaks, actuator faults etc…
– How to detect early enough?– How to distinguish between?
The goals
Get indication of– process disturbances and– malfunctions in process equipment
As early as possible to– Increase uniform quality of the products– Improve safety– Minimize maintenance costs
The process monitoring loop
FaultIdentification
FaultDiagnosis
Process Recovery
FaultDetection
Fault Detection = determining whether a fault has occurredFault Identification = identifying the variables most relevant for diagnosing the faultFault Diagnosis = determining which fault occurredProcess Recovery = Removing the effect of the fault
yes
no
Process monitoring methods
Process monitoring
Qualitative Quantitative
Data-BasedModel-Based
Neural Networks
Statistical
+ Includes process knowledge − Needs process models
+ Easier to implement− Don’t include process knowledge
Process monitoring methodsProcess monitoring
Qualitative Quantitative
Data-BasedModel-Based
Residual based observersParity-space basedCausal modelsSigned digraphs
Statistical
PCAPLSRPCA
Neural Networks
SOMMLPRBFN..
PCAPLSRPCA..
Trend analysisRule-based
Scope of the course
introductory
+ GA and some Control
Data selection
• Training data includes:– If goal is to verify that process is in normal state
(monitoring) the process data used for training should represent normal conditions
– If the goal is to identify if the process is in a normal or some specified faulty state the process data used for training should include all the possible faulty states as well as the normal state
• Testing data– Completely independent from training data!
Data pretreatment
• Main procedures in pretreatment are:– Removing variables
• The data set may contain variables that have no relevant information for monitoring
– Removing outliers• Outliers are isolated measurement values that are erroneous
and will cause biased parameter estimates for the method used. Methods for removing outliers include visual inspection and statistical tests.
– Autoscaling• Process data often needs to be scaled to avoid particular
variables dominating the process monitoring method (autoscaling = subtract mean and divide by standard deviation)
Univariate vs. multivariate statistics
• The most simple type of monitoring is based on univariate statistics
• Individual thresholds are determined for each variable (Shewart charts)
upper control limit
lower control limit
target
out-of-control
in-control
Univariate vs. multivariate statistics
• Tight thresholds result in a high false alarm rate but low missed alarm rate
• Limits too spread apart result in low false alarm rates but high missed alarm rates
• A trade off between false alarms and missed alarms
-5 0 5 100
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
threshold
normal faulty
false alarmsmissed alarms
Univariate vs. multivariate statistics
• Univariate methods determine threshold for each variable individually without considering other variables
• The fact that there are correlations between the variables is ignored
• The multivariate T2 statistic takes into account these correlations
• The T2 statistic is based on an eigenvector decompostion of the covariance matrix
Univariate vs. multivariate statistics
• Comparison of univariate statistics and T2
statistic
x1
x2
univariate statistical confidence region
T2 statistical confidenceregion
More on T2 later in the PCA part
Abnormal datacan be classifiedas ok!
Principal Component Analysis
Process monitoring
Qualitative Quantitative
Data-BasedModel-Based
Neural Networks
Statistical
PCA
Principal component analysis
• Linear method
• Greatly reduces the number of variables to be monitored – data compression
• Based on eigenvalue and eigenvector decomposition of the covariance matrix
• Simple indexes for monitoring
Principal component analysis• The idea of PCA is to form a minimum number of
new variables to describe the variation of the data by using linear combinations of the original variables
x2
PCA model
PC1: Direction of largest variationPC2: Direction of 2. largest variation
x1
PC1
PC2
PCA- principal components
• The new axes = principal components are selected according to the direction of highest variation in the original data set
• The new axes are orthogonal
x2
x1
PC2
PC1
PCA
PCA- principal components
x2
x1
pc2
pc1
• Principal components will rotate the data set so that the different groups might be separable
No separation offaulty data (red)with original axis
Separation of faulty dataon pc2-axis
PCA- scores
• The PCA scores are the values of the original data points on the principal component axes
PC1
PC2
Sample 1
PCA model calculation
• The direction of the principal components = eigenvectors
• The variation of the data along a eigenvector is given by the corresponding eigenvalue
x2
PCA model
x1
PC1=e1=w11x1+w12x
2
Principal component analysis
• Principal Component Analysis is based on a eigenvalue/eigenvector decomposition of the covariance matrix of the data set (X) with n observations (rows) and m variables (columns).
XX1-N
1C T
Covariancematrix C
The covariance matrix can be decomposed:
X is centered to have zero mean and scaled so that each variable has the same variance (especially if the variables have different units
M
1
M1
λ00
00
00λ
Λ
eeV
ΛC
TVV
Principal component analysis
• The decomposition can be done by solving
• In Matlab the eigenvalues and eigenvectors are obtained by:[eig_vec,eig_val] = eig(C)
• Each eigenvector is a column in eig_vec and the eigenvalues are on the diagonal of eig_val
• A score-matrix T can be calculated: T = XVk
• Vk is a transformation matrix containing the eigenvectors corresponding to the k largest eigenvalues. The user chooses k (more later)
• T is a ”compressed” version of X
0λICdet iii eλCe and
Principal component analysis
• The compressed data can be decompressed:
• There is a residual matrix E between the original data and the decompressed data
• Combining the equations and rewriting the TVkT term:
• pi are the principal components
• The scores ti are the distances along the principal component pi
XXE ˆ
TkTVX ˆ
EptETVXk
1i
Tii
Tk
Principal component analysis
• Matlab demo: Compressing and decompressing data via eigenvalue decomposition
Principal Component Analysis
• Example: Calculate (by hand) the eigenvalues and eigenvectors for:
Specify that
also verify
42
27C
TVVΛC
M
1
M1
λ00
00
00λ
Λ
eeV
Recall that 0λICdet
iii eλCe
1ei
Monitoring indexes
• SPE – variation outside the model, distance off the model plane
• Hotelling T2 – variation inside the model, distance from the origin along the model plane
PCA modelplane
Points with SPE violation
PC2
PC1
SPE
Points withT2 violation
Monitoring indexes
• T2: Measures systematic variations of the process. For individual observation:
• SPE: Measures the random variations of the process
tΛtxVΛVx)(xT -1K
Tnew
Tk
-1Kk
Tnewnew
2 ~~
newTkk
T x)VV(Ir whererrSPE ~
SPE
Different view
Monitoring indexes
• The process can be also monitored by tracking the score values for each principal component
PCA - calculate the model
1. Zero-mean the original data set.2. Compute the covariance matrix C3. Compute the eigenvalues and eigenvectors. Modify the
matrices so, that the eigenvalues are in decreasing order (remember to do the same operations to the eigenvectors)
4. Choose how many principal components to use. (Plot eigenvalues, captured variance)
5. Form the transformation matrix Vk (principal components) and eigenvalue matrix Λk.
6. Compute confidence limits for the scores of every PC7. Compute the Hotelling T2 & SPE limits
PCA - calculate the model1. Scale the original data set X (zero-mean, unit variance if necessary).
2. Calculate the covariance matrix
XX1-N
1C T
PCA - calculate the model
3. Compute the eigenvalues and eigenvectors. The eigenvalues are computed according to
The eigenvectors can be solved from the equation:
Remember to keep the eigenvectors in the same order as the eigenvalues
iii eλCe
0IλCdet i
m21 eeeV
PCA - calculate the model
4. Choose how many principal components to use. (Plot eigenvalues, captured variance)
100%λ
λ)Cvariance(P Captured m
1jk
ii
λ
var capt. %
tot. var capt. %
1 3.18 28.91 28.91
2 2.34 21.31 50.22
3 1.74 15.81 66.03
4 1.08 9.78 75.80
5 0.91 8.24 84.04
6 0.79 7.15 91.20
7 0.49 4.48 95.67
8 0.22 2.04 97.71
9 0.14 1.30 99.01
10 0.11 0.97 99.98
11 0.00 0.02 100.00
PCA - calculate the model
With 7 PCs 96% of the variance is captured
0 2 4 6 8 10 120
0.5
1
1.5
2
2.5
3
number of PC:s
96% of variance explained
PCA - calculate the model
5. Form the transformation matrix Vk (eigenvectors) and eigenvalue matrix ΛK .
K
2
1
K
λ00
0λ0
00λ
Λ
k21k eeeV
PCA - calculate the model
6. Compute confidence limits for the scores of every PC
α/2)1,t(Nλ)Conf(PC ii
sqrt(lamda)*tinv(alfa+(1-alfa)/2,N-1)In Matlab:
α
α = confidence level
PCA - calculate the model
7. Compute Hotelling T2 limit & SPE limit
αK,NK,FKN
1)K(NT2
lim
where the F(K,N-K,) corresponds to the probability point on the F-distribution with (K,N-K)
degrees of freedom and confidence level . N = # of data samples, K = # of PCs
0h1
21
002
1
20α1α Θ
1hhΘ1
Θ
2ΘhcΘSPE
22
310
m
1kj
iji 3Θ
Θ2Θ1hλΘ
where m= number of original variables, k=number of principal components in the model, cα= upper limit from normal distribution with conf. level α
“Unused” eigenvalues
PCA – the new data
1. Scale the new data set with training data scaling values .
2. Compute PCA transformation (i.e. scores for all the chosen principal components) using Vk.
3. Compare the scores to confidence limits. If inside the limits, then OK.
4. Compute the Hotelling T2 & SPE values for the new data set
5. Compare the Hotelling T2 & SPE values to the limits. If under the limit, then OK
PCA – the new data1. Scale the new data set with training data
scaling values
2. Compute PCA transformation (i.e. scores for all the chosen principal components) using Vk.
newTk xVt ~
train
trainnewnew σ
xxx
~
PCA – the new data
3. Compare the scores to confidence limits. If inside the limits, then OK4. Compute the Hotelling T2 & SPE values for the new data set
5. Compare the Hotelling T2 & SPE values to the limits. If under the limit, then OK
tΛtxVΛVx)(xT -1K
Tnew
Tk
-1Kk
Tnewnew
2 ~~
newTkk
T x)VV(Ir whererrSPE ~
Fault identification
• Once a fault has been detected, the next step is determine the cause of the out-of-control status
• One way to handle this is by using so called contribution plots
Contribution plots
• In response to T2 violations we can obtain contribution plots according to:
1. For observation xi, find the r cases when the normalized scores ti2/λi >
T2/a
2. Calculate the contribution of each variable xj to the out-of-control scores ti
3. When conti,j is negative set it equal to zero
4. Calculate the total contribution of the jth process variable
5. Plot CONTj for all process variables in a single plot
V matrix the of element j)(i, the is v xvλ
tcont th
ji,jji,i
iji,
~
r
1iji,j contCONT
Contribution plots
1 2 3 4 50
1
2
3
4
5
6
7
8Contribution plot
process variable
Contribution plots
• Can also be made from the SPE index
1 2 3 4 5 6 7 8 9 10 110
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
2iriSP
PCA derivation
• We want a linear combination of the elements of x that has maximal variance
• Next we look for a linear function eT2x that
is uncorrelated with eT1x but has maximum
variance and so on• We can write• We want to have unit length for the scaling
vector e (constraint)
nn1,11,1T xexexe1
1TT Ceexevar11
PCA derivation
• The maximization can be done using Lagrange multipliers
• We have
• Differentiation gives
• This is an eigenvalue problem. It is the largest eigenvalue that corresponds to the solution since
1eeλCeemax 1T11
T
1
Unit lenght
0λeCe 11
λeλeλeeCee 1T
1T
1T
111
Maximizing the variance = maximizing λ
PCA derivation
• For the second component we can write
• Differentiation gives
• Multiplication with eT1 gives
1T22
T22
T2 eφe1eeλCeemax
0φeλeCe 122
0φ
0eφeeλeCee 1T12
T12
T1
0λeCe 22
(A)
introduce in (A)
Again an eigenvalue problem
uncorrelated
PCA derivation
• Since e2 must be different from e1 and hence λ ≠ λ1
• We still want to maximize variation λ is the second largest eigenvalue
• A similar analysis can be done for the third, fourth etc. PCs.
0ee 1T2