Computers and Chemical Engineering 64 (2014) 13–23
Contents lists available at ScienceDirect
Computers and Chemical Engineering
j ourna l ho me pa g e: www.elsev ier .com/ locate /compchemeng
ultivariate video analysis and Gaussian process regression modelased soft sensor for online estimation and prediction of nickel pelletize distributions
ingyan Chena, Jie Yua,∗, Yale Zhangb
Department of Chemical Engineering, McMaster University, Hamilton, Ontario, Canada L8S 4L7Vale Base Metals Technology Development, Mississauga, Ontario, Canada L5K 1Z9
r t i c l e i n f o
rticle history:eceived 5 August 2013eceived in revised form 6 January 2014ccepted 14 January 2014vailable online 28 January 2014
ideo analysisdge detectionoft sensoraussian process regression
a b s t r a c t
Accurate measurement and prediction of pellet size distributions are critically important for materialprocessing because they are essential for model predictive control, real-time optimization, planning,scheduling and decision support of material production. Mechanical sieving is one of the traditionalmethods for pellet size measurement in industrial practice but cannot be applied in real-time fashion.Alternately, multivariate image analysis based pellet sizing methods may acquire the size informationnon-intrusively and thus can be implemented for on-line measurement in industrial applications. Nev-ertheless, the conventional multivariate image analysis based pellet sizing methods cannot effectivelydeal with the pellet overlapping effects in the still images, which may lead to inaccurate and unreliablemeasurements of size distributions. In our study, two novel video analysis based pellet sizing methods areproposed for measuring the pellet size distributions without any off-line and intrusive tests. The videos offree-falling pellets are first taken and then the free-falling tracks of pellets in video frames are analyzedthrough the two video analysis based pellet sizing approaches. In the first video analysis method, theSobel edge detection strategy is adopted to identify and isolate the free-falling tracks in order to estimate
the diameters of the corresponding pellets. For the second video analysis approach, the filtered gray-scale video frames are scanned row by row and then the particle diameters are estimated and predictedthrough the built Gaussian process regression (GPR) models and a fine designed counting rule so as toeliminate the overlapping effects of nickel pellets along the horizontal and vertical directions. The utilityof these two video analysis based pellet sizing methods is demonstrated through the measurement andestimation of free-falling nickel pellets in two test videos.
Particle size distribution is a crucially important quality variablen different industrial operations including mining, materials andharmaceutical processes. Specifically in mining industry, accurateellet size measurement and prediction can substantially improveroduct quality, production yield and energy efficiency. Mechani-al sieving serves as one of the traditional pellet sizing methods, inhich pellets are passed through the grids of mesh in order to deter-ine the corresponding size distributions (Koh, Miles, Morgan, &
ayes-Gill, 2009). However, the intrusive test requires representa-
ive samples manually taken from the pellet decomposers and such
off-line analysis is not suitable for automatic control and real-timeoptimization of pellet production processes.
Predictive model based soft sensors have attracted increasingattention from academia and industry in the past decades (Kadlec,Grbic, & Gabrys, 2011; Kano & Nakagawa, 2008; Yu, 2012a; Yu& Qin, 2008, 2009). Soft sensors usually make use of availableprocess measurement data or prior knowledge on process mecha-nism to build predictive models for estimating key product qualityvariables that cannot be easily measured by physical hardware ina real-time fashion (Kadlec, Gabrys, & Strandt, 2009; Lin, Recke,Knudsen, & Jørgensen, 2007; Yu, 2012b). There are two types of softsensors, namely mechanistic model based and process data drivensoft sensors. Traditional model-based soft sensors are mainly based
on first-principle process models along with extended Kalman fil-ter or adaptive observer techniques (Assis & Filho, 2000; Doyle,1998). However, the model development requires in-depth pro-cess knowledge on physical and chemical mechanisms and the
odeling effort can be quite heavy. Alternatively, data-drivenoft sensors rely on process data only and thus can alleviate theechanistic model development effort and knowledge require-ent. Different kinds of data-driven soft sensor methods have been
eveloped, including principal component analysis (PCA), partialeast squares (PLS), artificial neural networks (ANN), support vec-or machine (SVM) and Gaussian process regression (GPR) (Hoskins
Himmelblau, 1988; Kresta, Marlin, & MacGregor, 1994; Yan, Shao, Wang, 2004; Yu, 2012a, 2013; Yu, Chen, & Rashid, 2013). ThoughCA or PLS based soft sensors can deal with variable collinearity anddentify statistical models within lower-dimensional latent sub-pace, they are essentially linear models and thus may not accountor significant process nonlinearity. Alternatively, ANN, SVM andPR approaches can be adopted to construct data-driven soft sen-ors for nonlinear processes (Napoli & Xibilia, 2011; Qin & McAvoy,992; Rashid & Yu, 2012; Ruiz, Nougués, Calderon, Espu na, &uigjaner, 2000; Yu, 2012c, 2013). Soft sensor concept and meth-ds are definitely attractive for measuring pellet size distributionsn an on-line fashion instead of off-line lab analysis. With reliableoft sensors for online size measurement, model predictive controlnd real-time optimization of particle processes become possible.
soft sensor approach by integrating ANN and PCA is developed toynamically estimate the particle size distributions of grinding cir-uits, where on-line adaption of neural network model is achievedo deal with the time-varying nature of grinding circuits and mean-hile the structure of neural network is simplified through PCA
trategy (Du, del Villar, & Thibault, 1997). More recently, a soft sen-or approach relying on the parameter-constrained identificationlgorithm for on-line particle size estimation in wet grinding cir-uits is developed by taking into account prior process knowledgeSbarbaro, Ascencio, Espinoza, Mujica, & Cortes, 2008). In addition, aeural network based soft sensor is designed to predict the size dis-ributions of disarranged ore particles by utilizing particle imagesnd their uniformity (Ko & Shang, 2011).
Recently, multivariate image analysis techniques have beenidely explored for soft sensor based pellet size measurement
nd show significant advantages over traditional mechanical sie-ing approach that is labor and time intensive. The main purposef image analysis is to extract useful measurement informationrom digitized images by analyzing their pixel arrays. Depending onhe image features acquired, different methods are further devel-ped for analyzing and measuring various types of parametersuch as particle counts, shape characteristics, area fractions, andpatial and size distributions (MacGregor, Yu, Mu noz, & Flores-errillo, 2005; Torabi, Sayad, & Balke, 2005). Multivariate imagenalysis approaches typically involve latent subspace projectionsf image data based on PCA or PLS models in order to extracteometric features from still images and reduce dimensionalityf data matrices, which are different from the filtering and edgeetection steps of regular image processing (Bharati & MacGregor,998; Prats-Montalbán, De, & Ferrer, 2011). Multivariate imagesre then decomposed into orthogonal components through trans-ormation into a number of latent variables that retain most ofmage information (Yu & MacGregor, 2004). For the purpose ofsolating pellets from background images, the multivariate PCA
odel of blank background is built and the pellets are identified byomparing the background model so as to estimate the pellet sizeistributions (Sarkar, Doan, Ying, & Srinivasan, 2009). Moreover,
mage segmentation based on multi-flash imaging is introduced toapture the geometric edges around particles from shadow infor-ation (Koh et al., 2009). For quantitative prediction purpose, PLSodel is also built from histogram features within latent-variable
core plots in order to predict the coating concentration of snackroducts (Yu & MacGregor, 2003). Likewise, a method based onLS model and angle measuring technique is employed to predicthe particle size distributions of natural sands (Dahl & Esbensen,
l Engineering 64 (2014) 13–23
2007). Moreover, the fiber diameter distributions in nano-fibersare predicted by utilizing wavelet transformation and gray-levelco-occurrence matrices (Facco et al., 2010). In addition to activeacademic research, a commercial system termed as WipFrag isdeveloped to estimate the pellet size distributions in an on-linefashion, where size measurements are obtained by using thresh-olding, gradient operators, and morphological technique (Koh et al.,2009; Maerz, 1999). Though image analysis based pellet sizingmethods can acquire size distributions non-intrusively, the highprecision of estimation and prediction may not be guaranteed dueto some limiting factors such as the overlapping effects amongdifferent sizes of pellets and the undetected areas caused by thespecific positions of camera systems.
Image analysis based soft sensor approaches have been inten-sively investigated for particle size distribution measurementwithin emulsion and suspension polymerization systems of indus-trial crystallization processes. For instance, the light scatteringtechnique focuses a laser beam through a probe tip and thencollects the scattered laser light to obtain the crystal size infor-mation (Braatz & Hasebe, 2002; Monnier, Klein, Ratsimba, & Hoff,1996; Tähti, Louhi-Kultanen, & Palosaari, 1999). Nevertheless, thismethod is more appropriate for determining the suspension sizedistributions under low volume fractions. Alternatively, the laserbackscattering approach is explored to characterize the particlesize distributions in suspension polymerization reactors with highparticle densities but a large number of calibration experimentsare required (Togkalidou, Braatz, Johnson, Davidson, & Andrews,2001). Meanwhile, inverse modeling method is integrated withlaser backscattering approach to determine polymeric bead sizedistributions under the assumption that the backscattering lightis perfect at different angles (Fujiwara, Nagy, Chew, & Braatz,2005; Hukkanen & Braatz, 2003). Alternative effort has also beenattempted to estimate particle shape and size distributions bywavelet transform and multi-scale segmentation based image anal-ysis methods (Calderon De Anda, Wang, Lai, & Roberts, 2005;Calderon De Anda, Wang, & Roberts, 2005; Chen & Wang, 2005). Inorder to handle high particle concentrations, illumination throughreflected light is required in the above techniques, which may leadto poorly identified particle boundaries. Furthermore, model-basedobject recognition algorithm is applied to identify crystal objectswith a wide range of sizes and shapes by matching raw image fea-tures with pre-defined models (Larsen, Rawlings, & Ferrier, 2006,2007). However, the overlapping effect in still images given highparticle concentrations still poses a significant challenge towardthe precise estimation of size distributions.
Aimed at eliminating the particle overlapping effect that cannotbe easily handled by conventional image analysis based methods,two video analysis based approaches are proposed in this studyby making use of the videos of free-falling pellets for estimatingtheir size distributions without any intrusive tests. In the first videoanalysis approach, the edges of the free-falling particle tracks indifferent video frames are captured and thus the pellet diame-ters are equivalent to the widths of the corresponding free-fallingtracks. For the second video analysis method, the filtered gray-scalevideo frames are scanned row by row so as to obtain the filteredgray-scale curves. Then Gaussian process regression (GPR) modelsare constructed for decomposing and fitting different sub-curvesin order to estimate and predict the diameters of various pelletsalong the horizontal direction. Furthermore, a counting rule forpellet size distribution is developed to get rid of the overlappingeffect along the vertical direction of free-falling pellets. The perfor-mance of these two video analysis based pellet sizing methods is
demonstrated and compared through the lab-scale video clips offree-falling nickel pellets.
The remainder of this paper is organized as follows. The conven-tional image analysis based pellet sizing method and its challenges
J. Chen et al. / Computers and Chemical Engineering 64 (2014) 13–23 15
F riginap
abtStpS
2
eamppmpeppwfi
ig. 1. Illustrative example of the image analysis based pellet sizing method: (a) oellet size distribution and cumulative distribution.
re briefly described in Section 2. Then the two video analysisased pellet sizing methods along with the corresponding illus-rative examples of two test videos are shown in Section 3. Inection 4, the measurement results of pellet size distributions fromhese two video analysis based methods are presented and com-ared. Finally, the conclusions and future work are discussed inection 5.
. Image analysis based pellet sizing soft sensor method
Image analysis based pellet sizing techniques have been widelyxplored for on-line measurement and basically consist of imagecquisition, image preprocessing, feature extraction and size esti-ation steps. High resolution images are first captured from a
articular location within pellet processes and then digitized intoixel images in order to extract useful geometric features for esti-ating pellet size distributions. The success of image analysis based
ellet sizing method relies on the quality of images as well as theffectiveness of image analysis. Traditional image analysis based
ellet sizing methods require efficient edge detection of differentellets in the preprocessed and filtered images. First the image ofell mixed nickel pellets in a bin is taken. Then different layers oflters are applied to the image so as to extract significant features
l image; (b) pellet edge detection results; (c) pellet identification results; and (d)
and identify pellet edges. After edge detection, pellet diameterscan be identified from the local maximum distances and thus pel-let size distributions may be estimated by incorporating all pelletdiameters in the image.
In order to illustrate the conventional image analysis basedpellet sizing method, the image of nickel pellet samples is ana-lyzed to obtain pellet size distribution. The original image, its pelletedge detection results, and the corresponding pellet size estima-tion results are depicted in Fig. 1(a), (b), and (c), respectively.Moreover, the estimated pellet size distribution and cumulativedistribution compared with the actual ones are shown in Fig. 1(d).The significant inconsistency between the actual and estimatedpellet size distributions implies that the conventional image anal-ysis based pellet sizing method may not be accurate and reliable.The main reason of the poor prediction results lies in the over-lapping effect of different pellets in still images. Basically thosesmall pellets tend to move onto lower layers and thus are hiddenbehind the large pellets in upper layers. Consequently, the over-lapped small pellets can hardly be detected and identified from the
edge detection strategy. It is therefore desirable to develop newapproaches in order to eliminate the overlapping effect of pelletsand obtain size distribution prediction results with higher accu-racy.
16 J. Chen et al. / Computers and Chemical Engineering 64 (2014) 13–23
o ana
3
iatffuybotacpsdlttvfvcti
3
ftsfaetGa
I
wa2
Fig. 2. Illustrative procedure of the first vide
. Video analysis based pellet sizing soft sensor methods
Due to the limitation of overlapping effect in the conventionalmage analysis based pellet sizing method, two kinds of videonalysis based pellet sizing approaches are developed to estimatehe nickel pellet size distributions in our study. The videos of free-alling pellets are taken with proper lighting conditions. Then theree-falling tracks of nickel pellets in different video frames aretilized for measuring the pellet diameters. In the first video anal-sis method, the Sobel edge detection strategy is employed in thelack and white binary video frames in order to capture the featuresn pellet diameters and further estimate the size distributions. Inhe second video analysis approach, the filtered gray-scale framesre scanned row by row so that the diameters of different pelletsan be obtained from the filtered gray-scale curves. Then Gaussianrocess regression models are developed to decompose the gray-cale curves and predict the pellet diameters along the horizontalirection. Further, a counting rule is designed to eliminate the over-
apping effect of pellets along the vertical direction. It is assumedhat there are total P frames in the video and all the pellets takehe time of n video frames to fall from the top to the bottom of theideo region. As long as the pellet diameters in every n consecutiverames are measured, the size distribution of all the pellets in theideo can be obtained. In addition, the position and lighting of videoamera are fixed for all the videos to ensure that the ratio betweenhe number of pixels in the video frames and the number of inchesn geometric size remains the same.
.1. Pre-processing of video frames
In order to extract useful geometric features on pellet sizesrom different video frames, preprocessing procedure is designedo remove the color gradient and background illumination. Con-ider the red-green-blue (RGB) color image I1(i, j) in the pth videorame, where I1(i, j) denotes the RGB value for the ith pixel rownd the jth pixel column. Since the color is not the main feature forstimating pellet sizes, the RGB color image I1(i, j) is converted tohe gray-scale image I2(i, j) by performing a weighted sum of the R,, and B components of the corresponding pixel row and columns follows
here the coefficients represent human perception of red, greennd blue colors and are used in standard color video systems (Cadík,008). In order to remove the non-uniform background from the
lysis based pellet sizing soft sensor method.
gray-scale image I2(i, j), the background illumination I3(i, j) is esti-mated by conducting morphological opening on I2(i, j) as
I3 = I2 ◦ b = (I2 � b) ⊕ b (2)
where b is the disk-shaped structuring element with the corre-sponding size less than that of the smallest free-falling track whilethe morphological opening operation ◦ is equivalent to an erosion� followed by a dilation ⊕. The erosion and dilation operationsare defined as follows (Lillesand, Kiefer, & Chipman, 2008; Zhou,Srinivasan, & Lakshminarayanan, 2009)
f (i, j) = [I2 � b](i, j) = min(s,t)∈b
{I2(i + s, j + t)} (3)
and
[f ⊕ b](i, j) = max(s,t)∈b
{f (i − s, j − t)} (4)
Thus the filtered gray-scale image I4(i, j) with uniform back-ground can be obtained as
I4(i, j) = I2(i, j) − I3(i, j) (5)
The gray-scale value is zero in the background part of I4(i, j) sincethe background illumination is removed.
3.2. The first video analysis based pellet sizing soft senor method
The first video analysis based pellet sizing method is con-ducted by detecting the edges of free-falling tracks and thewidth of each track equals the diameter of the correspondingfree-falling pellet. The illustrative procedures of this method isshown in Fig. 2. The black and white binary image I5(i, j) is firstobtained by thresholding the filtered gray-scale frame I4(i, j) asfollows
I5(i, j) ={
1 (white), if I4(i, j) ≥ L
0 (black), if I4(i, j) < L(6)
where L is the normalized global threshold that can be obtainedfrom Otsu’s method by minimizing the intra-class variance of the
black and white pixels (Otsu, 1975). The Sobel operation for approx-imating the gradient of binary function at each image point is thenapplied to detect the geometric edges of free-falling tracks (Maaß,Rojahn, Hänsch, & Kraume, 2012; Parker, 2010). Assume that Gi(i,
J. Chen et al. / Computers and Chemical Engineering 64 (2014) 13–23 17
Fig. 3. Illustrative example of the first video analysis based pellet sizing soft sensor method: (a) original image; (b) filtered gray-scale image; (c) black and white image; and(
ja
G
G
wat
G
a
�
t
(
wds
3
t
d) edge detection results.
) and Gj(i, j) are the images containing the approximate horizontalnd vertical derivatives at each point
i(i, j) =
⎡⎣ 1 0 −1
2 0 −2
1 0 −1
⎤⎦ ∗ I5(i, j) (7)
j(i, j) =
⎡⎣ 1 2 1
0 0 0
−1 −2 −1
⎤⎦ ∗ I5(i, j) (8)
here the operator * denotes the 2-dimensional convolution oper-tion. Thus the gradient magnitude G(i, j) and direction �(i, j) athe image point (i, j) can be expressed as
(i, j) =√
G2i(i, j) + G2
j(i, j) (9)
nd
(i, j) = arctan
(Gj(i, j)Gi(i, j)
)(10)
Consequently, the edges can be identified at those points wherehe gradient of I5 is maximized
iE, jE) = arg max(i,j)
G(i, j) (11)
here (iE, jE) represents the identified edge point. Once the pelletiameters in n consecutive video frames are obtained, the pelletize distributions can then be estimated.
.2.1. An illustrative exampleThe first video analysis based pellet sizing method is applied to
wo lab-scale test videos of free-falling nickel pellets. The initial
velocity of free-falling motion of pellets is assumed to be zeroand the frame rate of the test videos is 29 frames per second.There are total 850 and 1160 frames in the first and secondtest videos respectively and every 10 consecutive video framesare extracted for analysis. One of the frames in the first testvideo is used as an illustrative example to explain the majorsteps of the first video analysis method, as shown in Fig. 3. TheRGB color image in Fig. 3(a) is first converted to the filteredgray-scale image in Fig. 3(b) by removing the color gradient andbackground illumination. Then the filtered gray-scale image istransformed into the binary black and white image as shown inFig. 3(c) and the Sobel operation is employed to capture the geo-metric edges of free-falling tracks as highlighted by red contoursin Fig. 3(d). Thus the width of each identified free-falling trackequals the diameter of the corresponding nickel pellet and total85 and 116 frames in the first and second test videos are pro-cessed.
The estimated size distributions and cumulative distributionsare compared with the actual ones that are obtained by lab-scalemechanical sieving. The comparison results for the two test videosare shown in Figs. 4 and 5. It is obvious that the first video analy-sis method performs better than the conventional image analysisbased pellet sizing method in terms of size distribution predictionaccuracy. Nevertheless, there are still some challenges in the firstvideo analysis approach that may cause some unreliable estimationresults. Specifically, the small pellets with relatively narrow free-falling tracks that are labeled as “1” in Fig. 6 cannot be preciselyidentified by the Sobel edge detection method. Moreover, largenumber of pellets can still result in overlapping effect along the
horizontal and vertical directions. The overlapped pellets labeledas “2” and “3” in Fig. 6 may be incorrectly captured as a singlepellet and thus can lead to biased estimation of pellet size distribu-tions.
18 J. Chen et al. / Computers and Chemical Engineering 64 (2014) 13–23
0 0.1 0.2 0.3 0.4 0.5 0.60
20
40
60
80
100Pellet Size Distribution
Size (inch)
Dis
trib
utio
n (%
)
0 0.1 0.2 0.3 0.4 0.5 0.60
20
40
60
80
100Pellet Size Cumulative Distribution
Size (inch)
Dis
trib
utio
n (%
)
Actual DistributionPredicted Distribution
Actual DistributionPredicted Distribution
Fa
3m
tvfitIfwIptvto
it
0 0.1 0.2 0.3 0.4 0.5 0.60
20
40
60
80
100Pellet Size Distribution
Size (inch)
Dis
trib
utio
n (%
)0 0.1 0.2 0.3 0.4 0.5 0.6
0
20
40
60
80
100Pellet Size Cumulative Distribution
Size (inch)
Dis
trib
utio
n (%
)
Actual DistributionPredicted Distribution
Actual DistributionPredicted Distribution
Fig. 5. Pellet size distribution and cumulative distribution results of the first videoanalysis based pellet sizing soft sensor method for the second test video.
ig. 4. Pellet size distribution and cumulative distribution results of the first videonalysis based pellet sizing soft sensor method for the first test video.
.3. The second video analysis based pellet sizing soft sensorethod
Instead of detecting the geometric edges of the free-fallingracks in the converted black and white video frames, the secondideo analysis method as illustrated in Fig. 7 is designed to scan theltered gray-scale image I4(i, j) row by row along the vertical direc-ion. Since the background illumination is removed, the values of4(i, j) are zero under uniform background. As such, if there is a free-alling pellet in the filtered gray-scale frame, the gray-scale valuesill be non-zero. For each pixel row in the filtered gray-scale image
4(i, j), a filtered gray-scale curve with a single peak indicates oneellet passing through that row and the width of the curve equalshe diameter of the corresponding pellet. In addition, the gray-scalealue tends to be larger in the central area and gradually decreaseso zero on pellet edges because it is always brighter in the middlef pellets.
The second video analysis based pellet sizing method isllustrated in Fig. 8, where the three dashed lines stand for the illus-rative rows in the filtered gray-scale frame and the corresponding
Fig. 6. Illustrative example of the challenges for the first video analysis based pelletsizing soft sensor method.
Fig. 7. Illustrative procedure of the second video analysis based pellet sizing soft sensor method.
J. Chen et al. / Computers and Chemical Engineering 64 (2014) 13–23 19
F gray-
fiitwfwmdrcwca
fibpgttGcogbFSjtrcTe
D
wf
k
wkito
ig. 8. Illustration of the proposed pixel row based scanning strategy of the filtered
ltered gray-scale curves are shown in each row. Given that theres one pellet with the diameter d1 passing through the first illus-rative row, there is a corresponding filtered gray-scale curve withidth d1 along that pixel row. For the second illustrative row, the
our overlapped pellets in the horizontal direction lead to a curveith four sub-curves and the GPR model can be further built to esti-ate the diameter of each pellet. There is also a small pellet with
iameter d2 passing through the second and the third illustrativeows, which results in the filtered gray-scale curve with the identi-al width d2 in both the second and the third rows. The small pelletsith relatively narrow free-falling tracks may not be accurately
aptured by the first video analysis method but can be detectednd identified in the second video analysis approach.
If the pellets are overlapped along the horizontal direction, theltered gray-scale curve includes multiple sub-curves and the num-er of the sub-curves is equivalent to the number of the overlappedellets. Because of the overlapping effects of pellets, the filteredray-scale sub-curves need to be decomposed and then the diame-ers of the overlapped pellets can be estimated from the widths ofhe decomposed sub-curves. In the second video analysis method,PR model is constructed to decompose the filtered gray-scaleurve into sub-curves and further estimate the diameters of theverlapped pellets. Suppose that there are S peaks in the filteredray-scale curve and thus S sub-curves need to be decomposedy GPR models to estimate the diameters of S overlapped pellets.irstly, the filtered gray-scale curve is split into S sub-curves by the
− 1 local minimum points. For the sth sub-curve, the samples in-axis and the corresponding filtered gray-scale values are assumedo be Js = {j1, j2, · · ·, jns } and Fs = {fs(j1), fs(j2), · · ·, fs(jns )}, where ns
epresents the number of gray-scale samples within the sth sub-urve while fs denotes the corresponding filtered gray-scale value.hus the training data for the GPR model of the sth sub-curve isxpressed as
s = {ji, fs(ji) | i = 1, . . ., ns} = {Js, Fs} (12)
The mean function of the Gaussian process is assumed to be zerohile the squared exponential function is chosen as the covariance
unction
(jm, jn) = �2f exp
[−(jm − jn)2
2l2
](13)
here �f and l are the maximum allowable covariance and Gaussian
ernel width (Rasmussen & Williams, 2006). Then the GPR models used to predict and extend the sth filtered gray-scale sub-curveo the j-axis in order to further obtain the diameter of the sthverlapped pellet. Given the predicted inputs Js∗ = {j1∗, j2∗, · · ·, jns∗}
scale frame in the second video analysis based pellet sizing soft sensor method.
with ns* representing the width of the sth sub-curve extended tothe j-axis, the joint distribution of the training samples Fs and thepredicted outputs Fs* for the sth sub-curve is given by[
Fs
Fs∗
]∼N(
0,
[Ks KT
s∗
Ks∗ Ks∗∗
])(14)
where Ks* denotes the covariance matrix evaluated between allpairs of training and predicted samples given by
Ks∗ =
⎡⎢⎢⎢⎢⎣
k(j1∗, j1) k(j1∗, j2) · · · k(j1∗, jns )
k(j2∗, j1) k(j2∗, j2) · · · k(j2∗, jns )
......
. . ....
k(jns∗, j1) k(jns∗, j2) · · · k(jns∗, jns )
⎤⎥⎥⎥⎥⎦ (15)
Meanwhile, Ks and Ks** are the covariance matrices withinthe training and predicted samples respectively and they can bedefined in the same way as the above covariance matrix Ks* (Stein,1999). In addition, the conditional probability density functionp(Fs*|Fs) follows a Gaussian distribution as
p(Fs∗|Fs)∼N(Ks∗K−1s Fs, Ks∗∗ − Ks∗K−1
s KTs∗) (16)
Thus the best prediction of Fs* is the following mean estimation
Fs∗ = Ks∗K−1s Fs (17)
and the uncertainty of the prediction can be quantified by its covari-ance as follows
cov(Fs∗) = Ks∗∗ − Ks∗K−1s KT
s∗ (18)
Moreover, in order to estimate the model parameters �s = {�f, l},the optimization problem can be formulated as
�∗s = arg max
�s
log p(Fs|Js, �s) (19)
where
log p(Fs|Js, �s) = −12
FTs K−1
s Fs − 12
log |Ks| − ns
2log(2�) (20)
Thus the diameter of the sth overlapped pellet equals the dis-tance between the two j-axis intersections of the predicted outputsFs∗ in the sth GPR model. Consequently, all the diameters of over-lapped pellets can be obtained in this way and the overlapping
effect along the horizontal direction is thus addressed.
The filtered gray-scale images of every n consecutive frames arescanned row by row and all the widths of the filtered gray-scalecurves and sub-curves are estimated and counted. Since the video
20 J. Chen et al. / Computers and Chemical Engineering 64 (2014) 13–23
ideo a
ofivtiloon
mdbb
M
Fr
Fig. 9. Schematic diagram of the two proposed v
nly records a short interval of free-falling motion, the lengths ofree-falling tracks are assumed to be identical for all the pelletsn the video and expressed as w pixels. If there is no overlap in theertical direction, one pellet should be scanned for w times becausehe filtered gray-scale frame is scanned row by row. Nevertheless,f there is any overlap in the vertical direction, the overlapped pel-ets would be scanned less than w times and the diameters of theverlapped pellets are counted less than w times accordingly. Inrder to obtain the precise prediction of pellet size distributions, theumber of pellets in the video frames needs to be counted reliably.
A counting rule is designed in this work to get accurate esti-ation of pellet size distributions. Assume that any particular
iameter d is counted for N(d) times. Then the total num-er of the pellets with the diameter d is expressed as M(d)
elow
(d) = ceil(
N(d)w
)(21)
ig. 10. Illustrative example of the second video analysis based pellet sizing soft sensor meesult of the 41st row.
nalysis based pellet sizing soft sensor methods.
where the function ceil(·) is defined to round up a numerical valueto the nearest integer. If there is no overlap in the vertical directionfor the pellets with the diameter d, Eq. (21) can be simplified as
M(d) = N(d)w
(22)
Thus the overlapping effect along the vertical direction can beavoided effectively by using Eq. (21). With the above countingrule, the predicted pellet size distributions in the second videoanalysis method can be closer to the actual ones as opposedto the first video analysis method. The schematic diagrams ofthe two video analysis based pellet sizing methods are shownin Fig. 9.
3.3.1. An illustrative exampleThe two lab-scale test videos used in the first video analysis
method are also used to examine the performance of the secondvideo analysis method and one frame in the first test video is
thod: (a) filtered gray-scale frame with the 41st pixel row marked; (b) the scanning
J. Chen et al. / Computers and Chemical Engineering 64 (2014) 13–23 21
F e Gaus
cisatTtaadotpic
Fv
ig. 11. The estimated sub-curves and the corresponding confidence intervals of thub-curves; (b) The second curve with three sub-curves.
hosen as an illustrative example. The filtered gray-scale frames shown in Fig. 10(a) with the dashed line indicating the 41-t pixel row and the filtered gray-scale curves for the 41-st rowre depicted in Fig. 10(b). It can be seen from Fig. 10(b) thathere are two filtered gray-scale curves with multiple sub-curves.he two sub-curves within the first gray-scale curve indicate thathere are two overlapped pellets along the horizontal directionnd the decomposed and estimated sub-curves from GPR modelsre shown in Fig. 11(a) along with the corresponding 95% confi-ence intervals. The diameters of the two overlapped pellets can bebtained from the geometric distance between the two intersec-
ions in the j-axis of each estimated sub-curve. In addition, the threeredicted sub-curves in the second gray-scale curve are shown
n Fig. 11(b) and the diameters of these three overlapped pelletsan be further estimated from the corresponding intersections. The
0 0.1 0.2 0.3 0.4 0.5 0.60
20
40
60
80
100Pellet Size Distribution
Size (inch)
Dis
trib
utio
n (%
)
0 0.1 0.2 0.3 0.4 0.5 0.60
20
40
60
80
100Pellet Size Cumulative Distribution
Size (inch)
Dis
trib
utio
n (%
)
Actual DistributionPredicted Distribution
Actual DistributionPredicted Distribution
ig. 12. Pellet size distribution and cumulative distribution results of the secondideo analysis based pellet sizing soft sensor method for the first test video.
ssian process regression models for the 41st pixel row: (a) The first curve with two
average length of the free-falling tracks used in the counting ruleis w = 30 pixels in both test videos. After total 85 and 116 framesin the first and the second test videos are scanned row by row, thepellet size distributions can be estimated and the prediction resultsare shown in Figs. 12 and 13 for the first and the second test videos,respectively.
4. Comparison of pellet size distribution prediction results
In order to compare the estimation accuracy and performanceof the two video analysis based pellet sizing soft sensor methods,the following mean absolute percentage error(MAPE) index is used
MAPE = 100%N
N∑i=1
|Yi − Yi
Yi| (23)
0 0.1 0.2 0.3 0.4 0.5 0.60
20
40
60
80
100Pellet Size Distribution
Size (inch)
Dis
trib
utio
n (%
)
0 0.1 0.2 0.3 0.4 0.5 0.60
20
40
60
80
100Pellet Size Cumulative Distribution
Size (inch)
Dis
trib
utio
n (%
)
Actual DistributionPredicted Distribution
Actual DistributionPredicted Distribution
Fig. 13. Pellet size distribution and cumulative distribution results of the secondvideo analysis based pellet sizing soft sensor method for the second test video.
22 J. Chen et al. / Computers and Chemica
Table 1Comparison of the MAPE values of predicted pellet size distributions between thetwo video analysis based pellet sizing soft sensor methods for two different testvideos.
Video analysis method 1 Video analysis method 2
wbcptteivev
5
mtdoiybvgptie
titvooaplbfps
R
A
B
B
C
C
C
Test video no. 1 2 1 2MAPE 33.46% 28.25% 9.98% 17.22%
here Yi and Yi are the actual and predicted percentages of the ithin of pellet size measurements and N is the total number of binsorresponding to different size intervals. The prediction results ofellet size distributions using the two video analysis methods forhe two different test videos are compared in Table 1. It can be seenhat the second video analysis method shows improved accuracy ofstimating and predicting pellet size distributions in terms of MAPEndex for both test videos. The smaller MAPE values of the secondideo analysis method are mainly due to its enhanced ability ofliminating the pellet overlapping effects along the horizontal andertical directions.
. Conclusions
In this paper, two video analysis based pellet sizing soft sensorethods are proposed to estimate and predict the size distribu-
ions of nickel pellets. These two approaches make use of theynamic video frames to predict the pellet size distributions with-ut any intrusive tests and show superiorities over the conventionalmage analysis based pellet sizing method. In the first video anal-sis approach, the diameters of free-falling pellets are identifiedy detecting the geometric edges of free-falling tracks in differentideo frames. For the second video analysis method, the filteredray-scale frames are scanned row by row to extract the features forellet diameters. In order to remove the overlapping effects alonghe horizontal and vertical directions, GPR models and a count-ng rule are developed for decomposing gray-scale sub-curves andstimating the pellet diameters with high accuracy.
These two video analysis based pellet sizing methods are appliedo two test videos for measuring nickel pellet size distributions. Its shown that the second video analysis approach performs bet-er than the first video analysis method in terms of smaller MAPEalues by avoiding the overlapping effects. It should be pointedut that the developed video analysis based pellet sizing meth-ds can be extended to other application as well. The only requiredssumption of the proposed methods is that the pellets or any otherarticles need to be free-falling with the identical initial speed. As
ong as the clear free-falling tracks are captured in the video frames,oth proposed methods should be applicable. Future research mayocus on further improving the soft sensor prediction accuracy ofellet size distributions and extending the video analysis based softensors toward different industrial applications.
eferences
ssis, A. J., & Filho, R. M. (2000). Soft sensors development for on-line bioreactorstate estimation. Computers and Chemical Engineering, 24(2), 1099–1103.
harati, M. H., & MacGregor, J. F. (1998). Multivariate image analysis for real-timeprocess monitoring and control. Industrial and Engineering Chemistry Research,37(12), 4715–4724.
raatz, R. D., & Hasebe, S. (2002). Particle size and shape control in crystallizationprocesses. In AIChE Symposium Series New York, NY, USA, (pp. 307–327).
ˇadík, M. (2008). Perceptual evaluation of color-to-grayscale image conversions.Computer Graphics Forum, 27(7), 1745–1754.
alderon De Anda, J., Wang, X. Z., & Roberts, K. J. (2005). Multi-scale segmenta-tion image analysis for the in-process monitoring of particle shape with batch
crystallisers. Chemical Engineering Science, 60(4), 1053–1065.
alderon De Anda, J., Wang, X. Z., Lai, X., & Roberts, K. J. (2005). Classifying organiccrystals via in-process image analysis and the use of monitoring charts tofollow polymorphic and morphological changes. Journal of Process Control, 15(7),785–797.
l Engineering 64 (2014) 13–23
Chen, J., & Wang, X. Z. (2005). A wavelet method for analysis of droplet and particleimages for monitoring heterogeneous processes. Chemical Engineering Commu-nications, 192(4), 499–515.
Dahl, C. K., & Esbensen, K. H. (2007). Image analytical determination of particle sizedistribution characteristics of natural and industrial bulk aggregates. Chemo-metrics and Intelligent Laboratory Systems, 89(1), 9–25.
Doyle, F. J., III. (1998). Nonlinear inferential control for process applications. Journalof Process Control, 8(5), 339–353.
Du, Y.-G., del Villar, R., & Thibault, J. (1997). Neural net-based softsensor for dynamicparticle size estimation in grinding circuits. International Journal of MineralProcessing, 52(2), 121–135.
Facco, P., Tomba, E., Roso, M., Modesti, M., Bezzo, F., & Barolo, M. (2010). Automaticcharacterization of nanofiber assemblies by image texture analysis. Chemomet-rics and Intelligent Laboratory Systems, 103(1), 66–75.
Fujiwara, M., Nagy, Z. K., Chew, J. W., & Braatz, R. D. (2005). First-principles and directdesign approaches for the control of pharmaceutical crystallization. Journal ofProcess Control, 15(5), 493–504.
Hoskins, J. C., & Himmelblau, D. M. (1988). Artificial neural network models ofknowledge representation in chemical engineering. Computers and ChemicalEngineering, 12(9), 881–890.
Hukkanen, E. J., & Braatz, R. D. (2003). Measurement of particle size distributionin suspension polymerization using in situ laser backscattering. Sensors andActuators B, 96(1), 451–459.
Kadlec, P., Gabrys, B., & Strandt, S. (2009). Data-driven soft sensors in the processindustry. Computers and Chemical Engineering, 33(4), 795–814.
Kadlec, P., Grbic, R., & Gabrys, B. (2011). Review of adaptation mechanisms for data-driven soft sensors. Computers and Chemical Engineering, 35(1), 1–24.
Kano, M., & Nakagawa, Y. (2008). Data-based process monitoring, process control,and quality improvement: Recent developments and applications in steel indus-try. Computers and Chemical Engineering, 32(1), 12–24.
Ko, Y.-D., & Shang, H. (2011). A neural network-based soft sensor for particle sizedistribution using image analysis. Powder Technology, 212(2), 359–366.
Koh, T. K., Miles, N. J., Morgan, S. P., & Hayes-Gill, B. R. (2009). Improving particle sizemeasurement using multi-flash imaging. Minerals Engineering, 22(6), 537–543.
Kresta, J. V., Marlin, T. E., & MacGregor, J. F. (1994). Development of inferentialprocess models using PLS. Computers and Chemical Engineering, 18(7), 597–611.
Larsen, P. A., Rawlings, J. B., & Ferrier, N. J. (2006). An algorithm for analyzing noisy,in situ images of high-aspect-ratio crystals to monitor particle size distribution.Chemical Engineering Science, 61(56), 5236–5248.
Larsen, P. A., Rawlings, J. B., & Ferrier, N. J. (2007). Model-based object recognition tomeasure crystal size and shape distributions from in situ video images. ChemicalEngineering Science, 62(5), 1430–1441.
Lillesand, T. M., Kiefer, R. W., & Chipman, J. W. (2008). Remote sensing and imageinterpretation (6th ed.). New York, NY, USA: John Wiley & Sons.
Lin, B., Recke, B., Knudsen, J. K. H., & Jørgensen, S. B. (2007). A systematic approachfor soft sensor development. Computers and Chemical Engineering, 31(5),419–425.
Maaß, S., Rojahn, J., Hänsch, R., & Kraume, M. (2012). Automated drop detectionusing image analysis for online particle size monitoring in multiphase systems.Computers and Chemical Engineering, 45, 27–37.
MacGregor, J. F., Yu, H., Mu noz, S. G., & Flores-Cerrillo, J. (2005). Data-based latentvariable methods for process analysis, monitoring and control. Computers andChemical Engineering, 29(6), 1217–1223.
Maerz, N. H. (1999). Online fragmentation analysis: Achievements in the miningindustry. In Proceedings of the 7th annual international center for aggregatesresearch (ICAR) symposium Austin, TX, USA.
Monnier, O., Klein, J.-P., Ratsimba, B., & Hoff, C. (1996). Particle size determina-tion by laser reflection: Methodology and problems. Particle & Particle SystemsCharacterization, 13(1), 10–17.
Napoli, G., & Xibilia, M. G. (2011). Soft sensor design for a Topping process in thecase of small datasets. Computers and Chemical Engineering, 35(11), 2447–2456.
Otsu, N. (1975). A threshold selection method from gray-level histograms. Automat-ica, 11(285-296), 23–27.
Parker, J. R. (2010). Algorithms for image processing and computer vision (2nd ed.).Indianapolis, IN, USA: Wiley Publishing, Inc.
Prats-Montalbán, J. M., De, J. A., & Ferrer, A. (2011). Multivariate image analysis:A review with applications. Chemometrics and Intelligent Laboratory Systems,107(1), 1–23.
Qin, S. J., & McAvoy, T. J. (1992). Nonlinear PLS modeling using neural networks.Computers and Chemical Engineering, 16(4), 379–391.
Rashid, M. M., & Yu, J. (2012a]). Hidden Markov model based adaptive independentcomponent analysis approach for complex chemical process monitoring andfault detection. Industrial and Engineering Chemistry Research, 51, 5506–5514.
Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning.Cambridge, MA, USA: MIT Press.
Ruiz, D., Nougués, J. M., Calderon, Z., Espu na, A., & Puigjaner, L. (2000). Neural net-work based framework for fault diagnosis in batch chemical plants. Computersand Chemical Engineering, 24(2), 777–784.
Sarkar, D., Doan, X.-T., Ying, Z., & Srinivasan, R. (2009). In situ particle size esti-mation for crystallization processes by multivariate image analysis. ChemicalEngineering Science, 64(1), 9–19.
Sbarbaro, D., Ascencio, P., Espinoza, P., Mujica, F., & Cortes, G. (2008). Adaptivesoft-sensors for on-line particle size estimation in wet grinding circuits. ControlEngineering Practice, 16(2), 171–178.
Stein, M. L. (1999). Interpolation of spatial data: Some theory for kriging. New York,NY, USA: Springer Verlag.
ähti, T., Louhi-Kultanen, M., & Palosaari, S. (1999). On-line measurement of crystalsize distribution during batch crystallization. In Proceedings of the 14th interna-tional symposium industrial crystallization Cambridge, UK.
ogkalidou, T., Braatz, R. D., Johnson, B. K., Davidson, O., & Andrews, A. (2001). Experi-mental design and inferential modeling in pharmaceutical crystallization. AIChEJournal, 47(1), 160–168.
orabi, K., Sayad, S., & Balke, S. T. (2005). On-line adaptive Bayesian classificationfor in-line particle image monitoring in polymer film manufacturing. Computersand Chemical Engineering, 30(1), 18–27.
an, W., Shao, H., & Wang, X. (2004). Soft sensing modeling based on support vectormachine and Bayesian model selection. Computers and Chemical Engineering,28(8), 1489–1498.
u, H., & MacGregor, J. F. (2003). Multivariate image analysis and regression forprediction of coating content and distribution in the production of snack foods.Chemometrics and Intelligent Laboratory Systems, 67(2), 125–144.
u, H., & MacGregor, J. F. (2004). Monitoring flames in an industrial boiler usingmultivariate image analysis. AIChE Journal, 50(7), 1474–1483.
u, J. (2012a]). A Bayesian inference based two-stage support vector regressionframework for soft sensor development in batch bioprocesses. Computers andChemical Engineering, 41, 134–144.
u, J. (2012b]). Multiway Gaussian mixture model based adaptive kernel partialleast squares regression method for soft sensor estimation and reliable quality
l Engineering 64 (2014) 13–23 23
prediction of nonlinear multiphase batch processes. Industrial and EngineeringChemistry Research, 51(40), 13227–13237.
Yu, J. (2012c]). A nonlinear kernel Gaussian mixture model based inferential moni-toring approach for fault detection and diagnosis of chemical processes. ChemicalEngineering Science, 68, 506–519.
Yu, J. (2013). Online quality prediction of nonlinear and non-Gaussian chemicalprocesses with shifting dynamics using finite mixture model based Gaussianprocess regression approach. Chemical Engineering Science, 82, 22–30.
Yu, J., & Qin, S. J. (2008). Multimode process monitoring with Bayesianinference-based finite Gaussian mixture models. AIChE Journal, 54,1811–1829.
Yu, J., & Qin, S. J. (2009). Multiway Gaussian mixture model based multiphasebatch process monitoring. Industrial and Engineering Chemistry Research, 48,8585–8594.
Yu, J., Chen, K., & Rashid, M. M. (2013). A Bayesian model averagingbased multi-kernel Gaussian process regression framework for nonlin-ear state estimation and quality prediction of multiphase batch processes
with transient dynamics and uncertainty. Chemical Engineering Science, 93,96–109.
Zhou, Y., Srinivasan, R., & Lakshminarayanan, S. (2009). Critical evaluation of imageprocessing approaches for real-time crystal size measurements. Computers andChemical Engineering, 33(5), 1022–1035.
![[7] Gaussian Elimination - Coding The ?· Gaussian Elimination [7] Gaussian Elimination. Starting to…](https://static.documents.pub/doc/80x56/5ba1840309d3f2bb6a8c8421/7-gaussian-elimination-coding-the-gaussian-elimination-7-gaussian-elimination.jpg)
