IEEE SENSORS JOURNAL, VOL. 5, NO. 6, DECEMBER 2005 1433
Fast and Robust Gas Identification System Usingan Integrated Gas Sensor Technology
and Gaussian Mixture ModelsSofiane Brahim-Belhouari, Amine Bermak, Senior Member, IEEE, Minghua Shi, and
Philip C. H. Chan, Senior Member, IEEE
Abstract—Among the most serious limitations facing the successof future consumer gas identification systems are the drift problemand the real-time detection due to the slow response of most oftoday’s gas sensors. This paper shows that the combination of anintegrated sensor array and a Gaussian mixture model permitssuccess in gas identification problems. An integrated sensor arrayhas been designed with the aim of combustion gases identification.Our identification system is able to quickly recognize gases withmore than 96% accuracy. Robust detection is introduced through adrift counteraction approach based on extending the training dataset using a simulated drift.
Index Terms—Drift counteraction, fast recognition, gas sensors,Gaussian mixture model (GMM).
I. INTRODUCTION
GAS identification on a real-time basis is very critical fora very wide range of applications in the civil and mili-
tary environments. The past decade has seen a significant in-crease in the application of multisensor arrays to gas classifica-tion and quantification. Most of this work has been focused onsystems using microelectronic gas sensors featuring small sizeand low-cost fabrication, making them attractive for consumerapplications. A number of interesting applications have alsoemerged in the last decade, whether related to hazard detection,poisonous and dangerous gases or to quality and environmentalapplications such as air quality control. Among various types ofmicroelectronic gas sensors, the microhotplate based SnO thinfilm sensors offer a number of interesting features and are partic-ularly attractive for their practical interest [1]. Indeed, these de-vices feature high sensitivity, low power consumption, as well ascompactness and good compatibility with semiconductor tech-nology. Unfortunately, thin film sensors (as do all gas sensors),suffer from a number of shortcomings such as cross selectivityto gases i.e., low selectivity, high sensitivity to humidity, nonlin-earities of the sensor’s response, drift, and slow response. Poorselectivity toward the monitored gas, or cross sensitivity towardother gases makes a sensor’s output unreliable. Long exposure
Manuscript received April 12, 2004; revised April 27, 2005. This work wassupported by the Research Grant Council (RGC) of Hong Kong under the com-petitive earmarked research Grant HKUST 6162/04E. The associate editor co-ordinating the review of this paper and approving it for publication was Prof.Michael Schoening.
Digital Object Identifier 10.1109/JSEN.2005.858926
cycles of the sensors as well as aging factors and poor stabilitycauses a sensor’s calibration curve drift with time [2]. The driftcan be explained as a random temporal variation of the sensorresponse when exposed to the same gases under identical condi-tions. These drifts are due to unknown dynamic processes in thesensor system (e.g., poisoning or ageing of sensors) or environ-mental changes (e.g., temperature and pressure conditions). Theslow response feature of most gas sensors is also a very criticalissue as fast detection is one of the most important requirementsin a number of applications, such as hazard, poisonous and dan-gerous gas detection. Indeed, gas sensors react slowly and thesteady-state response is typically obtained after few minutes. Aninteresting current tendency in research is to address the slow re-sponse using nano-structural engineering such as grain growthof metal oxide films. Response time in the order of few secondshas been reported in the literature [3]. Unfortunately technolog-ical potentiality of structural engineering are not yet fully ex-ploited and the techniques used are not studied sufficiently [2].An alternative solution to improve the response time is to ad-dress the issue at the algorithmic level by processing the tran-sient response rather than the steady-state response of the sensor.
Pattern recognition algorithms combined with a gas sensorarray have been traditionally used to address non selectivityand sensors nonlinearity issues [4]. In fact, a gas sensor arraypermits to improve the selectivity of the single gas sensor, andshows the ability to classify different gases. Significant researchwork has been carried out during the last decade in gas detection,preprocessing techniques as well as pattern recognition algo-rithms [4], [5]. However, robust and accurate gas discriminationstill remain a challenge even for most advanced pattern recog-nition techniques due to the problems previously mentioned.
In this paper, we present a fast and robust gas classification ap-proach for combustible gases application using micromachinedSnO gas sensors, and a pattern recognition algorithm basedon class-conditional density estimation using Gaussian mixturemodels (GMM). Fast recognition is achieved by developingnovel methods for selecting dynamic features of the sensorssignals, which not only improve detection performances, butalso speed it up considerably. Robust detection is also proposedthrough a drift counteraction approach based on extending thetraining data set using a simulated drift. The performance of theretrained GMM shows the effectiveness of the new approach inimproving the classification performance in the presence of drift.
The paper is organized as follows. Section II presents the in-tegrated micromachined gas sensor technology and the experi-
1434 IEEE SENSORS JOURNAL, VOL. 5, NO. 6, DECEMBER 2005
Fig. 1. Cross section of the gas sensor.
mental characterization of the integrated gas sensor array. Sec-tion III describes the pattern recognition system based on GMMclassifiers. Section IV presents the experimental results relatedto the classification. Special emphasis will be placed on fast androbust detection. Section V concludes this paper.
II. GAS SENSORS TECHNOLOGY AND EXPERIMENTAL
CHARACTERIZATION
A. Gas Sensor Technology
The tin oxide is still the most popularly used material forthe detection of combustible gases and toxic contaminants. Mi-croelectronic gas sensors based on tin oxide films are, there-fore, extensively used in gas detection applications. Such de-vices are sensitive to specific gases when heated at high temper-ature levels (around 300 C). To reach such high temperatures, amicrostructure called the microhotplate (MHP) was developed[6]. This structure is built using either front-side or back-sideetch bulk micromachining techniques. The thermally isolatedhotplate was fabricated using surface silicon micromachiningtechnique. The front-side surface machined MHP permits to re-tain all the desirable thermal characteristics that are essential tothe integrated gas sensor applications. The cross-sectional viewof the device is shown in Fig. 1.
The MHP is suspended by four microbridges at the four cor-ners. The bridges are 30 m wide and 58 m long. The areaoutside the MHP remains at the silicon substrate temperature,which reduces the thermal crosstalk between individual MHPsin a sensor array system and when supporting electronic circuityis to be integrated with the MHP on the same chip. In order toimprove the temperature uniformity of the MHP, a polysiliconheater ring is placed at the outer perimeter of the microstruc-ture. The polysilicon resistor at the center monitors the temper-ature of the MHP. The insulating air-gap was formed by etchingaway polysilicon sacrificial layer. Finite element thermal anal-ysis suggested that 1.5–2- m air-gap provides effective thermalisolation for the MHP.
The device was fabricated using our in-house 4- m designrule process. The device requires seven masks and occupies anarea of 120 120 m. The stability of the sensor as well asits sensitivity, defined as the ratio between the change in sensorresistance in the presence and in the absence of the gas, was
Fig. 2. Microphotograph of the integrated gas sensor array.
experimentally characterized [1]. It was found that the sensorexhibits very good stability (up to 500 cycles) and excellent sen-sitivity to CO as low as 1 ppm. Based on this sensor structure, anintegrated gas sensor array, including four individually control-lable units was developed. Fig. 2 shows a microphotograph ofthe manufactured chip including four sensors on a single chip.The main advantage of the surface machined process comparedwith the bulk silicon etching process is the high yield, whichis of primary importance when implementing an array. Indeed,the yield typically drops in an array based implementation ascompared to single sensor structure due to an increase in siliconarea and the requirement of different sensing films making theprocess more difficult. A single chip integrates four sensor ele-ments as shown in the microphotograph of Fig. 2.
Each sensor has its own heater and temperature sensor. Threedifferent sensing films are used to implement the sensor array.One sensor is based on Au/SnO (sensor 1), another sensor isbased on Pt/Cu(0.16 wt%)-SnO (sensor 2), and the remainingtwo sensors are based on Pt/SnO (sensors 3 and 4). Totally,two chips were used and calibrated by tuning their selectivity toa given set of gases using the temperature parameter. Before car-rying out electrical measurement, the temperature of the micro-hotplate is calibrated by first recording the current flow through
BRAHIM-BELHOUARI et al.: FAST AND ROBUST GAS IDENTIFICATION SYSTEM 1435
TABLE ISENSOR DESCRIPTION
the heating layer and the temperature (sensor resistance). Thesensitivity to CO was found to reach its peak at about 300 C forsensor 3 and sensor 4 Pt/SnO ), the remaining sensors also re-spond to variations in the CO concentration but to a lower extent.A good sensitivity to H was obtained at about 260 C, while agood sensitivity to CH was obtained at about 300 C. Table Isummarizes all the sensors used together with the different pa-rameters. It should be noticed that the two chips are identical;however, the operating temperature is different, allowing us totune the selectivity of the two chips to different gases. Usingthe temperature parameter to set the selectivity of gas sensorsis gaining greater interest by researchers in this field mainly be-cause the selectivity is greatly influenced by the operating tem-perature [7].
The two chips provide eight responses, which could be seenas a fingerprint or a signature corresponding to a given gasmixture, which can then be exploited by a pattern recognitionsystem in order to build a selective detection system, as will bedescribed in the next sections.
B. Experimental Characterization
An automated experimental setup was built in order to per-form electrical and gas sensing characterization. The setup canbe used to measure gas-sensing characteristics in well definedtemperature cycles and gas concentration levels. Fig. 3 illus-trates an overall view of the system including the gas chamber,the gas delivery system as well as the data acquisition system.The gas chamber with a diameter of 90 mm and a reactionvolume of 100 cm was used for the experiment. The chip car-rier is inserted into a chip socket and placed inside the chamberwith feed-through wires used for resistance measurement andtemperature control. The gas delivery system includes valves
and three mass flow controllers (MFC) (with a maximumflow rate of 500 ml/min) for the tested gas and one MFC for thesynthetic air (1l/min). The data acquisition board (DAQ) fromNational Instrument is used in order to control the valves andthe MFCs. The DAQ is also used to record the output of thesensors for further processing. The gas concentrations in thesensor chamber are adjusted by selecting the correct flow ratefor different gases. Input signals generated by the data acquisi-tion board and used to control the MFC are pulse signals cor-responding to different concentrations. The mode of operationis, therefore, an online measurement without reference gas withgradually increasing concentrations.
The temperature of the sensors is constantly monitored by pe-riodically reading data from the integrated temperature sensor.The microhotplates of each chip are heated to a particular tem-perature by flowing the precalibrated current through the heatingelement. A current flow of 2.8 mA/ m through the heatinglayer is required for an operating temperature of about 300 C.
Fig. 3. Experimental setup used to characterize the sensors. V stands for Valve,MFC stands for mass flow controller, and the DAQ is the data acquisition boardused to control the setup and to acquire the signals from the sensor array. DUTstands for device under test.
TABLE IISENSOR ARRAY CHARACTERISTICS AND CHARACTERIZATION
SETUP PARAMETERS
This corresponds to a maximum power consumption of about200 mW per chip. Table II summarizes the chip features as wellas the characterization set-up parameters.
The sensors output are raw voltage measurements in theform of exponential-like curves typically described by Fig. 4.Gases used in the experiment are methane, carbon monoxide,hydrogen, and two binary mixtures: one of methane and carbonmonoxide and another of hydrogen and carbon monoxide.Vapors were injected into the gas chamber at a flow rate deter-mined by the mass flow controllers (MFC).
The steady-state values of the array sensor were recordedwhile periodically injecting different gases and the baselineresponse of each sensor was normalized using the Euclideannorm. Normalization has been previously employed in gas dis-crimination applications where the identification must be basedon signature pattern, and not on the concentration dependentamplitudes [8]–[10]. On one hand, normalization is useful to setthe range of values for sensors’ output to range in order toavoid the data pattern with larger signal magnitude to dominatein the data space. On the other hand, normalization is applied toremove the concentration dependence within the data space. Be-cause the concentration for an unknown gas is also unknown, theidentification must be based on signature pattern, and not on theconcentration dependent amplitudes. Fig. 5 shows an example
1436 IEEE SENSORS JOURNAL, VOL. 5, NO. 6, DECEMBER 2005
Fig. 4. Raw response of an array of eight microelectronic gas sensors.
Fig. 5. Histograms showing the response patterns of the eight gas sensors exposed to CH , CO, and their mixture.
of a typical steady-state response for the sensor array exposed todifferent gases. We can note that the response of the two chips is
quite different due to different operating temperatures, as wellas mismatch in the fabrication process.
BRAHIM-BELHOUARI et al.: FAST AND ROBUST GAS IDENTIFICATION SYSTEM 1437
Fig. 6. Additive drift affecting the sensor baseline.
TABLE IIIREVIEW OF THE TYPICAL RESPONSE TIMES OF SnO -BASED GAS SENSORS ILLUSTRATING THE MATERIAL
USED, THE GASES INVOLVED, AND THEIR CONCENTRATIONS AS WELL AS THE OPERATING TEMPERATURE
A gas data set of 220 patterns (each pattern consists of eightsensor responses) was created to train the different density clas-sifiers and to evaluate their identification performances.
C. Sensor Nonidealities and Response Time
Our aim is to achieve identification of combustion gases withan array of tin oxide sensors. The pattern recognition strategyshould be robust against the inherent problems of the sensorsused. These include nonlinear response, poor selectivity, driftand slow response time. In addition to nonlinearity and nonse-lectivity, one of the most serious limitations of tin oxide gas sen-sors is the drift problem, which causes significant temporal vari-ations of the sensor response when exposed to the same gasesunder identical conditions. As a result of drift, the cluster distri-bution in the feature space becomes unstable over time, making
useless the decision surface built by the classifier during thetraining phase. The drift can affect both the baseline (additive)and the sensitivity of the sensor (multiplicative).
Fig. 6 illustrates an example of an additive drift problem inwhich we have reported the response of the sensor as functionof the concentration of gases periodically injected into a gaschamber in which the sensors are being placed. We can note thatthe baseline response of the sensor is shifted which complicatesthe classification problem even further.
Finally, detecting combustion and dangerous gases requiresfast and reliable detection. This requirement is made very chal-lenging due to the typically slow response of the tin oxide sen-sors. This issue is critical for most gas sensors with reaction timein the order of few minutes. Table III illustrates response timesof SnO based gas sensors, reported in the literature. Typical re-
1438 IEEE SENSORS JOURNAL, VOL. 5, NO. 6, DECEMBER 2005
sponse times are reported to be in the range of few minutes. Itshould be, however, noticed that response times cannot be statedsolely as they are dependent on many parameters such as typeof material, thickness of the membrane, operating temperatureas well as concentration of the gases in question. Our sensorspresent a response time of 1–3 min at about 300 C and 50-ppmCO. Recently, faster gas sensors based on nano-structures usingspecial chemical deposition process have been reported in theliterature [3] with response times in the order of 2–6 s at op-erating temperatures of about 200 C and H concentrations inthe range of 1000–8000 ppm. A recent study [2] suggested threemain methods for reducing the response time, which are 1) in-creasing the working temperature, 2) reducing the film thick-ness, and 3) introducing noble metals into the film.
Our aim is to achieve robust and fast identification of combus-tion gases with an array of tin oxide sensors using advanced pat-tern recognition algorithms overcoming the inherent problemsof our sensors. The robustness as well as the speed of classifica-tion is here tackled at the algorithmic level. It should be noticedthat there is currently a trend to tackle these issues at the phys-ical and fabrication level as suggested in [2].
III. PATTERN RECOGNITION SYSTEM
The algorithmic part of a gas identification system includes apreprocessing module and a classification procedure. The roleof the preprocessing module is to segment the pattern of interestfrom the background, remove noise, normalize the pattern andany other operation that contributes in defining a compact repre-sentation of the pattern. Classification tasks address the problemof identifying unknown sample as one from a set of recogniz-able gases.
A. Feature Selection
The data preprocessing stage operates on the gas sensorresponses in a way that improves the overall pattern analysisperformance. It can be achieved by extracting parameters that aredescriptive of the sensor array responses. Thus, the raw data istransformed into a characteristic feature vector. Numerous pre-processing techniques have been proposed in the literature [4].The most common procedure uses the steady state of the sensors’response as a feature vector and ignores the transient response[19]. A number of compression algorithms have been proposedto extract additional information from the transient response,resulting in improved selectivity and increased recognitionaccuracy [20]. However, these techniques normally require com-plicated analysis of the whole dataset, and so that, in any case, it isnecessary towait for thesteady-state response. Inorder to achievereliable recognition as fast as possible, we propose to select fea-tures over short time interval since gas injection. The problemof feature selection can be defined as that of selecting a subsetof features that achieve the best classification performance. Let
be the initial dataset containing a number of voltage features.Each feature corresponds to different time points of the searchinterval .The objective is to finda subset containingfew features that minimizes a selection criterion
(1)
In our case, classification error was used as the selection cri-terion since the focus is on classifying different gases. Severalsearch methods have been used to explore efficiently the fea-ture space. The simplest method is sequential forward selection(SFS) [21]. This procedure starts from the empty set and se-quentially adds features that achieve the lowest value for the se-lection criterion . This process continues until all features areincluded in the subset.
B. GMM Classifier
The objective of pattern recognition is to set a decision rule,which optimally partitions the data space into regions, one foreach class . A pattern classifier generates a class label for anunknown feature vector from a discrete set of previouslylearned classes. The most general classification approach is touse the posterior probability of class membership . Tominimize the probability of misclassification one should con-sider the maximum a posterior rule and assign to class
(2)
where is the class-conditional density and isthe prior probability. In the absence of prior knowledge,can be approximated by the relative frequency of examples inthe dataset. One way to build a classifier is to estimate the class-conditional densities by using representation models for howeach pattern class populates the feature space. In this approach,classifier systems are built by considering each of the class inturn, and estimating the corresponding class-conditional densi-ties from data.
The most widely used method of nonparametric density es-timation is the nearest neighbors (KNN). Despite the sim-plicity of the algorithm, it often performs very well and is animportant benchmark method. However, one drawback of KNNis that all the training data must be stored, and a large amount ofprocessing is needed to evaluate the density for a new input pat-tern. An alternative is to combine the advantages of both para-metric and nonparametric methods by allowing a very generalclass of functional forms in which the number of adaptive pa-rameters can be increased to build more flexible models. Thisleads us to a powerful technique for density estimation, calledmixture model [22]. In our work we focus on semi-parametricmodels based on Gaussian mixture distributions.
In a Gaussian mixture model, a probability density functionis expressed as a linear combination of basis functions. A modelwith components is described as mixture distribution [22]
(3)
where are the mixing coefficients and the parameters of thecomponent density functions vary with . Each mixturecomponent is defined by a Gaussian parametric distribution in
-dimensional space
BRAHIM-BELHOUARI et al.: FAST AND ROBUST GAS IDENTIFICATION SYSTEM 1439
The parameters to be estimated are the mixing coefficients ,the covariance matrix and the mean vector . The covari-ance matrix can be spherical where , diagonal i.e.,
or any positive definite fullmatrix.
The method for training mixture model is based on maxi-mizing the data likelihood. The log likelihood of the dataset
, which is treated as an error, is defined by
(4)
A specialized method is commonly used to produce optimumparameters, known as the expectation-maximization (EM) al-gorithm [23]. The EM algorithm iteratively modifies the modelparameters starting from the initial iteration . EM guar-antees a monotonically non decreasing likelihood although itsability to find a local maximum depends on parameter initial-ization. Convergence can be accelerated by modifying the max-imization step. For GMM, the EM optimization can be carriedout analytically with a simple set of equations [23], where themixing coefficients are estimated by
(5)
and the estimate for the means for each component is given by
(6)
and, finally, the update equation for the covariance matrix is
(7)
Minimum description length (MDL) criterion is able to selectan optimal number of components in the model and so partitionthe dataset. MDL was derived by Risanen [24] from an infor-mation theoretic perspective. Although the class-conditionaldistributions in feature space are generally non-Gaussian, theresulting multimodal approximation is remarkably accurate.GMM can approximate any continuous density with an ar-bitrary accuracy provided the model has a sufficiently largenumber of components, and provided the parameters of themodel are chosen correctly. The price we have to pay is thatthe training process is computationally intensive compared tothe simple procedure needed for parametric methods. Whenthe training algorithm constructs a mixture model with onlyone component, the GMM classifier has the same behavior asa Bayesian quadratic classifier, where the decision boundariesare hyper-ellipsoids or hyper-paraboloids.
IV. RESULTS AND ANALYSIS
In this section, we present results obtained by using severalconfigurations with our pattern recognition system. Experi-ments are based on the gas dataset of 220 patterns with eachpattern consisting of eight sensor responses.
TABLE IVGMM STRUCTURE SELECTION
A. Classification Results
In order to evaluate the classification performance of the pro-posed classifier, we first consider the steady-state response andperform three studies.
1) GMM Structure Selection: Several covariance matrixforms can be used to estimate the class-conditional density.Three competing structures based on full, diagonal or sphericalform are considered here. The GMM classifier is built by consid-ering each of the class in turn, and estimating the correspondingclass-conditional densities from the data. MDL crite-rion is used to select an optimal number of components for eachdensity model. The parameters of each model were adapted tothe training data in the maximum likelihood framework usingEM algorithm. These GMM structures have different numberof parameters. In order to compare the performance of differentstructures, a ten-fold cross validation approach was used to over-come the problem of the limited data set typically available in gassensors applications. In our ten-fold cross validation approach,the data set is split into two mutually exclusive subsets, one forlearning and one for testing. In order to minimize dependencybetween the data partition and the classification performance,10 different partitions were created. The classification accuracywas evaluated for each partition and the final result is expressedas the average performance.
The obtained results are reported in Table IV. The full struc-ture appears to be the best model in terms of classification accu-racy and complexity given by the structure parameters number.GMM with a full structure provides a better representation ofthe true density of the data set. The full Gaussian model permitsto build a very general structure in which the number of adap-tive parameters can be increased in a systematic way featuringa more flexible and accurate model.
2) Dimensionality Reduction: Previous results were ob-tained using the whole dataset. This second analysis wasmade on reduced datasets using a projection technique. Priorto applying the GMM classifier, a dimensionality reductiontechnique namely principal component analysis (PCA) wasused in order to perform redundancy removing and featurereduction. Fig. 7 presents the two-dimensional PCA scores forall the studied gas sensors steady-state voltages. We can notethat the decision boundaries are not well defined due to a strongoverlapping of the classes.
GMMs are built on projected datasets with different numberof principal components. Fig. 8 shows the class-conditional den-sity for the third class (CO and CH mixture) using GMM withthree component density functions and a full covariance ma-trix. The best performance is achieved when projecting to fiveprincipal components. The addition of components actually de-grades the performance of the classifier. Results of this analysisare shown in Table V. PCA projection improves the classifica-tion accuracy for all density model classifiers.
1440 IEEE SENSORS JOURNAL, VOL. 5, NO. 6, DECEMBER 2005
Fig. 7. PCA results for the microelectronic sensor array steady-state voltage. Measurement type: CO (circles), CH (plus signs), mixture CO-CH (diamonds),H (triangles), and mixture CO-H (squares).
Fig. 8. Class-conditional density for the third class (CO and CH mixture) using GMM with three component density functions and a full covariance matrix.
BRAHIM-BELHOUARI et al.: FAST AND ROBUST GAS IDENTIFICATION SYSTEM 1441
TABLE VCLASSIFICATION ACCURACY (%) WITH AND WITHOUT PCA PROJECTION
It can be concluded that the use of dimensionality reductiondepends on the relationship between the training dataset sizeand the number of features. If the number of training examplesis very large, then the classification error does not increase as thenumber of features increases. However, if the number of trainingexamples is small relative to the number of features, a dimen-sionality reduction technique is needed to guarantee an accept-able classification accuracy. This is typically the case regardlessof the GMM structure used as evidenced from Table V. Sinceonly a limited number of examples are typically available, thereis an optimal number of feature dimensions beyond which theperformance of the Gaussian mixture models start to degrade.The GMM performance was also evaluated using a single chipoperated at a single temperature of either 300 C or 260 Ccompared to the case of two chips operated simultaneously attwo different temperatures. For a single chip solution, the per-formance were found to be at best1 86%. This performance isclearly lower than that of the two chips operating at two tem-peratures which was found to be 94.2%.
3) GMM Versus Other Classifiers: The GMM classificationperformance was compared with widely used classificationtechniques in pattern recognition problems such as K-nearestneighbor (KNN), multilayer perceptron (MLP), support vectormachines (SVMs), and the probabilistic principal componentanalysis (PPCA). PPCA is a density model based classifiersimilar in its concept to the GMM algorithm [25]. Indeed, inPPCA, each component density function is given by a proba-bilistic PCA and the training of such a model can be done inthe maximum likelihood framework using an EM algorithm.SVMs are now used in gas sensors applications [26] becausethey are well-grounded in statistical learning theory and theyovercome many of the drawbacks seen in previously describedpattern recognition techniques. In this comparison, all classi-fiers were individually optimized and their performance wascompared for different principal components. GMM with afull covariance matrix is now considered as it has proven to bethe most effective structure. For MLP, an optimized structurehas been found with eight hidden units (117 weights). Bestgeneralization performance for KNN algorithm is given for Kequal to 3. For SVMs, the multiclass implementation of [27]was used. Generalization performances were estimated usingagain a ten-fold cross validation approach. Fig. 9 shows thatthe most accurate discriminant function is the SVM, while themost accurate density model is the GMM. For eight principalcomponents, the best performance is obtained for SVMs. SVMsare shown to work quite well when the dataset is small with re-spect to the input dimensionality. However, better performance
1The best-case performance is obtained by taking the maximum performancewhen operating the sensor at either 300 C or 260 C, and for all possible PCAprojections.
is obtained for GMM when projecting the data to the fivefirst principal components. The best performance is achievedusing GMM with a success rate of 94.2% obtained for fiveprincipal components. This points out to an important resultwhich suggests that higher generalization performance can beobtained by using feature reduction and selection techniques aspreprocessing techniques for increasing the ratio of the numberof training samples over the number of features.
B. Fast Recognition
The sensor array system reacts slowly and takes, on average,10 min to reach the stationary state. This time is a combinationof the time to fill the chamber and the sensors time response. Onepossible goal in pattern recognition is to achieve reliable resultsas fast as possible. In order to study the possibility of buildingfast recognition system based on dynamical response, we selectpatterns from different time points (over short period since gasinjection) of the transient sensors responses. For different searchintervals since gas injection, we set the initial data set to fea-tures corresponding to time points within each interval. We usethe sequential forward algorithm as a feature selector (1) withGMM and PCA dimensionality reduction. The optimization re-sults are shown in Table VI.
It was found that a maximum of 96.3% accuracy is obtainedin the time window [0, 30s] since gas injection with a subsetof three selected features, corresponding to 6, 18, and 27s, persensor. It is remarkable to note that, for times lower than 30s, aclassification performance of more than 94% is still achievableusing the optimized structure of GMM.
C. Robustness
The drift problem can cause a serious robustness issue as itcan be interpreted as temporal variations of the pattern distribu-tion in the feature space. The decision surface obtained duringthe training phase is, therefore, made obsolete, and, hence, re-training the entire system is necessary.
To compensate for the patterns dispersion movement, we pro-pose to extract robust features by generating simulated drift.The training data set is extended by extracting more featuresfrom the drifted measurements. This new learning space in-creases the classifier robustness to temporal variation of thesensor response. The efficiency of this procedure has been testedagainst simulated linear drift. The drift has been modeled as
where is the sensor output before the driftexperiment and was chosen randomly for each sensor[28]. Drift varying between 0 and 30% has been artificially gen-erated. The performance of the best classifier was evaluated overthe drifted measurements. Fig. 10 shows that drift affects therecognition ability of GMM as the classification success de-clines significantly (dashed line of Fig. 10). The drift counterac-tion strategy is to retrain GMM using drifted sensor responses(solid line of Fig. 10). The performance of the retrained GMMwas evaluated using the ten-fold cross validation method. It isshown that the counteraction procedure improves the perfor-mance of GMM in presence of 30% drift by a factor of over30%. The final assessment of this procedure has to be achievedby testing it over real sensor’s drift data.
1442 IEEE SENSORS JOURNAL, VOL. 5, NO. 6, DECEMBER 2005
Fig. 9. Accuracy as a function of the number of principal components.
Preprocessing the sensors signals and using advanced patternrecognition techniques are fundamental parts of gas identifica-tion systems. However, training a classifier for fast and robustdetection and recognition of different odors still remains chal-lenging partly because of slow response of the sensors, the tem-poral variability of the gas sensors, the large intra-class varianceas compared to the small inter-class separation and the smallamount of training data available. The use of dimensionality re-duction is an effective way to improve the performance of den-sity models classifiers particularly when the number of trainingexamples is small relative to the number of features. We haveshown that optimized GMM permits to achieve the highest ac-curacy when projecting the data to the five first principal compo-nents. We have also demonstrated that using advanced featureselection algorithms to select transient points can achieve im-proved and fast detection.
It is, however, important to notice that in order for GMM toproduce good generalization results, the test data should have
a relatively similar probability distribution compared to that ofthe training set. A general approach to this problem is to monitorthe likelihood of feature vectors during operation and comparethem with the range of likelihood in the training set. Any valuesthat fall outside this range are probably due to novel inputs, andthe corresponding model output should not be relied upon. Thisnovelty detection approach is a useful technique that should beconsidered for future investigation in order to improve our gasidentification performance.
We have also shown that drift counteraction approach basedon extracting robust features using a simulated drift is quite effi-cient in improving the robustness of the classifier. Further offsetcancellation techniques are currently being explored using hard-ware implementation in which the sensor information is dynam-ically obtained by subtracting the steady-state value with themost updated offset value. This approach is a real-time solutionwhich appears to be effective in reducing additive drift.
V. CONCLUSION
In this paper, we presented a gas identification approachbased on a microelectronic gas sensors technology and class-conditional density estimation using GMMs. In-house mi-croelectronic gas sensors based on tin oxide films and amicrostructure called the MHP were used. Extensive mea-surements and sensor characterization were performed usingan automated experimental setup for combustion gases iden-tification (methane, carbon monoxide, hydrogen, and theirmixtures). The GMM classifier structure was optimized by
BRAHIM-BELHOUARI et al.: FAST AND ROBUST GAS IDENTIFICATION SYSTEM 1443
Fig. 10. Classification performance as function of drift (expressed in %) before (dashed) and after (solid) retraining.
considering several covariance matrix forms used to estimatethe class-conditional density. It was found that the structurewith a full covariance matrix optimized with MDL criterion,presents the best classification accuracy for the gas sensorsdata set collected from the integrated gas sensor array. Theproposed classifier is shown to perform very well as comparedto traditional as well as advanced pattern recognition algo-rithms such as KNN, MLP, SVM, and PPCA. Indeed, the bestperformance is achieved using GMM with a success rate of94.2% obtained for five principal components. This points outto an important result which suggests that higher generalizationperformance can be obtained by using feature reduction andselection techniques as preprocessing techniques for increasingthe ratio of the number of training samples over the numberof features. Using the operating temperature as a parameter totune the selectivity of the sensor chip to different target gaseswas also proven to be an effective way to improve performanceof the overall system. In addition, fast recognition with anexcellent accuracy of more than 96% was obtained in the timewindow [0–30 s] since gas injection. It was found that theclassification accuracy can be trade for even faster recognition(94.5% for 15s). This classification success rate is achieved bythe combination of sensitive gas sensor array, dynamic featuresselected using SFS algorithm, and GMM with optimized struc-ture. It was, however, found that the drift seriously degradesthe classification performance of GMM. A drift counteractionapproach based on extracting robust features using a simulateddrift was proposed. The performance of the retrained GMM
was evaluated using a cross validation method which shows again of over 30% obtained for up to 30% drift.
ACKNOWLEDGMENT
The authors would like to thank Prof. G. Yan and Dr. D. Mar-tinez for their technical support and help.
REFERENCES
[1] P. C.H. Chan, G. Yan, L. Sheng, R. K. Sharma, Z. Tang, J.K.O. Sin,I.-M. Hsing, and Y. Wang, “An integrated gas sensor technology usingsurface micro-machining,” Sens. Actuators B, vol. 82, pp. 277–283,2002.
[2] G. Korotcenkov, “Gas response control through structural and chemicalmodification of metal oxide films: State of the art and approaches,” inSens. Actuators B, to be published.
[3] J. W. Gong, Q. F. Chen, W. F. Fei, and S. Seal, “Micromachinednanocrystalline SnO2 chemical gas sensors for electronic nose,” Sens.Actuators B, vol. 102, no. 1, pp. 117–125, 2004.
[4] R. Gutierrez-Osuna, “Pattern analysis for machine olfaction: A review,”IEEE Sensors J., vol. 2, no. 3, pp. 189–202, Jun. 2002.
[5] M. Pardo and G. Sberveglieri, “Learning from data: A tutorial with em-phasis on modern pattern recognition methods,” IEEE Sensors J., vol. 2,no. 3, pp. 189–202, Jun. 2002.
[6] G. Yan, L. Sheng, Z. Tang, J. Wu, P. C.H. Chan, and J.K.O. Sin, “A lowpower CMOS compatible integrated gas sensor using maskless tin oxidesputtering,” Sens. Actuators B, vol. 49, pp. 81–87, 1998.
[7] , “Multi-frequency temperature modulation for metal-oxide gassensors,” presented at the 8th Int. Symp. Olfaction and the ElectronicNose, Washington, DC, Mar. 2001.
[8] T. C. Pearce, “Computational parallels between the biological olfactorypathway and its analogue ‘The Electronic Nose’: Part II. Sensor-basedmachine olfaction,” Biosystems, vol. 41, pp. 69–90, 1997.
1444 IEEE SENSORS JOURNAL, VOL. 5, NO. 6, DECEMBER 2005
[9] R. Polikar, R. Shinar, V. Honavar, L. Udpa, and M. D. Porter, “Detectionand identification of odorants using an electronic nose,” in Proc. IEEEInt. Conf. Acoustics, Speech, and Signal Processing, vol. 5, 2001, pp.3137–3140.
[10] E. L. Hines, E. Llobet, and J. W. Gardner, “Electronic noses: A reviewof signal processing techniques,” Proc. Inst. Elect. Eng., Circuit DevicesSyst., vol. 146, no. 6, pp. 297–310, 1999.
[11] M. A. Aronova, K. S. Chang, I. Takeuchi, H. Jabs, D. Westerheim, A.Gonzalez-Martin, J. Kim, and B. Lewis, “Combinatorial libraries ofsemiconductor gas sensors as inorganic electronic noses,” Appl. Phys.Lett., vol. 83, no. 6, pp. 1255–1257, 2003.
[12] K. D. Mitzner, J. Sternhagen, and D. W. Galipeau, “Development of amicromachined hazardous gas sensor array,” Sens. Actuators B, vol. 93,no. 1–3, pp. 92–99, 2003.
[13] L. Chambon, J. P. Germain, A. Pauly, V. Demarne, and A. Grisel, “Ametallic oxide gas sensor array for a selective detection of the CO andNH3 gases,” Sens. Actuators B, vol. 60, no. 2–3, pp. 138–147, 1999.
[14] P. Althainz, A. Dahlke, M. Frietsch-Klarhof, J. Goschnick, and H. J.Ache, “Reception tuning of gas-sensor microsystems by selective coat-ings,” Sens. Actuators B, vol. 24–25, pp. 366–369, 1995.
[15] G. G. Mandayo, E. Castano, and F. J. Gracia, “Carbo monoxide detectorfabricated on the basis of a tin oxide novel doping method,” IEEE Sen-sors J., vol. 2, no. 4, pp. 322–328, Aug. 2002.
[16] R. P. Lyle and D. Walters, “Commercialization of silicon-based gas sen-sors,” in Proc. Transducers, Solid State Sensors and Actuators, Chicago,IL, 1997, pp. 975–978.
[17] J. W. Allen, B. T. Marquis, and D. J. Smith, “Investigating the long-termheating and analyte exposure effects on tin oxide thick-film sensors,” inProc. IEEE Sensors Conf., vol. 2, 2002, pp. 1260–1265.
[18] I. Sayago, M. D. C. Horrillo, S. Baluk, M. Aleixandre, M. J. Fernandez,L. Ares, M. Garcia, J. Santos, and J. Gutierrez, “Detection of toxic gasesby a tin oxide multisensor,” IEEE Sensors J., vol. 2, no. 5, pp. 387–393,Oct. 2002.
[19] J. W. Gardener, M. Craven, C. Dow, and E. L. Hines, “The predic-tion of bacteria type and culture growth phase by an electronic nosewith a multi-layer perceptron network,” Meas. Sci. Technol., vol. 9, pp.120–127, 1998.
[20] T. Eklov, P. Martensson, and I. Lundstrom, “Enhanced selectivity ofMOSFET gas sensors by systematical analysis of transient parameters,”Anal. Chim. Acta, vol. 353, pp. 291–300, 1997.
[21] , “Selection of variables for interpreting multivariate gas sensordata,” Anal. Chim. Acta, vol. 381, pp. 221–232, 1999.
[22] D. M. Titterington, A. F. M. Smith, and U. E. Makov, Statistical Analysisof Finite Mixture Distributions. New York: Wiley, 1985.
[23] C. M. Bishop, Neural Networks for Pattern Recognition. Oxford,U.K.: Clarendon, 1995.
[24] J. Rissanen, “Modeling by Shortest Data Description,” Automatica, vol.14, pp. 465–471, 1978.
[25] S. Brahim-Belhouari, A. Bermak, G. Wei, and P.C.H. Chan, “A compar-ative study of density models for gas identification using microelectronicgas sensor,” presented at the IEEE Conf. Signal Processing and Informa-tion Technology ISSPIT, Darmstadt, Germany, 2003.
[26] C. Distante, N. Ancona, and P. Siciliano, “Support vector machines forolfactory signals recognition,” Sens. Actuators B, vol. 88, no. 1, pp.30–39, 2003.
[27] Y. Guermeur, “Combining discriminant models with new multi-classSVMs,” Pattern Anal. Appl., vol. 5, no. 2, pp. 168–179, 2002.
[28] S. Marco, A. Ortega, A. Pardo, and J. Samitier, “Gas identification withtin oxide sensor array and self-organizing maps: Adaptive correction ofsensor drifts,” IEEE Trans. Instrum. Meas., vol. 47, no. 1, pp. 316–321,Feb. 1998.
Sofiane Brahim-Belhouari received the engineerdiploma in electrical engineering from the Poly-technic Institute of Algiers, Algeria, in 1993, andthe Ph.D. degree in automatic control and signalprocessing from the University of Paris XI, Paris,France, in 2000.
After receiving the Ph.D. degree, he held postdoc-torate positions at the Ecole Polytechnique Fédéralede Lausanne (EPFL), Lausanne, Switzerland, and theHong Kong University of Science and Technology,Kowloon. His main research interests are in data anal-
ysis, statistical signal processing, and pattern recognition.
Amine Bermak (M’99–SM’04) received the M.Eng.and Ph.D. degrees in electronic engineering fromPaul Sabatier University, France, in 1994 and 1998,respectively.
During his Ph.D. studies, he was part of the Mi-crosystems and Microstructures Research Group atthe French National Research Center LAAS-CNRSwhere he developed a 3-D VLSI chip for artifi-cial neural network classification and detectionapplications. He joined the Advanced Computer Ar-chitecture research group at York University, York,
U.K., where he was working as a postdoctorate on the VLSI implementationof CMM neural networks for vision applications in a project funded by theBritish Aerospace. In November 1998, he joined Edith Cowan University,Perth, Australia, first as a Research Fellow working on smart vision sensors,then as a Lecturer and a Senior Lecturer with the School of Engineeringand Mathematics. He is currently an Assistant Professor with the Electricaland Electronic Engineering Department, Hong Kong University of Scienceand Technology (HKUST), Kowloon, where he also serves as the AssociateDirector of the Computer Engineering Program. He has published more than70 papers in journals, book chapters, and refereed international conferences.
Dr. Bermak was awarded the “Bechtel Foundation Engineering Teaching Ex-cellence Award” at HKUST in 2004.
Minghua Shi received the B.S. degree in electronicengineering from the East China University of Sci-ence and Technology, Shanghai, where he graduatedwith highest honors. He is currently pursuing thePh.D degree at the Hong Kong University of Scienceand Technology (HKUST), Kowloon, under thesupervision of Prof. Amine Bermak.
In September 2002, he joined the Electrical andElectronic Engineering Department, HKUST. His re-search interests are related to hardware implementa-tion of pattern recognition algorithms for gas identi-
fication and electronic nose applications.Mr. Shi received the first prize scholarship from the East China University of
Science and Technology. He was also selected for the honor of Excellent Studentof Shanghai City.
Philip C. H. Chan (SM’95) was born in Shanghai,China, and raised in Hong Kong. He received his B.S.degree in electrical engineering from the Universityof California, Davis, where he graduated with highesthonors and departmental citation, and the receivedM.S. and Ph.D. degrees in electrical engineering fromthe University of Illinois, Urbana-Champaign, underProf. C. T. Sah.
He was with the University of Illinois as an IBMPostdoctoral Fellow and later as Visiting AssistantProfessor in electrical engineering. In 1981, he joined
Intel Corporation, Santa Clara, CA, as a Senior Engineer in the Technology De-velopment Computer-Aided Design Department, where he later became a Prin-cipal Engineer and Senior Project Manager and had corporate responsibilityfor circuit simulation tools, VLSI device modeling, and process characteriza-tion. In 1990, he joined the Design Technology Department of MicroproductsGroup, where he led a team of engineers that defined and developed a CADsystem to design multichip module products. This effort led to the first func-tional 486-based multichip module at Intel. He joined the Hong Kong Univer-sity of Science of Technology (HKUST), Kowloon, in April 1991 as a Reader.He became a Professor in 1997. He served as the Director of UndergraduateStudies, the founding Director of Computer Engineering Program, the Asso-ciate Dean of Engineering, and both the Acting Head and Head of the Depart-ment of Electrical and Electronic Engineering until 2002. He was also the Di-rector of the Microelectronic Fabrication Facility, the facility that supports allthe microelectronic related research at HKUST. He is also the Director of theAdvanced Electronic Packaging Laboratory, where he initiated his research inflip-chip technology. He became the Dean of Engineering in September 2003.His research interests include microelectronics devices, circuits, integrated sen-sors, and electronic packaging.
