Hindawi Publishing CorporationThe Scientific World JournalVolume 2013 Article ID 130134 9 pageshttpdxdoiorg1011552013130134
Research ArticleAn MR Brain Images Classifier System via Particle SwarmOptimization and Kernel Support Vector Machine
Yudong Zhang12 Shuihua Wang13 Genlin Ji1 and Zhengchao Dong2
1 School of Computer Science and Technology Nanjing Normal University Nanjing Jiangsu 210023 China2 Brain Imaging Lab and MRI Unit New York State Psychiatry Institute and Columbia University New York NY 10032 USA3 School of Electronic Science and Engineering Nanjing University Nanjing Jiangsu 210046 China
Correspondence should be addressed to Yudong Zhang zhangyudongnuaagmailcom
Received 22 July 2013 Accepted 13 August 2013
Academic Editors S Bourennane C Fossati and J Marot
Copyright copy 2013 Yudong Zhang et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited
Automated abnormal brain detection is extremely of importance for clinical diagnosis Over last decades numerous methodshad been presented In this paper we proposed a novel hybrid system to classify a given MR brain image as either normal orabnormal The proposed method first employed digital wavelet transform to extract features then used principal componentanalysis (PCA) to reduce the feature space Afterwards we constructed a kernel support vector machine (KSVM) with RBF kernelusing particle swarm optimization (PSO) to optimize the parameters C and 120590 Fivefold cross-validation was utilized to avoidoverfitting In the experimental procedure we created a 90 images dataset brain downloaded fromHarvardMedical School websiteThe abnormal brain MR images consist of the following diseases glioma metastatic adenocarcinoma metastatic bronchogeniccarcinoma meningioma sarcoma Alzheimer Huntington motor neuron disease cerebral calcinosis Pickrsquos disease Alzheimerplus visual agnosia multiple sclerosis AIDS dementia Lyme encephalopathy herpes encephalitis Creutzfeld-Jakob disease andcerebral toxoplasmosis The 5-folded cross-validation classification results showed that our method achieved 9778 classificationaccuracy higher than 8622 by BP-NN and 9133 by RBF-NN For the parameter selection we compared PSO with those ofrandom selection method The results showed that the PSO is more effective to build optimal KSVM
1 Introduction
Magnetic resonance imaging (MRI) is an imaging techniquethat produces high quality images of the anatomical struc-tures of the human body especially in the brain and pro-vides rich information for clinical diagnosis and biomedicalresearch The diagnostic values of MRI are greatly magnifiedby the automated and accurate classification of the MRIimages
Wavelet transform is an effective tool for feature extrac-tion from MR brain images because they allow analysis ofimages at various levels of resolution due to its multires-olution analytic property However this technique requireslarge storage and is computationally expensive [1] In orderto reduce the feature vector dimensions and increase the dis-criminative power the principal component analysis (PCA)has been used PCA is appealing since it effectively reduces
the dimensionality of the data and therefore reduces thecomputational cost of analyzing new data [2] Then theproblem of how to classify on the input data comes
In recent years researchers have proposed a lot ofapproaches for this goal which fall into two categories Onecategory is supervised classification including support vectormachine (SVM) [3] and 119896-nearest neighbors (119896-NN) [4]Theother category is unsupervised classification including self-organization feature map (SOFM) [3] and fuzzy 119888-means [5]While all these methods achieved good results yet the super-vised classifier performs better than unsupervised classifierin terms of classification accuracy (success classification rate)[6]
Among supervised classification methods the SVMs arestate-of-the-art classification methods based on machinelearning theory [7] Compared with other methods suchas artificial neural network decision tree and Bayesian
2 The Scientific World Journal
network SVMs have significant advantages of high accuracyelegant mathematical tractability and direct geometric inter-pretation Besides it does not need a large number of trainingsamples to avoid overfitting [8]
Original SVMs are linear classifiers In this paper weintroduced in the kernel SVMs (KSVMs) which extendsoriginal linear SVMs to nonlinear SVM classifiers by apply-ing the kernel function to replace the dot product formin the original SVMs [9] The KSVMs is allowed to fitthe maximum-margin hyperplane in a transformed featurespace The transformation may be nonlinear and the trans-formed space may be high dimensional thus though theclassifier is a hyperplane in the high-dimensional featurespace it may be nonlinear in the original input space [10]
The structure of the rest of this paperwas organized as fol-lows Section 2 gave the detailed procedures of preprocessingincluding the discrete wavelet transform (DWT) and princi-pal component analysis (PCA) Section 3 first introduced themotivation and principles of linear SVM and then extendedit to softmargin dual from Section 4 introduced themethodof PSO-KSVM It first gave the principles of KSVM and thenused the particle swarm optimization algorithm to optimizethe values of parameters 119862 and 120590 finally it used 119870-foldcross-validation to protect the classifier from overfittingThe pseudocodes and flowchart were listed Experiments inSection 5 created a dataset brain of 90 brain MR images andshowed the results of each step We compared our proposedPSO-KSVM method with traditional BP-NN and RBF-NNmethods Final Section 6 was devoted to conclusions anddiscussions
2 Preprocessing
21 Feature Extraction The most conventional tool of signalanalysis is Fourier transform (FT) which breaks down atime domain signal into constituent sinusoids of differentfrequencies thus transforming the signal from time domainto frequency domain However FT has a serious drawback asdiscarding the time information of the signal For exampleanalyst cannot tell when a particular event took place from aFourier spectrumThus the classification will decrease as thetime information is lost
Gabor adapted the FT to analyze only a small section ofthe signal at a time The technique is called windowing orshort-time Fourier transform (STFT) [11] It adds a windowof particular shape to the signal STFT can be regarded asa compromise between the time information and frequencyinformation It provides some information about both timeand frequency domain However the precision of the infor-mation is limited by the size of the window
Wavelet transform (WT) represents the next logical stepa windowing technique with variable size Thus it preservesboth time and frequency information of the signal Thedevelopment of signal analysis is shown in Figure 1
Another advantage of WT is that it adopts ldquoscalerdquo insteadof traditional ldquofrequencyrdquo namely it does not produce a time-frequency view but a time-scale view of the signal The time-scale view is a different way to view data but it is a morenatural and powerful way
Fouriertransform
Short-time-fourier
-transformWavelet
transform
Am
plitu
de
Freq
uenc
y
Frequency Time
Scal
e
Time
Figure 1 The development of signal analysis
22 Discrete Wavelet Transform The discrete wavelet trans-form (DWT) is a powerful implementation of the WT usingthe dyadic scales and positions The basic fundamental ofDWT is introduced as follows Suppose that 119909(119905) is a square-integrable function then the continuous WT of 119909(119905) relativeto a given wavelet 120595(119905) is defined as
Here the wavelet 120595119886119887(119905) is calculated from the motherwavelet 120595(119905) by translation and dilation 119886 is the dilationfactor and 119887 is the translation parameter (both real positivenumbers) There are several different kinds of waveletswhich have gained popularity throughout the developmentof wavelet analysis The most important wavelet is the Harrwavelet which is the simplest one and often the preferredwavelet in a lot of applications
Equation (1) can be discretized by restraining 119886 and 119887 to adiscrete lattice (119886 = 2
119887 amp 119886 gt 0) to give the DWT which canbe expressed as follows
ca119895119896 (119899) = DS[sum
119899
119909 (119899) 119892lowast
119895(119899 minus 2
119895119896)]
cd119895119896 (119899) = DS [sum
119899
119909 (119899) ℎlowast
119895(119899 minus 2
119895119896)]
(3)
Here ca119895119896 and cd119895119896 refer to the coefficients of the approx-imation components and the detail components respec-tively 119892(119899) and ℎ(119899) denote the low-pass filter and high-pass filter respectively 119895 and 119896 represent the wavelet scaleand translation factors respectively DS operator means thedownsampling
The above decomposition process can be iterated withsuccessive approximations being decomposed in turn so thatone signal is broken down into various levels of resolutionThe whole process is called wavelet decomposition treeshown in Figure 2
23 2D DWT In case of 2D images the DWT is applied toeach dimension separately Figure 3 illustrates the schematic
The Scientific World Journal 3
S
ca1 cd1
ca2 cd2
ca3 cd3
Figure 2 A 3-level wavelet decomposition tree
Image
g(n)
h(n)
darr
darr
darr
darr
darr
darr
g(n)
h(n)
g(n)
h(n)
LL
LH
HL
HH
Subband
Figure 3 Schematic diagram of 2D DWT
diagram of 2DDWTAs a result there are 4 subband (LL LHHH and HL) images at each scale The sub-band LL is usedfor the next 2D DWT
The LL subband can be regarded as the approximationcomponent of the image while the LHHL andHHsubbandscan be regarded as the detailed components of the image Asthe level of decomposition increased compacter but coarserapproximation component was obtainedThus wavelets pro-vide a simple hierarchical framework for interpreting theimage information In our algorithm level 3 decompositionvia Harr wavelet was utilized to extract features
24 Feature Reduction Excessive features increase compu-tation times and storage memory Furthermore they some-times make classification more complicated which is calledthe curse of dimensionality It is required to reduce thenumber of features [12]
PCA is an efficient tool to reduce the dimension of adata set consisting of a large number of interrelated variableswhile retaining most of the variations It is achieved bytransforming the data set to a new set of ordered variablesaccording to their variances or importance This techniquehas three effects it orthogonalizes the components of theinput vectors so that it uncorrelated with each other itorders the resulting orthogonal components so that thosewith the largest variation come first and it eliminates thosecomponents contributing the least to the variation in the dataset
It should be noted that the input vectors should benormalized to have zero mean and unity variance before
performing PCAThe normalization is a standard procedureDetails about PCA could be seen in [13]
3 SVM Classifier
The introduction of support vector machine (SVM) is alandmark of the field of machine learning [14] The advan-tages of SVMs include high accuracy elegant mathematicaltractability and direct geometric interpretation [15] Recentlymultiple improved SVMs have grown rapidly among whichthe kernel SVMs are the most popular and effective KernelSVMs have the following advantages [16] (1) work very wellin practice and have been remarkably successful in suchdiverse fields as natural language categorization bioinformat-ics and computer vision (2) have few tunable parametersand (3) training often employs convex quadratic optimization[17] Hence solutions are global and usually unique thusavoiding the convergence to local minima exhibited by otherstatistical learning systems such as neural networks
31 Principles of Linear SVMs Given a 119901-dimensional train-ing dataset of size 119873in the form
where 119910119899 is either minus1 or 1 corresponding to the class 1 or 2Each 119909119899 is a 119901-dimensional vector The maximum-marginhyperplane which divides class 1 from class 2 is the supportvector machine we want Considering that any hyperplanecan be written in the form of
wx minus 119887 = 0 (5)
where sdot denotes the dot product and w denotes the normalvector to the hyperplane We want to choose the w and119887 to maximize the margin between the two parallel (asshown in Figure 4) hyperplanes as large as possible while stillseparating the data So we define the two parallel hyperplanesby the equations as
wx minus 119887 = plusmn1 (6)
Therefore the task can be transformed to an optimizationproblem That is we want to maximize the distance betweenthe two parallel hyperplanes subject to prevent data fallinginto the margin Using simple mathematical knowledge theproblem can be finalized as
minw119887
w
st 119910119899 (w119909119899 minus 119887) ge 1 119899 = 1 119873
(7)
In practical situations the w is usually replaced by
minw119887
1
2
w2
st 119910119899 (w119909119899 minus 119887) ge 1 119899 = 1 119873
(8)
The reason leans upon the fact that w is involved in asquare root calculation After it is superseded with formula
4 The Scientific World Journal
wx minus b = minus1
wx minus b = 1
wx minus b = 0
Figure 4 The concept of parallel hyperplanes
(8) the solution will not change but the problem is alteredinto a quadratic programming optimization that is easy tosolve by using Lagrange multipliers and standard quadraticprogramming techniques and programs
32 Soft Margin However in practical applications theremay exist no hyperplane that can split the samples per-fectly In such case the ldquosoft marginrdquo method will choose ahyperplane that splits the given samples as clean as possiblewhile still maximizing the distance to the nearest cleanly splitsamples
Positive slack variables 120585119899 are introduced to measure themisclassification degree of sample 119909119899 (the distance betweenthe margin and the vectors 119909119899 that lying on the wrong sideof the margin) Then the optimal hyperplane separating thedata can be obtained by the following optimization problem
minw120585119887
1
2
w2+ 119862
119873
sum
119899=1
120585119899
st 119910119899 (w119909119899 minus 119887) ge 1 minus 120585119899
120585119899 ge 0
119899 = 1 119873
(9)
where 119862 is the error penalty Therefore the optimizationbecomes a tradeoff between a large margin and a small errorpenalty The constraint optimization problem can be solvedusing ldquoLagrange multiplierrdquo as
minw120585119887
max120572120573
1
2
w2+ 119862
119873
sum
119899=1
120585119899
minus
119873
sum
119899=1
120572119899 [119910119899 (w119909119899 minus 119887) minus 1 + 120585119899] minus
119873
sum
119899=1
120573119899120585119899
(10)
The min-max problem is not easy to solve so Cortes andVapnik proposed a dual form technique to solve it
33 Dual Form Thedual formof formula (9) can be designedas
The key advantage of the dual form function is that theslack variables 120585119899 vanish from the dual problem with theconstant 119862 appearing only as an additional constraint onthe Lagrange multipliers Now the optimization problem (11)becomes a quadratic programming (QP) problem which isdefined as the optimization of a quadratic function of severalvariables subject to linear constraints on these variablesTherefore numerous methods can solve formula (9) withinmilliseconds like interior point method active set methodaugmented Lagrangian method conjugate gradient methodsimplex algorithm and so forth
4 PSO-KSVM
41 Kernel SVMs Linear SVMs have the downside to linearhyperplane which cannot separate complicated distributedpractical data In order to generalize it to nonlinear hyper-plane the kernel trick is applied to SVMs [18] The resultingalgorithm is formally similar except that every dot product isreplaced by a nonlinear kernel function In another point ofview the KSVMs allow to fit the maximum-margin hyper-plane in a transformed feature space The transformationmay be nonlinear and the transformed space may be higherdimensional thus though the classifier is a hyperplane in thehigher-dimensional feature space it may be nonlinear in theoriginal input space For each kernel there should be at leastone adjusting parameter so as to make the kernel flexibleand tailor itself to practical data In this paper RBF kernel ischosen due to its excellent performanceThe kernel is writtenas
119896 (119909119898 119909119899) = exp(minus
1003817100381710038171003817119909119898 minus 119909119899
1003817100381710038171003817
21205902
) (12)
Put formula (12) into formula (11) and we got the final SVMtraining function as
1003817100381710038171003817119909119898 minus 119909119899
1003817100381710038171003817
21205902
)
st
0 le 120572119899 le 119862
119873
sum
119899=1
120572119899119910119899 = 0
119899 = 1 119873
(13)
It is still a quadratic programming problem and we choseinterior point method to solve the problem However thereis still an outstanding issue that is the value of parameters 119862and 120590 in (13)
The Scientific World Journal 5
42 PSO To determine the best parameter of 119862 and 120590traditionalmethod uses trial-and-errormethods It will causeheavy computation burden and cannot guarantee to findthe optimal or even near-optimal solutions Fei W [19] andChenglin et al [20] proposed to use PSO to optimize theparameters respectively and independently The PSO is apopulated global optimization method deriving from theresearch of the movement of bird flocking or fish schoolingIt is easy and fast to implement Besides we introduced inthe cross-validation to construct the fitness function used forPSO
PSO performs search via a swarm of particles which isupdated from iteration to iteration To seek for the optimalsolution each particle moves in the direction of its previouslybest position (119901best) and the best global position in the swarm(119892best) as follows
119901best119894 = 119901119894 (119896lowast)
st fitness (119901119894 (119896lowast)) = min119896=1119905
[fitness (119901119894 (119896))]
119892best = 119901119894lowast (119896lowast)
st fitness (119901119894lowast (119896lowast)) = min119894=1119875
119896=1119905
[fitness (119901119894 (119896))]
(14)
where 119894 denotes the particle index119875 denotes the total numberof particles 119896 denotes the iteration index and 119905 denotesthe current iteration number and 119901 denotes the positionThe velocity and position of particles can be updated by thefollowing equations
where V denotes the velocity The inertia weight 119908 is usedto balance the global exploration and local exploitation The1199031 and 1199032 are uniformly distributed random variables withinrange (0 1) The 1198881 and 1198882 are positive constant parameterscalled ldquoacceleration coefficientsrdquo Here the particle encodingis composed of the parameters 119862 and 120590 in (13)
43 Cross-Validation In this paper we choose 5-fold con-sidering the best compromise between computational costand reliable estimates The dataset is randomly divided into5 mutually exclusively subsets of approximately equal size inwhich 4 subsets are used as training set and the last subsetis used as validation set The abovementioned procedurerepeated 5 times so each subset is used once for validationThe fitness function of PSO chose the classification accuracyof the 5-fold cross-validation
Here 119910119904 and 119910119898 denote the number of successful classificationand misclassification respectively PSO is performed tomaximize the fitness function (classification accuracy)
44 Pseudocodes of Our Method In total our method can bedescribed as the following three stages and the flowchart isdepicted in Figure 5
ture reduction)Step 3 Fivefolded cross-validationStep 4 Determining the best parameter
Step 41 Initializing PSO The particles correspond to 119862
and 120590Step 42 For each particle 119894 computer the fitness values
Step 421 Decoding the particle to parameters 119862 and120590
Step 422 Using interior method to train KSVMaccording to (13)
Step 423 Calculating classification error accordingto (16) as the fitness values
Step 43 Updating the 119892best and 119901best according to (14)Step 44 Updating the velocity and position of each
particle according to (15)Step 45 If stopping criteria is met then jump to Step 46
otherwise return to Step 42Step 46 Decoding the optimal particle to corresponding
parameter 119862lowast and 120590lowast
Step 5 Constructing KSVM via the optimal 119862lowast and 120590
lowast
according to (13)Step 6 Submitting newMRI brains to the trained KSVM and
outputting the prediction
5 Experiments and Discussions
The experiments were carried out on the platform of P4IBM with 33GHz processor and 2GB RAM running underWindows XP operating system The algorithm was in-housedeveloped via the wavelet toolbox the biostatistical toolboxof 32 bitMATLAB 2012a (theMathWorks)The programs canbe run or tested on any computer platforms where MATLABis available
51 Database The datasets brain consists of 90 T2-weightedMR brain images in axial plane and 256 times 256 in-planeresolution which were downloaded from the website ofHarvard Medical School (URL httpwwwmedharvardeduaanlibhomehtml) The abnormal brain MR imagesof the dataset consist of the following diseases gliomametastatic adenocarcinoma metastatic bronchogenic carci-nomameningioma sarcoma Alzheimer Huntington motorneuron disease cerebral calcinosis Pickrsquos disease Alzheimerplus visual agnosia multiple sclerosis AIDS dementia Lymeencephalopathy herpes encephalitis Creutzfeld-Jakob dis-ease and cerebral toxoplasmosisThe samples of each diseaseare illustrated in Figure 6
6 The Scientific World Journal
MRIbrains
Featureextraction
Featurereduction
CandidateKSVM 1
DWT PCA
Used as trainingand validation set
Preprocessing
New MRIbrain
Normal orabnormal
Particle 1 Particle P
PSO
Particle 2
CandidateKSVM 2 middot middot middot
middot middot middot
middot middot middot
CandidateKSVM P
Fitness 1 Fitness 2 Fitness P
Update particle velocity and position
Are stoppingcriteria met
No
OptimalKSVM
Yes
Output
5-folded cross-validation
Update pbest and gbest
Figure 5 Methodology of our proposed PSO-KSVM algorithm
Figure 7 Illustration of 5-fold cross-validation of brain dataset(we divided the dataset into 5 groups and for each experiment 4groups were used for training and the rest one group was used forvalidation Each group was used once for validation)
We randomly selected 5 images for each type of brainSince there are 1 type of normal brain and 17 types ofabnormal brain in the dataset 5lowast(1 + 17) = 90 images wereselected to construct the brain dataset consisting of 5 normaland 85 abnormal brain images in total
The setting of the training images and validation imageswas shown in Figure 7 We divided the dataset into 5 equallydistributed groups each groups contain one normal brain
(a) (b)
Figure 8Theprocedures of 3-level 2DDWT (a) normal brainMRI(b) level 3 wavelet coefficients
and 17 abnormal brains Since 5-fold cross-validation wasused we would perform 5 experiments In each experiment4 groups were used for training and the left 1 group wasused for validation Each group was used once for validationIn total in this cross validation way 360 images were fortraining and 90 images were for validation
52 Feature Extraction The three levels of wavelet decom-position greatly reduce the input image size as shown inFigure 8 The top left corner of the wavelet coefficients imagedenotes for the approximation coefficients at level 3 of whichthe size is only 32times 32 = 1024The border distortion should beavoided In our algorithm symmetric padding method [21]was utilized to calculate the boundary value
53 Feature Reduction As stated above the extracted featureswere reduced from 65536 to 1024 by the DWT procedureHowever 1024 was still too large for calculation Thus PCAwas used to further reduce the dimensions of features The
8 The Scientific World Journal
Table 3 Parameters of comparison by random selection method (the final row corresponds to our proposed method)
Figure 9 The curve of variances against number of principlecomponents (here we found that 39 features can achieve 9502variances)
curve of cumulative sum of variance versus the number ofprinciple components was shown in Figure 9
The variances versus the number of principle componentsfrom 1 to 40 were listed in Table 1 It showed that only39 principle components (bold font in table) which wereonly 391024 = 381 of the original features could preserve9502 of total variance
54 Classification Accuracy The KSVM used the RBF asthe kernel function We compared our PSO-KVSM methodwith one hidden-layer Back Propagation-Neural Network(BP-NN) and RBF-Neural Network (RBF-NN) The resultswere shown in Table 2 It showed that BP-NN correctlymatched 388 cases with 8622 classification accuracy RBF-NN correctly matched 411 cases with 9133 classificationaccuracy Our PSO-KSVM correctly matched 440 brainimages with 9778 classification accuracy Therefore ourmethod had the most excellent classification performance
55 Parameter Selection The final parameters obtained byPSO were 119862 = 1433 and 120590 = 1132 We compared this casewith random selection method which randomly generatedthe values of 119862 in the range of (50 200) and 120590 in the rangeof [05 2] and then we compared them with the optimizedvalues by PSO (119862 = 1433 and 120590 = 1132) The resultsachieved by random selectionmethodwere shown in Table 3We saw that the classification accuracy variedwith the changeof parameters 120590 and 119862 so it was important to determine the
optimal values before constructing the classifierThe randomselection method was difficult to come across the best valuesso PSO was an effective method for this problem comparedto random selection method
6 Conclusions and Discussions
In this study we had developed a novel DWT + PCA + PSO-KSVM hybrid classification system to distinguish betweennormal and abnormalMRIs of the brainWepicked upRBF asthe kernel function of SVM The experiments demonstratedthat the PSO-KSVM method obtained 9778 classificationaccuracy on the 5-folded 90-image dataset higher than8622 of BP-NN and 9133 of RBF-NN
Future work should focus on the following four aspectsFirst the proposed SVM based method could be employedfor MR images with other contrast mechanisms such as T1-weighted proton density weighted and diffusion weightedimages Second the computation time could be acceleratedby using advanced wavelet transforms such as the lift-upwavelet Third Multiclassification which focuses on brainMRIs of specific disorders can also be explored Forth novelkernels will be tested to increase the classification accuracyand accelerate the algorithm
The DWT can efficiently extract the information fromoriginal MR images with litter loss The advantage of DWTover Fourier transforms is the spatial resolution namelyDWT captures both frequency and location information Inthis study we choose the Harr wavelet although there areother outstandingwavelets such asDaubechies seriesWewillcompare the performance of different families of wavelet infuture work Another research direction lies in the stationarywavelet transform and the wavelet packet transform
The importance of PCA was demonstrated in the Dis-cussion If we omitted the PCA procedures we meet ahuge feature space (1024 dimensions) which will cause heavycomputation burden and lowered the classification accuracyThere are some other excellent feature reduction methodssuch as ICA manifold learning In the future we will focuson investigating the performance of those algorithms
The reason we choose RBF kernel is that RBF takesthe form of exponential function which enlarge the sampledistances to the uttermost extent In the future we will try totest other kernel functions
The Scientific World Journal 9
The importance of introducing PSO is to determine theoptimal values of parameters119862 and120590 From random selectionmethod we found it is hard to get the optimal values at theparameter spaceTherefore the PSO is an effectiveway to findthe optimal values Integrating PSO to KSVM enhance theclassification capability of KSVM
The most important contribution of this paper is thepropose of a hybrid system integrating DWT PCA PSOKSVM and CV used for identifying normal MR brains fromabnormal MR brains It would be useful to help clinicians todiagnose the patients
References
[1] M Emin Tagluk M Akin and N Sezgin ldquoClassification ofsleep apnea by using wavelet transform and artificial neuralnetworksrdquo Expert Systems with Applications vol 37 no 2 pp1600ndash1607 2010
[2] J Camacho J Pico and A Ferrer ldquolsquoThe best approaches in theon-line monitoring of batch processes based on PCA does themodelling structure matterrsquo [Anal Chim Acta Volume 642(2009) 59ndash68]rdquo Analytica Chimica Acta vol 658 no 1 p 1062010
[3] S Chaplot LM Patnaik andN R Jagannathan ldquoClassificationof magnetic resonance brain images using wavelets as input tosupport vector machine and neural networkrdquo Biomedical SignalProcessing and Control vol 1 no 1 pp 86ndash92 2006
[4] C A Cocosco A P Zijdenbos and A C Evans ldquoA fullyautomatic and robust brain MRI tissue classification methodrdquoMedical Image Analysis vol 7 no 4 pp 513ndash527 2003
[5] J-Y Yeh and J C Fu ldquoA hierarchical genetic algorithm for seg-mentation of multi-spectral human-brainMRIrdquo Expert Systemswith Applications vol 34 no 2 pp 1285ndash1295 2008
[6] Y Zhang and L Wu ldquoClassification of fruits using computervision and a multiclass support vector machinerdquo Sensors vol12 no 9 pp 12489ndash12505 2012
[7] N S Patil P S Shelokar V K Jayaraman and B D KulkarnildquoRegression models using pattern search assisted least squaresupport vector machinesrdquo Chemical Engineering Research andDesign A vol 83 no 8 pp 1030ndash1037 2005
[8] D Li W Yang and S Wang ldquoClassification of foreign fibers incotton lint using machine vision and multi-class support vectormachinerdquo Computers and Electronics in Agriculture vol 74 no2 pp 274ndash279 2010
[9] T A F Gomes R B C Prudcncio C Soares A L DRossi and A Carvalho ldquoCombining meta-learning and searchtechniques to select parameters for support vector machinesrdquoNeurocomputing vol 75 pp 3ndash13 2012
[10] R Hable ldquoAsymptotic normality of support vector machinevariants and other regularized kernel methodsrdquo Journal ofMultivariate Analysis vol 106 pp 92ndash117 2012
[11] L Durak ldquoShift-invariance of short-time Fourier transform infractional Fourier domainsrdquo Journal of the Franklin Institutevol 346 no 2 pp 136ndash146 2009
[12] Y Zhang S Wang Y Huo L Wu and A Liu ldquoFeatureextraction of brain MRI by stationary wavelet transform and itsapplicationsrdquo Journal of Biological Systems vol 18 no 1 pp 115ndash132 2010
[13] Y Zhang L Wu and G Wei ldquoA new classifier for polarimetricSAR imagesrdquo Progress in Electromagnetics Research vol 94 pp83ndash104 2009
[15] P Martiskainen M Jarvinen J Skon J Tiirikainen MKolehmainen and J Mononen ldquoCow behaviour pattern recog-nition using a three-dimensional accelerometer and supportvectormachinesrdquoAppliedAnimal Behaviour Science vol 119 no1-2 pp 32ndash38 2009
[16] S Bermejo B Monegal and J Cabestany ldquoFish age catego-rization from otolith images using multi-class support vectormachinesrdquo Fisheries Research vol 84 no 2 pp 247ndash253 2007
[17] A M S Muniz H Liu K E Lyons et al ldquoComparisonamong probabilistic neural network support vector machineand logistic regression for evaluating the effect of subthalamicstimulation in Parkinson disease on ground reaction forceduring gaitrdquo Journal of Biomechanics vol 43 no 4 pp 720ndash7262010
[18] J Acevedo-Rodrıguez S Maldonado-Bascon S Lafuente-Arroyo P Siegmann and F Lopez-Ferreras ldquoComputationalload reduction in decision functions using support vectormachinesrdquo Signal Processing vol 89 no 10 pp 2066ndash20712009
[19] S-W Fei ldquoDiagnostic study on arrhythmia cordis based onparticle swarm optimization-based support vector machinerdquoExpert Systems with Applications vol 37 no 10 pp 6748ndash67522010
[20] Z Chenglin S Xuebin S Songlin and J Ting ldquoFault diagnosisof sensor by chaos particle swarm optimization algorithm andsupport vector machinerdquo Expert Systems with Applications vol38 no 8 pp 9908ndash9912 2011
[21] A Messina ldquoRefinements of damage detection methods basedon wavelet analysis of dynamical shapesrdquo International Journalof Solids and Structures vol 45 no 14-15 pp 4068ndash4097 2008
network SVMs have significant advantages of high accuracyelegant mathematical tractability and direct geometric inter-pretation Besides it does not need a large number of trainingsamples to avoid overfitting [8]
Original SVMs are linear classifiers In this paper weintroduced in the kernel SVMs (KSVMs) which extendsoriginal linear SVMs to nonlinear SVM classifiers by apply-ing the kernel function to replace the dot product formin the original SVMs [9] The KSVMs is allowed to fitthe maximum-margin hyperplane in a transformed featurespace The transformation may be nonlinear and the trans-formed space may be high dimensional thus though theclassifier is a hyperplane in the high-dimensional featurespace it may be nonlinear in the original input space [10]
The structure of the rest of this paperwas organized as fol-lows Section 2 gave the detailed procedures of preprocessingincluding the discrete wavelet transform (DWT) and princi-pal component analysis (PCA) Section 3 first introduced themotivation and principles of linear SVM and then extendedit to softmargin dual from Section 4 introduced themethodof PSO-KSVM It first gave the principles of KSVM and thenused the particle swarm optimization algorithm to optimizethe values of parameters 119862 and 120590 finally it used 119870-foldcross-validation to protect the classifier from overfittingThe pseudocodes and flowchart were listed Experiments inSection 5 created a dataset brain of 90 brain MR images andshowed the results of each step We compared our proposedPSO-KSVM method with traditional BP-NN and RBF-NNmethods Final Section 6 was devoted to conclusions anddiscussions
2 Preprocessing
21 Feature Extraction The most conventional tool of signalanalysis is Fourier transform (FT) which breaks down atime domain signal into constituent sinusoids of differentfrequencies thus transforming the signal from time domainto frequency domain However FT has a serious drawback asdiscarding the time information of the signal For exampleanalyst cannot tell when a particular event took place from aFourier spectrumThus the classification will decrease as thetime information is lost
Gabor adapted the FT to analyze only a small section ofthe signal at a time The technique is called windowing orshort-time Fourier transform (STFT) [11] It adds a windowof particular shape to the signal STFT can be regarded asa compromise between the time information and frequencyinformation It provides some information about both timeand frequency domain However the precision of the infor-mation is limited by the size of the window
Wavelet transform (WT) represents the next logical stepa windowing technique with variable size Thus it preservesboth time and frequency information of the signal Thedevelopment of signal analysis is shown in Figure 1
Another advantage of WT is that it adopts ldquoscalerdquo insteadof traditional ldquofrequencyrdquo namely it does not produce a time-frequency view but a time-scale view of the signal The time-scale view is a different way to view data but it is a morenatural and powerful way
Fouriertransform
Short-time-fourier
-transformWavelet
transform
Am
plitu
de
Freq
uenc
y
Frequency Time
Scal
e
Time
Figure 1 The development of signal analysis
22 Discrete Wavelet Transform The discrete wavelet trans-form (DWT) is a powerful implementation of the WT usingthe dyadic scales and positions The basic fundamental ofDWT is introduced as follows Suppose that 119909(119905) is a square-integrable function then the continuous WT of 119909(119905) relativeto a given wavelet 120595(119905) is defined as
Here the wavelet 120595119886119887(119905) is calculated from the motherwavelet 120595(119905) by translation and dilation 119886 is the dilationfactor and 119887 is the translation parameter (both real positivenumbers) There are several different kinds of waveletswhich have gained popularity throughout the developmentof wavelet analysis The most important wavelet is the Harrwavelet which is the simplest one and often the preferredwavelet in a lot of applications
Equation (1) can be discretized by restraining 119886 and 119887 to adiscrete lattice (119886 = 2
119887 amp 119886 gt 0) to give the DWT which canbe expressed as follows
ca119895119896 (119899) = DS[sum
119899
119909 (119899) 119892lowast
119895(119899 minus 2
119895119896)]
cd119895119896 (119899) = DS [sum
119899
119909 (119899) ℎlowast
119895(119899 minus 2
119895119896)]
(3)
Here ca119895119896 and cd119895119896 refer to the coefficients of the approx-imation components and the detail components respec-tively 119892(119899) and ℎ(119899) denote the low-pass filter and high-pass filter respectively 119895 and 119896 represent the wavelet scaleand translation factors respectively DS operator means thedownsampling
The above decomposition process can be iterated withsuccessive approximations being decomposed in turn so thatone signal is broken down into various levels of resolutionThe whole process is called wavelet decomposition treeshown in Figure 2
23 2D DWT In case of 2D images the DWT is applied toeach dimension separately Figure 3 illustrates the schematic
The Scientific World Journal 3
S
ca1 cd1
ca2 cd2
ca3 cd3
Figure 2 A 3-level wavelet decomposition tree
Image
g(n)
h(n)
darr
darr
darr
darr
darr
darr
g(n)
h(n)
g(n)
h(n)
LL
LH
HL
HH
Subband
Figure 3 Schematic diagram of 2D DWT
diagram of 2DDWTAs a result there are 4 subband (LL LHHH and HL) images at each scale The sub-band LL is usedfor the next 2D DWT
The LL subband can be regarded as the approximationcomponent of the image while the LHHL andHHsubbandscan be regarded as the detailed components of the image Asthe level of decomposition increased compacter but coarserapproximation component was obtainedThus wavelets pro-vide a simple hierarchical framework for interpreting theimage information In our algorithm level 3 decompositionvia Harr wavelet was utilized to extract features
24 Feature Reduction Excessive features increase compu-tation times and storage memory Furthermore they some-times make classification more complicated which is calledthe curse of dimensionality It is required to reduce thenumber of features [12]
PCA is an efficient tool to reduce the dimension of adata set consisting of a large number of interrelated variableswhile retaining most of the variations It is achieved bytransforming the data set to a new set of ordered variablesaccording to their variances or importance This techniquehas three effects it orthogonalizes the components of theinput vectors so that it uncorrelated with each other itorders the resulting orthogonal components so that thosewith the largest variation come first and it eliminates thosecomponents contributing the least to the variation in the dataset
It should be noted that the input vectors should benormalized to have zero mean and unity variance before
performing PCAThe normalization is a standard procedureDetails about PCA could be seen in [13]
3 SVM Classifier
The introduction of support vector machine (SVM) is alandmark of the field of machine learning [14] The advan-tages of SVMs include high accuracy elegant mathematicaltractability and direct geometric interpretation [15] Recentlymultiple improved SVMs have grown rapidly among whichthe kernel SVMs are the most popular and effective KernelSVMs have the following advantages [16] (1) work very wellin practice and have been remarkably successful in suchdiverse fields as natural language categorization bioinformat-ics and computer vision (2) have few tunable parametersand (3) training often employs convex quadratic optimization[17] Hence solutions are global and usually unique thusavoiding the convergence to local minima exhibited by otherstatistical learning systems such as neural networks
31 Principles of Linear SVMs Given a 119901-dimensional train-ing dataset of size 119873in the form
where 119910119899 is either minus1 or 1 corresponding to the class 1 or 2Each 119909119899 is a 119901-dimensional vector The maximum-marginhyperplane which divides class 1 from class 2 is the supportvector machine we want Considering that any hyperplanecan be written in the form of
wx minus 119887 = 0 (5)
where sdot denotes the dot product and w denotes the normalvector to the hyperplane We want to choose the w and119887 to maximize the margin between the two parallel (asshown in Figure 4) hyperplanes as large as possible while stillseparating the data So we define the two parallel hyperplanesby the equations as
wx minus 119887 = plusmn1 (6)
Therefore the task can be transformed to an optimizationproblem That is we want to maximize the distance betweenthe two parallel hyperplanes subject to prevent data fallinginto the margin Using simple mathematical knowledge theproblem can be finalized as
minw119887
w
st 119910119899 (w119909119899 minus 119887) ge 1 119899 = 1 119873
(7)
In practical situations the w is usually replaced by
minw119887
1
2
w2
st 119910119899 (w119909119899 minus 119887) ge 1 119899 = 1 119873
(8)
The reason leans upon the fact that w is involved in asquare root calculation After it is superseded with formula
4 The Scientific World Journal
wx minus b = minus1
wx minus b = 1
wx minus b = 0
Figure 4 The concept of parallel hyperplanes
(8) the solution will not change but the problem is alteredinto a quadratic programming optimization that is easy tosolve by using Lagrange multipliers and standard quadraticprogramming techniques and programs
32 Soft Margin However in practical applications theremay exist no hyperplane that can split the samples per-fectly In such case the ldquosoft marginrdquo method will choose ahyperplane that splits the given samples as clean as possiblewhile still maximizing the distance to the nearest cleanly splitsamples
Positive slack variables 120585119899 are introduced to measure themisclassification degree of sample 119909119899 (the distance betweenthe margin and the vectors 119909119899 that lying on the wrong sideof the margin) Then the optimal hyperplane separating thedata can be obtained by the following optimization problem
minw120585119887
1
2
w2+ 119862
119873
sum
119899=1
120585119899
st 119910119899 (w119909119899 minus 119887) ge 1 minus 120585119899
120585119899 ge 0
119899 = 1 119873
(9)
where 119862 is the error penalty Therefore the optimizationbecomes a tradeoff between a large margin and a small errorpenalty The constraint optimization problem can be solvedusing ldquoLagrange multiplierrdquo as
minw120585119887
max120572120573
1
2
w2+ 119862
119873
sum
119899=1
120585119899
minus
119873
sum
119899=1
120572119899 [119910119899 (w119909119899 minus 119887) minus 1 + 120585119899] minus
119873
sum
119899=1
120573119899120585119899
(10)
The min-max problem is not easy to solve so Cortes andVapnik proposed a dual form technique to solve it
33 Dual Form Thedual formof formula (9) can be designedas
The key advantage of the dual form function is that theslack variables 120585119899 vanish from the dual problem with theconstant 119862 appearing only as an additional constraint onthe Lagrange multipliers Now the optimization problem (11)becomes a quadratic programming (QP) problem which isdefined as the optimization of a quadratic function of severalvariables subject to linear constraints on these variablesTherefore numerous methods can solve formula (9) withinmilliseconds like interior point method active set methodaugmented Lagrangian method conjugate gradient methodsimplex algorithm and so forth
4 PSO-KSVM
41 Kernel SVMs Linear SVMs have the downside to linearhyperplane which cannot separate complicated distributedpractical data In order to generalize it to nonlinear hyper-plane the kernel trick is applied to SVMs [18] The resultingalgorithm is formally similar except that every dot product isreplaced by a nonlinear kernel function In another point ofview the KSVMs allow to fit the maximum-margin hyper-plane in a transformed feature space The transformationmay be nonlinear and the transformed space may be higherdimensional thus though the classifier is a hyperplane in thehigher-dimensional feature space it may be nonlinear in theoriginal input space For each kernel there should be at leastone adjusting parameter so as to make the kernel flexibleand tailor itself to practical data In this paper RBF kernel ischosen due to its excellent performanceThe kernel is writtenas
119896 (119909119898 119909119899) = exp(minus
1003817100381710038171003817119909119898 minus 119909119899
1003817100381710038171003817
21205902
) (12)
Put formula (12) into formula (11) and we got the final SVMtraining function as
1003817100381710038171003817119909119898 minus 119909119899
1003817100381710038171003817
21205902
)
st
0 le 120572119899 le 119862
119873
sum
119899=1
120572119899119910119899 = 0
119899 = 1 119873
(13)
It is still a quadratic programming problem and we choseinterior point method to solve the problem However thereis still an outstanding issue that is the value of parameters 119862and 120590 in (13)
The Scientific World Journal 5
42 PSO To determine the best parameter of 119862 and 120590traditionalmethod uses trial-and-errormethods It will causeheavy computation burden and cannot guarantee to findthe optimal or even near-optimal solutions Fei W [19] andChenglin et al [20] proposed to use PSO to optimize theparameters respectively and independently The PSO is apopulated global optimization method deriving from theresearch of the movement of bird flocking or fish schoolingIt is easy and fast to implement Besides we introduced inthe cross-validation to construct the fitness function used forPSO
PSO performs search via a swarm of particles which isupdated from iteration to iteration To seek for the optimalsolution each particle moves in the direction of its previouslybest position (119901best) and the best global position in the swarm(119892best) as follows
119901best119894 = 119901119894 (119896lowast)
st fitness (119901119894 (119896lowast)) = min119896=1119905
[fitness (119901119894 (119896))]
119892best = 119901119894lowast (119896lowast)
st fitness (119901119894lowast (119896lowast)) = min119894=1119875
119896=1119905
[fitness (119901119894 (119896))]
(14)
where 119894 denotes the particle index119875 denotes the total numberof particles 119896 denotes the iteration index and 119905 denotesthe current iteration number and 119901 denotes the positionThe velocity and position of particles can be updated by thefollowing equations
where V denotes the velocity The inertia weight 119908 is usedto balance the global exploration and local exploitation The1199031 and 1199032 are uniformly distributed random variables withinrange (0 1) The 1198881 and 1198882 are positive constant parameterscalled ldquoacceleration coefficientsrdquo Here the particle encodingis composed of the parameters 119862 and 120590 in (13)
43 Cross-Validation In this paper we choose 5-fold con-sidering the best compromise between computational costand reliable estimates The dataset is randomly divided into5 mutually exclusively subsets of approximately equal size inwhich 4 subsets are used as training set and the last subsetis used as validation set The abovementioned procedurerepeated 5 times so each subset is used once for validationThe fitness function of PSO chose the classification accuracyof the 5-fold cross-validation
Here 119910119904 and 119910119898 denote the number of successful classificationand misclassification respectively PSO is performed tomaximize the fitness function (classification accuracy)
44 Pseudocodes of Our Method In total our method can bedescribed as the following three stages and the flowchart isdepicted in Figure 5
ture reduction)Step 3 Fivefolded cross-validationStep 4 Determining the best parameter
Step 41 Initializing PSO The particles correspond to 119862
and 120590Step 42 For each particle 119894 computer the fitness values
Step 421 Decoding the particle to parameters 119862 and120590
Step 422 Using interior method to train KSVMaccording to (13)
Step 423 Calculating classification error accordingto (16) as the fitness values
Step 43 Updating the 119892best and 119901best according to (14)Step 44 Updating the velocity and position of each
particle according to (15)Step 45 If stopping criteria is met then jump to Step 46
otherwise return to Step 42Step 46 Decoding the optimal particle to corresponding
parameter 119862lowast and 120590lowast
Step 5 Constructing KSVM via the optimal 119862lowast and 120590
lowast
according to (13)Step 6 Submitting newMRI brains to the trained KSVM and
outputting the prediction
5 Experiments and Discussions
The experiments were carried out on the platform of P4IBM with 33GHz processor and 2GB RAM running underWindows XP operating system The algorithm was in-housedeveloped via the wavelet toolbox the biostatistical toolboxof 32 bitMATLAB 2012a (theMathWorks)The programs canbe run or tested on any computer platforms where MATLABis available
51 Database The datasets brain consists of 90 T2-weightedMR brain images in axial plane and 256 times 256 in-planeresolution which were downloaded from the website ofHarvard Medical School (URL httpwwwmedharvardeduaanlibhomehtml) The abnormal brain MR imagesof the dataset consist of the following diseases gliomametastatic adenocarcinoma metastatic bronchogenic carci-nomameningioma sarcoma Alzheimer Huntington motorneuron disease cerebral calcinosis Pickrsquos disease Alzheimerplus visual agnosia multiple sclerosis AIDS dementia Lymeencephalopathy herpes encephalitis Creutzfeld-Jakob dis-ease and cerebral toxoplasmosisThe samples of each diseaseare illustrated in Figure 6
6 The Scientific World Journal
MRIbrains
Featureextraction
Featurereduction
CandidateKSVM 1
DWT PCA
Used as trainingand validation set
Preprocessing
New MRIbrain
Normal orabnormal
Particle 1 Particle P
PSO
Particle 2
CandidateKSVM 2 middot middot middot
middot middot middot
middot middot middot
CandidateKSVM P
Fitness 1 Fitness 2 Fitness P
Update particle velocity and position
Are stoppingcriteria met
No
OptimalKSVM
Yes
Output
5-folded cross-validation
Update pbest and gbest
Figure 5 Methodology of our proposed PSO-KSVM algorithm
Figure 7 Illustration of 5-fold cross-validation of brain dataset(we divided the dataset into 5 groups and for each experiment 4groups were used for training and the rest one group was used forvalidation Each group was used once for validation)
We randomly selected 5 images for each type of brainSince there are 1 type of normal brain and 17 types ofabnormal brain in the dataset 5lowast(1 + 17) = 90 images wereselected to construct the brain dataset consisting of 5 normaland 85 abnormal brain images in total
The setting of the training images and validation imageswas shown in Figure 7 We divided the dataset into 5 equallydistributed groups each groups contain one normal brain
(a) (b)
Figure 8Theprocedures of 3-level 2DDWT (a) normal brainMRI(b) level 3 wavelet coefficients
and 17 abnormal brains Since 5-fold cross-validation wasused we would perform 5 experiments In each experiment4 groups were used for training and the left 1 group wasused for validation Each group was used once for validationIn total in this cross validation way 360 images were fortraining and 90 images were for validation
52 Feature Extraction The three levels of wavelet decom-position greatly reduce the input image size as shown inFigure 8 The top left corner of the wavelet coefficients imagedenotes for the approximation coefficients at level 3 of whichthe size is only 32times 32 = 1024The border distortion should beavoided In our algorithm symmetric padding method [21]was utilized to calculate the boundary value
53 Feature Reduction As stated above the extracted featureswere reduced from 65536 to 1024 by the DWT procedureHowever 1024 was still too large for calculation Thus PCAwas used to further reduce the dimensions of features The
8 The Scientific World Journal
Table 3 Parameters of comparison by random selection method (the final row corresponds to our proposed method)
Figure 9 The curve of variances against number of principlecomponents (here we found that 39 features can achieve 9502variances)
curve of cumulative sum of variance versus the number ofprinciple components was shown in Figure 9
The variances versus the number of principle componentsfrom 1 to 40 were listed in Table 1 It showed that only39 principle components (bold font in table) which wereonly 391024 = 381 of the original features could preserve9502 of total variance
54 Classification Accuracy The KSVM used the RBF asthe kernel function We compared our PSO-KVSM methodwith one hidden-layer Back Propagation-Neural Network(BP-NN) and RBF-Neural Network (RBF-NN) The resultswere shown in Table 2 It showed that BP-NN correctlymatched 388 cases with 8622 classification accuracy RBF-NN correctly matched 411 cases with 9133 classificationaccuracy Our PSO-KSVM correctly matched 440 brainimages with 9778 classification accuracy Therefore ourmethod had the most excellent classification performance
55 Parameter Selection The final parameters obtained byPSO were 119862 = 1433 and 120590 = 1132 We compared this casewith random selection method which randomly generatedthe values of 119862 in the range of (50 200) and 120590 in the rangeof [05 2] and then we compared them with the optimizedvalues by PSO (119862 = 1433 and 120590 = 1132) The resultsachieved by random selectionmethodwere shown in Table 3We saw that the classification accuracy variedwith the changeof parameters 120590 and 119862 so it was important to determine the
optimal values before constructing the classifierThe randomselection method was difficult to come across the best valuesso PSO was an effective method for this problem comparedto random selection method
6 Conclusions and Discussions
In this study we had developed a novel DWT + PCA + PSO-KSVM hybrid classification system to distinguish betweennormal and abnormalMRIs of the brainWepicked upRBF asthe kernel function of SVM The experiments demonstratedthat the PSO-KSVM method obtained 9778 classificationaccuracy on the 5-folded 90-image dataset higher than8622 of BP-NN and 9133 of RBF-NN
Future work should focus on the following four aspectsFirst the proposed SVM based method could be employedfor MR images with other contrast mechanisms such as T1-weighted proton density weighted and diffusion weightedimages Second the computation time could be acceleratedby using advanced wavelet transforms such as the lift-upwavelet Third Multiclassification which focuses on brainMRIs of specific disorders can also be explored Forth novelkernels will be tested to increase the classification accuracyand accelerate the algorithm
The DWT can efficiently extract the information fromoriginal MR images with litter loss The advantage of DWTover Fourier transforms is the spatial resolution namelyDWT captures both frequency and location information Inthis study we choose the Harr wavelet although there areother outstandingwavelets such asDaubechies seriesWewillcompare the performance of different families of wavelet infuture work Another research direction lies in the stationarywavelet transform and the wavelet packet transform
The importance of PCA was demonstrated in the Dis-cussion If we omitted the PCA procedures we meet ahuge feature space (1024 dimensions) which will cause heavycomputation burden and lowered the classification accuracyThere are some other excellent feature reduction methodssuch as ICA manifold learning In the future we will focuson investigating the performance of those algorithms
The reason we choose RBF kernel is that RBF takesthe form of exponential function which enlarge the sampledistances to the uttermost extent In the future we will try totest other kernel functions
The Scientific World Journal 9
The importance of introducing PSO is to determine theoptimal values of parameters119862 and120590 From random selectionmethod we found it is hard to get the optimal values at theparameter spaceTherefore the PSO is an effectiveway to findthe optimal values Integrating PSO to KSVM enhance theclassification capability of KSVM
The most important contribution of this paper is thepropose of a hybrid system integrating DWT PCA PSOKSVM and CV used for identifying normal MR brains fromabnormal MR brains It would be useful to help clinicians todiagnose the patients
References
[1] M Emin Tagluk M Akin and N Sezgin ldquoClassification ofsleep apnea by using wavelet transform and artificial neuralnetworksrdquo Expert Systems with Applications vol 37 no 2 pp1600ndash1607 2010
[2] J Camacho J Pico and A Ferrer ldquolsquoThe best approaches in theon-line monitoring of batch processes based on PCA does themodelling structure matterrsquo [Anal Chim Acta Volume 642(2009) 59ndash68]rdquo Analytica Chimica Acta vol 658 no 1 p 1062010
[3] S Chaplot LM Patnaik andN R Jagannathan ldquoClassificationof magnetic resonance brain images using wavelets as input tosupport vector machine and neural networkrdquo Biomedical SignalProcessing and Control vol 1 no 1 pp 86ndash92 2006
[4] C A Cocosco A P Zijdenbos and A C Evans ldquoA fullyautomatic and robust brain MRI tissue classification methodrdquoMedical Image Analysis vol 7 no 4 pp 513ndash527 2003
[5] J-Y Yeh and J C Fu ldquoA hierarchical genetic algorithm for seg-mentation of multi-spectral human-brainMRIrdquo Expert Systemswith Applications vol 34 no 2 pp 1285ndash1295 2008
[6] Y Zhang and L Wu ldquoClassification of fruits using computervision and a multiclass support vector machinerdquo Sensors vol12 no 9 pp 12489ndash12505 2012
[7] N S Patil P S Shelokar V K Jayaraman and B D KulkarnildquoRegression models using pattern search assisted least squaresupport vector machinesrdquo Chemical Engineering Research andDesign A vol 83 no 8 pp 1030ndash1037 2005
[8] D Li W Yang and S Wang ldquoClassification of foreign fibers incotton lint using machine vision and multi-class support vectormachinerdquo Computers and Electronics in Agriculture vol 74 no2 pp 274ndash279 2010
[9] T A F Gomes R B C Prudcncio C Soares A L DRossi and A Carvalho ldquoCombining meta-learning and searchtechniques to select parameters for support vector machinesrdquoNeurocomputing vol 75 pp 3ndash13 2012
[10] R Hable ldquoAsymptotic normality of support vector machinevariants and other regularized kernel methodsrdquo Journal ofMultivariate Analysis vol 106 pp 92ndash117 2012
[11] L Durak ldquoShift-invariance of short-time Fourier transform infractional Fourier domainsrdquo Journal of the Franklin Institutevol 346 no 2 pp 136ndash146 2009
[12] Y Zhang S Wang Y Huo L Wu and A Liu ldquoFeatureextraction of brain MRI by stationary wavelet transform and itsapplicationsrdquo Journal of Biological Systems vol 18 no 1 pp 115ndash132 2010
[13] Y Zhang L Wu and G Wei ldquoA new classifier for polarimetricSAR imagesrdquo Progress in Electromagnetics Research vol 94 pp83ndash104 2009
[15] P Martiskainen M Jarvinen J Skon J Tiirikainen MKolehmainen and J Mononen ldquoCow behaviour pattern recog-nition using a three-dimensional accelerometer and supportvectormachinesrdquoAppliedAnimal Behaviour Science vol 119 no1-2 pp 32ndash38 2009
[16] S Bermejo B Monegal and J Cabestany ldquoFish age catego-rization from otolith images using multi-class support vectormachinesrdquo Fisheries Research vol 84 no 2 pp 247ndash253 2007
[17] A M S Muniz H Liu K E Lyons et al ldquoComparisonamong probabilistic neural network support vector machineand logistic regression for evaluating the effect of subthalamicstimulation in Parkinson disease on ground reaction forceduring gaitrdquo Journal of Biomechanics vol 43 no 4 pp 720ndash7262010
[18] J Acevedo-Rodrıguez S Maldonado-Bascon S Lafuente-Arroyo P Siegmann and F Lopez-Ferreras ldquoComputationalload reduction in decision functions using support vectormachinesrdquo Signal Processing vol 89 no 10 pp 2066ndash20712009
[19] S-W Fei ldquoDiagnostic study on arrhythmia cordis based onparticle swarm optimization-based support vector machinerdquoExpert Systems with Applications vol 37 no 10 pp 6748ndash67522010
[20] Z Chenglin S Xuebin S Songlin and J Ting ldquoFault diagnosisof sensor by chaos particle swarm optimization algorithm andsupport vector machinerdquo Expert Systems with Applications vol38 no 8 pp 9908ndash9912 2011
[21] A Messina ldquoRefinements of damage detection methods basedon wavelet analysis of dynamical shapesrdquo International Journalof Solids and Structures vol 45 no 14-15 pp 4068ndash4097 2008
diagram of 2DDWTAs a result there are 4 subband (LL LHHH and HL) images at each scale The sub-band LL is usedfor the next 2D DWT
The LL subband can be regarded as the approximationcomponent of the image while the LHHL andHHsubbandscan be regarded as the detailed components of the image Asthe level of decomposition increased compacter but coarserapproximation component was obtainedThus wavelets pro-vide a simple hierarchical framework for interpreting theimage information In our algorithm level 3 decompositionvia Harr wavelet was utilized to extract features
24 Feature Reduction Excessive features increase compu-tation times and storage memory Furthermore they some-times make classification more complicated which is calledthe curse of dimensionality It is required to reduce thenumber of features [12]
PCA is an efficient tool to reduce the dimension of adata set consisting of a large number of interrelated variableswhile retaining most of the variations It is achieved bytransforming the data set to a new set of ordered variablesaccording to their variances or importance This techniquehas three effects it orthogonalizes the components of theinput vectors so that it uncorrelated with each other itorders the resulting orthogonal components so that thosewith the largest variation come first and it eliminates thosecomponents contributing the least to the variation in the dataset
It should be noted that the input vectors should benormalized to have zero mean and unity variance before
performing PCAThe normalization is a standard procedureDetails about PCA could be seen in [13]
3 SVM Classifier
The introduction of support vector machine (SVM) is alandmark of the field of machine learning [14] The advan-tages of SVMs include high accuracy elegant mathematicaltractability and direct geometric interpretation [15] Recentlymultiple improved SVMs have grown rapidly among whichthe kernel SVMs are the most popular and effective KernelSVMs have the following advantages [16] (1) work very wellin practice and have been remarkably successful in suchdiverse fields as natural language categorization bioinformat-ics and computer vision (2) have few tunable parametersand (3) training often employs convex quadratic optimization[17] Hence solutions are global and usually unique thusavoiding the convergence to local minima exhibited by otherstatistical learning systems such as neural networks
31 Principles of Linear SVMs Given a 119901-dimensional train-ing dataset of size 119873in the form
where 119910119899 is either minus1 or 1 corresponding to the class 1 or 2Each 119909119899 is a 119901-dimensional vector The maximum-marginhyperplane which divides class 1 from class 2 is the supportvector machine we want Considering that any hyperplanecan be written in the form of
wx minus 119887 = 0 (5)
where sdot denotes the dot product and w denotes the normalvector to the hyperplane We want to choose the w and119887 to maximize the margin between the two parallel (asshown in Figure 4) hyperplanes as large as possible while stillseparating the data So we define the two parallel hyperplanesby the equations as
wx minus 119887 = plusmn1 (6)
Therefore the task can be transformed to an optimizationproblem That is we want to maximize the distance betweenthe two parallel hyperplanes subject to prevent data fallinginto the margin Using simple mathematical knowledge theproblem can be finalized as
minw119887
w
st 119910119899 (w119909119899 minus 119887) ge 1 119899 = 1 119873
(7)
In practical situations the w is usually replaced by
minw119887
1
2
w2
st 119910119899 (w119909119899 minus 119887) ge 1 119899 = 1 119873
(8)
The reason leans upon the fact that w is involved in asquare root calculation After it is superseded with formula
4 The Scientific World Journal
wx minus b = minus1
wx minus b = 1
wx minus b = 0
Figure 4 The concept of parallel hyperplanes
(8) the solution will not change but the problem is alteredinto a quadratic programming optimization that is easy tosolve by using Lagrange multipliers and standard quadraticprogramming techniques and programs
32 Soft Margin However in practical applications theremay exist no hyperplane that can split the samples per-fectly In such case the ldquosoft marginrdquo method will choose ahyperplane that splits the given samples as clean as possiblewhile still maximizing the distance to the nearest cleanly splitsamples
Positive slack variables 120585119899 are introduced to measure themisclassification degree of sample 119909119899 (the distance betweenthe margin and the vectors 119909119899 that lying on the wrong sideof the margin) Then the optimal hyperplane separating thedata can be obtained by the following optimization problem
minw120585119887
1
2
w2+ 119862
119873
sum
119899=1
120585119899
st 119910119899 (w119909119899 minus 119887) ge 1 minus 120585119899
120585119899 ge 0
119899 = 1 119873
(9)
where 119862 is the error penalty Therefore the optimizationbecomes a tradeoff between a large margin and a small errorpenalty The constraint optimization problem can be solvedusing ldquoLagrange multiplierrdquo as
minw120585119887
max120572120573
1
2
w2+ 119862
119873
sum
119899=1
120585119899
minus
119873
sum
119899=1
120572119899 [119910119899 (w119909119899 minus 119887) minus 1 + 120585119899] minus
119873
sum
119899=1
120573119899120585119899
(10)
The min-max problem is not easy to solve so Cortes andVapnik proposed a dual form technique to solve it
33 Dual Form Thedual formof formula (9) can be designedas
The key advantage of the dual form function is that theslack variables 120585119899 vanish from the dual problem with theconstant 119862 appearing only as an additional constraint onthe Lagrange multipliers Now the optimization problem (11)becomes a quadratic programming (QP) problem which isdefined as the optimization of a quadratic function of severalvariables subject to linear constraints on these variablesTherefore numerous methods can solve formula (9) withinmilliseconds like interior point method active set methodaugmented Lagrangian method conjugate gradient methodsimplex algorithm and so forth
4 PSO-KSVM
41 Kernel SVMs Linear SVMs have the downside to linearhyperplane which cannot separate complicated distributedpractical data In order to generalize it to nonlinear hyper-plane the kernel trick is applied to SVMs [18] The resultingalgorithm is formally similar except that every dot product isreplaced by a nonlinear kernel function In another point ofview the KSVMs allow to fit the maximum-margin hyper-plane in a transformed feature space The transformationmay be nonlinear and the transformed space may be higherdimensional thus though the classifier is a hyperplane in thehigher-dimensional feature space it may be nonlinear in theoriginal input space For each kernel there should be at leastone adjusting parameter so as to make the kernel flexibleand tailor itself to practical data In this paper RBF kernel ischosen due to its excellent performanceThe kernel is writtenas
119896 (119909119898 119909119899) = exp(minus
1003817100381710038171003817119909119898 minus 119909119899
1003817100381710038171003817
21205902
) (12)
Put formula (12) into formula (11) and we got the final SVMtraining function as
1003817100381710038171003817119909119898 minus 119909119899
1003817100381710038171003817
21205902
)
st
0 le 120572119899 le 119862
119873
sum
119899=1
120572119899119910119899 = 0
119899 = 1 119873
(13)
It is still a quadratic programming problem and we choseinterior point method to solve the problem However thereis still an outstanding issue that is the value of parameters 119862and 120590 in (13)
The Scientific World Journal 5
42 PSO To determine the best parameter of 119862 and 120590traditionalmethod uses trial-and-errormethods It will causeheavy computation burden and cannot guarantee to findthe optimal or even near-optimal solutions Fei W [19] andChenglin et al [20] proposed to use PSO to optimize theparameters respectively and independently The PSO is apopulated global optimization method deriving from theresearch of the movement of bird flocking or fish schoolingIt is easy and fast to implement Besides we introduced inthe cross-validation to construct the fitness function used forPSO
PSO performs search via a swarm of particles which isupdated from iteration to iteration To seek for the optimalsolution each particle moves in the direction of its previouslybest position (119901best) and the best global position in the swarm(119892best) as follows
119901best119894 = 119901119894 (119896lowast)
st fitness (119901119894 (119896lowast)) = min119896=1119905
[fitness (119901119894 (119896))]
119892best = 119901119894lowast (119896lowast)
st fitness (119901119894lowast (119896lowast)) = min119894=1119875
119896=1119905
[fitness (119901119894 (119896))]
(14)
where 119894 denotes the particle index119875 denotes the total numberof particles 119896 denotes the iteration index and 119905 denotesthe current iteration number and 119901 denotes the positionThe velocity and position of particles can be updated by thefollowing equations
where V denotes the velocity The inertia weight 119908 is usedto balance the global exploration and local exploitation The1199031 and 1199032 are uniformly distributed random variables withinrange (0 1) The 1198881 and 1198882 are positive constant parameterscalled ldquoacceleration coefficientsrdquo Here the particle encodingis composed of the parameters 119862 and 120590 in (13)
43 Cross-Validation In this paper we choose 5-fold con-sidering the best compromise between computational costand reliable estimates The dataset is randomly divided into5 mutually exclusively subsets of approximately equal size inwhich 4 subsets are used as training set and the last subsetis used as validation set The abovementioned procedurerepeated 5 times so each subset is used once for validationThe fitness function of PSO chose the classification accuracyof the 5-fold cross-validation
Here 119910119904 and 119910119898 denote the number of successful classificationand misclassification respectively PSO is performed tomaximize the fitness function (classification accuracy)
44 Pseudocodes of Our Method In total our method can bedescribed as the following three stages and the flowchart isdepicted in Figure 5
ture reduction)Step 3 Fivefolded cross-validationStep 4 Determining the best parameter
Step 41 Initializing PSO The particles correspond to 119862
and 120590Step 42 For each particle 119894 computer the fitness values
Step 421 Decoding the particle to parameters 119862 and120590
Step 422 Using interior method to train KSVMaccording to (13)
Step 423 Calculating classification error accordingto (16) as the fitness values
Step 43 Updating the 119892best and 119901best according to (14)Step 44 Updating the velocity and position of each
particle according to (15)Step 45 If stopping criteria is met then jump to Step 46
otherwise return to Step 42Step 46 Decoding the optimal particle to corresponding
parameter 119862lowast and 120590lowast
Step 5 Constructing KSVM via the optimal 119862lowast and 120590
lowast
according to (13)Step 6 Submitting newMRI brains to the trained KSVM and
outputting the prediction
5 Experiments and Discussions
The experiments were carried out on the platform of P4IBM with 33GHz processor and 2GB RAM running underWindows XP operating system The algorithm was in-housedeveloped via the wavelet toolbox the biostatistical toolboxof 32 bitMATLAB 2012a (theMathWorks)The programs canbe run or tested on any computer platforms where MATLABis available
51 Database The datasets brain consists of 90 T2-weightedMR brain images in axial plane and 256 times 256 in-planeresolution which were downloaded from the website ofHarvard Medical School (URL httpwwwmedharvardeduaanlibhomehtml) The abnormal brain MR imagesof the dataset consist of the following diseases gliomametastatic adenocarcinoma metastatic bronchogenic carci-nomameningioma sarcoma Alzheimer Huntington motorneuron disease cerebral calcinosis Pickrsquos disease Alzheimerplus visual agnosia multiple sclerosis AIDS dementia Lymeencephalopathy herpes encephalitis Creutzfeld-Jakob dis-ease and cerebral toxoplasmosisThe samples of each diseaseare illustrated in Figure 6
6 The Scientific World Journal
MRIbrains
Featureextraction
Featurereduction
CandidateKSVM 1
DWT PCA
Used as trainingand validation set
Preprocessing
New MRIbrain
Normal orabnormal
Particle 1 Particle P
PSO
Particle 2
CandidateKSVM 2 middot middot middot
middot middot middot
middot middot middot
CandidateKSVM P
Fitness 1 Fitness 2 Fitness P
Update particle velocity and position
Are stoppingcriteria met
No
OptimalKSVM
Yes
Output
5-folded cross-validation
Update pbest and gbest
Figure 5 Methodology of our proposed PSO-KSVM algorithm
Figure 7 Illustration of 5-fold cross-validation of brain dataset(we divided the dataset into 5 groups and for each experiment 4groups were used for training and the rest one group was used forvalidation Each group was used once for validation)
We randomly selected 5 images for each type of brainSince there are 1 type of normal brain and 17 types ofabnormal brain in the dataset 5lowast(1 + 17) = 90 images wereselected to construct the brain dataset consisting of 5 normaland 85 abnormal brain images in total
The setting of the training images and validation imageswas shown in Figure 7 We divided the dataset into 5 equallydistributed groups each groups contain one normal brain
(a) (b)
Figure 8Theprocedures of 3-level 2DDWT (a) normal brainMRI(b) level 3 wavelet coefficients
and 17 abnormal brains Since 5-fold cross-validation wasused we would perform 5 experiments In each experiment4 groups were used for training and the left 1 group wasused for validation Each group was used once for validationIn total in this cross validation way 360 images were fortraining and 90 images were for validation
52 Feature Extraction The three levels of wavelet decom-position greatly reduce the input image size as shown inFigure 8 The top left corner of the wavelet coefficients imagedenotes for the approximation coefficients at level 3 of whichthe size is only 32times 32 = 1024The border distortion should beavoided In our algorithm symmetric padding method [21]was utilized to calculate the boundary value
53 Feature Reduction As stated above the extracted featureswere reduced from 65536 to 1024 by the DWT procedureHowever 1024 was still too large for calculation Thus PCAwas used to further reduce the dimensions of features The
8 The Scientific World Journal
Table 3 Parameters of comparison by random selection method (the final row corresponds to our proposed method)
Figure 9 The curve of variances against number of principlecomponents (here we found that 39 features can achieve 9502variances)
curve of cumulative sum of variance versus the number ofprinciple components was shown in Figure 9
The variances versus the number of principle componentsfrom 1 to 40 were listed in Table 1 It showed that only39 principle components (bold font in table) which wereonly 391024 = 381 of the original features could preserve9502 of total variance
54 Classification Accuracy The KSVM used the RBF asthe kernel function We compared our PSO-KVSM methodwith one hidden-layer Back Propagation-Neural Network(BP-NN) and RBF-Neural Network (RBF-NN) The resultswere shown in Table 2 It showed that BP-NN correctlymatched 388 cases with 8622 classification accuracy RBF-NN correctly matched 411 cases with 9133 classificationaccuracy Our PSO-KSVM correctly matched 440 brainimages with 9778 classification accuracy Therefore ourmethod had the most excellent classification performance
55 Parameter Selection The final parameters obtained byPSO were 119862 = 1433 and 120590 = 1132 We compared this casewith random selection method which randomly generatedthe values of 119862 in the range of (50 200) and 120590 in the rangeof [05 2] and then we compared them with the optimizedvalues by PSO (119862 = 1433 and 120590 = 1132) The resultsachieved by random selectionmethodwere shown in Table 3We saw that the classification accuracy variedwith the changeof parameters 120590 and 119862 so it was important to determine the
optimal values before constructing the classifierThe randomselection method was difficult to come across the best valuesso PSO was an effective method for this problem comparedto random selection method
6 Conclusions and Discussions
In this study we had developed a novel DWT + PCA + PSO-KSVM hybrid classification system to distinguish betweennormal and abnormalMRIs of the brainWepicked upRBF asthe kernel function of SVM The experiments demonstratedthat the PSO-KSVM method obtained 9778 classificationaccuracy on the 5-folded 90-image dataset higher than8622 of BP-NN and 9133 of RBF-NN
Future work should focus on the following four aspectsFirst the proposed SVM based method could be employedfor MR images with other contrast mechanisms such as T1-weighted proton density weighted and diffusion weightedimages Second the computation time could be acceleratedby using advanced wavelet transforms such as the lift-upwavelet Third Multiclassification which focuses on brainMRIs of specific disorders can also be explored Forth novelkernels will be tested to increase the classification accuracyand accelerate the algorithm
The DWT can efficiently extract the information fromoriginal MR images with litter loss The advantage of DWTover Fourier transforms is the spatial resolution namelyDWT captures both frequency and location information Inthis study we choose the Harr wavelet although there areother outstandingwavelets such asDaubechies seriesWewillcompare the performance of different families of wavelet infuture work Another research direction lies in the stationarywavelet transform and the wavelet packet transform
The importance of PCA was demonstrated in the Dis-cussion If we omitted the PCA procedures we meet ahuge feature space (1024 dimensions) which will cause heavycomputation burden and lowered the classification accuracyThere are some other excellent feature reduction methodssuch as ICA manifold learning In the future we will focuson investigating the performance of those algorithms
The reason we choose RBF kernel is that RBF takesthe form of exponential function which enlarge the sampledistances to the uttermost extent In the future we will try totest other kernel functions
The Scientific World Journal 9
The importance of introducing PSO is to determine theoptimal values of parameters119862 and120590 From random selectionmethod we found it is hard to get the optimal values at theparameter spaceTherefore the PSO is an effectiveway to findthe optimal values Integrating PSO to KSVM enhance theclassification capability of KSVM
The most important contribution of this paper is thepropose of a hybrid system integrating DWT PCA PSOKSVM and CV used for identifying normal MR brains fromabnormal MR brains It would be useful to help clinicians todiagnose the patients
References
[1] M Emin Tagluk M Akin and N Sezgin ldquoClassification ofsleep apnea by using wavelet transform and artificial neuralnetworksrdquo Expert Systems with Applications vol 37 no 2 pp1600ndash1607 2010
[2] J Camacho J Pico and A Ferrer ldquolsquoThe best approaches in theon-line monitoring of batch processes based on PCA does themodelling structure matterrsquo [Anal Chim Acta Volume 642(2009) 59ndash68]rdquo Analytica Chimica Acta vol 658 no 1 p 1062010
[3] S Chaplot LM Patnaik andN R Jagannathan ldquoClassificationof magnetic resonance brain images using wavelets as input tosupport vector machine and neural networkrdquo Biomedical SignalProcessing and Control vol 1 no 1 pp 86ndash92 2006
[4] C A Cocosco A P Zijdenbos and A C Evans ldquoA fullyautomatic and robust brain MRI tissue classification methodrdquoMedical Image Analysis vol 7 no 4 pp 513ndash527 2003
[5] J-Y Yeh and J C Fu ldquoA hierarchical genetic algorithm for seg-mentation of multi-spectral human-brainMRIrdquo Expert Systemswith Applications vol 34 no 2 pp 1285ndash1295 2008
[6] Y Zhang and L Wu ldquoClassification of fruits using computervision and a multiclass support vector machinerdquo Sensors vol12 no 9 pp 12489ndash12505 2012
[7] N S Patil P S Shelokar V K Jayaraman and B D KulkarnildquoRegression models using pattern search assisted least squaresupport vector machinesrdquo Chemical Engineering Research andDesign A vol 83 no 8 pp 1030ndash1037 2005
[8] D Li W Yang and S Wang ldquoClassification of foreign fibers incotton lint using machine vision and multi-class support vectormachinerdquo Computers and Electronics in Agriculture vol 74 no2 pp 274ndash279 2010
[9] T A F Gomes R B C Prudcncio C Soares A L DRossi and A Carvalho ldquoCombining meta-learning and searchtechniques to select parameters for support vector machinesrdquoNeurocomputing vol 75 pp 3ndash13 2012
[10] R Hable ldquoAsymptotic normality of support vector machinevariants and other regularized kernel methodsrdquo Journal ofMultivariate Analysis vol 106 pp 92ndash117 2012
[11] L Durak ldquoShift-invariance of short-time Fourier transform infractional Fourier domainsrdquo Journal of the Franklin Institutevol 346 no 2 pp 136ndash146 2009
[12] Y Zhang S Wang Y Huo L Wu and A Liu ldquoFeatureextraction of brain MRI by stationary wavelet transform and itsapplicationsrdquo Journal of Biological Systems vol 18 no 1 pp 115ndash132 2010
[13] Y Zhang L Wu and G Wei ldquoA new classifier for polarimetricSAR imagesrdquo Progress in Electromagnetics Research vol 94 pp83ndash104 2009
[15] P Martiskainen M Jarvinen J Skon J Tiirikainen MKolehmainen and J Mononen ldquoCow behaviour pattern recog-nition using a three-dimensional accelerometer and supportvectormachinesrdquoAppliedAnimal Behaviour Science vol 119 no1-2 pp 32ndash38 2009
[16] S Bermejo B Monegal and J Cabestany ldquoFish age catego-rization from otolith images using multi-class support vectormachinesrdquo Fisheries Research vol 84 no 2 pp 247ndash253 2007
[17] A M S Muniz H Liu K E Lyons et al ldquoComparisonamong probabilistic neural network support vector machineand logistic regression for evaluating the effect of subthalamicstimulation in Parkinson disease on ground reaction forceduring gaitrdquo Journal of Biomechanics vol 43 no 4 pp 720ndash7262010
[18] J Acevedo-Rodrıguez S Maldonado-Bascon S Lafuente-Arroyo P Siegmann and F Lopez-Ferreras ldquoComputationalload reduction in decision functions using support vectormachinesrdquo Signal Processing vol 89 no 10 pp 2066ndash20712009
[19] S-W Fei ldquoDiagnostic study on arrhythmia cordis based onparticle swarm optimization-based support vector machinerdquoExpert Systems with Applications vol 37 no 10 pp 6748ndash67522010
[20] Z Chenglin S Xuebin S Songlin and J Ting ldquoFault diagnosisof sensor by chaos particle swarm optimization algorithm andsupport vector machinerdquo Expert Systems with Applications vol38 no 8 pp 9908ndash9912 2011
[21] A Messina ldquoRefinements of damage detection methods basedon wavelet analysis of dynamical shapesrdquo International Journalof Solids and Structures vol 45 no 14-15 pp 4068ndash4097 2008
(8) the solution will not change but the problem is alteredinto a quadratic programming optimization that is easy tosolve by using Lagrange multipliers and standard quadraticprogramming techniques and programs
32 Soft Margin However in practical applications theremay exist no hyperplane that can split the samples per-fectly In such case the ldquosoft marginrdquo method will choose ahyperplane that splits the given samples as clean as possiblewhile still maximizing the distance to the nearest cleanly splitsamples
Positive slack variables 120585119899 are introduced to measure themisclassification degree of sample 119909119899 (the distance betweenthe margin and the vectors 119909119899 that lying on the wrong sideof the margin) Then the optimal hyperplane separating thedata can be obtained by the following optimization problem
minw120585119887
1
2
w2+ 119862
119873
sum
119899=1
120585119899
st 119910119899 (w119909119899 minus 119887) ge 1 minus 120585119899
120585119899 ge 0
119899 = 1 119873
(9)
where 119862 is the error penalty Therefore the optimizationbecomes a tradeoff between a large margin and a small errorpenalty The constraint optimization problem can be solvedusing ldquoLagrange multiplierrdquo as
minw120585119887
max120572120573
1
2
w2+ 119862
119873
sum
119899=1
120585119899
minus
119873
sum
119899=1
120572119899 [119910119899 (w119909119899 minus 119887) minus 1 + 120585119899] minus
119873
sum
119899=1
120573119899120585119899
(10)
The min-max problem is not easy to solve so Cortes andVapnik proposed a dual form technique to solve it
33 Dual Form Thedual formof formula (9) can be designedas
The key advantage of the dual form function is that theslack variables 120585119899 vanish from the dual problem with theconstant 119862 appearing only as an additional constraint onthe Lagrange multipliers Now the optimization problem (11)becomes a quadratic programming (QP) problem which isdefined as the optimization of a quadratic function of severalvariables subject to linear constraints on these variablesTherefore numerous methods can solve formula (9) withinmilliseconds like interior point method active set methodaugmented Lagrangian method conjugate gradient methodsimplex algorithm and so forth
4 PSO-KSVM
41 Kernel SVMs Linear SVMs have the downside to linearhyperplane which cannot separate complicated distributedpractical data In order to generalize it to nonlinear hyper-plane the kernel trick is applied to SVMs [18] The resultingalgorithm is formally similar except that every dot product isreplaced by a nonlinear kernel function In another point ofview the KSVMs allow to fit the maximum-margin hyper-plane in a transformed feature space The transformationmay be nonlinear and the transformed space may be higherdimensional thus though the classifier is a hyperplane in thehigher-dimensional feature space it may be nonlinear in theoriginal input space For each kernel there should be at leastone adjusting parameter so as to make the kernel flexibleand tailor itself to practical data In this paper RBF kernel ischosen due to its excellent performanceThe kernel is writtenas
119896 (119909119898 119909119899) = exp(minus
1003817100381710038171003817119909119898 minus 119909119899
1003817100381710038171003817
21205902
) (12)
Put formula (12) into formula (11) and we got the final SVMtraining function as
1003817100381710038171003817119909119898 minus 119909119899
1003817100381710038171003817
21205902
)
st
0 le 120572119899 le 119862
119873
sum
119899=1
120572119899119910119899 = 0
119899 = 1 119873
(13)
It is still a quadratic programming problem and we choseinterior point method to solve the problem However thereis still an outstanding issue that is the value of parameters 119862and 120590 in (13)
The Scientific World Journal 5
42 PSO To determine the best parameter of 119862 and 120590traditionalmethod uses trial-and-errormethods It will causeheavy computation burden and cannot guarantee to findthe optimal or even near-optimal solutions Fei W [19] andChenglin et al [20] proposed to use PSO to optimize theparameters respectively and independently The PSO is apopulated global optimization method deriving from theresearch of the movement of bird flocking or fish schoolingIt is easy and fast to implement Besides we introduced inthe cross-validation to construct the fitness function used forPSO
PSO performs search via a swarm of particles which isupdated from iteration to iteration To seek for the optimalsolution each particle moves in the direction of its previouslybest position (119901best) and the best global position in the swarm(119892best) as follows
119901best119894 = 119901119894 (119896lowast)
st fitness (119901119894 (119896lowast)) = min119896=1119905
[fitness (119901119894 (119896))]
119892best = 119901119894lowast (119896lowast)
st fitness (119901119894lowast (119896lowast)) = min119894=1119875
119896=1119905
[fitness (119901119894 (119896))]
(14)
where 119894 denotes the particle index119875 denotes the total numberof particles 119896 denotes the iteration index and 119905 denotesthe current iteration number and 119901 denotes the positionThe velocity and position of particles can be updated by thefollowing equations
where V denotes the velocity The inertia weight 119908 is usedto balance the global exploration and local exploitation The1199031 and 1199032 are uniformly distributed random variables withinrange (0 1) The 1198881 and 1198882 are positive constant parameterscalled ldquoacceleration coefficientsrdquo Here the particle encodingis composed of the parameters 119862 and 120590 in (13)
43 Cross-Validation In this paper we choose 5-fold con-sidering the best compromise between computational costand reliable estimates The dataset is randomly divided into5 mutually exclusively subsets of approximately equal size inwhich 4 subsets are used as training set and the last subsetis used as validation set The abovementioned procedurerepeated 5 times so each subset is used once for validationThe fitness function of PSO chose the classification accuracyof the 5-fold cross-validation
Here 119910119904 and 119910119898 denote the number of successful classificationand misclassification respectively PSO is performed tomaximize the fitness function (classification accuracy)
44 Pseudocodes of Our Method In total our method can bedescribed as the following three stages and the flowchart isdepicted in Figure 5
ture reduction)Step 3 Fivefolded cross-validationStep 4 Determining the best parameter
Step 41 Initializing PSO The particles correspond to 119862
and 120590Step 42 For each particle 119894 computer the fitness values
Step 421 Decoding the particle to parameters 119862 and120590
Step 422 Using interior method to train KSVMaccording to (13)
Step 423 Calculating classification error accordingto (16) as the fitness values
Step 43 Updating the 119892best and 119901best according to (14)Step 44 Updating the velocity and position of each
particle according to (15)Step 45 If stopping criteria is met then jump to Step 46
otherwise return to Step 42Step 46 Decoding the optimal particle to corresponding
parameter 119862lowast and 120590lowast
Step 5 Constructing KSVM via the optimal 119862lowast and 120590
lowast
according to (13)Step 6 Submitting newMRI brains to the trained KSVM and
outputting the prediction
5 Experiments and Discussions
The experiments were carried out on the platform of P4IBM with 33GHz processor and 2GB RAM running underWindows XP operating system The algorithm was in-housedeveloped via the wavelet toolbox the biostatistical toolboxof 32 bitMATLAB 2012a (theMathWorks)The programs canbe run or tested on any computer platforms where MATLABis available
51 Database The datasets brain consists of 90 T2-weightedMR brain images in axial plane and 256 times 256 in-planeresolution which were downloaded from the website ofHarvard Medical School (URL httpwwwmedharvardeduaanlibhomehtml) The abnormal brain MR imagesof the dataset consist of the following diseases gliomametastatic adenocarcinoma metastatic bronchogenic carci-nomameningioma sarcoma Alzheimer Huntington motorneuron disease cerebral calcinosis Pickrsquos disease Alzheimerplus visual agnosia multiple sclerosis AIDS dementia Lymeencephalopathy herpes encephalitis Creutzfeld-Jakob dis-ease and cerebral toxoplasmosisThe samples of each diseaseare illustrated in Figure 6
6 The Scientific World Journal
MRIbrains
Featureextraction
Featurereduction
CandidateKSVM 1
DWT PCA
Used as trainingand validation set
Preprocessing
New MRIbrain
Normal orabnormal
Particle 1 Particle P
PSO
Particle 2
CandidateKSVM 2 middot middot middot
middot middot middot
middot middot middot
CandidateKSVM P
Fitness 1 Fitness 2 Fitness P
Update particle velocity and position
Are stoppingcriteria met
No
OptimalKSVM
Yes
Output
5-folded cross-validation
Update pbest and gbest
Figure 5 Methodology of our proposed PSO-KSVM algorithm
Figure 7 Illustration of 5-fold cross-validation of brain dataset(we divided the dataset into 5 groups and for each experiment 4groups were used for training and the rest one group was used forvalidation Each group was used once for validation)
We randomly selected 5 images for each type of brainSince there are 1 type of normal brain and 17 types ofabnormal brain in the dataset 5lowast(1 + 17) = 90 images wereselected to construct the brain dataset consisting of 5 normaland 85 abnormal brain images in total
The setting of the training images and validation imageswas shown in Figure 7 We divided the dataset into 5 equallydistributed groups each groups contain one normal brain
(a) (b)
Figure 8Theprocedures of 3-level 2DDWT (a) normal brainMRI(b) level 3 wavelet coefficients
and 17 abnormal brains Since 5-fold cross-validation wasused we would perform 5 experiments In each experiment4 groups were used for training and the left 1 group wasused for validation Each group was used once for validationIn total in this cross validation way 360 images were fortraining and 90 images were for validation
52 Feature Extraction The three levels of wavelet decom-position greatly reduce the input image size as shown inFigure 8 The top left corner of the wavelet coefficients imagedenotes for the approximation coefficients at level 3 of whichthe size is only 32times 32 = 1024The border distortion should beavoided In our algorithm symmetric padding method [21]was utilized to calculate the boundary value
53 Feature Reduction As stated above the extracted featureswere reduced from 65536 to 1024 by the DWT procedureHowever 1024 was still too large for calculation Thus PCAwas used to further reduce the dimensions of features The
8 The Scientific World Journal
Table 3 Parameters of comparison by random selection method (the final row corresponds to our proposed method)
Figure 9 The curve of variances against number of principlecomponents (here we found that 39 features can achieve 9502variances)
curve of cumulative sum of variance versus the number ofprinciple components was shown in Figure 9
The variances versus the number of principle componentsfrom 1 to 40 were listed in Table 1 It showed that only39 principle components (bold font in table) which wereonly 391024 = 381 of the original features could preserve9502 of total variance
54 Classification Accuracy The KSVM used the RBF asthe kernel function We compared our PSO-KVSM methodwith one hidden-layer Back Propagation-Neural Network(BP-NN) and RBF-Neural Network (RBF-NN) The resultswere shown in Table 2 It showed that BP-NN correctlymatched 388 cases with 8622 classification accuracy RBF-NN correctly matched 411 cases with 9133 classificationaccuracy Our PSO-KSVM correctly matched 440 brainimages with 9778 classification accuracy Therefore ourmethod had the most excellent classification performance
55 Parameter Selection The final parameters obtained byPSO were 119862 = 1433 and 120590 = 1132 We compared this casewith random selection method which randomly generatedthe values of 119862 in the range of (50 200) and 120590 in the rangeof [05 2] and then we compared them with the optimizedvalues by PSO (119862 = 1433 and 120590 = 1132) The resultsachieved by random selectionmethodwere shown in Table 3We saw that the classification accuracy variedwith the changeof parameters 120590 and 119862 so it was important to determine the
optimal values before constructing the classifierThe randomselection method was difficult to come across the best valuesso PSO was an effective method for this problem comparedto random selection method
6 Conclusions and Discussions
In this study we had developed a novel DWT + PCA + PSO-KSVM hybrid classification system to distinguish betweennormal and abnormalMRIs of the brainWepicked upRBF asthe kernel function of SVM The experiments demonstratedthat the PSO-KSVM method obtained 9778 classificationaccuracy on the 5-folded 90-image dataset higher than8622 of BP-NN and 9133 of RBF-NN
Future work should focus on the following four aspectsFirst the proposed SVM based method could be employedfor MR images with other contrast mechanisms such as T1-weighted proton density weighted and diffusion weightedimages Second the computation time could be acceleratedby using advanced wavelet transforms such as the lift-upwavelet Third Multiclassification which focuses on brainMRIs of specific disorders can also be explored Forth novelkernels will be tested to increase the classification accuracyand accelerate the algorithm
The DWT can efficiently extract the information fromoriginal MR images with litter loss The advantage of DWTover Fourier transforms is the spatial resolution namelyDWT captures both frequency and location information Inthis study we choose the Harr wavelet although there areother outstandingwavelets such asDaubechies seriesWewillcompare the performance of different families of wavelet infuture work Another research direction lies in the stationarywavelet transform and the wavelet packet transform
The importance of PCA was demonstrated in the Dis-cussion If we omitted the PCA procedures we meet ahuge feature space (1024 dimensions) which will cause heavycomputation burden and lowered the classification accuracyThere are some other excellent feature reduction methodssuch as ICA manifold learning In the future we will focuson investigating the performance of those algorithms
The reason we choose RBF kernel is that RBF takesthe form of exponential function which enlarge the sampledistances to the uttermost extent In the future we will try totest other kernel functions
The Scientific World Journal 9
The importance of introducing PSO is to determine theoptimal values of parameters119862 and120590 From random selectionmethod we found it is hard to get the optimal values at theparameter spaceTherefore the PSO is an effectiveway to findthe optimal values Integrating PSO to KSVM enhance theclassification capability of KSVM
The most important contribution of this paper is thepropose of a hybrid system integrating DWT PCA PSOKSVM and CV used for identifying normal MR brains fromabnormal MR brains It would be useful to help clinicians todiagnose the patients
References
[1] M Emin Tagluk M Akin and N Sezgin ldquoClassification ofsleep apnea by using wavelet transform and artificial neuralnetworksrdquo Expert Systems with Applications vol 37 no 2 pp1600ndash1607 2010
[2] J Camacho J Pico and A Ferrer ldquolsquoThe best approaches in theon-line monitoring of batch processes based on PCA does themodelling structure matterrsquo [Anal Chim Acta Volume 642(2009) 59ndash68]rdquo Analytica Chimica Acta vol 658 no 1 p 1062010
[3] S Chaplot LM Patnaik andN R Jagannathan ldquoClassificationof magnetic resonance brain images using wavelets as input tosupport vector machine and neural networkrdquo Biomedical SignalProcessing and Control vol 1 no 1 pp 86ndash92 2006
[4] C A Cocosco A P Zijdenbos and A C Evans ldquoA fullyautomatic and robust brain MRI tissue classification methodrdquoMedical Image Analysis vol 7 no 4 pp 513ndash527 2003
[5] J-Y Yeh and J C Fu ldquoA hierarchical genetic algorithm for seg-mentation of multi-spectral human-brainMRIrdquo Expert Systemswith Applications vol 34 no 2 pp 1285ndash1295 2008
[6] Y Zhang and L Wu ldquoClassification of fruits using computervision and a multiclass support vector machinerdquo Sensors vol12 no 9 pp 12489ndash12505 2012
[7] N S Patil P S Shelokar V K Jayaraman and B D KulkarnildquoRegression models using pattern search assisted least squaresupport vector machinesrdquo Chemical Engineering Research andDesign A vol 83 no 8 pp 1030ndash1037 2005
[8] D Li W Yang and S Wang ldquoClassification of foreign fibers incotton lint using machine vision and multi-class support vectormachinerdquo Computers and Electronics in Agriculture vol 74 no2 pp 274ndash279 2010
[9] T A F Gomes R B C Prudcncio C Soares A L DRossi and A Carvalho ldquoCombining meta-learning and searchtechniques to select parameters for support vector machinesrdquoNeurocomputing vol 75 pp 3ndash13 2012
[10] R Hable ldquoAsymptotic normality of support vector machinevariants and other regularized kernel methodsrdquo Journal ofMultivariate Analysis vol 106 pp 92ndash117 2012
[11] L Durak ldquoShift-invariance of short-time Fourier transform infractional Fourier domainsrdquo Journal of the Franklin Institutevol 346 no 2 pp 136ndash146 2009
[12] Y Zhang S Wang Y Huo L Wu and A Liu ldquoFeatureextraction of brain MRI by stationary wavelet transform and itsapplicationsrdquo Journal of Biological Systems vol 18 no 1 pp 115ndash132 2010
[13] Y Zhang L Wu and G Wei ldquoA new classifier for polarimetricSAR imagesrdquo Progress in Electromagnetics Research vol 94 pp83ndash104 2009
[15] P Martiskainen M Jarvinen J Skon J Tiirikainen MKolehmainen and J Mononen ldquoCow behaviour pattern recog-nition using a three-dimensional accelerometer and supportvectormachinesrdquoAppliedAnimal Behaviour Science vol 119 no1-2 pp 32ndash38 2009
[16] S Bermejo B Monegal and J Cabestany ldquoFish age catego-rization from otolith images using multi-class support vectormachinesrdquo Fisheries Research vol 84 no 2 pp 247ndash253 2007
[17] A M S Muniz H Liu K E Lyons et al ldquoComparisonamong probabilistic neural network support vector machineand logistic regression for evaluating the effect of subthalamicstimulation in Parkinson disease on ground reaction forceduring gaitrdquo Journal of Biomechanics vol 43 no 4 pp 720ndash7262010
[18] J Acevedo-Rodrıguez S Maldonado-Bascon S Lafuente-Arroyo P Siegmann and F Lopez-Ferreras ldquoComputationalload reduction in decision functions using support vectormachinesrdquo Signal Processing vol 89 no 10 pp 2066ndash20712009
[19] S-W Fei ldquoDiagnostic study on arrhythmia cordis based onparticle swarm optimization-based support vector machinerdquoExpert Systems with Applications vol 37 no 10 pp 6748ndash67522010
[20] Z Chenglin S Xuebin S Songlin and J Ting ldquoFault diagnosisof sensor by chaos particle swarm optimization algorithm andsupport vector machinerdquo Expert Systems with Applications vol38 no 8 pp 9908ndash9912 2011
[21] A Messina ldquoRefinements of damage detection methods basedon wavelet analysis of dynamical shapesrdquo International Journalof Solids and Structures vol 45 no 14-15 pp 4068ndash4097 2008
42 PSO To determine the best parameter of 119862 and 120590traditionalmethod uses trial-and-errormethods It will causeheavy computation burden and cannot guarantee to findthe optimal or even near-optimal solutions Fei W [19] andChenglin et al [20] proposed to use PSO to optimize theparameters respectively and independently The PSO is apopulated global optimization method deriving from theresearch of the movement of bird flocking or fish schoolingIt is easy and fast to implement Besides we introduced inthe cross-validation to construct the fitness function used forPSO
PSO performs search via a swarm of particles which isupdated from iteration to iteration To seek for the optimalsolution each particle moves in the direction of its previouslybest position (119901best) and the best global position in the swarm(119892best) as follows
119901best119894 = 119901119894 (119896lowast)
st fitness (119901119894 (119896lowast)) = min119896=1119905
[fitness (119901119894 (119896))]
119892best = 119901119894lowast (119896lowast)
st fitness (119901119894lowast (119896lowast)) = min119894=1119875
119896=1119905
[fitness (119901119894 (119896))]
(14)
where 119894 denotes the particle index119875 denotes the total numberof particles 119896 denotes the iteration index and 119905 denotesthe current iteration number and 119901 denotes the positionThe velocity and position of particles can be updated by thefollowing equations
where V denotes the velocity The inertia weight 119908 is usedto balance the global exploration and local exploitation The1199031 and 1199032 are uniformly distributed random variables withinrange (0 1) The 1198881 and 1198882 are positive constant parameterscalled ldquoacceleration coefficientsrdquo Here the particle encodingis composed of the parameters 119862 and 120590 in (13)
43 Cross-Validation In this paper we choose 5-fold con-sidering the best compromise between computational costand reliable estimates The dataset is randomly divided into5 mutually exclusively subsets of approximately equal size inwhich 4 subsets are used as training set and the last subsetis used as validation set The abovementioned procedurerepeated 5 times so each subset is used once for validationThe fitness function of PSO chose the classification accuracyof the 5-fold cross-validation
Here 119910119904 and 119910119898 denote the number of successful classificationand misclassification respectively PSO is performed tomaximize the fitness function (classification accuracy)
44 Pseudocodes of Our Method In total our method can bedescribed as the following three stages and the flowchart isdepicted in Figure 5
ture reduction)Step 3 Fivefolded cross-validationStep 4 Determining the best parameter
Step 41 Initializing PSO The particles correspond to 119862
and 120590Step 42 For each particle 119894 computer the fitness values
Step 421 Decoding the particle to parameters 119862 and120590
Step 422 Using interior method to train KSVMaccording to (13)
Step 423 Calculating classification error accordingto (16) as the fitness values
Step 43 Updating the 119892best and 119901best according to (14)Step 44 Updating the velocity and position of each
particle according to (15)Step 45 If stopping criteria is met then jump to Step 46
otherwise return to Step 42Step 46 Decoding the optimal particle to corresponding
parameter 119862lowast and 120590lowast
Step 5 Constructing KSVM via the optimal 119862lowast and 120590
lowast
according to (13)Step 6 Submitting newMRI brains to the trained KSVM and
outputting the prediction
5 Experiments and Discussions
The experiments were carried out on the platform of P4IBM with 33GHz processor and 2GB RAM running underWindows XP operating system The algorithm was in-housedeveloped via the wavelet toolbox the biostatistical toolboxof 32 bitMATLAB 2012a (theMathWorks)The programs canbe run or tested on any computer platforms where MATLABis available
51 Database The datasets brain consists of 90 T2-weightedMR brain images in axial plane and 256 times 256 in-planeresolution which were downloaded from the website ofHarvard Medical School (URL httpwwwmedharvardeduaanlibhomehtml) The abnormal brain MR imagesof the dataset consist of the following diseases gliomametastatic adenocarcinoma metastatic bronchogenic carci-nomameningioma sarcoma Alzheimer Huntington motorneuron disease cerebral calcinosis Pickrsquos disease Alzheimerplus visual agnosia multiple sclerosis AIDS dementia Lymeencephalopathy herpes encephalitis Creutzfeld-Jakob dis-ease and cerebral toxoplasmosisThe samples of each diseaseare illustrated in Figure 6
6 The Scientific World Journal
MRIbrains
Featureextraction
Featurereduction
CandidateKSVM 1
DWT PCA
Used as trainingand validation set
Preprocessing
New MRIbrain
Normal orabnormal
Particle 1 Particle P
PSO
Particle 2
CandidateKSVM 2 middot middot middot
middot middot middot
middot middot middot
CandidateKSVM P
Fitness 1 Fitness 2 Fitness P
Update particle velocity and position
Are stoppingcriteria met
No
OptimalKSVM
Yes
Output
5-folded cross-validation
Update pbest and gbest
Figure 5 Methodology of our proposed PSO-KSVM algorithm
Figure 7 Illustration of 5-fold cross-validation of brain dataset(we divided the dataset into 5 groups and for each experiment 4groups were used for training and the rest one group was used forvalidation Each group was used once for validation)
We randomly selected 5 images for each type of brainSince there are 1 type of normal brain and 17 types ofabnormal brain in the dataset 5lowast(1 + 17) = 90 images wereselected to construct the brain dataset consisting of 5 normaland 85 abnormal brain images in total
The setting of the training images and validation imageswas shown in Figure 7 We divided the dataset into 5 equallydistributed groups each groups contain one normal brain
(a) (b)
Figure 8Theprocedures of 3-level 2DDWT (a) normal brainMRI(b) level 3 wavelet coefficients
and 17 abnormal brains Since 5-fold cross-validation wasused we would perform 5 experiments In each experiment4 groups were used for training and the left 1 group wasused for validation Each group was used once for validationIn total in this cross validation way 360 images were fortraining and 90 images were for validation
52 Feature Extraction The three levels of wavelet decom-position greatly reduce the input image size as shown inFigure 8 The top left corner of the wavelet coefficients imagedenotes for the approximation coefficients at level 3 of whichthe size is only 32times 32 = 1024The border distortion should beavoided In our algorithm symmetric padding method [21]was utilized to calculate the boundary value
53 Feature Reduction As stated above the extracted featureswere reduced from 65536 to 1024 by the DWT procedureHowever 1024 was still too large for calculation Thus PCAwas used to further reduce the dimensions of features The
8 The Scientific World Journal
Table 3 Parameters of comparison by random selection method (the final row corresponds to our proposed method)
Figure 9 The curve of variances against number of principlecomponents (here we found that 39 features can achieve 9502variances)
curve of cumulative sum of variance versus the number ofprinciple components was shown in Figure 9
The variances versus the number of principle componentsfrom 1 to 40 were listed in Table 1 It showed that only39 principle components (bold font in table) which wereonly 391024 = 381 of the original features could preserve9502 of total variance
54 Classification Accuracy The KSVM used the RBF asthe kernel function We compared our PSO-KVSM methodwith one hidden-layer Back Propagation-Neural Network(BP-NN) and RBF-Neural Network (RBF-NN) The resultswere shown in Table 2 It showed that BP-NN correctlymatched 388 cases with 8622 classification accuracy RBF-NN correctly matched 411 cases with 9133 classificationaccuracy Our PSO-KSVM correctly matched 440 brainimages with 9778 classification accuracy Therefore ourmethod had the most excellent classification performance
55 Parameter Selection The final parameters obtained byPSO were 119862 = 1433 and 120590 = 1132 We compared this casewith random selection method which randomly generatedthe values of 119862 in the range of (50 200) and 120590 in the rangeof [05 2] and then we compared them with the optimizedvalues by PSO (119862 = 1433 and 120590 = 1132) The resultsachieved by random selectionmethodwere shown in Table 3We saw that the classification accuracy variedwith the changeof parameters 120590 and 119862 so it was important to determine the
optimal values before constructing the classifierThe randomselection method was difficult to come across the best valuesso PSO was an effective method for this problem comparedto random selection method
6 Conclusions and Discussions
In this study we had developed a novel DWT + PCA + PSO-KSVM hybrid classification system to distinguish betweennormal and abnormalMRIs of the brainWepicked upRBF asthe kernel function of SVM The experiments demonstratedthat the PSO-KSVM method obtained 9778 classificationaccuracy on the 5-folded 90-image dataset higher than8622 of BP-NN and 9133 of RBF-NN
Future work should focus on the following four aspectsFirst the proposed SVM based method could be employedfor MR images with other contrast mechanisms such as T1-weighted proton density weighted and diffusion weightedimages Second the computation time could be acceleratedby using advanced wavelet transforms such as the lift-upwavelet Third Multiclassification which focuses on brainMRIs of specific disorders can also be explored Forth novelkernels will be tested to increase the classification accuracyand accelerate the algorithm
The DWT can efficiently extract the information fromoriginal MR images with litter loss The advantage of DWTover Fourier transforms is the spatial resolution namelyDWT captures both frequency and location information Inthis study we choose the Harr wavelet although there areother outstandingwavelets such asDaubechies seriesWewillcompare the performance of different families of wavelet infuture work Another research direction lies in the stationarywavelet transform and the wavelet packet transform
The importance of PCA was demonstrated in the Dis-cussion If we omitted the PCA procedures we meet ahuge feature space (1024 dimensions) which will cause heavycomputation burden and lowered the classification accuracyThere are some other excellent feature reduction methodssuch as ICA manifold learning In the future we will focuson investigating the performance of those algorithms
The reason we choose RBF kernel is that RBF takesthe form of exponential function which enlarge the sampledistances to the uttermost extent In the future we will try totest other kernel functions
The Scientific World Journal 9
The importance of introducing PSO is to determine theoptimal values of parameters119862 and120590 From random selectionmethod we found it is hard to get the optimal values at theparameter spaceTherefore the PSO is an effectiveway to findthe optimal values Integrating PSO to KSVM enhance theclassification capability of KSVM
The most important contribution of this paper is thepropose of a hybrid system integrating DWT PCA PSOKSVM and CV used for identifying normal MR brains fromabnormal MR brains It would be useful to help clinicians todiagnose the patients
References
[1] M Emin Tagluk M Akin and N Sezgin ldquoClassification ofsleep apnea by using wavelet transform and artificial neuralnetworksrdquo Expert Systems with Applications vol 37 no 2 pp1600ndash1607 2010
[2] J Camacho J Pico and A Ferrer ldquolsquoThe best approaches in theon-line monitoring of batch processes based on PCA does themodelling structure matterrsquo [Anal Chim Acta Volume 642(2009) 59ndash68]rdquo Analytica Chimica Acta vol 658 no 1 p 1062010
[3] S Chaplot LM Patnaik andN R Jagannathan ldquoClassificationof magnetic resonance brain images using wavelets as input tosupport vector machine and neural networkrdquo Biomedical SignalProcessing and Control vol 1 no 1 pp 86ndash92 2006
[4] C A Cocosco A P Zijdenbos and A C Evans ldquoA fullyautomatic and robust brain MRI tissue classification methodrdquoMedical Image Analysis vol 7 no 4 pp 513ndash527 2003
[5] J-Y Yeh and J C Fu ldquoA hierarchical genetic algorithm for seg-mentation of multi-spectral human-brainMRIrdquo Expert Systemswith Applications vol 34 no 2 pp 1285ndash1295 2008
[6] Y Zhang and L Wu ldquoClassification of fruits using computervision and a multiclass support vector machinerdquo Sensors vol12 no 9 pp 12489ndash12505 2012
[7] N S Patil P S Shelokar V K Jayaraman and B D KulkarnildquoRegression models using pattern search assisted least squaresupport vector machinesrdquo Chemical Engineering Research andDesign A vol 83 no 8 pp 1030ndash1037 2005
[8] D Li W Yang and S Wang ldquoClassification of foreign fibers incotton lint using machine vision and multi-class support vectormachinerdquo Computers and Electronics in Agriculture vol 74 no2 pp 274ndash279 2010
[9] T A F Gomes R B C Prudcncio C Soares A L DRossi and A Carvalho ldquoCombining meta-learning and searchtechniques to select parameters for support vector machinesrdquoNeurocomputing vol 75 pp 3ndash13 2012
[10] R Hable ldquoAsymptotic normality of support vector machinevariants and other regularized kernel methodsrdquo Journal ofMultivariate Analysis vol 106 pp 92ndash117 2012
[11] L Durak ldquoShift-invariance of short-time Fourier transform infractional Fourier domainsrdquo Journal of the Franklin Institutevol 346 no 2 pp 136ndash146 2009
[12] Y Zhang S Wang Y Huo L Wu and A Liu ldquoFeatureextraction of brain MRI by stationary wavelet transform and itsapplicationsrdquo Journal of Biological Systems vol 18 no 1 pp 115ndash132 2010
[13] Y Zhang L Wu and G Wei ldquoA new classifier for polarimetricSAR imagesrdquo Progress in Electromagnetics Research vol 94 pp83ndash104 2009
[15] P Martiskainen M Jarvinen J Skon J Tiirikainen MKolehmainen and J Mononen ldquoCow behaviour pattern recog-nition using a three-dimensional accelerometer and supportvectormachinesrdquoAppliedAnimal Behaviour Science vol 119 no1-2 pp 32ndash38 2009
[16] S Bermejo B Monegal and J Cabestany ldquoFish age catego-rization from otolith images using multi-class support vectormachinesrdquo Fisheries Research vol 84 no 2 pp 247ndash253 2007
[17] A M S Muniz H Liu K E Lyons et al ldquoComparisonamong probabilistic neural network support vector machineand logistic regression for evaluating the effect of subthalamicstimulation in Parkinson disease on ground reaction forceduring gaitrdquo Journal of Biomechanics vol 43 no 4 pp 720ndash7262010
[18] J Acevedo-Rodrıguez S Maldonado-Bascon S Lafuente-Arroyo P Siegmann and F Lopez-Ferreras ldquoComputationalload reduction in decision functions using support vectormachinesrdquo Signal Processing vol 89 no 10 pp 2066ndash20712009
[19] S-W Fei ldquoDiagnostic study on arrhythmia cordis based onparticle swarm optimization-based support vector machinerdquoExpert Systems with Applications vol 37 no 10 pp 6748ndash67522010
[20] Z Chenglin S Xuebin S Songlin and J Ting ldquoFault diagnosisof sensor by chaos particle swarm optimization algorithm andsupport vector machinerdquo Expert Systems with Applications vol38 no 8 pp 9908ndash9912 2011
[21] A Messina ldquoRefinements of damage detection methods basedon wavelet analysis of dynamical shapesrdquo International Journalof Solids and Structures vol 45 no 14-15 pp 4068ndash4097 2008
Figure 7 Illustration of 5-fold cross-validation of brain dataset(we divided the dataset into 5 groups and for each experiment 4groups were used for training and the rest one group was used forvalidation Each group was used once for validation)
We randomly selected 5 images for each type of brainSince there are 1 type of normal brain and 17 types ofabnormal brain in the dataset 5lowast(1 + 17) = 90 images wereselected to construct the brain dataset consisting of 5 normaland 85 abnormal brain images in total
The setting of the training images and validation imageswas shown in Figure 7 We divided the dataset into 5 equallydistributed groups each groups contain one normal brain
(a) (b)
Figure 8Theprocedures of 3-level 2DDWT (a) normal brainMRI(b) level 3 wavelet coefficients
and 17 abnormal brains Since 5-fold cross-validation wasused we would perform 5 experiments In each experiment4 groups were used for training and the left 1 group wasused for validation Each group was used once for validationIn total in this cross validation way 360 images were fortraining and 90 images were for validation
52 Feature Extraction The three levels of wavelet decom-position greatly reduce the input image size as shown inFigure 8 The top left corner of the wavelet coefficients imagedenotes for the approximation coefficients at level 3 of whichthe size is only 32times 32 = 1024The border distortion should beavoided In our algorithm symmetric padding method [21]was utilized to calculate the boundary value
53 Feature Reduction As stated above the extracted featureswere reduced from 65536 to 1024 by the DWT procedureHowever 1024 was still too large for calculation Thus PCAwas used to further reduce the dimensions of features The
8 The Scientific World Journal
Table 3 Parameters of comparison by random selection method (the final row corresponds to our proposed method)
Figure 9 The curve of variances against number of principlecomponents (here we found that 39 features can achieve 9502variances)
curve of cumulative sum of variance versus the number ofprinciple components was shown in Figure 9
The variances versus the number of principle componentsfrom 1 to 40 were listed in Table 1 It showed that only39 principle components (bold font in table) which wereonly 391024 = 381 of the original features could preserve9502 of total variance
54 Classification Accuracy The KSVM used the RBF asthe kernel function We compared our PSO-KVSM methodwith one hidden-layer Back Propagation-Neural Network(BP-NN) and RBF-Neural Network (RBF-NN) The resultswere shown in Table 2 It showed that BP-NN correctlymatched 388 cases with 8622 classification accuracy RBF-NN correctly matched 411 cases with 9133 classificationaccuracy Our PSO-KSVM correctly matched 440 brainimages with 9778 classification accuracy Therefore ourmethod had the most excellent classification performance
55 Parameter Selection The final parameters obtained byPSO were 119862 = 1433 and 120590 = 1132 We compared this casewith random selection method which randomly generatedthe values of 119862 in the range of (50 200) and 120590 in the rangeof [05 2] and then we compared them with the optimizedvalues by PSO (119862 = 1433 and 120590 = 1132) The resultsachieved by random selectionmethodwere shown in Table 3We saw that the classification accuracy variedwith the changeof parameters 120590 and 119862 so it was important to determine the
optimal values before constructing the classifierThe randomselection method was difficult to come across the best valuesso PSO was an effective method for this problem comparedto random selection method
6 Conclusions and Discussions
In this study we had developed a novel DWT + PCA + PSO-KSVM hybrid classification system to distinguish betweennormal and abnormalMRIs of the brainWepicked upRBF asthe kernel function of SVM The experiments demonstratedthat the PSO-KSVM method obtained 9778 classificationaccuracy on the 5-folded 90-image dataset higher than8622 of BP-NN and 9133 of RBF-NN
Future work should focus on the following four aspectsFirst the proposed SVM based method could be employedfor MR images with other contrast mechanisms such as T1-weighted proton density weighted and diffusion weightedimages Second the computation time could be acceleratedby using advanced wavelet transforms such as the lift-upwavelet Third Multiclassification which focuses on brainMRIs of specific disorders can also be explored Forth novelkernels will be tested to increase the classification accuracyand accelerate the algorithm
The DWT can efficiently extract the information fromoriginal MR images with litter loss The advantage of DWTover Fourier transforms is the spatial resolution namelyDWT captures both frequency and location information Inthis study we choose the Harr wavelet although there areother outstandingwavelets such asDaubechies seriesWewillcompare the performance of different families of wavelet infuture work Another research direction lies in the stationarywavelet transform and the wavelet packet transform
The importance of PCA was demonstrated in the Dis-cussion If we omitted the PCA procedures we meet ahuge feature space (1024 dimensions) which will cause heavycomputation burden and lowered the classification accuracyThere are some other excellent feature reduction methodssuch as ICA manifold learning In the future we will focuson investigating the performance of those algorithms
The reason we choose RBF kernel is that RBF takesthe form of exponential function which enlarge the sampledistances to the uttermost extent In the future we will try totest other kernel functions
The Scientific World Journal 9
The importance of introducing PSO is to determine theoptimal values of parameters119862 and120590 From random selectionmethod we found it is hard to get the optimal values at theparameter spaceTherefore the PSO is an effectiveway to findthe optimal values Integrating PSO to KSVM enhance theclassification capability of KSVM
The most important contribution of this paper is thepropose of a hybrid system integrating DWT PCA PSOKSVM and CV used for identifying normal MR brains fromabnormal MR brains It would be useful to help clinicians todiagnose the patients
References
[1] M Emin Tagluk M Akin and N Sezgin ldquoClassification ofsleep apnea by using wavelet transform and artificial neuralnetworksrdquo Expert Systems with Applications vol 37 no 2 pp1600ndash1607 2010
[2] J Camacho J Pico and A Ferrer ldquolsquoThe best approaches in theon-line monitoring of batch processes based on PCA does themodelling structure matterrsquo [Anal Chim Acta Volume 642(2009) 59ndash68]rdquo Analytica Chimica Acta vol 658 no 1 p 1062010
[3] S Chaplot LM Patnaik andN R Jagannathan ldquoClassificationof magnetic resonance brain images using wavelets as input tosupport vector machine and neural networkrdquo Biomedical SignalProcessing and Control vol 1 no 1 pp 86ndash92 2006
[4] C A Cocosco A P Zijdenbos and A C Evans ldquoA fullyautomatic and robust brain MRI tissue classification methodrdquoMedical Image Analysis vol 7 no 4 pp 513ndash527 2003
[5] J-Y Yeh and J C Fu ldquoA hierarchical genetic algorithm for seg-mentation of multi-spectral human-brainMRIrdquo Expert Systemswith Applications vol 34 no 2 pp 1285ndash1295 2008
[6] Y Zhang and L Wu ldquoClassification of fruits using computervision and a multiclass support vector machinerdquo Sensors vol12 no 9 pp 12489ndash12505 2012
[7] N S Patil P S Shelokar V K Jayaraman and B D KulkarnildquoRegression models using pattern search assisted least squaresupport vector machinesrdquo Chemical Engineering Research andDesign A vol 83 no 8 pp 1030ndash1037 2005
[8] D Li W Yang and S Wang ldquoClassification of foreign fibers incotton lint using machine vision and multi-class support vectormachinerdquo Computers and Electronics in Agriculture vol 74 no2 pp 274ndash279 2010
[9] T A F Gomes R B C Prudcncio C Soares A L DRossi and A Carvalho ldquoCombining meta-learning and searchtechniques to select parameters for support vector machinesrdquoNeurocomputing vol 75 pp 3ndash13 2012
[10] R Hable ldquoAsymptotic normality of support vector machinevariants and other regularized kernel methodsrdquo Journal ofMultivariate Analysis vol 106 pp 92ndash117 2012
[11] L Durak ldquoShift-invariance of short-time Fourier transform infractional Fourier domainsrdquo Journal of the Franklin Institutevol 346 no 2 pp 136ndash146 2009
[12] Y Zhang S Wang Y Huo L Wu and A Liu ldquoFeatureextraction of brain MRI by stationary wavelet transform and itsapplicationsrdquo Journal of Biological Systems vol 18 no 1 pp 115ndash132 2010
[13] Y Zhang L Wu and G Wei ldquoA new classifier for polarimetricSAR imagesrdquo Progress in Electromagnetics Research vol 94 pp83ndash104 2009
[15] P Martiskainen M Jarvinen J Skon J Tiirikainen MKolehmainen and J Mononen ldquoCow behaviour pattern recog-nition using a three-dimensional accelerometer and supportvectormachinesrdquoAppliedAnimal Behaviour Science vol 119 no1-2 pp 32ndash38 2009
[16] S Bermejo B Monegal and J Cabestany ldquoFish age catego-rization from otolith images using multi-class support vectormachinesrdquo Fisheries Research vol 84 no 2 pp 247ndash253 2007
[17] A M S Muniz H Liu K E Lyons et al ldquoComparisonamong probabilistic neural network support vector machineand logistic regression for evaluating the effect of subthalamicstimulation in Parkinson disease on ground reaction forceduring gaitrdquo Journal of Biomechanics vol 43 no 4 pp 720ndash7262010
[18] J Acevedo-Rodrıguez S Maldonado-Bascon S Lafuente-Arroyo P Siegmann and F Lopez-Ferreras ldquoComputationalload reduction in decision functions using support vectormachinesrdquo Signal Processing vol 89 no 10 pp 2066ndash20712009
[19] S-W Fei ldquoDiagnostic study on arrhythmia cordis based onparticle swarm optimization-based support vector machinerdquoExpert Systems with Applications vol 37 no 10 pp 6748ndash67522010
[20] Z Chenglin S Xuebin S Songlin and J Ting ldquoFault diagnosisof sensor by chaos particle swarm optimization algorithm andsupport vector machinerdquo Expert Systems with Applications vol38 no 8 pp 9908ndash9912 2011
[21] A Messina ldquoRefinements of damage detection methods basedon wavelet analysis of dynamical shapesrdquo International Journalof Solids and Structures vol 45 no 14-15 pp 4068ndash4097 2008
Figure 7 Illustration of 5-fold cross-validation of brain dataset(we divided the dataset into 5 groups and for each experiment 4groups were used for training and the rest one group was used forvalidation Each group was used once for validation)
We randomly selected 5 images for each type of brainSince there are 1 type of normal brain and 17 types ofabnormal brain in the dataset 5lowast(1 + 17) = 90 images wereselected to construct the brain dataset consisting of 5 normaland 85 abnormal brain images in total
The setting of the training images and validation imageswas shown in Figure 7 We divided the dataset into 5 equallydistributed groups each groups contain one normal brain
(a) (b)
Figure 8Theprocedures of 3-level 2DDWT (a) normal brainMRI(b) level 3 wavelet coefficients
and 17 abnormal brains Since 5-fold cross-validation wasused we would perform 5 experiments In each experiment4 groups were used for training and the left 1 group wasused for validation Each group was used once for validationIn total in this cross validation way 360 images were fortraining and 90 images were for validation
52 Feature Extraction The three levels of wavelet decom-position greatly reduce the input image size as shown inFigure 8 The top left corner of the wavelet coefficients imagedenotes for the approximation coefficients at level 3 of whichthe size is only 32times 32 = 1024The border distortion should beavoided In our algorithm symmetric padding method [21]was utilized to calculate the boundary value
53 Feature Reduction As stated above the extracted featureswere reduced from 65536 to 1024 by the DWT procedureHowever 1024 was still too large for calculation Thus PCAwas used to further reduce the dimensions of features The
8 The Scientific World Journal
Table 3 Parameters of comparison by random selection method (the final row corresponds to our proposed method)
Figure 9 The curve of variances against number of principlecomponents (here we found that 39 features can achieve 9502variances)
curve of cumulative sum of variance versus the number ofprinciple components was shown in Figure 9
The variances versus the number of principle componentsfrom 1 to 40 were listed in Table 1 It showed that only39 principle components (bold font in table) which wereonly 391024 = 381 of the original features could preserve9502 of total variance
54 Classification Accuracy The KSVM used the RBF asthe kernel function We compared our PSO-KVSM methodwith one hidden-layer Back Propagation-Neural Network(BP-NN) and RBF-Neural Network (RBF-NN) The resultswere shown in Table 2 It showed that BP-NN correctlymatched 388 cases with 8622 classification accuracy RBF-NN correctly matched 411 cases with 9133 classificationaccuracy Our PSO-KSVM correctly matched 440 brainimages with 9778 classification accuracy Therefore ourmethod had the most excellent classification performance
55 Parameter Selection The final parameters obtained byPSO were 119862 = 1433 and 120590 = 1132 We compared this casewith random selection method which randomly generatedthe values of 119862 in the range of (50 200) and 120590 in the rangeof [05 2] and then we compared them with the optimizedvalues by PSO (119862 = 1433 and 120590 = 1132) The resultsachieved by random selectionmethodwere shown in Table 3We saw that the classification accuracy variedwith the changeof parameters 120590 and 119862 so it was important to determine the
optimal values before constructing the classifierThe randomselection method was difficult to come across the best valuesso PSO was an effective method for this problem comparedto random selection method
6 Conclusions and Discussions
In this study we had developed a novel DWT + PCA + PSO-KSVM hybrid classification system to distinguish betweennormal and abnormalMRIs of the brainWepicked upRBF asthe kernel function of SVM The experiments demonstratedthat the PSO-KSVM method obtained 9778 classificationaccuracy on the 5-folded 90-image dataset higher than8622 of BP-NN and 9133 of RBF-NN
Future work should focus on the following four aspectsFirst the proposed SVM based method could be employedfor MR images with other contrast mechanisms such as T1-weighted proton density weighted and diffusion weightedimages Second the computation time could be acceleratedby using advanced wavelet transforms such as the lift-upwavelet Third Multiclassification which focuses on brainMRIs of specific disorders can also be explored Forth novelkernels will be tested to increase the classification accuracyand accelerate the algorithm
The DWT can efficiently extract the information fromoriginal MR images with litter loss The advantage of DWTover Fourier transforms is the spatial resolution namelyDWT captures both frequency and location information Inthis study we choose the Harr wavelet although there areother outstandingwavelets such asDaubechies seriesWewillcompare the performance of different families of wavelet infuture work Another research direction lies in the stationarywavelet transform and the wavelet packet transform
The importance of PCA was demonstrated in the Dis-cussion If we omitted the PCA procedures we meet ahuge feature space (1024 dimensions) which will cause heavycomputation burden and lowered the classification accuracyThere are some other excellent feature reduction methodssuch as ICA manifold learning In the future we will focuson investigating the performance of those algorithms
The reason we choose RBF kernel is that RBF takesthe form of exponential function which enlarge the sampledistances to the uttermost extent In the future we will try totest other kernel functions
The Scientific World Journal 9
The importance of introducing PSO is to determine theoptimal values of parameters119862 and120590 From random selectionmethod we found it is hard to get the optimal values at theparameter spaceTherefore the PSO is an effectiveway to findthe optimal values Integrating PSO to KSVM enhance theclassification capability of KSVM
The most important contribution of this paper is thepropose of a hybrid system integrating DWT PCA PSOKSVM and CV used for identifying normal MR brains fromabnormal MR brains It would be useful to help clinicians todiagnose the patients
References
[1] M Emin Tagluk M Akin and N Sezgin ldquoClassification ofsleep apnea by using wavelet transform and artificial neuralnetworksrdquo Expert Systems with Applications vol 37 no 2 pp1600ndash1607 2010
[2] J Camacho J Pico and A Ferrer ldquolsquoThe best approaches in theon-line monitoring of batch processes based on PCA does themodelling structure matterrsquo [Anal Chim Acta Volume 642(2009) 59ndash68]rdquo Analytica Chimica Acta vol 658 no 1 p 1062010
[3] S Chaplot LM Patnaik andN R Jagannathan ldquoClassificationof magnetic resonance brain images using wavelets as input tosupport vector machine and neural networkrdquo Biomedical SignalProcessing and Control vol 1 no 1 pp 86ndash92 2006
[4] C A Cocosco A P Zijdenbos and A C Evans ldquoA fullyautomatic and robust brain MRI tissue classification methodrdquoMedical Image Analysis vol 7 no 4 pp 513ndash527 2003
[5] J-Y Yeh and J C Fu ldquoA hierarchical genetic algorithm for seg-mentation of multi-spectral human-brainMRIrdquo Expert Systemswith Applications vol 34 no 2 pp 1285ndash1295 2008
[6] Y Zhang and L Wu ldquoClassification of fruits using computervision and a multiclass support vector machinerdquo Sensors vol12 no 9 pp 12489ndash12505 2012
[7] N S Patil P S Shelokar V K Jayaraman and B D KulkarnildquoRegression models using pattern search assisted least squaresupport vector machinesrdquo Chemical Engineering Research andDesign A vol 83 no 8 pp 1030ndash1037 2005
[8] D Li W Yang and S Wang ldquoClassification of foreign fibers incotton lint using machine vision and multi-class support vectormachinerdquo Computers and Electronics in Agriculture vol 74 no2 pp 274ndash279 2010
[9] T A F Gomes R B C Prudcncio C Soares A L DRossi and A Carvalho ldquoCombining meta-learning and searchtechniques to select parameters for support vector machinesrdquoNeurocomputing vol 75 pp 3ndash13 2012
[10] R Hable ldquoAsymptotic normality of support vector machinevariants and other regularized kernel methodsrdquo Journal ofMultivariate Analysis vol 106 pp 92ndash117 2012
[11] L Durak ldquoShift-invariance of short-time Fourier transform infractional Fourier domainsrdquo Journal of the Franklin Institutevol 346 no 2 pp 136ndash146 2009
[12] Y Zhang S Wang Y Huo L Wu and A Liu ldquoFeatureextraction of brain MRI by stationary wavelet transform and itsapplicationsrdquo Journal of Biological Systems vol 18 no 1 pp 115ndash132 2010
[13] Y Zhang L Wu and G Wei ldquoA new classifier for polarimetricSAR imagesrdquo Progress in Electromagnetics Research vol 94 pp83ndash104 2009
[15] P Martiskainen M Jarvinen J Skon J Tiirikainen MKolehmainen and J Mononen ldquoCow behaviour pattern recog-nition using a three-dimensional accelerometer and supportvectormachinesrdquoAppliedAnimal Behaviour Science vol 119 no1-2 pp 32ndash38 2009
[16] S Bermejo B Monegal and J Cabestany ldquoFish age catego-rization from otolith images using multi-class support vectormachinesrdquo Fisheries Research vol 84 no 2 pp 247ndash253 2007
[17] A M S Muniz H Liu K E Lyons et al ldquoComparisonamong probabilistic neural network support vector machineand logistic regression for evaluating the effect of subthalamicstimulation in Parkinson disease on ground reaction forceduring gaitrdquo Journal of Biomechanics vol 43 no 4 pp 720ndash7262010
[18] J Acevedo-Rodrıguez S Maldonado-Bascon S Lafuente-Arroyo P Siegmann and F Lopez-Ferreras ldquoComputationalload reduction in decision functions using support vectormachinesrdquo Signal Processing vol 89 no 10 pp 2066ndash20712009
[19] S-W Fei ldquoDiagnostic study on arrhythmia cordis based onparticle swarm optimization-based support vector machinerdquoExpert Systems with Applications vol 37 no 10 pp 6748ndash67522010
[20] Z Chenglin S Xuebin S Songlin and J Ting ldquoFault diagnosisof sensor by chaos particle swarm optimization algorithm andsupport vector machinerdquo Expert Systems with Applications vol38 no 8 pp 9908ndash9912 2011
[21] A Messina ldquoRefinements of damage detection methods basedon wavelet analysis of dynamical shapesrdquo International Journalof Solids and Structures vol 45 no 14-15 pp 4068ndash4097 2008
Figure 9 The curve of variances against number of principlecomponents (here we found that 39 features can achieve 9502variances)
curve of cumulative sum of variance versus the number ofprinciple components was shown in Figure 9
The variances versus the number of principle componentsfrom 1 to 40 were listed in Table 1 It showed that only39 principle components (bold font in table) which wereonly 391024 = 381 of the original features could preserve9502 of total variance
54 Classification Accuracy The KSVM used the RBF asthe kernel function We compared our PSO-KVSM methodwith one hidden-layer Back Propagation-Neural Network(BP-NN) and RBF-Neural Network (RBF-NN) The resultswere shown in Table 2 It showed that BP-NN correctlymatched 388 cases with 8622 classification accuracy RBF-NN correctly matched 411 cases with 9133 classificationaccuracy Our PSO-KSVM correctly matched 440 brainimages with 9778 classification accuracy Therefore ourmethod had the most excellent classification performance
55 Parameter Selection The final parameters obtained byPSO were 119862 = 1433 and 120590 = 1132 We compared this casewith random selection method which randomly generatedthe values of 119862 in the range of (50 200) and 120590 in the rangeof [05 2] and then we compared them with the optimizedvalues by PSO (119862 = 1433 and 120590 = 1132) The resultsachieved by random selectionmethodwere shown in Table 3We saw that the classification accuracy variedwith the changeof parameters 120590 and 119862 so it was important to determine the
optimal values before constructing the classifierThe randomselection method was difficult to come across the best valuesso PSO was an effective method for this problem comparedto random selection method
6 Conclusions and Discussions
In this study we had developed a novel DWT + PCA + PSO-KSVM hybrid classification system to distinguish betweennormal and abnormalMRIs of the brainWepicked upRBF asthe kernel function of SVM The experiments demonstratedthat the PSO-KSVM method obtained 9778 classificationaccuracy on the 5-folded 90-image dataset higher than8622 of BP-NN and 9133 of RBF-NN
Future work should focus on the following four aspectsFirst the proposed SVM based method could be employedfor MR images with other contrast mechanisms such as T1-weighted proton density weighted and diffusion weightedimages Second the computation time could be acceleratedby using advanced wavelet transforms such as the lift-upwavelet Third Multiclassification which focuses on brainMRIs of specific disorders can also be explored Forth novelkernels will be tested to increase the classification accuracyand accelerate the algorithm
The DWT can efficiently extract the information fromoriginal MR images with litter loss The advantage of DWTover Fourier transforms is the spatial resolution namelyDWT captures both frequency and location information Inthis study we choose the Harr wavelet although there areother outstandingwavelets such asDaubechies seriesWewillcompare the performance of different families of wavelet infuture work Another research direction lies in the stationarywavelet transform and the wavelet packet transform
The importance of PCA was demonstrated in the Dis-cussion If we omitted the PCA procedures we meet ahuge feature space (1024 dimensions) which will cause heavycomputation burden and lowered the classification accuracyThere are some other excellent feature reduction methodssuch as ICA manifold learning In the future we will focuson investigating the performance of those algorithms
The reason we choose RBF kernel is that RBF takesthe form of exponential function which enlarge the sampledistances to the uttermost extent In the future we will try totest other kernel functions
The Scientific World Journal 9
The importance of introducing PSO is to determine theoptimal values of parameters119862 and120590 From random selectionmethod we found it is hard to get the optimal values at theparameter spaceTherefore the PSO is an effectiveway to findthe optimal values Integrating PSO to KSVM enhance theclassification capability of KSVM
The most important contribution of this paper is thepropose of a hybrid system integrating DWT PCA PSOKSVM and CV used for identifying normal MR brains fromabnormal MR brains It would be useful to help clinicians todiagnose the patients
References
[1] M Emin Tagluk M Akin and N Sezgin ldquoClassification ofsleep apnea by using wavelet transform and artificial neuralnetworksrdquo Expert Systems with Applications vol 37 no 2 pp1600ndash1607 2010
[2] J Camacho J Pico and A Ferrer ldquolsquoThe best approaches in theon-line monitoring of batch processes based on PCA does themodelling structure matterrsquo [Anal Chim Acta Volume 642(2009) 59ndash68]rdquo Analytica Chimica Acta vol 658 no 1 p 1062010
[3] S Chaplot LM Patnaik andN R Jagannathan ldquoClassificationof magnetic resonance brain images using wavelets as input tosupport vector machine and neural networkrdquo Biomedical SignalProcessing and Control vol 1 no 1 pp 86ndash92 2006
[4] C A Cocosco A P Zijdenbos and A C Evans ldquoA fullyautomatic and robust brain MRI tissue classification methodrdquoMedical Image Analysis vol 7 no 4 pp 513ndash527 2003
[5] J-Y Yeh and J C Fu ldquoA hierarchical genetic algorithm for seg-mentation of multi-spectral human-brainMRIrdquo Expert Systemswith Applications vol 34 no 2 pp 1285ndash1295 2008
[6] Y Zhang and L Wu ldquoClassification of fruits using computervision and a multiclass support vector machinerdquo Sensors vol12 no 9 pp 12489ndash12505 2012
[7] N S Patil P S Shelokar V K Jayaraman and B D KulkarnildquoRegression models using pattern search assisted least squaresupport vector machinesrdquo Chemical Engineering Research andDesign A vol 83 no 8 pp 1030ndash1037 2005
[8] D Li W Yang and S Wang ldquoClassification of foreign fibers incotton lint using machine vision and multi-class support vectormachinerdquo Computers and Electronics in Agriculture vol 74 no2 pp 274ndash279 2010
[9] T A F Gomes R B C Prudcncio C Soares A L DRossi and A Carvalho ldquoCombining meta-learning and searchtechniques to select parameters for support vector machinesrdquoNeurocomputing vol 75 pp 3ndash13 2012
[10] R Hable ldquoAsymptotic normality of support vector machinevariants and other regularized kernel methodsrdquo Journal ofMultivariate Analysis vol 106 pp 92ndash117 2012
[11] L Durak ldquoShift-invariance of short-time Fourier transform infractional Fourier domainsrdquo Journal of the Franklin Institutevol 346 no 2 pp 136ndash146 2009
[12] Y Zhang S Wang Y Huo L Wu and A Liu ldquoFeatureextraction of brain MRI by stationary wavelet transform and itsapplicationsrdquo Journal of Biological Systems vol 18 no 1 pp 115ndash132 2010
[13] Y Zhang L Wu and G Wei ldquoA new classifier for polarimetricSAR imagesrdquo Progress in Electromagnetics Research vol 94 pp83ndash104 2009
[15] P Martiskainen M Jarvinen J Skon J Tiirikainen MKolehmainen and J Mononen ldquoCow behaviour pattern recog-nition using a three-dimensional accelerometer and supportvectormachinesrdquoAppliedAnimal Behaviour Science vol 119 no1-2 pp 32ndash38 2009
[16] S Bermejo B Monegal and J Cabestany ldquoFish age catego-rization from otolith images using multi-class support vectormachinesrdquo Fisheries Research vol 84 no 2 pp 247ndash253 2007
[17] A M S Muniz H Liu K E Lyons et al ldquoComparisonamong probabilistic neural network support vector machineand logistic regression for evaluating the effect of subthalamicstimulation in Parkinson disease on ground reaction forceduring gaitrdquo Journal of Biomechanics vol 43 no 4 pp 720ndash7262010
[18] J Acevedo-Rodrıguez S Maldonado-Bascon S Lafuente-Arroyo P Siegmann and F Lopez-Ferreras ldquoComputationalload reduction in decision functions using support vectormachinesrdquo Signal Processing vol 89 no 10 pp 2066ndash20712009
[19] S-W Fei ldquoDiagnostic study on arrhythmia cordis based onparticle swarm optimization-based support vector machinerdquoExpert Systems with Applications vol 37 no 10 pp 6748ndash67522010
[20] Z Chenglin S Xuebin S Songlin and J Ting ldquoFault diagnosisof sensor by chaos particle swarm optimization algorithm andsupport vector machinerdquo Expert Systems with Applications vol38 no 8 pp 9908ndash9912 2011
[21] A Messina ldquoRefinements of damage detection methods basedon wavelet analysis of dynamical shapesrdquo International Journalof Solids and Structures vol 45 no 14-15 pp 4068ndash4097 2008
The importance of introducing PSO is to determine theoptimal values of parameters119862 and120590 From random selectionmethod we found it is hard to get the optimal values at theparameter spaceTherefore the PSO is an effectiveway to findthe optimal values Integrating PSO to KSVM enhance theclassification capability of KSVM
The most important contribution of this paper is thepropose of a hybrid system integrating DWT PCA PSOKSVM and CV used for identifying normal MR brains fromabnormal MR brains It would be useful to help clinicians todiagnose the patients
References
[1] M Emin Tagluk M Akin and N Sezgin ldquoClassification ofsleep apnea by using wavelet transform and artificial neuralnetworksrdquo Expert Systems with Applications vol 37 no 2 pp1600ndash1607 2010
[2] J Camacho J Pico and A Ferrer ldquolsquoThe best approaches in theon-line monitoring of batch processes based on PCA does themodelling structure matterrsquo [Anal Chim Acta Volume 642(2009) 59ndash68]rdquo Analytica Chimica Acta vol 658 no 1 p 1062010
[3] S Chaplot LM Patnaik andN R Jagannathan ldquoClassificationof magnetic resonance brain images using wavelets as input tosupport vector machine and neural networkrdquo Biomedical SignalProcessing and Control vol 1 no 1 pp 86ndash92 2006
[4] C A Cocosco A P Zijdenbos and A C Evans ldquoA fullyautomatic and robust brain MRI tissue classification methodrdquoMedical Image Analysis vol 7 no 4 pp 513ndash527 2003
[5] J-Y Yeh and J C Fu ldquoA hierarchical genetic algorithm for seg-mentation of multi-spectral human-brainMRIrdquo Expert Systemswith Applications vol 34 no 2 pp 1285ndash1295 2008
[6] Y Zhang and L Wu ldquoClassification of fruits using computervision and a multiclass support vector machinerdquo Sensors vol12 no 9 pp 12489ndash12505 2012
[7] N S Patil P S Shelokar V K Jayaraman and B D KulkarnildquoRegression models using pattern search assisted least squaresupport vector machinesrdquo Chemical Engineering Research andDesign A vol 83 no 8 pp 1030ndash1037 2005
[8] D Li W Yang and S Wang ldquoClassification of foreign fibers incotton lint using machine vision and multi-class support vectormachinerdquo Computers and Electronics in Agriculture vol 74 no2 pp 274ndash279 2010
[9] T A F Gomes R B C Prudcncio C Soares A L DRossi and A Carvalho ldquoCombining meta-learning and searchtechniques to select parameters for support vector machinesrdquoNeurocomputing vol 75 pp 3ndash13 2012
[10] R Hable ldquoAsymptotic normality of support vector machinevariants and other regularized kernel methodsrdquo Journal ofMultivariate Analysis vol 106 pp 92ndash117 2012
[11] L Durak ldquoShift-invariance of short-time Fourier transform infractional Fourier domainsrdquo Journal of the Franklin Institutevol 346 no 2 pp 136ndash146 2009
[12] Y Zhang S Wang Y Huo L Wu and A Liu ldquoFeatureextraction of brain MRI by stationary wavelet transform and itsapplicationsrdquo Journal of Biological Systems vol 18 no 1 pp 115ndash132 2010
[13] Y Zhang L Wu and G Wei ldquoA new classifier for polarimetricSAR imagesrdquo Progress in Electromagnetics Research vol 94 pp83ndash104 2009
[15] P Martiskainen M Jarvinen J Skon J Tiirikainen MKolehmainen and J Mononen ldquoCow behaviour pattern recog-nition using a three-dimensional accelerometer and supportvectormachinesrdquoAppliedAnimal Behaviour Science vol 119 no1-2 pp 32ndash38 2009
[16] S Bermejo B Monegal and J Cabestany ldquoFish age catego-rization from otolith images using multi-class support vectormachinesrdquo Fisheries Research vol 84 no 2 pp 247ndash253 2007
[17] A M S Muniz H Liu K E Lyons et al ldquoComparisonamong probabilistic neural network support vector machineand logistic regression for evaluating the effect of subthalamicstimulation in Parkinson disease on ground reaction forceduring gaitrdquo Journal of Biomechanics vol 43 no 4 pp 720ndash7262010
[18] J Acevedo-Rodrıguez S Maldonado-Bascon S Lafuente-Arroyo P Siegmann and F Lopez-Ferreras ldquoComputationalload reduction in decision functions using support vectormachinesrdquo Signal Processing vol 89 no 10 pp 2066ndash20712009
[19] S-W Fei ldquoDiagnostic study on arrhythmia cordis based onparticle swarm optimization-based support vector machinerdquoExpert Systems with Applications vol 37 no 10 pp 6748ndash67522010
[20] Z Chenglin S Xuebin S Songlin and J Ting ldquoFault diagnosisof sensor by chaos particle swarm optimization algorithm andsupport vector machinerdquo Expert Systems with Applications vol38 no 8 pp 9908ndash9912 2011
[21] A Messina ldquoRefinements of damage detection methods basedon wavelet analysis of dynamical shapesrdquo International Journalof Solids and Structures vol 45 no 14-15 pp 4068ndash4097 2008
