USE OF SUPPORT VECTOR MACHINE FOR TEETH … · USE OF SUPPORT VECTOR MACHINE FOR TEETH RECOGNITION...

USE OF SUPPORT VECTOR MACHINE FOR TEETH RECOGNITION FROMOCCLUSAL INTRAORAL DIGITAL PHOTOGRAPHIC IMAGES

Ramon Augusto Sousa Lins*, Keylly Eyglys Araujo dos Santos*, Adriao Duarte DoriaNeto*, Luis Noro†, Angelo Giuseppe Roncalli†, Maria Cristina dos Santos Medeiros†,

Pedro Henrique Sette de Souza†, Samara Martins da Silva†

*Department of Computer and AutomationFederal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil.

†Department of OdontologyFederal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil.

Emails: [email protected], [email protected], [email protected],

[email protected], [email protected], [email protected],

[email protected], [email protected]

Abstract— In this work we propose the development of an intelligent system able to recognize teeth fromocclusal intraoral digital photographic images. The system combines traditional techniques of machine learningand digital image processing. At first, we perform a color-based segmentation to find the regions of interest, teethand non teeth, using the Support Vector Machine algorithm. After this, we use techniques based on morphologicaloperators (erosion, watershed transform) to detect the teeth boundaries. Therewith, we calculate the Fourierdescriptors to take into account their shapes and positions. Then, they are classified according to their types bythe Support Vector Machine algorithm with the one-against-all method. The problem of multiclass classificationis approached in two different ways. At first we consider three types of teeth classes: molar, premolar and nonteeth, while in the second approach we consider five classes: molar, premolar, canine, incisor and non teeth. Theresults are shown through methods of performance estimation such as random sampling and confusion matrix.

Keywords— Occlusal intraoral digital photographic images, support vector machine, Fourier transform, multi-class classification.

Resumo— Neste trabalho nos propomos o desenvolvimento de um sistema inteligente capaz de reconhecerdentes a partir de imagens fotograficas digitais intraorais oclusais. O sistema faz uso combinado de tecnicas deaprendizado de maquina e processamento digital de imagem. Inicialmente e realizada uma segmentacao baseadaem cores das regioes de interesse, dentes e nao dentes, por um classificador binario usando maquina de vetores desuporte. Apos a identificacao dessas regioes sao utilizadas tecnicas baseadas em operadores morfologicos (erosao,transformada watershed) para detectar os limites dos dentes. Com isso e possıvel extrair seus descritores de forma(descritores de Fourier) e seus descritores de posicao. Em seguida, os dentes sao classificados quanto aos seus tipospor uma maquina de vetores de suporte com uso do metodo um-contra-todos, usado em problemas de classificacaode multiplas classes. O problema de classificacao de multiplas classes e abordado de duas maneiras diferentes. Naprimeira abordagem, consideramos tres tipos de classes: molares, pre-molares e nao dentes, enquanto na segundaabordagem sao considerados cinco tipos de classes: molares, pre-molares, caninos, incisivos e nao dentes. Osresultados sao mostrados atraves dos metodos de estimativa de desempenho amostragem aleatoria e matriz deconfusao.

Palavras-chave— Imagens fotograficas digitais intraorais oclusais, maquinas de vetores de suporte, transfor-mada de Fourier, classificacao multi-classe.

1 Introduction

The fast development of intelligent medical imagesystems is improving diagnoses and their interpre-tations. In dental industry, computer aided proce-dures such as dental implants, orthodontic plan-ning, among others are making a widely use ofdigital dental images as radiographs, photographsand computerized tomographies in support tothese procedures. Many efforts have been madein this area to develop methods that generatesgood results for different applications. Some de-veloped techniques were based on statistics prop-erties (Choorat et al., 2011), mathematical mor-phology (Mahsa Sepehrian, 2013), mathematicaltransforms (Gottlieb et al., 2014), among others.

From the different branches of dentistry, thepublic health is the area in which the dentist isresponsible for the oral health diagnosis of a com-

munity. To this end, the dentist makes epidemi-ological surveys through oral visual inspectionsproducing informations (e.g. identification andcounting of troubled teeth) about the oral healthconditions of Brazilian population, subsidizing theplanning and evaluating the taken actions on dif-ferent management levels of the Health UniqueSystem (SUS, from portuguese Sistema Unico deSaude). This common process accomplished bydentists has some limiting factors, as for examplea limited number of qualified professionals, differ-ent interpretations of diagnosis and so forth.

In order to assist the epidemiological survey,we propose the development of an intelligent sys-tem able to accomplish the basic visual task of rec-ognize teeth from intraoral digital photographicimages. Initially we do a color-based segmenta-tion to find the regions of interest, teeth and nonteeth, present in the images using the Support

XIII Simposio Brasileiro de Automacao Inteligente

Porto Alegre – RS, 1o – 4 de Outubro de 2017

ISSN 2175 8905 1747

Vector Machine (SVM) algorithm. After identi-fying those regions the data is used to detect theycontours. At this stage, techniques based on mor-phological operators such as erosion and water-shed transform are used to detect the teeth con-tours. Therewith, the Fourier descriptors are cal-culated taking in consideration their shapes andpositions. Then they are classified according totheir types by the SVM algorithm using the one-against-all method. The classification problem ofmulticlass is approached in two different ways. Atfirst we consider three types of teeth classes (mostof the problems happen in these teeth): molar,premolar and non teeth, while in the second ap-proach we consider five classes: molar, premolar,canine, incisor and non teeth. The aim here is toanalyse the possibility of resolving it using an ap-proach still not explored in the literature, at leastto the knowledge of the authors of this work.

This paper is organized as follows: in sectionII is done a brief bibliography revision of SVM.In section III the proposed system is explained.The results are described in section IV. Finally, insection V, the conclusion is discussed.

2 Support Vector Machine

The SVM is a classifier based on stan-dard theory of statistic learning, proposed byVapnick(Vapnick, 1995), that finds an optimalseparation surface, minimizing the classificationerror. It can be used in linear problems solutionas well as in non linear problems.

Originally the SVM was developed to solvebinary problems (two classes). Extend its func-tionality remains current research object. Overthe years, some methods were proposed to solvethe problems of multiclass classification.

In the strategy one-against-all for every 𝑐classes 𝑐 classifiers are built; The negative classesare all the others classes if not the positive class.The classification of each new instance is done us-ing the methodology in which the winner-take-all,in other words, the classifier with highest outputassign the corresponding class.

In this method, are used 𝑘 SVM models, being𝑘 the number of classes. The 𝑖𝑡ℎ SVM model istrained with all the training data in the 𝑖 classlabeled positive and the others data classes la-beled as negative. Thus, for a given trainingset {(x𝑖

1, 𝑦𝑖1), ..., (x𝑖

𝑛, 𝑦𝑖𝑛)}, where x𝑖 ∈ ℜ𝑑 with

𝑑 = 1, ..., 𝑛 and 𝑦𝑖 ∈ {1, ..., 𝑘} is the class of 𝑥𝑖,the 𝑖𝑡ℎ SVM model is solved as follows:

minw𝑖,𝑏𝑖,𝜉𝑖

1

2||w𝑖||2 + 𝐶

⎛⎝ 𝑛∑𝑗=1

𝜉𝑖𝑗

⎞⎠ (1)

For the optimization problem there are 𝑘 de-

cisions functions:

(𝑤1)𝑇 Φ(x) + 𝑏1

...

(𝑤𝑘)𝑇 Φ(x) + 𝑏𝑘

Thus, it can be said that x belongs to the classwith highest value between decisions functions,being defined as:

𝑐𝑙𝑎𝑠𝑠 𝑜𝑓 𝑥 ≡ 𝑎𝑟𝑔 max𝑖=1,...,𝑘

((𝑤𝑖)𝑇 Φ(x) + 𝑏𝑖) (2)

For more details consult (Chih-Wei Hsu, 2002).

3 Proposed System

The process of teeth recognition from occlusal in-traoral digital images is defined basically in fourstages, see Figure 1. At first, we propose an im-age acquisition technique that fits the constraintsfaced by health workers in the field. Therewith,several measurements of inclination angles anddistances from the acquisition camera were made.From this standardization, we created an imagedatabase that is used as input of the system. Afterthis, the images go through a preprocessing stageto adapt their scale, color model and contrast ad-justment to the problem. Next, they pass by dif-ferent processes: segmentation based on colors bySVM, morphological operators, Fourier transformand multiclass classification by SVM so that thedata can be represented and described adequatelyfor the recognition process.

Figure 1: Block diagram of the proposed system.

3.1 Image acquisition

We use 40 images from compact cameras CANON,model A1300 with built-in flash. The images were



1748

taken of two ways: upper and lower occlusal usinga specific side mirror and lip retractor in adults.

The occlusal photograph is a two-dimensionalimage corresponding to the distribution of allteeth (third molar right to the third molar fromleft) in the dental arch (upper or lower) view fromthe top down or the opposite way as shown inFigure 2a.

3.2 Preprocessing

The preprocessing of images is a procedure fre-quently used in pattern recognition problems toadapt the input data for different purposes, to ob-tain the best possible solution to the problem. Weuse as input the image as a pixel matrix of RGBvalues (red, green, blue), with values ranging from0 to 255 for 8-bit depth.

There still other models of color images, inthis work is used the YCbCr color model, briefexplained in section 3.2.2.

3.2.1 Scale adaptation

Due to the different scales resulting from the im-age acquisition process, is performed a scale adap-tation, procedure that reduces or enlarges the im-age from a scale obtained according to the dimen-sion of the image and an maximum dimensions ofa fixed window 600 x 600 pixels, so that the fixeddimension is not exceeded.

This adjustment is accomplished through thebicubic interpolation method where the outputpixels values are the average values of a 4x4 win-dow pixels from the point neighborhood.

3.2.2 Conversion from RGB to YCbCr

The YCbCr model has redundancies that canbe eliminated without prejudice the image, mak-ing them files smaller without major visual loss.Another important feature is the Y componentthat contains high frequencies of luminance ingrayscale that facilitate the learning system pro-cess.

In this model, Y represents the luminance ofan image, while the chrominance Cb is the blue(B-Y) and red chrominance Cr (R-Y), for moredetails consult Acharya and Tsai (2005).

3.2.3 Contrast adjustment

After the image be scaled and converted to YCbCrmodel, it’s applied to each component a contrastenhancement to achieve better discrimination oftheir color pixels. The contrast adjustment is doneby mapping each image pixel in a correspondingpixel to an [0,1] interval.

(a) RGB image (b) YCbCr image (c) YCbCr imagewith adjust

Figure 2: RGB image converted to the YCbCrmodel with contrast adjustment.

3.3 Segmentation

3.3.1 Based-color segmentation

From the conversion and contrast adjustmentof the original image, the segmentation is donethrough an binary classifier (SVM). Each imagepixel is classified as teeth or non teeth, setted todifferent sample colors for teeth and restorationsin a class, and gums, tongue and other oral ele-ments in a second class. After the creation of thetraining set the optimal hyperplane is calculatedand the network parameters are defined. A testdata set is utilized as input of the classifier andthe output data forms a binary image with the re-gions of interest, non teeth and teeth, plus noiseand unwanted regions, see sub-picture 3a.

3.3.2 Morphological operators

The morphological operators are used in the ex-traction of image components, so they can beproperly used in the representation and descrip-tion of its forms. From the obtained binary imagewe perform an image opening process by a struc-ture element in a disk shape with radius equal’s1 extracting the image background, in a way thatthe most representative pixels of teeth are identi-fied.

Next, the image passes through a median filterwith 17x17 dimension. After the filtering processthe image passes through a hole fill. Accordingto Soille (Soille, 2002), binary image holes are de-fined as a set of background components that arenot connected to the image edges. Following thisidea, we can say that holes are collections of back-ground pixels (black) surrounded by foregroundpixels (white) that do not connect the edges ofobjects. This process is based on morphologicalreconstruction where masks and markers are de-fined from the filtered image.

After filling the holes, the image undergoesby an erosion process to remove some noises andunwanted components.

The combined use of these techniques makesthe binary image adequated to to calculate thedistance mapping of the regions of interest. Thedistance image is utilized by the watershed algo-rithm to identifying the boundaries of the teeth..



1749

(a) Binary image (b) Binary imagewith opening

(c) Binary image fil-tered

(d) Binary imagefilled

(e) Binary imageeroded

Figure 3: Identification stages of interest regions.

Meyer (1994), defines the distance image astopographical reliefs. This relief suffers a uni-form flooding process from its regional minimums.From the moment the floods begin to mix, barriersare erected to prevent this from happening, thesebarriers are known as watershed lines.

Applying the watershed transform in the dis-tance image it follows that the elements are la-beled into integers larger or equal to zero knownas watershed pixels, generating as output a labelmatrix that represents the regions of interest seg-mented. These steps can be better observed inFigure 4.

(a) Distance image. (b) Labeled image. (c) Binary imagewith frontiers.

Figure 4: Frontier detection process of teeth fromdistance images.

3.4 Teeth type classification

At this stage the binary image with teeth bor-ders are processed in order that the boundaries(Fourier) and position descriptors can be ex-tracted of each segmented object. First, each ob-ject has its border decomposed in a 2-D outlineas a polygonal curve of 𝑁 points on the 𝑥 and 𝑦plane. Next is defined an array of complex num-bers through the vertices coordinates. We applythe Discrete Fourier Transform (DFT) on a com-plex vector obtaining a Fourier descriptors vectorthat represents the boundaries of each tooth. TheDFT decomposes the data into a set of frequencycomponents. We use ten Fourier coefficients forrepresenting the data, number of descriptors thatbest described the teeth boundaries on the images

without significant loss.

It is important that the descriptors be in-variant to translation, rotation, scale and startingpoint given the different specific features of eachpatient’s teeth. Make them invariant to scale isalso important because of adversities found in theprocess of image acquisition.

According to Kaick et al. (2010), to obtain in-variant descriptors to rotation of its on contour inthe 2-D plane, the phase component must be dis-carded using only their magnitude. Discard thephase component also makes the descriptors in-variant to start points contour selection. Elimi-nating the DC component makes the descriptorsbecome invariant to translation. The DC compo-nent, which is the first sign of the coefficient cor-responds to the average value of the signal shape.Finally, to obtain invariance to scale each coeffi-cient is divided by the magnitude of the first fre-quency component, taking into account that theDC component was discarded.

The descriptors area and position are ob-tained from its area and angle. The area is nor-malized in function of the total area of segmentedregions. The angle is obtained in function of eachobject relative to the central vertical axis and thecenter point of each image, normalized in functionof the angles sum.

For the teeth type classification problem weuse two approaches in trying to handle in the bestpossible way for this problem, since most of theproblems occur into the molars and premolars. Infirst approach the objects are arranged in threeclasses for the training process and classification.To the first class are considered the molar teeth, insecond class the premolars and in the third classthe canines, the incisors and the objects that weconsidered non teeth.

In second approach we considered five classes.To the first class are considered the molar teeth, insecond class the premolars, in third class canines,the incisors to fourth class and to the fifth classthe objects that we considered non teeth. Theresult of both approaches can be seen on Figure5.

(a) Classification of theteeth types for threeclasses.

(b) Classification ofthe teeth types for fiveclasses.

Figure 5: Classification process of the teeth types.



1750

4 Results

4.1 Classification algorithm assessment for thebased-color segmentation

4.1.1 Random Sampling

In the segmentation by colors we use a data setconsisting of 22353 points with three dimensions(YCbCr), representing the pixels belonging toclasses teeth and non teeth. The data set is di-vided in: half training, half testing. The trainingand test processes are carried out repeatedly fordifferent kernel functions. By the end of the re-peating process we obtained the statistical infor-mation results of average and variance shown inTable 1.

Table 1: Segmentation by teeth colors.Function Kernel Accuracy(%) VarianceQuadratic 96.59 0.02Radial Base Function(RBF) 98.13 0.01Sigmoid 72.83 4.46

4.2 Watershed segmentation performance

From the acquired images we recorded a total of463 teeth, in which 117 of them are molars, 118premolars, 78 canines and 150 incisors. Throughthe border detection method is possible by an ex-pert count obtain a rate of 83.37% in the correctsegmentation of the teeth present in images. TheTable 2 shows concisely the results obtained dur-ing the segmentation process.

Table 2: Watershed segmentation of the teeth.Teeth Present Segmented Accuracy(%)Molars 117 94 80.34Premolars 118 102 86.44Canines 78 68 87.18Incisors 150 122 81.33Total 463 386 83.37

4.3 Evaluation of the classification algorithm forthe three classes approach

4.3.1 Random sampling

In the classification of teeth we use a data set con-sisting of 470 objects that represents the essenceof the classes: molars, premolars and non teeth.The dataset is divided into 60% for training and40% for testing. The training and testing pro-cesses is carried out repeatedly for different kernelfunctions. By the end of the repeating processwe obtained the statistical information results ofaverage and variance shown in the Table 3:

We can inferred from the classification withRBF kernel that the count of molar and premo-lars has an average hit rate of 88.13 % given the

Table 3: Classification of teeth type for the threeclasses approach.

Kernel Function Accuracy(%) VarianceQuadratic 85.87 10.16Polynomial 84.16 7.55RBF 88.13 3.01

amount of test data used. The others teeth (in-cisors and canines) can not be distinguished in thisapproach because they are considered in the sameobjects class labeled as non teeth.

4.3.2 Optimal classifier performance

For choosing the SVM parameters we use the gridsearch technique for a full search of parameters.The optimal classifier is set to RBF kernel, witha 𝜎 = 0.9 and a regularization parameter 𝐶 = 2.The optimal classifier is assessed from their errormetrics and confusion matrix.

Based on the confusion matrix the perfor-mance of the optimal classifier is shown in Table4.

Table 4: Confusion matrix for the three classesapproach

Truemolar premolar non teeth

Pre

dic

t molar 37 3 0premolar 1 36 2non teeth 1 3 106

accuracy(%) 94.7

4.4 Evaluation of the classification algorithm forthe five classes approach

4.4.1 Random sampling

The five class approach evaluation is similar to theprocess described in 4.3.1, however two classes areinserted, canines and incisors. The statistical in-formation results are shown in the following Ta-ble 5:

Table 5: Teeth type classification for five classesapproach.

Kernel function Accuracy(%) VarianceQuadratic 77.72 11.05Polynomial 74.51 12.81RBF 80.95 5.09

We can inferred from the classification withRBF kernel that the teeth count has an average hitrate of 80.95 % given the amount of test data used.All teeth can be distinguished in this approachbecause the canines and incisors are now distinctclasses among themselves and objects labeled asnon teeth.



1751

4.4.2 Optimal classifier performance

For choosing the SVM parameters we use the gridsearch technique for a full search of parameters.The optimal classifier is set to RBF kernel witha 𝜎 = 0.9 and a regularization parameter 𝐶 = 2.The optimal classifier is assessed from their errormetrics and confusion matrix. Based on the con-fusion matrix the classifier performance is shownin Table 6.

Table 6: Confusion matrix for the five classes ap-proach

Truemolar premolar canine incisor non teeth

Pre

dic

t molar 36 3 0 0 0premolar 1 38 4 0 0

canine 0 0 20 3 0incisor 0 0 3 47 3

non teeth 2 1 1 1 26accuracy(%) 88.4

5 Conclusion and Future Work

This work proposes to use intelligent image pro-cessing techniques for interpretation of digitalintraoral occlusal photos facing the public oralhealth.

As usual, working with appropriate digitalimages is always a limiting factor in intelligentsystems development. Using photographs takenthrough common low cost cameras is even morechallenging, comparing with the use of X-ray orcomputed tomography images. External factorssuch as brightness, noise, resolution might evenimpair the system use.

With these challenges in mind, we tried to usesimpler classical techniques already established inintelligent image processing area. We opted forthe use of SVM that jointly with the morpholog-ical operators techniques, showed to be efficientand robust techniques for boundaries detection.Considering also the use of the SVM in classifi-cation problems of multiple classes from methodone-versus-all, together with the position descrip-tors and shape, we observed that this approachalso shown satisfactory results to teeth recog-nition. The system demonstrates that machinelearning with digital image processing techniquescan assist the dentist in carrying out epidemio-logical surveys in public oral health and can con-tribute to the collection of oral health informationfrom a population effectively.

In the future the intelligent system can be im-proved and modified by inserting new restrictiveelements, such as information as caries, periodon-tal disease, malocclusion, prosthesis, fluorosis, andoral lesions. Therewith more information aboutthe oral situation of the population can be ob-tained allowing a local planning increasingly opti-mized and efficient facing the principle of fairness.

An intuitive next step will be to use the deepconvolutional neural networks to attack this prob-lem and possibly observe better outcomes.

Acknowledgment

This work was funded by the National Coun-cil for Scientific and Technological Development(CNPq).

The authors would like to thank the odonto-logic department of the Federal University of RioGrande do Norte - UFRN for the idealization ofthis work and all assistance provided.

References

Acharya, T. and Tsai, P.-S. (2005). JPEG2000Standard for Image Compression: Concepts,Algorithms and VLSI Architectures, 1 edn,Wiley.

Chih-Wei Hsu, C.-J. L. (2002). A comparison ofmethods for multiclass support vector ma-chines, pp. 415–425.

Choorat, P., Chiracharit, W. and Chamnongthai,K. (2011). A single tooth segmentation us-ing structural orientations and statistical tex-tures, 4th Biomedical Engineering Interna-tional Conference pp. 294–297.

Gottlieb, R., Sarrett, D., Belle, A., Najarian, K.and Hobson, R. (2014). An Automated Den-tal Caries Detection and Scoring System forOptical Images of Tooth Occlusal Surface,pp. 1925–1928.

Kaick, O. V., Hamarneh, G., Ward, A. D.,Schweitzer, M. and Zhang, H. (2010). Learn-ing Fourier Descriptors for Computer-AidedDiagnosis of the Supraspinatus 1, 1: 1–33.

Mahsa Sepehrian, Ali M. Deylami, R. A. Z.(2013). Individual Teeth Segmentation inCBCT and MSCT Dental Images Using Wa-tershed, 20th Iranian Conference on Biomed-ical Engineering pp. 27–30.

Meyer, F. (1994). Topographic distance and wa-tershed lines, Signal Processing 38: 113–225.

Soille, P. (2002). Morphological Image Analysis, 2edn, Springer.

Vapnick, V. (1995). The nature of statistical learn-ing theory, Springer-Verlag .



1752

Date post:	23-Nov-2018
Category:	Documents
Upload:	vukhanh
View:	223 times
Download:	0 times

USE OF SUPPORT VECTOR MACHINE FOR TEETH … · USE OF SUPPORT VECTOR MACHINE FOR TEETH RECOGNITION...

Documents