Contributions in Computer Assisted Diagnosis: BreastCancer and Autoimmune Diseases
Praful AgrawalIIITD-MTech-CS-PHD-11-004
April 6, 2014
Indraprastha Institute of Information TechnologyNew Delhi
AdvisorsMayank VatsaRicha Singh
Submitted in partial fulfillment of the requirementsfor the Degree of M.Tech. in Computer Science and Engineering
Keywords: Mammography, Saliency, Autoimmunity, Connective Tissue Diseases, Breast Cancer,Indirect Immunofluorescence, and Laws Texture Features.
c©2014 Praful AgrawalAll rights reserved
Certificate
This is to certify that the thesis titled “Contributions in Computer Assisted Diagnosis: Breast Cancerand Autoimmune Diseases” submitted by Praful Agrawal for the partial fulfillment of the requirementsfor the degree of Master of Technology in Computer Science & Engineering is a record of the bonafidework carried out by him under our guidance and supervision in the Image Analysis and Biometrics groupat Indraprastha Institute of Information Technology, Delhi. This work has not been submitted anywhereelse for the reward of any other degree.
Dr. Mayank Vatsa and Dr. Richa SinghIndraprastha Institute of Information Technology, New Delhi
Abstract
With the advent of research, it has been established that many leading diseases among women, such asbreast cancer, cervical cancer, and autoimmune diseases, can be prevented if diagnosed at initial stage.This research aims at development and analysis of computer assisted systems for accurate diagnosisof such diseases. Among various diseases, this thesis focus upon developing automated systems forscreening breast cancer and autoimmune diseases.
Significant research efforts are being made to detect breast cancer symptoms on screening mammograms,however, mass detection has been the most challenging task. The complexity of the task is attributed tovarying shape and size of masses and presence of artifacts and pectoral muscles. In this research, wepursue the idea of visual saliency and propose a novel framework to detect mass(es) from screeningmammogram(s). The concept of visual saliency is based properties of human vision, therefore, it mayhelp in performing the ”intuitive” tasks which human eye perform with ease such as finding the regionof interest. We use the saliency algorithm to segment candidate regions which may contain masses. Thequalitative analysis shows that saliency algorithm is capable of detecting mass containing regions withoutany prior segmentation of pectoral muscles. Extensive feature analysis is performed to obtain the optimalset of features to detect masses using Support Vector Machine based classification. Experiments areconducted on publicly available MIAS database using existing protocols. Results from the comparativeanalysis show that the proposed framework outperforms the state-of-art algorithms.
Identification of antigen patterns from HEp-2 cells is crucial for the diagnosis of autoimmune diseases.The manual inspection under microscope as well as computer screens is prone to inter-observer vari-ability and lack of standardization. Therefore, efforts are being made to automate the antigen patternclassification from HEp-2 cell images. In this research, we propose a feature categorization to ana-lyze the existing research associated with HEp-2 cell image classification. We also propose an efficientclassification system for antigen pattern identification based on Laws texture features. Experiments areconducted using public datasets and existing protocols. Comparison with state-of-the-art techniquesclearly indicate that Laws texture features are more efficient for the given task.
Acknowledgments
Towards the completion of my Masters degree, I would like to pay my heartily tributes to people whocontributed in many ways. Many companions have witnessed my tough as well as cherishing momentsthroughout the span of two and half years in IIIT-Delhi. This dissertation is only a part of learning andenormous experience achieved in due course.
After expressing gratitude towards God and my loving parents, I would like to thank my advisors Dr.Mayank Vatsa and Dr. Richa Singh for their support and guidance throughout the journey. Their constantguidance and inputs have helped me prosper towards a more confident and improved personality. Theymade best of their efforts in supporting me through various possible ways. Their advise has alwaysserved me gain more knowledge and selecting better options. I would like to thank my undergraduateadvisor Prof. Sanjay Goel, his remarkable vision and kind guidance has been and will remain a constantsource of motivation. Also, I would like to thank Dr. Rahul Purandare, Dr. Debajyoti Bera, and Dr.Gaurav Gupta for giving me the cherishable moments at IIIT-Delhi.
I would like to acknowledge the organizers of MIAS and DDSM databases as well as the HEp-2 celldatabases - MIVIA and ICIP 2013 cell competition. The present dissertation could not have been possiblewithout the public databases being swiftly available for research. I thank and dearly appreciate theresearchers releasing their databases for research purpose.
Other than my parents, my brother has always been a constant companion with unconditional love andsupport for which any expression of thanks does not suffice. I would like to give special mention toShruti for being there as a constant source of inspiration and motivating me in the worst as well as besttimes of my life. I would also like to thank all my friends and companions from IIIT-Delhi who havemade my stay dearly memorable and cherishable especially, Denzil, Damodaram, Himanshu, Shiva,Paridhi, Anupama, Monika, Dipto, Ankita, Aditi, Kuldeep, Pandarasamy, Trasha, Anuj, Tejas, Anush,and Ayushi. This section can not be complete without a vote of thanks to the staff at IIIT-Delhi whichhas always been very helpful in many ways. In the end, I would like to once again thank the Almightyfor adding this wonderful chapter to my life.
i
Dissemination of Research Results
1. AGRAWAL, P., VATSA, M., AND SINGH, R. HEp-2 cell image classification - A computationalreview. Under Review.
2. AGRAWAL, P., VATSA, M., AND SINGH, R. Saliency based mass detection from screeningmammograms. Signal Processing 99, (2014), 29-47.
3. AGRAWAL, P., VATSA, M., AND SINGH, R. HEp-2 Cell Image Classification: A ComparativeAnalysis. In Machine Learning in Medical Imaging (2013), pp. 195-202.
ii
Contents
1 Introduction 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Medical Image Analysis for Screening and Diagnostic Purposes . . . . . . . . . . . . . 3
1.3 Research Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Saliency based mass detection from screening mammograms 6
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1.1 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1.2 Research Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2 Proposed Framework for Mass Detection . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2.1 Pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2.2 Saliency based ROI Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2.3 Grid based Sampling of ROI . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2.4 Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2.5 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3 Results and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3.1 Segmentation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3.2 Feature Extraction and Classification Results . . . . . . . . . . . . . . . . . . . 22
2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3 HEp-2 cell image classification using Laws texture features 29
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.1.1 Diagnostic tests for Autoimmune Diseases . . . . . . . . . . . . . . . . . . . . 30
3.1.2 ANA Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.1.3 Research Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.2 Pattern identification from IIF slides . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.3 HEp-2 Cell Image Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3.1 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.3.2 Databases and Existing Results . . . . . . . . . . . . . . . . . . . . . . . . . . 37
iii
3.4 Proposed Laws Texture Features Based HEp-2 Cell Image Classification . . . . . . . . . 40
3.4.1 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.4.2 Experimental Database and Protocol . . . . . . . . . . . . . . . . . . . . . . . . 41
3.4.3 Results and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4 Conclusion and Future Directions 46
5 Appendices 57
5.1 Fourier Domain Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.2 Intensity Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.3 Laws Texture Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
5.4 Statistical Texture Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.5 Run Length Texture Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
iv
List of Figures
2.1 Sample images: (a) normal mammogram, (b) cluster of micro-calcification, (c) a part ofmammogram containing mass, and (d) left and right mammograms of the same patientshowing a case of bilateral asymmetry. Source: MIAS Database [92]. . . . . . . . . . . 7
2.2 Pectoral muscles indicated on a MLO view mammogram containing masses. . . . . . . . 8
2.3 Sample of mass regions from the MIAS database [92]. . . . . . . . . . . . . . . . . . . 8
2.4 Illustrating the steps involved in the proposed framework. . . . . . . . . . . . . . . . . . 13
2.5 Low contrast mammogram with occlusion and noisy background. . . . . . . . . . . . . 13
2.6 Steps involved in pre-processing of mammograms. . . . . . . . . . . . . . . . . . . . . 14
2.7 (a) Enhanced image without cropping, (b) saliency map without cropping, (c) enhancedimage after cropping, and (d) saliency map pertaining to cropped and enhanced image. . 14
2.8 Saliency map generated from enhanced image and suspicious regions obtained afterthresholding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.9 (a) Circular regions extracted from ROI and (b) representation of each circle for featureextraction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.10 Sample results of the proposed saliency based segmentation algorithm. (a) Successfulsegmentation and (b) false segmentation. . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.11 Sample results of the saliency algorithms. Each row corresponds to the output of thefour saliency algorithms for corresponding mammogram image shown in the left mostcolumn. The green color in saliency maps denotes the ROI segmented after thresholdingon the saliency map and pink color represents the ground truth region containing mass. . 23
2.12 ROC curves for the individual sets of features (Best viewed in color). . . . . . . . . . . . 24
2.13 ROC curves for different mother wavelets with combined features from DWT and RDWT(Best viewed in color). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.14 ROC curves for different combinations of feature sets (Best viewed in color). . . . . . . 28
3.1 Steps involved in an automated system for diagnosis of IIF images. . . . . . . . . . . . . 32
3.2 A sample of positive and intermediate intensity IIF images. . . . . . . . . . . . . . . . . 32
3.3 Segmentation mask for corresponding IIF image. . . . . . . . . . . . . . . . . . . . . . 33
3.4 Sample images from the ICIP 2013 cell image classification contest training dataset [48].The cell pattern types of images from left to right - Centromere, Golgi, Homogeneous,Nucleolar, Nuclear Membrane, and Speckled. . . . . . . . . . . . . . . . . . . . . . . . 34
3.5 Diagrammatic representation of proposed Laws texture features based HEp-2 cell imageclassification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
v
3.6 Comparison results using 10 fold cross validation on ICIP 2013 and MIVIA datasets.Left hand side graphs correspond to results on ICIP 2013 cell image classification contesttraining dataset [48] and the right hand side graphs present results on MIVIA dataset [32]. 45
5.1 Feature masks generated using the kernels proposed by Laws [59]. . . . . . . . . . . . . 59
vi
List of Tables
1.1 Age standardized adult mortality rate by cause during 2008 as published in WHO 2013report [1]. The numbers reported are per 100,000 population in the corresponding regionsfor people with ages 30-70 years. Minimum, median, and maximum values are alsoreported as observed from data for all WHO member nations. . . . . . . . . . . . . . . . 1
1.2 World statistics on physicians (including specialist and general physicians) as well thenecessary infrastructure to conduct image based diagnosis in corresponding regions [1].Minimum, median, and maximum values are also reported as observed from data for allWHO member nations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.1 Summary of key existing techniques for mass detection. . . . . . . . . . . . . . . . . . . 9
2.2 Kernels proposed by Laws [59]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.3 Symptom wise description of the MIAS database. . . . . . . . . . . . . . . . . . . . . . 21
2.4 Comparative analysis of saliency algorithms. . . . . . . . . . . . . . . . . . . . . . . . 22
2.5 Classification results of individual sets of features. . . . . . . . . . . . . . . . . . . . . . 24
2.6 Comparing the performance of DWT and RDWT entropy features with different motherwavelets. The results are reported in terms of Area Under the Curve (Az) of ROC curves. 25
2.7 Analyzing the effect of mRMR based feature selection on individual feature sets. Perfor-mance of individual set of features after mRMR based feature selection. . . . . . . . . . 26
2.8 Analyzing classification performance with different combinations of feature sets. . . . . 27
2.9 Comparing the performance of proposed algorithm with existing algorithms on the MIASdatabase. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.1 Review of existing literature based on the proposed feature categorization and classifica-tion techniques used for HEp-2 cell image classification. . . . . . . . . . . . . . . . . . 35
3.2 Summary of the MIVIA dataset [32] and training dataset provided during ICIP 2013 cellimage classification contest [48]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.3 Overall cell classification accuracy (%) reported in existing literature on MIVIA dataset. 38
3.4 This table presents the number of correctly classified cell images for each IIF image asreported by existing techniques using leave-one out cross validation protocol on MIVIAdataset. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.5 Comparison results for all the baseline features using different protocols on MIVIAdataset. Overall cell classification accuracy values are reported for all feature and clas-sifier combinations on the three protocols. On the lines of existing literature, all threeprotocols are applied on combined (both positive and intermediate) dataset. . . . . . . . 44
vii
viii
Chapter 1
Introduction
1.1 Introduction
According to 2013 report by World Health Organization (WHO) [1], worldwide leading causes of death
include cancer, cardiovascular diseases, diabetes, and chronic respiratory disorders. Table 1.1 presents
recently published statistics on age standardized adult mortality rate by causes in the year 2008. During
recent decades, diagnosis of these diseases heavily depend on the medical imaging technologies such as
X-Ray scan, Magnetic Resonance Imaging (MRI), Computed Axial Tomography (CAT), Intra-Vascular
Ultra Sound (IVUS), Ultrasonography (USG), Electrocardiography (ECG), and microscopic imaging.
These image based techniques are being used during various phases of treatment, starting from early
screening to advanced surgery. Such an advent in medical technology is helping clinicians in accurate
diagnosis and conducting efficient treatments.
Table 1.1: Age standardized adult mortality rate by cause during 2008 as published in WHO 2013 report [1].The numbers reported are per 100,000 population in the corresponding regions for people with ages 30-70 years.Minimum, median, and maximum values are also reported as observed from data for all WHO member nations.
Region All causes Cancer Cardiovascular dis-eases and diabetes
Chronic respira-tory conditions
African Region 1716 147 382 92Region of the Americas 532 136 169 24South-East Asia Region 987 125 322 109European Region 626 166 238 17Eastern Mediterranean Region 881 127 344 46Western Pacific Region 545 168 184 41India 1002 108 328 133Global 764 150 245 52Minimum 220 59 59 2Median 774 140 284 29Maximum 3147 284 1427 195
1
Experts have suggested that some of the incurable diseases, such as breast cancer and cervical cancer, can
be prevented if detected at initial stages [80]. WHO has suggested screening programs primarily based on
medical imaging technologies to prevent large number of deaths caused due to such diseases. In addition,
routine screening is recommended for diseases with high prevalence. Screening programmes have been
found effective in reducing the mortality rates by 30% [16]. Generally, during a screening test, an image
of internal body part is captured and the aim is to find any symptoms which may indicate prospective
incidence of the disease. These medical images are examined by experts and diagnosis is suggested
based on the findings from the captured images. With increasing number of screening and diagnostic
tests based on imaging technologies, abundant number of medical images are being captured. Since the
examination of these images is crucial for the diagnosis, there are important issues to be addressed:
• There is an amount of subjectivity associated with the findings reported by the examining expert.
It has been found that two different experts can suggest different findings from a common sample
image. This mismatch in opinion is essentially due to the difference in their level of expertise and
lack of adequate experience required for accurate examination of medical images [6].
• A large number of images being captured requires a number of human experts. Even in most
developed economies such as USA and Europe, the number of available experts is significantly
lower. Therefore, in most of the laboratories, experts have to examine large number of samples on
daily basis which may affect the overall diagnosis. Table 1.2 presents the number of physicians in-
cluding specialist physicians as well as general physicians and the data on corresponding essential
health infrastructure available worldwide.
Table 1.2: World statistics on physicians (including specialist and general physicians) as well the necessary infras-tructure to conduct image based diagnosis in corresponding regions [1]. Minimum, median, and maximum valuesare also reported as observed from data for all WHO member nations.
Region Physicians per10,000 popula-tion (2005-2012)
Computed tomogra-phy units per 1,000,000population (2010)
Radiotherapy unitsper 1,000,000 popu-lation (2010)
African Region 2.5 0.4 0.1Region of the Americas 20.4 NA 5.4South-East Asia Region 5.5 NA 0.3European Region 33.3 NA 4Eastern Mediterranean Region 10.8 1.9 0.3Western Pacific Region 15.2 NA 1.6India 6.5 NA 0.4Global 13.9 NA 1.8Minimum 0.1 <0.05 <0.05Median 14.2 3.8 0.7Maximum 70.6 141.2 28.2
2
1.2 Medical Image Analysis for Screening and Diagnostic Purposes
Efforts are being made to evolve the computer aided systems for screening and diagnostic purposes.
Such systems would eradicate the subjectivity in reported findings and fasten the treatment process. Re-
searchers have been trying to develop the automatized procedures using the tools from computer vision,
pattern recognition and machine learning [25], [40]. Other than automation of image interpretation, sig-
nificant research has been done to develop robotic tools to automate the complex and time consuming
tasks in diagnosis. The computer based image analysis systems in combination with robotic tools are
also being used during several operated surgeries. Largely, automated procedures in medical diagnosis
are dependent on accurate image based techniques. The usefulness of image analysis in different stages
of medical treatment process can be categorized as:
• Enhancement: Medical images are captured by large cameras which require multiple mechanical
calibrations for an accurate and clear image capture. However, due to various practical purposes,
most of the times the resultant image contains noise such as out of focus and salt and pepper
noise. Therefore, the medical images are required to be processed with a set of image processing
techniques to enhance or restore the image for meaningful interpretation and diagnosis.
• Segmentation: Medical images are generally captured for diagnosis of a specific organ or region,
however, the viewpoint of a sophisticated camera system should be able to capture larger area.
Therefore, it is seldom required to carve out the organ or area of interest from captured images.
In a manual inspection, doctors are aware of the region of interest and human eyes are capable of
easily segmenting the required region. However, in case of automated analysis, segmentation of
region of interest is one among the most complex tasks especially due to the surrounding tissues
which collude with the delineating boundary. Sophisticated algorithms based on image processing
and computer vision are being developed to segment the region of interest from medical images.
• Registration: Nowadays, complicated surgeries are being conducted under the purview of sophis-
ticated camera systems where sometimes a tiny camera is inserted into the human body to estimate
the ongoing process. In many other cases, multiple cameras are utilized to obtain the internal im-
ages. In all such cases, images are simulated and registered with respect to an atlas model. Image
processing and linear algebra based techniques have been developed to accurately align the cap-
tured images with the existing model. In case of computer assisted surgeries, accurate estimation
of the position of inserted tools with respect to the organ is important for a successful surgery.
• Visualization: With emerging technologies, more complex image modalities are being used for
diagnosis such as Functional Magnetic Resonance Imaging, Photoacoustic imaging, Echocardiog-
raphy, and Functional Near Infrared Spectroscopy. Such complex images have immense informa-
tion to be observed and analyzed by doctors to suggest conclusive diagnosis. Therefore, in order
to present the highly complex information visualization algorithms are developed. These algo-
rithms project the high dimensionality image on to a computer screen for better visualization as
well as they may also indicate important regions to be investigated. Visualization algorithms are
also crucial in projecting the run time images captured during computer assisted surgeries.
3
• Fusion: In current scenario, images from multiple modalities are required to diagnose complex
diseases. Images captured using various technologies help doctors understand different aspects of
the symptoms. Since these images convey different information, they are examined independently,
and seldom required to model the complete information. Therefore, image fusion algorithms are
being developed to combine the information from different modality images.
• Storage: Large number of medical images are being captured for diagnosis in several ways. It is
essential to store this humongous data for several important purposes such as to conduct research,
to demonstrate important findings to trainees and inexperienced doctors, to assess the impact of
treatment and to provide a follow up diagnosis based on historical data. However, two important
issues need to be addressed while storing such a valuable data: vast storage space required to
save the data, variable formats and the security of data. In order to efficiently store large medical
issues, lossless image compression techniques have been developed. Also, a meta data based
image format DICOM (Digital Imaging and Communications in Medicine)1 has been standardized
to bring images of different modalities to a common structure. This standardization is also helpful
in communicating different modality images across hospitals. The security of medical information
is crucial as it may contain sensitive personal information. Therefore, watermarking techniques
are being utilized to encode the identity of patient to make it accessible only to legitimate users.
In order to develop efficient computer based models, tools from pattern recognition and machine learning
are also required. Most of the automated systems rely on combination of image processing and decision
making techniques to suggest a meaningful diagnosis. Such systems being used at several stages of med-
ical treatment has motivated researchers to develop efficient techniques. The proposed techniques need
to be thoroughly evaluated on existing data in order to establish their clinical utility. As a result, most of
the exciting research being conducted is validated and thoroughly evaluated by existing benchmarks and
public datasets.
1.3 Research Contributions
Worldwide, leading causes of death among women include breast cancer, cervical cancer, autoimmune
diseases, arthritis, anemia, heart disease, and osteoporosis. Efforts are being made to generate awareness
among women to consult physicians as soon as they experience any symptoms. Moreover, screening
programs have been started by various countries to detect most prevalent diseases such as breast cancer
and cervical cancer. Though such programs are essential and have proven to be effective, they still require
human experts to analyze the image based medical data and propose a relevant diagnosis. Therefore,
computer based detection/diagnosis systems are being developed to automate the role of human experts.
In this thesis, we focus on two aspects: breast cancer screening and diagnosis using mammograms and
HEp-2 (human epithelial type 2) cell classification. The research contributions in this thesis can be
summarized as:1http://dicom.nema.org/
4
• Screening mammography has been successful in early detection of breast cancer, which has been
one of the leading causes of death for women worldwide. Among commonly detected symptoms
on mammograms, mass detection is a challenging problem as the task is affected by high complex-
ity of breast tissues, presence of pectoral muscles as well as varying shape and size of masses. In
this research, a novel framework is proposed which automatically detects mass(es) from mammo-
gram(s) even in the presence of pectoral muscles. The framework uses saliency based segmentation
which does not require removal of pectoral muscles, if present. From segmented regions, different
features are extracted followed by Support Vector Machine classification for mass detection. The
experiments are performed using an existing experimental protocol on the MIAS database and the
results show that the proposed framework with saliency based region segmentation outperforms
the state-of-art algorithms.
• Indirect Immunofluorescence (IIF) testing is the state-of-art procedure to detect antinuclear au-
toantibodies. However, due to lower level of standardization and automation, large number of
experts are required to analyze the test images. In this research, we focus on automated identifica-
tion of antigen patterns from HEp-2 cell images. Recently, many techniques have been proposed
to automate this task. We propose a feature categorization based on different object properties and
analyze the existing literature based on this categorization. We also propose to utilize Laws texture
features for the HEp-2 cell image classification. Experimental analysis is performed to evaluate
and compare the performance of Laws texture features with other widely used image based fea-
tures. Comparative results on two public datasets using different protocols conclude that Laws
texture features are efficient for the given task.
Chapters 2 and 3 present the proposed computer aided systems to automate the diagnosis of breast can-
cer and autoimmune diseases respectively using image interpretation and decision making algorithms.
Finally, Chapter 4 summarizes the contributions of the thesis with discussion on possible future research
directions.
5
Chapter 2
Saliency based mass detection fromscreening mammograms
2.1 Introduction
For the past few decades, cancer has been a major cause of deaths worldwide. World Health Organization
(WHO) has predicted that the number of deaths due to cancer will increase by 45% from 2007 to 2030.
In 2030, new cases of cancer are expected to reach 15.5 million from 11.3 million reported in 20071.
Detailed statistics from WHO show that breast cancer accounts for the maximum number of newly
reported cases and deaths caused due to cancer among women worldwide. The symptoms of breast
cancer can be detected with the help of X-ray of breasts, termed as mammograms. Experts have suggested
that the best possible cure for breast cancer is the early prognosis of these symptoms.
With increasing awareness about the disease, thousands of mammograms are captured annually in the
screening centers. Even in most of the developed economies such as USA and Europe, there is a mis-
match between the number of experts available and the number that is required to analyze the screening
mammograms. Due to the lack of medical experts required to analyze the screening mammograms,
scientists have realized the role of Computer Aided Detection/Diagnosis systems to automate the diag-
nosis of screening mammograms. Computer Aided Detection systems (CADe) are used to detect early
symptoms from medical images and Computer Aided Diagnosis systems (CADx) are used to diagnose
abnormal findings. Some studies [4], [13], [14] claim that the use of CAD systems has actually improved
the diagnosis of breast cancer, however other experiments [76] do not agree with these results. Amid such
varied results, scientists emphasize on using CAD systems as secondary opinion for doctors.
For breast cancer prognosis, a CADe system detects early symptoms from screening mammograms. As
shown in Figure 2.1, the commonly detected symptoms on mammograms include
• clustered micro-calcification,
• mass,1http://www.who.int/features/qa/15/en/index.html, last accessed on June 12, 2013.
6
( a ) ( b ) ( c )
( d )
Figure 2.1: Sample images: (a) normal mammogram, (b) cluster of micro-calcification, (c) a part of mammogramcontaining mass, and (d) left and right mammograms of the same patient showing a case of bilateral asymmetry.Source: MIAS Database [92].
• architectural distortion, and
• bilateral asymmetry.
Clustered micro-calcification is a very prominent symptom observed in new cases of breast cancer. As
illustrated in Figure 2.1, they appear as a group of white dots on a mammogram. A mass is defined as
a space-occupying lesion seen in more than one projection [93]. They vary in size and shape [71], and
therefore are difficult to detect in mammograms. Figure 2.1 illustrates a case with cancerous mass present
in a mammogram. Architectural distortion and bilateral asymmetry are not very common symptoms and
multiple images are required to detect these symptoms. For example, architectural distortion can be
detected by comparing the screening mammograms captured before and after the distortion is developed.
Bilateral asymmetry corresponds to any significant difference in the tissue structure between the two
breasts (left and right). Figure 2.1 shows a sample case of bilateral asymmetry.
Mammograms are generally captured from multiple views to collect as much information as possible be-
fore detection/diagnosis. Some widely captured views are Carnio-caudal (CC) and Multi-lateral Oblique
(MLO). A CC view mammogram is taken from above a horizontally-compressed breast and shows as
much as possible of the glandular tissue, the surrounding fatty tissue, and the outermost edge of the
chest muscle. On the other hand, a MLO view mammogram is captured from the side at an angle of
a diagonally compressed breast thus allowing more of the breast tissue to be imaged as compared to
other views. However, with this procedure, pectoral muscles also appear in MLO view mammograms.
As shown in Figure 2.2, these pectoral muscles have high intensity values in the mammogram and may
be misinterpreted as mass. Therefore, algorithms generally involve removing the pectoral muscles or a
segmentation step to suppress pectoral muscle region followed by detection of symptoms. Even with the
7
availability of multiple views, only few automatic mammogram analysis algorithms combine informa-
tion and perform case based analysis using multiple views simultaneously [98]. Most of the algorithms
consider and process each view as an individual image.
Pectoral Muscles
Masses
Figure 2.2: Pectoral muscles indicated on a MLO view mammogram containing masses.
Many sophisticated algorithms have been proposed to detect the symptoms from screening mammo-
grams [37], [79], [80]. As shown in Figure 2.1, there are multiple symptoms of breast cancer. While
efficient algorithms for detection and diagnosis of micro-calcification exist [42], the same is not true for
masses [86]. Detection and diagnosis of masses in a mammogram are challenging problems due to the
varying size, shape, and appearance of masses as well as varying tissue density (Figure 2.3). Detection
and diagnosis corresponds to two different stages of the complete diagnosis. Detection refers to finding
the possible location(s) of mass in the complete mammogram, while diagnosis is the final step of finding
the exact boundaries of the mass present and/or classify the segmented mass as benign or malignant [71].
Even though there are algorithms that can address both the problems simultaneously, researchers gen-
erally try to solve them independently. Successful detection is crucial for meaningful diagnosis and
therefore, in this research, we focus on mass detection from screening mammograms.
( a ) ( b )
( c ) ( d ) ( e ) ( f )
Figure 2.3: Sample of mass regions from the MIAS database [92].
8
2.1.1 Literature Review
In literature, several researchers have proposed algorithms to detect masses in mammograms. Table 2.1
summarizes some of the key contributions for mass detection. Vyborny and Giger [99] discussed the
algorithms to detect abnormalities such as mass and micro-calcification from mammograms using CAD
systems. They concluded that for a fair comparison, the algorithms should be evaluated on a public
database. However, in the absence of any such database, existing algorithms compare their results di-
rectly with the radiologist’s assessment. In the same year, Mammographic Image Analysis Society [92]
released a public database comprising 322 digitally scanned film based mammograms. Later, another
database known as the Digital Database for Screening Mammography (DDSM) [45] was published for
research purpose. These databases and few others have helped the researchers to identify and solve some
key problems associated with the development of CAD systems for screening mammograms. Cheng et
al. [20] summarized the existing algorithms for one of the key challenges in screening mammograms
i.e, detection and classification of masses. They discussed the various steps involved in an automated
approach and the key processing algorithms proposed in literature for modeling these steps. They also
compared the results of different classifiers with radiologist’s performance on the Nijmegen and DDSM
databases. Tang et al. [93] analyzed the advent of CAD systems using mammography in a more generic
way and focused on computer aided detection of four major symptoms in screening mammograms. A
categorical review of existing algorithms was presented and the authors suggested that CAD systems
could assist radiologists in validating their assessment. They also suggested that further improvement
is required if such systems are to be used in an independent manner. Oliver et al. [71], in their recent
survey, analyzed existing algorithms according to various computer vision paradigms used for segmen-
tation of suspicious regions and the features used to model the variations in normal and cancerous tissue
patterns. They also compared seven key mass detection approaches on a common set of images using
one experimental protocol and observed that in general, the performance varies with shape and size of
masses as well as with the density of breast tissues.
Table 2.1: Summary of key existing techniques for mass detection.
Technique Description Database Results
Lai et
al. [58] 1989
Adaptive thresholding is used to de-
tect suspicious regions from enhanced
image. Template matching is used
to identify masses from candidate re-
gions.
Training: 7 images
with 11 masses. Test-
ing: 17 images with 19
masses.
100% TP @ 1.1
FP/image.
Petrick et
al. [75] 1996
Edge detection is applied to find the
boundary of the objects highlighted in
DCWE filtered image. Morphological
features are used to classify objects as
mass or non-mass.
25 images containing
one mass each.
96% TP @ 4.5
FP/image.
Continued on next page
9
Table 2.1 – continued from previous pageTechnique Description Database Results
Karssemeijer
et al. [53]
1996
Pixel orientation map is obtained with
the help of three dimensional second
order Gaussian derivative operators.
Orientations at a scale with maximum
response of Gaussian operators are se-
lected for further processing. Two sta-
tistical features are computed from the
orientation map to detect stellate pat-
terns.
31 normal breast im-
ages, 9 images with
malignant stellate le-
sions and 10 images
with architectural dis-
tortions are used from
MIAS database.
90% TP @ 1
FP/image.
Polakowski
et al. [77]
1997
Difference of Gaussian smoothed im-
ages followed by thresholding is used
to segment Regions of Interest (ROI)
from pre-processed image. Texture
features based classification is used to
classify segmented ROI as mass or
non-mass.
Total 272 images used
with 36 malignant and
53 benign cases.
92% accuracy for
locating masses.
100% TP @ 1.8
FP/image.
Eltonsy et
al. [28] 2007
Multiple concentric layer model is de-
signed to detect the suspicious regions
in a mammogram. Minimum distance
between two segmented regions and
probability of presence of a suspicious
region in the given mammogram are
used to reduce the false accepts.
Training: 135 malig-
nant cases.
Testing: 135 malig-
nant, 82 normal and
135 benign cases. All
images from DDSM
database.
92% TP @ 5.4 and
5 FP/image respec-
tively for malignant
and normal cases,
61.6% TP @ 5.1
FP/image for be-
nign cases.
Kom et
al. [54] 2007
Image enhanced by linear filter is sub-
tracted from original image to obtain
the suspicious regions. Local Adap-
tive Thresholding is applied to the sub-
tracted image to obtain the detected
masses.
34 images containing
49 lesions and 27 nor-
mal images used.
95.91% TP @ 15%
FPR.
Varela et
al. [97] 2007
Iris filter is applied in a multiscale fash-
ion followed by adaptive thresholding
is used to segment the suspicious re-
gions. Gray level, texture, contour,
and morphological features are used
for classification.
66 malignant and 49
normal cases used
with 4 images per
case.
88% and 94% TP
@ 1.02 FP/image
for per image and
per case evaluation
respectively.
Continued on next page
10
Table 2.1 – continued from previous pageTechnique Description Database Results
Hong and
Sohn [50]
2010
Suspicious regions are modeled with
the help of a topographic representa-
tion termed as isocontour maps. A hi-
erarchial representation, inclusion tree
is used to capture the relationship be-
tween contours and minimum nesting
depth is used to identify the masses.
400 images selected
from DDSM database.
100% TP @ 3.8
FP/image.
Gao et
al. [39] 2010
Combination of Morphological Com-
ponent Analysis (MCA) and Concen-
tric Layer Model is used for mass de-
tection.
Training: 40 malig-
nant and 10 benign
cases.
Testing: 100 malig-
nant and 50 benign
cases. All images from
DDSM database.
99% TP @ 2.7
FP/image for ma-
lignant cases and
88% TP @ 3.1
FP/image for be-
nign cases.
Mencattini
et al. [65]
2010
Suspicious regions are detected based
on measure of convergence of gradi-
ent vectors in a circle from center pixel
in a pre-processed image. Parametric
thresholding is used as false positive re-
duction.
136 images selected
from DDSM database
containing one mass
each.
100% TP @ 5
FP/image.
Sampaio et
al. [8] 2011
Cellular Neural Networks are used to
detect the suspicious regions in a pre-
processed image. Texture features
derived from geostatic functions and
shape features are used for classifica-
tion.
623 images randomly
selected from DDSM
database.
80% TP @ 0.84
FP/image.
Some commercial systems such as ImageChecker CAD2 and SecondLook Digital3 are also available and
are nearly accurate in detecting micro-calcifications, however the accuracies of mass detection require
improvement [86]. Based on our observation, following are the key challenges in designing an efficient
mass detection algorithm:
• Existing databases consist of noisy mammograms. Noise in mammograms is mainly due to the
old film based X-rays, such as mechanical noise in the image background and certain irregularities
such as tape markings and occlusions. Therefore, mammogram analysis starts with an overhead
step of image cleaning, which generally involves masking out the breast region. However, this2http://www.hologic.com/en/imagechecker-cad, last accessed on June 12, 2013.3http://www.icadmed.com/products/mammography/secondlookdigital.htm, last accessed on June
12, 2013.
11
masking disrupts the natural texture on the outer boundary of breast region thereby creating a hard
edge. Such hard edges can easily confuse the segmentation algorithms. In order to develop more
useful and efficient algorithms, such unwanted hard edges on the breast boundary must be diluted
before segmentation.
• In mass detection algorithms, pectoral muscle segmentation is an important step as the pres-
ence of pectoral muscles may increase false alarms generated by automatic segmentation algo-
rithms [38]. Though several researchers have proposed dedicated pectoral muscle segmentation
algorithms [12], [31], [38], [57], [67], it is still challenging to accurately segment these muscles
when the tissue density around the muscles is high. Therefore, it is important to design more robust
algorithms that do not require segmenting the pectoral muscle boundaries.
• False positive reduction is used to remove the falsely segmented regions. Existing algorithms
consider the segmented Regions of Interest (ROI) as a single entity for the false positive reduction
step. Very often these regions contain mass surrounded by normal tissue regions or mass present
in different orientations. Such regions can lead to ambiguity in feature values and therefore reduce
the performance.
• Features are generally derived from the properties of objects present in an image such as tex-
ture, shape, gradient, and intensity. Many such features have been proposed/used for mass detec-
tion [20], [71]. However, only subsets of these features have been used by researchers repeatedly.
There is no study that analyzes a comprehensive feature space in a common framework and deter-
mine their effectiveness.
2.1.2 Research Contribution
In this research, a framework for mass detection from mammograms is proposed which attempts to
answer the key issues in existing approaches as identified in the previous subsection.
• The pre-processing step in the proposed framework utilizes image blending to diffuse the hard
edges formed due to masking.
• Visual saliency is proposed to segment probable mass containing regions in a pre-processed mam-
mogram. One of the key findings of the study is that the saliency based segmentation is robust to
the presence of pectoral muscles.
• For improved mass detection, candidate regions obtained from visual saliency based segmentation
are examined. A grid based approach is utilized to examine the regions of interest and different
features are extracted, analyzed, and compared individually. Further, feature selection and dimen-
sionality reduction algorithms are investigated to obtain the optimal set of features. Using the
protocol of Oliver et al. [71] on the MIAS database, classification results show that the proposed
algorithm yields better performance than existing algorithms.
12
2.2 Proposed Framework for Mass Detection
In this research, a framework for detection of masses from screening mammograms is proposed. The
proposed framework uses visual saliency based segmentation and a set of optimal features to detect
mass in screening mammograms. Figure 2.4 illustrates the steps involved in the proposed framework -
pre-processing, ROI segmentation, grid based sampling of ROI, feature extraction, and classification.
Figure 2.4: Illustrating the steps involved in the proposed framework.
2.2.1 Pre-processing
The mammogram images are generally low on contrast and have noise in background such as tape mark-
ings and labels (as shown in Figure 2.5) which may affect the segmentation results. Therefore, contrast
enhancement and background segmentation are crucial pre-processing steps for a CAD system to ana-
lyze mammograms [56], [81], [88]. In this research, a set of completely automated pre-processing steps
are used to remove the background noise and enhance the image quality of mammogram images. The
pre-processing steps illustrated in Figure 2.6 are,
Figure 2.5: Low contrast mammogram with occlusion and noisy background.
• Masking: The first step of pre-processing estimates the breast boundary using a gradient based
approach proposed by Kus and Karagoz [56]. Adaptive global thresholding followed by histogram
stretching are used to compute the outline of breast region and generate the mask. To ensure that
any breast region is not missed, the mask is dilated using a circular structuring element of pixel
size five. Finally, the dilated mask, as shown in Figure 2.6, is used to remove the background
noise.
• Enhancement: In literature, several contrast enhancement techniques have been proposed for the
enhancement of mammograms [88]. In this research, adaptive histogram equalization is applied to
enhance the contrast of masked mammogram image [41]. Figure 2.6 shows the contrast enhanced
image thus obtained.
13
• Blending: In masking, the background region pixels are assigned zero intensity, which creates
hard edge along the border of the segmented region. This artificial hard edge leads to undue
saliency accumulation along the border. Therefore, we utilize the concept of image blending to
dissolve the hard edges. Gaussian-Laplacian pyramid [15] based image blending is used to blend
the masked image with the original image only along the outer breast boundary. The two images
are decimated into a Gaussian- Laplacian Pyramid upto seven levels. Blending is applied at each
level of the pyramid and finally the image is reconstructed.
• Cropping: Though image blending removes the hard edges at the outer boundary, it fails to dis-
solve the thick vertical edge on one side of the mammogram. Such edges are obtained when the
breast region lies in the middle of the image; an example is shown in Figure 2.7. To address this is-
sue, as shown in Figure 2.7, image is cropped such that the breast region is aligned to its respective
sides.
Preprocessed Image
Input Image Generated Mask
Masking
Enhancement
Masked Image
Enhanced Image
Mask Generation
Blending & Cropping
Figure 2.6: Steps involved in pre-processing of mammograms.
( a ) ( b ) ( c ) ( d )
Figure 2.7: (a) Enhanced image without cropping, (b) saliency map without cropping, (c) enhanced image aftercropping, and (d) saliency map pertaining to cropped and enhanced image.
2.2.2 Saliency based ROI Segmentation
After preprocessing, the region of interest (region where mass is expected) is to be segmented. The
anatomy of breast is a complex structure due to the presence of pectoral muscles as well as the varied
14
density of breast parenchyma. For an expert, it is easy to analyze breast tissues without getting confused
with pectoral muscles. However, for an automatic algorithm, it is difficult to differentiate between pec-
toral muscles and mass. Therefore, generally, pectoral muscles are removed before segmenting the ROI.
Automatic pectoral muscle segmentation is a difficult task, and it is also an overhead in processing the
mammograms without pectoral muscles such as Carnio-Caudal (CC) view mammograms. Therefore, in
this research, we propose the visual saliency based ROI segmentation in mammograms which does not
require the location of pectoral muscles at any time of processing to detect the ROI.
Visual saliency models the ability of humans to perceive salient features in an image. In computer vision,
visual saliency models are bottom-up techniques which emphasize on particular image regions such as
regions with different characteristics [10]. Visual saliency models can be classified as space-based or
object-based models. Object-based models assign higher saliency to the regions containing objects. On
the other hand, space-based models produce a saliency map of the input image. It is represented as a
probabilistic map of an image where the value at a pixel location corresponds to the saliency of that pixel
with respect to the surroundings. It can also be very useful in cases where some structures are implicit
with respect to the image such as pectoral muscles in mammograms. A simple visual saliency based
segmentation algorithm can be designed by applying thresholding on the saliency map (or probabilistic
map).
Several visual saliency based algorithms have been proposed in literature [9], [10]. Since the screen-
ing mammograms are gray scale images, algorithms which utilize the multi-channel properties of color
images may not be useful. Saliency algorithms which are capable of computing visual saliency from
a grayscale image such as Esaliency [2], GBVS [44], Hou and Zhang [51], and Liu et al. [63], are
considered to segment ROI from the pre-processed mammogram. Any of these algorithms can be used
for generating saliency maps, however, we experimentally observed that GBVS yields the best results.
Therefore, in this research, we use saliency maps generated by GBVS to extract the ROI.
GBVS computes saliency of a region with respect to its local neighbourhood using the directional con-
trast. Saliency computation is directly correlated with the feature map used. In screening mammograms,
it has been observed that the contrast of mass containing regions is significantly different from the re-
maining breast parenchyma. As discussed earlier, mass surrounded by dense tissues are tough to detect,
however, the directional contrast with respect to the local neighbourhood helps in identifying such masses
along with the masses present in fatty regions. It is to be noted that the fatty regions surrounded by dense
tissues may also get falsely detected, such regions are discarded at the final classification stage. The steps
involved in computing the saliency map are explained below4.
1. Mass differs in contrast from the neighboring regions, therefore feature maps are computed from
contrast values along four different orientations of 2D Gabor filters (0◦, 45◦, 90◦, and 135◦).
2. The saliency maps can be derived in a naıve way by squaring all the values in the feature map
obtained in the previous step. However, this naıve method can result in a large number of falsely
detected regions indicated as salient regions. Therefore, activation maps are computed as the4Please refer to the original paper by Harel et al. [44] for more details on GBVS algorithm.
15
equilibrium distribution of ergodic Markov chain [60], obtained using the initial feature maps. The
equilibrium distribution will lead to higher weights only for the edges present in salient regions.
Moreover, the higher edge weights initialised in regions other than salient regions will get diffused
in the equilibrium distribution. Ergodic Markov chains are modeled on a fully connected directed
graph obtained from feature maps. The graph is generated by connecting nodes in a feature map
using weighted connections. The weight of an edge connecting node (i, j) to node (p, q) in the
graph is assigned as,
w((i, j), (p, q)) , D · F (i− p, j − q), (2.1)
where, F (a, b) , exp
(−a
2 + b2
2σ2
)(2.2)
D ,
∣∣∣∣log(M(i, j)
M(p, q)
)∣∣∣∣ (2.3)
M(i, j) represents a node in the feature map and σ is set to 0.15 times the image width5.
3. Final step in saliency algorithms is generating saliency map from activation maps. However, in
general, some individual activation maps lack accumulation of weights in salient regions, therefore,
an additional step of normalization of activation map is performed to avoid uniform saliency maps.
GBVS normalize activation maps using a similar approach as used in the previous step, i.e. the
equilibrium distribution of ergodic Markov chain to accumulate high activation values in salient
regions. Markov chains are obtained from activation maps in a similar manner as discussed in
the previous step, however, the function D in Eq. 2.1 now maps to the value at location (p, q) in
activation map (Eq. 2.4) and value of the parameter σ in Eq. 2.2 is 0.06 times the image width5,
D , A(p, q) (2.4)
where A(p, q) represents a node in the activation map.
4. Finally, normalized activation maps are combined using sum rule to obtain the saliency map.
Once the saliency map is computed, a threshold equal to half of the maximum value6 in saliency map
is used to obtain the ROI. The regions containing less than 100 pixels are discarded for further analysis.
Figure 2.8 illustrates some examples of saliency maps generated from pre-processed images and ROI
segmentation from the saliency maps.
2.2.3 Grid based Sampling of ROI
Since GBVS algorithm extracts salient regions in a more generic manner, therefore, along with mass,
the segmented regions contain some normal tissues as well. The proposed framework utilizes a grid
based search to detect the mass containing regions and discard the falsely detected normal tissue regions.5 We empirically found that σ value suits all images in the MIAS database.6Threshold=0.5 is empirically selected to obtain the optimal size ROIs.
16
Figure 2.8: Saliency map generated from enhanced image and suspicious regions obtained after thresholding.
Within the bounding box covering the segmented ROI, a grid of seed points is marked with each point
marked 40 pixels apart in both horizontal and vertical directions. Overlapping circular regions of radii
varying from 30 pixels to 210 pixels are extracted using all the seed points and features are extracted
from these regions. The size of circular regions is selected to be varying from 30 pixels to 210 pixels
in order to model a generalized approach independent of mass size. Figure 2.9 illustrates an example of
extracting circular regions from the segmented ROI.
s1
ci
R
s2
Figure 2.9: (a) Circular regions extracted from ROI and (b) representation of each circle for feature extraction.
2.2.4 Feature Extraction
In this research, we have extracted a large number of features and clustered these features among three
different categories namely, Spatial domain features, Fourier domain features, and Wavelet entropy fea-
tures. The categorization is based on the information used to compute these features. Various features
belonging to these three categories have been explored by researchers for discriminating between mass
and non-mass regions. However, current literature lacks the understanding of individual features and
their combination. One of the key contributions of this research includes analysis of the large pool of
features in top-down as well as bottom-up approach to find the optimal set of features. This section
further explains the extracted features as well as the feature selection techniques utilized to obtain the
relevant feature set.
Fourier Domain Features
Spectral energy features have been used by researchers to differentiate mitotic nuclei from others present
in microscopic images of biopsy slides for breast cancer [11]. Fourier domain representation of the pixels
in ROI is used to compute the spectral energies which represent the energy in different frequency bands.
Spectral energies are calculated from the Fast Fourier Transform (FFT) of one-dimensional signal formed
17
by the pixels in the ROI, FFT is computed using Eq. 2.5. From this FFT representation, 35 energy values
are computed from different frequency bands as explained in Section 5.1.
FFT ROI =N−1∑n=0
x(n)e−j2πnk/N , k = 0, 1, ..., N − 1 (2.5)
where, x(n) represents the pixels of region R in spatial domain and N represents the total number of
pixels in region R.
Wavelet Entropy Features
In some cases, the masses may be surrounded with highly dense tissues along different directions. Dis-
crete Wavelet Transform (DWT) encodes the details present in an image along different directions. Wang
et al. [100] has shown that entropy features computed in wavelet domain can efficiently differentiate be-
tween the masses and normal breast tissues. Nine features encoding the entropy of upto three level DWT
detailed sub-bands are extracted from the circular regions. As the ROI size is much smaller compared
to the complete mammogram, meaningful decimation can be upto level three only. Beyond level three,
the smaller sized masses remain for few pixels only. On the other hand, Redundant Discrete Wavelet
Transform (RDWT) [33] is undecimated, therefore it can be used to extract the entropy information even
at higher levels. Here, RDWT upto level four is used to extract the entropy features.
Let I be the mammogram image and IHl, IV l, IDl be the detailed sub-bands at level l representing the
horizontal, vertical and diagonal high level coefficients respectively. RHl,RV l, andRDl represent the set
of DWT coefficients from the detailed sub-bands IHl, IV l, and IDl respectively mapped to the pixels in
region R on a mammogram. Entropy is calculated for each sub-band region as Ali = −∑pil ∗ log(pil),
where i = H,V, and D. AlH , AlV , and AlD represent the entropy of horizontal, vertical and diagonal
coefficients respectively at level l from DWT representation and pil represents the normalized histogram
of values in Ril. In a similar manner, the entropy features are computed from the RDWT sub-bands,
Bli = −
∑pil ∗ log(pil), where i = H,V,D whereBl
H ,BlV , andBl
D represent the entropy of horizontal,
vertical and diagonal coefficients at level l from RDWT representation respectively.
Spatial Domain Features
The spatial domain features include Laws texture features, intensity features, run-length texture features,
and statistical texture features commonly used by existing algorithms [20], [61]. To extract the spatial
domain features, let us assume circle ci be the set comprising all the boundary points of ROI,R be the set
containing all points in the region of interest, square s1 be the square region with area and center same
as ci, and region s2 be the square with each side exactly five pixels more than s1 and center same as ci.
This provides us two squares and a circle as shown in Figure 2.9, we now extract five different features
from the distribution formed by pixels in these regions.
• Intensity Features: It has been observed that the intensity of mass varies with respect to the sur-
18
rounding tissues. Based on this intuition, researchers have proposed some thresholding based and
morphology based algorithms [28], [71], [93]. However, we model this variation with the help
of five features computed from the pixel values within and out of the suspicious regions namely,
Mean Intensity, Intensity Variation, Mean Intensity Difference, Skewness, and Kurtosis. Detailed
information about computing these features is available in Section 5.2.
• Laws Texture Features: Laws [59] proposed five linear kernels for texture based segmentation
that represent edges, ripples, waves, lines, and spots in a square region as shown in Table 2.2.
Polakowski et al. [77] studied 25 feature masks generated by the combination of these kernels
and found that eight of them are the most discriminating for detecting masses in mammograms.
These eight features are used in this experiment to map the texture information present in the
segmented regions. Each of these features are computed with the help of feature masks described
in Section 5.3. The preprocessed image is convolved with these feature masks and the mean value
corresponding to pixel locations in circular regions are considered as feature values.
Table 2.2: Kernels proposed by Laws [59].
Kernel Label Kernel Values
l5 [1 4 6 4 1]s5 [-1 0 2 0 -1]e5 [1 -4 6 -4 1]r5 [-1 -2 0 2 1]w5 [-1 2 0 -2 1]
• Statistical Texture Features: Statistical texture features, also known as Spatial Gray-Level De-
pendence (SGLD) features, use statistical measures to model the pattern of intensity values in a
region [43]. These features have been used by many existing approaches to differentiate between
breast tissues as normal or cancerous [20]. The features are computed using the Gray Level Co-
occurrence Matrix (GLCM) which quantifies the number of occurrences of gray level i in a spatial
relation with gray level j. The image is discretized to eight gray levels and GLCM matrices are
obtained for four orientations (0◦, 45◦, 90◦, and 135◦). The four GLCM matrices obtained are
normalized and 13 features are extracted from each of the four matrices. Section 5.4 contains de-
tailed description of these features. Further, 13 additional features are computed as mean of these
individual features.
• Run Length Texture Features: Another widely used measure of texture includes features based
on the Gray Level Run Length (GLRL) matrix [94]. These features measure the simultaneous
presence of intensity values for varying continuous lengths in different directions. GLRL matrix
G(x, y|θ) represents the run length matrix in a given direction θ. These can be helpful in modeling
complex textures such as breast tissues and have been used for mass detection in some existing ap-
proaches [20]. In total, 20 features are computed from GLRL matrices in four different directions.
GLRL matrix for four directions (0◦, 45◦, 90◦, and 135◦) are computed and the features derived
from these matrices are explained in 5.5.
19
Feature Selection
154 features are extracted from each circular region. These features are evaluated in the proposed frame-
work resulting in selection of optimal features to discriminate between mass and non-mass regions. The
features are analyzed individually as well as in combination using feature selection and dimensionality
reduction algorithms. In this research, feature concatenation is used to combine any two feature sets.
Firstly, the performance of all individual set of features is evaluated. Mutual information based feature
selection [73] is used to further analyze these feature sets and obtain optimal features out of these individ-
ual sets. Features with significant performance are then combined to seek better classification. Mutual
information based feature selection [73] uses a minimum Redundancy Maximum Relevance (mRMR)
criterion to measure the discrimination power of features. Mutual information between two random
variables x and y with probability density functions p(x), p(y) and p(x, y) is computed using Eq. 2.6.
MI(x; y) =
∫ ∫p(x, y)log
p(x, y)
p(x)p(y)dxdy (2.6)
According to the mRMR criteria, optimal features must have high mutual information with the target
labels in order to represent the maximum dependency. The theoretical analysis of mRMR based feature
selection algorithm is discussed by Peng et al. [73]. The mRMR based feature selection arranges the
features in decreasing order of their discrimination ability using the input feature values and optimal
number of features is obtained by evaluating different number of features selected from top of the list.
Based on the initial results of individual feature sets, a combined set of features is obtained using feature
concatenation. The combined set of features is analyzed with the help of mRMR feature selection and
Principal Component Analysis (PCA) [3] based dimensionality reduction.
2.2.5 Classification
The features extracted from circular ROIs are classified using a 2−class SVM with the classes being
mass and non-mass. SVM [19] with polynomial kernel of degree three is trained to obtain the decision
boundary. The training data for SVM is {x, y}, where x represents the feature vector extracted from the
circular ROI with training label y ∈ {+1,−1}. +1 represents the positive (mass) class and−1 represents
the negative (non-mass) class. The actual labels of circular ROIs are obtained using the ground truth
information available with the database. Any circular ROI is assigned a positive label if the center of the
mass is present within the ROI or there is at least 50% overlap between the ground truth and extracted
ROI. The classifier trained on optimal set of features is tested using the remaining (unseen) regions as
probe instances.
2.3 Results and Analysis
Images from the MIAS database [92] are used for evaluating the proposed saliency based framework for
mass detection. The spatial resolution of images is 50µm × 50µm and grayscale intensity quantized to
20
8 bits. The MIAS database is one of the most popular public databases used for evaluating breast cancer
detection and diagnosis techniques. Though the database is old and many sophisticated algorithms have
been applied to detect symptoms using this challenging database, the intricacy of the database is clearly
visible from the results in the comparison study by Oliver et al. [71]. The database contains varied
density mammograms for both mass and non-mass classes, which increases the intra-class variation and
makes the problem even more challenging. There are total 322 MLO view mammograms in the database,
both left and right breast images for 161 cases. Among the 322 mammograms, 207 are normal and 115
mammograms have one of the four symptoms of breast cancer. Further details about the number of
images per symptom are summarized in Table 2.3.
Table 2.3: Symptom wise description of the MIAS database.
Description # Images (# Symptoms)Fatty Fatty-Glandular Dense-Glandular
Architectural Distortion 6(6) 6(6) 7(7)Bilateral Asymmetry 4(4) 4(4) 7(7)Mass 24 (27 ) 20 (20 ) 12 (12 )Micro-calcification Cluster 6(6) 9(9) 10(15)Normal 66 65 76
Total 106 104 112
In order to compare the performance of the proposed framework with existing algorithms, experiments
are conducted with the experimental protocol used by Oliver et al. [71] and the performance is compared
with seven state-of-the-art algorithms. The protocol includes classification of segmented regions as mass
and non-mass using three times 10 fold cross-validation. The performance is compared in terms of mean
and standard deviation values of Area Under the Curve (AUC) of Receiver Operating Characteristic
(ROC) curves obtained after classification. Since we have used the same database and experimental
protocol, performance of the proposed framework is directly compared with the results reported by Oliver
et al. [71]. The performance of segmentation and classification steps of the proposed framework are
individually discussed in the following sub-sections.
2.3.1 Segmentation Results
ROI segmentation using GBVS yields highly accurate results and does not generate any false alarm
due to pectoral muscles on the MIAS database. 49 out of total 58 masses are detected by the saliency
based segmentation, while nine are missed. Some false rejects are due to the noise in breast region,
for example, as shown in Figure 2.10, when a label is coinciding with the breast tissues and therefore
can not be completely removed. This results in false saliency accumulation at that point, resulting in
missing the mass region as well as adding one false positive. The results from GBVS are promising and
overall classification results (described in later subsections) justify the intuition to use saliency based
segmentation for analyzing mammograms.
In this research, we also compare the performance of GBVS with three existing visual saliency algo-
rithms - Esaliency [2], Hou and Zhang [51], and Liu et al. [63]. The algorithms are considered on the
21
( a )
( b )
Figure 2.10: Sample results of the proposed saliency based segmentation algorithm. (a) Successful segmentationand (b) false segmentation.
basis of their capability to generate saliency map from an input grayscale image. Though the effective-
ness of GBVS as a generic visual saliency algorithm can be analyzed in the benchmark study by Borji
et al. [10], comparative analysis from Figure 2.11 and Table 2.4 show that GBVS outperforms other
saliency algorithms for mammogram ROI segmentation. As shown in Table 2.4, three algorithms are
able to detect more than half of the masses at the cost of high false positives. Additionally, unlike GBVS,
other three algorithms are not able to distinguish between the pectoral muscle region and other breast
parenchyma. The contrast of mass containing regions vary from other breast parenchyma, therefore,
contrast maps being the basis of saliency map generation in GBVS makes it suitable for mammogram
analysis.
Table 2.4: Comparative analysis of saliency algorithms.
Comparison Metric GBVS [44] Liu et al. [63] Hou & Zhang [51] Esaliency [2]
Mass detection ratio(threshold on saliencymap)
49/58 (0.5) 58/58 38/58 (0.4) 40/58 (0.7)
False positives 2− 3/image NA 6− 7/image > 10/imageObservation Results are not
affected by thepresence of pec-toral muscles
Entire breast re-gion is the out-put salient object
Incorrect high saliencyvalue in different re-gions of mammogramother than mass re-gions such as pectoralmuscles and all cor-ners of the image
It is not ableto differenti-ate betweenthe pectoralmuscles andother breastparenchyma
2.3.2 Feature Extraction and Classification Results
In the proposed framework, 30, 462 overlapping circular regions are extracted from the probable regions
marked by applying saliency based segmentation on 55 images containing 58 masses. 3, 470 regions
contain at least 50% mass according to the ground truth data; these regions are considered as positive
class (mass) samples. 154 features from seven different sets of features are computed to classify each
22
GBVS [41] Liu [59]et al. Hou & Zhang[47]
Esaliency [2]InputMammogram
Figure 2.11: Sample results of the saliency algorithms. Each row corresponds to the output of the four saliencyalgorithms for corresponding mammogram image shown in the left most column. The green color in saliencymaps denotes the ROI segmented after thresholding on the saliency map and pink color represents the ground truthregion containing mass.
23
circular region as mass and non-mass. In the proposed framework, the performance of these features
is evaluated thoroughly, starting from classification using individual sets of features. Feature selection
techniques are applied on these feature sets to achieve optimal performance with minimum computational
effort. The features with good classification performance are combined using feature concatenation to
further enhance the classification performance. The results obtained from complete feature analysis are
explained in this section.
Individual Feature Set Results
The performance of individual sets of features for mass vs non-mass classification of extracted regions
can be compared in Table 2.5 and Figure 2.12. The results obtained can be summarized as follows,
Table 2.5: Classification results of individual sets of features.
Feature Category Features No. of Features AUC (Az)
Spatial Domain Features
GLRL Features 20 0.548± 0.016Intensity Features 5 0.532± 0.016Laws Texture Features 8 0.436± 0.059SGLD Features 65 0.687± 0.016
Fourier Domain Features Spectral Energy Features 35 0.495± 0.015
Wavelet FeaturesDWT Entropy Features 9 0.876± 0.001RDWT Entropy Features 12 0.870± 0.001
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
False Positive Rate
Tru
e P
osi
tive
Rat
e
RDWTDWTSGLDLawsIntensityGLRLFFT
Figure 2.12: ROC curves for the individual sets of features (Best viewed in color).
• As compared to other feature sets, spatial domain features do not perform efficiently. One of the
major reasons for the reduced performance could be varying orientation, shape and size of masses
present in the circular ROIs. The results show that the intensity or texture information alone may
not be sufficient to model the highly complex masses.
• Fourier domain spectral energies also could not yield high performance for the classifying circular
regions. Since spatial information is absent in Fourier representation, spectral energies derived
24
from frequency bands alone may not be sufficient, therefore exploring additional features may be
helpful for performance improvement.
• Entropy features from Discrete Wavelet Transform achieve show very good classification perfor-
mance without being affected by the size or orientation of masses. Wavelet entropies model the
randomness in edge maps along different orientations of a localized neighborhood. Therefore,
they may be able to better encode the differences between mass and non-mass irrespective of their
shape and size. Since the performance of DWT and RDWT features is dependent on the mother
wavelet used, we have evaluated the results with 12 different mother wavelets. The results of this
comparison are reported in Table 2.6. The results show that DWT entropy features derived from
the first three levels of Bi-orthogonal 2.2 wavelet transform yields the best AUC of 0.876± 0.001.
Table 2.6: Comparing the performance of DWT and RDWT entropy features with different mother wavelets. Theresults are reported in terms of Area Under the Curve (Az) of ROC curves.
Wavelet Used DWT RDWT DWT+RDWT
Bi-orthogonal 1.3 0.863± 0.001 0.870± 0.001 0.874± 0.000Bi-orthogonal 2.2 0.876± 0.001 0.848± 0.015 0.864± 0.000Bi-orthogonal 5.5 0.821± 0.006 0.767± 0.044 0.863± 0.020Coiflet 1 0.832± 0.020 0.792± 0.064 0.860± 0.000Coiflet 5 0.812± 0.022 0.674± 0.044 0.847± 0.000Daubechies 2 0.843± 0.012 0.784± 0.055 0.867± 0.001Daubechies 10 0.788± 0.029 0.744± 0.052 0.874± 0.004Discrete Meyer 0.775± 0.040 0.671± 0.064 0.842± 0.005Reverse Bi-orthogonal 1.3 0.797± 0.009 0.618± 0.025 0.851± 0.000Reverse Bi-orthogonal 2.2 0.855± 0.000 0.703± 0.051 0.891± 0.001Reverse Bi-orthogonal 5.5 0.780± 0.058 0.786± 0.009 0.858± 0.008Symlets 2 0.785± 0.036 0.721± 0.075 0.869± 0.001
• As discussed earlier, DWT can yield significant entropy features upto three levels of decima-
tion and beyond that, the size of some decimated masses is insignificant for feature computation.
Therefore, we have also evaluated the performance of Redundant Wavelet Transform features as
there is no decimation step in RDWT. We empirically found that information upto level four is
useful for the MIAS database. As shown in Table 2.6, the performance of RDWT entropy differs
significantly from the corresponding DWT entropy features, which indicates that RDWT provides
some additional information. Out of the 12 mother wavelets used for evaluation, entropy features
derived from the RDWT representation using Bi-orthogonal 1.3 mother wavelet yields the best
performance of AUC = 0.870± 0.001.
Feature Selection Results
From the results reported in Table 2.5, it is clear that the performance of different feature sets differ
significantly. Also, some features in these sets may be providing redundant information. Therefore, to
reduce the computational effort and discard the redundant and irrelevant features in these individual fea-
25
ture sets, mRMR based feature selection [73] is applied to find the optimal features from these individual
sets. The results obtained after feature selection are summarized in Table 2.7. It is observed that accu-
racy improved after feature selection for Spatial domain and Fourier domain features. Entropy features
derived from DWT and RDWT depend on edge orientations within the region. However, due to varying
shape and size of masses, their relevance can vary. Therefore, to keep the feature set generalizable, no
further feature selection is applied on wavelet entropy features.
Table 2.7: Analyzing the effect of mRMR based feature selection on individual feature sets. Performance ofindividual set of features after mRMR based feature selection.
Feature Description No. of Selected Features AUC (Az)
DWT 9 0.876± 0.001RDWT 12 0.870± 0.001Intensity 3 0.560± 0.047FFT 21 0.523± 0.016SGLD 52 0.719± 0.052GLRL 15 0.631± 0.047Laws 4 0.479± 0.093
Feature Combination Results
After thoroughly evaluating the individual sets of features, the feature sets are combined and classifica-
tion performance is analyzed. Feature combination is performed at two levels - before feature selection
and after feature selection, further described below.
• Before feature selection: As shown in Tables 2.5, the entropy values from DWT and RDWT repre-
sentations yield best result among all features. Therefore, it is our assertion that any combination
of features should contain these entropy features. Correlation analysis is also used to validate the
combination. The class-wise correlation between the distance scores obtained after SVM clas-
sification is calculated for both the feature sets. The correlation for the positive class (mass) is
found to be 0.33, which indicates that True Positive (TP) to False Positive (FP) ratio on the ROC
curve can be improved when DWT and RDWT features are combined. DWT features from all 12
mother wavelets are combined with their corresponding RDWT features. Comparison results from
Table 2.6 and Figure 2.13 show that the features extracted using Reverse Bi-orthogonal 2.2 mother
wavelet yields the best TP-FP ratio with AUC = 0.891 ± 0.001. The combination of best per-
forming DWT and RDWT features from Bi-orthogonal 2.2 and 1.3 mother wavelets respectively
could not perform better than DWT and RDWT features derived from Reverse Bi-orthogonal 2.2
mother wavelet.
• After feature selection: The feature sets obtained after feature selection are concatenated with
combination of DWT and RDWT to analyze the scope for further improvement in the classifi-
cation performance. Feature combination results are summarized in Table 2.8 and Figure 2.14.
Along with evaluating individual features, feature set obtained by combining the individual sets
26
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
False Positive Rate
Tru
e P
osi
tive
Rat
e
Biorthogonal 1.3Biorthogonal 2.2Biorthogonal 5.5Coiflet 1Coiflet 5Daubechies 2Daubechies 10Discrete MeyerReverse Biorthogonal 1.3Reverse Biorthogonal 2.2Reverse Biorthogonal 5.5Symlet 2
Figure 2.13: ROC curves for different mother wavelets with combined features from DWT and RDWT (Bestviewed in color).
of features is also analyzed using mRMR based feature selection and PCA based dimensionality
reduction. The combined feature set consists of DWT, RDWT, SGLD, GLRL, and FFT features
(intensity and Laws features are discarded for the combination due to their poor performance).
The results derived from feature combination, as reported in Table 2.8, show that the highest clas-
sification performance is achieved using the combination of DWT and RDWT features only.
Table 2.8: Analyzing classification performance with different combinations of feature sets.
Feature Description No. of Features AUC (Az)
DWT + RDWT 21 0.891± 0.001DWT + RDWT + Intensity 24 0.754± 0.039DWT + RDWT + GLRL 36 0.747± 0.025DWT + RDWT + FFT 42 0.489± 0.042DWT + RDWT + SGLD 73 0.693± 0.020
DWT + RDWT + FFT + GLRL + SGLD(Complete Feature Set)
141 0.457± 0.073
mRMR based Feature Selection (On Com-plete Feature Set)
114 0.788± 0.030
PCA based Dimensionality Reduction (OnComplete Feature Set)
10 0.463± 0.033
Comparison with Existing Algorithms
The proposed framework uses saliency based region of interest detection. The segmented regions are
further refined using SVM based classification with entropy features derived from DWT and RDWT
representations. The performance of the proposed framework can be directly compared with the results
reported by Oliver et al. [71] as the experimental protocol and database are same. Therefore, the com-
parison is performed with seven state-of-the-art algorithms evaluated in [71] that use computer vision
27
algorithms such as Difference of Gaussian (DoG) and isocontour maps for segmentation of regions of
interest, followed by machine learning approach for false positive reduction. The algorithms are ana-
lyzed in the literature review section in Table 2.1 and the results are shown in Table 2.9. The results in
Table 2.9 clearly shows that the proposed algorithm achieves significant improvement in reducing false
positives while detecting masses in digital mammograms with improved sensitivity.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
False Positive Rate
Tru
e P
osi
tive
Rat
e
DWT + RDWTDWT +RDWT + FFTDWT + RDWT + GLRLDWT + RDWT + IntensityDWT + RDWT + SGLDComplete feature setmRMR based selectionPCA based selection
Figure 2.14: ROC curves for different combinations of feature sets (Best viewed in color).
Table 2.9: Comparing the performance of proposed algorithm with existing algorithms on the MIAS database.
Algorithm AUC (Az)
Detection using Difference of Gaussian [77] 0.601± 0.028Detection using concentric layers of gray regions [28] 0.614± 0.032Detection based on a classifier [53] 0.673± 0.016Pattern matching based approach [58] 0.675± 0.005Thresholding based approach [54] 0.685± 0.002Laplacian edge detector based approach [75] 0.758± 0.005Detection using Iris filter [97] 0.787± 0.003Proposed framework with DWT and RDWT features 0.891± 0.001
2.4 Summary
This chapter presented a novel framework to detect mass(es) from screening mammograms for diagnosis
of breast cancer. The detection of mass(es) from screening mammograms has been a challenging prob-
lem due to inherent complexity of size, shape, and surrounding tissue structures. Several techniques have
been proposed and comparative studies have been conducted to evaluate the state-of-art techniques on
public datasets. The proposed framework utilize the saliency based algorithm to detect mass containing
regions which does not require any segmentation of pectoral muscles unlike most of the existing tech-
niques. The experimental analysis conducted on public database using existing protocol concludes that
proposed framework outperforms state-of-the-art techniques.
28
Chapter 3
HEp-2 cell image classification using Lawstexture features
3.1 Introduction
Human immune system consists of cells and tissues which work together to defend the body from foreign
infections. The immune system is composed of two components, innate immune system and acquired
immune system. The innate immune system is responsible for activation of white blood cells to protect
the body against foreign infections without using any antibodies (which are developed as part of the adap-
tive or acquired immune system as the human grows). Acquired immune system learns to fight infections
and preserve the learning for possible future infections. It activates immune cells and generate proteins
i.e., antibodies according to the learned patterns in order to protect against foreign invasions. When the
acquired immune system mistakenly learns to fight against self-tissues, the antibodies thus generated
are called autoantibodies. This phenomenon is referred as autoimmunity and the resulting syndromes
are known as autoimmune diseases. The reasons for such a malfunctioning in the acquired immune
system are unclear; however, they are considered to be the combined effect of genetic, environmental,
and regulatory changes [84]. For instance, a patient with genetic conditions favorable to development
of autoimmunity, under specific environmental conditions, may become a victim of a foreign intrusion
which might result in triggering a particular disease. The effects of autoimmune disease vary according
to the targeted part of the body which can be heart, brain, glands, digestive system, or blood vessels.
Some autoimmune diseases such as Systemic Lupus Erythematosus (SLE) and Type 1 diabetes can even
affect multiple parts of the body including essential organs such as kidneys, muscles, and blood vessels.
Collectively, autoimmune diseases are one of the leading causes of deaths worldwide. Satoh et al. [85]
have shown that in USA alone, autoantibodies are present in more than 32 million individuals. Further,
Fairweather et al. [29] suggest that the autoimmune diseases are more prevalent among women and their
overall prevalence is increasing. Mostly autoimmune diseases are chronic syndromes with debilitating
health issues and a never ending clinical treatment; however, some of them may even lead to death.
29
3.1.1 Diagnostic tests for Autoimmune Diseases
The most common symptom of autoimmunity is inflammation which leads to reddish scars, pain, and
swelling. While the cure for autoimmune diseases is under exploratory phases, currently treatments are
conducted to reduce the symptoms and effects associated with the particular disease. In current treatment
process, it is important to detect and diagnose autoimmune diseases, for which the following tests are
useful [17]:
• Laboratory Evaluation: Autoimmune diseases resulted due to inflammation cause significant
changes in the normal hematology. Therefore, laboratory evaluations such as Complete Blood
Count (CBC) and urinalysis are conducted to investigate any abnormality in hematologic parame-
ters such as white blood cell count and red blood cell count.
• Inflammatory Markers: In order to confirm commonly observed abnormalities in autoimmune
diseases, tests indicative of inflammation are conducted. Some of the popular tests include Ery-
throcyte Sedimentation Rate (ESR) and C-Reactive Protein (CRP).
• Autoantibody Tests: The presence of autoantibodies combined with appropriate symptoms helps
to support diagnosis for autoimmune diseases. Autoantibodies are of various types, therefore,
different tests are conducted to investigate the presence of particular autoantibodies. However,
among various types, the presence of Antinuclear Antibodies (ANA) has been commonly observed
for many autoimmune diseases. Therefore, Antinuclear Antibody Test is conducted as a screening
test for autoimmune diseases.
Among all the diagnostic tests, detecting the presence of autoantibodies is vital for diagnosis of autoim-
mune diseases. However, ANA testing is a time consuming and expensive process due to the involvement
of human experts and lack of automated techniques [21] [26]. Significant efforts are being made to de-
velop automated systems which could fasten the diagnostic process.
3.1.2 ANA Testing
ANA tests are conducted to detect the presence of antinuclear antibodies which help in diagnosis of
several autoimmune diseases. Enzyme-Linked Immunosorbent Assay (ELISA) and Indirect Immunoflu-
orescence (IIF) are the two widely used methods for detection of anitnuclear antibodies. Even though
both ELISA and IIF achieve high sensitivity, IIF is preferred and the recommended procedure [66]. IIF
achieves high sensitivity as well as specificity for ANA detection and can be used to detect many antin-
uclear antigen patterns unlike ELISA which is a targeted test to detect specific patterns [36]. Similar to
several other medical diagnostic tests, the clinical procedure of IIF is time consuming and requires hu-
man experts for visualization. Firstly, experts categorize the digitally captured images of IIF slides based
on the fluorescence intensity levels in the images. The images categorized as appropriate for clinical
diagnosis are then further examined to identify the antigen patterns being exhibited as a result of antigen
reactions taking place within Human Epithelial Type 2 (HEp-2) cells. The increasing number of tests
being conducted resulted in need for automated systems in place of manual examination of test slides.
30
To the best of our knowledge, the IIF procedure has been automated up to the generation of test slides
with the help of robotics and vision based techniques [83]. Very few commercial systems have auto-
mated the steps in manual examination of IIF images [5]. However, a completely automated computer
aided diagnostic (CAD) system for this test is still not available. The automation of pattern identification
from test slides would not only reduce the need for large number of human experts, it would also help in
standardization of the test. Such a standardization would establish confidence among the test results by
omitting the possibility of intra-observer variability.
3.1.3 Research Contribution
As mentioned previously, the state-of-art procedure for ANA testing is IIF test which reports the presence
of antinuclear antibodies within the patient’s serum. Though IIF is very useful as clinical screening for
autoimmunity, this procedure lacks standardized completely automated processing and requires human
experts to examine the fluorescence microscopic images. Therefore, significant efforts are underway to
develop efficient techniques to automate this procedure [7], [82]. Most challenging task in automating
the IIF procedure is identification of patterns from HEp-2 cells within the test images. Recently, public
databases are made available that have significantly driven the research efforts in automating the task
of pattern identification [32]. Many sophisticated pattern classification approach based techniques have
been developed and high performance results are reported on common protocols. In this study:
• We review the existing literature and critically analyze the shortcomings observed by most of the
researchers. Using innate characteristics of a cell image, we propose to categorize the features used
by existing techniques and some other related features that could also be used for classification.
• We propose to use Laws texture features [59] for HEp-2 cell image classification and present com-
parative results with other features using different classification algorithms. Also, results are com-
puted on different databases to analyze the impact of data from different labs on the classification
performance.
• We analyze the classification performance of different feature sets on multiple classification algo-
rithms using public databases on existing and a k-fold cross validation based protocols.
3.2 Pattern identification from IIF slides
The IIF test slide images are generated by capturing fluorescent radiation exhibited as a result of antigen
reaction within the HEp-2 cells. The exhibited patterns help in identification of different types of antigens
present in the serum thereby leading to detection of rheumatic diseases. Since the fluorescence radiation
is vulnerable to ambient light, the image of IIF test slide is generally categorized as positive, intermediate,
and negative fluorescence image [83]. The categorization is based on the quality of captured image
thereby indicating the usability of image for diagnosis. Positive images are high quality fluorescence
images as they are high contrast images enabling the expert observers to easily identify the patterns
31
being exhibited. The images labelled with intermediate fluorescence are lower in contrast, therefore,
it is relatively difficult to identify antigen patterns from intermediate images as compared to positive
images. Finally, negative fluorescence images do not qualify for any further clinical usage. The positive
and intermediate images are examined to identify antigen patterns. Identification of patterns from the
fluorescent HEp-2 cell images can be viewed as a classical pattern recognition problem with more than
50 possible target classes. This section emphasizes on the efforts being made to automate the process of
pattern identification from IIF test slides. As illustrated in Figure 3.1, an automated system for analysis
of IIF images comprises of the following steps:
Figure 3.1: Steps involved in an automated system for diagnosis of IIF images.
• IIF Image Classification: Digitally captured IIF image is first labeled as positive, intermediate
or negative intensity image based on the quality of fluorescent radiation as perceived from the
digital image. Only positive and intermediate intensity images are considered for further analysis.
Figure 3.2 illustrates samples of positive and intermediate intensity IIF images.
Figure 3.2: A sample of positive and intermediate intensity IIF images.
• Pre-processing: IIF images are colored images, however, Cordelli and Soda [22] suggest that
converting color images to grayscale representation improves the efficiency of automated process.
Moreover, mechanically captured images may incur some calibration errors resulting in out of
32
focus images, therefore, some pre-processing steps may also be required to correct such errors.
Collectively a pre-processing step is essential before further analysis is performed on IIF images.
• Cell Segmentation: Image segmentation techniques are required to segment the HEp-2 cells from
the IIF image. An example is shown in Figure 3.3. As discussed earlier, positive intensity IIF
images are higher in contrast as compared to intermediate images; therefore, any segmentation
algorithm for IIF images need to be designed carefully. A common segmentation algorithm may
result in less accurate results for both types of images. Mostly existing techniques have utilized
Otsu thresholding [72] for cell segmentation [52], [27], [46]; however, most of these techniques
fail when cell structures are not segmented accurately [23]. Significant efforts are required to
develop efficient techniques for HEp-2 cell segmentation from IIF images.
Figure 3.3: Segmentation mask for corresponding IIF image.
• HEp-2 Cell Image Classification: The segmented HEp-2 cell images are then classified among
target antigen patterns using pattern recognition and machine learning techniques. The public
datasets have focused on commonly observed antinuclear antigen patterns such as Centromere,
Golgi, Homogeneous, Nucleolar, Nuclear Membrane, and Speckled. Figure 3.4 illustrates a sam-
ple of these antigen patterns in both positive and intermediate intensity IIF images. Similar to the
segmentation step, classification techniques need to consider the difference in properties between
intermediate and positive intensity images.
3.3 HEp-2 Cell Image Classification
Efforts are being made to promote research and development of techniques to automate HEp-2 cell
image classification. Public databases are being released through workshops and competitions in reputed
international conferences such as International Conference on Pattern Recognition (ICPR) [32] [49] and
International Conference on Image Processing (ICIP) [48]. Availability of such public databases have
largely driven the research for HEp-2 cell image classification. The initial approaches submitted during
33
Intermediate Intensity Cell Images
Positive Intensity Cell Images
Figure 3.4: Sample images from the ICIP 2013 cell image classification contest training dataset [48]. The cellpattern types of images from left to right - Centromere, Golgi, Homogeneous, Nucleolar, Nuclear Membrane, andSpeckled.
the ICPR 2012 contest on HEp-2 cell image classification have evaluated various image based features
in order to find the optimal set of features [32].
In this research, we categorize the features used by most of the existing approaches based on various
object based properties of a HEp-2 cell image. Under the purview of proposed feature categorization, we
present a review of existing literature and discuss the research gaps observed in research associated with
HEp-2 cell image classification. We propose the use of Laws texture features for HEp-2 cell image clas-
sification task validated by comparative experiments. Also, we analyze the results reported by existing
techniques and the protocols with supportive experimental analysis.
3.3.1 Literature Review
In the initial study conducted by Foggia et al. [32], many researchers used geometric features along
with other intensity based features for HEp-2 cell image classification. As summarized in Table 3.1,
recent techniques have largely explored only intensity based features; specifically descriptor features.
The results from a comparison study by Snell et al. [89] report high performance using shape based
features. However, most of the techniques incorporate texture or descriptor based features in combination
with standard classification algorithms, such as Support Vector Machines (SVM) [96] and k-Nearest
Neighbour (kNN) [35], to develop efficient models for HEp-2 cell image classification. In this study,
existing literature is summarized with respect to a proposed feature categorization. We broadly categorize
different features used by existing techniques for HEp-2 cell image classification in four categories.
34
Tabl
e3.
1:R
evie
wof
exis
ting
liter
atur
eba
sed
onth
epr
opos
edfe
atur
eca
tego
riza
tion
and
clas
sific
atio
nte
chni
ques
used
forH
Ep-
2ce
llim
age
clas
sific
atio
n.
Ref
eren
ceFe
atur
esTe
chni
que
Dat
abas
eG
eom
etri
cTe
xtur
eD
escr
ipto
rO
ther
sPe
rner
etal
.[74
]Y
Dec
isio
ntr
eePr
ivat
eda
taSo
daan
dIa
nnel
lo[9
0]Y
Agg
rega
tion
ofbi
nary
clas
sifie
rsPr
ivat
eda
taD
iCat
aldo
etal
.[18
]Y
YY
Subc
lass
Dis
crim
inan
tAna
lysi
sM
IVIA
Fara
kiet
al.[
30]
YB
agof
wor
dsM
IVIA
Kon
get
al.[
55]
YSp
arse
codi
ngM
IVIA
Liu
and
Wan
g[6
2]Y
Supp
ortV
ecto
rMac
hine
MIV
IAN
anni
etal
.[68
]Y
Supp
ortV
ecto
rMac
hine
MIV
IAN
osak
aan
dFu
kui[
69]
YSu
ppor
tVec
torM
achi
neM
IVIA
Pono
mar
evet
al.[
78]
YSu
ppor
tVec
torM
achi
neM
IVIA
Shen
etal
.[87
]Y
Bag
ofw
ords
MIV
IASn
elle
tal.
[89]
YY
YSu
ppor
tVec
torM
achi
neM
IVIA
Stok
lasa
etal
.[91
]Y
YY
kN
Nba
sed
clas
sifie
rM
IVIA
The
odor
akop
oulo
set
al.[
95]
YSp
arse
repr
esen
tatio
nM
IVIA
Wili
emet
al.[
101]
YB
agof
wor
dsM
IVIA
Yan
get
al.[
102]
YSu
ppor
tVec
torM
achi
neM
IVIA
35
Feature Categorization
The proposed feature categorization is based on the properties of a HEp-2 cell image which could be
captured as features and may be helpful in antigen pattern classification:
1. Geometric Features: Features that focus on various geometric aspects and relate to the charac-
teristics associated with shape, size, and circumference of HEp-2 cells are referred as geometric
features.
2. Texture Features: Features which directly relate to the texture or intensity pattern of a cell image
are generally defined as texture features. The feature representations in this category are Laws
texture features [59], Spatial gray level dependence features (SGLD) [43], and Gray level run
length features (GLRL) [94].
3. Descriptor Features: Descriptor features correspond to the feature computations which involve
specific keypoint based description or generate histogram based descriptors from feature maps.
Some examples are Local Binary Pattern (LBP) [70], Scale Invariant Feature Transform (SIFT) [64],
and Histogram of Oriented Gradients (HOG) [24].
4. Others: This category includes feature representations such as statistical features computed from
the intensity values in a cell image [41], features based on dictionary learning [30], and features
derived based on ICA or PCA projections [62], [102].
Literature Analysis
According to the proposed categorization, existing literature is summarized below:
• Geometric Features: Ponomarev et al. [78] utilize the shape and structure based features for SVM
based classification. Perner et al. [74] developed a data mining based decision tree to classify the
HEp-2 cell images using the shape and size based features.
• Texture Features: DiCataldo et al. [18] proposed a Subclass Discriminant Analysis based ap-
proach which uses a combination of texture and shape based features. Stoklasa et al. [91] com-
bined multiple shape and texture features to propose an efficient kNN based technique. Soda and
Iannello [90] proposed an aggregation of classifiers based technique which incorporates various
texture features.
• Descriptor Features: Nanni et al. [68] proposed an ensemble of variants of LBP features in
combination with SVM based classification. Nosaka and Fukui [69] explored a rotation invari-
ant co-occurrence among adjacent LBPs in combination with SVM based classification. Other
than standard classification models, researchers have also utilized learning based models. Kong
et al. [55] combined LBP and HOG features to develop a dictionary learning based approach.
Theodorakopoulos et al. [95] utilized SIFT and LBP to develop a sparse representation based
technique. Wiliem et al. [101] proposed an advanced learning approach based on bag of visual
36
words which incorporates SIFT based features. Shen et al. [87] proposed a framework which uses
intensity order pooling based rotation invariant local gradient features for a bag of words based
classification.
• Others: Researchers have also tried to learn discriminating features from cell images to develop
task specific classification models. Liu and Wang [62] learned discriminating features from images
in a learning based approach. Yang et al. [102] proposed to learn features using Independent Com-
ponent Analysis based framework for a SVM based multi-class classification. Faraki et al. [30]
adopted a dictionary learning approach to develop fisher tensors based technique.
Significant efforts are also being made by the biomedical industry to develop efficient CAD techniques
for IIF testing. Bizarro et al. [5] analyzed and compared the performance of six commercial systems
being used by clinicians for IIF testing. Their analysis was mainly aimed at comparing the classification
performance of six systems for positive vs negative fluorescence categorization of IIF images. The
results show that commercial systems yield good performance for positive vs negative classification task.
Though, they have also reported results for the HEp-2 cell image classification; however, the commercial
systems do not show high performance.
3.3.2 Databases and Existing Results
HEp-2 cell image classification research has surged since the release of MIVIA dataset during ICPR
2012 [32]. The MIVIA dataset comprises of about 1400 cell images pertaining to six cell categories.
Another bigger database has been released as training set during ICIP 2013 cell classification competi-
tion [48]. Table 3.2 provides summary of MIVIA and ICIP 2013 datasets. As shown in Table 3.1, the
existing techniques are generally evaluated using the two predefined protocols on MIVIA dataset.
• Protocol 1 - An equal split of total cell images into single training and testing datasets as provided
during the ICPR 2012 HEp-2 cell image classification contest.
• Protocol 2 - Leave-one-out cross validation on entire dataset i.e, during each fold of 28 fold cross
validation, 27 images are to be used for training and the left out image is used to test the perfor-
mance of trained classifier.
The results from existing literature on the two protocols using MIVIA dataset are summarized in Ta-
bles 3.3 and 3.4. The results reflect following key insights:
• The results from existing literature are encouraging and show the HEp-2 cell image classification is
a complex pattern recognition problem with maximum overall accuracy of 75.10% and 89.55% on
protocol 1 and protocol 2, respectively. However, significant efforts are required to attain clinically
usable accuracy value close to 100%.
• The results summarized in Table 3.4 depict that coarse and fine speckled are the most difficult to
identify among the six antigen patterns and the highest accuracies are observed for homogeneous
and centromere patterns.
37
• The image wise results show a huge variation in correct classification rate across the 28 images.
Most of the positive intensity images are classified correctly, whereas, the accuracy for intermedi-
ate intensity images is significantly low.
• Even for positive intensity category, poor performance is observed in few cases. The variations
in accuracy may be attributed to the complexity of combined model for positive and intermediate
intensity images.
Table 3.2: Summary of the MIVIA dataset [32] and training dataset provided during ICIP 2013 cell image classi-fication contest [48].
Database Pattern Positive Intermediate Total
MIV
IA
Centromere 173 184 357Coarse Speckled 136 74 210Cytoplasmic 36 75 111Fine Speckled 97 111 208Homogeneous 222 108 330Nucleolar 129 112 241Total 793 664 1457
ICIP
2013
Centromere 1378 1363 2741Golgi 349 375 724Homogeneous 1087 1407 2494Nucleolar 934 1664 2598Nuclear Membrane 943 1265 2208Speckled 1457 1374 2831Total 6148 7448 13596
Table 3.3: Overall cell classification accuracy (%) reported in existing literature on MIVIA dataset.
Citation Protocol 1 Protocol 2
DiCataldo et al. [18] 72.21 89.55Faraki et al. [30] 70.16 71.70Kong et al. [55] 66.76 63.16Liu and Wang [62] 66.60 58.92Nanni et al. [68] 70.00 67.20Nosaka and Fukui [69] 68.53 70.65Ponomarev et al. [78] 70.57 54.77Shen et al. [87] 74.39 69.39Snell et al. [89] 56.50 53.70Stoklasa et al. [91] 64.40 64.30Theodorakopoulos et al. [95] 75.10 64.90Wiliem et al. [101] 67.40 56.80Yang et al. [102] 64.60 60.60
38
Table 3.4: This table presents the number of correctly classified cell images for each IIF image as reported byexisting techniques using leave-one out cross validation protocol on MIVIA dataset.
Sr. Pattern Catg. #Img [18] [30] [55] [62] [68] [69] [78] [87] [91] [95] [101]
1 Homogeneous P 61 61 57 61 60 54 59 49 60 61 54 432 Fine Speckled I 48 42 25 48 29 14 17 16 18 33 32 193 Centromere P 89 89 89 89 88 79 88 86 87 83 88 874 Nucleolar I 66 57 21 12 29 17 31 27 44 3 26 325 Homogeneous I 47 46 16 47 35 35 42 27 30 43 28 266 Coarse Speckled P 68 65 66 68 10 26 63 63 29 63 36 447 Centromere I 56 49 47 56 49 22 37 47 54 34 51 548 Nucleolar P 56 55 17 56 0 51 30 11 25 43 5 309 Fine Speckled P 46 40 6 46 15 7 23 6 14 34 4 2310 Coarse Speckled I 33 29 22 26 5 32 11 12 33 14 23 1611 Coarse Speckled I 41 39 30 41 34 35 30 5 41 13 34 3212 Coarse Speckled P 49 49 36 38 35 33 37 37 43 43 41 3513 Centromere P 46 46 42 34 34 44 44 38 35 40 42 3014 Centromere I 63 14 40 63 4 45 25 5 8 0 3 3015 Fine Speckled I 63 39 17 33 21 11 23 14 24 16 46 3616 Centromere P 38 37 36 35 35 38 36 35 33 38 38 3617 Coarse Speckled P 19 14 5 2 0 3 0 9 4 2 2 1418 Homogeneous P 42 41 42 42 26 23 32 7 27 17 18 1419 Centromere I 65 63 57 57 64 62 61 48 62 48 52 5620 Nucleolar I 46 45 44 46 7 42 38 6 43 30 2 4121 Homogeneous I 61 44 22 3 23 37 25 45 20 21 17 5022 Homogeneous P 119 115 104 86 102 84 83 65 81 78 92 7823 Fine Speckled P 51 49 41 48 35 18 31 1 14 34 16 2824 Nucleolar P 73 69 66 65 66 68 65 56 68 55 63 4525 Cytoplasmic I 24 22 13 24 4 19 20 11 18 9 19 1026 Cytoplasmic P 36 34 33 34 33 33 32 30 34 32 34 3327 Cytoplasmic I 38 37 36 38 35 34 32 31 37 35 37 3528 Cytoplasmic I 13 13 13 13 13 13 13 10 13 13 12 10
39
3.4 Proposed Laws Texture Features Based HEp-2 Cell Image Classifica-tion
Above literature analysis depicts that intensity based features are generally more efficient for HEp-2 cell
image classification. However, most discriminating features are still unclear and yet to be established.
Other than several texture features, such as SGLD features [43] and GLRL features [94], being used for
computer vision applications, Laws texture features [59] have been used effectively for specific medical
image analysis applications such as mammography [77]. Laws texture features are derived from basic
linear kernels that represent edges, ripples, waves, lines, and spots in a square region. Based on the
insights from patterns within the HEp-2 cell images [48], we propose to use Laws texture features for the
HEp-2 cell image classification, as shown in Figure 3.5. We perform experimental analysis to evaluate
and compare the performance of Laws texture features in the proposed system with other widely used
image based features for HEp-2 cell image classification such as LBP and HOG features.
Figure 3.5: Diagrammatic representation of proposed Laws texture features based HEp-2 cell image classification.
3.4.1 Algorithm
The input to the proposed system, as illustrated in Figure 3.5, is the HEp-2 cell image segmented from
IIF test slide image. In order to identify the antigen pattern in input image, Laws texture features are
extracted. Laws texture features consist of total 102 statistical values computed from the 34 feature maps
(illustrated in Figure 3.5) obtained from the convolution of feature masks with the HEp-2 cell image, see
40
Appendix 5.3 for more details. These features are then served to a trained classifier model for pattern
identification. The classifier model is developed using the features computed from the training samples.
Rest of this section evaluates the effectiveness of the proposed Laws texture features based system and
compares the performance with other widely used feature representations.
3.4.2 Experimental Database and Protocol
In order to evaluate the effectiveness of Laws texture features, we perform several comparison exper-
iments. Various features are tested using four different classifiers to ensure optimal performance. The
results are obtained using two public datasets - MIVIA [32] and the training dataset provided during ICIP
2013 [48]. Thus, the aim of entire analysis are:
• to compare and evaluate the performance of Laws texture features for the HEp-2 cell image clas-
sification,
• to validate the comparative results using images from two different datasets, and
• to validate our assertion that instead of using a single classifier model with same parameter values
for both positive and intermediate intensity images, different classifier models need to be developed
for better classification.
Features and Classifiers
In order to compare the performance of Laws texture features, we have considered features from spa-
tial, frequency as well as wavelet domain representations of cell images. Wavelet transforms are used
to represent the frequency domain so that the spatial information also remain intact while computing
important frequency domain characteristics. The entire pool of features according to above mentioned
categorization is summarized below:
• Geometric Features: Geometric features include shape, size, and boundary based features such
as area, convex hull features, solidity, aspect ratio, compactness, and Fast Fourier Transform based
spectral energy features computed via boundary points [11].
• Texture Features: Gray Level Run Length features (GLRL) [94], Laws texture features [59],
Energy and entropy values computed from high frequency bands of Redundant Discrete Wavelet
Transform representation [34], Gabor wavelet features, and Spatial Gray Level Dependence fea-
tures (SGLD) [43] are considered for texture based features.
• Descriptor Features: Descriptors computed from HOG [24] and uniform circular LBP [70] rep-
resentations are considered for descriptor features.
• Others: Statistical features computed from intensity distribution of HEp-2 cell images, such as
mean, absolute mean, and standard deviation are also used for comparison.
41
Classification algorithms considered for the analysis are: k-Nearest Neighbor [35], Random Decision
Forests [47], SVM [96] with linear kernel, and SVM with non-linear RBF kernel. Parameter selection
for SVM is performed using grid search [19] to ensure best results.
Experimental Protocol
The experiments are conducted to evaluate our assertion that separate models should be developed for
positive and intermediate intensity images. Therefore, we divided both datasets into respective positive
and intermediate intensity datasets. Along with the positive and intermediate models, another model is
developed using the entire data, i.e. combined positive and intermediate intensity images, to analyze its
impact on performance. All the three models are evaluated using existing protocols of MIVIA dataset
(discussed in Section 3.3.2; 50-50 split protocol 1 and leave one out protocol 2) as well as 10 fold cross-
validation protocol (i.e. 10 equal size folds for each class are created, nine folds are used for training
and one fold is used for testing. This training and testing is repeated 10 times and average accuracy is
reported).
3.4.3 Results and Analysis
Table 3.5 summarizes the comparison of classification results of different feature and classifier combina-
tions using the two protocols of MIVIA and the 10 fold cross validation protocol on MIVIA dataset. The
high accuracy values using 10 fold cross validation protocol clearly indicate the limitation in existing
protocols of MIVIA dataset. Also, the cross-validation based experimental protocol convey similar re-
sults irrespective of the dataset, see Figure 3.6. The overall analysis from Figure 3.6 can be summarized
as:
• Laws texture features are effective for HEp-2 cell image classification. They perform the best
on ICIP 2013 contest training dataset as well as show high accuracy similar to LBP and HOG
features for MIVIA dataset. The results on two different datasets also signify the robustness of
these features across data from multiple sources.
• Texture and descriptor features outperform all other feature categories for the positive intensity
and intermediate intensity sets. As discussed earlier, though the images pertaining to intermediate
intensity set are difficult to categorize, intensity based features yield significant improvement in
classifying these images.
• The results further indicate that cells across different types of cell patterns have similar shape and
size thereby increasing the inter-class similarity. On the other hand, grayscale distribution within
different cell classes vary significantly thus increasing the inter-class variability. Therefore, the
features based on grayscale values such as texture and descriptor features yield better performance
for cell image classification than geometric features.
• Among the four classification techniques used for experiments, SVM with RBF kernel yields the
highest accuracy for HEp-2 cell image classification. The results of RDF and SVM-RBF are
42
comparable with RDF providing the second best classification accuracy.
• The intermediate intensity images are generally lower in contrast as compared to positive intensity
images. The results in Figure 3.6 present a comparison of the three models trained on positive,
intermediate, and combined (positive as well as intermediate images) datasets. The significant
difference in results clearly show that independent models should be trained to classify positive
and intermediate intensity images to achieve optimal performance. Overall results indicate that cell
image classification for images pertaining to the intermediate intensity set is difficult as compared
to the positive intensity images.
• Since number of samples of HEp-2 cell images are much larger in ICIP 2013 dataset as compared
to MIVIA dataset, the classifiers are able to learn better and yield higher classification accuracies
on ICIP 2013 database.
The results from Table 3.5 indicate that the performance results using different protocols vary signifi-
cantly. The variation in results may be due to inability of classification algorithms to learn adequately
when only few samples are available. Also, images in the dataset may not contain the adequate samples
per class and therefore causing the effect of imbalance learning. Based on the results from this study,
we find that the k-fold cross validation protocol may be preferred over leave-one-out cross validation
protocol. The leave-one-out cross validation protocol in such cases only provides accuracy for one class
which may not indicate the overall accuracy of the trained model. However, k-fold cross validation pro-
tocol provides the classification performance for all classes using equal proportions of data for each class.
Therefore, it is our assertion that with 10 fold cross validation, texture features such Laws features with
SVM or RDF classifiers can potentially provide a viable solution for HEp-2 cell image classification
3.5 Summary
This chapter discussed the emerging research efforts to develop computer aided systems for diagnosis of
autoimmune diseases. Among various tasks in diagnosis of autoimmune diseases, the focus is to develop
an efficient automated system for HEp-2 cell image classification. The existing research is analyzed
based on a proposed feature categorization and the results reported on existing database. Laws texture
features based system is proposed for the HEp-2 cell image classification. Experimental analysis is
conducted using two public datasets and existing protocols. Results from comparative analysis concludes
that the proposed system outperforms state-of-the-art techniques.
43
Table 3.5: Comparison results for all the baseline features using different protocols on MIVIA dataset. Overall cellclassification accuracy values are reported for all feature and classifier combinations on the three protocols. On thelines of existing literature, all three protocols are applied on combined (both positive and intermediate) dataset.
Category Feature Classifier Protocol 1 Protocol 2 10 fold CV
Geo
met
ric
Boundary
kNN 22.62 27.64 31.82RDF 28.34 31.03 52.64
SVM-RBF 25.20 22.71 35.40SVM-Linear 26.43 27.19 28.11
Shape & Size
kNN 29.70 27.03 35.87RDF 31.88 27.27 53.19
SVM-RBF 32.70 28.21 41.44SVM-Linear 35.42 27.12 30.03
Text
ure
Gabor
kNN 27.11 22.44 49.21RDF 31.74 27.50 68.86
SVM-RBF 27.11 33.72 67.97SVM-Linear 17.03 37.34 61.31
RDWT
kNN 26.57 23.94 50.72RDF 27.79 22.45 66.59
SVM-RBF 20.30 0.00 24.54SVM-Linear 20.30 14.29 14.23
SGLD
kNN 27.25 20.23 40.06RDF 28.20 18.50 51.95
SVM-RBF 35.42 24.54 55.74SVM-Linear 25.07 21.01 44.12
GLRL
kNN 30.38 26.97 49.83RDF 32.29 30.53 70.37
SVM-RBF 37.06 22.37 45.29SVM-Linear 33.92 28.98 41.79
Laws
kNN 27.66 20.40 66.32RDF 23.84 19.17 85.78
SVM-RBF 20.03 16.00 65.15SVM-Linear 48.37 42.25 70.31
Des
crip
tor LBP
kNN 30.52 25.01 72.24RDF 44.14 32.61 82.75
SVM-RBF 53.54 40.88 82.13SVM-Linear 48.64 39.24 73.81
HOG
kNN 40.60 34.29 62.74RDF 41.83 34.03 75.11
SVM-RBF 45.50 42.47 80.21SVM-Linear 46.05 46.49 73.68
Oth
ers
Statistical
kNN 28.61 21.98 64.54RDF 37.19 22.97 73.34
SVM-RBF 30.52 21.26 74.36SVM-Linear 45.23 31.47 49.35
44
(a) k-Nearest Neighbour
(b) Random Decision Forest
(c) SVM with linear kernel
(d) SVM with RBF kernel
Figure 3.6: Comparison results using 10 fold cross validation on ICIP 2013 and MIVIA datasets. Left hand sidegraphs correspond to results on ICIP 2013 cell image classification contest training dataset [48] and the right handside graphs present results on MIVIA dataset [32].
45
Chapter 4
Conclusion and Future Directions
In this thesis, we proposed computer aided systems to assist the screening of two dominant diseases
among women - breast cancer and autoimmune diseases. To assist the clinicians in screening of breast
cancer, a visual saliency based framework is proposed which detects potential mass locations in screen-
ing mammograms. The contributions of the proposed framework are two fold: (1) utilizing saliency
maps for ROI segmentation and (2) classifying the regions into mass or non-mass classes using entropy
features derived from both DWT and RDWT. We observe that unlike existing segmentation techniques,
saliency approach does not require a prior step of pectoral muscle removal and can provide better seg-
mentation. The segmented regions are analyzed in a grid based approach using different feature sets
(e.g. texture, spatial, frequency, and wavelet) to detect masses via SVM classification. The detailed
feature analysis, including feature selection approaches, suggests that entropy features derived from both
DWT and RDWT yield the maximum classification accuracy. Based on the results from this research, we
conclude that a visual saliency algorithm trained specifically to detect symptoms from screening mam-
mograms would produce better results in the proposed framework. Clinical studies need to be performed
to establish the key properties of such a visual saliency algorithm. Further efforts would be required to
model the findings of these studies into an efficient algorithm.
In order to improve the screening practices of autoimmune diseases, we focus on the crucial HEp-2 cell
image classification problem. We proposed to use Laws texture features for identification of antigen
patterns from HEp-2 cell images. The effectiveness of Laws texture features is presented with the help
of thorough experimental analysis. To facilitate a meaningful comparison, a feature categorization based
on the object based properties of a cell image is proposed. The experimental analysis concludes that
among various properties the features derived using the intensity patterns within the cell image are most
efficient. Among various intensity based features, Laws texture features show high performance for the
HEp-2 cell image classification on both the datasets. Two key insights of this research are: 1) HEp-2 cell
image classification algorithms should consider the difference in image based properties of positive and
intermediate category images for better performance and 2) more datasets with large number of images
such as ICIP 2013 Cell Image Competition dataset are essential to adequately learn the automated models
for complex HEp-2 cell image classification task. HEp-2 cell image classification is a relatively new
area of research and based on our analysis of the existing literature, following research gaps need to be
46
addressed:
• Independent efforts have been made to automate individual modules of the complete automated
analysis (presented in Figure 3.1). Future research should focus to completely automate the entire
process.
• Efficient protocols based on practices in pattern recognition community, such as 10 fold cross
validation, should be adopted to perform evaluation of proposed techniques in a more generic
manner.
• More public datasets with large number of sample images would be helpful in substantial evalua-
tion of the proposed techniques.
47
Bibliography
[1] World Health Statistics. Tech. rep., World Health Organization (WHO), 2013.
http://www.who.int/gho/publications/world_health_statistics/
EN_WHS2013_Full.pdf?ua=1.
[2] AVRAHAM, T., AND LINDENBAUM, M. Esaliency (extended saliency): Meaningful attention us-
ing stochastic image modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence
32, 4 (2010), 693–708.
[3] BELHUMEUR, P., HESPANHA, J., AND KRIEGMAN, D. Eigenfaces vs. Fisherfaces: Recognition
Using Class Specific Linear Projection. IEEE Transactions on Pattern Analysis and Machine
Intelligence 19, 7 (1997), 711–720.
[4] BERMAN, C. G. Recent advances in breast-specific imaging. Cancer Control Journal of the
Moffitt Cancer Center 14, 4 (2007), 338–349.
[5] BIZZARO, N., ANTICO, A., PLATZGUMMER, S., TONUTTI, E., BASSETTI, D., PESENTE, F.,
TOZZOLI, R., TAMPOIA, M., AND VILLALTA, D. Automated antinuclear immunofluorescence
antibody screening: A comparative study of six computer-aided diagnostic systems. Autoimmunity
Reviews 13, 3 (2014), 292–298.
[6] BIZZARO, N., TOZZOLI, R., TONUTTI, E., PIAZZA, A., MANONI, F., GHIRARDELLO, A.,
BASSETTI, D., VILLALTA, D., PRADELLA, M., AND RIZZOTTI, P. Variability between meth-
ods to determine ANA, anti-dsDNA and anti-ENA autoantibodies: a collaborative study with the
biomedical industry. Journal of Immunological Methods 219, 12 (1998), 99–107.
[7] BONROY, C., VERFAILLIE, C., SMITH, V., PERSIJN, L., DE WITTE, E., DE KEYSER, F.,
AND DEVREESE, K. Automated indirect immunofluorescence antinuclear antibody analysis is a
standardized alternative for visual microscope interpretation. Clinical Chemistry and Laboratory
Medicine 51, 9 (2013), 1–9.
[8] BORGES SAMPAIO, W., MORAES DINIZ, E., CORREA SILVA, A., CARDOSO DE PAIVA, A.,
AND GATTASS, M. Detection of masses in mammogram images using CNN, geostatistic func-
tions and SVM. Computers in Biology and Medicine 41, 8 (2011), 653–664.
[9] BORJI, A., AND ITTI, L. State-of-the-Art in Visual Attention Modeling. IEEE Transactions on
Pattern Analysis and Machine Intelligence 35, 1 (2013), 185–207.
48
[10] BORJI, A., SIHITE, D. N., AND ITTI, L. Quantitative Analysis of Human-Model Agreement in
Visual Saliency Modeling: A Comparative Study. IEEE Transactions on Image Processing 22, 1
(2013), 55–69.
[11] BOUCHERON, L. E. Object- and Spatial-Level Quantitative Analysis of Multispectral
Histopathology Images for Detection and Characterization of Cancer. PhD thesis, University
of California, Santa Barbara, 2008.
[12] BRANDT, S., KAREMORE, G., KARSSEMEIJER, N., AND NIELSEN, M. An Anatomically Ori-
ented Breast Coordinate System for Mammogram Analysis. IEEE Transactions on Medical Imag-
ing 30, 10 (2011), 1841–1851.
[13] BREM, R. F., HOFFMEISTER, J. W., ZISMAN, G., DESIMIO, M. P., AND ROGERS, S. K. A
computer-aided detection system for the evaluation of breast cancer by mammographic appearance
and lesion size. American Journal of Roentgenology 184, 3 (2005), 893–896.
[14] BREM, R. F., RAPELYEA, J. A., ZISMAN, G., HOFFMEISTER, J. W., AND DESIMIO, M. P.
Evaluation of breast cancer with a computer-aided detection system by mammographic appear-
ance and histopathology. Cancer 104, 5 (2005), 931–935.
[15] BURT, P., AND ADELSON, E. The Laplacian Pyramid as a Compact Image Code. IEEE Trans-
actions on Communications 31, 4 (1983), 532–540.
[16] CADY, B., AND CHUNG, M. Mammographic screening: no longer controversial. American
journal of clinical oncology 28, 1 (2005), 1–4.
[17] CASTRO, C., AND GOURLEY, M. Diagnostic testing and interpretation of tests for autoimmunity.
Journal of Allergy and Clinical Immunology 125, 2 (2010), 1–23.
[18] CATALDO, S. D., BOTTINO, A., ISLAM, I. U., VIEIRA, T. F., AND FICARRA, E. Subclass dis-
criminant analysis of morphological and textural features for hep-2 staining pattern classification.
Pattern Recognition 47, 7 (2014), 2389–2399.
[19] CHANG, C. C., AND LIN, C. J. LIBSVM: A library for support vector machines. ACM Transac-
tions on Intelligent Systems and Technology 2, 3 (2011), 1–27.
[20] CHENG, H., SHI, X., MIN, R., HU, L., CAI, X., AND DU, H. Approaches for automated
detection and classification of masses in mammograms. Pattern Recognition 39, 4 (2006), 646–
668.
[21] COPPLE, S. S., GILES, S. R., JASKOWSKI, T. D., GARDINER, A. E., WILSON, A. M., AND
HILL, H. R. Screening for IgG Antinuclear Autoantibodies by HEp-2 Indirect Fluorescent An-
tibody Assays and the Need for Standardization. American Journal of Clinical Pathology 137, 5
(2012), 825–830.
[22] CORDELLI, E., AND SODA, P. Color to grayscale staining pattern representation in IIF. In
Computer Based Medical Systems (2011), pp. 1–6.
49
[23] CREEMERS, C., GUERTI, K., GEERTS, S., VAN COTTHEM, K., LEDDA, A., AND SPRUYT,
V. HEp-2 Cell Pattern Segmentation for the Support of Autoimmune Disease Diagnosis. In
International Symposium on Applied Sciences in Biomedical and Communication Technologies
(2011), pp. 28:1–28:5.
[24] DALAL, N., AND TRIGGS, B. Histograms of oriented gradients for human detection. In IEEE
Conference on Computer Vision and Pattern Recognition (2005), vol. 1, pp. 886–893.
[25] DOI, K. Computer-aided diagnosis in medical imaging: historical review, current status and future
potential. Computerized Medical Imaging and Graphics 31, 4-5 (2007), 198–211.
[26] EGERER, K., ROGGENBUCK, D., HIEMANN, R., WEYER, M.-G., BUTTNER, T., RADAU, B.,
KRAUSE, R., LEHMANN, B., FEIST, E., AND BURMESTER, G.-R. Automated evaluation of
autoantibodies on human epithelial-2 cells as an approach to standardize cell-based immunofluo-
rescence tests. Arthritis Research & Therapy 12, 2 (2010), R40.
[27] ELBISCHGER, P., GEERTS, S., SANDER, K., ZIERVOGEL-LUKAS, G., AND SINAH, P. Algo-
rithmic framework for HEp-2 fluorescence pattern classification to aid auto-immune diseases di-
agnosis. In IEEE International Symposium on Biomedical Imaging: From Nano to Macro (2009),
pp. 562–565.
[28] ELTONSY, N., TOURASSI, G., AND ELMAGHRABY, A. A Concentric Morphology Model for
the Detection of Masses in Mammography. IEEE Transactions on Medical Imaging 26, 6 (2007),
880–889.
[29] FAIRWEATHER, D., FRISANCHO-KISS, S., AND ROSE, N. R. Sex differences in autoimmune
disease from a pathological perspective. The American Journal of Pathology 173, 3 (2008), 600–
609.
[30] FARAKI, M., HARANDI, M. T., WILIEM, A., AND LOVELL, B. C. Fisher tensors for classifying
human epithelial cells. Pattern Recognition 47, 7 (2014), 2348–2359.
[31] FERRARI, R. J., RANGAYYAN, R. M., DESAUTELS, J. E. L., BORGES, R. A., AND FRERE,
A. F. Automatic identification of the pectoral muscle in mammograms. IEEE Transactions on
Medical Imaging 23, 2 (2004), 232–245.
[32] FOGGIA, P., PERCANNELLA, G., SODA, P., AND VENTO, M. Benchmarking HEp-2 Cells
Classification Methods. IEEE Transactions on Medical Imaging 32, 10 (2013), 1878–1889.
[33] FOWLER, J. The redundant discrete wavelet transform and additive noise. IEEE Signal Processing
Letters 12, 9 (2005), 629–632.
[34] FOWLER, J. E. The redundant discrete wavelet transform and additive noise. IEEE Signal Pro-
cessing Letters 12, 9 (2005), 629–632.
[35] FRIEDMAN, J. H., BENTLEY, J. L., AND FINKEL, R. A. An algorithm for finding best matches
in logarithmic expected time. ACM Transactions on Mathematical Software 3, 3 (1977), 209–226.
50
[36] FRITZLER, M. J. The antinuclear antibody test: last or lasting gasp? Arthritis and Rheumatism
63, 1 (2011), 19–22.
[37] GANESAN, K., ACHARYA, U., CHUA, C., MIN, L., ABRAHAM, K., AND NG, K.-H. Computer-
Aided Breast Cancer Detection Using Mammograms: A Review. IEEE Reviews in Biomedical
Engineering 6 (2013), 77–98.
[38] GANESAN, K., ACHARYA, U. R., CHUA, K. C., MIN, L. C., AND ABRAHAM, K. T. Pectoral
muscle segmentation: A review. Computer Methods and Programs in Biomedicine 110, 1 (2013),
48–57.
[39] GAO, X., WANG, Y., LI, X., AND TAO, D. On Combining Morphological Component Analysis
and Concentric Morphology Model for Mammographic Mass Detection. IEEE Transactions on
Information Technology in Biomedicine 14, 2 (2010), 266–273.
[40] GIGER, M. L., CHAN, H.-P., AND BOONE, J. Anniversary Paper: History and status of CAD
and quantitative image analysis: The role of Medical Physics and AAPM. Medical Physics 35, 12
(2008), 5799–5820.
[41] GONZALEZ, R. C., AND WOODS, R. E. Digital Image Processing, 2nd ed. 1992.
[42] HALKIOTIS, S., BOTSIS, T., AND RANGOUSSI, M. Automatic detection of clustered microcal-
cifications in digital mammograms using mathematical morphology and neural networks. Signal
Processing 87, 7 (2007), 1559–1568.
[43] HARALICK, R. M., SHANMUGAM, K., AND DINSTEIN, I. Textural Features for Image Classifi-
cation. IEEE Transactions on Systems, Man and Cybernetics, 6 (1973), 610–621.
[44] HAREL, J., KOCH, C., AND PERONA, P. Graph-based visual saliency. In Advances in Neural
Information Processing Systems (2007), vol. 19, pp. 545–552.
[45] HEATH, M., BOWYER, K., KOPANS, D., KEGELMEYER JR, P., MOORE, R., CHANG, K., AND
MUNISHKUMARAN, S. Current status of the Digital Database for Screening Mammography. In
International Workshop on Digital Mammography. 1998, pp. 457–460.
[46] HIEMANN, R., HILGER, N., MICHEL, J., NITSCHKE, J., BOHM, A., ANDERER, U., WEIGERT,
M., AND SACK, U. Automatic analysis of immunofluorescence patterns of HEp-2 cells. Annals
of the New York Academy of Sciences 1109, 1 (2007), 358–371.
[47] HO, T. K. Random decision forests. In IEEE International Conference on Document Analysis
and Recognition (1995), vol. 1, pp. 278–282.
[48] HOBSON, P., PERCANNELLA, G., VENTO, M., AND WILIEM, A. Competition on cells classi-
fication by fluorescent image analysis. In International Conference on Image Processing (2013).
http://nerone.diiie.unisa.it/contest-icip-2013/index.shtml.
51
[49] HOBSON, P., PERCANNELLA, G., VENTO, M., AND WILIEM, A. Contest on performance
evaluation of indirect immunofluorescence image analysis systems. In International Conference
on Pattern Recognition (2014). http://i3a2014.unisa.it/?page_id=91.
[50] HONG, B. W., AND SOHN, B. S. Segmentation of regions of interest in mammograms in a to-
pographic approach. IEEE Transactions on Information Technology in Biomedicine 14, 1 (2010),
129–139.
[51] HOU, X., AND ZHANG, L. Saliency detection: A spectral residual approach. In IEEE Conference
on Computer Vision and Pattern Recognition (2007), pp. 1–8.
[52] HUANG, Y.-L., JAO, Y.-L., HSIEH, T.-Y., AND CHUNG, C.-W. Adaptive Automatic Segmen-
tation of HEp-2 Cells in Indirect Immunofluorescence Images. In IEEE International Conference
on Sensor Networks, Ubiquitous, and Trustworthy Computing (2008), pp. 418–422.
[53] KARSSEMEIJER, N., AND TE BRAKE, G. Detection of stellate distortions in mammograms. IEEE
Transactions on Medical Imaging 15, 5 (1996), 611–619.
[54] KOM, G., TIEDEU, A., AND KOM, M. Automated detection of masses in mammograms by local
adaptive thresholding. Computers in Biology and Medicine 37, 1 (2007), 37–48.
[55] KONG, X., LI, K., CAO, J., YANG, Q., AND WENYIN, L. Hep-2 cell pattern classification with
discriminative dictionary learning. Pattern Recognition 47, 7 (2014), 2379–2388.
[56] KUS, P., AND KARAGOZ, I. Fully automated gradient based breast boundary detection for digi-
tized X-ray mammograms. Computers in Biology and Medicine 42, 1 (2012), 75–82.
[57] KWOK, S., CHANDRASEKHAR, R., AND ATTIKIOUZEL, Y. Automatic pectoral muscle segmen-
tation on mammograms by straight line estimation and cliff detection. In Intelligent Information
Systems Conference (2001), pp. 67–72.
[58] LAI, S., LI, X., AND BISCOF, W. On techniques for detecting circumscribed masses in mammo-
grams. IEEE Transactions on Medical Imaging 8, 4 (1989), 377–386.
[59] LAWS, K. I. Textured Image Segmentation. PhD thesis, University of Southern California, 1980.
[60] LEVIN, D. A., PERES, Y., AND WILMER, E. L. Markov Chains and Mixing Times. American
Mathematical Society, 2009.
[61] LI, H. D., KALLERGI, M., CLARKE, L. P., JAIN, V. K., AND CLARK, R. A. Markov random
field for tumor detection in digital mammography. IEEE Transactions on Medical Imaging 14, 3
(1995), 565–576.
[62] LIU, L., AND WANG, L. Hep-2 cell image classification with multiple linear descriptors. Pattern
Recognition 47, 7 (2014), 2400–2408.
52
[63] LIU, T., YUAN, Z., SUN, J., WANG, J., ZHENG, N., TANG, X., AND SHUM, H.-Y. Learning
to Detect a Salient Object. IEEE Transactions on Pattern Analysis and Machine Intelligence 33,
2 (2011), 353–367.
[64] LOWE, D. G. Distinctive Image Features from Scale-Invariant Keypoints. International Journal
of Computer Vision 60, 2 (2004), 91–110.
[65] MENCATTINI, A., RABOTTINO, G., SALMERI, M., AND LOJACONO, R. Assessment of a Breast
Mass Identification Procedure Using an Iris Detector. IEEE Transactions on Instrumentation and
Measurement 59, 10 (2010), 2505–2512.
[66] MERONI, P. L., AND SCHUR, P. H. ANA screening: an old test with new recommendations.
Annals of the Rheumatic Diseases 69, 8 (2010), 1420–1422.
[67] MUSTRA, M., AND GRGIC, M. Robust automatic breast and pectoral muscle segmentation from
scanned mammograms. Signal Processing 93, 10 (2013), 2817–2827.
[68] NANNI, L., PACI, M., AND BRAHNAM, S. Indirect immunofluorescence image classification
using texture descriptors. Expert Systems with Applications 41, 5 (2014), 2463–2471.
[69] NOSAKA, R., AND FUKUI, K. Hep-2 cell classification using rotation invariant co-occurrence
among local binary patterns. Pattern Recognition 47, 7 (2014), 2428–2436.
[70] OJALA, T., PIETIKAINEN, M., AND HARWOOD, D. A comparative study of texture measures
with classification based on featured distributions. Pattern Recognition 29, 1 (1996), 51–59.
[71] OLIVER, A., FREIXENET, J., MARTI, J., PEREZ, E., PONT, J., DENTON, E. R. E., AND
ZWIGGELAAR, R. A review of automatic mass detection and segmentation in mammographic
images. Medical Image Analysis 14, 2 (2010), 87–110.
[72] OTSU, N. A Threshold Selection Method from Gray-Level Histograms. IEEE Transactions on
Systems, Man, and Cybernetics 9, 1 (1979), 62–66.
[73] PENG, H., LONG, F., AND DING, C. Feature selection based on mutual information criteria of
max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis
and Machine Intelligence 27, 8 (2005), 1226–1238.
[74] PERNER, P., PERNER, H., AND MULLER, B. Mining knowledge for HEp-2 cell image classifi-
cation. Artificial Intelligence in Medicine 26, 1 (2002), 161–173.
[75] PETRICK, N., CHAN, H. P., SAHINER, B., AND WEI, D. An adaptive density-weighted contrast
enhancement filter for mammographic breast mass detection. IEEE Transactions on Medical
Imaging 15, 1 (1996), 59–67.
[76] PISANO, E. D., GATSONIS, C., HENDRICK, E., YAFFE, M., BAUM, J. K., ACHARYYA, S.,
CONANT, E. F., FAJARDO, L. L., BASSETT, L., D’ORSI, C., JONG, R., AND REBNER, M.
Diagnostic Performance of Digital versus Film Mammography for Breast-Cancer Screening. New
England Journal of Medicine 353, 17 (2005), 1773–1783.
53
[77] POLAKOWSKI, W. E., COURNOYER, D. A., ROGERS, S. K., DESIMIO, M. P., RUCK, D. W.,
HOFFMEISTER, J. W., AND RAINES, R. A. Computer-aided breast cancer detection and di-
agnosis of masses using Difference of Gaussians and derivative-based feature saliency. IEEE
Transactions on Medical Imaging 16, 6 (1997), 811–819.
[78] PONOMAREV, G. V., ARLAZAROV, V. L., GELFAND, M. S., AND KAZANOV, M. D. Ana hep-
2 cells image classification using number, size, shape and localization of targeted cell regions.
Pattern Recognition 47, 7 (2014), 2360–2366.
[79] RAHMATI, P., ADLER, A., AND HAMARNEH, G. Mammography segmentation with maximum
likelihood active contours. Medical Image Analysis 16, 6 (2012), 1167–1186.
[80] RANGAYYAN, R., AYRES, F., AND LEO DESAUTELS, J. A review of computer-aided diagnosis
of breast cancer: Toward the detection of subtle signs. Journal of the Franklin Institute 344 (2007),
312–348.
[81] RANGAYYAN, R., SHEN, L., SHEN, Y., DESAUTELS, J. E. L., BRYANT, H., TERRY, T.,
HORECZKO, N., AND ROSE, M. Improvement of sensitivity of breast cancer diagnosis with
adaptive neighborhood contrast enhancement of mammograms. IEEE Transactions on Informa-
tion Technology in Biomedicine 1, 3 (1997), 161–170.
[82] RIGON, A., BUZZULINI, F., SODA, P., ONOFRI, L., ARCARESE, L., IANNELLO, G., AND
AFELTRA, A. Novel opportunities in automated classification of antinuclear antibodies on HEp-2
cells. Autoimmunity Reviews 10, 10 (2011), 647–652.
[83] RIGON, A., SODA, P., ZENNARO, D., IANNELLO, G., AND AFELTRA, A. Indirect Immunoflu-
orescence in Autoimmune Diseases : Assessment of Digital Images for Diagnostic Purpose. Cy-
tometry. Part B: Clinical Cytometry 72, 6 (2007), 472–477.
[84] RIOUX, J. D., AND ABBAS, A. K. Paths to understanding the genetic basis of autoimmune
disease. Nature 435, 7042 (2005), 584–589.
[85] SATOH, M., CHAN, E. K. L., HO, L. A., ROSE, K. M., PARKS, C. G., COHN, R. D., JUSKO,
T. A., WALKER, N. J., GERMOLEC, D. R., WHITT, I. Z., CROCKETT, P. W., PAULEY, B. A.,
CHAN, J. Y. F., ROSS, S. J., BIRNBAUM, L. S., ZELDIN, D. C., AND MILLER, F. W. Prevalence
and sociodemographic correlates of antinuclear antibodies in the United States. Arthritis and
Rheumatism 64, 7 (2012), 2319–2327.
[86] SENTHILKUMAR, B., AND UMAMAHESHWARI, G. A Review on Computer Aided Detection and
Diagnosis - Towards the treatment of Breast Cancer. European Journal of Scientific Research 52,
4 (2011), 437–452.
[87] SHEN, L., LIN, J., WU, S., AND YU, S. Hep-2 image classification using intensity order pooling
based features and bag of words. Pattern Recognition 47, 7 (2014), 2419–2427.
54
[88] SINGH, S., AND BOVIS, K. An evaluation of contrast enhancement techniques for mammo-
graphic breast masses. IEEE Transactions on Information Technology in Biomedicine 9, 1 (2005),
109–119.
[89] SNELL, V., CHRISTMAS, W., AND KITTLER, J. Hep-2 fluorescence pattern classification. Pat-
tern Recognition 47, 7 (2014), 2338–2347.
[90] SODA, P., AND IANNELLO, G. Aggregation of classifiers for staining pattern recognition in antin-
uclear autoantibodies analysis. IEEE Transactions on Information Technology in Biomedicine 13,
3 (2009), 322–329.
[91] STOKLASA, R., MAJTNER, T., AND SVOBODA, D. Efficient k-nn based hep-2 cells classifier.
Pattern Recognition 47, 7 (2014), 2409–2418.
[92] SUCKLING, J., PARKER, J., DANCE, D., ASTLEY, S., HUTT, I., BOGGIS, C., RICKETTS, I.,
STAMATAKIS, E., CERNEAZ, N., KOK, S., TAYLOR, P., BETAL, D., AND SAVAGE, J. The
mammographic images analysis society digital mammogram database. Experta Medica Interna-
tional Congress Series 1069 (1994), 375–378.
[93] TANG, J., RANGAYYAN, R. M., XU, J., EL NAQA, I., AND YANG, Y. Computer-aided detec-
tion and diagnosis of breast cancer with mammography: recent advances. IEEE Transactions on
Information Technology in Biomedicine 13, 2 (2009), 236–251.
[94] TANG, X. Texture information in run-length matrices. IEEE Transactions on Image Processing
7, 11 (1998), 1602–1609.
[95] THEODORAKOPOULOS, I., KASTANIOTIS, D., ECONOMOU, G., AND FOTOPOULOS, S. Hep-
2 cells classification via sparse representation of textural features fused into dissimilarity space.
Pattern Recognition 47, 7 (2014), 2367–2378.
[96] VAPNIK, V. N. Statistical Learning Theory, 1st ed. 1998.
[97] VARELA, C., TAHOCES, P. G., MENDEZ, A. J., SOUTO, M., AND VIDAL, J. J. Computerized
detection of breast masses in digitized mammograms. Computers in Biology and Medicine 37, 2
(2007), 214–226.
[98] VELIKOVA, M., LUCAS, P. J., SAMULSKI, M., AND KARSSEMEIJER, N. A probabilistic frame-
work for image information fusion with an application to mammographic analysis. Medical Image
Analysis 16, 4 (2012), 865–875.
[99] VYBORNY, C. J., AND GIGER, M. L. Computer vision and artificial intelligence in mammogra-
phy. American Journal of Roentgenology 162, 3 (1994), 699–708.
[100] WANG, Y., GAO, X., AND LI, J. A Feature Analysis Approach to Mass Detection in Mam-
mography Based on RF-SVM. In International Conference on Image Processing (2007), vol. 5,
pp. 9–12.
55
[101] WILIEM, A., SANDERSON, C., WONG, Y., HOBSON, P., MINCHIN, R. F., AND LOVELL, B. C.
Automatic classification of human epithelial type 2 cell indirect immunofluorescence images using
cell pyramid matching. Pattern Recognition 47, 7 (2014), 2315–2324.
[102] YANG, Y., WILIEM, A., ALAVI, A., LOVELL, B. C., AND HOBSON, P. Visual learning and
classification of human epithelial type 2 cell images through spontaneous activity patterns. Pattern
Recognition 47, 7 (2014), 2325–2337.
56
Chapter 5
Appendices
5.1 Fourier Domain Features
Three high level features are calculated which represent energy at high, medium, and low frequencies
(Eq. 5.1- 5.3), other 32 energies are extracted from entire frequency spectrum at intervals of 32 using
Eq. 5.4.
FFT low =340∑n=0
|FFT ROI(n)|2 (5.1)
FFTmed =683∑
n=341
|FFT ROI(n)|2 (5.2)
FFThigh =1023∑n=684
|FFT ROI(n)|2 (5.3)
FFTh =
32·(h+1)−1∑n=32·h
|FFT ROI(n)|2, where h = 0, 1, ..., 31 (5.4)
5.2 Intensity Features
Mean Intensity: Mean Intensity corresponds to the mean value of pixels in region R [61].
m =1
N
∑(i,j)∈R
I(i, j) (5.5)
57
Intensity Variation: Intensity variation corresponds to the standard deviation of pixels in region R [61].
var =
√1
N
∑(I(i, j)−m)2 , where (5.6)
m =1
N
∑I(i, j) (5.7)
where I(i, j) represents the image pixel value at (i, j) location and N the number of points in R.
Mean Intensity Difference: Mean intensity difference is the difference between the mean intensity of pix-
els within the region R and the mean intensity of pixels in the neighborhood Ne [61]. The neighborhood
Ne is defined as region obtained after subtracting circular region ci from square region s2:
Ne = s2− ci (5.8)
Skewness: Skewness measures the asymmetry among data around the sample mean. Skewness of the
distribution formed by pixels of region R is defined as third standardized moment of the distribution:
s =E(x− µ)3
σ3(5.9)
Kurtosis: Kurtosis measures the robustness of the distribution towards outliers. Kurtosis of the distribu-
tion formed by pixels of region R is defined as fourth standardized moment of the distribution:
k =E(x− µ)4
σ4(5.10)
where µ and σ are mean and standard deviation of the distribution andE(x) gives expectation of variable
x.
5.3 Laws Texture Features
Laws [59] proposed three basic kernels [1 2 1], [−1 0 −1], [−1 2 −1] to represent texture information.
Combinations of the three basic kernels further produce five kernels [1 4 6 4 1], [−1 0 2 0 −1], [1 −46 −4 1], [−1 −2 0 2 1], [−1 2 0 −2 1] that represent edges, ripples, waves, lines, and spots in a square
region. Texture filters are generated from these kernels by multiplication of two same size kernels, as a
result 9 filters of size 3 × 3 and 25 filters of size 5 × 5 can be formed. These 34 filters are convolved
with the given image to obtain the texture maps. From these texture maps three statistical measures -
mean, absolute mean, and standard deviation are computed as feature values. In total, 102 features are
derived from the given image. Eight feature masks used to compute Laws texture features are illustrated
in Figure 5.1.
58
-1 -2 0 2 1
2 4 0 -4 -2
0 0 0 0 0
-2 -4 0 4 2
1 2 0 -2 -1
-1 -2 0 2 1
-2 -4 0 4 2
0 0 0 0 0
2 4 0 -4 -2
1 2 0 -2 -1
-1 0 2 0 1
0 0 0 0 0
2 0 -4 0 2
0 0 0 0 0
-1 0 2 0 1
-1 4 -6 4 -1
0 0 0 0 0
2 -8 12 -8 2
0 0 0 0 0
-1 4 -6 4 -1
-1 0 2 0 -1
4 0 -8 0 4
-6 0 12 0 -6
4 0 -8 0 4
-1 0 2 0 -1
-1 0 2 0 -1
-4 0 8 0 -4
-6 0 12 0 -6
-4 0 8 0 -4
-1 0 2 0 -1
-1 2 0 -2 1
4 -8 0 8 -4
-6 12 0 -12 6
4 -8 0 8 -4
-1 2 0 -2 1
1 -4 6 -4 1
4 -16 24 -16 4
6 -24 36 -24 6
4 -16 24 -16 4
1 -4 6 -4 1
( a ) w5r5 ( b ) r5r5 ( c ) s5s5 ( d ) s5e5
( e ) e5s5 ( f ) l5s5 ( g ) e5w5 ( h ) l5e5
Figure 5.1: Feature masks generated using the kernels proposed by Laws [59].
5.4 Statistical Texture Features
Following equations define the notations helpful in describing features extracted from GLCM matrices,
Cx(i) =
Ng∑j=1
C(i, j) (5.11)
Cy(j) =
Ng∑i=1
C(i, j) (5.12)
Cx+y(k) =
∑Ng
i=1
∑Ng
j=1 C(i, j), k = 2, 3, ..., 2Ng
i+ j = k(5.13)
Cx−y(k) =
∑Ng
i=1
∑Ng
j=1 C(i, j), k = 0, 1, ..., Ng − 1
|i− j| = k(5.14)
where Ng is the number of distinct gray levels, 8 in our case. With the help of notations defined above,
13 features are computed using the equations below:
• Contrast or Inertia:
f1 =
Ng−1∑n=0
n2
{∑Ng
i=1
∑Ng
j=1 C(i, j)
|i− j| = n
}(5.15)
• Correlation:
f2 =
∑i
∑j ijC(i, j)− µxµy
σxσy(5.16)
59
where µx, µy, σx, σy are the means and standard deviations of matrices Cx and Cy respectively.
• Energy or Angular Second Moment:
f3 =∑i
∑j
C(i, j)2 (5.17)
• Sum of squares:
f4 =∑i
∑j
(i− µ)2C(i, j) (5.18)
where µ is the mean of the GLCM matrix.
• Entropy:
f5 = −∑i
∑j
C(i, j)log(C(i, j)) (5.19)
• Inverse Difference Moment:
f6 =∑i
∑j
1
1 + (i− j)2C(i, j) (5.20)
• Sum Average:
f7 =
2Ng∑k=2
kCx+y(k) (5.21)
• Sum Variance:
f8 =
2Ng∑k=2
(k − f7)2Cx+y(k) (5.22)
• Sum Entropy:
f9 = −2Ng∑k=2
Cx+y(k)log(Cx+y(k)) (5.23)
• Difference Variance:
f10 =∑k
(k − µx−y)2Cx−y(k) (5.24)
where µx−y is the mean of Cx−y.
• Difference Entropy:
f11 = −Ng−1∑k=0
Cx−y(k)log(Cx−y(k)) (5.25)
• Information measures of Correlation:
f12 =HXY −HXY1max(HX,HY )
(5.26)
60
f13 = [1− e−2(HXY2−HXY )]12 (5.27)
HXY = −∑i
∑j
C(i, j)log(C(i, j)) (5.28)
HX = −∑i
Cx(i)log(Cx(i)) (5.29)
HY = −∑j
Cy(j)log(Cy(j)) (5.30)
HXY1 = −∑i
∑j
C(i, j)log(Cx(i)Cy(j)) (5.31)
HXY2 = −∑i
∑j
Cx(i)Cy(j)log(Cx(i)Cy(j)) (5.32)
C is one of the four GLCM matrices, similar computations are made for all four matrices.
5.5 Run Length Texture Features
Run length distribution r(j|θ) and gray-level distribution g(i|θ) are derived from each of the four GLRL
matrices G(x, y|θ) as,
r(j|θ) =∑i
G(i, j|θ) (5.33)
g(i|θ) =∑j
G(i, j|θ) (5.34)
The total number of runs in the image S are computed as,
S =∑i
∑j
G(i, j|θ) (5.35)
Following features are extracted from each of the four GLRL matrices using notations defined in Eq. 5.33-
5.35,
• Short Runs Emphasis (SRE):
SRE =1
S
∑j
r(j|θ)j2
(5.36)
• Long Runs Emphasis (LRE):
LRE =1
S
∑j
r(j|θ)j2 (5.37)
61
• Gray Level Non-uniformity (GLN):
GLN =1
S
∑i
g(i|θ)2 (5.38)
• Run Length Non-uniformity (RLN):
RLN =1
S
∑j
r(j|θ)2 (5.39)
• Run Percentage (RP):
RP =1
Area
∑j
r(j|θ) (5.40)
where Area is the number of pixels in region R.
62