1
Sumaiya Iqbal and Md Tamjidul Hoque gratefully acknowledge the Louisiana Board of Regents through the Board of Regents Support Fund, LEQSF (2013-16)-RD-A-19. • The usual goal of feature selection is to identify and remove all irrelevant and redundant features • Redundant features provide an opportunity to mitigate or at least predict performance loss due to missing data • Selected features may provide insights of genes correlated with the disease • Feature selection may be a form of overfitting training data • A validation dataset is crucial to the feature selection process Ideally, we would only use features present in the top row to build our final model. However, we may choose to build additional models with features from additional rows. If future test samples are missing features in the top row we may choose a model constructed with an alternative set of features. DNA microarray data allows the analysis of the expression level of thousands of genes simultaneously. This process can capture the current state of the gene regulation within a cell by capturing mRNA expressions, instead of tedious quantitate and qualitative measurement of protein expressions, which would have been more accurate measure of the cellular activities. As we are measuring the indirect interaction using mRNA expression, we therefore need to have robust approaches to infer the true statistics. This approach will make it possible to have clinically and/or scientifically useful predictions such as diagnosing diseases, the identification of tumor types and treatment selection. Many statistical classification methods are available for this type of task. Further, a central difficulty in such statistical classification is that, some of the features (variables) in the data may be irrelevant or redundant to the prediction task. Irrelevant and redundant data complicate and confound the classification process, therefore, it is desirable to identify and eliminate variables that are not useful for the classification task. The aim of this research is to propose a robust methodology for classifying DNA microarray data using feature selection, which is the process of identifying and eliminating features that are irrelevant or redundant. The proposed method performs effective feature selection to identify a subset of genes that best describe a disease. Two well-known DNA microarray datasets were used to validate the method. A Hybrid Evolutionary Feature Selection Method for Microarray Data Denson Smith, Sumaiya Iqbal, Md Tamjidul Hoque email: {dsmith8, siqbal1, thoque}@uno.edu Department of Computer Science, University of New Orleans, New Orleans, LA, USA Method Abstract Results and Discussion Conclusions Future Work Acknowledgements MCC = ( TP × TN ) − (FP × FN ) ( TP + FP )( TP + FN )( TN + FP )( TN + FN ) where , TP = the number of true positives TN = the number of true negatives FP = the number of false positives FN = the number of false negatives Extra Tree Classifier For validation and the final model, the ET is tuned to maximize classification performance. Other classifiers such as deep neural network and support vector machine may also be trained on the selected features. The extra tree classifier provides the genetic algorithm with two pieces of information about each candidate features set. Predictions from the ET are used to generate fitness estimates for the genetic algorithm. Feature importance estimates from the ET are used to remove features estimated to be unimportant from some of the current generation’s offspring. If these features are indeed unimportant (irrelevant) then the offspring will have an equal or higher fitness estimate compared with its parents. Heatmap of the breast cancer candidate feature sets ranked by Matthews Correlation Coefficient Detail of heatmap Darker colors indicate features that appear in more candidate feature sets. Lighter colors indicate features that appear in fewer candidate feature sets. Features that do not appear in any candidate feature set are likely to be irrelevant. Rows with equal or near equal performance but different features likely contain features that are mutually redundant. A set of 10 candidate features is generated for each fitness metric: 1. MCC 2. AUC 3. accuracy 4. F1 5. (MCC+AUC)/2 6. (F1+AUC)/2 7. (accuracy+AUC)/2 8. (precision+recall)/2 During the feature selection process, the ET parameters are tuned to maximize the accuracy of feature importances. Features that generate higher information gain at more nodes are estimated to be more important. Information gain is measured by Gini purity or information entropy. best MCC found metric: accuracy+AUC elite: 4 # features 32 AUC 0.8571 accuracy 0.9474 precision 1.0000 recall 0.8571 F1 0.9231 MCC 0.8895 all features metric: None # features 24187 AUC 0.8393 accuracy 0.8421 precision 0.8333 recall 0.7143 F1 0.7692 MCC 0.6548 Performance [1] Huerta, E. B., Duval, B. and Hao, J.-K. Gene selection for microarray data by a LDA-based genetic algorithm. Springer, City, 2008. [2] Sahu, B. and Mishra, D. A novel feature selection algorithm using particle swarm optimization for cancer microarray data. Procedia Engineering, 382012), 27-31. [3] Garro, B. A., Rodríguez, K. and Vázquez, R. A. Classification of DNA microarrays using artificial neural networks and ABC algorithm. Applied Soft Computing, 382016), 548-560. [4] Sasikala, S., alias Balamurugan, S. A. and Geetha, S. A Novel Feature Selection Technique for Improved Survivability Diagnosis of Breast Cancer. Procedia Computer Science, 502015), 16-23. References • PSO – particle swarm optimization • ABC – artificial bee colony • GFFS – genetic forest feature selector • GA – genetic algorithm • J48 – decision tree • LDAGA – linear discriminate analysis genetic algorithm • Filter – correlation of individual gene expression with target class Overfitting? • Some candidate feature sets that performed well with the training data performed very poorly with the validation data. • This is likely due to spurious relationships between irrelevant features and the target class. • If this is the cause then feature selection may be viewed as a form of overfitting the training data. • This illustrates why a validation dataset is crucial. Classifica?on technique Selec?on technique # of genes % accuracy Reference SVM PSO 20 1.0000 [2] SVM ABC 5 0.9470 [3] ET GFFS 32 0.9470 Proposed method J48 GA 41 0.9381 [4] SVM Filter + LDA-GA 44 0.8421 [1] Comparison with Other Methods • Dimensionality greatly reduced • Substantial improvement of all performance metrics • The best MCC was generated from a candidate set selected with accuracy+AUC as the fitness metric for the GA • Reapply feature selection using only the candidate feature sets to determine if results improve • Attempt to reduce overfitting of the training data during feature selection • Formalize the method of choosing an alternative feature set in the case of missing data • Complete the process on additional microarray datasets • Complete the process on datasets from different problem domains
