Date post: | 27-Dec-2015 |
Category: |
Documents |
Upload: | arthur-rice |
View: | 220 times |
Download: | 0 times |
Neural Network Ensemble based on Feature Selection for Non-Invasive Recognition of
Liver Fibrosis StageBartosz KRAWCZYK, Michał WOŹNIAK, Tomasz ORCZYK,
Piotr PORWIK, Joanna MUSIALIK, Barbara BŁOŃSKA-FAJFROWSKA
Presentation agenda Overview. Current diagnostic methods. Proposed method.
Analyzed data. Data analysis methods. Result comparison.
Conclusions.
Overview Liver fibrosis:
Accumulation of tough, fibrous scar tissue in the liver. ~1,75% of Poland’s population is infected with HCV. Unthreated may cause Liver Cirrhosis and death.
Risk factors: Chronic infection with hepatitis C or hepatitis B virus (HCV,
HBV). Immune system compromise (HIV or immunosuppressive
drugs). Heavy alcohol consumption.
Gradation indexes: Knodell Histological Activity Index (HAI Score). Ishak system. METAVIR system.
Current diagnostic methods Invasive
Liver biopsy Risk of health complications or even death. Up to 45% uncertainty depending on bioptate quality
and size. Still assumed as a „gold standard”.
Non-invasive ELF Test FibroTest & FibroScan
Expensive Not very accurate
Proposed method Non-invasive
Blood test based
Inexpensive Only regular blood tests
Comparable with other non-invasive methods Similar error level to FibroTest
Proposed method: Analyzed data and problems
Data characteristics: 127 patients mostly with HCV (70%) and Liver
Fibrosis. All patients otherwise healthy and not under
therapy. 34 parameters measured.
Problems: Low data samples count. Unequal distribution of diagnosed fibrosis level. Incomplete records. Many poor quality biopsies.
Proposed method: Analyzed data and problems
Age* (years) 50 (13)
Male, n(%) 75 (59%)
Biopsy result, n(%)
F0 2 (2%)
F1 35 (28%)
F2 5 (4%)
F3 16 (13%)
F4 67 (53%)
HCV/HBV/other 70% / 9% / 21%
HB* (g/L) 14 (1.91)RBC* (106/UL) 5 (0.74)WBC* (103/UL) 6 (2.31)PLT* (103/UL) 161 (70.75)PT* (sec.) 13 (9.04)PTP* (%) 90 (17.82)APTT* (sec.) 38 (12.53)INR* 1 (0.26)ASPT* (IU/L) 65 (51.01)ALAT* (IU/L) 72 (61.81)ALP* (IU/L) 104 (55.11)BIL* (mg/dL) 2 (2.69)GGTP* (IU/L) 89 (94.43)KREA* (mg/dL) 1 (0.23)GLU* (mg/dL) 95 (19.02)Na* (mmol/L) 138 (3.48)
K* (mmol/L) 5 (5.16)Fe* (mmol/L) 104 (70.23)CRP* (IU/L) 4 (25.38)TG* (mg/dL) 107 (50.83)CHO* (mg/dL) 189 (51.04)Ur. acid* (mg/dL) 6 (1.39)TP* (g/dL) 7 (0.81)TIBC* 322 (120.47)Neutr* (103/UL) 3 (1.35)Lymph* (103/UL) 2 (0.55)Mono* (103/UL) 1 (0.19)Eos* (103/UL) 0 (0.13)Baso* (103/UL) 0 (0.02)Albu* (%) 58 (7.79)Glb. α1
* (%) 3 (1.33)Glb. α2
* (%) 8 (2.52)Glb. β* (%) 11 (2.43)Glb. γ* (%) 19 (7.21)
Proposed method: Neural Network Ensemble
The introduced method of classifier ensemble design consists of three main steps:
Building the pool of individual classifiers.
Pruning the acquired pool by discarding redundant predictors.
Using a sophisticated trained fuser to deliver the ensemble.
Proposed method:Building the pool of classifiers
Models should be complementary to each other, exhibiting at the same time high accuracy and high diversity.
There is no single optimal approach for feature selection task and results obtained on the basis of different methods may differ significantly.
Instead of selecting a single best feature selection method we use several of them to reduce the dimensionality of the feature space.
Proposed method:Ensemble pruning
There are several different ways in the literature on how to select valuable members to the committee.
Ideal ensemble consists of classifiers of high individual accuracy and high diversity.
Among diversity measures there are two major types: Pairwise (shows how two classifiers differ from each
other). Non-pairwise (measure the diversity of the whole
ensemble). For measuring the diversity of whole ensemble
we used the entropy measure.
Proposed method: Fusion of individual classifiers
Classifier fusion algorithms can make decisions on the basis of class labels given by individual classifiers or they can construct new discriminant functions on the basis of individual classifier support functions: The first group includes voting algorithms. The second group is based on discriminant
analysis.
The design of improved fusion classification models, especially trained fusers, is the focus of current research.
Proposed method: Fusion of individual classifiers
Assume that we have K classifiers in a pool after the pruning procedure.For a given object each individual classifier decides for class based on the values of discriminants. Let denote a function that is assigned to class i for a given value of x, and that is used by the l-th classifier . The combined classifier uses the decision rule , where .The weights can be set dependent on the classifier and class number: weight is assigned to the l-th classifier and the i-th class, and given classifier weights assigned to different classes may differ.
)()2()1( ,...,, K
Xx Mi ,...,1
xiF l ,)(
)(l
),(ˆmax),(ˆ)( xkFxiFifixMk
K
i
lK
l
ll andxiFxiF1
)(
1
)()( 1),(),(ˆ
)()( il
Proposed method: Feature selection algorithms
Eight different feature selection algorithms were used, namely: ReliefF, Fast Correlation Based Filter, Genetic Wrapper, Simulated Annealing Wrapper, Forward Selection, Backward Selection, Quick Branch & Bound, Las Vegas Incremental.
Neural network architecture was as follows: the number of neurons in the input layer was equal to the number of selected features, the number of output neurons was equal to the number of classes and the number of hidden neurons was equal to the half of the sum of number of neurons from the former layers.
Proposed method:Set-up
As reference methods we have selected most popular ensembles - Bagging, Boosting, Random Forest and Random Subspace.
Additionally we have compared our method with the single best classier from the pool, all classifiers from the pool and with simple majority voting.
The combined 5x2 CV F test [1] was carried out to asses the statistical signicance of obtained results.
Proposed method: Results
The proposed neural network ensemble, based on feature selection methods, has outperformed all the previously used MCS for this problem.
The weakest results were returned by single best model approach, which highlights the usefulness of utilizing more than one classier to fully exploit the outputs of feature selection methods.
Second biggest accuracy boost lies in the used fuser-trained fusion of individual classiers allows to derive an optimal linear combination of them.
The pruning step had smallest but still statistically significant impact on the ensemble design.
Conclusions: The presented paper shows that, despite some
problems it is possible to reach similar or even lower error level than commercial tests.
It is also worth to mention that liver biopsy result, according to the other research, is also only a prediction with classification error varying from 35% up to 45% , depending on the sample size and count.
we proved that each of the three steps embedded in the proposed committee design has an important impact on the quality of the final prediction and thus should not be omitted.