8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
1/29
TRANSFORM BASED FUSION
ALGORITHM FOR MEDICAL IMAGES
Presented By---
Sneha Singh
M.Tech. 2ndyear
(12528023)
Department of Electrical Engineering
Indian Institute of Technology Roorkee, Roorkee
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
2/29
TOPICS TO BE DISCUSSED
Medical imaging
Fusion algorithms
Process flow
Image datasets
Performance measures
Results and Discussions
Comparative Analysis
Conclusion
References
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
3/29
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
4/29
LITERATURE REVIEW
S. No. Authors Refs. Year Fusion Techniques
1 H. Li et al. [3] 1995 This fusion scheme is based on discrete wavelet transform (DWT).
2 S. Li et al. [6] 2002 This paper presented the combination of DWT and artificial neural
networks (ANN) for pixel level multifocus image fusion.
3 G. Piella [8] 2003 This paper proposed a general framework on pixel to pixel fusion
schemes.
4. B. Xu et al. [53] 2004 This paper presented fusion algorithm based on pulse coupled neural
network (PCNN).
5 W. Li and X
Zhu
[11] 2005 In this paper the algorithm is based on wavelet packet analysis (WPA)
and pulse-coupled neural networks (PCNN).
6 W. Li and X-F
Zhu
[10] 2005 This technique represents medical image fusionbased on pulse-coupled
neural networks (PCNN).
7 Q. Miao and
B. Wang
[13] 2006 Contourlet transform is used in this paper for image fusion .
8 A. Wang et al. [14] 2006 This method demonstrate the application of wavelet transformation to
multi-modality medicalimage fusion.
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
5/29
LITERATURE REVIEW
S. No. Authors Refs. Year Fusion Techniques
9 B. Yang et al. [15] 2007 This technique presented the use of nonsubsampled contourlet transform
(NSCT) for image fusion.
10 D. Agrawal
and J. Singhai
[21] 2010 In this method a modifiedapproach of PCNNis proposed by reducing the
processing time and computational complexity.
11 Q.-g. Miao etal.
[45] 2011 In this paper image fusion by using Shearlet transform is presented.
12 C. Yuan et al. [49] 2011 This paper proposed image fusion by using Nonsubsampled shearlet
transform (NSST).
13 S. Das and
M.K. Kundu
[30] 2012 This paper proposed NSCTand the PCNNwith modified spatial frequency
(MSF) in order to obtain better fusion results .
14 P. Geng et.al. [31] 2012 Image fusion with PCNN in shearlet domain is presented.
15 X. Sun et al. [34] 2013 This paper presented image fusion based on NSCT domain with improved
contrast.
16 P. Ganasala
and V. Kumar
[54] 2014 This paper presented CT and MR image fusion scheme in NSCT domain
with improved image contrast.
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
6/29
PROCESS FLOW
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
7/29
CT images
MR images
IMAGE DATASETS
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
8/29
Block representation of fusion technique in transform domain
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
9/29
PERFORMANCE MEASURES
S.NO. Performance Measures Mathematical Expression
1 Entropy (EN) )
=
2 Standard deviation (STD)
,
=
=
3 Mutual Information (MI) ; ; where ; ,(,) , (,)()()
=
=
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
10/29
PERFORMANCE MEASURES
S.NO.
Performance Measures Mathematical Expression
4 Spatial Frequency (SF) where
1
(1) ( , 1 (, ))
=
=
1
(1)
( 1, (, ))
=
=
5 Image Quality Index (IQI) 2
2 and
0 , (, )
2
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
11/29
PERFORMANCE MEASURES
S.NO. Performance Measures Mathematical Expression
6 Edge Strength ,
, (,)== , (, , (,)==where
+( ,) and 1 ( , )
, , , ; , > , , , ;
, 1 , ,
/2
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
12/29
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
13/29
SUBJECTIVE ANALYSIS OF VARIOUS METHODS
WT AVG MAX (M1) NSCT AVG MAX (M2)
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
14/29
WT-AVG-MAX (M1)
DatasetPERFORMANCE MEASURES
EN MI STD SF Q0 QXY/F
# 1 5.2127 2.5634 65.3796 5.1216 0.2822 0.2769
# 2 4.8940 2.5240 59.7501 6.6219 0.3052 0.2586
# 3 4.4542 2.3354 66.8901 5.5161 0.3290 0.2592
# 4 5.2507 2.1770 57.0081 7.1815 0.3244 0.2502
# 5 4.8650 2.3399 65.7486 6.5576 0.2964 0.2405
# 6 4.1145 2.6323 62.4138 5.5684 0.2883 0.2669
# 7 4.0108 2.4357 69.7617 6.3112 0.3655 0.2758
# 8 4.7939 2.3288 62.6742 5.8714 0.3781 0.2929
# 9 4.5105 2.4310 61.4288 6.0779 0.2856 0.2395
NSCT-AVG-MAX (M2)
Datase
t
PERFORMANCE MEASURES
EN MI STD SF Q0 QXY/F
# 1 5.1201 3.0783 63.8424 4.8754 0.3525 0.3784
# 2 4.8601 2.7564 57.1828 6.2705 0.3208 0.3460
# 3 4.5035 2.4772 65.1020 5.6107 0.3224 0.3443
# 4 5.2960 2.4050 60.2343 7.1192 0.3894 0.3401
# 5 4.7970 2.5112 63.1778 6.2101 0.3595 0.3328
# 6 4.2301 2.7863 60.4923 5.3852 0.3306 0.3456
# 7 4.0645 2.5961 68.9085 5.8846 0.3389 0.4086
# 8 4.8013 2.5628 60.3235 6.0536 0.3573 0.3930
# 9 4.6494 2.6086 59.0970 6.2537 0.2988 0.3118NSST-AVG-MAX (M3)
Datase
t
PERFORMANCE MEASURES
EN MI STD SF Q0 QXY/F
# 1 5.2057 3.2190 67.5122 5.2468 0.4204 0.4380
# 2 4.8638 2.9961 61.8963 6.5585 0.4097 0.4592# 3 4.6208 2.6345 68.3783 6.0983 0.3997 0.4059
# 4 5.2286 2.7054 63.3878 7.5095 0.4502 0.4963
# 5 4.8337 2.7793 67.7438 6.7447 0.4336 0.4847
# 6 4.2791 3.1108 63.5787 5.5623 0.4872 0.4736
# 7 4.1262 2.8338 71.4323 5.9743 0.5218 0.5163
# 8 4.8554 2.8409 65.3852 6.1661 0.4452 0.5448
# 9 4.6864 2.8091 62.9468 6.5367 0.4295 0.4102
NSCT-MAX-SF (M4)
Datase
t
PERFORMANCE MEASURES
EN MI STD SF Q0 QXY/F
# 1 5.1121 3.1058 86.1278 5.3696 0.4415 0.4299
# 2 4.9348 3.0139 81.3054 6.3475 0.4128 0.4636# 3 4.6876 2.9970 91.6373 5.6933 0.4173 0.4172
# 4 5.2871 2.7727 83.2657 7.2480 0.4278 0.4692
# 5 4.8924 2.9413 89.1206 6.3264 0.4225 0.4723
# 6 4.2841 3.1173 85.3086 5.4073 0.5131 0.4957
# 7 4.1826 3.0357 95.0274 6.0055 0.5681 0.4805
# 8 4.8586 2.8499 84.3038 6.1553 0.4662 0.5490
# 9 4.6937 3.0268 83.9519 6.5796 0.4333 0.4164
NSCT-MAX-MSF-PCNN (M5)Datase
t
PERFORMANCE MEASURES
EN MI STD SF Q0 QXY/F
# 1 5.2727 3.1384 86.2748 5.4210 0.4542 0.4375
# 2 4.9361 3.0489 81.5871 6.6242 0.4346 0.5088
# 3 4.6935 3.0389 91.6671 5.8879 0.4354 0.4234
# 4 5.3184 2.8223 83.3683 7.3292 0.4365 0.4866
# 5 4.9108 2.9532 89.4888 6.6292 0.4303 0.4729
# 6 4.3675 3.1358 85.6309 5.5829 0.5304 0.4975
# 7 4.1897 3.0390 95.4015 6.2845 0.5702 0.4825
# 8 4.8604 2.8607 84.3626 6.2186 0.4722 0.5772
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
15/29
NSST-MAX-SF-PCNN (M6)
Datase
t
PERFORMANCE MEASURES
EN MI STD SF Q0 QXY/F
# 1 5.2989 3.2226 86.6951 5.5828 0.4709 0.4494
# 2 4.9914 3.0644 81.7370 6.6959 0.4482 0.5134
# 3 4.7247 3.0310 92.1739 6.1315 0.4396 0.4261
# 4 5.3486 2.8432 83.4918 7.6077 0.4553 0.5122
# 5 4.9528 2.9722 89.6534 6.8515 0.4470 0.4919
# 6 4.4379 3.1570 85.9482 5.7063 0.5482 0.5112
# 7 4.5431 3.0913 95.7893 6.3453 0.5756 0.5231
# 8 4.8970 2.8813 84.6587 6.4386 0.5023 0.5797
# 9 4.7826 3.0502 84.4332 6.8568 0.4518 0.4244
NSST-MAX-MSF-PCNN (M7)
Datase
t
PERFORMANCE MEASURES
EN MI STD SF Q0 QXY/F
# 1 5.3401 3.2462 86.8412 5.6992 0.4764 0.4631
# 2 4.9941 3.0792 81.9542 6.8099 0.4526 0.5164
# 3 4.7301 3.0594 92.3936 6.1480 0.4490 0.4347
# 4 5.3937 2.8469 83.7206 7.6801 0.4597 0.5143
# 5 4.9768 2.9865 89.8980 6.9478 0.4583 0.4964
# 6 4.5661 3.1730 86.1057 5.7910 0.5495 0.5283
# 7 4.6669 3.0978 95.7932 6.3712 0.5825 0.5399
# 8 4.9093 2.8877 84.6746 6.5236 0.5058 0.5856
# 9 4.7990 3.0569 84.7801 6.8796 0.4527 0.4295
NSST-RE-NSML-PCNN (M9)
Datase
t
PERFORMANCE MEASURES
EN MI STD SF Q0 QXY/F
# 1 5.3430 3.2956 87.1860 5.7153 0.4839 0.4685
# 25.0335 3.0853 82.0347 6.8577 0.4627 0.5496
# 3 4.8078 3.0724 92.6860 6.2712 0.4833 0.5002
# 4 5.4243 2.8509 83.8141 8.8396 0.4649 0.5255
# 5 4.9927 2.9886 89.9084 7.7340 0.4602 0.5188
# 6 4.6794 3.2026 86.1525 5.8769 0.5592 0.5514
# 7 4.7696 3.1514 96.1604 6.4666 0.5857 0.5970
# 8 4.9426 2.8933 85.0326 6.5816 0.5153 0.5930
# 9 4.9018 3.0973 84.8521 6.9213 0.4557 0.4969
NSCT-RE-NSML-PCNN (M8)
Datase
t
PERFORMANCE MEASURES
EN MI STD SF Q0 QXY/F
# 1 5.2839
3.1822
86.5459
5.5582
0.4616
0.4480
# 2 4.9362
3.0604
81.5441
6.3185
0.4401
0.5098
# 3 4.7231
3.0454
92.1624
5.8938
0.4387
0.4236
# 4 5.3255
2.8377
83.3934
7.4228
0.4423
0.4962
# 5 4.9413
2.9641
89.6431
6.6472
0.4388
0.4843
# 6 4.4298
3.1544
85.6424
5.6298
0.5422
0.5008
# 7 4.2629
3.0902
95.4214
6.2919
0.5746
0.5211
# 8 4.8878
2.8624
84.6310
6.3458
0.4855
0.5784
# 9 4.7392
3.0475
84.3938
6.7025
0.4511
0.4211
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
16/29
Comparative analysis of the averaged performance measures obtained by the
fusion methods
Fusion Methods EN MI STD SF Q0 QXY/F
WT-AVG-MAX 4.6785 0.44012.4186 0.140463.4506 3.89916.0920 0.64510.3172 0.03510.2623 0.0177
NSCT-AVG-MAX 4.7024 0.39412.6424 0.204462.0401 3.56995.9626 0.63560.3411 0.02680.3556 0.0311
NSST-AVG-MAX 4.7444 0.37072.8810 0.190865.8068 3.15016.2664 0.67020.4441 0.03870.4699 0.0468
NSCT-MAX-SF-PCNN 4.7703 0.35852.9845 0.113386.6721 4.41636.1258 0.59640.4558 0.05240.4660 0.0421
NSCT-MAX-NMSF-PCNN 4.8081 0.37133.0086 0.110286.9047 4.43416.2976 0.60170.4679 0.04940.4785 0.0490
NSST-MAX-SF-PCNN 4.8863 0.30733.0348 0.121887.1756 4.52856.4685 0.62670.4821 0.04930.4924 0.0508
NSST-MAX-NMSF-PCNN 4.9307 0.2848 3.0482 0.126787.3512 4.48716.5389 0.62300.4874 0.04860.5009 0.0510
NSCT-RE-NSML-PCNN 4.8366 0.34993.0271 0.118987.0419 4.49686.3123 0.58190.4750 0.05030.4870 0.0503
NSST-RE-NSML-PCNN 4.9883 0.25083.0708 0.142387.5363 4.55736.8071 0.96930.4968 0.04700.5334 0.0435
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
17/29
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
18/29
CONCLUSIONS
Several image fusion techniques using different transformation methods are
discussed.
DWT capable to preserve the spectral information but unable to express spatial
characteristics efficiently.
NSCT improve the performance and overcome the limitations of the wavelet .
Pixel-level spatial domain fusion such as averaging usually lead to contrast
reduction while improve the overall frequency of pixels.
PCNN gives good decision output as it biologically inspired by human visual system
hence better contrast.
Spatial and modified spatial frequency is become good focus indicator as total
activity and clarity level measure.
NSST improves the shortcoming of NSCT and gives better directionality hence
ensures good image quality.
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
19/29
CONCLUSIONS
Three fusion schemes using NSST are presented and compared with another rules
based on same fusion combination but using WT and NSCT.
WT based on AVG-MAX gives high entropy and spectral information but lacks of
contrast and localization.
NSCT improve the performance with high degree of localization and directionality
but due to AVG it degraded with low standard deviation.
NSST outperforms the two methods and gives better evaluation measures.
Compared with other approaches PCNN-based fusion are of higher clarity and
contrast.
The sum-modified-Laplacian (SML) can well reflects the feature information of
the edges of the image.
The energy of image reflects the brightness level of scene.
The fusion scheme based on NSST using regional energy and NSML motivated
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
20/29
REFERENCES
References.docx
http://localhost/var/www/apps/conversion/tmp/scratch_2/References.docxhttp://localhost/var/www/apps/conversion/tmp/scratch_2/References.docx8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
21/29
Activity Level Measures
S.No.
Activity Measures Mathematical Expression
1 Pixel Averaging (AVG) (, ) ( , (, ))
2 Maximum Coefficientselection (MAX) , (, ), (, ) (, )(, ), otherwise
3 Regional Energy (RE)
, , (,)==
where (,)is weight template of size 33
(,) 1
9
1 1 11 1 11 1 1
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
22/29
Activity Level Measures
S.No. Activity Measures Mathematical Expression
4 Novel sum-modified-laplacian (NSML)
, 2 , 1, 1, 2 , , 1 , 1
, , . , where (,)is weight template of size 33 ,
1/15 2/15 1/152/15 3/15 2/151/15 2/15 1/15
5 Modified SpatialFrequency(MSF)
1(1)(1) , 1, 1
=
=
1(1)(1) 1, , 1 ==
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
23/29
Wavelet Transform (WT)
Wavelet transform for image fusion
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
24/29
Nonsubsampled Contourlet Transform(NSCT)
Block diagram Nonsubsampled contourlet transform based imagedecomposition
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
25/29
Nonsubsampled Contourlet Transform(NSCT)
Block diagram Nonsubsampled contourlet transform based imagedecomposition
Nonsubsampled Shearlet T f
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
26/29
Nonsubsampled Shearlet Transform(NSST)
Block diagram Nonsubsampled shearlet transform based imagedecomposition
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
27/29
Pulse Coupled Neural Network (PCNN)
PCNN Neuron Model
28
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
28/29
Mathematical equations supports PCNN neuron model:
28
, , , 1 ,,,,[ 1 ],
,
,
1
,,,
,
[ 1 ],
, , 1 , , , 1 ,[]
, 1, , > ,0,
8/11/2019 Transform Based Fusion Algorithm for Medical Images_updated
29/29