U-Net Fixed Point Quantization For MedicalImage Segmentation
MohammadHossein AskariHemmatSina Honari
Lucas RouhierChristian S. PeroneJulien Cohen-Adad
Yvon SavariaJean-Pierre David
MICCAI2019, October 17th
Presentation outline:
1 Medical Image Segmentation With Deep Neural Networks(DNNs)
2 Quantization in Deep Neural Networks
3 U-Net Fixed-Point Quantization for Medical ImageSegmentation
4 Results
5 Conclusion
U-Net Fixed Point Quantization For Medical Image Segmentation 2
3/24
Medical Image Segmentation With Deep Neural Networks (DNNs)
• Medical Image Segmentation are performed on MRI imagesto help doctors diagnose diseases.
Figure: Segmentation of a brain tumor in an MRI image [M.Havaie et al 2016].
U-Net Fixed Point Quantization For Medical Image Segmentation 3
4/24
Medical Image Segmentation With Deep Neural Networks (DNNs)
• Medical health centers and hospitals are equipped withpre-trained models used in medical CADs to analyse MRIimages.
• However, to perform these tasks with DNNs, a lot ofcomputation needs to be done.
• But how many operations are really needed?
U-Net Fixed Point Quantization For Medical Image Segmentation 4
5/24
Computation Cost in Deep Neural Networks (DNNs)
Training Computation Cost :Finishing a 90-epoch ImageNet-1k training with ResNet-50 on a NVIDIAM40 GPU takes 14 days. This training requires 1018 single precisionoperations in total [Y. You et al ”ImageNet Training in Minutes”].
Inference Computation Cost :Finishing a full pass of Imagenet with input size of 224x224 with batch sizeof 128 reuqires 13 GB feature memory and 497 GFLOPs [S. AlbanieGitHub:convnet-burden].
U-Net Fixed Point Quantization For Medical Image Segmentation 5
6/24
Cost and energy consumption source in DNNs:
• Number of parameters in the network (Data Movement).• Per operation computation cost.
Figure: Deep compression [H. Song, et al., 2015]
U-Net Fixed Point Quantization For Medical Image Segmentation 6
7/24
Quantization For Accelerating Computation in DNNs :
One way to accelerate computation and save energy in a DNN isto use less precision for computation:
• Less number of bits are required to be stored in memory,which reduces data movement.
• Floating point calculation is costly and slow. Less energyand computation will be spent per operation.
U-Net Fixed Point Quantization For Medical Image Segmentation 7
8/24
Quantization For Accelerating Computation in DNNs :
What is Quantization in DNN?Quantization is a technique to reduce memory consumption and thecomputation time of deep neural networks by lowering the precision ofparameters.
U-Net Fixed Point Quantization For Medical Image Segmentation 8
9/24
Quantization For Accelerating Computation in DNNs :
• Quantization not only uses less memory but it is moreenergy efficient:
Figure: Energy consumption of multiplication and accumulationin a 45nm process (Horowitz, 2014)
• In Intel Core i7 4770 3.40GHz, 32-bit multiplication is morethan 3 times faster for fixed point data compared tofloating point data. [https://goo.gl/7Y7GWt].
U-Net Fixed Point Quantization For Medical Image Segmentation 9
10/24
Quantization For Accelerating Computation in DNNs :
Does it Work?
U-Net Fixed Point Quantization For Medical Image Segmentation 10
11/24
Quantization For Accelerating Computation in DNNs :
• In [M. Courbariaux et al. ”BinaryConnect”] it has beenshown that for small models (MNIST size) using even 1-bitfor weights results in test accuracy drop of only 1%.
• For bigger models (like ImageNet) using 8-bit integersinstead of 32-bit floating point shows state-of-the-artperformance.
• Pytorch just added quantization option!
U-Net Fixed Point Quantization For Medical Image Segmentation 11
12/24
Existing works for quantization of medical images:
• FCN Quantization [Xu, X., et al. CVPR 2018]: Appliedquantization on Fully Convolutional Networks forbiomedical application.
• TernaryNet [Heinrich, M.P et al, IJCARS 2017]: Firstquantization results for U-Net using ternary net.
U-Net Fixed Point Quantization For Medical Image Segmentation 12
13/24
Our research objective:
Quantization for Medical Image Segmentation:In this work, we wanted to know does quantization works for life threateningtasks such as medical imaging?
• We wanted to use a well known model that is widely usedfor medical imaging.
• We wanted to have only fixed point operations so that itcan be used with an integer processors.
U-Net Fixed Point Quantization For Medical Image Segmentation 13
14/24
U-Net Fixed-Point Quantization for Medical ImageSegmentation
• In our work, we transformed all floating point computationto fixed-point computation.
xf = abs(x) − floor(abs(x)), xi = floor(abs(x)) (1)
quantize(x , n) = (round(clamp(x , n) << n)) >> n (2)
clamp(x , n) =
2n − 1 when x ≥ 2n − 1
x when 0 < x < 2n − 1
0 when x ≤ 0
(3)
• Qp i .f : fixed point quantization of parameter p by using ibits to represent the integer part and f bits to represent thefractional part.U-Net Fixed Point Quantization For Medical Image Segmentation 14
15/24
Challenges for Training:
• Dropout and Quantization: we found that when Dropout isapplied along with quantization, the accuracy drops a lot:
U-Net Fixed Point Quantization For Medical Image Segmentation 15
16/24
Challenges for Training:
• Keeping the last layer in full precision has much moreimpact than keeping the first layer in full precision.
• At inference, we used Pytorch batch-norm folding.Effectively including batch-norm parameters in thequantized model as part of the quantized weights.
U-Net Fixed Point Quantization For Medical Image Segmentation 16
17/24
Results For Quantization of U-Net Model for Medical Image:• We used three different datasets:
NIH
Pan
crea
sG
M S
pina
l Cor
dE
M D
roso
phila
Bra
in
Figure: Segments in show false positive, segments in show false negative and segments inshow true positive.
U-Net Fixed Point Quantization For Medical Image Segmentation 17
18/24
Results For Quantization of U-Net Model for Medical Image:
• Qa6.0,Qw0.4 compared to other methods:
Figure: shows best score overall and shows best score between three quantiation methods.
U-Net Fixed Point Quantization For Medical Image Segmentation 18
19/24
Results and Discussion:
• We found out that instead of using 32-bit floating pointvalues, we can use 4 bits for weights and 6 bits foractivations.
U-Net Fixed Point Quantization For Medical Image Segmentation 19
20/24
Results and Discussion:
• Fully fixed point data path:
CONV +ak-1
wt
bt
ak
Quant
Quant
Fixed Point Fixed Point
Fixed Point Fixed PointFixed Point
CONV +ak-1
wt
bt
ak
Ternary
Ternary
Ternary Float
Ternary Float
Ternary
Tern
aryN
etO
ur M
etho
d
U-Net Fixed Point Quantization For Medical Image Segmentation 20
21/24
Results and Discussion
• Relu Vs Tanh performance:
Figure: Comparing ReLU and Tanh run time using Intel’sOpenVino [Deanne. D rt. al Release notes for intel R© 2019]
U-Net Fixed Point Quantization For Medical Image Segmentation 21
22/24
Our Code is on GitHub:https://github.com/hossein1387/U-Net-Fixed-Point-Quantization-for-Medical-Image-Segmentation.
U-Net Fixed Point Quantization For Medical Image Segmentation 22
23/24
Our Contribution in this work:
• We report the first fixed point quantization results on theU-Net architecture for the medical image segmentation taskand show that the current quantization methods availablefor U-Net are not efficient for the hardware commonlyavailable in the industry.
• We quantify the impact of fixed point quantization on theperformance of the U-Net model using three differentmedical imaging datasets.
• We report results comparable to a full precisionsegmentation model by using only 6 bits for activation and4 bits for weights, effectively reducing the weights size by afactor of 8x and the activation size by a factor of 5x.
U-Net Fixed Point Quantization For Medical Image Segmentation 23
24/24
Thank you for your attention!
U-Net Fixed Point Quantization For Medical Image Segmentation 24