+ All Categories
Home > Documents > ImageNet Classification Using Binary Convolutional Neural...

ImageNet Classification Using Binary Convolutional Neural...

Date post: 16-Aug-2020
Category:

Documents
Upload: others
View: 5 times
Download: 0 times
Download Report this document
Share this document with a friend
13
XNOR-Net ImageNet Classification Using Binary Convolutional Neural Networks Mohammad Rastegari Vicente Ordonez Joseph Redmon Ali Farhadi Presentation by Naveen
Transcript
Page 1: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks

XNOR-NetImageNet Classification Using Binary

Convolutional Neural Networks

Mohammad Rastegari Vicente OrdonezJoseph Redmon

Ali Farhadi

Presentation by Naveen

Page 2: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks

Deep Neural Networks are Complicated(And Huge!)

Remember HW2 - Size of AlexNet?

Page 3: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks

CPU vs GPU

SmallWeak

Scrawny

BigStrong

Powerful

Page 4: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks

Possible ApproachesShallow Approximation Compression

Page 5: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks

Binary-Weights Network

Basic Idea: Too much information in each convolutional layer. Can we store less?

Page 6: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks

I * W ~ (I W) a~

I = Input TensorW = Weightsa = scaling factor

Binary-Weights Network

Page 7: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks

Training

Binarize weights in forward pass and backward

propagation

Use real valued weights in gradient descent (Why?)

Also, if we are using real valued weights somewhere, what’s the

point?!

Page 8: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks

XNOR-Net

Page 9: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks

Training

BinActivComputes the K and sign(I)

BinConvPerform earlier Binary

Convolution

Page 10: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks

Experiments

Efficiency

58x CPU Speedups

Page 11: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks

ExperimentsAccuracy

Cifar-10

Binary-Weight Network: 9.88% Error XNOR-Net: 10.17% Error

Page 12: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks

ExperimentsAccuracy

Page 13: ImageNet Classification Using Binary Convolutional Neural ...vicente/recognition/2016/presentations/xnornet.pdfImageNet Classification Using Binary Convolutional Neural Networks

Questions?


Recommended
Traffic sign classification with deep convolutional neural ...publications.lib.chalmers.se/records/fulltext/238914/238914.pdf · Traffic sign classification with deep convolutional

Traffic sign classification with deep convolutional neural ...publications.lib.chalmers.se/records/fulltext/238914/238914.pdf · Traffic sign classification with deep convolutional

Documents

Human Pose Estimation and Activity Classification …cs231n.stanford.edu/reports/2015/pdfs/cdong-paper.pdfHuman Pose Estimation and Activity Classification Using Convolutional Neural

Human Pose Estimation and Activity Classification …cs231n.stanford.edu/reports/2015/pdfs/cdong-paper.pdfHuman Pose Estimation and Activity Classification Using Convolutional Neural

Documents

Circulant Binary Convolutional Networks: Enhancing …openaccess.thecvf.com/content_CVPR_2019/papers/Liu...Circulant Binary Convolutional Networks: Enhancing the Performance of 1-bit

Circulant Binary Convolutional Networks: Enhancing …openaccess.thecvf.com/content_CVPR_2019/papers/Liu...Circulant Binary Convolutional Networks: Enhancing the Performance of 1-bit

Documents

TEX-Nets: Binary Patterns Encoded Convolutional Neural ...€¦ · TEX-Nets: Binary Pa‡erns Encoded Convolutional Neural Networks for Texture RecognitionICMR ’17, , June 6–9,

TEX-Nets: Binary Patterns Encoded Convolutional Neural ...€¦ · TEX-Nets: Binary Pa‡erns Encoded Convolutional Neural Networks for Texture RecognitionICMR ’17, , June 6–9,

Documents

Complex Network Classification with Convolutional Neural Network · 2019. 4. 25. · Complex Network Classification with Convolutional Neural Network Ruyue Xin1, Jiang Zhang1,*,

Complex Network Classification with Convolutional Neural Network · 2019. 4. 25. · Complex Network Classification with Convolutional Neural Network Ruyue Xin1, Jiang Zhang1,*,

Documents

ImageNet Classification with Deep Convolutional Neural ...usman/courses/cs675_summer20/...ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky University

ImageNet Classification with Deep Convolutional Neural ...usman/courses/cs675_summer20/...ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky University

Documents

On Binary Classification with Single–Layer Convolutional Neural … · 2015-09-15 · On Binary Classification with Single–Layer Convolutional Neural Networks Soroush Mehri

On Binary Classification with Single–Layer Convolutional Neural … · 2015-09-15 · On Binary Classification with Single–Layer Convolutional Neural Networks Soroush Mehri

Documents

3D model classification using convolutional neural networkcs229.stanford.edu/proj2015/146_report.pdf · 2017-09-23 · 3D model classification using convolutional neural network

3D model classification using convolutional neural networkcs229.stanford.edu/proj2015/146_report.pdf · 2017-09-23 · 3D model classification using convolutional neural network

Documents

Local Binary Convolutional Neural Networks · 2017. 5. 31. · Local Binary Convolutional Neural Networks Felix Juefei-Xu Carnegie Mellon University felixu@cmu.edu Vishnu Naresh Boddeti

Local Binary Convolutional Neural Networks · 2017. 5. 31. · Local Binary Convolutional Neural Networks Felix Juefei-Xu Carnegie Mellon University [email protected] Vishnu Naresh Boddeti

Documents

A Convolutional Transformation Network for Malware ...convert binary executables into color images that convey the semantics of the binary data. We develop a Convolutional Transformation

A Convolutional Transformation Network for Malware ...convert binary executables into color images that convey the semantics of the binary data. We develop a Convolutional Transformation

Documents

Food Image Classification with Convolutional Neural Networks

Food Image Classification with Convolutional Neural Networks

Documents

Scene classification with Convolutional Neural Networkscs231n.stanford.edu/reports/2017/pdfs/102.pdf · Scene classification with Convolutional Neural Networks Josh King jking9@stanford.edu

Scene classification with Convolutional Neural Networkscs231n.stanford.edu/reports/2017/pdfs/102.pdf · Scene classification with Convolutional Neural Networks Josh King [email protected]

Documents

Behavioral Malware Classification using …spiros/papers/malcon2018a.pdfBehavioral Malware Classification using Convolutional Recurrent Neural Networks Bander Alsulami Drexel University

Behavioral Malware Classification using …spiros/papers/malcon2018a.pdfBehavioral Malware Classification using Convolutional Recurrent Neural Networks Bander Alsulami Drexel University

Documents

Convolutional Neural Networks for Font Classification

Convolutional Neural Networks for Font Classification

Documents

Binary Convolutional Codes with Application to Magnetic - NativeI

Binary Convolutional Codes with Application to Magnetic - NativeI

Documents

Convolutional Neural Networks for Fashion Classification ...cs231n.stanford.edu/reports/2015/pdfs/BLAO_KJAG_CS...Convolutional Neural Networks for Fashion Classification and Object

Convolutional Neural Networks for Fashion Classification ...cs231n.stanford.edu/reports/2015/pdfs/BLAO_KJAG_CS...Convolutional Neural Networks for Fashion Classification and Object

Documents

ImageNet Classification with Deep Convolutional Neural …hinton/absps/imagenet.pdf · ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky ... and 17.0%

ImageNet Classification with Deep Convolutional Neural …hinton/absps/imagenet.pdf · ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky ... and 17.0%

Documents

DeepSentiBank: Visual Sentiment Concept Classification with ... · DeepSentiBank: Visual Sentiment Concept Classification with Deep Convolutional Neural Networks Tao Chen1, Damian

DeepSentiBank: Visual Sentiment Concept Classification with ... · DeepSentiBank: Visual Sentiment Concept Classification with Deep Convolutional Neural Networks Tao Chen1, Damian

Documents