BayesianSegNet:ModelUncertaintyinDeepConvolutionalEncoder-DecoderArchitecturesforSceneUnderstandingAlexKendall,VijayBadrinarayananandRobertoCipolla

http://mi.eng.cam.ac.uk/projects/segnet/

References1. VijayBadrinarayanan,AlexKendall,RobertoCipolla.SegNet:ADeepConvolutionalEncoder-DecoderArchitectureforImageSegmentation. PAMI,2017.2. AlexKendall,VijayBadrinarayananandRobertoCipolla.BayesianSegNet:ModelUncertaintyinDeepConvolutionalEncoder-DecoderArchitecturesfor

SceneUnderstanding.BMVC,2017.3. AlexKendallandYarinGal.WhatUncertaintiesDoWeNeedinBayesianDeepLearningforComputerVision?arXivpreprintarXiv:1703.04977,2017.4. AlexKendall,YarinGalandRobertoCipolla.Multi-TaskLearningUsingUncertaintytoWeighLossesforSceneGeometryandSemantics.arXivpreprint

arXiv:1705.07115,2017.

Insights• Wecanobtainper-classmodeluncertaintyestimatesforsceneunderstandingmodels• Bayesianinferencemoreimportantinlateencoderandearlydecoderlayers• Improvessegmentationperformanceby2-3% acrosspopularmodels• MCdropoutoutperformsweightaveragingafter6samplesandconvergesafter40samples• Especiallyeffectiveforsmalldatasets(e.g.CamVid)• Modeluncertaintyincreasesforrareanddifficultclasses• Modeluncertaintyisusefulforsafeautonomousdecisionmaking,activelearningandlabelpropagation

FurtherApplications• DistinguishAleatoric (sensor)

uncertaintyandEpistemic (model)uncertainty[3]

• Useuncertaintytoimprovemulti-tasklearning[4]

• Semanticsegmentation,instancesegmentationanddepth regressionfromasingleinputimage[4]

Model Standard Bayesian

DilationNet 71.3% 73.1%

FCN 62.2% 65.4%

SegNet 59.1% 60.5%

Convolutional Encoder-DecoderInputSegmentation

Model Uncertainty

Stochastic DropoutSamples

Conv + Batch Normalisation + ReLUDropout Pooling/Upsampling Softmax

mean

variance

RGB Image

WeuseMonteCarlodropoutsamplingattesttimetogenerateaposteriordistributionofpixelclasslabels.

BayesianSegNetarchitecturePASCALVOC2012

TestServerPerformance

InputImageGroundTruthModelSegmentationAleatoricUncertaintyEpistemicUncertainty