Post on 05-Apr-2018
transcript
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 1/47
!""#$%&'()*%+$),'
-.,")/)0%1/$2+'34'5$%/)/26$2)#'7.&%#+'
'!"#$%&'()'#)*+,-)./0.)
Erik SudderthBrown University
Work by E. Sudderth, A. Torralba, W. Freeman, & A. Willsky
IJCV 2008:Describing Visual Scenes using Transformed Objects & Parts
CVPR 2006:Depth from Familiar Objects: A Hierarchical Model for 3D Scenes
NIPS 2005:Describing Visual Scenes using Transformed Dirichlet Processes
Building on work by Y. W. Teh, M. Jordan, M. Beal, & D. Blei
JASA 2006:Hierarchical Dirichlet Processes
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 2/47
8%)/,$,9':$16';."$2'7.&%#+
model
neural
stochastic
recognition
nonparametric
gradient
dynamical
Bayesian
!
Framework for unsupervised discovery oflow-dimensional latent structure frombag of word representations
Algorithms
Neuroscience
Statistics
Vision
!
!! pLSA: Probabilistic Latent Semantic Analysis(Hofmann 2001)
!! LDA: Latent Dirichlet Allocation(Blei, Ng, & Jordan 2003)
!! HDP: Hierarchical Dirichlet Processes(Teh, Jordan, Beal, & Blei 2006)
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 3/47
5$%/)/26$2)#'<$/$26#%1'=/.2%++'
E
[πj ] =
β
G0 ∼ DP(γ , H )
Gj ∼ DP(α, G0)
J groups of data:documents, images,!
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 4/47
>6$,%+%'?%+1)@/),1'A/),26$+%'
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 5/47
8.2)#'B$+@)#'A%)1@/%+C'D@"%/"$E%#+
•!Partition image into ~1,000superpixels
•!Goal: Reduce dimensionality, aggregateinformation spatially –hopefully not
across object boundaries!
Inspired by the successes oftopic modelsfor text data,
some have proposed learning fromlocal image features
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 6/47
8.2)#'B$+@)#'A%)1@/%+C'F,1%/%+1'?%9$.,+'
Affinely AdaptedHarris Corners
Maximally StableExtremal Regions
Linked Sequencesof Canny Edges
•!Some invariance to lighting & pose variations•!Dense, multiscaleover-segmentationof image
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 7/47
!'<$+2/%1%'A%)1@/%'B.2)G@#)/*'
•! !"#$%&'()*+,'-."/#%$-+"0+
"#')1.%2"1+)1)#/3+
•! 4"$56.)+789:::+;"#*+
*'<2"1%#3+='%+>?$)%1-+
•! @%5+)%<,+0)%.6#)+."+
1)%#)-.+!"#$%&'()*+'
appearance of
featureiin image j
2D position of
featureiin image j
SIFT Descriptors
Lowe, IJCV 2004
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 8/47
;6%'H./#&')+')'()9'.I'B$+@)#'H./&+'
Fei-Fei & Perona,CVPR 2005 Sivic, Russell, Efros, Zisserman, & Freeman,ICCV 2005
Topics asvisual themescomposing a
known set of scene categories
Topics asvisual object classeswithin a
(carefully chosen) image collection
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 9/47
F0)9%+')+'0./%'16),'()9+'.I'A%)1@/%+'
•! A";+*"+B+C1";+.,'-+'-+"<)%1+D)1)%.,+%+<&)%#+-C3E+
•! A";+$%13+D'<3<&)-+%1*+.#'<3<&)-+%$+B+&""C'1/+%.E+
,-.'%*/'(/'0*."12'0)'#3$//4/'"5%2/#'"10)'0)6"7'5)+/'
9-/*/'%*/'5%1.'5)*/'0))&#'%!%"&%:&/':.'%+%6;12'
1)16%*%5/0*"7'%1+'-"/*%*7-"7%&'<%./#"%1'5)+/&#='
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 10/47
B$+@)#'JGK%21'>)1%9./$L)M.,'
•! GOAL:Visuallyrecognize andlocalize object categories
•!Robustlylearn appearance models from few examples
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 11/47
Part-Based Models for Objects
Pictorial Structures Fischler & Elschlager, 1973
Generalized Cylinders Marr & Nishihara, 1978
Recognition by Components Biederman, 1987
Constellation Model Perona, Weber, Welling,
Fergus, Fei-Fei, 2000 to!
Efficient MatchingFelzenszwalb & Huttenlocher, 2005
Discriminative PartsFelzenszwalb, McAllester,
Ramanan, 2008 to!
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 12/47
>.@,M,9'JGK%21+'N'=)/1+'
How many parts? How many objects?
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 13/47
O%,%/)MP%'7.&%#'I./'JGK%21+'
For each image:Sample a reference position
For each feature:
!!Randomly choose one part!!Sample from that part’s feature distribution
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 14/47
JGK%21+')+'<$+1/$G@M.,+'
•!Parts are defined by parameters, whichencode distributions on visual features:
Featureappearance
Featureposition
Pr(appearance | part) Pr(position | part)
•!Objects are defined bydistributionson the
infinitely many potential part parameters:
Pr(part)
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 15/47
!'-.,")/)0%1/$2'=)/1Q()+%&'7.&%#'
4 Images 16 Images 64 Images
# images
# p
a r t s
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 16/47
O%,%/)#$L$,9'!2/.++'>)1%9./$%+'
Can we transfer knowledge from one object category to another?
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 17/47
8%)/,$,9'D6)/%&'=)/1+'
•! FDG)<.-+%#)+"H)1+&"<%&&3+-'$'&%#+'1+%55)%#%1<)+
•! I'-<"=)#+ 6%*0#+-,%#)*+%<#"--+<%.)/"#')-+ –A";+$%13+.".%&+5%#.-+-,"6&*+;)+-,%#)E+
–A";+$%13+5%#.-+-,"6&*+)%<,+<%.)/"#3+6-)E+
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 18/47
5$%/)/26$2)#'<='JGK%21'7.&%#'
G0
J
G!"
#"
H
R
Nw
$"
%"
&"
v
L
G0
1G G2
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 19/47
5$%/)/26$2)#'<='JGK%21'7.&%#'
G0
J
G!"
#"
H
R
Nw
$"
%"
&"
v
L
G0
1G G2
Discrete Data:
Teh et. al., 2004
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 20/47
>6$,%+%'?%+1)@/),1'A/),26$+%'
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 21/47
D6)/$,9'=)/1+C'RS'>)1%9./$%+'
•!Caltech 101 Dataset (Li & Perona)•!Horses (Borenstein & Ullman)•!Cat & dog faces (Vidal-Naquet & Ullman)
•!Bikes from Graz-02 (Opelt & Pinz)•!Google!
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 22/47
B$+@)#$L)M.,'.I'D6)/%&'=)/1+'
Pr(appearance | part)
Pr(position | part)
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 23/47
B$+@)#$L)M.,'.I'D6)/%&'=)/1+'
Pr(appearance | part)
Pr(position | part)
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 24/47
B$+@)#$L)M.,'.I'D6)/%&'=)/1+'
Pr(appearance | part)
Pr(position | part)
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 25/47
B$+@)#$L)M.,'.I'=)/1'<%,+$M%+'
Hierarchical Clustering of Pr(part | object)
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 26/47
<%1%2M.,';)+T'
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 27/47
<%1%2M.,'?%+@#1+'
6 Training Images per Category(ROC Curves)
Shared Parts more accurate than
Unshared Parts
Modeling feature positions
improves shared detection, but
hurts unshareddetection
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 28/47
<%1%2M.,'?%+@#1+'
6 Training Images per Category(ROC Curves)
Detection vs. Training Set Size(Area Under ROC)
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 29/47
D6)/$,9'D$0"#$U%+'7.&%#+'
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 30/47
D2%,%+V'JGK%21+V'),&'=)/1+'
Features
Parts
Objects
Scene
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 31/47
>.,1%E1@)#';/),+I%/'8%)/,$,9'
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 32/47
JGK%21'P+4'B$+@)#'>)1%9./$%+'
•! Assume training data contains object category labels•!Discover underlying visual categories automatically
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 33/47
7@#M"#%'JGK%21'D2%,%+'
•!How many cars are there?•!Where are those cars in the scene?
Standard dependent Dirichlet process models (Gelfand et. al., 2005) inappropriate
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 34/47
D")M)#';/),+I./0)M.,+'
•!Let global DP clusters model objectsin acanonical coordinate frame
•!Generate images via a randomset of transformations:
Parameterized family
of transformations
Shift cluster from canonical
coordinate frame to object
location in a given image
Layered Motion Models (Darrell & Pentland 1991, Wang & Adelson 1994, Jojic & Frey 2001)
Nonparametric Transformation Densities (Learned-Miller & Viola 2000)
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 35/47
!';.*'H./#&C''()/+'N'(#.G+'
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 36/47
;/),+I./0%&'<$/$26#%1'=/.2%++'
G0G0
1 2 3G G G
J
G j!"
#"
H
R
Mixture
Parameters Transformations
Nv
$" %"
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 37/47
F0"./1),2%'.I';/),+I./0)M.,+'
HDP TDP
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 38/47
>.@,M,9'N'8.2)M,9'JGK%21+''
•!How many cars are there?•!Where are those cars in the scene?
Dirichlet Processes
Transformations
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 39/47
B$+@)#'D2%,%';<=''
G0#"
H
R
GlobalDensityObject category Part size & shape
Transformation prior
'"
F(
J
G j!" TransformedDensitiesObject category Part size & shape
Instance locations
w v
2D ImageFeatures AppearanceLocation
N
$"
%"o
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 40/47
D1/%%1'D2%,%'B$+@)#'>)1%9./$%+'
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 41/47
D1/%%1'D2%,%'D%90%,1)M.,+'
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 42/47
D%90%,1)M.,'=%/I./0),2%'
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 43/47
WE1%,+$.,C'X<'D2%,%+
•!Segmentation easier in 3D•!Identifying known objectsregularizes depth estimation
Red
Green
Office
Scene
Far
Near
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 44/47
X<'D1/@21@/%'I/.0'D1%/%.
Reference (left) Image Potential Matches Depth Densities
Overhead View
Depth =Disparity
)"
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 45/47
O/%%&*'<%"16'W+M0)1%+
Reference (left) Image Potential Matches Depth Densities
Red Far
Green Near
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 46/47
X<';/),+I./0%&'<=C'JY2%'D2%,%+
Computer Screen
DeskBookshelvesBackground
7/31/2019 BNP2hierarchy Cvpr2012 Applied Bayesian Nonparametrics
http://slidepdf.com/reader/full/bnp2hierarchy-cvpr2012-applied-bayesian-nonparametrics 47/47
D1%/%.';%+1'F0)9%
Simultaneousobject recognition & coarse3D reconstruction