Gable Roof Description by Self-Avoiding Polygon *Qiongchen Wang, Zhiguo Jiang, Junli Yang, Danpei Zhao, and Zhenwei Shi. ACCV 2009

Outline

•Introduction•Model Representation and Probabilistic

Distribution▫SAP Model▫Probabilistic Distribution▫Learning Local Appearance Model

•Model Inference Algrorithm•Experiment•Discussion and Conclusion

Introduction(1/2)

• roof delineation from aerial imagery in important to computer vision, remote sensing and cartography.

•So work use fusing multiple information (DEM) or using 3D information.

•Always Using more than one nadir-view aerial image.

•Two reason: Shape variety and Local ambiguity.

Introduction(2/2)

•This paper address the problem of shape variety by proposing the self-avoiding polygon (SAP)

•To avoid local ambiguity:▫Learning based local structure detectors.▫Searching the entire state space of all possible SAPs.

• A search algorithm is using.∗

Related Work

•Two major categories:•1.parameterized models :

▫Described buildings that by prototype shapes.▫Di cult to combine the large number of parameters.ffi

•2.component based models:▫Great flexibility and expressive power in modeling

various building types.▫The major challenge is local ambiguity.

Gable Roof

Self-Avoiding Polygon(1/3)

•The state space of gable corners C on image lattice Λ with domain D is defined as a dictionary:

ΩC = { C (x,y,s,θ,t), (x,y,s,∀ θ,t) } (1)• (x,y) D is the center position of C∈• s { k,k =1,...,K } ∈ denotes scale • θ { 2π(w/16),w =0,...,15 } ∈ denotes orientation• t { 1,2 } ∈ represents two types of gable roof corners

P = n,(C 1,...,C i,...,C n,C n+1 ) , (2)• n [minN,maxN] ∈ means the number of corners in P

C 1 = C n+1 , (3)

Self-Avoiding Polygon(2/3)

C i C ⇒ j , j =i +1 , i =1,...,n. (3)• Ci =C (xi,yi,si,θi,ti ) , then Ci C⇒ j if and only if there exists (θi,θj

) such that:

si = sj ;θj = θi+π or θi−π/2, if ti=1, tj=1,θj = θi+π, if ti=1, tj=2,θj = θi+π or θi−π/2, if ti=2, tj=1,θj = θi+π, if ti=2, tj=2. (4)

Self-Avoiding Polygon(3/3)

•Given a P, we can generate a shape template of gable roof by connecting all open bonds of each node into a loop.

Probabilistic Distribution(1/6)

•Z is the partition function.•E(P) represents a set of constraints serving as shape

priors.▫Constraints specified in previous equation.▫Penalize when point (xi+1,yi+1 ) is not close to the line

crossing point (xi,yi ) toward direction θi.▫Penalize when (xi,yi ),(xi+1,yi+1 ) are too close.

Probabilistic Distribution(2/6)

•E(P,I) measures how good a gable rooftop generated from P fits the image I.

Probabilistic Distribution(3/6)

• s ∈ Λ , x ∈ Λ is a pixel.• I(x) is the intensity value of x.• N(x)/x is all pixels in the neighborhood except x.• R Λ is the image domain occupied by P.⊂• Λ/R means on the image lattice Λ except R.• y(x)=1 means x is on the rooftop, y(x)=0 means x is on the background.

ProbabilisticDistribution(4/6)

Probabilistic Distribution(5/6)•First right-hand-side term can be ignored.•Divide R into three sets: •corners(C), boundaries(B) and interior area (A)

• lc(x)=1 or 0 indicate pixel x is “on” or “o ” the ffcorner of a gable roof. (just like lb(x), la(x) )

log

Probabilistic Distribution(6/6)

•equation(8), (9), (10) are independent to each other.• (8),(9) and (10) represent posterior probability ratios

of a pixel x belonging to correspondence patch.

Learning Local Appearance Model(1/2)

•Cornor : adopt the active basis model [7]

• B = (Bi, i =1,...,n) is a template composed of a set of Gabor wavelets Bi .• rj is convolution response of the jth Gabor wavelets with image I • λj is the jth coe cients.ffi• zi is normalizing constant.• N = 40 Gabor wavelets.

[7] Wu, Y.N., Si, Z.Z., Fleming, C., Zhu, S.C.: Deformable template as active basis. In: ICCV, pp. 1–8 (2007)

Learning Local Appearance Model(2/2)

•Bound and Area:•Learned by training discriminative

classifiers based on local cues within the neighborhood.

•Similar to the approach in [6].•Use a N(s) =30 × 30 neighborhood to learn

an approximated posterior probability.•Computed at a given scale•M=45 of instances for each type.

Dollar, P., Tu, Z.W., Belongie, S.J.: Supervised learning of edges and object boundaries. In:CVPR (2006)

Model Inference Algrorithm

•Find a P∗ that maximize the posteriori probability given I.

•Using A* search algorithm.

Data Structures In A* Search Algorithm

• A Saliency Map Array• A Priority Queue• A Closed Set• A Neighborhood Tree

A* Search Algorithm

•came_from(X)▫stores the previous node of X in the path

•cost_so_far(X,Y) = path cost + path length

▫The smallest cost from the initial node to current node.•cost_to_goal(X,G) = path length

▫Heuristic distance from current node to goal.• total_cost = cost_so_far + cost_so_far

An A* Example

Experiment(1/2)

•Evaluate SAP model on a number of challenging gable roofs.

•Use a preprocess to determine the scale.•Compute saliency maps of roof corners at 10

continous scales. The scale with maximun score is selected as the global scale.

Experiment(2/2)

•Demonstrate some final results by connecting the searched corners using straight lines.

•The whole search space is very huge, but the A ∗algorithm only search a few branches with lowest cost in the Neighborhood Tree.

•Computing time grows linearly (not exponentially) with search depth.

Discussion and Conclusion

•SAP model and an efficient A∗ search algorithm for complex gable roof representation and inference.

Future Work

•Focus on developing more efficient addmisible heuristics to guide the search to convergy to the goal faster.

•To generalize the SAP model to more types of roofs.

•Add more shape prior to constrain SAP model in a smaller space.