Low-Rank Based Algorithms for Rectification, Repetition Detection and De-Noising in Urban Images
A dissertation proposal by
Juan Liu
Committee
Committee:
Professor Ioannis Stamos, Mentor, Hunter College
Professor Yingli Tian, City College
Professor Zhigang Zhu, City College
Outside Member:
Professor Emmanouil Z. Psarakis, University of Patras, Greece
Current Work
2D image rectification
Façade texture selection
Efficient Kronecker Product model
Automatic repeated patterns detection
Reconstruction & photorealistic rendering of urban environments, symmetry detection, hole filling, etc.
Façade Image Rectification
Vanishing points (VPs) detection
TILT
Vanishing points Draw m hypothetical lines at angular intervals for each vanishing point
Output
Façade Texture Selection
Let C be the non-negative m×m matrix, with each element representing the number of Harris corners inside a block
Slide a window of size r×c along C
Compute the sample mean deviation of the sample median of matrix C, μC
Façade Texture Selection
Kronecker Product Model
A Toy Example
Input facade Repeated patterns
Repeated Patterns Detection via Kronecker Product Model
• Define the cost function:
• The minimization problem:
Solution to The Minimization Problem
Theorem 1: Let be the Singular Value Decomposition of the rearranged counterpart of matrix . Then partition matrices , patterns and weighting factors should satisfy the following:
Solution to The Minimization Problem
Lemma 1: Assuming that matrices are known, then the optimal and are related as follows:
Spatial Periods Estimation via Cross Correlation
• Consider that are known, then we can estimate the periods for facade partition
3×10 partition blocks Fi,j
Column-wise Cross-Correlation sequences: distance between the
adjacent peaks provides the period information.
Partition
Block Vectorization
• Vectorization of partition blocks
• The equivalent cost function:
Vectorization
rank = 30
Estimating K by Clustering
Reshape
rank = 4
Estimation of Mκ, κ =1, 2, …, K★
• Reshape the K★ indicators vectors from Algorithm 2
Computing Patterns and Weighting Factors
• Using Lemma 1 and Algorithm 2
Group 1
Group 2
Group 3
Group 4
Classification
Classification
Classification
Classification
Refinement
Refinement
Refinement
Refinement
1-0 patterns re-construction via Kronecker Product Model
Experiments and Evaluation
• Experiment 1: image rectification and texture selection
• Experiment 2: repeated patterns detection 89 façade images with ground-truth
Success rate 96%
Pixel-wise comparison
Success threshold: 91% match with ground-truth
Results
Proposed Work
Improve the estimation of K★
Extend the method to model nested patterns
Apply the model to 3D point clouds
Improve the estimation of K★
• Limitation: the accuracy and efficiency is drawn back by
– K-means clustering algorithm
Period Computation for Nested Patterns
• The current model may cut a bigger pattern into pieces due to that:– The most frequently appeared pattern dominates the final period
Apply the model to 3D point clouds
Timeline
• My work plan to complete the dissertation is arranged as follows:– Jan. 9 2015 - Jan. 20 2015:
• Finalize the implementation of new methods.
• Run experiments to test the new methods.
• Analyze the performance based upon experiment results.
– Jan. 21 2015 - Feb. 21 2015: Complete the rest part of my thesis and prepare for defense.
– Finally: Defend in March 2015.
Related Publications
• One paper “Automatic Kronecker product model based detection of repeated patterns in 2D urban images” that is related to this proposal has been published in IEEE International Conference on Computer Vision (ICCV) 2013 (accepatance rate 28%, 1600 submissions).
• Another related paper has been submitted to the IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) recently.