Date post: | 16-Dec-2015 |
Category: |
Documents |
Upload: | andrea-randall |
View: | 215 times |
Download: | 1 times |
GM-Carnegie Mellon Autonomous Driving CRL
Structured Hough Voting for Vision-
based Highway Border Detection
1
Zhiding YuCarnegie Mellon University
GM-Carnegie Mellon Autonomous Driving CRL
Sensors Setup on SRX Platform
Images from: Junqing Wei et al., “Towards a Viable Autonomous Driving Research Platform,” IEEE Intelligent Vehicles Symposium (IV), 2013
GM-Carnegie Mellon Autonomous Driving CRL
Sensors: Price vs Information
Price
Info
rmat
ion
Radar
Lidar
Camera
GM-Carnegie Mellon Autonomous Driving CRL
Computer Vision Applications
Object detection (pedestrian, vehicle, bicycle…) Road parsing (lane/border detection, road segmentation,
vanishing point estimation…) Localization and tracking Driver status monitoring Many other applications……
GM-Carnegie Mellon Autonomous Driving CRL
Motivation, Description and Goal
7
Goal
– Development for future driving assistance system and autonomous driving system
– Robust detection within 0.5 to 6 meters detection range. Achieve near 100% accuracy in daytime and over 90% in nighttime on the right most lane
– Handling various scenarios including highway entrance and exit
– Extend to the joint system with front view
GM-Carnegie Mellon Autonomous Driving CRL
Concrete Barrier Guard Rail Soft Shoulder
High-Level Idea: Learning based Method
Guard Rail
Soft Shoulder
ConcreteBarrier
Lane Marking
Densely Fired scanning windows
Returned Voting Points
Border / lane marking hypotheses
Structured Hough Voting
GM-Carnegie Mellon Autonomous Driving CRL
Overall 1592 training images:1. Concrete Barrier (839 images)
2. Guard Rail (300 images)
3. Soft Shoulder (453 images)
Overall 2638 testing images:
Dataset Collection
GM-Carnegie Mellon Autonomous Driving CRL
Training Patch Alignment
Negative Samples:
Positive Samples:
Concrete Natural Steel Lane Marker
GM-Carnegie Mellon Autonomous Driving CRL
Filter Bank
Patches that are discriminative to HOG
Patches that are discriminative to filter banks
Concatenated Filter Bank Feature
Feature Extraction
HOG
Concatenated HOG Feature
GM-Carnegie Mellon Autonomous Driving CRL
Extract features from all training patches (based on previous page)
Perform Fisher discriminant analysis
Train an RBF kernel SVM
Scanning window detection (Deliberately having a lot of positive firing)
Classification & Detection
Guard Rail
Soft Shoulder
ConcreteBarrier
Lane Marking
GM-Carnegie Mellon Autonomous Driving CRL
Structured Hough Voting: Intuitions Basic philosophy: A model that assumes voting results are correlated rather
than independent
Inter-frame structural info on hypotheses (Temporal smoothness)
Intra-frame structural info (Geometric relationship)
Multiple candidate hypotheses generation (Proposals with diversity)
1. Constrained Hough Voting on detected voting points (Detection + Tracking)
2. Arbitrary Hough Voting on detected voting points (Detection)
3. Constrained Hough Voting on image gradients (Pure Tracking)
GM-Carnegie Mellon Autonomous Driving CRL
Deals most of the frames where hypotheses from consecutive frames have strong correlation.
Purpose of Candidate 1
GM-Carnegie Mellon Autonomous Driving CRL
Automatically corrects result through searching for “much better” voting configurations (This is the power of detection, avoids error from tracking)
Purpose of Candidate 2
GM-Carnegie Mellon Autonomous Driving CRL
In the worst case where Type 1 voters fail, perform tracking by gradients from previous pose configuration.
Purpose of Candidate 3
GM-Carnegie Mellon Autonomous Driving CRL
Modeling under CRF: Background
A Conditional Random Field (CRF) discriminatively defines the joint posterior probability as the product of a set of potentials
The potentials are functions with hypotheses Hi being the variables. They are modeled in such a way that a larger potential value generally indicates a better hypothesis configuration.
CRF inference seeks to find the joint hypothesis configuration H that maximizes
H1
X1
H2 HN…
Unary Potential Pairwise Potential
X2 XN
GM-Carnegie Mellon Autonomous Driving CRL
Modeling under CRF: Intuition
What are the hypothesis Hi?
E.g.: image pixel labels (FG/BG, Object Class, etc.), if it is a segmentation problem.
In our problem, Hi is the Hough Voting hypothesis: Hi = (r, θ).
X is the observation of voting point coordinates and their weights.
The unary potential corresponds to the exponential of Hough voting weights: exp(v(Hi)).
The pairwise potential corresponds to the inter-frame smoothness (tracking) constraint.
H1
X1
H2 HN…
X2 XN
GM-Carnegie Mellon Autonomous Driving CRL
No Structural Information
Hbd,1…
Simplest Case: frame-wise independent Hough voting
Hbd,2 Hbd,N
X1 X2 XN
Hln,1…Hln,2 Hln,N
X1 X2 XN
GM-Carnegie Mellon Autonomous Driving CRL
Adding Inter-frame Structural Info.
Hbd,1…
Adding temporal smoothness: Hough voting constrained by neighboring frames
Hbd,2 Hbd,N
X1 X2 XN
Hln,1…Hln,2 Hln,N
X1 X2 XN
GM-Carnegie Mellon Autonomous Driving CRL
Adding Intra-frame Structural Info.
Hbd,1…
Adding Geometric Constraint: Hough voting constrained by both neighboring frames and intra-frame hypotheses
Hbd,2 Hbd,N
X1 X2 XN
Hln,1…Hln,2 Hln,N
X1 X2 XN
GM-Carnegie Mellon Autonomous Driving CRL
The Structured Hough Voting Model
• • •
• • •
Candidate Hypotheses Generation Unit
Mode Selection Potential
Coupled Structure Potential
GM-Carnegie Mellon Autonomous Driving CRL
Use decision tree to guide the mode selection.
The mode selection basically forces the output to be one of the candidate hypotheses, but allows discrepancy with the decision tree prediction with a penalty.
Mode Selection Potential
GM-Carnegie Mellon Autonomous Driving CRL
The coupled structure potential captures two most important relations between a border hypothesis and a lane hypothesis
Parallelism
Distance
Coupled Structure Potential
GM-Carnegie Mellon Autonomous Driving CRL
Inference
Conducting a whole inference each time given a new frame is computationally infeasible.
Relaxation: Initialize with the inferred state variable configuration of the previous t-1 frames and infer the current state variables, updating in an incremental way.
Inference procedure at t = 1:1. Perform Hough voting for both border and lane marking2. Perturbate hypotheses if geometric relationship violated (optional)
Inference procedure at t > 1:1. Generate the 3 candidate hypotheses for both border and lane marking2. Use decision tree to help selecting the best candidate3. Perturbate candidate hypotheses if geometric relationship violated (optional)4. Re-select the best candidate
GM-Carnegie Mellon Autonomous Driving CRL
Experiments: Qualitative Results
Ground Truth and Baseline methods:1. Ground Truth2. Independent Hough voting in each frame using the fired detector voting points3. Hough voting using the triggered detector voting points constrained by previous frame 4. Adding gradient tracking to Baseline 2.5. Kalman filter.6. Proposed Method
GM-Carnegie Mellon Autonomous Driving CRL
Summary
Proposed the Structured Hough Voting Model
The proposed model can be theoretically formulated under a CRF
Fast real-time feature extraction and online inference
Achieves very robust and good performance under challenging scenarios
and low quality inputs from production camera