Automatic Visual Inspection of Tunnels (AVIT):
Image Mosaicing via Robust Quadric Surface Estimation
M ti tiMotivation
U t i t k i i f t t• Urgent maintenance works: aging infrastructure
• Current visual inspection method is ineffective in time and costsCurrent visual inspection method is ineffective in time and costs
• Urgent need for an automatic system to detect anomalies
• Advancement in Computer Vision technology
System Outline
Homography-based MosaicingHomography based Mosaicing
Di erging parallel lines MisalignmentObtained from the ‘Autopano’ software
• More distortion when mosaicing in the ‘along’ direction
Diverging parallel lines Misalignment
g g
• Algorithm works well when a camera undergoes pure rotation
Homography-based MosaicingHomography based Mosaicing
MisalignmentDi i ll l li
Obtained from the Microsoft `Image Composite Editor software
• More distortion when mosaicing in the ‘along’ direction
Diverging parallel lines
g g
• Algorithm works well when a camera undergoes pure rotation
Aldwych tunnel data setAldwych tunnel data set• Good pairwise reconstruction
• Sufficient 3D informationTunnel liningsy x
• Reconstruction of the entire sequence possible
z
3-1-2
3-1-1
2-1-2Overlap 40%-50%
Camera Overlap
1-1-12-1-1
2-1-21-1-2Camera 1
Overlap 40-50%
Tunnel linings
Reconstruction after RefinementReconstruction before Refinement
Reconstruction after RefinementSurface EstimationSVM classifier
Curvature due to incorrect surfaceincorrect surface
SVM classifier
Texture mappingSVM classifierpp g
Mosaics on Curved SurfacesMosaics on Curved Surfaces
Parallelism is preserved Local misalignment can be refined
Blending algorithm to smooth colour at the overlapping region can be refined
Mosaics on Curved SurfacesMosaics on Curved Surfaces
Microsoft Image Composite Editor on prewarped images
More ResultsMore Results
More ResultsMore Results
Local misalignment due to 3D oca sa g e due o 3structures
ProblemProblem
ProblemProblem
ProblemProblem
• Ransac algorithm for 3D goutlier removal
C d l i f• Correspondence solving for more accurate registration and reconstruction
C t S tCurrent System
Outline of the future systemOutline of the future systemVideo Based
ReconstructionVideo Capture
System
Surface Estimation with
Priors
Multiple View Reconstruction
Image warping and
Final MosaicImage
Retrieval
Labelled SVM LearningImage Dataset
SVM Learning
Registration
and
Change Detection
Correspondence
Solving Change Detectiong
• Database can be created from the initial set of labelled images
F t kFuture work• Correspondence solving with the aid• Correspondence solving with the aid
of 1) crack and deterioration patterns, 2) regions (tunnel linings, pipes), 3) Fourier transform (good for
2003 2007
cracks), 4) photometric changes (color, illumination)・・ 5) structure removal
• RF classifier (key point tracking),tree structure for multiple hypotheses (david lowe’s)(david lowe s)
• Applications: Image registration and change (crack) detectiong ( )
・ Several tens of crack and anomalies images, prototype of the above
Image from Sinha et. al. (2006)
Acknowledgement
Supervisor : Prof. Kenichi Soga and p gProf. Roberto Cipolla
Collaborator : Fabio Viola and Dr. Taekyun Kim
G t EPSRC COT d Ch i t’ d tGrants : EPSRC, COT, and Christ’s graduate award
Thank You
F t kFuture works• Image registration and extending the sequence• Image registration and extending the sequence
• Automatic crack detection
• Change detection
Image from Sinha et. al. (2006)2003 2007
2003 2007
C• Change detection by accurate geometrical registration
• Semantic Texton Forests for ImageSemantic Texton Forests for Image Categorization and Segmentation (Shotton et. al. 2008)
• Systematic database collection