Computer Vision for Stuctural Damage in Tunnels
Research and new products
Konstantinos Loupos (Senior Project Manager) Institute of Communication and Computer Systems (ICCS)
Greece May 22-28, 2015
National Technical University of Athens (Greece) Institute of Communication and Computer Systems (Greece)
i-sense group Smart Integrated Systems Team
A few words for my team…
• Sensors - signal and image processing
• Communication systems • Computer vision algorithms
(capturing and processing) • Monitoring, control and
automation systems • H/W (digital and analog
electronics)
• S/W engineering and computer technologies
• Control and Robotics • Microwave, MMW and optical
sensors
• Non-profit organization • Very active participation into EC Co-Funded Projects
– More than 35 running EC projects – Some National
• Smart Integrated Systems Team – Structural Health Monitoring – Security Applications – Sensorial Systems Design and Development – Communication Systems Design and Implementation – Water Demand Management and Applications
A few words for my team…
Summary
Introduction
Visual inspection system description
Data collection details
Preliminary 3D information extraction
Conclusions
Overview
ROBO-SPECT – EC Co-funded Project
•Collaborative project (STREP) - FP7 - ICT
•THEME 3 - Cognitive Systems and Robotics
•Project full title: ROBotic System with Intelligent Vision and Control for Tunnel Structural INSPECTion and Evaluation
Recent and exploitable research
in
ROBO-SPECT is an innovative, integrated, robotic system that, in one pass, will perform inspection and assessment of transportation tunnel linings, minimizing humans’ interaction and has the potential to be commercialized in the short to medium term
scans the intrados for potential defects on the surface
detects and measures r a d i a l d e f o r m a t i o n between parallel cracks
Advanced functionalities of the ROBO-SPECT system will include: •Interweaving of computer vision and sensing techniques with intelligent control of a multi-degree-of-freedom robot to enable it to provide, speedily, all the required parameters for structural assessment with the required accuracy •Ability to automatically adapt to different use cases •A crack meter sensor for the simultaneous measurement of crack width and depth and its integration with the robotic arm •A novel quantitative structural assessment tool that, based on the inspection of the lining intrados, will assess the structural condition of the inspected reinforced concrete tunnel lining
• NDT techniques for detection and recording of defects (Davies & Mamlouk, 1985), ! many image-based methods
• Haack et al. (1995) review of mechanical, radiation, electric and optic techniques
• Image processing, segmentation, feature extraction, pattern recognition, classifications and reconstruction have been reported
• Sinha et al. (2003) review of computer vision techniques on concrete pipelines
Introduction
▪ Automation in tunnel inspection is still low
▪ Advances in 2D/ 3D Vision are not exploited
Graphical representation
Autonomous system for tunnel inspection
Initial integration
Autonomous system for tunnel inspection
Computer Vision contribution
Computer Vision Modules for Tunnels Inspection
Detection of defects 3D modelling
Cracks in real-time
Other defects offline
Local 3D around cracks
Robot guidance
Production year 2014Model PointGrey-Grasshopper GS3-U3-91S6C-
CNumber of pixels 9.1 MegaPixelsPixel size 3.69 µmWidth 3376Height 2704CCD diagonal 1"Focal length 12.5mm FoV crop factor 1.7Sensor type Sony ICX814 CCD, Global shutterWeight 2 x {90g (sensor) + 290g (lens)}Single Camera size 44x29x58 mm (sensor) + 120x90 mm
(lens)Video 9 fps
Camera technical specifications• The requested scale K of recorded
images should be better than 1:135 • 1.7m distance from the stereo rig to
the lining • Area captured in a frame has 1.68m
length • Depth of field for the aforementioned
configuration and f/2.8 is between 1.5m and 2.0m.
Computer Vision Equipment
• The detection and reconstruction algorithms are adaptive to different tunnels.
• At the first evaluation data from Egnatia motorway tunnels (Northern Greece) were gathered (main and ventilation tunnels).
• Tunnel heights are from 3.5m in ventilation tunnels to 11m in main tunnels.
Data Collection
Difficulties of Computer Vision in tunnelsImaging confronts many problems not known to common computer vision tasks: •operational difficulties •accessibility difficulties (eg. strong airflow, draining water, puddling water) •blurred images due to low shutter speed, or •artifacts due to strong flash lights •vibrations due to self-movement and traffic •Low texture •Weak geometry of the depicted object
Data Collection
Calibration of the stereo pairUse stereo to: •improve calibration results •constrain the geometry •ensure good stereo through sufficient overlap.
➔ The circular geometry of the tunnel and the low texture forces the use of pre-calibration.
Preliminary 3D information extraction
3D reconstruction of a crack• adaptive local stereo-matching
algorithm • Inputs to the structural
assessment are: - measurable width - position and - orientation of the crack. • noise in the reconstructed
surface • challenging images through
conditions described above • ongoing work
Preliminary 3D information extraction
• Shape – Cracks is expected to present large length and small width – Cracks is expected to be “not straight” lines
• Intensity – Pixels belonging to cracks are expected to be darker than their neighbouring
pixels that do not belong to cracks
Cracks‘ Characteristics
• Image Processing techniques – Based on the intensity of pixels and their spatial relations – Morphological operations – Filtering – Simple shape analysis of detection results
• Filtering based on areas • Filtering based on “sphericity”
Pros • No need of annotated data • Operations can be parallelized and achieve better than real-time
performance (25fps) Cons • Low generalization ability
– Methods are dataset oriented (fine tuned for a specific dataset)
Methodology for Crack Detection
• Machine Learning techniques – Based on the intensity of pixels and their spatial relations – Exploitation of low level image features for
• Detecting cracks • Hierarchically constructing high level features
– Artificial Neural Networks – Markov Random Fields
Pros • Generalization ability • Learning machines can achieve real time predictions (depended on their
structure) Cons • Need of annotated data
Methodology for Crack Detection
• At these stage our analysis is based only on image processing techniques • Based on cracks characteristics our approach consists of the following steps
(pipeline): – Lines enhancing – Noise removal – Straight lines removal – Shape filtering – Morphological reconstruction
Methodology for Crack Detection
• Lines enhancing – The intensity of pixels that are darker than the average intensity of their
neighbours is set to zero
Image Processing techniques
• Noise removal (1) – Median filter is applied on binary enhanced image to remove “salt and pepper”
noise
Image Processing techniques
• Noise removal (2) – Connected components whose area is smaller than a threshold are discarded
Image Processing techniques
• Straight lines removal – Straight lines are detected and discarded
Image Processing techniques
• Sphericity filtering – Filtering is based on the observation that the minimum bounding circle of a
crack must have much larger area than the crack.
Image Processing techniques
• Morphological reconstruction – The sphericity filtered image is used as seed and binary filtered image is used as
mask
Image Processing techniques
• Cracks detection vs Original image
Egnatia tunnel
Image Processing techniques
• Cracks detection vs Original image
VSH tunnel
Image Processing techniques
3D laser scanner data• Slices of the tunnel are
scanned to detect deformations
• Several scanners are tested in
tunnels and a calibration field for
their accuracy and precision • Measurement of dimensions of the deformed tunnel cross-
sections
Preliminary 3D information extraction
▪ Specialized computer vision system for detection and accurate 3D reconstruction of tunnel defects
▪ Tests on real motorway tunnels (Egnatia on northern Greece) ▪ Algorithms to be evaluated on railway tunnels (London Underground) ▪ Focus on pre-casted concrete lining ▪ Preliminary results from sub-mm 3D reconstruction ▪ Online dataset for evaluation of algorithms to be published ▪ Ongoing research
Conclusions
Thank you …➢ … for your attention! ➢ … time for questions.
Konstantinos Loupos (MEng, MSc, PMP) Senior Project Manager Institute of Communication and Computer Systems [email protected]
