Obstacle avoidance system for an ROV

Master thesis by: Lars BruslettoSupervisor: Professor Mart in Ludvigsen

Problem

Autonomous underwater operations need to be able to avoid obstacles.

Is it possible to do this with computer vision in underwater situations?

This thesis work is about obstacle avoidance methods that work in low light, scattering, blurring, and dif f icult camera calibration.

Disparity map obstacle avoidance

We don't want to crash into unknown objects Stereo rig

If (obstacle)

Calculate center of given object.

Calculate possible path away from object

Send calculated path directions using UDP to path program

A BStereo block

match algorithm

Rectify image Aquire images using VimbaSDK

SLIC Superpixel LBP method -->Train (Machine Learning) model

Uniform Local Binary Pat tern

-low computat ional

-high qual ity descript ion

-rotat ion invariant texture

classif icat ion

- i l luminat ion invariant

Machine Learning

Linear Support Vector

Classif ication

Feature extract ionClassif ier t raining

Image database Segmentation

Simple l inear i terat ive

clustering (SLIC)

Fit model

Compute the histogram of the LBP

Histogram computat ion

To be able to calculate disparity:

- need a good camera calibration

Underwater this is tricky:

Figure: Chessboard placed at 80 m depth

Because: Calculate intrinsic and extrinsic camera parameters.

Co-Supervisors: Phd. candidate Trygve Fossum, Phd. candidate Stein M Nornes Phd. candidate Mauro Candeloro

--> Development of the SLIC Superpixel LBP method

--> Development of the Disparity method

--> Development of a way to acquire images from allied vision cameras with python

---> Development of simulation environment module to test computer vision obstacle avoidance algorithms .

--> Development of SLIC Superpixel LBP classif ication algorithm that might be used in dif ferent kinds of applications, i,e autonomous pipe following , rust detection , shell f ish recognition ,star f ish recognition and more.

Main Contribut ions

The obstacle avoidance system takes control of the path choice

when it detects obstacles.

Sends calculated path to path program

Part of autonomy program

Disparity image with a f itted ell ipse

Resul tsUsing model to Predict and avoid obstacle

Conclusion

The disparity method and the SLIC Superpixel LBP method is successfully implemented and tested. The disparity method is proven to work in sea trials in the Trondheimsfjord.

Under simulation experiments the two methods have been compared and they both give very satisfying results in obstacle avoidance.

The disparity method is faster, and therefore better suited for real t ime applications.

The SLIC Superpixel LBP method has great potential also for other underwater applications as it is rotation and il lumination invariant. This means it is robust under dif ferent rotation and light conditions.

Future work

One could experiment with training the classif ier for more objects as the model has capacity to classify more than two classes. It could be interesting to train the classif ier for "reef", "sand", "subsea installation" and try to preform autonomous scene interpretation based on such a classif ier.

The disparity image contains noise that makes it hard to reconstruct 3D models form the image, one could investigate further how one would f ix the noise problem.

Implement underwater computer vision SLAM, if one is able to get disparity images with litt le noise.

One could create and test a DP anchor module based on the SLIC Superpixel LBP method, as it is robust to light and rotation. And the user could specify the object it wants to anchor to.

Make the SLIC Superpixel LBP method run faster using GPU multi-threading and CPU threading. And also vectorize more of the for loops in python.

Using disparity method to avoid obstacle

Achanta R, Shaji A, Smith K, et al . (2012) SLIC superpixels compared to state-of -the-art superpixel methods. IEEE t ransact ions on pat tern analysis and machine intel l igence 34(11): 2274?2282. Available f rom: ht tp:/ / dx.doi.org/10.1109/TPAMI.2012.120.

Machine Learning - Hands-On for Developers by Jason Bel l (Wiley, 2015).pdf (n.d.).

Ojala T, Piet ikäinen M and Mäenpää T (n.d.) Mul t iresolut ion Gray Scale and Rotat ion Invariant Texture Classif icat ion with Local Binary Pat terns.

Piet ikäinen M and Heikkilä J (n.d.) Image and Video Descript ion with Local Binary Pat tern Variants. Available f rom: ht tp:/ /www.ee.oulu.f i / research/ imag/mvg/ f i les/pdf /CVPR-tutorial -f inal .pdf .

Rodriguez-Teiles FG, Geovani Rodriguez-Teiles F, Ricardo P-A, et al . (2014) Vision-based react ive autonomous navigat ion with obstacle avoidance: Towards a non-invasive and caut ious explorat ion of marine habitat . In: 2014 IEEE Internat ional Conference on Robot ics and Automat ion (ICRA). Available f rom: ht tp:/ / dx.doi.org/10.1109/ icra.2014.6907412.

References

SLIC Superpixel LBP obstacle avoidance

Uniform Local Binary Pat tern

-low computat ional

-high qual ity descript ion

-rotat ion invariant texture classif icat ion

- i l luminat ion invariant

Machine Learning

Linear Support Vector

Classif ication

Feature extract ion Classif ier deciderAcquire image Segmentation

Simple l inear i terat ive

clustering

Classify model

Compute the histogram of the LBP

Histogram computat ion

If (segment(classifer) == "other")

Calculate center of given object.

Calculate possible path away from object

Send calculated path directions using UDP to

path program

Mask image

The disparity methodPros

+ fast

+robust

+ possibil ity to create 3D

point clouds from data

Cons

-need calibration

-need 2 cameras( heavier

payload + more

expensive)

SLIC Superpixel LBP

Pros

+robust in dif ferent l ighting

+no need for camera

calibration

+ need only one camera

Cons

-slower

-no depth information

No obstacle case

Obstacle case