Bohr Robot GroupOpenCV

ECE479John Chhokar

J.C. AradaRichard Dixon

Goals

• Learn OPENCV• Implement Face Tracking Robot using Haar Cascade• Implement Color Detecting Robot• Implement Obstacle Avoidance Robot/Navigation

Face Detection• OpenCV makes face detection easy using the Haar Cascade

Classifier stored in a XML file.• Classifier uses data stored in the XML file to classify each image• The Haar sequence was proposed in 1909 by Alfred Haar• Viola-Jones method is the fundamental technique used in face

detection where their work models wavelets (Publish 2001, Conference on Computer Vision and Pattern Recognition)

• Haar Feature Rectangles are used to detect Haar features.• Computations of Haar Rectangles are performed by a technique

called an Integral Image. The pre-processing of the image to significantly increase Haar like features.

Face Detection

• 4 = A + B + C + D• 2 = A + B• 3 = A + C• 1 = A

Integrals Image Value, the sum of all the pixels above it and to the left

D = (x4,y4) – (x2,y2) – (x3,y3) + (x1,y1)

• Sum of all pixels inside of a rectangle needs only 4 array references. S = A – B – C + D. Two rectangle sums = 8.

Face DetectionOpenCV Haar Features

)Sum(r)Sum(r black i, whitei, if

thresholdfif

thresholdfifxh

i

ii 1

1)(

• Sum of the pixels which lie within the white rectangles are subtracted from the sum of the pixels within the grey rectangles

• Rectangle features can be compiled very quickly.

Face DetectionFiltering keys on differences in facial features.

Eye region is often darker than cheeks.

Intensity across bridge of nose and eye region.

All filters must pass for image to be recognized as face. If one fails image is not considered

Face Detection Demo

Color Detection• cvCvtColor• cvInRangeS• cvSmooth• CvSeq* circles = cvHoughCircles

Color Detection

Color Detection #2

Obstacle Avoidance/Self Navigation

Work in progress • The goal of this portion of our project is to have the Bohr Robot

self navigate a hallway or corridor and possibly negotiate a right or left turn.

• To accomplish this that we implement a series of functions and transformations – Canny edge detection – Hough Line transform

How Color Detection Works• Colors are represented by a combination of Red Green and Blue• We simplify the color scale by defining colors in a 24bit scale

– Red, Blue, and Green are represented each by 8 bits

• Bitmap images can best be described as a matrix populated by the 24bit values. – We use a reference value for a color and start matching any

color that color that corresponds to that value

Simplified Algorithm• Below is the simplified algorithm on how OpenCV detects colors

• Get image size– Get X and Y values of images

• While I <= X– While J <= Y

• If Reference == Color– Then match == 1;– Else match == 0;

• J = J+1;• End while

– End while

How the Program Works• We use the X value obtained from the match

– Average out the values of X in the frame• Move the motor towards X

– Done by defining the center as the reference– We then move the motor left or right dependent on the

reference point

Problems encountered• Latency is too slow

– Movement of motor dependant an event that already occurred.

– Bad feedback• Algorithm is not optimized

– Using sleep instead of polling

• Workaround is to delay sending of commands– Average out movement.

Obstacle Avoidance/Self Navigation

• Laplace function

• Canny edge detection – Contours – Formed by thresholding image pixels– If gradient is of pixel

• lower it is rejected• higher it is accepted • Between accepted if connected to a pixel in has a

higher gradient– 2:1 or 3:1

Obstacle Avoidance/Self Navigation

• cvCvtColor(frame, grey, CV_BGR2GRAY);• cvCanny( grey, edges,50, 150, 3);

Obstacle Avoidance/Self Navigation

• Hough Transformation• Standard Hough Transformation(SHT)

– Uses the slop intercept form (y = mx + b)– Range + infinity

• Probabilistic Hough Transformation (PPHT)– P=x cos(t) + y sin(t) – Accumulates a fraction of points

Obstacle Avoidance/Self Navigation

• cvHoughLines2( edges, storage, CV_HOUGH_PROBABILISTIC, 1, CV_PI/180, 50, 50, 10 );

Obstacle Avoidance/Self Navigation

• What Next• Implementation of our decision algorithm

– Calculate the slope the lines in our ROI (~ 2 ft)• If there are no line in ROI

– Move forward• If slope is -.1 to .1

– stop( object in front)– Cal slope (ROI) Left (~1ft)

• If no line then turn• Slope?

– Cal slope (ROI) Right (~1ft)• …

Obstacle Avoidance/Self Navigation

Obstacle Avoidance/Self Navigation

References

• Paul Viola and Michael Jones Original Paper 2001 at Computer Vision and Pattern Recognition Conference

http://research.microsoft.com/enus/um/people/viola/Pubs/Detect/violaJones_CVPR2001.pdf• Cognitive Robotics Seeing with OpenCV Finding Faces http://www.cognotics.com/opencv/servo_2007_series/part_2/index.html