Computer Vision

• DARPA Challenge Seeks Robots To Drive Into Disasters.

• DARPA's Robotics Challenge offers a $2 million prize if you can build a robot capable of driving and using human tools.

• DARPA Wants to build rescue robots for the military?

• The new challenge; to develop robots that can navigate disaster-response scenarios.

• Robots will be required to complete a number of discrete tasks, including traveling across rubble, removing debris from a blocked entryway, climbing a ladder, and entering and driving a car.

• Bigdog

• Petman

• Robot Swag

• Aquatic synthetic life

Current capabilities show that robot physical forms are quite advanced.

• Why robots?

• They can go places humans can’t or shouldn’t go!– Natural disasters areas too dangerous for people

– Uninhabitable environments

– Nuclear, chemical or biological disasters

Why, then, if my robot is so useful is my android paranoid?

• As mobile and dexterous as my robot is it needs a way to know where it is and how to get to a desired location or to know the location of objects in the surrounding environment.

• This task is referred to as localization.

• Where am I? How did I get here? How do I get out of here, or go to the next place?

• Localization also refers to knowledge of the position or location of important objects in the environment.

• Our robot could use GPS, except after a natural disasters like hurricanes or earthquakes nothing is where it is supposed to be. Major features of a city or locale may no longer exist.

After the Tsunami in Japan all GPS references to buildings and streets was useless

The same was true after this earthquake in China

• Our robot may be in an area where GPS is unavailable – under the sea, in a building, a geographical location with no reference points.

This building could collapse at any time. Perfect for an android.

But, the android would need a method of creating navigational information for these new situations.

• In general, important objects within the environment may be destroyed, moved from their original location, temporarily positioned or too insignificant to have GPS reference data.

• The Goal for our robot is to autonomously develop an independent and real time understanding of its location and position vis-à-vis existing locations and objects using visual information.

• The problem is to develop the machine’s capability to visually recognize objects within the immediate environment.

• However:– Humans see and immediately recognize specific

objects; trees, faces, buildings, etc. We automatically know where we are in relation to these objects.

– This is the result of many complex neurological and cognitive processes.

• Computers don’t see things the way humans do.

• Computers see light and dark, color intensity.

• Computers store numbers and analyze (think) in equations.

• We need a way for computers to identify objects in the environment as reference points for locating themselves and other objects in the environment.

• They will not see objects as we do. Though, they will be able to consistently recognize an object as having been “seen” before and associate the object with a specific location and position.

• Robots see using digital cameras. Images are stored as digital information.

• We need our robot to recognized objects within the digital images of that object taken a different times and at different angles.

• Regardless of ambient conditions of light, weather, time of day, rotation of image and angle at which the picture was taken

• As mentioned, images are stored as digital information.

• This information can be stored as an array of numbers that represent individual dots of the image called pixels.

• For grayscale images similar to “black and white” photographs the intensity of blackness is stored as a number from 0(black) to 255(white)

• For color images the image information can be stored as an array of numbers that represent individual pixels as well.

• Instead of storing information on the intensity of blackness, information is stored about the levels of red, green and blue colors for each pixel (RGB).

• The intensity for these colors is stored as a number from 0(maximum) to 255(minimum)

• Color images can be easily converted to Grayscale by weighting the values for Red, Green, and Blue and summing the resulting values.

• Often color will no provide useful information for our process, but increase the processing burden. We will work in grayscale.

• We need our robot to recognized an object within the digital images of that object taken a different times and different angles.

• Matching an object’s features across different images is a common problem in computer vision.

• When you have different images with different scales and rotations, you need to use an program called Scale Invariant Feature Transform or SIFT.

• The SIFT algorithm analyses sections of a digital image and identifies regions of significant change in grayscale referred to as SIFT points.

• These SIFT points are then described by the size and intensity of the region’s changes and the general direction (orientation) of these changes.

• The region’s location and its relative position to other SIFT points is recorded.

• Through this process SIFT creates a unique “fingerprint” for an object that is independent of many ambient features of a photograph or image.

Photograph of rooftops in [Italy?]

Converted to grayscale and SIFT process applied

The SIFT point are detected

This slide displays information about each SIFT point.

Two photos which share objects

We have found SIFT points that match from the two photos!!!

• Using SIFT our robot can identify a geographic or architectural feature and associate the image with a position either by GPS or another positioning technology.

• Even though our goal is for our robot to employ an autonomous locating ability separate from GPS technology, we can use GPS technology in the development of our system.

• Current research employees geolocalization using GPS and Google street view images.

• I can take a photograph on any street in downtown Orlando and using SIFT and other technologies determine my location by comparing SIFT points in my photograph and SIFT points in images found in Google street view for Orlando.

• The success rate in obtaining a good location is quite high.

• Ultimately my paranoid android may be calmed by the certain knowledge of his location in the world.

• Frankly, the mathematics involved in these tasks is far beyond the scope of any of your current classes.

• Calculus, Statistic and probability theory as well as finite mathematics are employed.

• People who are able to develop Artificial intelligence and robotic capabilities are and will be some of the most sought after employees in the modern work force.

• All These technologies are still developing technologies and creative fundamental work still needs to be done.

• If this type of work interests you, please contact:

• Dr. Mubarak Shah or Dr. Neils Lobo for more information.

• Questions