1Scientific SystemsScientific Systems

Scientific Systems Company, Inc

Presentation at Meeting No. 96

Aerospace Control and Guidance Systems Committee

Harbour Town Resorts

Hilton Head, S. Carolina

19-21 October, 2005

By

Raman K. Mehra

Phone: 781-933-5355

Email: rkm@ssci.com

Scientific SystemsScientific Systems

VISTA: Visual Threat Awareness for Unmanned Air Vehicles

3Scientific SystemsScientific Systems

VISTA Overview

Problem: Situational awareness is required for “nap of the earth” UAV autonomy

Goal: Research, design and flight test a system for real time, visual collision obstacle detection

Proposed solution: Real time stereo + Perceptual organization + Image segmentation + Region tracking = VISTA (Visual Threat Awareness) system

4Scientific SystemsScientific Systems

Challenges of UAV visual obstacle detection

• 3D Reconstruction: Difficulty of computing 3D scene structure from two or more 2D images

• Performance tradeoffs

– Missed detections and false alarms vs. available computation

– Scene resolution vs. available computation

• Stereo vision: correspondence errors due to low contrast, occlusions, specular reflections, foreshortening, periodic features, threshold choice, minimum distance violation

– Traditional block matching stereo is too noisy for practical use in general!

5Scientific SystemsScientific Systems

VISTA Achievements

• First application of 640x480@23Hz stereo in UAV flight

• Real time (3-5Hz) perception algorithm accelerated by Sarnoff Corp’s Acadia I vision processor for improved performance over traditional stereo

• Proof of concept visual obstacle detection and avoidance in two flight experiments on Georgia Tech’s GT-Max helicopter against rural obstacles

Obstacle Map: Green = Nearest Collision Obstacle, Yellow = Obstacle Disparity Map: Light = Close, Dark = Far, White = Undefined

6Scientific SystemsScientific Systems

VISTA Architecture

• Stereo: Large baseline (50cm) binocular stereo for range measurement

• Foveation: Compression appropriate for collision detection

• Segmentation: Measurement fusion, Grouping for obstacle hypotheses

• Detection: False alarm rejection, Obstacle tracking

7Scientific SystemsScientific Systems

Segmentation examples

Synthetic Sphere Synthetic Sign

Building Hallway

Large Tree Small Tree

8Scientific SystemsScientific Systems

Flight experiment 1.1.10: Detection video

Obstacle Map: Green = Nearest Collision Obstacle, Yellow = Tracked obstacles

Disparity Map: Light = Close, Dark = Far, White = Undefined

9Scientific SystemsScientific Systems

Flight experiment 2: Detection and avoidance

Flight 2.1.6 Flight 2.2.5

Flight Obstacle Time SpeedCollision

Volume

Keepout

Radius

Replanning

Radius

Detection Error @ Obstacle Distance

2.1.6 “Sign” 11:52 am 10 ft/s 60m/5m 40 ft 65 ft 7ft @ 65ft

2.2.5 “Sign” 3:23 pm 10 ft/s 60m/5m 40 ft 82 ft 15ft @ 82ft

Avoidance Triggered

Avoidance Triggered

10Scientific SystemsScientific Systems

Flight experiment 2.1.6: Ground video

11Scientific SystemsScientific Systems

Flight experiment 2.1.6: Detection video

Obstacle Map: Green = Nearest Collision Obstacle, Yellow = Tracked obstacles

Disparity Map: Light = Close, Dark = Far, White = Undefined

12Scientific SystemsScientific Systems

VISTA Summary

Major Accomplishments:

• First application of 640x480@23Hz stereo hardware in UAV flight

• Real time (3-5Hz) algorithm that improves collision obstacle detection performance over traditional stereo only

• Proof of concept in two flight experiments against real obstacles.

We have demonstrated proof of concept for a real time visual collision detection system in a rural environment.