Advanced Cubesat Imaging

Nathan Bossart, Joe Mayer, Bob Urberger

RASCAL ACIP

Team Introduction

http://acip.us Facilitator: Bob Urberger Computer Engineering majors Space Systems Research Lab

RASCAL Mission “Rascal is a two-spacecraft

mission to demonstrate key technologies for proximity operations…”

“After the on-orbit checkout, one 3U spacecraft is released and passively drifts away.... After a suitable distance, the released spacecraft will activate its propulsion system and return to within a few meters of the base. The second spacecraft will be released and the process repeated...”

RASCAL ACIP

Imaging Payload Awareness of Cubesat

EnvironmentComputer Vision

Low-Level ProcessingObject DetectionDistance Determination

High-Level DataNavigationThruster Control

RASCAL ACIP

Functional BreakdownUnique Face Identifier

Capture Images

Transfer Data

Process Images

Output/Store Control Data

RASCAL ACIP

Known Pattern

RawData

StructuredData

High-LevelData

Modules

RASCAL ACIP

LEDs

Camera

Computational Hardware

Unique Face Identification

Capture Images

Transfer Data

Process Images

Output/Store Control Data

LEDs Required to perform

classification Not enough detail

visible for other features

Three approaches Unique pattern of

LEDs for each face Unique combination

of colors for each face Both unique patterns

and colors

Unique pattern works regardless of camera spectrum Fails when face

partially visible Color combinations

only work with visible spectrum cameras Can classify cube

corners as well as faces with well chosen color patterns

RASCAL ACIP

Camera Potential Camera

Choices: FLIR Tau 640

640x480, 14-bit Visible spectrum

image sensor 2-5MP - 16 to 24 bit

color Parallel data output

from cameras

Component Requirements: FLIR

PCB integration Control signaling simple Low resolution,

monochromatic 16.1 MB/s input data rate @

30Hz Visible spectrum

Requires lens fixture Complex control signaling High resolution, wide color

range 2MP with 24 bit color:

○ 57.6 MB/s input data rate @ 30Hz

RASCAL ACIP

Processing Hardware Processing blocks in hardware Caching and system control managed in

software Timing and Gate Consumption Alternatives:

Pure software implementationPure hardware implementation

RASCAL ACIP

Imaging Functions Image Processing

Pre- Processing

DistanceDetection

ObjectDetection

ObjectClassification

RASCAL ACIP

ImageDataStructure

Distance Data

ImageEdges

Objects In Frame

Preprocessing Noise suppression

Color Conversion

Object enhancement

Image segmentation

Conversion and downsampling

RASCAL ACIP

Distance Detection Identify depth from a

single image Monocular Cues

Relative sizeComparison of

imaged objects to known shape scale at particular depth

Structured geometry identified should be easy to identify scale regular structureSquare or equilateral

triangle

RASCAL ACIP

Distance Detection in RASCAL

Square LED pattern on spacecraft faceCritical point identificationHomography estimationProjective transformPoint correspondence for scale

Hardware DomainParallel matrix multiplication

RASCAL ACIP

Object Detection Identifying Objects

in an Image

Region or Contour Based

Edge DetectionRelies heavily on Pre-

processing

RASCAL ACIP

(Columbia University)

Object Detection in RASCAL

Hardware DomainCanny/Deriche Sobel Operator

ConstraintsCubesat sizeEnvironmentalResolution

RASCAL ACIP

(Columbia University)

Objection Classification Post-object

detection / image segmentation

Support vector machine (metric space classification)

Assign a class based upon pre-programmed control data

RASCAL ACIP

Object Classification in RASCAL

RASCAL ACIP

Completed with bare-metal software

ARM Assembly / C Minimum distance principle

(efficient) Multi-tiered and/or multi-

dimensional space from attributes given

Determine a number of attributes with significant differences between faces

Testing: expect a very high level (>95%) of correct classifications

Constraints of Object Classification Must work with a

variety of backgrounds (Earth, Moon, Sun, Space, etc.)

Ideally real time (bounded) and low latency

Updated at >=10 Hz

Must function with different sizes (patterns can vary from a few pels to larger than the frame)

Definitive discrimination functions with high reliability

Alternative algorithm: neural nets, fuzzy logic

Output to Control System will output calculated information about

placement, attitude, distance, etc. In the future, a separate team will construct a

system to interpret data and convert to control signals/data

Since this is out of the scope of our project, the output format/setup is ultimately our choice

RASCAL ACIP

Functional Testing Output Unique Pattern

Capture ImagesStream ImagesVerify Control Signals

Transfer DataOscilloscopeFrame Buffer

RASCAL ACIP

Process ImageSoftware VerificationHardware Verification

Output/Store DataBuffer

System Testing Camera integration Hardware timing constraints Block connectivity verification

Blocks signal each other as intended Full pipeline simulation

Blocks interact as expected Physical synthesis testing

Data produced from each frame

RASCAL ACIP

Timeline

RASCAL ACIP

Data In and Out of System

Obtain Camera

Separate Processing into Blocks

Interface Camera with Hardware

Algorithm Verification in Software

Store Camera Data in Hardware

Preprocess Image

Achieve Block Functionalality

Merge Processing Blocks

Confirm Full Integration

Project Wrap-Up

10/28 11/17 12/7 12/27 1/16 2/5 2/25 3/17 4/6 4/26

Dates of Years 2013-2014

Project Task

Estimated Costs

Designed for very low budget and small amount of needed materials

Largely out of SSRL funding

Function Part Low Estimate High Estimate Notes

Obtain vision data Camera $4,000 $10,000 SSRL funding

  Camera Specification $0 $10,000 SSRL funding

Display patterns LEDs $1 $10 SSRL funding

Algorithm Resources Books $0 $0 library & creative commons

Development Tools Zedboard $0 $0 donation

  Xilinx Vivado $0 $0 donation

  Oscilloscope $0 $0 provided

  Desktop PC $0 $0 provided

TOTAL COST: $4,001 $20,010 RASCAL ACIP

Future Work for Integration

Thruster control system

Placement into spacecraft

Radiation, vibration, space-readiness

We will provide thorough documentation for future groups

RASCAL ACIP

Bibliography http://cubesat.slu.edu/AstroLab/SLU-03__Rascal.html Jan Erik Solem, Programming Computer Vision with

Python. Creative Commons. Dr. Ebel, Conversation Dr. Fritts, Conversation Dr. Mitchell, Conversation Milan Sonka, Vaclav Hlavac, Roger Boyle, Image

Processing, Analysis, and Machine Vision. Cengage Learning; 3rd edition.

http://www.cs.columbia.edu/~jebara/htmlpapers/UTHESIS/node14.html

RASCAL ACIP

Questions?

RASCAL ACIP