Image Processing for Remote Sensing

Matthew E. Nelson

Joseph Coleman

ImaP-RS

One method for detecting plant health is to look in the Near Infra-red band (700 ~ 1000 nm). A camera system was developed by the SSCL for this type of remote sensing application. This project continued to improve the efficiency and functionality of the system by moving to a FPGA based system

ImaP-RS

Goals Implement a working Linux kernel on the

ML507 with USB Support Cross Compile existing tools to the

PowerPC architecture the ML507 usesgPhoto2libgphoto2

Implement a co-processor design to handle the image processing

ImaP-RSLinux on the ML507 Xilinx provides a reference base system for

installing a Linux kernel on the ML507Limited functionality, but good starting pointUSB is not enabled in the base system

ML507 Cypress CY7C67300 chipsetNOT supported by Linux

Plan BDigilent SMSC EVB-USB3300-XLX Daughter

boardIt is supported by Linux

ImaP-RS

Demo – Booting Linux on the ML507

ImaP-RS

USB on the ML507

ImaP-RS

USB on the ML507

ImaP-RSCross Compiling the Tools Two software packages are critical for the communication between

the Canon 5D and Linux libgphoto

○ Provides camera interface libraries gphoto

○ Provides command line tools

libexifoptional

libpopt

libusbRequired for

USB cameraslibgphoto2

gphoto2

ImaP-RSCross Compiling the software Cross Compiling libgphoto2 and gPhoto2 proved to

be more difficult than originally planned ELDK would not compile libgphoto2 Would not compile against a library it had, others trying to

compile reported similar problems, either ELDK or libgphoto2 problem

In addition, one other software package would be helpful UFRaw

○ Can convert from Canon’s RAW format to PNG or TIFF○ Can also do some limited image manipulation○ This was also difficult to cross compile due to a number of

dependencies required

ImaP-RSImage Processing When gPhoto is in place, we can obtain images

directly from the Canon 5DWe can also send commands to the camera for

camera settings and to trigger a photograph Images will be stored on a mounted CF Card

ImaP-RS

Image Processing Considerations File Formats Selection Process Results Connectivity Simulink

ImaP-RSImage Processing Considerations Need to get pictures off of flash drive in

camera Cannot remove the flash drive because of

application environment (tethered balloon at ~50-500 ft.

Need high reliability and scripting capabilities to facilitate autonomous action

Onboard image processing for immediate data availability

Low power consumption for extended sessions

ImaP-RSCanon 5D Supported Image File Formats: RAW TIFF JPEGUFRaw Supported Image File Formats: PNG TIFF JPEGMatlab Imread( ) Supported Image File Formats: BMP JPEG PNG TIFF GIF Etc.

ImaP-RS

Format Selection Criteria: Non-compressed for intuitive pixel-

based processing Compatible with all platforms Open standard for GPS information

insertion later

ImaP-RS

Results: TIFF selected for image format FPGA to be used for any filter operations Power PC running Linux to be used for any

Matlab serial image functions (loading image, flattening, etc.)

USB interface to Canon EOS 5D Simulink to be used with Xilinx System

Generator (sysgen) for DSP

ImaP-RS

Dataflow Diagram – Hardware Standalone ML507 does not

support USB host mode Camera does not support USB

mass storage device mode

Digilent USB Daughterboard

CF CameraMemory

(kept empty)

Canon EOS 5D

ML507Power PC

LinuxXilinx custom kernel version

2.6.30

Device

USB

Host

8-bit Data Bus

ImaP-RS

Dataflow Diagram – Software Side

ML507Power PC

LinuxXilinx custom kernel version

2.6.30

GPhoto HD

CF ML507 Memory

RAM

Notify

Trigger

Writes C-Program

Imread( )Rgb2gray( )

ExecutionControl

Imwrite( )

FPGA

ImaP-RS

Rgb2gray( )

3D Truecolor Image Data Array

2D Grayscale Image Data Array

Grayscale Image Creation Steps:•Imread( ) parses file and creates a 3D matrix•Rgb2gray( ) flattens the image to 2D•Grayscale Intensity = 0.2989 * R + 0.5870 * G + 0.1140 * B

ImaP-RS

Simulink with Xilinx Sysgen for DSPbenefits: Built in simulink blocks for FIR filters,

FFTs, error correction, arithmetic, memory access (RAM, etc.) & digital logic

Automatic VHDL code generation for specific Xilinx IP cores

Hardware co-simulation

ImaP-RS

ImaP-RS

Xilinx FPGA boundaries in Sysgen

ImaP-RS

Problems Non-linux supported USB chipset on the

ML507 Issue with cross compiling libgphoto Xilinx Sysgen for DSP availability Lack of documentation Too much documentation Closed protocol on Canon Cameras

ImaP-RS

Future work We plan to continue this work

Finish working on getting the camera talking to the FPGA

Integrate image processing into the ML507 board

Fine tune settings

ImaP-RS

Thank you

Any Questions?

ImaP-RSReferences

Xilinx Application Note 1140http://www.xilinx.com/support/documentation/application_notes/xapp1140.pdf

XPS USB Host Controller Driverhttp://xilinx.wikidot.com/usb-host-controller-driver

Configuring, Building and Loading PowerPC Linuxhttp://xilinx.wikidot.com/powerpc-linux

gPhoto2http://gphoto.sourceforge.net/Matlab documentationhttp://www.mathworks.com/Xilinx System Generator Documentationhttp://www.xilinx.com/tools/sysgen.htmWikipedia Graphic format comparisonhttp://en.wikipedia.org/wiki/Comparison_of_graphics_file_formats