definition • design • development D3Engineering
Hyperspectral Camera Design Project
definition • design • development D3Engineering
Team Members
• Will Shaffer– Electrical Engineering
• Jeff Sidoni– Electrical Engineering
• Dan Scorse– Mechanical Engineering
• Jordan Gartenhaus– Mechanical Engineering
• Sponsored by:D3 Engineering222 Andrews StreetRochester, NY 14604
definition • design • development D3Engineering
Hyperspectral Overview
• Imaging system capable of capturing several tens to several hundreds of narrow, contiguous bands of the electromagnetic spectrum.
• Covers visible spectrum, extends from NIR to long wave IR.
• Data sets are large, but allow for the detection of subtle differences overlooked by traditional and multispectral systems.
definition • design • development D3Engineering
Spectral Reflectance
• Defined as the ratio of reflected energy to incident energy as a function of wavelength
• Materials have a unique spectral signature that is used for identification after a data set is acquired.
0 5 10 15-10
0
10
20
30
40
50
60
70
Wavelength λ (μm)
% R
efle
ctan
ce
AsphaltDry GrassGraniteGrassDark Soil
definition • design • development D3Engineering
Data Acquisition
• Hyperspectral images are obtained using an imaging spectrometer; a device used to measure various properties of light over the spectrum.
• Typically contains an optical system, a dispersing element, possibly in the form or a prism or grating and an array of detectors.
definition • design • development D3Engineering
Linear Array Spectrometer
• Uses a single row of detectors. Allows for the spectrum of a given point in a target to be acquired.
• Scanning mirror adds a dimension to capture an entire row of thetarget.
• Second dimension is typically accomplished by the movement of a plane or satellite.
definition • design • development D3Engineering
2D Sensor Method
• Using a two dimensional sensor, an entire row can be captured in a single frame.
• Scanning mirror adds second dimension.• In remote sensing applications where the camera is in a plane,
data sets can be captured without the need of a scanning mirror.
definition • design • development D3Engineering
Data Organization
• Hyperspectral data sets, by nature, are very large
• Usually consist of several hundred bands of the spectrum.
• Organized in a cube where two of the axes are spatial and one is spectral.
definition • design • development D3Engineering
Identification of Materials
• Unique spectral signature for materials
• Many libraries available containing hundreds or thousands of signatures for natural and man-made materials
• Research and evaluation of algorithms to improve classification and identification
definition • design • development D3Engineering
Applications
• Remote Sensing Applications– Military
• Terrain Mapping• Target Detection
– Agricultural• Crop Yield Prediction• Soil Quality Evaluation
– Mineral Exploration– Environmental Research
• Close Range Applications– Medical
• Early cancer detection
– Food Industry• Bacteria detection and
illness prevention
– Counterfeit Currency
definition • design • development D3Engineering
Defense Applications
• Used for target identification in combat areas• Target below is a camouflaged tank, hidden to traditional and
even multispectral imaging systems.
definition • design • development D3Engineering
Project Outline
• Purpose is to design and build a functional hyperspectral camera system for use by D3 Engineering’s design staff.
definition • design • development D3Engineering
Project Requirements
• Prototype a Hyperspectral Camera that– Uses a single CMOS or CCD imager– Captures spectra from ~400 to ~850nm (Visible Light)– Has spectral resolution of 25-50nm– Develop back end software to acquire and process a
hyperspectral data set and extract basic information such as material identification
– Utilizes hardware already developed and tested by D3 Engineering for the image acquisition and motor control for the scanning mirror
definition • design • development D3Engineering
System Block Diagram
Processing Board
Motion Board
Imager Board
Image Sensor DSPVideoData
USB
Communication
I2C Serial Communications
Motion Control
SDRAM
FLASH
5VDC
Mirror
Target
definition • design • development D3Engineering
Project TimelineID Task Name Duration Start Finish
1 Hyperspectral Camera Design 124 days Mon 12/5/05 Thu 5/25/062 Requirements Generation 2 w ks Mon 12/5/05 Fri 12/16/053 Design Documentation 2 w ks Mon 12/19/05 Fri 12/30/054 Initial Prototyping w / CDK 2 w ks Mon 12/19/05 Fri 12/30/055 Working Camera 2 w ks Mon 1/2/06 Fri 1/13/066 Preliminary Design Review 1 day Fri 2/24/06 Fri 2/24/067 CDK 61 days Mon 1/2/06 Mon 3/27/068 Camera Softw are Development 9.2 w ks Mon 1/2/06 Mon 3/6/069 1Mpixel 1 w k Tue 3/7/06 Mon 3/13/06
10 Frequency registration 2 w ks Tue 3/14/06 Mon 3/27/0611 Optical Subsystem 40 days Mon 2/27/06 Fri 4/21/0612 Use Spectrograph Optics 7 w ks Mon 2/27/06 Fri 4/14/0613 Optical Design 2 w ks Mon 2/27/06 Fri 3/10/0614 Lightpath Diagram 2 w ks Mon 3/13/06 Fri 3/24/0615 Simulation 2 w ks Mon 3/27/06 Fri 4/7/0616 Testing 2 w ks Mon 4/10/06 Fri 4/21/0617 Scanning Mirror 20 days Mon 2/27/06 Fri 3/24/0618 Linear stepper motor 4 w ks Mon 2/27/06 Fri 3/24/0619 Mechanical Structure 4 w ks Mon 2/27/06 Fri 3/24/0620 Software 45 days Mon 2/27/06 Fri 4/28/0621 Raw Data Dow nlaod 1 w k Mon 2/27/06 Fri 3/3/0622 Conversion to Matlab 2 w ks Mon 3/6/06 Fri 3/17/0623 Frequency Resolution 3 w ks Mon 3/20/06 Fri 4/7/0624 Create data Cube 3 w ks Mon 4/10/06 Fri 4/28/0625 Mechanical 50 days Mon 2/27/06 Fri 5/5/0626 Mirror 2 w ks Mon 2/27/06 Fri 3/10/0627 Enclosure 4 w ks Mon 3/13/06 Fri 4/7/0628 Optical Mounts 4 w ks Mon 4/10/06 Fri 5/5/0629 Intergration 20 days Mon 4/24/06 Fri 5/19/0630 Testing 3 w ks Mon 4/24/06 Fri 5/12/0631 Verify vs Requirements 1 w k Mon 5/15/06 Fri 5/19/0632 Final Report 1 w k Fri 5/19/06 Thu 5/25/06
1/2
4 11 18 25 1 8 15 22 29 5 12 19 26 5 12 19 26 2 9 16 23 30 7 14 21 28 4 11Dec '05 Jan '06 Feb '06 Mar '06 Apr '06 May '06 Jun '06
definition • design • development D3Engineering
Task Partitioning
• Four main design areas– Optics System
• Development of custom optics system• Evaluation of ImSpector Optics System
– Image System / Algorithm Development• Interfacing image capture to MATLAB• Algorithms to normalize capture data and identification
– Scanning Mirror Motor Control• Precise movement of scanning mirror
– Mechanical Components• Enclosure and mounting system for optics
definition • design • development D3Engineering
Optical Design
definition • design • development D3Engineering
Background
• As a part-time employee at ITT Space Systems Division basic knowledge in optics is benificial
• Took on optical design as a learning experience
definition • design • development D3Engineering
Requirements
• Initially minimal
• Achieve a wavelength of 400-700nm
• Resolution 25-50nm
definition • design • development D3Engineering
Initial Research
• Initial inquiries made at ITT about optical design
• Dr. Conrad Wells assisted with providing relevant books and guidance to understanding calculations
• Dr. Wells also provided initial design paths as possibilities
definition • design • development D3Engineering
Initial Optical Path
TAR
GE
T
MIR
RO
R
ENC
LOSU
RE
BA
FFLE
PR
ISM
LEN
S(3
) PL
ACE
S
definition • design • development D3Engineering
TAR
GE
T
MIR
RO
R
EN
CLO
SUR
E
BA
FFLE
PR
ISM
LEN
S(3
) PL
ACES
Alternate Path
definition • design • development D3Engineering
Relevant Work
• Upon looking into existing companies who make hyperspectral camera’s, Specim, a company in Finland was brought to attention by Scott Reardon
• On the website, the dispersion element, prism-grating-prism (PGP) was discussed
• Searching PGP turned up Mauri Aikio’s dissertation on hyperspectral imaging and his invention of the PGP
definition • design • development D3Engineering
Work Continued
• Calculations for optimization of focal length while optimizing a prism size were preformed.
• Focal length results were in the range of one meter which was unacceptable for making a compact design.
• Alternate dispersion elements looked at.
definition • design • development D3Engineering
lamda vs. h
-30
-20
-10
0
10
20
30
0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1
1.05
lamda (nm)
h (m
m)
Series1
Lambda vs. Height
definition • design • development D3Engineering
Wavelength vs. Optimized Refracted Angle
-0.03
-0.02
-0.01
0
0.01
0.02
0.03
0.4
0.45 0.5
0.55 0.6
0.65 0.7
0.75 0.8
0.85 0.9
0.95 1
1.05
Lamda (nm)
Thet
a_t2
-thet
a_no
t (de
g)
Series1
definition • design • development D3Engineering
Detector size Vs Input angle
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70
Incident Angle (deg)
Foca
l Pla
ne S
ize
(mm
)
Series1
definition • design • development D3Engineering
detector size vs. prism angle
0
5
10
15
20
25
30
35
40
45
50
0 10 20 30 40 50 60 70 80
prism angle (deg)
dete
ctor
siz
e (m
m)
Series1Series2Series3Series4Series5Series6Series7Series8Series9Series10Series11Series12Series13Series14Series15
definition • design • development D3Engineering
Work Continued
• Research on dispersion elements and spectrographs performed
• Result of research turned up prism-grating-prism (PGP) element
• Correspondences with Edmund Optics (E.O.): suggested a monochromator
• Additional searching turned up handheld spectrograph
definition • design • development D3Engineering
Dispersion Elements
• Two different dispersion elements considered
• Prism
• Blazed holographic grating
definition • design • development D3Engineering
Prism ApplicationsLittrow Prism
definition • design • development D3Engineering
Prism Apps. Cont.
definition • design • development D3Engineering
Prism Apps. Cont.
definition • design • development D3Engineering
Prism Test Sets
definition • design • development D3Engineering
Prism Test Sets Cont.
definition • design • development D3Engineering
Prism Test Sets Cont.
• This test set utilizes the Edmund Optics handheld spectrograph.
• The light source is an Hg pencil lamp.
• The video camera is recording the spectrum.
definition • design • development D3Engineering
Prism-Grating-Prism
• Design discovered by Mauri Aikio of Finland
• A mate of a prism, longpass filter, covering glass, grating, substrate glass, shortpass filter, and another prism
• Provides for a compact and straight thru design for spectrographs
definition • design • development D3Engineering
PGP Element
Prism 1LPF
Coveringglass
Grating
SPF
Prism 2
Substrate glass
definition • design • development D3Engineering
Handheld Spectrograph
• This unit is approximately 4” long and 7/8” in diameter.
• It utilizes a multiple prism alignment or mate similar to the PGP element in the ImSpector.
• Very easy to use and test.
• Can easily be adapted to an existing system.
definition • design • development D3Engineering
Handheld Spectrograph Prism Element Concept
definition • design • development D3Engineering
Monochromator
• E.O. suggested a monorchromator in place of ordering a holographic grating and slit separately.
• Using the monochromator E.O. suggested allows you to tune it to what wavelength you want to look at.
definition • design • development D3Engineering
Solution Guidelines
• Prism calculations yielded a large focal length
• D3 is attempting to demonstrate data acquisition
• Calibration of optics must be done by optical technician
definition • design • development D3Engineering
Optical Design Solution
• The Handheld spectroscope, monocromator, or ImSpector camera all stood out to be the best optical solutions to meet requirements
• An optical system will be created in parallel by D. Scorse utilizing a Littrow prism as the dispersion element.
definition • design • development D3Engineering
Imaging System
• Embedded System comprised of an imager and a DSP to capture and process frames
• Initial design will use USB to send acquired data to MATLAB for processing– Ease development of algorithms for normalization and
classification– Eventually move final processing inside the DSP
definition • design • development D3Engineering
Imaging Hardware
• Several tradeoffs while choosing hardware for this application– CMOS vs. CCD
• Pre-existing interface hardware to CMOS sensor• CCD has complex circuitry, but better dynamic range and bit
depth• Preliminary design will use CMOS, future revisions may
implement a CCD sensor– Texas Instruments DSP
• C6416 vs. DM642
definition • design • development D3Engineering
Image AcquisitionProcessing Board
Imager Board
1.3 MegaPixel10 Bit CMOS
Imager
DSP
C6416
VideoData
USB
I2C Serial Communications
SDRAM
FLASH
5VDC
Imager0
Optics
definition • design • development D3Engineering
Image Classification
• Initial development of algorithms done using MATLAB
• Several associated issues– Camera calibration– Reflectance conversion– Image normalization– Spectral Mixing Correction– Material Classification against reference libraries
definition • design • development D3Engineering
Scanning Mirror ControlStages of Development:• Generate PWM signals to rotate and stop mirror at 1° increments as a
baseline for data acquisition• Increase resolution to achieve a smooth steady rotation while varying
angular velocity to accommodate imager• Implement close-loop Kruse control to maximize resolution and provide
rotor position feedback• Incorporate additional interrupts to handle serial communications and
external velocity control
definition • design • development D3Engineering
Driver Schematic for Stepper Motor Phase A
100
10k
100
10k
10k
100
10k
100
47k
47k
1k
1k .033 .033
IRFZ44E
IRFZ44EIRFZ44E
IRFZ44E
1uF
IN2
SHDN3LO
5
HO7
COM4
VB8
VCC1
VS6
IR21040
PWM_A1+
820nF
ZMM5251B100 uF
1uF
0
0
0
0
0
Vdd Vdd
31
2-
+
MtrDA1- MtrDA1+
820nF
Vmm
IN2
SHDN3 LO
5
HO7
COM4
VB8
VCC1
VS6
IR2104
V3_3/2
ViA1
Motor Current Sense
Half - Bridge DriverHalf - Bridge Driver
0
0
PWM_A1-
definition • design • development D3Engineering
PWM Generation
• The top waveform is generated by selecting a base PWM carrier frequency
•The bottom waveform is the modulated signal produced through comparison with the percentage values stored in a table
•Thus, the angular velocity of the rotor can be controlled by slowing down the transition through the look-up table
definition • design • development D3Engineering
PWM Generation
• Motor windings are modeled as a series LC circuit; and thus a low-pass filter
• Each PWM signal drives one half-bridge driver – need four signals per motor
• Other signals are generated identically except for either inversion or 90° phase shift
Effective Driver Signal
definition • design • development D3Engineering
Mechanics
• Mirror is required to rotate +/- 20° relative to the center of the target image
• The distance between the mirror and the target is remains variable to allow for optimized data acquisition
definition • design • development D3Engineering
Slit Width Analysis
tan(20 )y d= ⋅ °
Target Width 2y=
1 tan( )y dΔ = ⋅ ΔΘ 2 1tan(2 )y d yΔ = ⋅ ΔΘ − Δ 3 2 1tan(3 )y d y yΔ = ⋅ ΔΘ − Δ − Δ
( )1
1
tan( ) 1i
i jj
y d i y i−
=
Δ = ⋅ ΔΘ − Δ ∀ >∑
( ) ( )1 2 2
1 1 1tan( ) tan( ) tan ( 1)
i i i
i j j jj j j
y d i y d i d i y y− − −
= = =
⎛ ⎞Δ = ⋅ ΔΘ − Δ = ⋅ ΔΘ − − ΔΘ − Δ − Δ⎜ ⎟
⎝ ⎠∑ ∑ ∑
( ) ( )tan tan ( 1)iy d i d iΔ = ⋅ ΔΘ − ⋅ − ΔΘ
definition • design • development D3Engineering
d [m] y [m] μStep Size Δθ
# Slits/ Half Image
Avg. Δy [mm] Min Δy [mm] Max Δy [mm]
0.0625 0.225 88 4.136 3.927 4.4300.03125 0.1125 177 2.056 1.963 2.2200.015625 0.05625 355 1.025 0.982 1.111
0.0078125 0.028125 711 0.512 0.491 0.5560.00390625 0.0140625 1422 0.256 0.245 0.278
0.0625 0.225 88 8.272 7.854 8.8600.03125 0.1125 177 4.113 3.927 4.4390.015625 0.05625 355 2.051 1.963 2.222
0.0078125 0.028125 711 1.024 0.982 1.1120.00390625 0.0140625 1422 0.512 0.491 0.556
0.0625 0.225 88 12.408 11.781 13.2890.03125 0.1125 177 6.169 5.890 6.6590.015625 0.05625 355 3.076 2.945 3.333
0.0078125 0.028125 711 1.536 1.473 1.6670.00390625 0.0140625 1422 0.768 0.736 0.834
0.0625 0.225 88 16.544 15.708 17.7190.03125 0.1125 177 8.225 7.854 8.8780.015625 0.05625 355 4.101 3.927 4.444
0.0078125 0.028125 711 2.048 1.963 2.2230.00390625 0.0140625 1422 1.024 0.982 1.112
0.0625 0.225 88 20.680 19.635 22.1490.03125 0.1125 177 10.282 9.817 11.0980.015625 0.05625 355 5.126 4.909 5.555
0.0078125 0.028125 711 2.560 2.454 2.7790.00390625 0.0140625 1422 1.280 1.227 1.390
1.092
5 1.820
4 1.456
3
3.6 ° per Full Step:
0.3641
2 0.728
Comparison of Open-Loop Control Variables
definition • design • development D3Engineering
Close-Loop Kruse Control
• Technique for accurately calculating the rotor angle in real-time and without the use of an encoder
• Achieved by feeding back emf from sense windings and integrating to produce voltage signals proportional to the rotor’s displacement angle
• Allows for 40,000-60,000 micro-steps per revolution
definition • design • development D3Engineering
d [m] y [m] μSteps / Full
Rev. Δθ# Slits/ Half
Image Avg. Δy [mm] Min Δy [mm]Max Δy [mm]
40000 0.00900 2222 0.163802986 0.1570796 0.177874045000 0.00800 2500 0.145588094 0.1396263 0.158115250000 0.00720 2777 0.131065983 0.1256637 0.142294355000 0.00655 3055 0.119139193 0.1142397 0.129362260000 0.00600 3333 0.109201990 0.1047198 0.118584940000 0.00900 2222 0.327605971 0.3141593 0.355747945000 0.00800 2500 0.291176187 0.2792527 0.316230450000 0.00720 2777 0.262131966 0.2513274 0.284588655000 0.00655 3055 0.238278386 0.2284795 0.258724460000 0.00600 3333 0.218403981 0.2094395 0.237169840000 0.00900 2222 0.491408957 0.4712389 0.533621945000 0.00800 2500 0.436764281 0.4188790 0.474345650000 0.00720 2777 0.393197948 0.3769911 0.426882955000 0.00655 3055 0.357417579 0.3427192 0.388086660000 0.00600 3333 0.327605971 0.3141593 0.355754740000 0.00900 2222 0.655211943 0.6283185 0.711495945000 0.00800 2500 0.582352375 0.5585054 0.632460850000 0.00720 2777 0.524263931 0.5026548 0.569177255000 0.00655 3055 0.476556772 0.4569589 0.517448860000 0.00600 3333 0.436807962 0.4188790 0.474339640000 0.00900 2222 0.819014929 0.7853982 0.889369845000 0.00800 2500 0.727940469 0.6981317 0.790576150000 0.00720 2777 0.655329914 0.6283185 0.711471555000 0.00655 3055 0.595695964 0.5711987 0.646811160000 0.00600 3333 0.546009952 0.5235988 0.5929245
Kruse Control
1 0.364
2 0.728
5 1.820
3 1.092
4 1.456
Slit Width Analysis for Kruse Controlled System
definition • design • development D3Engineering
Mechanical Components
Main components:
– Project enclosure
– Scanning mirror fixture
– Optical mounts
definition • design • development D3Engineering
ImSpector Imager Mount
• Threaded for standard C-mount fixture• Tapped for mounting of an imager adapter plate
definition • design • development D3Engineering
Hyperspectral Enclosure
• Optical breadboard based• Light-weight lift-off aluminum box top• Draw latches for easy open/close
definition • design • development D3Engineering
Scanning Mirror Fixture
• Machined mounting plate• Attached to stepper motor shaft• Adjustability of mirror height and angle• Set screw allows for easy manipulation
definition • design • development D3Engineering
Lenses
• Commercially available mounting hardware• Modified to suit requirements