SAND REPORT SAND2002-3846 Unlimited Release Printed February 2003 High Speed 2D Hadamard Transform Spectral Imager Joseph Wehlburg, Christine Wehlburg, Craig Boney, Jody Smith, Olga Blum Spahn, Stephen Gentry and Mark Smith Prepared by Sandia National Laboratories Albuquerque, New Mexico 87185 and Livermore, California 94550 Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under Contract DE-AC04-94AL85000. Approved for public release; further dissemination unlimited.
Transcript
SAND REPORT
SAND2002-3846 Unlimited Release Printed February 2003 High Speed 2D Hadamard Transform Spectral Imager
Joseph Wehlburg, Christine Wehlburg, Craig Boney, Jody Smith, Olga Blum Spahn, Stephen Gentry and Mark Smith
Prepared by Sandia National Laboratories Albuquerque, New Mexico 87185 and Livermore, California 94550 Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under Contract DE-AC04-94AL85000. Approved for public release; further dissemination unlimited.
2
Issued by Sandia National Laboratories, operated for the United States Department of Energy by Sandia Corporation.
NOTICE: This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government, nor any agency thereof, nor any of their employees, nor any of their contractors, subcontractors, or their employees, make any warranty, express or implied, or assume any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represent that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government, any agency thereof, or any of their contractors or subcontractors. The views and opinions expressed herein do not necessarily state or reflect those of the United States Government, any agency thereof, or any of their contractors. Printed in the United States of America. This report has been reproduced directly from the best available copy. Available to DOE and DOE contractors from
U.S. Department of Energy Office of Scientific and Technical Information P.O. Box 62 Oak Ridge, TN 37831 Telephone: (865)576-8401 Facsimile: (865)576-5728 E-Mail: [email protected] Online ordering: http://www.doe.gov/bridge
Available to the public from
U.S. Department of Commerce National Technical Information Service 5285 Port Royal Rd Springfield, VA 22161 Telephone: (800)553-6847 Facsimile: (703)605-6900 E-Mail: [email protected] Online order: http://www.ntis.gov/help/ordermethods.asp?loc=7-4-0#online
3
SAND2002-3846 Unlimited Release
Printed February 2003
High Speed 2D Hadamard Transform Spectral Imager
Joseph C. Wehlburg, Jody L. Smith, Mark W. Smith, Stephen Gentry and Craig M. Boney
Remote Sensing & Data Exploitation Department
Christine M. Wehlburg Information Detection, Extraction, and Analysis Department
Olga Blum Spahn
RF Microsystems Technologies Department
Sandia National Laboratories P.O. Box 5800-0972
Albuquerque, NM 87185-0972
Abstract Hadamard Transform Spectrometer (HTS) approaches share the multiplexing advantages found in Fourier transform spectrometers. Interest in Hadamard systems has been limited due to data storage/computational limitations and the inability to perform accurate high order masking in a reasonable amount of time. Advances in digital micro-mirror array (DMA) technology have opened the door to implementing an HTS for a variety of applications including fluorescent microscope imaging and Raman imaging. A Hadamard transform spectral imager (HTSI) for remote sensing offers a variety of unique capabilities in one package such as variable spectral and temporal resolution, no moving parts (other than the micro-mirrors) and vibration tolerance. Two approaches to for 2D HTS systems have been investigated in this LDRD. The first approach involves dispersing the incident light, encoding the dispersed light then recombining the light. This method is referred to as spectral encoding. The other method encodes the incident light then disperses the encoded light. The second technique is called spatial encoding. After creating optical designs for both methods the spatial encoding method was selected as the method that would be implemented because the optical design was less costly to implement.
4
Contents Abstract ........................................................................................................................................... 3 Contents .......................................................................................................................................... 4 Figures............................................................................................................................................. 5 Tables.............................................................................................................................................. 6 Introduction..................................................................................................................................... 7 Simulations ..................................................................................................................................... 9 Optical Design .............................................................................................................................. 10 TI DMD Experiments ................................................................................................................ 18 SNL Micro-mirror Array Experiments ......................................................................................... 27 Experimental Setup for the HTSI ................................................................................................. 36 Conclusions................................................................................................................................... 37 References..................................................................................................................................... 39 Appendix A “Optimization and Characterization of an Imaging Hadamard Spectrometer”........ 40 Appendix B “Theoretical description and numerical simulations of a simplified Hadamard transform imaging spectrometer” ................................................................................................. 51 Appendix C (Optical Design Report) ........................................................................................... 61 Appendix D (Patent application) ................................................................................................ 105 Distribution: ................................................................................................................................ 118
5
Figures Figure 1. Offner relay design. 10 Figure 2. Image area shown (green) inscribed on DMD. 11 Figure 3. Offner Relay HTS design with focal plane windows. 11 Figure 4. Spot diagram. System is near diffraction limited. 12 Figure 5. Optical path difference plots. 12 Figure 6. Footprint diagram showing dispersed image at the DMD. 13 Figure 7. MTF plot. 13 Figure 8. Spot diagram for 1.2 micron light. 14 Figure 9. Spot diagram for 2.4 micron light. 14 Figure 10. Alternate design for HTS with two holographic correctors prior to focal plane. 16 Figure 11. Rejection Ratio for TI DMD. 20 Figure 12. Rejection Ratio for 400nm-1000nm. 21 Figure 13. Rejection Ratio without realignment for optimal signal. 22 Figure 14. Optical Setup 1 for DMD tests. 24 Figure 15. Optical Setup 2 for DMD tests. 25 Figure 16. CAD layout of RS298, SUMMITV die under investigation. Mirror array of interest has a red box around it. 27 Figure 17. Schematic representation of the polysilicon layers constituting the compound pivot micro mirror. 27 Figure 18. Schematic representation of of SNL MEMS. 28 Figure 19. Schematic representation of the experimental set up. 29 Figure 20. Data representing one cycle of tilt angle (left y-axis, black curve) and applied bias to the energized pad (right y-axis, red curve) as a function of time for a 50 µm x 50 µm mirror. 30 Figure 21. Trend in the mean values of the maximum and minimum tilt angle values for a 50 µm x 50 µm micro mirror (pads 77, 78) over a 24 hr period. 31 Figure 22. Initial behavior of the mean values and standard deviations of the maximum and minimum tilt angle during the first 30 minutes of operation for a 50 µm x 50 µm micro mirror (pads 77, 78). 32 Figure 23. Trend in the mean values of the maximum and minimum tilt angle values for a 50 µm x 50 µm micro mirror (pads 75, 76) over a 24 hr period. 33 Figure 24. Initial behavior of the mean values and standard deviations of the maximum and minimum tilt angle during the first 60 minutes of operation for a 50 µm x 50 µm micro mirror (pads 75, 76). 33 Figure 25. Trend in the mean values of the maximum and minimum tilt angle values for a 50 µm x 50 µm micro mirror (pads 73, 74) over a 17 hr period. 34 Figure 26. Initial behavior of the mean values and standard deviations of the maximum and minimum tilt angle during the first 30 minutes of operation for a 50 µm x 50 µm micro mirror (pads 73, 74). 34
6
Tables Table 1. Design constraints and mitigation strategies. 16 Table 2. Results from the SNL MEMS micro-mirror tests. 35
7
Introduction An important consideration in developing a remote sensing instrument is signal to noise. Two available choices in spectroscopy that incorporate the multiplexing advantage to increase signal to noise are Fourier and Hadamard transform instruments1. The development of the Michelson interferometer and advanced computing resources have contributed to the availability of commercial one-dimensional, two-dimensional imaging and hyperspectral imaging Fourier transform spectrometers2,3. Concurrent advances in Hadamard transform techniques were hampered by limitations in encoding mask technology. Recent developments in digital micro-mirror array (DMA) technology and the availability of two-dimensional detector focal plane arrays charge have motivated our group to build a Hadamard transform spectral imager (HTSI) for remote sensing. An HTSI would have the same multiplexing advantages as a Fourier transform system, however, there are additional benefits that are uniquely beneficial to remote sensing. The HTSI would have no moving parts, be insensitive to vibrations and capable of incorporating variable spectral and temporal resolution. The development of Hadamard transform spectrometers as a viable multiplexing alternative to Fourier transform techniques has progressed in the past decade with improvements in encoding mask technology. Liquid crystal spatial light modulators (LC-SLM’s) were utilized by the Hammaker-Fateley group as stationery Hadamard encoding masks to replace moving encoding masks4,5. At the time, moving encoding masks were limited by their ability to reproducibly position the masking patterns6-8. A major limitation of an LC-SLM is the non-ideal transmission and rejection. Hadamard encoding masks, in a traditional cyclic S-matrix configuration, require a 1 or 0 condition. In the 1 condition, the transmission (T1) is ideally 100%. For the 0 condition, the transmission (T0) is ideally 0% (100% rejection)9. Hammaker and Fateley’s Raman imaging system was developed with an LC-SLM that was also small in size (10 X 10 pixels). Recent work by Jovin and coworkers using an 800x600 pixel format liquid LC-SLM as the encoding mask for Hadamard transform fluorescence microscopy imaging reported the acquisition of two-dimensional spectral images despite less than ideal T1 and T0 behavior10. Improvements in translation device technology resurrected the use of moving encoding masks, however, they remain relatively slow in the encoding sequence and both the mask dimensions and element size are fixed11,12. More recent advances in DMA technology and the commercial availability of the Texas Instruments digital micro-mirror device (DMD) have enabled researchers to consider micro-mirror arrays as an alternative encoding mask over LC-SLM’s. The Hammaker-Fateley group, building on their pioneering work in Hadamard spectroscopy‡, utilizes the DMA as an SLM for generating a stationary Hadamard encoding mask. With the DMA in a dispersive flat-field near-infrared spectrometer that could be operated in both conventional raster scanning (CRS) and Hadamard transform spectrometer modes, DeVerse, Hammaker and Fateley were able to experimentally document the Hadamard multiplex advantage. The DMA in the flat-field
‡ An exhaustive list of citations documenting the Hadamard transform spectroscopy effort at Kansas State
University until 1997 can be found in Ref. 11.
8
spectrometer was utilized as a 1D Hadamard mask for spectral encoding13. Due to the cost-prohibitive nature of 2D multi-channel detectors for future expected work in the near-infrared, the current Raman imaging systems designed by DeVerse and coworkers employs a single element detector for imaging in the visible spectral region. A 1D Hadamard cyclic S-matrix encoding mask (spectral encoding) is folded into a 2D Hadamard encoding mask and together with sample rastering a single element detector can be used for Hadamard transform imaging14,15. Hanley, Verveer and Jovin, in addition to developing a Hadamard transform fluorescence microscopy imager with an LC-SLM, have reported data for fluorescence imaging with a DMA, as well16. Some of the differences between the Raman imaging system (DeVerse and coworkers) and the fluorescence microscope imaging system (Hanley and coworkers) include the use of a 2D Hadamard cyclic S-matrix for spatial (not spectral) encoding and a 2D charge couple device (CCD) detector operating in the visible region. The initial focus of this LDRD was to develop a 1.2 µm – 2.4 µm wavelength HTSI and to evaluate the functionality of this form of Hyper-spectral Imaging. Further investigations led to changes in the wavelength, to the 450nm-800 nm range, do to cost and time constraints. As the project developed a new method of encoding the information was invented, now referred to as spatial encoding. After simulating the new method it was decided to switch from the original concept, called spectral encoding, to the new spatial encoding so the LDRD worked on completed the optical design and procuring the components needed for the spatial encoding HTSI.
9
Simulations A simulation for the spectral encoding method of the HTSI was developed at the start of the LDRD to allow different engineering options to be evaluated without the cost of actually purchasing hardware. As the simulation progressed a paper detailing this version of the simulation was writen and accepted for publication and is included in Appendix A. When the concept spatial encoding was developed a simulation was created to test the idea. A paper detailing the spatial encoding concept and the simulation for it was also published and is included in Appendix B.
10
Optical Design The initial design requirements for the system were as follows: Mechanical:
Weight: Less than 15 lbs Optical Envelope: 864 in2 (6”x12”x12”) Optical: Spectral Coverage: 1.2 – 2.4 microns Spectral Resolution: 40 nm Image format: 256 x 256 30 micron pixels MTF: .1 at 70 lines/mm, 04 at 30 lines/mm There were also design constraints controlled by existing equipment. Encoding constraints:
Existing Texas Instruments Digital Micro-mirror Device 768x1024 pixels Pixel size of 17 microns Total DMD usable space of 17x13mm Rotation of individual mirror elements at 45 degrees Usable mirror positions at +/- 10 degrees Imaging constraints: Existing 256x256 IR detector with 30 micron pixel size Usable detector area of 7.68x7.68mm
The initial design idea for the Hadamard instrument was a double Offner relay as shown in figure 1. The all-reflective system provides good imaging for the full desired wavelength range. This design uses the symmetry advantage of the Offner relay, but is limited by the Offner relay small field and one to one imaging.
The Digital Micro-mirror Device[3] rotates on the diagonal axis of the square “pixels”. This, in conjunction with the dispersive element [1] introduced large distortion into the system. This distortion can not be mitigated with the downstream optics. The solution to this problem is to rotate the DMD 45 degrees to provide a single plane for the optical axis. This in turn limits the spatial extent of the DMD that is usable. Since the dispersed image will represent a rectangle on the DMD, the maximum useable area for the image is reduced to provide adequate spectral resolution. Figure 2 shows the useable area of the DMD inscribed on the 768x1024 DMD array.
Figure 1. Offner relay design.
1 2
3
4
5 6
11
This arrangement mitigates the distortion issue, however, the reduction in image extent severely affects the required grating frequency for the holographic dispersive element. The mapping from the DMD pixels to the focal plane pixels is also affected, since the mapping is now a diamond shaped DMD pixel onto a square FPA pixel. Modeling this configuration, we concluded that this would essentially introduce noise into the final image. The design shown in Figures 3 thru 9 illustrate the system design and performance with the DMD rotated 45 degrees. This arrangement results in an image height at the DMD of 3.5mm.
Figure 3. Offner Relay HTS design with focal plane windows.
Figure 2. Image area shown (green) inscribed on DMD.
DMD area
12
Figure 4. Spot diagram. System is near diffraction limited.
Figure 5. Optical path difference plots.
13
Figure 6. Footprint diagram showing dispersed image at the DMD.
Figure 7. MTF plot.
14
Figure 8. Spot diagram for 1.2 micron light.
Figure 9. Spot diagram for 2.4 micron light.
15
The Schleimphluge condition whereby the image plane is tilted to compensate for the tilt introduced by the DMD requires a large aperture at the focal plane for the ray bundle to pass through the window and cold filter to the focal plane pixels. Our camera design, as purchased had an entrance aperture at the cold filter of 13.3858mm. This coupled with the distance from the cold stop to the actual focal plane array of 26.59mm was not sufficient for this design. Several options were considered, however the vendor could not provide us with any assurances of the camera operation after modifications at their facility. In parallel with the work on the camera issues, various vendors were contacted to manufacture the two convex holographic elements. The fact that these elements were convex was a considerable hurdle however there are a few manufacturers that have this capability. Because of the optical path difference introduced by the DMD, the recombination element [4] is required to be a holographic optical element. Due to the two dimensional nature of this instrument, the groove frequency required is approximately 7 lines/mm. This is extremely low groove frequency for the holographic process, which puts the construction points for the laser source only approximately 3 mm apart. Additionally, the element is convex which further complicates the manufacturing process. The Hyperion sensor has a convex grating in the SWIR bands that was produced by Carl Ziess. We contacted Ziess with our specifications and began discussions. Since the construction points for this element are virtual, ie behind the grating substrate, this element cannot be written directly. The solution to this problem was developed in conjunction with vendor discussions. It is possible to create a master on a concave surface and then replicate the convex element using that master. The lead time for these elements was approximately 14 months, which was past the end of the project. We investigated alternatives to the convex diffraction grating. These included flat gratings and prism systems. The limitation of the flat, ruled grating is that recombination of the image will not include any aberration correction. The limitation of the prism system is the unavailability of glass that transmits over the desired spectral range. One grating method that was investigated was a convex ruled grating, which could be manufactured from a concave master. The ruled grating manufacturing process lends itself to lower grating frequencies. To compensate for the aberration induced by the spherical elements and the DMD, two transmissive holographic corrector elements were placed prior to the camera. This arrangement provided adequate imaging, but the signal at the detector was estimated at approximately 1.5% of the input signal based simply on the transmission of the diffractive and holographic elements. This would require input signals so large that the DMD would potentially sustain damage.
16
Figure 20. Alternate design for HTS with two holographic correctors prior to focal plane.
In light of the technical and financial constraints, alternatives were considered that would loosen the original constraints to allow more degrees of freedom in the design. Critical constraints and their ramifications are summarized in table 1. Table 2 Design constraints and mitigation strategies
Constraint Problem associated with this constraint Potential Mitigation methods
Camera pixel size of 30 microns Pixel size not an integer number of the DMD Pixels. Causes noise.
1) Replace camera 2) Develop design with appropriate magnification
Current Camera has a 3-5 micron cold filter
No transmission in the wavelength region of interest.
1) Modify camera 2) Replace camera
Camera ray bundle small acceptance angle
Schleimphluge condition exists in all designs with the DMD. Image will be completely vignetted in existing camera.
1) Modify camera 2) Replace camera
Recombination of spectra requires holographic elements for aberration correction.
Original design (convex HOE) lead time too long. Alternate design requires very high input signal which poses risk to DMD.
1) Encode spatial dimension
Wavelength region 1.2-2.4 microns
Requires reflective system due to glass transmission ranges.
1) Narrow wavelength region
Purchase of a new camera system with specifications tailored to our design was deemed the appropriate course of action. This removed three major constraints from the system, but still left
17
the problems associated with the holographic elements and potential risk to the DMD by using a high input signal. An alternate encoding and processing scheme which does not require recombination of the spectrally spread image was developed in the project. This method encodes the spatial dimension of the image prior to dispersion. Full design requirements and final design are included in the design report from Optical Research Associates which is included in Appendix C.
18
TI DMD Experiments The DMD video board was installed in a computer based upon a Pentium 90 MHz microprocessor (vintage ~1995). The video driver settings were adjusted to 640 x 480 resolution with a monochrome display (standard VGA driver). In this configuration, the DMD displayed the same output as the computer’s monitor. This display was affected by changes made in the control software and by changes in the contents of the video signal itself. The board containing the DMD was mounted on a multi-axis positioning stage for optimizing the signal. An application was written (named Window_ver1.vi in LabVIEW 4.1) that converted most of the pixels in the display to one 8-bit gray level. This was used to control the positions of the pixels in the DMD. With the pixels in the controlled display window set to 255, the DMD was considered in the “On” position since most of the light was reflected to the +10° position. With the pixels set to level 0, it was considered in the “Off” position since most of the light was reflected to the -10° position. As previously mentioned, all of the presented data were taken with the detector in the +10° position. The Oriel MS257 monochromator and a 250W QTH lamp comprised the source of irradiance on the DMD. The monochromator included a chopper, two slits, and an order-sorting filter wheel. The image of the output slit of the monochromator was focused upon the DMD using an uncoated bi-convex lens at a speed of f/3.5 (see Appendix 1). An aperture was placed between the chopper and the input slit to reduce flaring of the image. Since these tests employed the use of uncoated fused silica and MgF coated achromat lenses, two tests were performed to account for variation in image magnification at the detector over the wavelength range. In the first test, the slits were adjusted to an input = 1 mm and an output = 2 mm. These settings reduced flaring and maximized the signal throughput providing an image that under-filled the active area in use on the DMD and overfilled the detector. In the second test, the slits were changed to an input = 500µ and an output = 1200µ. In addition, a φ=1mm pinhole was inserted at the DMD position in the previous set-up to provide a clear image and under-fill both the DMD active area and the detector. Additional optics used in the latter case are described and shown in Figure 5. In both set-ups an f/2.0 achromat was positioned to collect the signal reflected from the +10° position of the DMD and to focus an approximate 1x magnification image onto the surface of a detector element. The DMD had been placed in the Run mode and had been reset. Orientation at an azimuth 45° to the plane vertical to the surface of the table effectively placed the tilt axes of the DMD pixels such that the incident light and both the +10° and -10° reflections off the DMD were along a plane parallel to the surface of the table. The +10° reflected beam was centered, by eye, in the receiving lens and aligned to overfill the detector element. With this arrangement, the FOV seen by the detector remained constant. Two detectors were used to measure the DMD output. A silicon detector operating in the photo-voltaic mode was used to measure the wavelengths from 500nm to 1000nm and a lead sulfide detector (biased with 50V DC) in the photo-conductive mode was used to measure wavelengths
19
from 1000nm to 2500nm. In each case the output signal from the detector was input to a preamplifier and then to a lock-in amplifier. The lock-in was synchronized to the chopper frequency to eliminate background light. The RMS output of the lock-in amp was accepted as the output signal. For each test, a measurement was made every 100nm of both the “On” and “Off” signal. The ratios of these values were calculated and plotted with respect to wavelength. The gain of the preamplifier, the position of the DMD array, and the alignment of the collection optic/detector was adjusted to maximize the “On” signal prior to measurement. Summary: The Texas Instruments DMD was designed to work in the multimedia industry and, as expected, has the highest contrast ratios in the visible wavelength spectrum. Literature on the TI DMD mentions contrast ratios of 2500:1 from the visible to the IR at incident angles of 20° to “Park” position Normal. Intrigued by that figure, a quick experiment at 500nm and 600nm was done to see what contrast would be obtained with a “rough” alignment at that publicized angle of incidence. As one might expect, the contrast measured at 1350 at 500nm and 1617 at 600nm which are significant increases over the “rough” alignment contrast ratio of ~650 when the DMD is used at the incident angle in our remaining tests. With the incident angle along the “Park” position Normal, contrast ratios of better than 650 can be achieved throughout the visible region but the mean ratio drops by at least a factor of 2 above 1000nm. Peaks do occur at ~1100nm, ~1400nm and ~2200nm with contrast ratios from 500 to 600 but the majority of wavelengths from 1000nm to 2500nm have contrast ratios from 150 to 400. Alignment of the optical system is the key to high contrast ratios. Tests in which the radial angle of the DMD was varied relative to the incident light showed very little effect in the overall contrast with proper collection lens/detector alignment. Several measurements were made in which the contrast ratio reached 1300 at 500nm (even with our preferred incidence angle) but those results were practically impossible to repeat. An easier target of ~750 at 500nm is more typical. This alignment issue would complicate the use of the DMD should it be used across the entire range tested. One test was performed to provide a worst case scenario should the operation be optimized in the visible and used out to 2500nm(see Figure 3). As you can see, the IR contrast ratios suffer significant loss if the alignment isn’t adjusted for maximum “On” signal. Additional tests to perform would include: 1) measuring the contrast ratio over wavelength from an under-filled DMD pixel; 2) measuring the detector output as an ac signal synchronized to the 1kHz DMD “flicker”; 3) checking for polarization affects; 4) determining the affect of timing differences between the chopped incident light and the DMD “flicker”.
20
DATA
Legend Notes:
1. Series 1 is detector overfilled from 500nm to 2500nm. 2. Series 2 is detector overfilled from 2500nm to 500nm. 3. Series 3 is detector under-filled from 1000nm to 2500nm. 4. Series 4 is detector under-filled from 2500nm to 1000nm.
Figure 11. Rejection Ratio for TI DMD.
Figure 11 shows the complete wavelength range from 500nm to 2500nm. Notice that the DMD performance, particularly in the IR, maintains a high degree of repeatability in both the overfilled and the under-filled conditions. Therefore, the focusing effect (which actually becomes a magnification issue at the detector) has a negligible affect on the overall measured efficiency when the system is properly aligned.
TI DMD Contrast Ratio Optimizing "On" Signal by Adjusting Detector, DMD, and Collection Lens
w/o Adjusting DMD Radial Angle
0.00
200.00
400.00
600.00
800.00
1000.00
500 1000 1500 2000 2500 3000
Wavelength (nm)
Series1Series2Series3Series4
21
Legend Notes:
1. Series 1 is detector over-filled from 500nm to 1000nm. 2. Series 2 is detector over-filled from 1000nm to 500nm. 3. Series 3 is detector under-filled from 500nm to 1000nm. 4. Series 4 is detector under-filled from 1000nm to 500nm.
Figure 12. Rejection Ratio for 400nm-1000nm.
Although the DMD contrast ratio curves in the visible have similar shape, the values are noticeably smaller in the under-fill condition, as seen in Figure 12. Apparently some alignment or other optical affect is reducing the contrast by 50%. Repeatability in the visible spectrum in
TI DMD Contrast Ratio Variation Observed in the 500 to 1000nm Range - No Change in Radial
an ascending/descending test is not as good as for 1000nm to 2500nm. Perhaps there is some detector inconsistencies, but more likely is the alignment issue. The trick is in returning to the original spot. The IR wavelengths seem to have less dependence on DMD position for good contrast ratios.
Figure 13. Rejection Ratio without realignment for optimal signal.
Figure 13 shows the contrast ratio obtained by aligning the system at 500nm and then simply changing wavelength. The detector was in the overfilled condition. Notice the overall form of the curve is the same as Figure 11 and that improved alignment (particularly tip/tilt and focal position of the DMD) can yield as much as a factor of 2 improvement in the contrast ratio in the IR.
Micromirror wavelength sensitivity 004 - aligned only at 500 nm.
0
100
200
300
400
500
600
700
800
200 600 1000 1400 1800 2200 2600
Wavelength (nm)
Rat
io (O
n/O
ff)
Ratio (On/Off) PbS
23
Optical Calculations: Set-up 1: Given:
Monochromator has f/# of 3.9 Micromirror wants input light of f/# of 3.5 Object to image distance of 300mm
Formulas: 1/S + 1/S’ = 1/ fL S + S’ = 300mm (arbitrarily chosen) S/S’ = 3.9/3.5 therefore, S’ = 3.5S/3.9 Then, S + 3.5S/3.9 = 300mm S(1 + 3.5/3.9) = 300mm S = 300mm/(1 + 3.5/3.9) S = 300mm/1.8974 S ≅ 158mm And S’ = 3.5/3.9 * 158mm S’ ≅ 142mm Finally, 1/ fL = 1/158mm + 1/142mm fL ≅ 75mm (we used a Newport bi-convex UV Fused Silica lens Part # SBX 052) Since f/# = S/D =S’/D D = 158mm/3.9 = 40.5mm (our lens is a 50.8mm diameter) Or for the array, D = 142mm/3.5 = 40.5mm The array 1/2∠ ≅ arctan 20.25mm/142mm or 8.1°
24
For size and positioning reasons a 25.4mm diameter f/2 lens of fL = 50.8mm was chosen to collect and focus the light when the array was in the +10° position (relative to the “park” position). To get 1:1 image the f/# would be 3.5 for object and image giving S = S’. Using a fL = 50.8mm lens S = S’ = 101.6mm D = 101.6mm/3.5 = 29mm Set-up 1: NOTE: An aperture was inserted between the QTH and the input slit to the monochromator to yield a vertical height slightly larger than the vertical size of the detectors and to reduce image flaring. This effectively shrunk the size of the beam at the lenses (particularly the 1” diameter one) so that the beam would be small enough to mostly fill the clear aperture of the 1” lens. Optical Layout: Detector overfilled with reflected beam
Chopper/lightsource
Figure 14. Optical Setup 1 for DMD tests..
25
Set-up 2: Under-fill Detector with reflected beam: Optics change after the 75mm bi-convex lens. The addition of a ∅=1mm pinhole inserted at the location of the DMD in set-up 1 is employed to reduce the image size at the detector. After optimizing the image a fL = 200mm lens is inserted into the beam to collimate the image. A fL = 63.5mm lens is used to focus the image on the DMD. The f# is 63.5mm/19mm (clear aperture for 1” lens holder) is 3.34. The same fL = 50.8mm is used to collect and focus the image off the DMD onto the detector. 7
1mm pinhole
FL = 75.8mm
monochromator
Light source
FL = 200mm
FL = 63.5mm
DMD
Si or Pb
FL = 50.8mm
Light dump
Figure 15. Optical Setup 2 for DMD tests.
26
Equipment List
1ea Oriel Instruments Model MS257 Monochromator having gratings covering wavelengths from 250 – 1300nm and 1100 – 5000nm. Long pass filters at open, 450nm, 850nmm, and 1700nm will be used. Entrance slit set for 1000µ; Exit slit set for 2000µ 1ea Oriel Instruments chopper Model 75152 1ea Oriel Instruments monochromator illuminator Model 7340 with lamp Model 73402 1ea Oriel Instruments Radiometric power supply Model 68831 1ea Oriel Instruments MERLIN chopper power supply Model 70100 1ea Ithaco Model 1211 Current Pre-amp 1ea Texas Instruments DLP DDE PC-DCI interface board w/micromirror array, video card, and software 1ea ZEOS Pentium computer system running LabVIEW software 1ea Judson IRI 23033488 lead sulfide (PbS) detector 1ea HP Model 6206B DC power supply (used to provide dc bias for PbS detector) 1ea EG&G PAR UV100 silicon detector 1ea EG&G PAR Model 5209 Lock-in Amplifier 1ea EG&G PAR Model 5113 Pre-amp 1ea Tektronix Model 2465B 400MHz oscilloscope 1ea Newport Model PAC040 Achromatic lens MgF2 1ea Newport Model 425 X-Y translation stage 1ea Newport Model 37 3-axis tilt stage 1ea Newport Model SBX 052 uncoated UV Fused Silica bi-convex lens 1ea Newport Model 496 rotary stage 1ea Newport Model 270 lab jack 1ea Newport Model BK-3 kinematic base plate 1ea Newport ModelPMC100 motion controller 1ea Newport Model LH-100 1”dia lens holder 1ea Newport Model LH-200 2”dia lens holder 2ea Newport Model BUP-2 universal base 3ea Newport Model VPH-4 post holders 3ea Newport Model SP-3 posts
27
SNL Micro-mirror Array Experiments We have investigated reliability and accuracy of the pointing angle of compound pivot micro mirrors fabricated in SUMMITV. Figure 16 shows the layout of the die investigated, RS298 with the mirror array under investigation circled in red.
Mirror array studied contains six individually addressed columns of four mirrors each. The four mirrors are connected electrically and upon application of voltage to the appropriate pads, actuate together.
Figure 16. CAD layout of RS298, SUMMITV die under investigation. Mirror array of
interest has a red box around it.
poly 3 mirror
poly 1 electrode
poly 2 springs
Right poly 0electrode
Left poly 0electrode
poly 3 mirror
poly 1 electrode
poly 2 springs
Right poly 0electrode
Left poly 0electrode
Figure 17. Schematic representation of the polysilicon layers constituting the compound pivot
micro mirror.
28
Devices were fabricated in a multi-layer polysilicon surface micromachining process (SUMMIT V™) at Sandia National Laboratories. As shown in Figure 17, mirrors are implemented using the poly3 surface layer (blue), springs with the poly2 layer (red), capacitive plates with the poly1 layer (gray) and contact layer is implemented using poly0 (pink). Individual devices are 50 µm x 50 µm in size with a 2.25 µm thick reflective surface supported by a capacitive electrode pair, which is electrostatically switched to the desired angle. These mirrors are of the compound pivoting type, tilting about two shifted axes until the top capacitive plate stops on the grounded substrate plane. This is designed to result in a tilt angle of ~7°, obtained from an inverse tangent of the vertical travel distance (∆x) over the pivot arm length (∆y) and illustrated in Figure 18b.
Left SpringLeft Pivot Axis Right Pivot Axis
(a)
(b) (c)
Left Capacitive Plate
Right Capacitive Plate
Right Spring
∆y∆x
Left SpringLeft Pivot Axis Right Pivot Axis
(a)
(b) (c)
Left Capacitive Plate
Right Capacitive Plate
Right Spring
∆y∆x
Figure 18. Schematic representation of (a) plan view of the mirror with the top surface of the mirror removed. Left and right capacitive plates are shown in solid gray as well as labeled. The spring which stretches when the mirror is tilted to the left is shown in solid black and is labeled
as “left spring”. The spring which stretches when the mirror is tilted to the right is shown in cross hatch and is labeled “right spring. Also indicated are the left and right pivot axes. (b) Is a
cross-sectional view of the mirror when it is tilted to the left, along with an un-tilted view below. Also indicated are the pivot arm distance (∆y) and vertical displacement distance (∆x). (c) Is a cross-sectional view of the mirror when it is tilted to the right, along with an un-tilted
view below. Left and right pivot axes are indicated with arrows continuing from (a).
29
In Figure 18a, the plan view of the basic structure of the mirror is shown as well (with the top reflective surface not shown). The left and right pivot axes are shifted to the positions indicated by the arrows. Upon application of voltage to the left or right bias pad (not shown in the figure), the left or right capacitive plate deflects downwards, onto a mechanical stop (this is where the capacitive plate touches the substrate). Left and right springs (indicated in Figure 3 in black and cross hatch pattern respectively) stretch to allow the tilting motion and provide the restoring force upon removal of the bias. Unbiased and tilted mirror positions are shown in cross-section in Figure 18b and c. The tilt angle was measured by bouncing a HeNe laser beam off the surface of the mirror and measuring its position on a position sensitive detector (PSD). Upon application of an appropriate bias, generated by LabView software and a digital acquisition (DAQ) board and amplified by an amplifier, the mirror would tilt, displacing the laser beam on the position sensitive detector. The change in the beam position resulted in a different voltage put out by the PSD and acquired by the DAQ board. After appropriate calibration, the change in position of the laser beam on the PSD provided the measurement of the mirror tilt angle. Figure 19 shows the experimental set up schematically
Devices were tested in a micro clean environment in order to minimize the effect of particle contamination, although no effort was made to regulate temperature and humidity. Mirrors were un-packaged and bias was applied with probes, that is no wire bonding was done. Each experiment was carried out on a new mirror column. The device under tested was actuated in one direction only, that is bias was applied to the left pad only and the right pad was left floating. This is the simplest way to actuate the devices, although possibly not the optimal one. Theoretically, best results are expected for the “zero” tilt angle when both pads are actuated at a lower level then the “tilting bias”. For example, if the bias necessary to tilt the mirror to the left is on the order of 120V, the “zero” bias on both pads needed to give a stable “zero” position maybe on the order of ~50V. Application of “zero” bias to both pads ensures that the “zero” tilt state is determined by a mechanical contact where the mirror “sits down” on both the left and right pivot points (refer to Figure 3 for location of the
LabVIEW™
DAQ board 10X Amplifier
Position SensitiveDevice
MirrorsLaser
LabVIEW™LabVIEW™
DAQ board 10X Amplifier
Position SensitiveDevice
MirrorsLaser
Figure 19. Schematic representation of the experimental set up.
30
pivot axis). If the “zero” position is obtained by removing the bias from the actuating pad and floating the other pad, only the restoring action of the poly2 spring is responsible for bringing the mirror to the flat “zero” position. Although this is not the optimal way to operate the mirror, that is what we did as a first order (and worst case) scenario. The device under test was operated at 1kH continuously for approximately a 24 hr period. This was repeated for several mirrors on a single die. Data representing a full cycle (1 msec) of the mirror tilt state was acquired every 8 sec, resulting in a sample of 10000 cycles out of 8.64 x 107 cycles during a 24 hr period. During each cycle that was acquired, mirror position was obtained with a 1 µsec resolution, such that 1 msec was represented by 1000 data points. Figure 20 shows mirror tilt data for a single cycle (1 msec) as well as the bias on the actuating pad.
As measured previously, the turn on time, that is time between the turn on of the bias voltage and full tilt of the mirror is ~30 µsec. Upon actuation, the tilt angle rings for an additional ~50 µsec before settling out at a steady state value. Turn off time is ~ 22 µsec and settling time an additional ~17 µsec. The turn off times are associated with stiffness of the poly2 spring, which pulls the mirror back to the “zero” position. These can be affected by appropriate choice of the “zero” position biasing scheme as mentioned above. The ringing time can be decreased by appropriate choice of actuation voltage. In this case, actuation voltage of 154 V was chosen, which is greater then the minimum voltage required for switching. For each data set, like the one shown in Figure 20, a mean value and standard deviation of the minimum tilt angle (flat “zero” position) and maximum tilt angle (~7°) was calculated. This was
0 200 400 600 800 1000
0
2
4
6
8
0.0
40.0
80.0
120.0
160.0
Tilt
Ang
le (d
egre
es)
Time (µsec)
Tilt Angle Applied Bias
Applied Bias (V
)
Figure 20. Data representing one cycle of tilt angle (left y-axis, black curve) and applied bias to
the energized pad (right y-axis, red curve) as a function of time for a 50 µm x 50 µm mirror.
31
accomplished by first sorting the data points between minimum (<20% of peak value – minimum value) and maximum (>80% of peak value-minimum value), such that the points during the transition between minimum and maximum were ignored. Then first 50 points of the minimum and maximum data set were ignored. This crudely allowed us to disregard the oscillation or ringing of the tilt angle. Since the extent of the ringing can be controlled by the biasing scheme, a set of bias conditions can be found which minimize it, thus we ignored it in the calculation of the statistics of the minimum and maximum tilt angle. Thus two values and their standard deviations (sd) were obtained for each data set like the one in Figure 20: mean maximum tilt (maxmean) and its sd and mean minimum tilt (minmean) and its sd. Although we have 10000 of these values over a 24 hr period, we plotted only on the order of 100, representing a mean value of the maximum and minimum tilt every 15 minutes (this was done in order to save computation time and to clarify the plot). In the areas of interest, where there is a change in the mean values, we plotted every data point, representing 8-second intervals. Figure 21 shows a plot representing the trend in value of maxmean and minmean over a 24 hr period.
As can be seen from the above plot, there is an initial change in the mean value of the maximum and minimum tilt angle. In order to more closely investigate this phenomenon, we plotted data points for data taken every 8 seconds in that region. Figure 22 shows the initial behavior of the mean values of the maximum and minimum tilt angle. For the mirror whose data is shown in Figure 21, average maximum tilt angle over the period of 24 hrs 6.92171± 0.08331and average minimum tilt angle is 0.16774 ± 0.08321. If the initial 30
00:00 04:48 09:36 14:24 19:12 24:00
6.5
7.0
7.5
0.0
0.5
1.0
Max
Tilt
Ang
le (d
egre
es)
Time (hh:min)
MAXMEANFrequency = 1kHzDataset spacing = 15 min MINMEAN
Min Tilt A
ngle (degrees)
Figure 21 Trend in the mean values of the maximum (black, left y-axis) and minimum (red, right
y-axis) tilt angle values for a 50 µm x 50 µm micro mirror (pads 77, 78) over a 24 hr period. Error bars indicate the standard deviation values for each mean value.
32
minutes of data are ignored (the “burn-in period), the average maximum tilt angle is 6.91047± 0.04578 and the average minimum tilt angle is 0.16905± 0.08413.
Figure 23 shows the tilt angle behavior of another 50 µm x 50 µm mirror during the 24 hr operation. The mirror under test was adjacent to the mirror whose tilt characteristics are shown in Figures 21 and 22. Same bias of 154V was applied to the actuating pad. Similarly to the mirror of Figure 21 and 22, there appears to be an initial period of operation where the mean maximum and minimum tilt angles change. Figure 24 shows the detail of the tilt angle performance corresponding to the first 60 minutes of operation. In this case the mean maximum and minimum tilt angles are 6.67529± 0.09261 and 0.25375 ± 0.0451, respectively for the 24 hr period. If the initial 60 minutes are ignored (where most of the change occurs), the mean maximum and minimum tilt angles are 6.65712 ±0.02885 and 0.25997 ± 0.03358, respectively. Results for the third mirror (adjacent to the second mirror described above) are shown in Figures 25 and 26. Analogously to the previous case, Figure 25 shows the survey of the tilt performance over 24 hr period, whereas Figure 26 shows the detail of the time period during which mean values of the tilt angle are changing. For mirror #3, mean maximum and minimum tilt angles are 6.95053 ± 0.03208 and 0.18769 ± 0.04084, respectively for a 24 hr period. If the first 30 minutes of operation are ignored, the mean maximum and minimum tilt angles are 6.95282 ± 0.02847 and 0.19119 ± 0.03909, respectively. Table 1 summarizes these results.
00:00 00:07 00:14 00:21 00:28
6.6
6.8
7.0
7.2
7.4
7.6
0.0
0.2
0.4
0.6
0.8
1.0M
ax T
ilt A
ngle
(deg
rees
)
Time (hh:min)
MAXMEAN
Frequency = 1kHzDataset spacing = 8 sec
MINMEAN
Min Tilt A
ngle (degrees)
Figure 22 Initial behavior of the mean values and standard deviations of the maximum (black
curve left y-axis) and minimum (red, right y-axis) tilt angle during the first 30 minutes of operation for a 50 µm x 50 µm micro mirror (pads 77, 78).
33
00:00 04:48 09:36 14:24 19:12 24:006.0
6.3
6.6
6.9
7.2
0.0
0.3
0.6
0.9
1.2
Max
Tilt
Ang
le (d
egre
es)
Time (hh:min)
MAXMEANFrequency = 1kHzDataset spacing = 15 min MINMEAN
Min Tilt A
ngle (degrees)
Figure 23 Trend in the mean values of the maximum (black, left y-axis) and minimum
(red, right y-axis) tilt angle values for a 50 µm x 50 µm micro mirror (pads 75, 76) over a 24 hr period. Error bars indicate the standard deviation values for each mean value.
00:00 00:14 00:28 00:43 00:576.0
6.3
6.6
6.9
7.2
0.0
0.3
0.6
0.9
Max
Tilt
Ang
le (d
egre
es)
Time (hh:min)
MAXMEAN
Frequency = 1kHzDataset spacing = 8 sec
MINMEAN
Min Tilt A
ngle (degrees)
Figure 24 Initial behavior of the mean values and standard deviations of the maximum (black curve left y-axis) and minimum (red, right y-axis) tilt angle during the first 60
minutes of operation for a 50 µm x 50 µm micro mirror (pads 75, 76).
34
00:00 04:48 09:36 14:24
6.4
6.6
6.8
7.0
7.2
0.0
0.2
0.4
0.6
0.8
Max
Tilt
Ang
le (d
egre
es)
Time (hh:min)
MAXMEANFrequency = 1kHzDataset spacing = 15 min MINMEAN
Max Tilt A
ngle (degrees)
Figure 25 Trend in the mean values of the maximum (black, left y-axis) and minimum (red, right y-axis) tilt angle values for a 50 µm x 50 µm micro mirror (pads 73, 74) over a 17 hr
period. Error bars indicate the standard deviation values for each mean value.
Figure 26 Initial behavior of the mean values and standard deviations of the maximum (black
curve left y-axis) and minimum (red, right y-axis) tilt angle during the first 30 minutes of operation for a 50 µm x 50 µm micro mirror (pads 73, 74).
35
Table 2. Results of SNL MEMS micro-mirror test.
Mirror #
Burn in period (min)
Contact pads
Mean Max (all)
Mean Min (all)
Mean Max (w/o burn in period)
Mean Min (w/o burn in period)
1 30 77, 78 6.92171± 0.08331
0.16774 ± 0.08321
6.91047± 0.04578
0.16905± 0.08413
2 60 76, 75 6.67529± 0.09261
0.25375 ± 0.0451
6.65712 ±0.02885
0.25997 ± 0.03358
2 30 76, 75 6.67529± 0.09261
0.25375 ± 0.0451
6.66015 ± 0.03528
0.25875 ± 0.03424
3 30 74, 73 6.95053 ± 0.03208
0.18769 ± 0.04084
6.95044 ± 0.03118
0.18916 ± 0.04019
As Table 2 indicates, omission or inclusion of the burn-in period affects the standard deviation of the mean maximum tilt angle more than any other variable. The origin of the burn-in period is under investigation. Future work should include study of the effect of metallization, drying method (super critical drying vs. self-assembled monolayers drying, mirrors 1-3 were dried by both methods), packaging, dormancy period, bias scheme, frequency of operation, environmental variables, such as temperature and humidity and others. Also, since mirrors show no sign of failure at 24 hrs of continuous operation, it is necessary to extend the period of data acquisition to several days or weeks if the mean time to failure is required.
36
Experimental Setup for the HTSI To simplify the data acquisition system all the electronics are COTS (commercial-off-the-shelf). The chassis for the electronics is a National Instruments (NI) PXI-1000B 8-slot PXI chassis. The controller is an NI 8176 embedded controller using a 1.26 MHz Pentium III with 512 MB of ram running Windows 2000. There are three NI PXI-6534s, which are 32-bit high-speed digital I/O card to control the Texas Instruments digital micro-mirror device (DMD). The camera for the system is a Dalsa 1M15 1024x1024 camera that can support a frame rate of 15 frames per second (fps). The frame grabber and the digital I/O are all controlled using LabVIEW code. There is also the option of controlling the DMD using a Universal Serial Port (USB) if this is determined to have any advantages.
37
Conclusions The Hadamard LDRD work has produced two different imaging designs that have different capabilities. The spectral encoding design is DMD size restricted. To optimize the system there needs to be control over the size and design of the DMD, this was what led us to investigate the Sandia National Laboratories MEMS mirror arrays. The spatial encoding design is FPA size restricted. This was what led us to move forward with the spatial encoding design. There were many different selections that were possible for the FPA that simplified the entire system design without limiting the overall system. A software simulation was done for each method to evaluate design parameter changes. After the completion of the initial optical designs and based on simulation results a patent application was submitted and is pending approval. The data acquisition system has been setup and is awaiting the optical components. During the course of the project two papers have be published on the HTSI the first one is, C. M. Wehlburg, J. C. Wehlburg, S. M. Gentry, and J. Smith, “Optimization and Characterization of an Imaging Hadamard Spectrometer”, Proc. SPIE, 4381, pp. 506-515 (2001) which is included in Appendix C and the second one is M. W. Smith, J. L. Smith, G K. Torrington, C. M. Wehlburg, and J. C. Wehlburg, “Theoretical description and numerical simulations of a simplified Hadamard transform imaging spectrometer”, Proc. SPIE, 4816 (pending printing) which is included in Appendix D. Initial investigations into using the Sandia National Laboratories MEMS technologies to fabricate micro mirror arrays for the HTSI have been started and the results show that the techniques is capable of producing mirrors with an angular tilt accuracy of better then ±0.08º. This is not accurate enough for a final production system, but it is promising for prototype mirrors that were not packaged. The mirrors were also tested for 24hours at 1kHz with no failures. Currently Sandia National Laboratories Department 1742 is continuing to test the mirrors for a longer period to get long-term reliability data. The ability to have micro-mirror arrays custom fabricated with different mirror sizes, coating and tilt angles would enhance the HTSI capabilities in both spectral and spatial resolution. If the system is going to go beyond a laboratory demonstration system this will be a vital part. The results from this LDRD were the creation of two simulations for the analysis of spectral and spatial encoded two-dimensional HTSIs. An optical design for the spatial encoding method was competed and the optical components were constructed that will operate in the 450 nm – 800 nm spectral band and provided 10 nm spectral resolution per CCD pixel. Two optical designs for the spectral encoding design were investigated one for ~400nm-750nm and the other for 1.2 µm-2.4 µm. These designs were not competed after it was determined that there was not sufficient time or funding to produce the needed optical components. One of the major components of the optical design is the TI DMD which was evaluated for the spectral ranges that were of interest to this LDRD. TI has now also developed an IR enhanced version, which improves upon the reflection efficiency in the 1 µm -2.5 µm range with a loss of reflection efficiency in the visible
38
wavelengths. The option of custom fabricating our own micro-mirror array was evaluated and though the technology is not at a stage that could be used for this LDRD the work underway shows that future efforts should include Sandia National Laboratories MEMS as an option for improving the optical efficiency and allowing more flexibility in the optical design for the HTSI. The research done during this LDRD has shown that the HTSI can be manufactured with existing technology and can provide a performance in the temporal and hyper-spectral region that cannot currently be achieved with other imaging systems.
39
References 1. M. Harwit and N. J. A. Sloane, Hadamard Transform Optics, p. 3, Academic Press , New York, 1979. 2. A. A. Michelson, Light Waves and Their Uses, Univ. of Chicago Press, Chicago, 1902 reprinted 1961. 3. P. R. Giffiths, “Interferometry in the Seventies,” Analyt. Chem., 46, 645A-654A, 1974. 4. W. G. Fateley, R. M. Hammaker, J.V. Paukstelis, S. L. Wright, E. A. Orr, A. N. Mortensen and K. J. Latas,
“Application of a 2-Dimensionsl Hadamard Encoding Mask for the Imaging of Thin-Layer Chromatography Plates by Laser-Induced Fluorescence or Surface-Enhanced Raman-Scattering and for Use with a Photoacoustic Detector to Generate 3-Dimensional Photoacoustic Images”, Appl. Spectrosc., 47, pp. 1464-1470, 1993.
5. R. M. Hammaker, A. N. Mortensen, E. A. Orr, M. K. Bellamy, J. V. Paukstelis and W. G. Fateley, “Multidimensional Hadamard-Transform Spectrometry”, J. Mol. Struc., 348, pp. 135-138, 1995.
6. M. Harwit and N. J. A. Sloane, Hadamard Transform Optics, pp. 109-180 and 229-243, Academic Press , New York, 1979.
7. R. M. Hammaker, J. A. Graham, D. C. Tilotta and W. G. Fateley, in J. R. Durig (Ed.), Vibrational Spectra and Structure Vol. 15, pp. 401-485, Elsevier, Amsterdam, 1986.
8. R. M. Hammaker, A. P. Bohlke, R. D. Freeman, J. M. Jarvis, K. J. Latas, D. Lin-Vien, A. M. Mortensen, E. A. Orr, J. D. Tate, D. C. Tilotta, J. S. White, S. L. Wright, J. V. Paukstelis and W. G. Fateley, in: J. J. Laserna (Ed.), Modern Techniques in Raman Spectroscopy, pp. 143-226, Wiley, Chichester, 1996.
9. M. Harwit and N. J. A. Sloane, Hadamard Transform Optics, pp. 12-19, Academic Press , New York, 1979. 10. Quentin S. Hanley, Peter J. Verveer and Thomas M. Jovin, “Spectral Imaging in a Progammable Array
Microscope by Hadamard Transform Fluorescence Spectroscopy”, Appl. Spectrosc., 53, pp. 1-10, 1999. 11. M. K. Bellamy, A. N. Mortensen, R. M. Hammaker and W. G. Fateley, “Chemical Mapping in the Mid- and
Near-IR Spectral Regions by Hadamard Transform/FT-IR Spectrometry,”, Appl. Spectrosc., 51, pp. 477-486, 1997.
12. R. A. DeVerse, T. A. Mangold, R. M. Hammaker and W. G. Fateley, in: J. A. de Haseth (Ed.), Fourier Transform Spectroscopy Eleventh International Conference, AIP Conference Proceedings 430, pp. 443-446, AIP, Woodbury, 1998.
13. R. A. DeVerse, R. M. Hammaker and W. G. Fateley, “Realization of the Hadamard Multiplex Advantage Using a Programmable Optical Mask in a Dispersive Flat-Field Near-Infrared Spectrometer,” Appl. Spectrosc., 54, pp. 1751-1758, 2000.
14. R. A. DeVerse, R. M. Hammaker and W. G. Fateley, “Hadamard transform Raman imagery with a digital micro-mirror array,” Vib. Spectrosc., 19, pp. 177-186, 1999.
15. R. A. DeVerse, R. M. Hammaker and W. G. Fateley, “An improved Hadamard encoding mask for multiplexed Raman imaging using single channel detection,” J. Mol. Struc., 521, pp. 77-88, 2000.
16. Qentin S. Hanley, Peter J. Verveer and Thomas M. Jovin, “Optical Sectioning Fluorescence Spectroscopy in a Programmable Array Microscope,” Appl. Spectrosc., 52, pp. 783-789, 1998.
17. Kimberly D. Barnes, Nikhil Taluja, Ruth A. Dyer and Stephen A. Dyer, “Development and Implementation of a General-Purpose Hadamard-Transform Spectrometer Simulation Program,” IEEE Transactions on Instrumentation and Measurement, 47, pp. 163-167, 1998.
18. Warren J. Smith (Ed.), Modern Optical Engineering, p.154, McGraw-Hill, New York, 1990. 19. D. Kwo, G. Lawrence andM. Chrisp, “Design of a Grating Spectrometer from a 1:1 Offner Mirror System,”
Current Developments in Optical Engineering II, R. E. Fisher and W. J. Smith, Eds., Proc. SPIE 818, pp. 275-278, 1987.
40
Appendix A “Optimization and Characterization of an Imaging Hadamard Spectrometer”
41
Optimization and Characterization of an Imaging Hadamard Spectrometer
Christine M. Wehlburg*, Joseph C. Wehlburg, Stephen M. Gentry and Jody Smith, Sandia National Laboratories, 1515 Eubank SE, Albuquerque, NM 87185†
ABSTRACT
Hadamard Transform Spectrometer (HTS) approaches share the multiplexing advantages found in Fourier transform spectrometers. Interest in Hadamard systems has been limited due to data storage/computational limitations and the inability to perform accurate high order masking in a reasonable amount of time. Advances in digital micro-mirror array (DMA) technology have opened the door to implementing an HTS for a variety of applications including fluorescent microscope imaging and Raman imaging. A Hadamard transform spectral imager (HTSI) for remote sensing offers a variety of unique capabilities in one package such as variable spectral and temporal resolution, no moving parts (other than the micro-mirrors) and vibrational insensitivity. An HTSI for remote sensing using a Texas Instrument digital micro-mirror device (DMD) is being designed for use in the spectral region 1.25–2.5 µm. In an effort to optimize and characterize the system, an HTSI sensor system simulation has been concurrently developed. The design specifications and hardware components for the HTSI are presented together with results calculated by the HTSI simulation that include the effects of digital (vs. analog) scene data input, detector noise, DMD rejection ratios, multiple diffraction orders and multiple Hadamard mask orders.
1. INTRODUCTION An important consideration in developing a remote sensing instrument is signal to noise. Two available choices in
spectroscopy that incorporate the multiplexing advantage to increase signal to noise are Fourier and Hadamard transform instruments1. The development of the Michelson interferometer and advanced computing resources have contributed to the availability of commercial one-dimensional, two-dimensional imaging and hyperspectral imaging Fourier transform spectrometers2,3. Concurrent advances in Hadamard transform techniques were hampered by limitations in encoding mask technology. Recent developments in digital micro-mirror array (DMA) technology and the availability of two-dimensional detector focal plane arrays charge have motivated our group to build a Hadamard transform spectral imager (HTSI) for remote sensing. An HTSI would have the same multiplexing advantages as a Fourier transform system, however, there are additional benefits that are uniquely beneficial to remote sensing. The HTSI would have no moving parts, be insensitive to vibrations and capable of incorporating variable spectral and temporal resolution.
The development of Hadamard transform spectrometers as a viable multiplexing alternative to Fourier transform techniques has progressed in the past decade with improvements in encoding mask technology. Liquid crystal spatial light modulators (LC-SLM’s) were utilized by the Hammaker-Fateley group as stationery Hadamard encoding masks to replace moving encoding masks4,5. At the time, moving encoding masks were limited by their ability to reproducibly position the masking patterns6-8. A major limitation of an LC-SLM is the non-ideal transmission and rejection. Hadamard encoding masks, in a traditional cyclic S-matrix configuration, require a 1 or 0 condition. In the 1 condition, the transmission (T1) is ideally 100%. For the 0 condition, the transmission (T0) is ideally 0% (100% rejection)9. Hammaker and Fateley’s Raman imaging system was developed with an LC-SLM that was also small in size (10 X 10 pixels). Recent work by Jovin and coworkers using an 800x600 pixel format liquid LC-SLM as the encoding mask for Hadamard transform fluorescence microscopy imaging reported the acquisition of two-dimensional spectral images despite less than ideal T1 and T0 behavior10. Improvements in translation device technology resurrected the use of moving encoding masks, however, they remain relatively slow in the encoding sequence and both the mask dimensions and element size are fixed11,12.
More recent advances in DMA technology and the commercial availability of the Texas Instruments digital micro-mirror device (DMD) have enabled researchers to consider micro-mirror arrays as an alternative encoding mask over LC-SLM’s. The Hammaker-Fateley group, building on their pioneering work in Hadamard spectroscopy‡, utilizes the DMA as an SLM for generating a stationary Hadamard encoding mask. With the DMA in a dispersive flat-field near-infrared spectrometer that could be operated in both conventional raster scanning (CRS) and Hadamard transform spectrometer modes, DeVerse, Hammaker and Fateley were able to experimentally document the Hadamard multiplex advantage. The DMA in the flat-field spectrometer was utilized as a 1D Hadamard mask for spectral encoding13. Due to the cost-prohibitive nature of 2D multi-channel detectors for future expected work in the near-infrared, the current Raman imaging systems designed by DeVerse and coworkers employs a single element detector for imaging in the visible spectral region. A 1D Hadamard cyclic S-matrix encoding mask (spectral encoding) is folded into a 2D Hadamard encoding mask and together with sample rastering a single element detector can be used for Hadamard transform imaging14,15. Hanley, Verveer and Jovin, in addition to developing a Hadamard transform fluorescence microscopy imager with an LC-SLM, have reported data for fluorescence imaging with a DMA, as well16. Some of the differences between the Raman imaging system (DeVerse and coworkers) and the fluorescence microscope imaging system (Hanley and coworkers) include the use of a 2D Hadamard cyclic S-matrix for spatial (not spectral) encoding and a 2D charge couple device (CCD) detector operating in the visible region.
Our group is designing an HTSI for visible to near-infrared (1.25 –2.5 µm) remote sensing applications. As DeVerse and coworkers mention in their work on a Hadamard Raman imager, the near-IR 2D detector arrays should outperform the raster scanning mode necessary with a single detector14,15. Although not as cost effective as the single element detector, our design includes an InSb focal plane array. In an effort to optimize the spectrometer, an HTSI simulation has been developed. Previous Hadamard spectrometer simulation programs for 1D spectral data developed by Deyer and coworkers have modeled the effects of various encoding and spectrum recovery schemes on input data and also included the effect of detector noise17. Our HTSI simulation for 2D hyperspectral image cubes includes modeling the effects of various optical components and design elements including multiple diffraction orders, multiple order left-cyclic S-matrix masks, physical placement of optical components, DMA size and rejection specifications and detector size and noise specifications. The simulation is being developed as a tool to model the effect of possible optical design changes as the HTSI is built and avoid costly errors in design that were not anticipated because of the lack of data for HTSI systems. In addition to results calculated with the simulation, we will present specifications for the detector, DMA, optical design and data acquisition and control systems that are currently completed for the HTSI.
2. THEORY 2.1 Hadamard Transform Basics
The optical multiplexing advantage that Hadamard transform spectroscopy achieves has its historical basis in the development of optimal weighing design schemes. In the case of spectra, measuring multiple wavelength intensities according to a weighing scheme or Hadamard masking order, a corresponding increase in accuracy is observed. There are two types of weighing designs available in developing a Hadamard transform system, ‘H’ and ‘S’ matrices. An ‘H’ or Hadamard weighing design consists of entries corresponding to –1, 0 and +1. A subset type of Hadamard matrix design is the simplex or S-Matrix mask that utilizes +1 and 0 in the weighing design. Due to the relative ease in translating a Hadamard encoding mask, the use of left-cyclic S-matrices is common in optical applications of Hadamard weighing schemes9.
Figure 1A represents a left-cyclic Hadamard S-matrix weighing design with an order of 7. A black square indicates an off or “0” condition while the white are on or “1”. Theoretically, each position in the Hadamard mask could correspond to a specific wavelength’s intensity. The resultant 7 observables would then be multiplied by the inverse of the Hadamard S-matrix, as shown in Figure 1B, in order to solve for the individual wavelength intensities. Given the assumption that detector noise is independent of the amount of light reaching the detector holds true, the signal-to-noise improvement for a left-cyclic S matrix encoding over single-slit measurements approaches ~ n /29. If every nth observable consisted of measuring the intensity at a single wavelength (i.e. the weighing scheme had 6 off positions and 1 on position per observable), then the system would be equivalent to a conventional dispersive spectrometer. Using Figure 1 as a specific example of an ideal 7th order left-cyclic Hadamard S-matrix, the multiplexing advantage is realized because for the same number of observables (7) each wavelength intensity is measured 4 times instead of once.
‡ An exhaustive list of citations documenting the Hadamard transform spectroscopy effort at Kansas State University
until 1997 can be found in Ref. 11.
43
2.2 Diffraction Grating
The equation for dispersion implemented in the simulation code was that for a diffraction grating18,
Tminout λθθ += sinsin (1)
where the incident angle (θin) is related to the diffraction angle out (θout) by the wavelength, λ (µm), the dispersion order m (0, 1, 2 …) and the grating constant T (µm-1). The actual offset (cm) for each λ can be calculated according to the geometry in Figure 2 and Eqn. 1,
DISTANCETmINOFFSET *)}(sintan{sin 1 λθλ += − (2)
3. EXPERIMENTAL 3.1 Hardware
A schematic of the hardware components for the HTSI is shown in Figure 3. The camera is a Santa Barbara Focalplane SBF-125 system that contains a 320x256 InSb focal plane array (FPA) sensitive from 0.5-2.5 µm and includes a closed-cycle dewar for cooling. The FPA will be windowed to 256x256 pixels to allow a frame rate of 400 Hz. The data will
n0
n1
n2
n3
n4
n5
n6
0 1 2 3 4 5 6n0 = ψ0 + ψ1 + ψ2 + ψ4 + ε0
n1 = ψ0 + ψ1 + ψ3 + ψ6 + ε1
n2 = ψ0 + ψ2 + ψ5 + ψ6 + ε2
n3 = ψ1 + ψ4 + ψ5 + ψ6 + ε3
n4 = ψ0 + ψ3 + ψ4 + ψ5 + ε4
n5 = ψ2 + ψ3 + ψ4 + ψ6 + ε5
n6 = ψ1 + ψ2 + ψ3 + ψ5 + ε6
n0
n1
n2
n3
n4
n5
n6
ψ0
ψ1
ψ2
ψ3
ψ4
ψ5
ψ6
=
-1
x
A B
Figure 1. (A) Graphical representation of a left-cyclic Hadamard S-matrix, with white and black squares representing ones and zeros, respectively, with the equations for observables for seven channels. (B) Hadamard transform solution.
Figure 2. Schematic of geometry defining incident and diffraction angles and distances between the diffraction grating and the DMD.
inθ outθ
A
B
C
DMD
DiffractionGrating
A≡λOFFSETB≡distanceC≡DMD
OFFSET
44
be streamed off to a PCI based StreamStor system made by Boulder Instruments that can support a data rate of 60 MB/s and exceeds the ~52 MB/s output of the camera.
Data processing will be handled by a PCI based DSP board using the TI C6701 chip that allows the real-time processing of the Hadamard encoded data as it is collected. The DSP will also send the control signals to the board that implements the Hadamard encoding mask on the Texas Instruments 1024x768 pixel DMD. Figure 4 illustrates how the DMD generates the masking pattern by turning the appropriate micro-mirrors “on” and “off”. The “on” mirrors direct light through the optical system, while the “off” mirrors redirect light to an optical stop. An Intel-based computer will house the DSP and the StreamStor systems and will handle the long-term (post-experiment) storage. The user interface to the DSP will be coded in LabView to allow ease of modification, simplicity of design and allow post processing of data on the host computer.
Qualifying the performance of the DMD is critical for ascertaining the effects of alignment and rejection on the HTSI. The on/off specifications for the DMD are not exactly 1 or 0. Figure 5 represents experimental data for the rejection ratio [signalon/signaloff] of the micro-mirror array collected over the wavelength region 0.5–2.5 µm. The rejection ratio was measured at 100 nm increments and the “on” signal was optimized for each wavelength by adjusting the detector response and the DMD (particularly the tip/tilt and focal position). The tip/tilt and focal position of the DMD in the actual HTSI will be fixed, so the parameters will need to be adjusted to maximize the rejection ratio across the spectral region of interest and the subsequent functional dependence will be measured and incorporated into the model.
Figure 4. Diagram of two micro-mirrors on DMD. Each 16 µm2 mirror consists of three physical layers and two “airgap” layers that allow the mirror’s angle to tilt +10°, “on”, or -10°, “off”.
Figure 3. Block diagram of components in the Hadamard transform spectral imager.
Source
DiffractionGrating
RefocusingOptics
DMD
2DDetector
Array
PCI basedDSP board
Computer
45
3.2 Optical Design
The optical design for the HTSI is essentially two split Offner relays. An Offner relay has all the advantages of a symmetrical system containing no coma, distortion or lateral color. Offner relays are commonly used as the optical system in slit system spectrometers19. However, since the HTSI is an imaging system, the optical design is somewhat more challenging. Inherent to an Offner relay design, and advantageous for an imaging system, is a large field of view and good image quality. The symmetry of the optical design is inherently telecentric and a small defocusing displacement will not change the image size. The split Offner configuration will be implemented in order to accommodate the pathlength variations incurred at the gratings and provide an additional degree of freedom for future adjustments. The system performance goal is a modulation transfer function (MTF) of 0.1 at 70 line pairs per millimeter, and 0.4 at 30 line pairs per
millimeter. Preliminary design work indicates that this will be attainable. The approximate working focal ratio of the system is F/4. Fore-optics not shown in Figure 5 create the image at the object plane (input image plane). The first Offner relay will bring the image to the grating, disperse the wavelengths then relay the spectrally dispersed image to the DMD. Our current design for the first Offner relay has spot sizes in the range of 5-8 µm in the x- and y-direction. The dispersed and Hadamard encoded image is passed from the DMD to the second split Offner relay containing another convex grating that recombines the spectrally dispersed elements and relays the encoded image to the detector. Because the signals are expected to be relatively small, stray light will be an important consideration. In order to address stray light considerations, the Offner relay design offers sufficient room for baffling not necessarily available in other configurations.
3.3 Model
In a real, analog HTSI system, all wavelengths and diffraction orders of the image would be dispersed across the DMD, which would be encoded for the appropriate masking pattern, and the encoded image for each Hadamard order would
Figure 5. DMD rejection ratio [signalon/signaloff] as a function of wavelength. The “on” signal was optimized at each 100 nm data collection.
0
200
400
600
800
1000
200 600 1000 1400 1800 2200 2600
Wavelength (nm)
Rat
io (O
n/O
ff)
Figure 6. Double split Offner relay optical design for the HTSI.
InSb DetectorArray
Input ImagePlane
Convex Grating
DMD
Convex Grating
46
be refocused onto the detector. The encoding sequence for each Hadamard order in the HTSI simulation begins with the initial S-Matrix vector (see Figure 1, n0 vector). As shown in Figure 7, each element (0 or +1) in the Hadamard vector is expanded by the value of the DMD pixel width divided by the Hadamard order rounded down to the nearest integer. The element size of the Hadamard encoding mask on the DMD becomes the number of pixels each element was expanded to fit the entire DMD in the x-direction. For each subsequent Hadamard encoding mask, the vector is rotated in a left-cyclic direction by a magnitude corresponding to the extended Hadamard element (# pixels) size.
Each extended Hadamard vector must encode each wavelength slice of the input data cube. The simulation has the additional option of generating multiple diffraction orders for each wavelength slice, each of which must be encoded. Figure 8 illustrates how the simulation applied the expanded Hadamard vector across each wavelength/diffraction order slice according to the calculated pixel shift from the edge of the DMD using the offset in Eqn. 2. Detector simulation is implemented by adding together all the encoded wavelength/diffraction order slices per Hadamard order. The summed encoded slices for each order are interpolated out to the 256x256 pixel dimensions of the detector and a Gaussian white noise based on a percentage of the maximum value of the measurement is added to the results. The detector slices for each
Hadamard order are the array slices in the untransformed Hadamard data cube or encodegram. The Hadamard transformed output data cube is calculated by multiplying the vector for each pixel in the encodegram by the inverse of the original left-cyclic S-matrix. The HTSI simulation was coded in LabView and inquiries regarding the source code can be directed to the contact author.
1022 pixels
146 pixels
768
pixe
ls
2“Unused”
pixels
DMD(1024x768 pixels)
7th OrderHadamard
Cyclic S-Matrix
Figure 7. Each row of the cyclic S-matrix corresponds to a Hadamard masking pattern. The encoding of the DMD involves turning the micro-mirrors “on” or “off” to generate the mask. The pattern is interpolated in both the x- and y-directions to fill the DMD.
λ1
λ2
λ3
λ4
Masking Pattern acrossDMD
...
...
... Figure 8. Representation of the wavelength slices in the simulation being encoded by a masking pattern. Each wavelength slice, for each diffraction order, is offset (dispersed) on the DMD according to the value calculated in Eqn. 2.
47
4. RESULTS AND DISCUSSION
The input image for the simulation code was a 32 pixel (spatial) x 32 pixel (spatial) x 256 wavelength (0.5-5.0 µm) hyperspectral image cube generated by Synthetic Scene Generation Model version 99.2 that integrates databases for generating simulated remote sensing data cubes. One of the first observations noted when running the simulation, was the appearance of strong banding in the output image slices as the Hadamard mask order was increased. The banding was caused by data aliasing as the element width decreased with an increase in Hadamard order. In a real HTSI, the dispersed analog signal would have an infinite number of wavelengths per mask element. The banding was effectively decreased, with an increase in Hadamard mask order, by interpolating the wavelength axis of the input data out to a larger number. Figure 9A is a wavelength slice of the input image cube. Figure 9B is the same data cube run through the HTSI simulation with a mask order of 59. Figure 9C is the result calculated by the simulation when the Hadamard order was increased to 127 and a strong banding pattern was observed in a wavelength slice of the transformed Hadamard image. By interpolating the 32x32x512 input image out to 32x32x1024, the HTSI simulation with a mask order of 127 (Figure 9D) showed a marked decreased in
the apparent banding in the image slice. The smoothing of data in the x- and y-spatial dimensions observed between the input image cube and the output data cubes in Figure 9 resulted from data interpolation in the model. The spatial 32x32 dimensions of the input data are spread to 256x256 pixels on the simulated detector.
Figure 10 is series of pixel spectra illustrating the effect of various model parameters on the Hadamard transformed output with a mask order of 59. Figure 10A is the pixel spectrum for the input data and Figure 10B is the spectrum undersampled at a factor (~17), approximately equivalent to the DMD pixel length divided by the Hadamard order. The geometry of the optical design was optimized via input parameters so that the 1st order diffraction wavelength slices (0.5-5.0 µm) are dispersed across the entire DMD. Under those conditions, the best result that can be achieved with the Hadamard model would be the input data cubed undersampled or averaged by a factor of ~17. The output of the Hadamard simulation without added noise, rejection ratio or diffraction orders, achieved similar results (Figure 10C) as the undersampled input.
The model can also be given values for the DMD rejection ratio and detector noise. The DMD rejection ratio for the current version of the simulation is single-value input. As Figure 10D demonstrates, using the lowest DMD rejection ratio shown in Figure 5 does not degrade the spectral information. Advanced modeling of the DMD will include the functional dependence of the rejection ratio versus wavelength in order to ascertain the effect of a non-constant value. A more dramatic effect can be seen with the addition of detector noise to the HTSI model. Detector noise resulting in the spectrum shown in Figure 10E was calculated as a 1% additive error based on the maximum pixel intensity of the encoded data at the detector. The 1% additive error based on the maximum value may be too large, but the output data illustrates the importance of minimizing detector noise. Maximizing the signal-to-noise ratio in a multiplexing instrument like the HTSI requires that the detector noise be independent of the measurand. As DeVerse and coworkers showed for a dispersive flat-
B C D
AFigure 9. Image slices of hyperspectral data cubes (A)
32x32x512 data input to simulation, (B) Resultant output data of simulation and Hadamard Order of 59 (C) Same as B, except Hadamard Order of 127, (D) Output data with input data interpolated to 1024 wavelength slices, Hadamard Order of 127.
48
field near-IR spectrometer, if the detector noise is independent, the Hadamard instrument will have better signal-to-noise than a slit spectrometer13.
Another source of error for the HTSI is the possible overlap of 2nd order diffraction on the DMD. Figure 10F represents the HTSI model with 2nd order diffraction light applied across the DMD in addition to the 1st order light. The intensity of light for the 1st order was set to 60% of the input while the 2nd order diffraction was given 10% of the input energy. The remaining 30% were assumed to be present in the zero order diffraction light. For these results, the DMD was
geometrically placed to effectively eliminate zero order light hitting the DMD. The spectrum suffered a loss in resolution with the addition of 2nd order light. The optical design and grating specifications will be modified to minimize the deleterious effects of zero and 2nd order diffraction.
Given the 256x256-pixel dimension of the detector, the hypothesis is that the maximum Hadamard matrix order that can be spatially detected by the 2D focal plane array is an order of 127. The assumption is that the complex alignment requirements in the imaging system will limit the focal plane array to detecting one Hadamard element vector per 2 pixel columns. Figure 11A is a pixel spectrum of the input data cube at a different point on the image than represented in Figure 10. At a Hadamard order of 127, the best spectral resolution that can be observed is the input data cube undersampled by a factor of ~8 and that benchmark is shown in Figure 11B. Figure 11C is the pixel spectrum at the same position and for a Hadamard order of 59 with input data cube dimensions of 32x32x512. The resolution improvement with a mask order of 127 with a 32x32x1024 input data cube is shown in Figure 11D. As discussed previously, the interpolation of the wavelength
Figure 10. Input and output data for HTSI simulation model with an input image with dimensions 32x32x512 (0.5 – 5.0 µm) and Hadamard mask order of 59, (A) Spectrum at single pixel of input image (B) Spectrum of single pixel of input image undersample at factor ~17, (C) Output data with no rejection ratio or detector noise (D) Output data with rejection ratio added to simulation, (E) Output data with detector noise added to simulation, (F) Output data with 2nd order diffraction light added to simulation.
Hadamard Mask Order = 59 Detector Noise AddedDMD Rejection Ratio=200
A B
D C E
F
2nd Order Diffraction Added
49
axis from 512 to 1024 was necessary when increasing the mask order in order to avoid banding due to aliasing. Figures 11C and 11D are data from the HTSI simulation with 1% additive detector noise and a DMD rejection ratio of 200 included with the input parameters. If the detector noise is removed from the simulation (Figure 11E), the output’s resolution is close to the undersampled (benchmark) spectrum shown in Figure 11B. Again, the result reinforces the advantage of minimizing measurand-dependent detector noise in the HTSI.
5. CONCLUSIONS The work presented thus far for an HTSI illustrates the complexity of an optical design for a non-raster imaging
system utilizing a DMD and a 2D detector array. Analysis of various designs indicated that an Offner relay-type system would provide the HTSI with the necessary 1:1 ratio and flexibility for baffling stray light effects. The DMD will need to be qualified over 1.25-2.5 µm with a single optimization that maximizes the rejection ratio across the entire spectral region. The HTSI simulation results indicate that the current dispersion, masking and refocusing systems in the code are generating expected results under ideal conditions. Basically, the encoding and transform of the encodegrams is mathematically correct. A constant DMD rejection ratio at reasonably expected values does not significantly degrade the simulation results. However, a functional rejection ratio (vs. wavelength) may be detrimental and warrants further investigation.
The major criteria for implementation of the multiplexing advantage in spectroscopy is that the detector noise be independent of the total signal reaching the detector. By adding a small percentage of detector noise that is dependent on the total signal reaching the detector, the simulation demonstrated the degree of spectral degradation we can expect. Further work must include minimizing the measurand-dependent detector noise, verifying the actual noise (constant or functional?) and including the detector characteristics in the simulation.
The impetus for developing the HTSI simulation was as a tool to illustrate how a Hadamard imaging spectrometer operates. As the HTSI instrument development progresses, the model has become useful in providing feedback regarding the potential deleterious or advantageous effects of changes in the design parameters. Future changes in the optical design, including an analysis of stray light and rotating the DMD 45°, will be incorporated into the simulation to provide realistic expectations of their effect on the HTSI performance before the instrument is completely assembled. Likewise, as individual components in the optical design are characterized and optimized, the actual values (i.e. DMD rejection ratio vs. wavelength, grating characteristics, detector noise) will also be included in the model. As the data acquisition and processing
Hadamard Mask Order=127 No Detector NoiseHadamard Mask Order=59
A B
C D E
Figure 11. Input and output data for HTSI simulation model at two Hadamard mask orders (A) Spectrum at single pixel of input image 32x32x1024 (0.5 – 5.0 µm) (B) Spectrum of single pixel of input image undersample averaged at ~8, (C) Output data at Hadamard Order = 59 (32x32x512 input data) and detector noise added, (D) Output data at Hadamard order = 127 (32x32x1024 input data) and detector noise added, (E) Same as (D) with no detector noise added.
50
system progresses, the ability to produce non-transformed encodegrams for testing those particular components of the HTSI will be an additional advantage of the simulation.
ACKNOWLEDGMENTS
We gratefully acknowledge the work by Mike K. Hinckley and Craig M. Boney, both at Sandia National Laboratories, in characterizing the rejection ratio of the Texas Instruments Digital Light Processor (DLP) Micro-mirror Device. Sandia National Laboratories is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under contract DE-AC04-94AL85000.
REFERENCES 1. M. Harwit and N. J. A. Sloane, Hadamard Transform Optics, p. 3, Academic Press , New York, 1979. 2. A. A. Michelson, Light Waves and Their Uses, Univ. of Chicago Press, Chicago, 1902 reprinted 1961. 3. P. R. Giffiths, “Interferometry in the Seventies,” Analyt. Chem., 46, 645A-654A, 1974. 4. W. G. Fateley, R. M. Hammaker, J.V. Paukstelis, S. L. Wright, E. A. Orr, A. N. Mortensen and K. J. Latas, “Application
of a 2-Dimensionsl Hadamard Encoding Mask for the Imaging of Thin-Layer Chromatography Plates by Laser-Induced Fluorescence or Surface-Enhanced Raman-Scattering and for Use with a Photoacoustic Detector to Generate 3-Dimensional Photoacoustic Images”, Appl. Spectrosc., 47, pp. 1464-1470, 1993.
5. R. M. Hammaker, A. N. Mortensen, E. A. Orr, M. K. Bellamy, J. V. Paukstelis and W. G. Fateley, “Multidimensional Hadamard-Transform Spectrometry”, J. Mol. Struc., 348, pp. 135-138, 1995.
6. M. Harwit and N. J. A. Sloane, Hadamard Transform Optics, pp. 109-180 and 229-243, Academic Press , New York, 1979.
7. R. M. Hammaker, J. A. Graham, D. C. Tilotta and W. G. Fateley, in J. R. Durig (Ed.), Vibrational Spectra and Structure Vol. 15, pp. 401-485, Elsevier, Amsterdam, 1986.
8. R. M. Hammaker, A. P. Bohlke, R. D. Freeman, J. M. Jarvis, K. J. Latas, D. Lin-Vien, A. M. Mortensen, E. A. Orr, J. D. Tate, D. C. Tilotta, J. S. White, S. L. Wright, J. V. Paukstelis and W. G. Fateley, in: J. J. Laserna (Ed.), Modern Techniques in Raman Spectroscopy, pp. 143-226, Wiley, Chichester, 1996.
9. M. Harwit and N. J. A. Sloane, Hadamard Transform Optics, pp. 12-19, Academic Press , New York, 1979. 10. Quentin S. Hanley, Peter J. Verveer and Thomas M. Jovin, “Spectral Imaging in a Progammable Array Microscope by
Hadamard Transform Fluorescence Spectroscopy”, Appl. Spectrosc., 53, pp. 1-10, 1999. 11. M. K. Bellamy, A. N. Mortensen, R. M. Hammaker and W. G. Fateley, “Chemical Mapping in the Mid- and Near-IR
Spectral Regions by Hadamard Transform/FT-IR Spectrometry,”, Appl. Spectrosc., 51, pp. 477-486, 1997. 12. R. A. DeVerse, T. A. Mangold, R. M. Hammaker and W. G. Fateley, in: J. A. de Haseth (Ed.), Fourier Transform
Spectroscopy Eleventh International Conference, AIP Conference Proceedings 430, pp. 443-446, AIP, Woodbury, 1998. 13. R. A. DeVerse, R. M. Hammaker and W. G. Fateley, “Realization of the Hadamard Multiplex Advantage Using a
Programmable Optical Mask in a Dispersive Flat-Field Near-Infrared Spectrometer,” Appl. Spectrosc., 54, pp. 1751-1758, 2000.
14. R. A. DeVerse, R. M. Hammaker and W. G. Fateley, “Hadamard transform Raman imagery with a digital micro-mirror array,” Vib. Spectrosc., 19, pp. 177-186, 1999.
15. R. A. DeVerse, R. M. Hammaker and W. G. Fateley, “An improved Hadamard encoding mask for multiplexed Raman imaging using single channel detection,” J. Mol. Struc., 521, pp. 77-88, 2000.
16. Qentin S. Hanley, Peter J. Verveer and Thomas M. Jovin, “Optical Sectioning Fluorescence Spectroscopy in a Programmable Array Microscope,” Appl. Spectrosc., 52, pp. 783-789, 1998.
17. Kimberly D. Barnes, Nikhil Taluja, Ruth A. Dyer and Stephen A. Dyer, “Development and Implementation of a General-Purpose Hadamard-Transform Spectrometer Simulation Program,” IEEE Transactions on Instrumentation and Measurement, 47, pp. 163-167, 1998.
18. Warren J. Smith (Ed.), Modern Optical Engineering, p.154, McGraw-Hill, New York, 1990. 19. D. Kwo, G. Lawrence andM. Chrisp, “Design of a Grating Spectrometer from a 1:1 Offner Mirror System,” Current
Developments in Optical Engineering II, R. E. Fisher and W. J. Smith, Eds., Proc. SPIE 818, pp. 275-278, 1987.
51
Appendix B “Theoretical description and numerical simulations of a simplified Hadamard transform imaging spectrometer”
Theoretical description and numerical simulations of a simplified Hadamard transform imaging spectrometer
Mark W. Smith*, Jody L. Smith, Geoffrey K. Torrington, Christine M. Wehlburg, and Joseph C. Wehlburg, Sandia National Laboratories, 1515 Eubank SE, Albuquerque, NM 87185†
ABSTRACT A familiar concept in imaging spectrometry is that of the three dimensional data cube, with one spectral and two spatial dimensions. However, available detectors have at most two dimensions, which generally leads to the introduction of either scanning or multiplexing techniques for imaging spectrometers. For situations in which noise increases less rapidly than as the square root of the signal, multiplexing techniques have the potential to provide superior signal-to-noise ratios. This paper presents a theoretical description and numerical simulations for a new and simple type of Hadamard transform multiplexed imaging spectrometer. Compared to previous types of spatially encoded imaging spectrometers, it increases etendue by eliminating the need for anamorphically compressed re-imaging onto the entrance aperture of a monochromator or spectrophotometer. Compared to previous types of spectrally encoded imaging spectrometers, it increases end-to-end transmittance by eliminating the need for spectral re-combining optics. These simplifications are attained by treating the pixels of a digital mirror array as virtual entrance slits and the pixels of a 2-D array detector as virtual exit slits of an imaging spectrometer, and by applying a novel signal processing technique. Keywords: Hadamard transform, imaging spectrometer, spectral imager, hyperspectral
1. INTRODUCTION A variety of terms, such as imaging spectrometry, spectral imaging, hyperspectral imaging, etc., are used to describe the process of acquiring data that provides a continuous spectrum for each pixel in an image. The resulting data set can be conceived of as a three dimensional data cube, with one spectral and two spatial dimensions. However, available detectors have at most two dimensions, which generally leads to the introduction of either scanning or multiplexing techniques for imaging spectrometers. The scanning approach is typified by the Hyperspectral Digital Imagery Collection Experiment (HYDICE) system1. HYDICE is an airborne imaging spectrometer that uses a telescope to image a strip of ground onto the entrance slit of a prism spectrometer. The long axis of the slit is oriented in the cross-track direction (relative to the motion of the aircraft) and images are built up line by line as the aircraft’s motion creates a push broom scan.
However, for situations in which noise increases less rapidly than as the square root of the signal, multiplexing techniques have the potential to provide signal-to-noise ratios (SNRs) that are superior to those produced by scanning systems. The SNR advantage of Hadamard multiplexed operation relative to scanned slit operation has been demonstrated both theoretically2,3 and experimentally4-6. While the first clear exposition of the spectral multiplexing advantage is often attributed to Fellgett7 in connection with Fourier transform spectrometry, two papers that described multiplexing techniques in the context of multi-slit dispersive instruments were published by Golay at about the same time8,9. Golay presented the fundamental concepts for applying binary orthogonal sequences to spectral multiplexing and demonstrated experimental confirmation of these ideas. These techniques were rediscovered in 1968 and thereafter rapidly gained recognition under the label “Hadamard encoding”10,11. Most so-called Hadamard encoding techniques are actually based on the use of S-matrices12, which are closely related to Hadamard matrices. The rows in an S-matrix are code words in a simplex code, and are sometimes alternatively called pseudo-random sequences or pseudo-random noise13. All of these are binary sequences that obey certain orthogonality relations. This paper uses the term Hadamard encoding in order to comply with common usage. The use of binary orthogonal sequences to encode multiplexed information is an extremely general technique that finds many applications. 2-D spatial encoding is especially germane for spectral imaging. Gottlieb proposed in 1968 the application of binary pseudo-noise codes to spatially multiplexed television scanning14. A host of spatial multiplexing applications have been proposed and implemented since that time. The most straightforward way to implement a multiplexed imaging spectrometer is to separate the spatial and spectral encoding from each other. A prototypical system was developed around 1976 by Swift and colleagues15. This system replaced the entrance slit of a spectrograph with a 2-D spatial encoding mask, and placed a second 1-D spectral encoding mask at the normal location of the exit aperture. Radiation was then spectrally re-combined by making a near retro-reflective pass through the spectrograph, and was then finally imaged onto a detector. The spatial and spectral encoding masks were stepped individually to build up a complete 3-D spatial-spectral data cube. The systems described by Treado and Morris16 and Chen et al.17 imaged a 2-D spatial encoding mask pattern onto the entrance slit of a monochromator. The monochromator was spectrally scanned, rather than being spectrally multiplexed. Bellamy and colleagues imaged a 2-D spatially encoded mask pattern onto the entrance aperture of a Fourier transform interferometer18. All of these techniques used a detector with only a single element. Other approaches make use of 2-D detector arrays. A typical instrument built along these lines uses a barcode type pattern of lines (which form cyclical simplex sequences) to encode spatial information in a dimension that is perpendicular to the entrance slit of a spectrograph. Anamorphic optics are then used to re-image the scene onto the entrance slit or aperture of the spectrograph19,20. This is similar to the HYDICE configuration, but with the addition of spatial multiplexing in one dimension. Because spatial and/or spectral encoding can be employed in combination with single element, linear array, or 2-D array detectors, many variations on the Hadamard multiplexing theme are possible. A group from Sandia National Laboratories proposed using a Texas Instrument digital micro-mirror device (DMD) to perform spectral encoding at the output of an Offner relay that used a holographic diffraction grating to produce spectral dispersion. A second Offner relay,
53
also incorporating a holographic grating, would then be used to spectrally recombine the data and produce a spectrally multiplexed image on a 2-D detector array21. The work that will be presented in this paper results from the realization that, given a 2-D array detector, neither anamorphic image compression nor spectral recombination are strictly necessary in order to apply Hadamard encoding as a multiplexing technique in imaging spectrometry. This permits a great simplification in optical design. More importantly, there should be a direct increase in system throughput because the etendue (product of area and solid angle) and/or the end-to-end optical transmittance can be increased. The penalties are a minor increase in the complexity of the decoding algorithm, and the fact that spectral resolution becomes directly linked to spatial resolution.
54
2. THEORY
2.1 Generic instrument configuration A generic configuration of the proposed new system (see Figure 1) first images the input scene onto a digital mirror array (DMA). The DMA is used to impose a cyclical Hadamard (or more properly a cyclical simplex) encoding along one spatial dimension of the 2-D input scene. This dimension is parallel to the direction in which the dispersion will later be applied. The use of mirrors to implement the encoding permits, but does not require, the use of two separate detection channels, since radiation that is switched away from one channel can be switched towards the second channel. General considerations related to the use of DMAs (or DMDs) as Hadamard encoding masks have been presented in the literature20-22. The primary channel is the dispersed image channel, which will be comprised of a set of overlapping, wavelength dispersed images. This channel carries mixed spatial and spectral information. A secondary channel is an optional panchromatic channel. While this channel is not required, it does add some operational flexibility, and makes use of signal power that otherwise would simply be lost. This second channel could be used to correct for temporal variations in the scene illumination that occur during the encoding sequence. Photons that arrive at the primary and secondary image planes are detected by independent 2-D focal plane array (FPA) detectors. The FPA for the dispersed channel must be somewhat oversized in the direction in which the dispersion is applied. The number of columns in this FPA must be at least equal to the sum of the number of columns in the final hyperspectral image cube plus the number of pixels of dispersion minus 1.
2.3 Hadamard encoding and forward equations The Hadamard encoding that is imposed by the DMA permits the unambiguous separation of spatial and spectral information that is present in the primary channel. The way in which this is accomplished can be explained by referring to Figure 2, which is a cross sectional view through the dispersed 3-D spatial-spectral data space. The vertical dimension corresponds to wavelength, and the horizontal dimension is the spatial dimension parallel to the dispersion (the second spatial dimension, which is not shown, is perpendicular to the dispersion
Figure 3 Generic layout for proposed new Hadamard transform imaging spectrometer
Figure 4 Schematic illustration of dispersion in the Hadamard transform imaging spectrometer.
55
and to the page). The top object in Figure 2 represents the image at the DMA plane, before it is dispersed. The objects labeled as red image sheet, green image sheet, and blue image sheet represent the images that correspond to particular spectral bands (with only 3 bands shown for the sake of simplicity) after dispersion. The effect of the dispersing optics is to offset each of these image sheets by a distance that is equal to the amount of linear dispersion experienced by each wavelength. The diagonal arrow in Figure 2 traces the polychromatic radiation that originates at pixel k in the DMA image plane as it is dispersed prior to arriving at the FPA. Of course the FPA does not respond to each dispersed image sheet separately; it responds to the sum of all the image sheets. This is a projection down the vertical arrow in Figure 2. This can be expressed mathematically as:
∑=k
kkhkhk ISR '' (1)
where =Rhk ' Response of FPA pixel k’ for state h of the Hadamard mask
=Shk Element h of the Hadamard (simplex) code word for DMA pixel k
=I kk ' Radiance that originated at DMA pixel k and arrived at FPA pixel k’ Summation extends over all DMA pixels that can contribute photons to FPA pixel k’. The Hadamard (simplex) code words must be cyclic and they must contain at least as many elements as are included in the summation.
2.3 Inverse equations The FPA is read out once for each element of the Hadamard encoding sequence, or once for each state of the DMA. This produces a sequence of data frames that carry mixed spatial and spectral information. The spatial and spectral information are separated using the following inversion technique:
∑ −=h
hjjhjj RSI '1
'ˆ (2)
where =I jjˆ ' Inverted radiance that originated at DMA pixel j and arrived at FPA pixel j’
=−S jh1 Element h of the inverse Hadamard code word for DMA pixel j
=Rhj' Response of FPA pixel j’ for state h of the Hadamard mask
Summation extends over the sequence length of the Hadamard (simplex) encoding. That is, the length of the inverse code words is equal to the length of the forward code words. We can verify that Eq. 2 does in fact recover Ijj’ by substituting Eq. 1 for Rhj’ in Eq. 2:
56
'1
'ˆ ISSI kjhkjhkh
jj−= ∑∑ (3)
The simplex code words and their inverses have a Kronecker delta orthogonality property24:
1 δ=∑ −jk
hhkjh SS (4)
and therefore:
III jjkjk
jkjj ' ''ˆ == ∑δ (5)
The inversion process is able to recover unambiguously the quantity Ijj’, which is the radiance that originated at DMA image pixel j and arrived at FPA detector pixel j’. By referring back to Figure 2 we see that this quantity is the radiance from DMA image pixel j that underwent a dispersion equal to ∆j = j’ – j. There is a direct correspondence between wavelength and amount of dispersion (although the exact relationship must be determined by instrument calibration). So, knowing the amount of dispersion, ∆j, is equivalent to knowing the wavelength to assign to Ijj’. The inversion process eliminates the need for actual spectral recombination since it allows the unambiguous separation of spatial information (subscript j) and spectral information (subscript j’ = j + ∆j, where ∆j is wavelength dependent) from the mixed spatial-spectral data frames. The DMA pixels can be regarded as virtual entrance slits, and the FPA pixels as virtual exit slits.
3. NUMERICAL SIMULATIONS Numerical simulations were carried out using MATLAB to verify that the algebra described above actually works when applied to a 3-D data cube of the type that could be produced by a multiplexed, dispersive, imaging spectrometer. The original input data cube, with a size of 32 pixels by 32 pixels by 256 wavelength bands (0.35–3.5 um), was generated using version 99.2 of the synthetic scene generation model (SSGM) . (SSGM was written by the Naval Research Laboratory). This cube was expanded to 64 pixels by 64 pixels by 512 wavelength bands for the purpose of the simulations. Figure 3 shows an image slice taken at the first wavelength band of the input data cube. The Hadamard (simplex) code words were generated using a maximal length shift register sequence24. This produces cyclical S-matrices whose rows are simplex code words of length M = 2N-1. Simulations were run for M = 63 and M = 127. The dispersing
Figure 5 Image slice taken at the first wavelength band (0.35 um) of the input data cube, which was generated using SSGM.
57
element was modeled as a diffraction grating operating in the 1st order. In each case, the groove density was chosen to provide one pixel of linear dispersion for each wavelength band (or 63 pixels of total dispersion for M = 63 and 127 pixels of total dispersion for M = 127). The purpose of these simulations was to demonstrate, by means of explicit numerical calculations, the correctness of the ideas that were formulated analytically in section 2. The numerical simulations did not include the effects of noise, aberrations and optical distortion, diffraction and scattering of stray light by the DMA, or other instrument imperfections. The general capability of producing accurate results in the presence of realistic amounts of noise using Hadamard transform techniques has been demonstrated experimentally by other groups 4-6. The impact of various types of mask imperfections has also been studied and reported14,18,26.
4. RESULTS AND DISCUSSION Figure 4 shows the an image slice taken at the first wavelength band (0.35 um) for the M = 127 simulation. The location of individual features is very good in comparison to the original scene, which is shown in Figure 3. Figure 5 shows spectra for the middle pixel of the scene for the spectral range of 0.5-1.5 microns. The solid line shows the original spectrum. The open circles show the inverted spectrum. The crosses show the difference (inverted minus original) spectra. The original spectrum was higher resolution (512 wavelength
bands) than the inverted spectrum (127 wavelength bands), and the results around 0.7-0.9 microns reflect this. The system proposed in this paper is superficially similar to the one developed by Hanley et al. for microscopy work, in so far as the Hadamard encoding is done using a 1-D barcode type pattern in a spatial dimension that is perpendicular to the dispersion20. However, the technique of Hanley et al. requires the use of anamorphic optics to compress the encoded dimension of the scene down to the width of the entrance aperture of a spectrograph. The technique proposed here does not require this anamorphic compression step, and allows the dispersing optics to collect signal radiation from an extended area, rather than from a narrow slit. This will
Figure 6 Inverted image slice taken from the M = 127 simulation.
Figure 7 Inverted spectrum for the M = 127 simulation.
58
translate directly into a throughput advantage, since the input area can be much larger for a given solid angle of collection. Furthermore, the inversion algorithms employed by Hanley et al. required a multiple step inversion process, with the final step being the application of a wavelength shift correction in order to produce spectrally registered images. This is more complicated than the single step inversion described by Eq. 2 in this paper. Figure 1 suggests a concave holographic diffraction grating as the single optical element that is used for both dispersion and reimaging in Channel 1. However, this could alternatively be an Offner relay with a convex grating, or refractive optics with a prism, or any other optical system that performs dispersion plus imaging. The key point is that, relative to the system proposed earlier by the Sandia group21, about half of the optics are eliminated. Eliminating half of the optics carries several advantages. First, the end-to-end system transmittance will be significantly higher. This is especially important when one considers that the efficiency of holographically produced diffraction gratings is generally only 25% - 50% over an extended spectral range. Eliminating one grating therefore doubles to quadruples the amount of radiation that is transmitted to the FPA. Second, the system can be packaged into a significantly smaller volume and will also be lighter. This is important for any application in which weight and volume are at a premium. Third, simple systems are generally cheaper to manufacture and more robust in operation than complicated systems. There are various options for achieving variable resolution with the proposed new system, if so desired. One option is to operate the DMA pixels in clusters of 2x2, or 4x4, etc. and to decrease the Hadamard order correspondingly. For example, the instrument might be operated at a Hadamard order of M = 127 with individual DMA pixel switching, at an order of M = 63 with 2x2 clustered switching, and at an order of M = 31 with 4x4 pixel switching. Note that because M decreases slightly more quickly than a simple factor of 2, it would be necessary to set the dispersion based on the lowest resolution operation, and this would result in a slight zero padding of the last few spectral elements of the higher orders. The time for acquisition of a complete spectrum would be cut approximately in half with each doubling of the DMA cluster size, and the data volume would be reduced by a factor of approximately 8. While the spatial resolution in the dimension parallel to the dispersion is necessarily linked to the spectral resolution, the spatial resolution in the orthogonal dimension could, in principle, be maintained at a single value. Alternatively, the full spatial resolution could be maintained in both dimensions and the variable spectral resolution could be introduced via variable dispersion. This could be achieved, for example, by using counter-rotating prisms. Another approach would be to use a limited set of prisms in a filter wheel type of assembly, and select the prism based on the spectral resolution that is desired.
4. SUMMARY AND CONCLUSIONS A theoretical description and numerical simulations for a new and simple type of Hadamard transform imaging spectrometer have been presented. Compared to previous types of spatially encoded imaging spectrometers, such as the one described by Hanley et al.20, it increases etendue by eliminating the need for anamorphically compressed re-imaging onto the entrance aperture of a monochromator or spectrophotometer. Compared to previous types of spectrally encoded imaging spectrometers, such as the one described by Wehlburg et al.21, it increases end-to-end transmittance by eliminating the need for spectral re-combining optics. These simplifications and performance improvements are attained by treating the pixels of a digital mirror array as
59
virtual entrance slits and the pixels of a 2-D array detector as virtual exit slits of an imaging spectrometer, and by applying a novel signal processing technique. While this approach couples the spatial and spectral resolution of the instrument, various options exist for varying the spectral resolution. Simple systems are generally cheaper to manufacture and more robust in operation than complicated systems, which are both points in favor of the proposed new design. The actual construction of an instrument that operates according to the principals outlined here is underway at Sandia National Laboratories (Patent pending).
ACKNOWLEDGEMENTS Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under contract DE-AC04-94AL85000.
REFERENCES 1. R. W. Basedow, D. C. Carmer, and M. E. Anderson, “HYDICE System, implementation and performance”, Proc. SPIE, 2480, pp. 258-267 (1995). 2. M. Harwit and N. J. A. Sloane, Hadamard Transform Optics, pp. 96-108, Academic Press, New York, 1979. 3. B. Carli and V. Natale, “Efficiency of spectrometers”, Appl. Opt., 18, pp. 3954-3958 (1979). 4. J. A. Decker, Jr., “Experimental Realization of the Multiplex Advantage with a Hadamard-Transform Spectrometer”, Appl. Opt., 10, pp. 510-514 (1971). 5. P. Hansen and J. Strong, “High Resolution Hadamard Transform Spectrometer”, Appl. Opt., 11, pp. 502-506 (1972). 6. R. A. DeVerse, R. M. Hammaker, and W. G. Fateley, “Realization of the Hadamard Multiplex Advantage Using a Programmable Optical Mask in a Dispersive Flat-Field Near-Infrared Spectrometer”, Appl. Spectrosc., 54, pp. 1751-1758 (2000). 7. P. Fellgett, J. Phys. Radium, 19, pp. 187 (1958). 8. M. J. E. Golay, “Multi-Slit Spectrometry”, J. Opt. Soc. Am., 39, pp. 437-444 (1949). 9. M. J. E. Golay “Static Multislit Spectrometry and Its Application to the Panoramic Display of Infrared Spectra”, J. Opt. Soc. Am., 41, pp. 468-472 (1951). 10. R. N. Ibbett, D. Apinall, J. F. Grainger, “Real-time Multiplexing of Dispersed Spectra in Any Wavelength Region”, Appl. Opt., 7, pp. 1089-1093 (1968). 11. J. A. Decker, Jr. and M. O. Harwitt, “Sequential Encoding with Multislit Spectrometers”, Appl. Opt., 7, pp. 2205-2209 (1968). 12. Ibid., M. Harwit and N. J. A. Sloane, Hadamard Transform Optics, pp. 1-19. 13. Ibid., M. Harwit and N. J. A. Sloane, Hadamard Transform Optics, pp. 14. 14. P. Gottlieb, “A Television Scanning Scheme for a Detector-Noise-Limited System”, IEEE Trans. Inform. Theory, IT-14, pp. 428-433 (1968). 15. R. D. Swift, R. B. Wattson, J. A. Decker, Jr., R. Paganetti, and M. Harwit, “Hadamard transform imager and imaging spectrometer”, Appl. Opt., 15, pp. 1595-1609 (1976). 16. P. J. Treado and M. D. Morris, “A Hadamard Transform Raman Microprobe”, Appl. Spectrosc., 43, pp. 190-193 (1989). 17. G. Chen, E. Mei, W. Gu, X. Zen, Y. Zeng, “Instrument for Hadamard transform three-dimensional fluorescence microscope image analysis”, Analyt. Chim. Acta, 300, pp. 261-267 (1995).
60
18. M. K. Bellamy, A. N. Mortensen, R. M. Hammaker, W. G. Fateley, “Chemical mapping in the Mid- and Near-IR Spectral Regions by Hadamard Transform/FT-IR Spectrometry”, Appl. Spectrosc., 51, pp. 477-486 (1997). 19. P. J. Treado, A. Govil, M. D. Morris, K. D. Sternitzke, R. L. McCreery, “Hadamard Transform Raman Microscopy of Laser-Modified Graphite Electrodes”, Appl. Spectrosc., 44, pp. 1270-1275 (1990). 20. Q. S. Hanley, P. J. Verveer, T. M. Jovin, “Spectral Imaging in a Programmable Array Microscope by Hadamard Transform Fluorescence Spectroscopy”, Appl. Spectrosc., 53, pp. 1-10 (1999). 21. C. M. Wehlburg, J. C. Wehlburg, S. M. Gentry, and J. Smith, “Optimization and Characterization of an Imaging Hadamard Spectrometer”, Proc. SPIE, 4381, pp. 506-515 (2001). 22. R. A. DeVerse, R. M. Hammaker, W. G. Fateley, J. A. Graham, and J. D. Tate, “Spectrometry and imaging using a digital micromirror array”, Amer. Labor., pp. 112S-120S (1998). 23. Ibid., M. Harwit and N. J. A. Sloane, Hadamard Transform Optics, pp. 12-19. 24. Ibid., M. Harwit and N. J. A. Sloane, Hadamard Transform Optics, pp. 205-208. 25. D. C. Tilotta, R. M. Hammaker, and W. G. Fately, “Multiplex advantage in Hadamard transform spectrometry utilizing solid-state encoding masks with uniform, bistable optical tranmission defects”, Appl. Opt., 26, pp. 4285-4292 (1987).
61
Appendix C (Optical Design Report) Final Report for the Preliminary Optical Design of an Imaging Spectrometer for Remote Sensing Applications
September 3, 2002 Ms. Jody Smith SANDIA NATIONAL LABORATORY Remote Sensing and Data Exploitation PO Box 5800 Albuquerque NM 87185 Subject: Final Report for the Preliminary Optical Design of an Imaging
Spectrometer for Remote Sensing Applications References: 1. Telcon, J. Smith/Sandia to B. Crowther/ORA on 05/03/02. 2. Telcon, J. Smith/Sandia to B. Crowther/ORA on 05/07/02.
3. Telcon, J. Smith/Sandia to B.Crowther and J. Rogers/ORA on 05/03/02. 4. ORA Statement of Work dated, 06/05/02. 5. Telcon, J. Smith/Sandia and B. Crowther/ORA on 06/25/02. 6. E-mail, L. Foo/ORA to J. Smith/Sandia on 08/23/02. 7. Telcon, L. Foo/ORA and J. Smith/Sandia on 08/23/02. 8. E-mail, L. Foo/ORA to J. Smith/Sandia on 08/26/02. 9. E-mail, L. Foo/ORA to J. Smith/Sandia on 09/03/02. Dear Ms. Smith: Optical Research Associates (ORA®) has enclosed the final report for the Preliminary Optical Design of an Imaging Spectrometer for Remote Sensing Applications. Please note that we completed this report using a frequency of 29.4 cy/mm for the tolerance analysis, as called for in the Reference 4 Statement of Work. However, that frequency was chosen for the analysis because it corresponds to the Nyquist frequency of a DMD with a pixel pitch of 17 µm, which was the plan at the time the SOW was written. Since then however, Sandia has selected a DMD with a pixel pitch of 13.68 µm, giving an actual Nyquist frequency of 36.5 cy/mm. We expect the as-built performance at 36.5cy/mm to be somewhat lower than at 29.4 cy/mm. (The nominal performance of the system at the updated frequency may be read from Figures 8 - 22.) ORA has calculated the as-built performance at 36.5 cy/mm and forwarded this information to Sandia in Reference 9.
62
This completes ORA’s work on the above-referenced project. Please review the enclosed report and do not hesitate to contact me if you have questions. Sincerely, OPTICAL RESEARCH ASSOCIATES Leslie D. Foo Senior Systems Engineer/Optical Design Optical Engineering Services E-mail: [email protected] LDFRSNLSP004:0903:rjm:e-mail,fed-ex cc: S. Chapman, R. J. Maushardt, J. Rogers, Ph.D./ORAW enc: Final Report, Table, Figures, Appendices
63
Prepared for: SANDIA NATIONAL LABORATORY REMOTE SENSING AND DATA EXPLOITATION FINAL REPORT PRELIMINARY OPTICAL DESIGN OF AN IMAGING SPECTROMETER FOR REMOTE SENSING APPLICATIONS September 3, 2002 Prepared by Leslie D. Foo Blake G. Crowther, Ph.D. ORA Project No. SNLSP004 OPTICAL RESEARCH ASSOCIATES 5210 East Williams Circle, Suite 510 Tucson, AZ 85711-4479
TABLE OF CONTENTS Figure 11 – MTF Plot 700 nm Position 1 Figure 12 – MTF Plot 700 nm Position 2 Figure 13 – MTF Plot 700 nm Position 3 Figure 14 – MTF Plot 600 nm Position 1 Figure 15 – MTF Plot 600 nm Position 2 Figure 16 – MTF Plot 600 nm Position 3 Figure 17 – MTF Plot 500 nm Position 1 Figure 18 – MTF Plot 500 nm Position 2 Figure 19 – MTF Plot 500 nm Position 3 Figure 20 – MTF Plot 450 nm Position 1 Figure 21 – MTF Plot 450 nm Position 2 Figure 22 – MTF Plot 450 nm Position 3 Figure 23 – Ray Fans 800 nm Position 1 Figure 24 – Ray Fans 800 nm Position 2 Figure 25 – Ray Fans 800 nm Position 3 Figure 26 – Ray Fans 600 nm Position 1 Figure 27 – Ray Fans 600 nm Position 2 Figure 28 – Ray Fans 600 nm Position 3 Figure 29 – Ray Fans 450 nm Position 1 Figure 30 – Ray Fans 450 nm Position 2 Figure 31 – Ray Fans 450 nm Position 3 Figure 32 – Full-Field Distortion Maps (DMD-to-CCD) Figure 33 – DMD in On-Off position and Light Trap LIST OF APPENDICES: Appendix I: CODE V Prescription and Fabrication Listing of the System Appendix II: Optical Performance Figures Appendix III: Preliminary Sensitivity Analysis Results Appendix IV: Preliminary Light Baffle Design Results Appendix V: Original ORA Statement of Work
66
FINAL REPORT PRELIMINARY OPTICAL DESIGN OF AN IMAGING SPECTROMETER FOR REMOTE SENSING APPLICATIONS (09/03/02) References: 1. Telcon, J. Smith/Sandia to B. Crowther/ORA on 05/03/02. 2. Telcon, J. Smith/Sandia to B. Crowther/ORA on 05/07/02. 3. Telcon, J. Smith/Sandia to B.Crowther and J. Rogers/ORA on 05/03/02. 4. ORA Statement of Work dated, 06/05/02. 5. Telcon, J. Smith/Sandia and B. Crowther/ORA on 06/25/02. 6. E-mail, L. Foo/ORA to J. Smith/Sandia on 08/23/02. 7. Telcon, L. Foo/ORA and J. Smith/Sandia on 08/23/02. 8. E-mail, L. Foo/ORA to J. Smith/Sandia on 08/26/02. 9. E-mail, L. Foo/ORA to J. Smith/Sandia on 09/03/02. 1.0 SUMMARY Sandia National Laboratory has requested support from Optical Research Associates (ORA) for support in developing an imaging spectrometer for remote sensing applications. A key component of this device is the Texas Instruments Digital Micromirror Device (DMD), which is used to selectively “turn on” or “turn off” portions of the image to allow the spectral nature of the images scene to be decoded. A preliminary optical design of the primary optical assemblies of this spectrometer consisting of an imaging objective lens, DMD, relay lens and dispersive grating, and a CCD imaging module have been defined. In particular, ORA has designed an imaging objective lens that images a scene onto the DMD array, which is focusable from 3 m to infinity. An 1:1 image relay system has also been designed to map the active area of the DMD onto a CCD detector array. The 1:1 image relay system contains a collimated region where a dispersive grating element is placed to produce spectral separation at the final image plane. ORA has also completed a preliminary sensitivity analysis of the primary optical assemblies of this spectrometer to determine the approximate difficulty of fabrication to Sandia and potential fabricators.
67
2.0 BACKGROUND A key component of the imaging spectrometer is the Texas Instruments Digital Micromirror Device (DMD). The DMD also complicates optical design of the spectrometer. As outlined in Reference 4, the DMD selected by Sandia deviates incident light at an angle of 24° with respect to the incoming light. This produces an image and/or object plane that is tilted with respect to the objective or relay optics. To mitigate the induced image tilt and distortion effects of the tilted object plane caused by the DMD, the conjugate image plane of the relay assembly must also be tilted according to the Scheimpflug condition. Optical systems which have tilted object and image planes often exhibit “keystone” distortion, where magnification is constant in one direction and varies linearly in the other direction. In addition, tilted plane systems often produce anamorphically distorted images in that the magnification in orthogonal directions are different. The difficulties and constraints that the DMD induced object tilt has on the system will be described later. 3.0 OBJECTIVE LENS DESIGN An F/7 objective lens has been designed to cover the field of view specified in Table 1. As shown in Figure 1, the objective lens consists of eight elements in three groups. Translating the entire lens assembly with respect to the DMD a total of 32.2 mm makes focus compensation for object distances from infinity to 3 meters. In order to achieve good optical image performance over a 450 nm – 800 nm spectral band, special dispersion glasses have been utilized, namely Schott Glass KZFSN2 and FK51. This combination of glasses helps to reduce the secondary color of the lens system over this wide spectral band. Optical performance of the objective lens is given in Figures 2-5 with spot diagrams, ray fan diagrams and polychromatic MTF all evaluated at the DMD plane. 4.0 SPECTRAL SEPARATION COMPONENT Two methods of providing spectral dispersion have been proposed for the DMD spectrometer system: a dispersing prism and a diffraction grating. Although a reflective diffraction grating seemed like a plausible solution, it was rejected because of the large number of closely spaced diffracted orders that would result from the grating structure. In order to provide 10 µm of spatial dispersion over a 10 nm spectral band using a lens with a focal length long enough to provide clearance at the diffraction grating, the frequency of the grating would have to be as low as 6.6 line/mm.
68
However, at a grating frequency of 6.6 lines/mm, the diffracted orders are separated by only 0.228°. Since the relay will collect light within a ±2.6° cone, more than 20 diffracted orders of light from the diffraction grating would be collected. This would result in an insurmountable stray light problem and would render the system useless as a spectrometer. The initial image relay configuration designed by B. Crowther used a single two-glass dispersing prism located at the pupil of the relay assembly in collimated space. This prism was achromatized over the 450 nm – 800 nm spectral band and provided 10 nm spectral resolution per CCD pixel. However, we discovered that the prism introduced a large amount of asymmetric distortion to the overall system. The distortion resulted from the asymmetry the prism introduced into the symmetrical relay configuration. To restore symmetry to the relay system, we re-designed the dispersion prism to be symmetrical about the stop and also produce no ray deviation at the central wavelength while also producing the required 10 µm of spatial dispersion over a 10 nm spectral band. A three-glass double prism configuration provided the solution to achieving the required dispersion while maintaining system symmetry to correct the anamorphic distortion seen in the earlier prism configuration. 5.0 IMAGE RELAY DESIGN As described in Section 2.0, the DMD presents a “tilted” object to the image relay system. To mitigate the image tilt produced by this tilted object plane, the CCD image plane of the relay must be tilted according to the Scheimpflug condition. Since the CCD has a gamma rotation of 45 degrees with respect to the DMD, any anamorphism would result in mapping square DMD pixels to diamond shaped pixels at the CCD image plane. This would also result in spatial and spectral information being “mixed” together and indistinguishable at the CCD image plane. Thus, not only must the relay operate according to the Scheimpflug condition, but it must also be free of anamorphic distortion so that spatial and spectral information remain orthogonal to each other at the CCD image plane. One of the easiest methods optical designers use to correct distortion is to use symmetry. A unity magnification optical system symmetrical about a centrally located stop is distortion free. This principle was maintained in designing the image relay and spectral dispersion component in this spectrometer. Requiring the image relay to operate at unity magnification places restrictions on the mapping of the DMD pixels to the corresponding CCD image plane pixels. The first choice would be to find a CCD with pixels that are exactly the same size as those of the DMD. If this can't be done, the task is still manageable, as we described in Reference 5.
69
First, it is important to note that choosing a magnification other than 1:1 is not a realistic option, as this would cause “mixing” of the spatial and spectral information, as we have described above. Therefore, to avoid Moiré effects between the DMD and the CCD in the spatial direction, the technique is to allow the DMD to determine the spatial mapping and let the CCD determine the spectral mapping. Pixel (spatial) contamination at the DMD is eliminated by "turning off" adjacent pixels in the same row. Once the corresponding spectral information is read and stored, turn the off pixels on and the on pixels off to read out the adjacent pixel spectral information. This can be viewed as an interlaced detection system similar to the interlacing done on a TV (or possibly computer monitor). The penalty is a slower image cube acquisition rate. Given that this is a proof of concept instrument, this scheme may be acceptable. 6.0 SYSTEM OPTICAL PERFORMANCE ANALYSIS The optical performance of the DMD spectrometer system was analyzed in terms of spot diagrams, ray aberration plots and MTF plots (at 29.4 cy/mm) at selected wavelengths as requested by Sandia in Reference 6 and delivered in Reference 7. This data is reproduced in Appendix II for your convenience. 7.0 PRELIMINARY SENSITIVITY ANALYSIS In order to determine the feasibility of fabricating the DMD spectrometer system, a preliminary sensitivity and optical performance analysis has been conducted. The DMD spectrometer system has been divided into three primary sub-sytems: the objective and the two identical halves of the relay. An optical component in each respective group has also been identified to provide lateral compensation in addition to longitudinal focus compensation at the DMD plane (objective group) or the CCD image plane ( relay groups). A total of 324 tolerance parameters with 10 possible compensators were evaluated to predict the as-built performance of the DMD spectrometer system. To reduce computation complexity, the as-built performance was measured as the MTF at 29.4 cy/mm, at a wavelength of 600 nm, at the center and four extreme corners of the field, and with the system focused for an object at infinity. These results are summarized in Appendix III. Note that 29.4 cy/mm corresponds to the Nyquist frequency of the DMD having the originally foreseen pixel pitch of 17 µm, as listed in the Reference 4 SOW. This frequency specification was not updated when Sandia selected a DMD having 13.68 µm pixels. The Nyquist frequency for 13.68 µm pixels is 36.5 cy/mm, and the as-built MTF at this frequency is expected to be somewhat worse than at 29.4 cy/mm. ORA has calculated the as-built performance at 36.5 cy/mm and forwarded this information to Sandia in Reference 9.
70
8.0 PRELIMINARY LIGHT TRAP AND BAFFLE DESIGN The DMD directs the incident light by tilting individual pixels ±12° with +12° for instance being the “on” position and -12° the “off” position. In the “on” position the DMD directs energy from the object scene into the relay assembly. In the “off” position it is desirable to “trap” the light reflected by the DMD inside an absorbing structure so that it does not contaminate or add unwanted noise to “on” position light. This light trap can be a fairly simple mechanical assembly shaped like a cone whose interior is coated with a highly absorptive black coating in the 400-800 nm spectral band . A layout of the DMD spectrometer with the DMD in the “off” position is shown in Figure 33 (in DMD On-Off Figure.ppt). The “off” beam is indicated in the figure and is “trapped” by an absorbing structure at the location shown. The Reference 4 SOW also mentions the degradation of image contrast due to diffraction from the periodic structure of the DMD. Using a wavelength of 0.575 microns and a “grating” period of 13.68 µm, one calculates a diffraction angle of 2.4 degrees. This diffraction angle must be applied to two different unwanted beams: the first being the light that strikes the interstices between the pixels, and the second being the light that strikes the “off” pixels. Assuming that the spaces between the pixels are 1 micron wide (known to be the case with DMDs having 17 µm nominal pitches), we calculate that the amount of energy striking the interstices is:
%14.00014.0)68.13/68.12(1 2 ==− This energy is diffracted into orders spaced every 2.4 degrees, according to a diffraction efficiency given by the duty cycle of the “grating.” Since the imaging beam lies 24 degrees off-axis, it will collect the light in orders 9, 10, and 11. For 1 µm reflective structures on spaces of 13.68 µm, we calculate that the diffraction efficiency will fall off as 0.052 x Sinc2(0.229 M), where M is the order number. (Note that the efficiency of the zeroth order is relatively low, because the fall-off is slow, and many orders contribute significantly to the power.) For orders 9, 10, and 11, the diffraction efficiency (relative to a total power of 1.0) is given by 0.96%, 0.56%, and 0.28%, respectively. Multiplying these diffraction efficiencies by the fractional power (14%) that strikes the interstices gives values for the collected scattered light of 0.134%, 0.078%, and 0.039%, respectively. The sum of these numbers (relative to the total energy falling on the detector) is 0.251%. However, considering that the entrance pupil is 2-dimensional, more than three orders are collected. A square grid of 3 x 3 orders would be a factor of 3 higher than this, but the energy contained in the inscribed circular pupil is smaller by a factor of pi/4, so an estimate for the collected stray light from the interstices is:
%59.00059.04/300251.0 ==×× π (This is likely to be a pessimistic estimate, since it assumes that the interstices are perfect planar reflectors.)
71
The light that strikes the pixels rather than the interstices represents 86% of the energy. However, the “off” pixels aim the light in a direction that is 48° from the imaging direction. Again, the “grating” pitch is 13.68 microns, so the orders appear every 2.4 degrees, and in this case we are interested in orders 19, 20, and 21. This time, however, the pixels are 12.68 µm in width, (92% duty cycle) and the diffraction efficiency is calculated to fall off as 0.85 x Sinc2(0.9256 M), where M is the order number. Calculating this for orders 19, 20, and 21 and multiplying by the 86% fractional power gives 0.186%, 0.020%, and 0.051%, respectively, or a sum of 0.257%. Accounting for the circularitry of the pupil as above gives a total estimate of 0.61% for the collected stray light due to the “off” beam. Adding these two sources of stray light gives 0.61% + 0.59% = 1.2%, relative to the total energy striking the DMD. If we instead compare to the energy that is expected to be in the “signal beam” (approximately 73%), we have an estimated level of stray light equal to 1.6% of the “signal.” This means that at any given spatial frequency, the MTF will be reduced to 98.4% of the calculated value. That is, at zero frequency, the contrast ratio is 98.4%, and at a frequency where the MTF is calculated to be (for example) 30%, the actual contrast ratio is 29.5%. 9.0 CONCLUSIONS A preliminary optical design of a DMD based spectrometer has been completed. This spectrometer consists of an objective lens capable of imaging objects located 3 meters to infinity onto a DMD. The DMD in turn directs spatial information from an object scene into a relay assembly that contains a spectral dispersing prism assembly to image onto a CCD array which has been oriented to detect the spectral information generated by the relay assembly as specified in Table 1.
72
TABLE 1 SPECIFICATIONS AND GOALS PRELIMINARY OPTICAL DESIGN OF AN IMAGING SPECTROMETER FOR REMOTE SENSING APPLIATIONS (09/03/02) Parameter Specification Preliminary Design 1. Configuration Objective, Texas Instruments
DMD Scene Generator, Optical Relay system, Dispersive element
Objective Lens TI DMD Scene Generator 1:1 Optical Relay Lens Dispersing double prism
2. Spectral Range 400-800 nm equally weighted (Goal)
450 – 800 nm (per Sandia)
3. Spectral Resolution 10 nm (goal)
10 nm
4. Illumination Source “Natural” incoherent light
Same
5. DMD Model Active Area Pixel Size Pixel Pitch Pixel Tilt (on/off)
TBD Sandia TBD Sandia, approx. 13.1 mm x 17.4 mm 16 µm 17 µm TBD, either 10°On/-10° Off or 12°On/-12°Off
TBD Sandia 13.1 mm x 17.4 mm 13.68 µm 12°On/-12°Off
6. Object Distance ~3 m to infinity
3 m to infinity
7. Detector Type Pixel Size Pixel Pitch
CCD (TBD Sandia) TBD Sandia TBD Sandia
Dalstar 1M15 (per Sandia) 14 µm x 14 µm
73
TABLE 1 (CON’T) SPECIFICATIONS AND GOALS PRELIMINARY OPTICAL DESIGN OF AN IMAGING SPECTROMETER FOR REMOTE SENSING APPLIATIONS (09/03/02) Parameter Specification Preliminary Design 8. Field of View 1.5° to 2.5° Half-field
(HFOV) in object space (maximize illuminated DMD pixels)
1.7° Half-field (HFOV)
9. F/# Goal of F/4 to F/7 TBD by Sandia with information provided by ORA
F/7
10. Image Quality Design MTF ~0.5 at 29.4 cycles/mm (Goal) at the detector plane with object at ~3 m
> 0.423 at 29.4 cy/mm
11. Contrast reduction due to diffracted light
Report 1.6% stray light (MTF reduced to 98.4% of calculated value)
12. Distortion Reprot only for this Phase. (10% of a DMD pixel Goal)
Two stages: Objective to DMD: < 0.086% object at infinity <0.10% object at 3 m Relay (DMD to CCD): <0.868 µm (uncalibrated) < 0.516 µm (calibrated)
13. Package Size Mimimize. (No specific requrement, but the device should not be larger than necessary.)
See Layout drawings
74
APPENDIX I CODE V Prescription of the System and Fabrication Data Listing
-------------------------------------------------------------------------------------------------- NOTES - Positive radius indicates the center of curvature is to the right Negative radius indicates the center of curvature is to the left - Dimensions are given in millimeters - Thickness is axial distance to next surface - Image diameter shown above is a paraxial value, it is not a ray traced value - Other glass suppliers can be used if their materials are functionally equivalent to the extent needed by the design; contact the designer for approval of substitutions. -------------------------------------------------------------------------------------------------- ARRAY DATA ARRAY( 1) RECTANGULAR ARRAY Spacing in X direction 0.013680 Spacing in Y direction 0.013680 -------------------------------------------------------------------------------------------------- DECENTERING CONSTANTS DECENTER X Y Z ALPHA BETA GAMMA -------------------------------------------------------------------------------------------------- D( 1) 0.0000 0.0000 0.0000 12.0000 0.0000 0.0000 (RETU) D( 2) 0.0000 0.0000 0.0000 24.0000 0.0000 0.0000 D( 3) 0.0000 0.0000 0.0000 -22.0378 0.0000 0.0000 (RETU) D( 4) 0.0000 0.0000 0.0000 -22.0378 0.0000 0.0000 (RETU) D( 5) 0.0000 0.0000 0.0000 3.3502 0.0000 0.0000 (RETU) D( 6) 0.0000 0.0000 0.0000 3.3502 0.0000 0.0000 (RETU) D( 7) 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 (RETU) D( 8) 0.0000 0.0000 0.0000 -0.0569 0.0000 0.0000 D( 9) 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 (RETU) D(10) 0.0000 0.0000 0.0000 -3.3502 0.0000 0.0000 (RETU) D(11) 0.0000 0.0000 0.0000 -3.3502 0.0000 0.0000 (RETU) D(12) 0.0000 0.0000 0.0000 22.0378 0.0000 0.0000 (RETU) D(13) 0.0000 0.0000 0.0000 22.0378 0.0000 0.0000 (RETU) D(14) 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 D(15) 0.0000 0.0000 0.0000 24.0000 0.0000 0.0000 (RETU) A decenter defines a new coordinate system (displaced and/or rotated) in which subsequent surfaces are defined. Surfaces following a decenter are aligned on the local mechanical axis (z-axis) of the new coordinate
system. The new mechanical axis remains in use until changed by another decenter. The order in which displacements and tilts are applied on a given surface is specified using different decenter types and these generate different new coordinate systems; those used here are explained below. Alpha, beta, and gamma are in degrees. DECENTERING CONSTANT KEY: TYPE TRAILING CODE ORDER OF APPLICATION -------------------------------------------------------------------------------- DECENTER DISPLACE (X,Y,Z) TILT (ALPHA,BETA,GAMMA) REFRACT AT SURFACE THICKNESS TO NEXT SURFACE DECENTER & RETURN RETU DECENTER (X,Y,Z,ALPHA,BETA,GAMMA) REFRACT AT SURFACE RETURN (-GAMMA,-BETA,-ALPHA,-Z,-Y,-X) THICKNESS TO NEXT SURFACE -------------------------------------------------------------------------------------------------- REFERENCE WAVELENGTH = 600.0 NM SPECTRAL REGION = 450.0 - 800.0 NM -------------------------------------------------------------------------------------------------- * ZOOM PARAMETERS POS. 1 POS. 2 POS. 3 *1 = INF 20000.0000 3000.0000 *2 = 100.0000 104.8379 131.9220 -------------------------------------------------------------------------------------------------- This is a decentered system. If elements with power are decentered or tilted, the first order properties are probably inadequate in describing the system characteristics. POS. 1 POS. 2 POS. 3 INFINITE CONJUGATES EFL = 311.5601 311.5586 311.5502 BFL = -120.1203 -115.2824 -88.1993 FFL = 31.4771 31.4771 31.4771 F/NO = -7.0014 -7.0013 -7.0011 AT USED CONJUGATES REDUCTION = 0.0000 -0.0156 -0.1028 FINITE F/NO = -7.0014 -7.0013 -7.0014
APPENDIX III Preliminary Sensitivity and As-built Optical Performance Analysis
TOLERANCE SUMMARY
Sequence saved with tolerances in file sys_double_prism_082202_tor.seq 26-Aug-02 POSITION 1 P E R F O R M A N C E S U M M A R Y POLYCHROMATIC MODULATION TRANSFER FUNCTION Obj DMD Relay 0 Dev prism 3G -------------------------------------------------------------------------------------------------------- WAVELENGTH WEIGHT 600.0 NM 1 -------------------------------------------------------------------------------------------------------- RELATIVE FREQ AZIM WEIGHT DESIGN DESIGN COMPENSATOR RANGE (+/-) * FIELD L/MM DEG + TOL * DLZ S14 obj DLZ S62 Ry2 DLZ S62 Ry1 DSY S8..10 obj DSX S8..10 obj DSY S29..30 Ry1 DSX S29..30 Ry1 DSY S52..53 Ry2 DSX S52..53 Ry2 DLZ S62 Psm 0.00, 0.00 29.40 90.0 1.00 0.814 0.719 0.948389 1.472601 1.475165 0.478395 0.449203 0.119909 0.119198 0.118014 0.115941 0.059825 -0.14, 1.00 29.40 90.0 1.00 0.776 0.551 0.948389 1.472601 1.475165 0.478395 0.449203 0.119909 0.119198 0.118014 0.115941 0.059825 -1.00, 0.14 29.40 90.0 1.00 0.435 0.384 0.948389 1.472601 1.475165 0.478395 0.449203 0.119909 0.119198 0.118014 0.115941 0.059825 0.14,-1.00 29.40 90.0 1.00 0.782 0.481 0.948389 1.472601 1.475165 0.478395 0.449203 0.119909 0.119198 0.118014 0.115941 0.059825 1.00,-0.14 29.40 90.0 1.00 0.561 0.477 0.948389 1.472601 1.475165 0.478395 0.449203 0.119909 0.119198 0.118014 0.115941 0.059825 -------------------------------------------------------------------------------------------------------- * The probable change and cumulative probability results are based on the assumption that the distribution of the MTF converges to a Gaussian form The compensator range is a mean plus 2 Sigma value. Linear compensators are in units of millimeters. Angular compensators are in radians.
26-Aug-02 POSITION 1 P E R F O R M A N C E S U M M A R Y POLYCHROMATIC MODULATION TRANSFER FUNCTION Obj DMD Relay 0 Dev prism 3G -------------------------------------------------------------------------------------------------------- WAVELENGTH WEIGHT 600.0 NM 1 -------------------------------------------------------------------------------------------------------- RELATIVE FREQ AZIM WEIGHT DESIGN DESIGN COMPENSATOR RANGE (+/-) * FIELD L/MM DEG + TOL * DLZ S14 obj DLZ S62 Ry2 DLZ S62 Ry1 DSY S8..10 obj DSX S8..10 obj DSY S29..30 Ry1 DSX S29..30 Ry1 DSY S52..53 Ry2 DSX S52..53 Ry2 DLZ S62 Psm 0.00, 0.00 29.40 0.0 1.00 0.831 0.744 0.950051 1.484079 1.484338 0.441646 0.478530 0.118993 0.120962 0.116039 0.117676 0.039656 -0.14, 1.00 29.40 0.0 1.00 0.524 0.450 0.950051 1.484079 1.484338 0.441646 0.478530 0.118993 0.120962 0.116039 0.117676 0.039656 -1.00, 0.14 29.40 0.0 1.00 0.779 0.425 0.950051 1.484079 1.484338 0.441646 0.478530 0.118993 0.120962 0.116039 0.117676 0.039656 0.14,-1.00 29.40 0.0 1.00 0.518 0.456 0.950051 1.484079 1.484338 0.441646 0.478530 0.118993 0.120962 0.116039 0.117676 0.039656 1.00,-0.14 29.40 0.0 1.00 0.761 0.402 0.950051 1.484079 1.484338 0.441646 0.478530 0.118993 0.120962 0.116039 0.117676 0.039656 -------------------------------------------------------------------------------------------------------- * The probable change and cumulative probability results are based on the assumption that the distribution of the MTF converges to a Gaussian form The compensator range is a mean plus 2 Sigma value. Linear compensators are in units of millimeters. Angular compensators are in radians.
47 INF -15.00000 0.10000 NPK51 48 INF / 1.00 0.00000 49 -87.47355 50 1357.82780 10.0000 2.0/ 0.50 -10.00000 0.50000 SF57 0.00200 51 424.50610 1.0000 2.0/ 0.50 -20.91779 0.50000 52 -45.22340 0.0200 2.0/ 0.50 -12.00000 0.02000 NFK51 0.00020 53 159.87288 0.2000 2.0/ 0.50 -9.23793 0.02000 54 65.76414 0.0200 2.0/ 0.50 -10.98582 0.30000 NKZFS11 0.00200 55 61.55814 0.0200 2.0/ 0.50 -3.00000 0.02000 56 72.79864 0.0400 2.0/ 0.50 -14.74137 0.10000 NKZFS11 0.00040 57 -38.56187 0.0200 2.0/ 0.50 -26.22564 0.30000 58 -95.67725 0.4000 2.0/ 1.00 -10.00000 0.50000 NFK51 0.00200 59 76.76011 0.2000 2.0/ 1.00 -5.00000 0.50000 60 47.82239 0.0200 2.0/ 1.00 -19.24977 0.06000 SF57 0.00200 61 57.96362 0.0200 2.0/ 1.00 -120.00000 62 0.00000 -------------------------------------------------------------------------------------------------------- Radius, radius tolerance, thickness and thickness tolerance are given in mm. Fringes of power and irregularity are at 546.1 nm. over the clear aperture Irregularity is defined as fringes of cylinder power in test plate fit
26-Aug-02 POSITION 1 D E C E N T E R E D T O L E R A N C E S Obj DMD Relay 0 Dev prism 3G -------------------------------------------------------------------------------------------------------- ELEMENT FRONT BACK ELEMENT WEDGE ELEMENT TILT EL. DEC/ROLL(R) NO. RADIUS RADIUS TIR ARC MIN TIR ARC MIN TIR mm. 1 210.42117 267.84681 0.0500 3.8 0.1145 8.6 0.0261 0.5000 2 105.52451 -245.78142 0.0100 0.8 0.0119 0.0200 (R) 2- 3 105.52451 50.71358 0.0297 0.1000 (R) 2- 4 105.52451 178.05473 0.0786 6.9 0.0423 0.2000 3 -245.78142 50.71358 0.0300 2.5 4 50.71358 178.05473 0.0300 2.6 5 -46.16134 INF 0.0900 9.6 0.0699 0.1000 (R) 5- 6 -46.16134 -60.61251 0.1613 17.2 6 INF -60.61251 0.0800 7.9 7 117.31165 117.31165 0.0300 2.9 0.1807 17.2 0.0052 0.5000 8 INF -101.90128 0.1000 9.5 0.0904 8.6 0.0714 0.2000 9 INF (MIRROR) 10 -57.96362 -47.82239 0.0400 4.0 0.1731 17.2 0.0177 0.4000 11 -76.76011 95.67725 0.1000 9.8 0.0879 8.6 0.1656 0.2000 12 38.56187 -72.79864 0.0200 2.3 0.0088 1.0 0.1218 0.1000 13 -61.55814 -65.76414 0.0100 1.0 0.0354 3.4 0.0016 0.1000 14 -159.87288 45.22340 0.0200 1.6 0.0126 1.0 15 -424.50610 -1357.82780 0.1000 8.3 0.1038 8.6 0.0344 0.5000 16 INF INF 17 INF INF 18 INF INF 19 INF INF 20 INF INF 21 INF INF 22 1357.82780 424.50610 0.1000 8.3 0.1037 8.6 0.0344 0.5000 23 -45.22340 159.87288 0.0200 1.6 0.0126 1.0 24 65.76414 61.55814 0.0100 1.0 0.0354 3.4 0.0016 0.1000 25 72.79864 -38.56187 0.0200 2.3 0.0088 1.0 0.1218 0.1000 26 -95.67725 76.76011 0.1000 9.8 0.0879 8.6 0.1656 0.2000 27 47.82239 57.96362 0.0400 4.0 0.1731 17.2 0.0177 0.4000 -------------------------------------------------------------------------------------------------------- Radii are given in units of mm. For wedge and tilt, TIR is a single indicator measurement taken at the smaller of the two clear apertures. For decenter and roll, TIR is a measurement of the induced wedge and is the maximum difference in readings between two indicators, one for each surface, with both surfaces measured at their respective clear apertures. The direction of measurement is parallel to the original optical axis of the element before the perturbation is applied. TIR is measured in mm. Decenter or roll is measured perpendicular to the optical axis in mm.
final sand report HTSI(GL) 93 2/4/2003
APPENDIX IV Preliminary Light Baffle Design and Analysis
final sand report HTSI(GL) 94 2/4/2003
APPENDIX V Original Statement of Work
final sand report HTSI(GL) 95 2/4/2003
June 5, 2002 Ms. Jody Smith SANDIA NATIONAL LABORATORY MS 0972 1515 Eubank Blvd. Albuquerque, NM 87123 Subject: Statement of Work, Preliminary Optical Design of an Imaging Spectrometer for Remote Sensing Applications References: 1. Telcon, J. Smith/Sandia to B. Crowther/ORA on 05/03/02. Telcon, J. Smith/Sandia with B. Crowther/ORA on 05/07/02. Telcon, J. Smith/Sandia with B. Crowther and J. Rogers/ORA on 05/17/02. Dear Ms. Smith: Optical Research Associates (ORA) is providing the attached Statement of Work (SOW), and cost/schedule quotation for the Preliminary Optical Design of an Imaging Spectrometer for Remote Sensing Applications. We understand that you are would like to have a working “proof-of-concept” model by the end of September 2002. We estimate that the Preliminary Design will require approximately four (4) weeks to complete, with the project Final Report delivered five (5) weeks ARO. In order to meet this schedule, we would like to initiate work as soon as possible. Our Firm-Fixed-Price (FFP) quotation for this design is $32,318. Please review the Statement of Work, and contact us with any questions you may have. In order to initiate work on this project, please send a PO to our contracts specialist, Scott Chapman, at our Pasadena, CA office. He can be reached by phone at (626) 795-9101, or by email at [email protected].
final sand report HTSI(GL) 96 2/4/2003
Thank you for considering ORA as your design partner. We would very much like to work with you on this project. Sincerely, OPTICAL RESEARCH ASSOCIATES Blake G. Crowther, Ph.D. Senior Systems Engineer/Optical Design Engineering Services E-mail: [email protected] Scott Chapman Contracts Specialist E-mail: [email protected] BGCPSNL0517:bab:email, fedex cc: G. Torrington/Sandia, J. Rogers, Ph.D., K. Thompson, Ph.D., R. Maushardt/ORAW enc: Statement of Work
final sand report HTSI(GL) 97 2/4/2003
STATEMENT OF WORK PRELIMINARY OPTICAL DESIGN OF AN IMAGING SPECTROMETER FOR REMOTE SENSING APPLICATIONS (06/05/02) References: 1. Telcon, J. Smith/Sandia with B. Crowther/ORA on 05/03/02. Telcon, J. Smith/Sandia with B. Crowther/ORA on 05/07/02. 3. Telcon, J. Smith/Sandia with B. Crowther and J. Rogers/ORA on 05/17/02. BACKGROUND Sandia National Laboratory (“Sandia”) is developing an imaging spectrometer for remote sensing applications. A key component of this device is the Texas Instruments Digital Micromirror Device (DMD). Sandia envisions using the DMD to allow portions of the image to be selectively “turned off” or “turned on”, allowing the spectral nature of the imaged scene to be decoded. To assist in this development, ORA is providing this Statement of Work (SOW), and cost quotation for the Preliminary Optical Design of an imaging spectrometer for remote sensing applications, according to the specifications and goals listed in Table 1. 2.0 PRELIMINARY OPTICAL DESIGN ISSUES As discussed in the (Reference 1) telecon, the ability to selectively “turn off” or “turn on” different portions of the imaged scenes requires that the DMD correspond to an intermediate image plane. Sandia also expressed a desire that the DMD map to the detector pixels such that each of the 17 µm DMD mirror facets correspond to an integer number of detector pixels (i.e. 1x1, 2x2, or 3x3). While this may be possible across the purely spatial dimension of the array, this mapping will not be preserved across the spectral dimension. The separation of the scene into its spectral components means there will be significant spectral smear at the image plane. Therefore, the image will be elongated in the spectral dimension of the array and will not be the same size as it is in the spatial dimension. A complicating factor associated with the use of a DMD and related to the mapping discussed above is the fact that the DMD deviates the light at an angle of 20° or 24°, depending on the model of DMD, (Reference 1) even though the global DMD surface is not at an angle of 10° or 12° with respect to the incoming ray. This produces an image, and/or object plane that is tilted with respect to the objective or relay optics. The complication here is that the corresponding image plane must also be tilted according to the Scheimpflug condition. Systems using tilted object and image planes often suffer “keystone distortion,” where the object is mapped to the image plane with a magnification that varies linearly with position in one dimension along the image instead of being constant. In addition, such systems often produce images that are
final sand report HTSI(GL) 98 2/4/2003
anamorphically distorted; that is, the magnification is different in two orthogonal directions. These effects can be controlled by adjusting the tilt angles of the object plane, the tilt angle of the optics, the tilt angle of the detector, the telecentricity of the system, and the magnification. Your desire for high-fidelity mapping from the object to DMD, and then to the detector means that distortion will be a major factor in the optimization merit function. The distortion to which we refer here is the traditional rotationally symmetric distortion in addition to the keystone distortion discussed above. One problem frequently encountered when using DMDs is that of stray light. When the device toggles to the “off” angle, the incident light is redirected out of the usable optical beam with little attenuation. Therefore, effective baffling (light traps) will need to be considered as part of the optical design. In addition to the “off-beam”, some light will reflect from the untilted periphery of the DMD. This reflected light will propagate back toward the objective lens. ORA recommends that this reflected light be analyzed in the Final Optical Design Phase. In our (Reference 1) telecon, we discussed several possible element types that could be used to “spread” or disperse the spectrum of the imaged scene. We decided in that conversation that a diffraction grating was a likely type of element to be used. Subsequently, we evaluated the specifications in greater detail and concluded that a prism may be actually the preferred dispersive element type. We discussed this in the (Reference 3) telecon. We anticipate using a prism in the design, but will hold open the possibility of using a diffraction grating should the prism not produce adequate performance. If a grating is used, we will try to use a grating formed on a flat substrate in order to reduce costs and delivery time in an effort to accommodate your schedule. Prisms and diffraction gratings, like most dispersive elements, depend on the angle at which the light is incident and operate best in a collimated beam with a fairly narrow field. Using such dispersive elements in a highly converging or diverging beam can introduce significant aberrations into the beam, which are subsequently difficult to remove. Therefore, we will attempt to collimate the beam reflected from the DMD and place the grating in this collimated beam, preferably at a pupil position.
final sand report HTSI(GL) 99 2/4/2003
Given the preliminary design issues discussed above, the design of the imaging spectrometer will most likely consist of an objective, the DMD, an optical relay, and a diffraction grating. The object will form an image of the scene on the DMD. The beam reflected from the DMD will be collimated by the first half of the relay to the extent possible and directed onto the dispersive element (prism). The second half of the relay will then take the spectrally dispersed beam, and produce the final image at the detector plane. We anticipate that the objective and relay elements will be composed of refractive elements, which should enable a short fabrication schedule. We will incorporate commercial off the shelf components where possible. However, we anticipate that custom optics will be required to achieve or even approach the goals listed in Table 1. In particular, this pertains to the correction of chromatic aberration. We anticipate that special glass types will be required for this design. Because of the periodic nature of the DMD pixels, some of the incident light will be diffracted into specific orders by the DMD, and a small fraction (typically, less than a percent) of this diffracted light will propagate in the direction of the “on-beam”, thereby reducing the contrast of the signal slightly. This is effect is fundamental to the use of DMDs and cannot be avoided. This effect is smaller for slower beams. The desired spectral range (Reference 2) is fairly large, running from 400 to 800 nm. We understand that Sandia is willing to compromise the spectral range if it means that other performance parameters, especially the spectral resolution, is would be compromised by the extended spectral range. ORA will work with Sandia National Laboratories on these issues to provide a design that is balanced between range and resolution, given the constraints that arise during the design process. Spectral resolution, for the purposes of this design, is defined as a geometric mapping in which wavelengths differing by 10 nm are imaged to different detector pixels at the final image plane. We understand that Sandia has not yet made a decision on a detector for the system, but will procure a detector that best matches the output image size. For optical design reasons, ORA anticipates using a 1:1 imager the design if possible. This means that the detector would match the format of the DMD. We anticipate working with Sandia over the course of the project to assist in making a detector decision. Since this is a proof of concept design and the specifications are not firmly fixed, we anticipate some adjustments will need to be made to various specifications as the design proceeds. We anticipate working closely with you and providing tradeoff data in cases where a specification may need to be adjusted or changed. We understand that our success is ultimately judged by the success of our customers and are therefore genuinely concerned that the project be successful. 3.0 DESIGN TASKS ORA will perform the following tasks relative to the specifications and goals in Table 1. 3.1 Design of the objective
final sand report HTSI(GL) 100 2/4/2003
ORA will design an objective that will image the scene onto the active area of the 768 x 1024 pixel DMD. Sandia will provide specification for two possible DMDs. ORA will incorporate either one in their design as they deem appropriate for imaging and design considerations. As discussed in the (Reference 2) telecon, we will fix the angle of the DMD with respect to the objective, and not allow it to be a variable in the design. If it becomes apparent during the course of the design that performance would be enhanced by allowing the objective to DMD angle to vary, we will inform you of that information, and assist you in making a decision regarding the angle. In this design, we will treat the tilt angle of the relay optics with respect to the axis of the beam reflected from the DMD as a variable in our design. We will also vary the angle of the image plane with respect to the relay optics. These variables, in addition to other free parameters, will be used to produce a sharp image and control the keystone distortion. The axis of rotation for the DMD pixels is 45 degrees. The preferred configuration of the system is to have square DMD pixels map to square focal plane pixels which may complicate the dispersing half of the system. Rotating the DMD assembly 45 degrees in the z-axis will be employed if necessary to obtain good image quality. 3.2 Design of the prism or grating ORA will design and specify a dispersive element after the DMD to produce the desired 10 nm spectral separation goal. During the course of the design, it may become necessary to adjust the spectral bandpass or resolution of the system. ORA will make those adjustments in consultation with Sandia after explaining the design difficulties and tradeoffs. As part of this task, ORA will make an estimate, based on scalar wave theory followed by LightTools® simulations, of the amount of light that diffracts from the DMD in the direction of the “on-beam” and the effect has on the contrast of the system.
final sand report HTSI(GL) 101 2/4/2003
3.3 Design an image relay system ORA will design an image relay system, operating at an appropriate magnification to map the square area inscribed in the DMD active area to the detector. This image relay system will incorporate the dispersive element to spread the input spectrum. 3.4 Indicate baffle and light trap locations and preliminary designs To ensure that the configuration selected for the spectrograph is not incompatible with the “off-beam” produced by the DMD, ORA will indicate positions and preliminary forms for necessary light traps and baffles as part of this design. Using LightTools, our proprietary software for illumination and stray light analysis, we will determine preliminary locations for necessary baffles and light traps. We will then verify that the specified baffles effectively eliminate the zeroth-order stray light; that is, all unwanted light strikes a baffle. 3.5 Preliminary sensitivity analysis ORA will perform a preliminary sensitivity analysis on the resultant design to determine the approximate difficulty of fabrication. This analysis is intended to give Sandia and potential fabricators an indication of the design sensitivities, but will not yield final tolerances. We expect that there will be design changes beyond the scope of the preliminary design and that additional tolerancing work will need to be done as well. This analysis should provide Sandia with the information required to obtain initial fabrication estimates; these would most likely require refinement after the Final Design Phase is complete. This preliminary sensitivity analysis will not include the effect of tolerances on the baffles or their function. 3.6 Final report ORA will prepare a written Final Report describing the results of Tasks 3.1 - 3.5. This will include a discussion of the design approach as well as layout drawings, performance analysis and the lens prescription. (Please note that although the preliminary lens prescription will be provided in this report, ORA does not consider the design mature enough for fabrication until the Final Optical Design is complete.) 4.0 DELIVERABLE ITEMS The deliverable items will be the Final Report described in Task 3.6.
final sand report HTSI(GL) 102 2/4/2003
5.0 LIMITATIONS AND ASSUMPTIONS This is a Preliminary Design Phase, which is intended to determine the general layout of the system and the general level of performance. ORA does not consider the design to be mature enough for fabrication until completion of the Final Optical Design Phase. Although ORA will indicate the position and perform the preliminary design a beam stop to trap the unwanted zeroth-order “off” beam of the system, this Preliminary Optical Design Phase does not include an analysis of ghost images, such as those caused by reflections from antireflection coated optical surfaces. No analysis of the performance across temperature is included in this quotation. 6.0 COMMUNICATIONS Communications will be by phone, fax, mail, e-mail, and Federal Express if needed. Representatives of Sandia National Laboratory are welcome to visit ORA to discuss the project at mutually agreeable times. 7.0 SCHEDULE The Preliminary Optical Design and Final Report is due five (5) weeks after receipt of order. 8.0 COST/TERMS Our Firm Fixed Price (FFP) cost quotation for Tasks 3.1 – 3.6 according to the specifications and goals listed in Table 1 is $32,318. ORA will invoice you $16,159 within 30 days ARO with payment terms of Net 30 days, and the remaining balance of $16,159 when we deliver our Final Report, with payment terms of net 30 days. ORA reserves the right to invoice the expended portion of the funds in the event that the customer places the project “on hold” for a substantial period of time. ORA also reserves the right to review a customer’s terms and conditions before accepting a purchase order. This quotation is valid until May 31, 2002.
final sand report HTSI(GL) 103 2/4/2003
TABLE 1 SPECIFICATIONS AND GOALS PRELIMINARY OPTICAL DESIGN OF AN IMAGING SPECTROMETER FOR REMOTE SENSING APPLICATIONS (06/05/02) Parameter Specification 1. Configuration Objective, Texas Instruments DMD Scene
2. Spectral Range 400 - 800 nm equally weighted (Goal)
3. Spectral Resolution 10 nm (Goal)
4. Illumination Source
“Natural” incoherent light
5. DMD Model Active area Pixel size Pixel pitch Pixel tilt (On/off)
TBD Sandia TBD Sandia, approx. 13.1 mm x 17.4 mm 16 µm 17 µm TBD, either 10° ON/-10° OFF or 12° ON/-12° OFF
6. Object Distance ~ 3 m to infinity
7. Detector Type Pixel Size Pixel Pitch
CCD (TBD Sandia) TBD Sandia TBD Sandia
8. Field of View 1.5° to 2.5° Half-field (HFOV) in object space (maximize illuminated DMD pixels)
9. F/# Goal of F/4 to F/7 TBD by Sandia with information provided by ORA
10. Image quality
Design MTF ~ 0.5 at 29.4 cycles/mm (Goal) at the detector plane with the object at ~ 3m
final sand report HTSI(GL) 104 2/4/2003
TABLE 1 (CONT’D) SPECIFICATIONS AND GOALS PRELIMINARY OPTICAL DESIGN OF AN IMAGING SPECTROMETER FOR REMOTE SENSING APPLICATIONS (06/05/02) Parameter Specification 11. Contrast reduction due to diffracted light
Report
12. Distortion Report only for this Phase. (10% of a DMD pixel Goal)
13. Package size Minimize. (No specific requirements, but the device should not be larger than necessary.)
final sand report HTSI(GL) 105 2/4/2003
Appendix D (Patent application) Patent Application
Staring 2-D Hadamard Transform Spectral Imager
CROSS REFERENCE TO RELATED APPLICATIONS (Not Applicable)
[01] The United States Government has rights in this invention pursuant to Department of Energy
Contract No. DE-AC04-94AL85000 with Sandia Corporation. BACKGROUND OF THE INVENTION
[02] There is a need for staring imaging systems capable of real-time high-speed imagery in a large number of spectral bands. (A staring image is a two-dimensional image that is captured at a single point in time, such as a camera image. A scanning image is a one-dimensional image that is scanned over different points in space as a function in time to form a 2-dimensional image.) A staring system would be useful for determining if momentary spectral changes occur in a two-dimensional field of view. Such momentary changes could easily be missed by a scanning system.
[03] Existing spectral imaging technologies do not support such high-speed imagery because mechanical scanning mechanisms are too slow. Techniques that employ electronic scanning methods and two-dimensional focal planes are required. In addition, techniques that support changing spectral sampling on-the-fly are able to match changing target and background conditions, particularly when it is necessary to optimize signal-to-noise ratio (SNR). In addition, under certain noise conditions, spectral multiplexing techniques have the potential of increasing SNR and decreasing the sample rate for a given measurement.
[04] Multiplexing is well-known in spectroscopy because of its ability to improve SNR. One of the available multiplexing techniques, Hadamard Transform Spectroscopy (HTS). An excellent review of HTS and its mathematical derivations is available in M. Harwit et al., Hadamard Transform Optics, Academic Press, 1979.
[05] The optical multiplexing advantage of Hadamard transform spectroscopy is due to a weighing design scheme. By simultaneously measuring multiple wavelength intensities according to a weighing scheme or Hadamard masking order, a corresponding increase in accuracy is observed. Two types of weighing schemes are available. The Hadamard H-matrix weighing design consists of entries corresponding to –1, 0 and +1. The simplex or S-Matrix mask utilizes +1 and 0 in the weighing design. Due to the relative ease in transforming the Hadamard mask encoded data, the additional requirement for a left-cyclic rotation of the S-matrix for each order is common for optical applications of a Hardamard weighing scheme.
[06] Figure 1A represents a left-cyclic Hadamard S-Matrix weighing design with an order of 7. A black square indicates an off or ‘0’ condition, while a white square is an on or ‘1’. In the example, each position in the Hadamard mask corresponds to specific wavelength intensity. The resulting 7 observables are multiplied by the inverse of the Hadamard S-matrix, as
final sand report HTSI(GL) 106 2/4/2003
shown in Figure 1B, to solve for the individual wavelength intensities. Assuming that detector noise is independent of the amount of light reaching the detector, the SNR
improvement approaches ~ 2n . If every nth observable consisted of measuring the
intensity at a single wavelength (i.e., the weighting scheme had 6 off positions and 1 on position), the system would be equivalent to a conventional dispersive spectrometer. However, using the example of Figure 1, the multiplexing advantage is illustrated because for the same seven observables each wavelength intensity is measured four times instead of once. For the Hadamard order of 7, the SNR improvement is negligible. Typical Hadamard orders implemented in an actual instrument are 1 to 2 orders of magnitude higher than the example shown in Fig. 1.
[07] Hadamard transform spectral imaging approaches have the potential to achieve the aforementioned improvements in SNR. Significant work in this area has been reported by a team at Kansas State University where a digital micro-mirror device (DMD™) from Texas Instruments (TI, see L. Hornbeck, U.S. Patent No. 5,535,047) has been utilized to implement a Hadamard transform spectrometer (HTS). (See R.A. DeVerse et al., Spectrometry and imaging using a digital micromirror array, American Laboratory, Oct. 1998, pp. 112S-120S; R.A. DeVerse et al., Hadamard transform Raman imagery with a digital micro-mirror array, Vibrational Spectroscopy 19 (1999) 177-186; R.A. DeVerse et al., An improved Hadamard encoding mask for multiplexed Raman imaging using single channel detection, Journal of Molecular Structure 521 (2000) 77-88; and W.G. Fateley et al. Modulations used to transmit information in spectrometry and imaging, Journal of Molecular Structure (550-551 (2000).) The papers describe the use of a DMD (or DMA) as a spatial light modulator for generating a stationary Hadamard encoding mask. The DMD in a dispersive flat-field spectrometer was utilized as a 1D Hadamard mask for spectral encoding. Due to the relatively high cost of 2D multichannel detectors in the near and mid infrared, the Raman imaging systems of DeVerse et al employ a single element detector for imaging in the visible spectral region. A 1D Hadamard cyclic S-matrix encoding mask (spectral encoding) is folded into a 2D Hadamard encoding mask and, together with sample rastering a single element detector, can be used for Hadamard transform imaging. While alluding to economic reasons, these papers do not discuss the technical problems that impede the development of viable staring 2D imaging system using a 2D detector and encoding the spectral dimension.
[08] Prior art systems have been developed and proposed that utilize Hadamard encoding of the spatial dimension using a DMD or 2D spatial light modulator (SLM) and a 2D detector. The systems encode one spatial dimension by focusing the input image on the DMD or SLM and passing the encoded images through a spectrograph (Q. S. Hanley et al, Spectral Imaging in a Programmable Array Microscope by Hadamard Transform Fluorescence Spectroscopy, Applied Spectroscopy, Vol 53, No.1, 1999) or utilize a Fourier Transformed Infrared light source (T. J. Tague, Jr., et al, U.S. Patent No. 5,923,036).
[09] The DMD has a 1024 x 768 matrix of discrete, rectangular mirrors that are supported on two opposing corners and which may be controllably tilted between a first position where one free corner is down and the opposite free corner is up and a second position where the one free corner is up and the opposite corner is down. The range of motion of each mirror is ± 10°.
final sand report HTSI(GL) 107 2/4/2003
SUMMARY OF THE INVENTION [10] The present invention provides a staring 2D imaging system utilizing the TI digital
micromirror device (DMD) and a 2D detector. The DMD is programmed to apply a cyclic S-matrix Hadamard encoding scheme, in either the spatial or spectral dimension, to the entire input scene with the resulting encodegram image cube undergoing the Hadamard transform to decode the encoded dimension. Unlike the prior art for encoding the spatial dimension, one embodiment of the present invention does not require the use of a spectral discriminator (i.e. spectrograph or Fourier transformed infrared light source) to elicit the spectral components of the image. Although only the spatial information is Hadamard encoded, the design and transform mathematics allow the spectral information to decode in conjunction with the Hadamard transform of the spatially encoded image cube. The alternate embodiment of the present invention, which encodes the spectral dimension of the image, differs from the previous art because it does not utilize a slit system with the requisite mechanical rastering components for collecting multiple 1-D slices to build a 2D image.
[11] One embodiment of the staring 2D imager focuses the input scene on the DMD. For each encoding pattern of the appropriate Hadamard matrix, the x-dimension pixels of the DMD apply a cyclic S-matrix encoding scheme to the x-dimension axis of the input image e scene. Each spatially encoded DMD image is spectrally dispersed in the x-dimension across a 2D detector. The set of dispersed encodegrams produces a data cube that contains both the Hadamard encoded spatial information and the spectral information that is optically convolved with the spatial. By applying a moving window inverse Hadamard transform to the data cube, the encoded spatial information is separated from the spectral information resulting in a traditional hyperspectral data cube containing two spatial and one spectral dimension.
[12] In an alternative embodiment, the 2D imager disperses the entire 2D input image scene and focuses the dispersed image onto the DMD. For each encoding pattern of the appropriate Hadamard matrix, the x-dimension pixels of the DMD apply a cyclic S-matrix encoding scheme to the spectrally dispersed 2D image, thus encoding the spectral dimension. Each encoded, spectrally dispersed image is then de-dispersed and focused on to a 2D detector thus maintaining the integrity of the spatial information and only the spectral information is Hadamard encoded. The inverse Hadamard transform is applied to the data cube of de-dispersed encodegram resulting in a traditional hyperspectral data cube containing two spatial and one spectral dimension.
[13] Additional advantages, and novel features of the invention will become apparent to those skilled in the art upon examination of the following description or may be learned by practice of the invention. The objects and advantages of the invention may be realized and attained as particularly pointed out in the appended claims.
BRIEF DESCRIPTION OF THE DRAWINGS [14] The accompanying drawings, which are incorporated in and form part of the specification,
illustrate an embodiment of the present invention and, together with the description, serve to explain the principles of the invention.
[15] Fig. 1 shows an example of a Hadamard cyclic S-matrix applied to n=7 observables and the inverse transform solution.
final sand report HTSI(GL) 108 2/4/2003
[16] Fig. 2 is a schematic view of a 2D Hadamard Transform Spectral Imager (HTSI) that encodes the spatial dimension according to a first embodiment of the present invention.
[17] Fig. 3 is a schematic view of a second embodiment of the invention that encodes the spectral dimension.
[18] Fig. 4 is a diagram of the optical design detailing the components of the split double Offner Relay and gratings required to correct for the linear axial chromatic aberration in the alternative embodiment shown in Fig. 3.
[19] Fig. 5 shows the transformation technique for the first embodiment. DETAILED DESCRIPTION OF THE INVENTION
[20] Depicted schematically in Fig. 2 are the primary components for an imaging device according to the present invention. An incoming 2D image source 1 is focused on a digital micromirror array such as DMD 2. After encoding in the spatial x-dimension on the DMD 2, the image is spectrally dispersed by a diffraction grating 4 and the spectrally dispersed, spatially Hadamard-encoded image is directed to a 2D detector array 5. The 2D detector array is read out for each element of the Hadamard encoding sequence, or once for each state of the DMD 2. This produces a sequence of data frames that carry mixed spatial and spectral information. The spatial and spectral information are separated in computer 3 by a Hadamard transform using the inversion technique with a moving window (as indicated by R(i:i+h-1,j)) outlined in Fig. 5.
[21] A second embodiment of the present invention is shown in Fig. 3. The dispersed 2D image source 1 is focused on to a DMD 3 by a split-Offner relay. A diffraction grating 2 in the split-Offner relay spectrally disperses the image..
[22] After encoding the dispersed spectral dimension on the DMD 3, the image is refocused by a second split-offner relay with a holographic grating 4 and the spectrally Hadamard-encoded image is directed to a 2D detector array 6. The 2D detector array 6 is read out for each Hadamard order and produces a series of image frames, an encodegram image cube, where the x- and y-spatial dimensions have been retained by the 1:1 imaging of the double split-Offner relay optical design and the spectral dimension is Hadamard encoded. The vector of responses at each pixel in the encodegram image cube is multiplied by the inverse of the Hadamard matrix to solve for the wavelength intensities. The total number of wavelengths for the system will correspond to the Hadamard order, thus allowing the staring imager the flexibility of varying the spectral and temporal resolution depending upon the system under observation.
[23] For the preferred embodiment shown in Fig. 3, there is a tilt in the axis of the receiving optics relative to the plane of the mirrors on DMD 3 from which the light is reflected that causes a shift in the axial source focus as a function of wavelength. The resulting uncorrected axial chromatic aberration in the final image causes the image to be out of focus as a function of wavelength. This linear axial chromatic aberration must be removed or the resulting image will be severely defocused and of unacceptable quality for imaging applications. For this imaging application, the Scheimpflug Condition is used to remove the tilt in the object plane for off-axis field points. Fig. 4 shows a practical implementation of this design approach
final sand report HTSI(GL) 109 2/4/2003
using an Offner-relay spectrometer design. The grating terms in the two mirrors 2 and 5 are identical, but the second mirror 5 has an added power term to correct for the spectral tilt.
[24] It should be apparent that there are many modifications possible with this invention. For example, the specifications of Texas Instrument’s DMD include dimensions of 768 pixels in the y-direction, 1024 pixels in the x-direction and a wavelength suitability region of 600 nm to 2.5 µm. However, the invention is not limited to use at these wavelengths or according to the physical dimensions of this particular digital micromirror array. The invention could use digital micromirror array and spatial light modulator technology with varying spatial and wavelength specifications as long as the devices are capable of generating the cyclic S-matrix masking patterns.
[25] In addition, each masking pattern of the cyclic Hadamard S-matrix is generated in the x-direction on the DMD with the pattern extending down the y-direction pixels. Using the cyclic Hadamard S-matrix with an order of 7 shown in Figure 1 as an example, the DMD would generate 7 consecutive masking patterns that would encode the spatial x dimension. The 1024 pixels in the x-direction would be turned “on” or “off” according to the masking pattern (white=on, black=off). The masking pattern would be repeated across the x-direction resulting in 1024/7 consecutive patterns. The “on” or “off” condition of the mask would be continuous in the y-direction pixels. The current micromirror array technology has, for example, 1024 x 768 separate mirrors that can turn “on” or “off”. However, since the “on” or “off” status for the Hadamard masking pattern is continuous in the y-direction, a preferred alternate embodiment for the present invention would include a digital micromirror array with pixel width mirrors that are the full length or fractions thereof in the y-direction of the incoming image. A digital signal process (DSP) board on computer implements the Hadamard masking pattern on the DMD .
[26] It is intended that the scope of the invention be defined by the appended claims.
final sand report HTSI(GL) 110 2/4/2003
CLAIMS 1. A staring imaging method comprising:
inputting a 2D spatial image containing multi-frequency spectral information; 2 encoding one dimension of the image with a cyclic Hadamard S-matrix; detecting the encoded image with a spatial 2D detector; and 4 transforming the encoded image.
2. The staring imaging method of claim 1 wherein said encoding step comprises: applying the input image optical scene to a 2D optical encoder; and encoding the x-dimension spatial information from the optical scene with a cyclic Hadamard S-matrix applied to the optical encoder.
3. The staring imaging method of claim 2 further comprising spectrally dispersing in the x-dimension the Hadamard spatially encoded image. 4. The staring imaging method of claim 3 wherein the spectrally encoded, spectrally dispersed input image is focused on a 2D detector. 5. The staring imaging method of claim 4 wherein:
an untransformed, spatially encoded, and spectrally dispersed data cube is built 2 from the detector images collected for each Hadamard order;
a moving window Hadamard transform is applied to the data cube wherein the spectral 4 dimension is deconvolved with the transformation of the encoded spatial dimension; and
the resulting separation of the x-dimension spatial information from the spectral 6 information forms a hyperspectral data cube containing two spatial and one spectral dimension.8
6. The staring imaging method of claim 5 wherein the optical encoder is a computer controlled digital micromirror array. 7. The staring imaging method of claim 6 wherein the digital micromirror array is a Texas Instruments DMD™. 8. The staring imagine method of claim 6 wherein the digital micromirror array has pixel width mirrors in the x-direction while the y-dimension of the mirrors can extend to the size of the input image. 9. The staring imaging method of claim 5 wherein the optical encoder is a spatial light modulator. 10. The staring imaging method of claim 1 wherein the encoding step comprises:
spectrally dispersing the input image optical scene along the x-dimension; applying the spectrally dispersed image to a 2D optical encoder; and encoding the spectral dimension of the dispersed optical scene with a cyclic Hadamard S-matrix applied across the x-dimension of the optical encoder.
final sand report HTSI(GL) 111 2/4/2003
11. The staring imaging method of claim 10 wherein, the spatial x-dimension of the Hadamard encoded, spectrally dispersed image is retained by refocusing (de-dispersing) the image after optically encoding. 12. The staring imaging method of claim 11 wherein the de-dispersed, spectrally encoded image is focused on a 2D detector. 13. The staring imaging method of claim 12 wherein,:
an untransformed, spectrally encoded data cube is built from the detector images 2 collected for each Hadamard order;
the data cube is unencoded by multiplying the vector at each encodegram pixel by the inverse of 4 the Hadamard cyclic S-matrix; and the resulting transform of the encodegram is a hyperspectral data cube containing two 6 spatial and one spectral dimension.
14. The staring imaging method of claim 13 wherein,
the optical encoder is a computer controlled digital micromirror array, wherein said array introduces linear axial chromatic aberrations into the image; and the de-dispersion is caused by a grating utilizing the Scheimpflug Condition to correct for the linear axial chromatic aberration by including a power term in the grating to correct for the spectral tilt.
15. The staring imaging method of claim 14 wherein the digital micromirror array is a Texas Instruments DMD™. 16. The staring imagine method of claim 14 wherein the digital micromirror array has pixel width mirrors in the x-direction while the y-dimension of the mirrors can extend to the size of the input image. 17. A staring 2D imager comprising: a 2D spatial input image; a first optical path for applying the input image to a first diffraction grating, wherein the image is spectrally dispersed; a second optical path for applying the dispersed image to a 2D digital mirror array; a computer for controlling said mirror array to encode the spectrally dispersed image with a cyclic S-mask Hardamard transform; a third optical path for applying the transformed image from said array to a second diffraction grating, the diffraction of said second grating canceling the dispersion of said first grating; a fourth optical path for applying the transformed image from said second diffraction grating to a 2D detector; wherein said computer is connected to said detector for converting said image into data representative of two spatial and one spectral dimension. 18. The imager of claim 17 wherein said encoder is a digital mirror array that introduces spectral tilt into the dispersed image, and wherein said second diffractive grating further includes a power term to compensate for said spectral tilt. 19. The imager of claim 18 wherein said digital mirror array is a DMD™.
112
ABSTRACT
A staring imaging system inputs a 2D spatial image containing multi-frequency spectral information. This image is encoded in one dimension of the image with a cyclic Hadamard S-matrix. The resulting image is detecting with a spatial 2D detector; and a computer applies a Hadamard transform to recover the encoded image.
113
n0
n1
n2
n3
n4
n5
n6
0 1 2 3 4 5 6n0 = ψ0 ψ1 + 2 + 4 + 0
n1 = ψ0 ψ1 + 3 + 6 + 1
n2 = ψ0 ψ2 + 5 + 6 + 2
n3 = ψ1 ψ4 + 5 + 6 + 3
n4 = ψ0 ψ3 + 4 + 5 + 4
n5 = ψ2 ψ3 + 4 + 6 + 5
n6 = ψ1 ψ2 + 3 + 5 + 6
n0
n1
n2
n3
n4
n5
n6
ψ0
ψ1
ψ2
ψ3
ψ4
ψ5
ψ6
=
-1
x
A B
Figure 1
114
Source
Diffraction Grating
DMD
2D Detector
Array
PCI based DSP board
Computer
(1)
(2)
(3)(4) (5)
Figure 2
115
Source
Diffraction Grating
Refocusing Optics
DMD
2D Detector
Array
PCI based DSP board
Computer
(1) (2)
(3) (4)
(5)
(6)
Figure 3
116
(1)
(2)
(3)
(5)
(6)
(B)
(A)
Figure 4
117
λ7 λ6 λ5 λ4 λ3 λ2 λ1
si,j R(i:i+(h-1),j), for all n
observables(hi)-1
n0
n1
n2
n3
n4
n5
n6
i= 1 2 3 4 5 6 7
e6
e5
e4
e3
e2
e1
e0-1
= x
For an image with pixel dimensions (I x J) Si,j≡ spectrum at pixel (i,j)
R ≡ detector response h ≡ hadamard order (shown e.g. h=7)
h ≡ hadamard vector for column i
Figure 5
118
Distribution:
2 MS 570 J. Wehlburg 5712 1 886 C. Wehlburg 1812 1 972 C. Boney 5712 1 570 J. Smith 5712 1 603 O. Spahn 1742 1 972 M. Smith 5712 1 980 R. Mata 5715 1 972 S. Gentry 5712 1 886 N. Jackson 1812 1 188 D. Chavez LDRD Office 1 9018 Central Technical Files, 8945-1 1 899 Technical Library, 9616 1 612 Review and Approval Desk, 9612
![Fast Background Initialization with Recursive Hadamard ... · 2. The Hadamard Transform The Hadamard transform [7, 9] belongs to the gener-alized class of Fourier transforms and it](https://static.documents.pub/doc/80x56/5f13c09011c737592655ec87/fast-background-initialization-with-recursive-hadamard-2-the-hadamard-transform.jpg)