ACQUISITION AND ANALYSIS OF
GIGA-PIXEL SPECTRAL
INFORMATION
IJARC CORE6 Project
11th Jan. 2010
Rei Kawakami
IMAGING DEVICES
• Limited to RGB or Monochrome images
Cameras are everywhere, but …
SPECTRAL IMAGING DEVICES
LCTF filter35,000 $
Hyper-spectral camera55,000 $
Line spectral scanner25,000 $
• High cost• Limited resolution• Few softwares available
PROPOSAL Giga-pix hyperspectral imaging system
High resolution (Giga-pix order)Off-the-shelf (~ 1,000 $)
Software to analyze high-res hyper-spectral data
WHY DO WE NEED IT? Why spectra?
Pure physical quantity Represents precise object’s optical property Sensor independent
Why high-resolution?Spectrum at microscopic level of details
Material analysis Preservation
POSSIBLE APPLICATIONS E-heritage
Preservation of precise color
Medical imagingAnalysis of skins, neurons, blood, etc.
Material analysis / biological analysisSemiconductorsFossilsMicro-organismsBirds, insects
OUTCOMES Imaging system Software to analyze data
PROPOSED IMAGING SYSTEM
M. Ezra et al.
Giga-pixel Camera@ Microsoft research
Spectral transmittancevaries linearly
Large-format lens CCD
LINEAR VARIABLE FILTER
Filter
Monochrome camera
Rotational/translational stage
ADVANTAGES Giga-pixel spectral information
Microscopic level
Efficient combination Translational motion
Low costCost of the filter: ~ 1,000 $FWHF: ~ 4 nm to 8 nmTransmittance: ~ 30 to 40 %
TECHNICAL ISSUES Stitching
Filter causes intensity change
Low intensity vs. Acquisition timeDenoising
Data compression Wavelength ~ 80 dimension
Efficient data-acquisitionSize of the filter = bottleneck
CCDFilter
Useless
METHODS FOR VISUALIZATION & ANALYSIS Visualization
Development of GUI for user assistance
AnalysisMaterial estimation (spectral unmixing) Illumination/Surface reflectance separationSegmentation
SCHEDULEJuly Aug Sep Oct Nov Dec Jan Feb Mar Apr May June
A
B
C
D
E
F
G
H
I
Survey on related cameras
Development of the camera
Writing a paper
Survey on spectral analysis
Developing a new spectral analysis method
Writing a paper
Presenting a paper about the sensor
Discussions with collaborators
Experiments with the sensor and the method
HOW TO CHECK PROGRESS Number of meetings
with collaborators
Number of days I spend at Microsoft research
Number of papers2
BUDGETTravel cost JPY Total
Travel to Beijing(Round trip: \50,000 x 2) 100,000
Travel expense for a conference(Travel cost: \300,000, Registration fee: \100,000) 400,000
Travel expense to measure cultural assets 100,000 600,000
Salaries JPY Total
Housing expense and salaries at Microsoft Research(\300,000 x 5 months) 1,500,000 1,500,000
Purchase of equipments JPY Total
Variable interference filter 100,000
Filter attachment 200,000
Artificial daylight for experiments 200,000
PC 100,000
Miscellaneous expense 300,000 900,000
MICROSOFT TOOLS & TECHNIQUES Giga-pixel camera
Visual studio
Office Live Meeting
Collaboration with Microsoft researchers
MY RESEARCH EXPERIENCE PHD candidate @ University of Tokyo, 05-
08 Surface color estimation
ICCV 2005, JOSA 2007, CVPR 2009 Surface reflectance/illumination separation
Project researcher @ University of Tokyo, 08- Shadow removal
VRST 2008 Optical property estimation of layered
surfacesMIRU 2009 (oral presentation, in Japanese)