ACQUISITION AND ANALYSIS OF

GIGA-PIXEL SPECTRAL

INFORMATION

IJARC CORE6 Project

11th Jan. 2010

Rei Kawakami

IMAGING DEVICES

• Limited to RGB or Monochrome images

Cameras are everywhere, but …

SPECTRAL IMAGING DEVICES

LCTF filter35,000 $

Hyper-spectral camera55,000 $

Line spectral scanner25,000 $

• High cost• Limited resolution• Few softwares available

PROPOSAL Giga-pix hyperspectral imaging system

High resolution (Giga-pix order)Off-the-shelf (~ 1,000 $)

Software to analyze high-res hyper-spectral data

WHY DO WE NEED IT? Why spectra?

Pure physical quantity Represents precise object’s optical property Sensor independent

Why high-resolution?Spectrum at microscopic level of details

Material analysis Preservation

POSSIBLE APPLICATIONS E-heritage

Preservation of precise color

Medical imagingAnalysis of skins, neurons, blood, etc.

Material analysis / biological analysisSemiconductorsFossilsMicro-organismsBirds, insects

OUTCOMES Imaging system Software to analyze data

PROPOSED IMAGING SYSTEM

M. Ezra et al.

Giga-pixel Camera@ Microsoft research

Spectral transmittancevaries linearly

Large-format lens CCD

LINEAR VARIABLE FILTER

Filter

Monochrome camera

Rotational/translational stage

ADVANTAGES Giga-pixel spectral information

Microscopic level

Efficient combination Translational motion

Low costCost of the filter: ~ 1,000 $FWHF: ~ 4 nm to 8 nmTransmittance: ~ 30 to 40 %

TECHNICAL ISSUES Stitching

Filter causes intensity change

Low intensity vs. Acquisition timeDenoising

Data compression Wavelength ~ 80 dimension

Efficient data-acquisitionSize of the filter = bottleneck

CCDFilter

Useless

METHODS FOR VISUALIZATION & ANALYSIS Visualization

Development of GUI for user assistance

AnalysisMaterial estimation (spectral unmixing) Illumination/Surface reflectance separationSegmentation

SCHEDULEJuly Aug Sep Oct Nov Dec Jan Feb Mar Apr May June

A

B

C

D

E

F

G

H

I

Survey on related cameras

Development of the camera

Writing a paper

Survey on spectral analysis

Developing a new spectral analysis method

Writing a paper

Presenting a paper about the sensor

Discussions with collaborators

Experiments with the sensor and the method

HOW TO CHECK PROGRESS Number of meetings

with collaborators

Number of days I spend at Microsoft research

Number of papers2

BUDGETTravel cost JPY Total

Travel to Beijing(Round trip: \50,000 x 2) 100,000

Travel expense for a conference(Travel cost: \300,000, Registration fee: \100,000) 400,000

Travel expense to measure cultural assets 100,000 600,000

Salaries JPY Total

Housing expense and salaries at Microsoft Research(\300,000 x 5 months) 1,500,000 1,500,000

Purchase of equipments JPY Total

Variable interference filter 100,000

Filter attachment 200,000

Artificial daylight for experiments 200,000

PC 100,000

Miscellaneous expense 300,000 900,000

MICROSOFT TOOLS & TECHNIQUES Giga-pixel camera

Visual studio

Office Live Meeting

Collaboration with Microsoft researchers

MY RESEARCH EXPERIENCE PHD candidate @ University of Tokyo, 05-

08 Surface color estimation

ICCV 2005, JOSA 2007, CVPR 2009 Surface reflectance/illumination separation

Project researcher @ University of Tokyo, 08- Shadow removal

VRST 2008 Optical property estimation of layered

surfacesMIRU 2009 (oral presentation, in Japanese)