Post on 27-Dec-2015
transcript
Computational Photography
Digital Camera
What isComputational Photography?
Second breakthrough by IT First : electronic image sensor (digital camera)
Digital representation of “image formed by lens” Second : Re-definition of whole camera (optics, usage)
Image is reconstructed by computation
Imageprocessing
Image
Image sensor
Optics
Whole part of camera is affected by computational photography
Film camera
Digital Camera
Light field(Light space)
€
(X,Y,Z)
€
(θ,φ)
What is camera?
Camera is a machine to record the distribution of the light in a scene
How to represent the distribution of the light in the scene? 3-D coordinate of the point where the light passing through : X, Y, Z Direction of the light : θ , Φ Wavelength of the light (color of the light) : λ Time : t
The 7 parameters function P which represent the distribution of the light is called “Plenoptic function”
€
I = P(X,Y,Z,θ,φ,λ , t)Light source
Object
Light field (light ray) Optics (lens) Image sensor (pixel)
€
(X,Y,Z)
€
(θ,φ)
Integration of camera
Camera integrates the light for all 7 parameters Position ( range of X, Y, Z : aperture size should not be zero) Direction ( range of θ , Φ : pixel size is not zero) Wavelength ( range of λ : No single wavelength filter) Exposure time ( range of t : shutter speed should not be too fast)
Multiple samples - θ,Φ:number of pixel, λ : RGB , t : burst shot So, what is multiple sampling for X, Y, Z?
€
I = P(X,Y,Z,θ,φ,λ , t)
Camera array
Measuring the distribution of the light at multiple position
ProFUSION25 (ViewPlus, Inc.)
The StanfordMulti-Camera Array(Marc Levoy @Stanford University)
Use of camera array
Free-viewpoint image
Defocus generation by synthetic aperture
3-D video (Matsuyama lab, Kyoto Univ.) Synthetic aperture(Vaish@Stanford)
Defocus control by Uncalibrated Synthetic Aperture
Natsumi Kusumoto, Shinsaku Hiura and Kosuke Sato, Uncalibrated Synthetic Aperture for Defocus Control, CVPR2009 (Jun. 2009)
€
(X,Y,Z)
€
(θ,φ)
Reviewing “integration”
Some part of information is lost by integration Sine wave which period is just as same as the integration duration
is lost
Blur of object within an exposure time Defocus by misfocus
€
I = P(X,Y,Z,θ,φ,λ , t)
€
Asin(nt)dt0
2π
∫ = 0
× = 0
Coded Exposure
Coded exposure : exposure is coded in time axis
Flutter Shutter Camera (Raskar@MERL)
This
TraditionalCoded
Exposure
Image of Static Object
Deblurred Image
Deblurred Image
Slide by R. Raskar
Coded Exposure
Temporal 1-D broadband code: Motion Deblurring
Coded Aperture
Spatial 2-D broadband mask: Focus Deblurring
Slide by R. Raskar
Coded Aperture
Levin@MIT(2007) Depth estimation by single
image (manual operation is necessary)
Multi-focus camera withCoded Aperture
Stabilizing the depth estimation and deblur by coded aperture
Simultaneous capture of 3 images with different focused distance
Hiura et al, CVPR(1998),SSII(1999)
Invariant integration Defocus : changed according to the distance
Blur : changed according to the speed of the object
Reconstruction is not easy because the estimation of the speed or distance is necessaryIs it possible to make defocus or blur invariant to the distance or speed?
Invariant integration Defocus
Special optics : Wavefront Coding
Motion of the image sensor while exposure
Blur Reciprocal motion of the camera
CDM Optics, Inc.
Motion of the image sensorfor invariant defocus
H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, Flexible Depth of Field Photography, ECCV2008
H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, Flexible Depth of Field Photography, ECCV2008
Motion of the image sensorfor invariant defocus
Deblur by reciprocal motion of the camera
A. Levin, P. Sand, T. S. Cho, F. Durand, W. T. Freeman. Motion-Invariant Photography. SIGGRAPH2008.
Input image Deblurred image
A. Levin, P. Sand, T. S. Cho, F. Durand, W. T. Freeman. Motion-Invariant Photography. SIGGRAPH2008.
Equipment Conceptual figure forLight sources with
different speed
Deblur by reciprocal motion of the camera
More..
Resources on www Wikipedia : computational photography http://computationalphotography.org/ http://www1.cs.columbia.edu/CAVE/projects/what_is/ http://projects.csail.mit.edu/photo/
Conferences International Conference on Computational Photography SIGGRAPH, CVPR, .. Session about computational photography