MIT Media Lab | Camera Culture

Shoot Now, Relight LaterIllumination Multiplexing with Lock-In Time of Flight SensorsAchuta Kadambi, Ayush Bhandari, Refael Whyte, Adrian Dorrington, Ramesh Raskar

achoo@mit.edu

How Can We

Relight a

Photograph?In this paper we propose a fusion of two

popular contexts, time of flight range cameras

and illumination multiplexing. Time of flight

cameras are a low cost, consumer-oriented

technology capable of acquiring range maps

at 30 frames per second. Such cameras have

a natural connection to conventional

illumination multiplexing strategies as both

paradigms rely on the capture of multiple

shots and synchronized illumination. While

previous work on illumination multiplexing has

exploited coding at millisecond intervals, we

repurpose sensors that are ordinarily used in

time of flight imagstrategiesing to demultiplex

via nanosecond coding.

ICCP 2014

Illumination Multiplexing

Illumination multiplexing is not a new problem, but requires fast captures. Figure from Schechner 2007

Time of Flight 3D Sensors

Time of Flight 3D cameras are an emerging camera technology. Such technology forms the basis for the new Kinect.

Time of Flight Operating Principle

Operation of ToF range cameras. The time difference of arrival between the emitted and received code is encoded in the phase offset. Thus, phase encodes depth.

Nanosecond vs Millisecond Coding

Case 1: Conventional High Speed MultiplexingCase 2: Using a lock-in ToF sensor allows for nanosecond coding.Case 3: Optimizing the Nanosecond Codes is the goal.

Illumination Multiplexing

Illumination multiplexing is not a new problem, but requires fast captures. Figure from Schechner 2007

Demonstration: Relighting Post-Capture

Demonstration: Scene Relighting.

Demonstration: Multiplexing Wavelength

Demonstration: Color ToF camera by multiplexing RGB lights.

Hardware Prototype

We validate our technique with a hardware prototype. It consists of an FPGA, Lock-in Sensor, and Solid State Illumination sources.

www.media.mit.edu/~achoo/demux