+ All Categories
Home > Documents > Computational Imaging - Massachusetts Institute of...

Computational Imaging - Massachusetts Institute of...

Date post: 01-Feb-2018
Category:
Upload: duongnhu
View: 224 times
Download: 0 times
Share this document with a friend
81
COMPUTATIONAL IMAGING Berthold K.P. Horn
Transcript
Page 1: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

COMPUTATIONAL IMAGING

Berthold K.P. Horn

Page 2: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

What is Computational Imaging?

• Computation inherent in image formation

Page 3: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

What is Computational Imaging?

• Computation inherent in image formation

(1) Computing is getting faster and cheaper

—precision physical apparatus is not

Page 4: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

What is Computational Imaging?

• Computation inherent in image formation

(1) Computing is getting faster and cheaper

—precision physical apparatus is not

(2) Can’t refract or reflect some radiation

Page 5: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

What is Computational Imaging?

• Computation inherent in image formation

(1) Computing is getting faster and cheaper

—precision physical apparatus is not

(2) Can’t refract or reflect some radiation

(3) Detection is at times inherently coded

Page 6: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Computational Imaging System

Berthold K.P. Horn
Line
Page 7: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Examples of Computational Imaging:

(1) Synthetic Aperture Imaging

(2) Coded Aperture Imaging

(3) Diaphanography—Diffuse Tomography

(4) Exact Cone Beam Reconstruction

Page 8: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

(1) SYNTHETIC APERTURE IMAGING

Traditional approach:

• Coupling of resolution, DOF, FOV to NA

• Precision imaging — “flat” illumination

with: Michael Mermelstein, Jekwan Ryu,Stanley Hong, and Dennis Freeman

Page 9: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Objective Lens Parameter Coupling

Page 10: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Synthetic Aperture Imaging

Traditional approach:

• Coupling of resolution, DOF, FOV to NA

• Precision imaging — “flat” illumination

New approach:

• Precision illumination — Simple imaging

• Multiple images — Textured illumination

Page 11: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation
Page 12: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation
Page 13: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation
Page 14: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation
Page 15: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Synthetic Aperture Imaging

• Precision illumination — Simple imaging

• Multiple images — Textured illumination

• Image detail in response to textures

• Non-uniform samples in FT space

Page 16: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

SAM M6

Page 17: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Creating Interference Pattern

Page 18: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Creating Interference Pattern

Page 19: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Fourier Transform of Texture Pattern

Page 20: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Interference Pattern Texture

Page 21: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Synthetic Aperture Microscopy

• Interference of many Coherent Beams

• Amplitude and Phase Control of Beams

Page 22: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Amplitude and Phase Control

Page 23: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Amplitude and Phase Control

Page 24: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Synthetic Aperture Microscopy

• Interference of many Coherent Beams

• Amplitude and Phase Control of Beams

• On the fly calibration

• Non-uniform inverse FT Least Squares

Page 25: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Wavenumber Calibration using FT

Page 26: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Hough Transform Calibration

Page 27: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Least Squares Match in FT

Page 28: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Fourier Transform of Texture Pattern

Berthold K.P. Horn
Rectangle
Page 29: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Uneven Fourier Sampling

Page 30: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Polystyrene Micro Beads (1µm)

Page 31: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Resolution Enhancement

• Reflective Optics Illumination

Vaccum UV — Short Wavelength

Page 32: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Reflective Optics M6

Page 33: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Resolution Enhancement

• Reflective Optics Illumination

Vaccum UV — Short Wavelength

• Fluorescence Mode

Resolution Determined by Illumination

Page 35: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

(2) CODED APERTURE IMAGING

• Can’t refract or reflect gamma rays

• Pinhole — tradeoff resolution and SNR

with: Richard Lanza, Roberto Accorsi,Klaus Ziock, and Lorenzo Fabris.

Page 36: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Coded Aperture Imaging

• Can’t refract or reflect gamma rays

• Pinhole — tradeoff resolution and SNR

• Multiple pinholes

• Complex masks can “cast shadows”

Page 37: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Masks — Fresnel Camera

Page 38: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Coded Aperture Principle

Page 39: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Decoding Method Rationale

Page 40: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Coded Aperture Imaging

• Can’t refract or reflect gamma rays

• Pinhole — tradeoff resolution and SNR

• Complex masks can “cast shadows”

• Decoding by Correlation

• Special Masks with Flat Power Spectrum

Page 41: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Mask Design — Inverse Systems

Page 42: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Maximizing SNR

minn∑i=1

w2i subject to

n∑i=1

wi = 1

yields wi = 1n

Page 43: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Masks — Legri URA

Page 44: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Masks — XRT Coarse

Page 45: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Mask Design — 1D

Definition: q is a quadratic residue (mod p)if ∃n s.t. n2 ≡ q(mod p)

Legendre symbol(ap

)={

1 if a is quadratic residue−1 otherwise

Correlation with zero shift (p − 1)/2Correlation with non-zero shift (p − 1)/4

Page 46: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Mask Design

• Auto Correlation

a(i) = (p − 1)4

(1+ δ(i))

• Power Spectrum

A(j) = (p − 1)4

(δ(j)+ 1)

Page 47: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Masks — Hexagonal

Page 48: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Coded Aperture Extensions

• Artifacts due to Finite Distance

• Mask / Countermask Combination

Page 49: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation
Page 50: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation
Page 52: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Coded Aperture Extensions

• Artifacts due to Finite Distance

• Mask / Countermask Combination

• Multiple Detector Array Positions

• “Synthetic Aperture” radiography

Page 53: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Coded Aperture Applications

• Detection of Fissile Material

• Large Area Detector Myth

• Signal and Background Amplified

Page 54: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Spatially Varying Background

Page 55: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Large Area Alone Doesn’t Help

Page 56: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Imaging and Large Area Do!

Page 57: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Coded Aperture Example

• Imaging — 1/R instead of 1/R2

Page 58: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Coded Aperture Detector Array

Page 59: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Computational Imaging System

Berthold K.P. Horn
Line
Page 60: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Coded Aperture Example

Three weak, distant radioactive sources

Reconstruction Animation

Page 61: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Coded Aperture Applications

• Detection of Fissile Material

• Imaging — 1/R instead of 1/R2

• Increasing Gamma Camera Resolution

• Replacing Rats with Mice

.

Page 62: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

(3) DIAPHANOGRAPHY

(Diffuse Optical Tomography)

• Highly Scattering — Low Absorption

• Many Sources — Many Detectors

with: Xiaochun Yang, Richard Lanza,Charles Sodini, and John Wyatt.

Page 63: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Diaphanography

• Randomization of Direction

• Scalar Flux Density

Page 64: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Diaphanography

• Approximation: Diffusion Equation

∆v(x,y)+ ρ(x,y)c(x,y) = 0

v(x,y) flux density

ρ(x,y) scattering coefficient

c(x,y) absorption coefficient

• Forward: given c(x,y) find v(x,y)

Page 65: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Diaphanography

• Approximation: Diffusion Equation

• Leaky Resistive Sheet Analog (2D)

Page 66: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Diaphanography

• “Invert” Diffusion Equation

• Regions of Influence

.

Page 67: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

(4) EXACT CONE BEAM ALGORITHM

• Faster Scanning—Fewer Motion Artifacts

• Lower Exposure—Uniform Resolution

with: Xiaochun Yang

Page 68: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Exact Cone Beam Reconstruction

• Faster Scanning—Fewer Motion Artifacts

• Lower Exposure—Uniform Resolution

• Parallel Beam → Fan Beam

• Planar Fan → Cone Beam

Page 69: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Parallel Beam to Fan Beam

Coordinate Transform in 2D Radon Space

Page 70: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Cone Beam Geometry — 3D

Page 71: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Radon’s Formula

• In 2D: ~ derivatives of line integrals

• In 3D: derivatives of plane integrals

• Can’t get plane integrals from projections∫ (∫f(r , θ)dr

)dθ

∫ ∫1rf(x,y)dx dy

Page 72: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Radon’s Formula in 3D

f(x) = − 18π2

∫S2

∂2R f(l,β)∂l2

∣∣∣∣∣l=x·β

where

R f(l,β) =∫f(x) δ(x · β− l)dV

Page 73: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Grangeat’s Trick

∂∂z

∫ ∫f(x,y, z)dx dy =

∂∂θ

∫ ∫f(r ,φ,θ)dr dφ

Page 74: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Exact Cone Beam Reconstruction

• Data Sufficiency Condition

• Good “Orbit” for Radiation Source

Page 75: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Radon Space — 2D

Page 76: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Circular Orbit is Inadequate (3D)

Page 77: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Data Insufficiency

Page 78: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Good Source Orbit

Page 79: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

Exact Cone Beam Reconstruction

• Data Sufficiency Condition

• Good “Orbit” for Radiation Source

• Practical Issue: Spiral CT Scanners

• Practical Issue: “Long Body” Problem

.

Page 80: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

COMPUTATIONAL IMAGING

(1) Synthetic Aperture Imaging

(2) Coded Aperture Imaging

(3) Diaphanography—Diffuse Tomography

(4) Exact Cone Beam Reconstruction

Page 81: Computational Imaging - Massachusetts Institute of …scpv.csail.mit.edu/Horn_Computational_Imaging.pdf · What is Computational Imaging? • Computation inherent in image formation

COMPUTATIONAL IMAGING


Recommended