Rice/Duke | Compressive Optical Devices | August 2007 Richard Baraniuk Kevin Kelly Rice University...

Rice/Duke | Compressive Optical Devices | August 2007

Richard BaraniukKevin Kelly

Rice University

Compressive Optical Imaging Systems –

Theory, Devices, Implementation

David BradyRebecca Willett

Duke University


Project Overview

Richard Baraniuk

Digital Revolution

camera arrays hyperspectral cameras

distributed camera networks

Sensing by Sampling

• Long-established paradigm for digital data acquisition– sample data at Nyquist rate (2x bandwidth) – compress data (signal-dependent, nonlinear)– brick wall to resolution/performance

compress transmit/store

receive decompress

sample

sparsewavelet

transform

Compressive Sensing (CS)

• Directly acquire “compressed” data

• Replace samples by more general “measurements”

compressive sensing transmit/store

receive reconstruct

Compressive Sensing

• When data is sparse/compressible, can directly acquire a condensed representation with no/little information lossthrough dimensionality reduction

measurementssparsesignal

sparsein some

basis

Compressive Sensing

• When data is sparse/compressible, can directly acquire a condensed representation with no/little information loss

• Random projection will work

measurementssparsesignal

sparsein some

basis

[Candes-Romberg-Tao, Donoho, 2004]for signal reconstruction

Compressive Optical Imaging Systems –Theory, Devices, and Implementation

• $400k budget for roughly April 2006-2007– administered by ONR – Rice portion expended; Duke portion in NCE

• Goals:– forge collaboration between Rice and Duke teams– demonstrate new Compressive Imaging technologies

hardware testbeds/demos at Rice and Duke new theory/algorithms

– quantify performance– articulate emerging directions

• Collaborations:– telecons, visits, joint projects, joint papers, artwork


Gerhard Richter 4096 Farben / 4096 Colours

1974254 cm X 254 cmLaquer on CanvasCatalogue Raisonné: 359

Museum Collection:Staatliche Kunstsammlungen Dresden (on loan)

Sales history: 11 May 2004Christie's New York Post-War and Contemporary Art (Evening Sale), Lot 34US$3,703,500


Gerhard Richter Dresden Cathedral

Stained Glass

Agenda

• Rebecca Willett, Duke [theory/algorithms]

• Kevin Kelly, Rice [hardware]

• David Brady, Duke [hardware]

• Richard Baraniuk, Rice [theory/algorithms]

• Discussion and Conclusions


Compressive Image Processing

Richard Baraniuk

Mike Wakin

Marco Duarte

Mark Davenport

Shri Sarvotham

PetrosBoufounos

Matthew Moravec

Mona Sheikh

Jason Laska


Image Classification/Segmentation

using Duke Hyperspectral System

(with Rebecca Willett)

Information Scalability

• If we can reconstruct a signal from compressive measurements, then we should be able to perform other kinds of statistical signal processing:

– detection– classification– estimation …

• Hyperspectral image classification/segmentation

Classification Example

spectrum 2

spectrum 1spectrum 3

Nearest Projected Neighbor

• normalize measurements

• compute nearest neighbor

Naïve Results

block size32 16 8

Results

naïve independent classification

tree-based classification

Voting / Cycle Spinningblock radius in pixels

16 20 24

28 32

Summary

• Direct hyperspectral classification/segmentation without reconstructing 3D data cube

• Future directions– replace nearest projected neighbor with more sophisticated

methods smashed filter projected SVM quad-tree based multiscale segmentation (HMTseg, …)

• Joint paper in the works


Performance Analysis of

Multiplexed Cameras

Single-Pixel Camera Analysis

randompattern onDMD array

DMD DMD

photon detector

imagereconstruction

orprocessing

• Analyze performance in terms of – dynamic range and #bits of A/D– MSE due to photon counting noise– number of measurements


Single Pixel Image Acquisiton

For a N-pixel, K-sparse image under T-second exposure:

• Raster Scan: Acquire one pixel at a time, repeat N times

• Basis Scan: Acquire one coefficient of image in a fixed basis at a time, repeat N times

• CS Scan: Acquire one incoherent/randomprojection of the image at a time,repeat times


Worst-Case Performance

• N: Number of pixels• P: Number of photons per pixel• T: Total capture time• M: Number of measurements• CN: CS noise amplification constant

• Sensor array shown as baseline• Table shows requirements to match worst-case

performance• CS beats Basis Scan if


Single Pixel Camera Experimental Performance

N = 16384M = 1640 = Daub-8

Multiplexed Camera Analysis

randompattern onDMD array

DMD DMD

S photon detectors

imagereconstruction

orprocessing

lens(es)

Dude, you gotta

multiplex!


S-Pixel Camera Performance


Sensor array shown as baselineM measurements split across S sensorsSingle pixel camera: S = 1


S-Pixel Camera Performance


Sensor array shown as baselineM measurements split across S sensorsSingle pixel camera: S = 1CS beats Basis Scan if


Smashed Filter –

Compressive Matched Filtering

Information Scalability

• If we can reconstruct a signal from compressive measurements, then we should be able to perform other kinds of statistical signal processing:

– detection– classification– estimation …

• Smashed filter: compressive matched filter

Matched Filter• Signal classification in additive white Gaussian noise

– LRT: classify test signal as from Class i if it is closest to template signal i

– GLRT: when test signal can be a transformed version of template, use matched filter

• When signal transformations are well-behaved, transformed templates form low-dimensional manifolds– GLRT

= matched filter= nearest manifold classification

M1

M2M3

Compressive LRT

• Compressive observations

• By the Johnson-Lindenstrauss Lemma, random projection preserves pairwise distances with high probability

Smashed Filter• Compressive observations of transformed signal

• Theorem: Structure of smooth manifolds is preserved by random projection w.h.p. provided

distances, geodesic distances, angles, volume, dimensionality, topology, local neighborhoods, …[Wakin et al 2006; to appear in Foundations on Computational Mathematics]

M1

M2M3

M1

M2M3

Stable Manifold EmbeddingTheorem:

Let F ½ RN be a compact K-dimensional manifold with– condition number 1/ (curvature, self-avoiding)

– volume V

Let be a random MxN orthoprojector with

[Wakin et al 2006]

Then with probability at least 1-, the following

statement holds: For every pair x,y 2 F

Manifold Learning from Compressive Measurements

ISOMAP HLLELaplacian

Eigenmaps

R4096

RM

M=15 M=15M=20

Smashed Filter – Experiments

• 3 image classes: tank, school bus, SUV

• N = 65536 pixels• Imaged using single-pixel CS camera with

– unknown shift– unknown rotation

Smashed Filter – Unknown Position

• Object shifted at random (K=2 manifold)• Noise added to measurements• Goal: identify most likely position for each image class

identify most likely class using nearest-neighbor test

number of measurements Mnumber of measurements M

avg

. sh

ift

est

imate

err

or

class

ifica

tion

rate

(%

)more noise

more noise

Smashed Filter – Unknown Rotation

• Object rotated each 2 degrees

• Goals: identify most likely rotation for each image classidentify most likely class using nearest-neighbor

test

• Perfect classification withas few as 6 measurements

• Good estimates of rotation with under 10 measurements

number of measurements M

avg

. ro

t. e

st.

err

or

How Low Can M Go?

• Empirical evidence that many fewer than measurements are needed for effective classification

• Late-breaking results (experimental+nascent theory)

Summary – Smashed Filter

• Compressive measurements are info scalablereconstruction > estimation > classification > detection

• Random projections preserve structure of smooth manifolds (analogous to sparse signals)

• Smashed filter: dimension-reduced GLRT for parametrically transformed signals– exploits compressive measurements and manifold structure– broadly applicable: targets do not have to have sparse

representation in any basis– effective for detection/classification– number of measurements required appears to be

independent of the ambient dimension


Compressive Phase Retrieval

for Fourier Imagers

Coherent Diffraction Imaging

• Image by sampling in Fourier domain

• Challenge: we observe only the magnitude of the Fourier measurements

Phase Retrieval

• Given: Fourier magnitude+additional constraints (typically support)

• Goal: Estimate phase of Fourier transform

• Compressive Phase Retrieval (CPR)

replace image support constraint with a sparsity/compressibility constraint

nonconvex reconstruction


Conclusions and

Future Directions


Project Outcomes

• Forged collaboration between Rice and Duke teams– several joint papers in progress

• Demonstrated new Compressive Imaging technologies– hardware testbeds/demos

hyperspectral, low-light, infrared DMD cameras coded aperture spectral imagers

– new theory/algorithms spectral image reconstruction/classification methods smashed filter

• Quantified performance– coded aperture tradeoffs– multiplexing tradeoff– number of measurements required for reconstruction/classification

Emerging Directions• Nonimaging cameras

– exploit information scalability– attentive/adaptive cameras– meta-analysis– separating “imaging process” from “display”

• Multiple cameras– image beamforming, 3D geometry imaging, …

• Deeper links between physics and signal processing– significance of coherence and spectral projections

• Links to analog-to-information program– nonidealities as challenges vs. opportunities

• Other modalities – THz, LWIR/MWIR, UV, soft x-rays, …


N- Pixel Camera Performance


Sensor array shown as baseline1 sensor per pixel - CS is unnecessary


Smashed Filter under Poisson noise

• Problem: vehicle image classification under variable parameter (shift, rotation, etc.)

• Image acquisition: M random projections under signal-dependent (Poisson) noise with single pixel camera

• Limited capture time T split among M projections• Solution: use articulation manifold structure and

generalized maximum likelihood classification (smashed filter)


Smashed Filter performance under Poisson noise

Shift (2D manifold) Rotation (1D manifold)

• Small number of measurements M for good performance• “Sweet spot” on M for shorter exposures T

CS Hallmarks

• CS changes the rules of the data acquisition game– exploits a priori signal sparsity information

• Universal – same random projections / hardware can be used for

any compressible signal class (generic)

• Democratic– each measurement carries the same amount of information– simple encoding– robust to measurement loss and quantization

• Asymmetrical (most processing at decoder)

• Random projections weakly encrypted


Smashed Filter:

How Low Can M Go?


Preservation of Manifold Structure

• Manifold Learning

Used for classification, visualization of high dimensional data, robust parameter estimation

• Network of single-pixel cameras ==== Randomly projected version of low-dimensional image manifold.

New result: stable manifold learning is possible without ever reconstructing the original images

Number of measurements sufficient for arbitrarily small learning error: linear in the information level K of the manifold


Translating disk manifold

• Learning algorithm: LTSA (Zhang, Zha. 2004.)

25 random projections



N = 64 x 64 = 4096, K = 2

Learning with original data:(N = 4096)


Manifold learning using random projections

• Demonstrates that random projections contain sufficient information about the manifold structure

• Two stages in manifold learning– Intrinsic dimension estimation– Construction of nonlinear map into low-dimensional

Euclidean space– New result: estimation errors in both stages due to

dimensionality-reducing projections can be controlled up to arbitrary accuracy with small number of measurements

• Ideal for distributed networks; sensors need to transmit very few pieces of information to the centralized learning algorithm


Intrinsic Dimension estimation • GP algorithm used directly on random projections

of hyperspheres• Empirically compute the number of measurements

required for estimate to be within 10% of the original.

• Observation: M linear in the intrinsic dimension K


Real data: Hand rotation database

N = 64 x 60 = 3840, K = 2


New Bound for Classification?

• Smashed Filter – Nearest Neighbor classifier

• Indyk, Naor. 2007 : preservation of approximate nearest neighbors requires merely O(K) random projections

• Minimum number of measurements required for classification in noiseless case (where D is the minimum separation between signal classes ):

M =O K log 2 / D / D 2


Experiment: Hyperspherical manifolds

• 1000 labeled training samples each from two unit 3-dimensional hyperspheres, separated by a distance D along an arbitrary direction in 2000-dimensional space

• Generate unlabeled samples, perform nearest neighbor classification in the compressed (“smashed”) domain

• Determine minimum number of measurements M required to obtain 99% classification rate.

• Bound: M decreases as square of the separation distance.


Hyperspherical manifolds: empirical verification of bound

Why Does CS Work (1)?• Random projection not full rank, but stably embeds

– sparse/compressible signal models (CS) – point clouds (JL)

into lower dimensional space with high probability• Stable embedding: preserves structure

– distances between points, angles between vectors, …





provided M is large enough: Compressive Sensing

K-dim planes

K-sparsemodel

CS Signal Recovery

• Recover sparse/compressible signal x from CS measurements y via linear programming

K-dim planes

K-sparsemodel

recovery





provided M is large enough: Johnson-Lindenstrauss

Q points

Tree-based classification

• Refine classification of blocks having neighbors from a different class





Date post:	17-Dec-2015
Category:	Documents
Upload:	angelina-hodges
View:	215 times
Download:	0 times

Rice/Duke | Compressive Optical Devices | August 2007 Richard Baraniuk Kevin Kelly Rice University...

Documents