Center for Subsurface Sensing Imaging Systems€¦ · Hanu Singh (WHOI) Richard Radke (RPI) Badri...

transcript

Center forSubsurface Sensing & Imaging Systems

NSF Year Six Site VisitApril 4-5, 2006NSF Year Six Site VisitApril 4-5, 2006

Thrusts R2B & D: Localized Probing, Mosaicing, Image Understanding

Badri Roysam (RPI)Hanu Singh (WHOI)Richard Radke (RPI)

Overview of Thrust ActivitiesOverview of Thrust Activities

R2B & D

Shape DynamicsModeling

Image-GuidedIntervention

ChangeDetection

Registration &Mosaicing

SpatialNavigation

ChangeInterpretation

SensorFusion

FeatureExtraction,

Segmentation

General Trends:

1. Maturing of core registration & mosaicing tools2. Shift towards toolkits & application frameworks3. Growing interactions across thrusts

Progress on Core RegistrationProgress on Core Registration

Dual-Bootstrap Algorithm introduced 2 years ago

Feature & intensity-BasedDriven by retinal imaging>99.5 % success on 15,000 image pairsHybrid Feature & intensity basedDeformable 3-D & 4-D models

Recent AdvancementsGeneralized algorithm based on generic featuresImproved LPM Toolkit on CenSSIS websiteLicensed to 2 companies

More Details…R3 session

Dual-Bootstrap Algorithm introduced 2 years ago

Feature & intensity-BasedDriven by retinal imaging>99.5 % success on 15,000 image pairsHybrid Feature & intensity basedDeformable 3-D & 4-D models

Recent AdvancementsGeneralized algorithm based on generic featuresImproved LPM Toolkit on CenSSIS websiteLicensed to 2 companies

More Details…R3 session

Retinal

Generic features

Underwater

Ongoing: Generic Features for HyperspectralImagesOngoing: Generic Features for HyperspectralImages

Classified Multi-scale HSI Interest Points

A AA A AB BB B BB BB B CC C CC CC C CD DD D DD D

UnderseaLand-Sea Border

Land 1 Land 2La Parguera

250x239x118

First step towards extending registration toolkit to HSI dataA promising algorithm:

Based on well-behaved earthmover spectral distance measureRobust to noise, spectral distortions, non-uniform spectral samplingMulti-scale, invariant to rotation, scale, and limited viewpoint changesInterest points can be classified based on spectral signatures

First step towards extending registration toolkit to HSI dataA promising algorithm:

Based on well-behaved earthmover spectral distance measureRobust to noise, spectral distortions, non-uniform spectral samplingMulti-scale, invariant to rotation, scale, and limited viewpoint changesInterest points can be classified based on spectral signatures

Progress on Tube Feature ExtractionProgress on Tube Feature Extraction

Tube-like structures continue to be important

VasculatureNeurites

Advancements:Deformable 3-D super-ellipsoid modelSuper-quadric models of secondary structures (spines)Robust estimation and detection algorithms

Software tool now in routine use at MGH

Validated on ~1000 datasets> 95% accuracy

Tube-like structures continue to be important

VasculatureNeurites

Advancements:Deformable 3-D super-ellipsoid modelSuper-quadric models of secondary structures (spines)Robust estimation and detection algorithms

Software tool now in routine use at MGH

Validated on ~1000 datasets> 95% accuracy

3D Star-convexSuper-ellipsoid

3D in vivo Multi-Photon Image of Tumor Vessels

3D in vivo Multi-Photon Image of Neurites

Spines

(Microvascular Research. 2005)

Another Student AwardAnother Student AwardCo

Red: Post-synaptic proteinGreen: Pre-synaptic ProteinBlue: Neurites

Patterned SurfacePatterned Surface

Hari Iyer:2005 MSA Presidential Award

Smooth Surface

PatternedSurface

Smooth SurfaceSmooth Surface

New NIH Grant: Application to Stem CellsNew NIH Grant: Application to Stem Cells

Blood vesselsBlood vessels

AstrocytesAstrocytes

BulbsBulbs

OrdinaryOrdinary

GFAP+ GFAP+ LeX+ LeX+

GFAP+ , LeX+ GFAP+ , LeX+

EEndothelialEndothelial

R01 NS051531, 2006 → 2011Neurovascular Stem-Cell Niche

Mosaicing Image Analysis ResultsMosaicing Image Analysis ResultsCo

Change Analysis – a Pervasive ProblemChange Analysis – a Pervasive Problem

Feature/ObjectFeature/ObjectExtractionExtraction Short-Term

ChangeAnalysis

Change Description

Structured ChangesStructured Changes(accept)(accept)

Unstructured ChangesUnstructured Changes(score manually)(score manually)

No-Change Regions(ignore)

Library ofLow-order

Change Models

Registration

Robust Change

Detection

Objects,Attributes

ChangeMask

Correspondences, Alignment Parameters

Physics-BasedCorrections

Longer-TermEvent Analysis

Library ofHigh-order

Change Models

PhysicsPhysics--BasedBasedIntensityIntensity

CorrectionsCorrections

BackgroundEstimation{ ( )}I t

570 nm 600 nmVein Occlusion

Macular Degeneration (FA)

Vascular Changes

Functional Changes from 2-wavelength

J. Biomedical Optics (2005),IEEE-TBME (2005)

Application to Retinal ImagesApplication to Retinal Images

Geographic Atrophy

neurite length

0 25 50 75 100 125 150 175 200 225 250 275

Kinase Concentration

0 25 50 75 100 125 150 175 200 225 250 275

Neurite Length GFP Signal

Application to NeurobiologyApplication to Neurobiology

IEEE-TBME (2006)

Application to Cultured Progenitor CellsApplication to Cultured Progenitor CellsCo

Cell Cycle(Feb 2006)

Application to Embryo DevelopmentApplication to Embryo Development

4-D Spinning Disk ConfocalSequence, every 2 minsCo

H1-GFP C. elegans worm

Cell Tracking

Automated Lineaging & Migration AnalysisAutomated Lineaging & Migration Analysis

Black:LineageRed:Cell Movement

R2B: Summary & Future PlansR2B: Summary & Future Plans

SummaryMature registration and mosaicing capabilitiesPowerful toolkit released, being generalized furtherGrowing focus on applications

E.g., change analysis, multi-sensor, multi-spectralGrowing interactions across thrusts

The Near TermMosaicing of image understanding resultsExpanded multi-modality supportInterface with external toolkit efforts (e.g., Insight, MATLAB)

The Longer TermGeneralized change interpretation systems

Information-theoretic approachesModel-based separation of changes and motion

Widen Applications Base