Center forSubsurface Sensing & Imaging Systems
Center forSubsurface Sensing & Imaging Systems
NSF Year Six Site VisitApril 4-5, 2006NSF Year Six Site VisitApril 4-5, 2006
Thrusts R2B & D: Localized Probing, Mosaicing, Image Understanding
Thrusts R2B & D: Localized Probing, Mosaicing, Image Understanding
Badri Roysam (RPI)Hanu Singh (WHOI)Richard Radke (RPI)
Badri Roysam (RPI)Hanu Singh (WHOI)Richard Radke (RPI)
Overview of Thrust ActivitiesOverview of Thrust Activities
R2B & D
Shape DynamicsModeling
Image-GuidedIntervention
ChangeDetection
Registration &Mosaicing
SpatialNavigation
ChangeInterpretation
SensorFusion
FeatureExtraction,
Segmentation
General Trends:
1. Maturing of core registration & mosaicing tools2. Shift towards toolkits & application frameworks3. Growing interactions across thrusts
Progress on Core RegistrationProgress on Core Registration
Dual-Bootstrap Algorithm introduced 2 years ago
Feature & intensity-BasedDriven by retinal imaging>99.5 % success on 15,000 image pairsHybrid Feature & intensity basedDeformable 3-D & 4-D models
Recent AdvancementsGeneralized algorithm based on generic featuresImproved LPM Toolkit on CenSSIS websiteLicensed to 2 companies
More Details…R3 session
Dual-Bootstrap Algorithm introduced 2 years ago
Feature & intensity-BasedDriven by retinal imaging>99.5 % success on 15,000 image pairsHybrid Feature & intensity basedDeformable 3-D & 4-D models
Recent AdvancementsGeneralized algorithm based on generic featuresImproved LPM Toolkit on CenSSIS websiteLicensed to 2 companies
More Details…R3 session
Retinal
Generic features
Underwater
Ongoing: Generic Features for HyperspectralImagesOngoing: Generic Features for HyperspectralImages
Classified Multi-scale HSI Interest Points
A AA A AB BB B BB BB B CC C CC CC C CD DD D DD D
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
A AA A AB BB B BB BB B CC C CC CC C CD DD D DD D
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
UnderseaLand-Sea Border
Land 1 Land 2La Parguera
250x239x118
First step towards extending registration toolkit to HSI dataA promising algorithm:
Based on well-behaved earthmover spectral distance measureRobust to noise, spectral distortions, non-uniform spectral samplingMulti-scale, invariant to rotation, scale, and limited viewpoint changesInterest points can be classified based on spectral signatures
First step towards extending registration toolkit to HSI dataA promising algorithm:
Based on well-behaved earthmover spectral distance measureRobust to noise, spectral distortions, non-uniform spectral samplingMulti-scale, invariant to rotation, scale, and limited viewpoint changesInterest points can be classified based on spectral signatures
Progress on Tube Feature ExtractionProgress on Tube Feature Extraction
Tube-like structures continue to be important
VasculatureNeurites
Advancements:Deformable 3-D super-ellipsoid modelSuper-quadric models of secondary structures (spines)Robust estimation and detection algorithms
Software tool now in routine use at MGH
Validated on ~1000 datasets> 95% accuracy
Tube-like structures continue to be important
VasculatureNeurites
Advancements:Deformable 3-D super-ellipsoid modelSuper-quadric models of secondary structures (spines)Robust estimation and detection algorithms
Software tool now in routine use at MGH
Validated on ~1000 datasets> 95% accuracy
3D Star-convexSuper-ellipsoid
3D in vivo Multi-Photon Image of Tumor Vessels
3D in vivo Multi-Photon Image of Neurites
Spines
(Microvascular Research. 2005)
Another Student AwardAnother Student AwardCo
llabo
rati
on:
Nat
alie
Dow
ell-
Mes
fin,
Wad
swor
th
Red: Post-synaptic proteinGreen: Pre-synaptic ProteinBlue: Neurites
Red: Post-synaptic proteinGreen: Pre-synaptic ProteinBlue: Neurites
Patterned SurfacePatterned Surface
Hari Iyer:2005 MSA Presidential Award
0.36
0.37
0.38
0.39
0.4
0.41
0.42
0.43
0.36
0.37
0.38
0.39
0.4
0.41
0.42
0.43
Syna
pse
Form
atio
n Ef
fici
ency
Smooth Surface
PatternedSurface
Smooth SurfaceSmooth Surface
New NIH Grant: Application to Stem CellsNew NIH Grant: Application to Stem Cells
B
DC
A
Blood vesselsBlood vessels
AstrocytesAstrocytes
BulbsBulbs
OrdinaryOrdinary
GFAP+ GFAP+ LeX+ LeX+
GFAP+ , LeX+ GFAP+ , LeX+
B
DC
A
EEndothelialEndothelial
R01 NS051531, 2006 → 2011Neurovascular Stem-Cell Niche
Mosaicing Image Analysis ResultsMosaicing Image Analysis ResultsCo
llabo
rati
on:
S. E
arle
y, G
. Pl
oppe
r (R
PI)
Change Analysis – a Pervasive ProblemChange Analysis – a Pervasive Problem
Feature/ObjectFeature/ObjectExtractionExtraction Short-Term
ChangeAnalysis
Change Description
Structured ChangesStructured Changes(accept)(accept)
Unstructured ChangesUnstructured Changes(score manually)(score manually)
No-Change Regions(ignore)
Library ofLow-order
Change Models
Registration
Robust Change
Detection
Objects,Attributes
ChangeMask
Correspondences, Alignment Parameters
Physics-BasedCorrections
Longer-TermEvent Analysis
Library ofHigh-order
Change Models
PhysicsPhysics--BasedBasedIntensityIntensity
CorrectionsCorrections
BackgroundEstimation{ ( )}I t
570 nm 600 nmVein Occlusion
Macular Degeneration (FA)
Vascular Changes
Functional Changes from 2-wavelength
J. Biomedical Optics (2005),IEEE-TBME (2005)
Application to Retinal ImagesApplication to Retinal Images
Geographic Atrophy
Colla
bora
tion
: G
ary
Bank
er (
U.
Ore
gon)
neurite length
0 25 50 75 100 125 150 175 200 225 250 275
time
leng
th
1
2
3
4
5
6
7
8
9
10
11
12
Kinase Concentration
0 25 50 75 100 125 150 175 200 225 250 275
time
prot
ein
1
2
3
4
5
67
8
9
10
11
12
Neurite Length GFP Signal
Application to NeurobiologyApplication to Neurobiology
IEEE-TBME (2006)
Application to Cultured Progenitor CellsApplication to Cultured Progenitor CellsCo
llabo
rati
on:
Sally
Tem
ple
(Alb
any
Med
ical
Col
lege
)
Cell Cycle(Feb 2006)
Application to Embryo DevelopmentApplication to Embryo Development
4-D Spinning Disk ConfocalSequence, every 2 minsCo
llabo
rati
on:
Bill
Moh
ler,
Uni
vers
ity
of C
onne
ctic
ut
H1-GFP C. elegans worm
Cell Tracking
Automated Lineaging & Migration AnalysisAutomated Lineaging & Migration Analysis
Black:LineageRed:Cell Movement
R2B: Summary & Future PlansR2B: Summary & Future Plans
SummaryMature registration and mosaicing capabilitiesPowerful toolkit released, being generalized furtherGrowing focus on applications
E.g., change analysis, multi-sensor, multi-spectralGrowing interactions across thrusts
The Near TermMosaicing of image understanding resultsExpanded multi-modality supportInterface with external toolkit efforts (e.g., Insight, MATLAB)
The Longer TermGeneralized change interpretation systems
Information-theoretic approachesModel-based separation of changes and motion
Widen Applications Base
SummaryMature registration and mosaicing capabilitiesPowerful toolkit released, being generalized furtherGrowing focus on applications
E.g., change analysis, multi-sensor, multi-spectralGrowing interactions across thrusts
The Near TermMosaicing of image understanding resultsExpanded multi-modality supportInterface with external toolkit efforts (e.g., Insight, MATLAB)
The Longer TermGeneralized change interpretation systems
Information-theoretic approachesModel-based separation of changes and motion
Widen Applications Base
