3D Reconstruction of Anatomical Structures from Serial EM images
Biological Motivation• Reconstruct 3D structures
from serial stack of EM images to understand– Distribution of cell type and
organelles.– Connectivity and
vasculature.• Requires
– Tracing of cell membranes, organelles, blood vessels, etc.
– Identification of structures for serial correspondence.
• Currently, reconstruct only a small fraction of volume (very few objects).– Time consuming (~20hours
per specimen).– Wealth of information in
surround structures not utilized.
Synapse Web, Kristen M. Harris, PI
http://synapses.clm.utexas.edu/
Serial TEM Dataset• typical volume:
– 20-50 slices – 8 x 5 x 0.05 m per slice– 40 to 100 m3 volume– fly brain volume: 0.1mm3
• resolution: – xy: 2.6 nm/pixel
(350~400 pixels/ m) – z: 0.05 m ~20 pixels
apart• storage size:
– small volume: ~100MB– fly brain: 3.2x1014 pixels– compression of 4 would
result in 73 terabytes. – (source: Fiala, BU)
serial direction
Data from Synapse Web, Kristen M. Harris, PI
http://synapses.clm.utexas.edu/
Challenges for Computer Vision• Segmenting objects
– EM images are inherently noisy.– Gaps in membrane.– Adjacent structures share weak
membrane boundary.– Organelles too small to use
common descriptors such as texture.
• Identification and correspondence– Structures can merge, split,
appear, or disappear (yellow arrow).
– z-axis structures (red) are easier to maintain correspondences than lateral structures (green).
– z resolution much lower than xy resolution (large changes serially).
– Automatic registration difficult (no ground truth)
• 3D reconstruction– good software available, but
getting to this step is the challenge.
Preliminary 2D Segmentation
• Parametric snakes• Red-initial contour• Green-final contour• Highly sensitive to
initialization (bottom)
• Automatic initialization is a big challenge.
Preliminary 2D Segmentation
• Geometric active contours.
• Provides topological flexibility.
• Less sensitive to initialization.
• Adjacent objects often merge (bottom).
Preliminary 2D Segmentation
• Level set with elastic edge interaction*
• Zero level contour of v provides “good” initialization.
• Still many problems.
*Xiang et al. J. Comp. Phys. 2006
@Á@t
=µπr ¢
r Ájr Áj
+ v¶jr Áj
v = ¡Z
≠
r ¢r (G ? I )r3
dxdy
@Á@t
=µπr ¢
r Ájr Áj
+ v¶jr Áj
v = ¡Z
≠
r ¢r (G ? I )r3
dxdy
Preliminary 2D Segmentation
• Previous method produces binary masks of cross sections.
• Correspondences can be made based on distance and area of overlap.
• Inconsistencies occur often (green)
Preliminary 3D Reconstruction
• Reconstructing “everything” at the same time produces confusing volume.
• Inconsistencies in segmentation and correspondence produce artifacts.
Open Issues
• 2D Segmentation challenges– Automatic initialization.– Segmenting adjacent objects sharing weak
edges.– Noise.
• Cross section correspondence– Identifying objects (synapse, mitochondria,
etc.)– Tracking contours serially and detecting
merging/splitting events.– Automatic registration.
• Current Work: simultaneous segmentation and correspondence.