NA-MICNational Alliance for Medical Image Computing http://na-mic.org

Techniques for Visualization of Anatomy and Function from MRISteve Pieper, PhD

VizBi 2010Heidelberg

National Alliance for Medical Image Computing http://na-mic.org

Outline

• Motivation

• Standard Clinical MRI Visualization

• Research Imaging– Registration– Segmentation and Parcellation– Functional Imaging

• Applications and Open Issues

National Alliance for Medical Image Computing http://na-mic.org

Example: Morphometry Group Statistics

• Regional Cortical Thickness Correlation with Aging and Cognitive Impairment– Greater Sensitivity than Clinical Tests– Localize Disease within Brain Anatomy– Impact of Treatment on Imaging Biomarkers

Fischl, Greve, et al - MGH

National Alliance for Medical Image Computing http://na-mic.org

Example: Multimodality Image Guided Neurosurgery

• Detailed Pre-Operative Model– Registered to Tracked

Instruments– Superimposed on Intraoperative

MRI

• Extracted Anatomy and Function– Roadmap for Surgical Decision

Making

• What is the Best Surgical Approach to Preserve Function?

Golby, Kikinis, Lemaire, BWH Neurosurgery

National Alliance for Medical Image Computing http://na-mic.org

MRI Context• MRI: Typically Volumetric at ~1mm

Resolution– Unlike CT, No Physical Units for MR

– Contrast Determined by Scan Protocol

• Much of clinical MRI is pattern recognition– Seeing things that “don’t look right”

• Anatomical Conventions– Like Maps having North at the Top– Axial (Transverse), Sagittal, Coronal

• Analysis Results– Spatially Localized Data– Often visualization is used as a

‘reasonableness check’ on the automated calculations

Wikipedia

National Alliance for Medical Image Computing http://na-mic.org

Basic Clinical Visualization

• Window/Level• Corner Annotations• Pseudocolor• Mosaic/Lightbox• Cine

National Alliance for Medical Image Computing http://na-mic.org

3D Clinical Visualization

MR Angiogram MIP from Siemens Leonardo Workstation

• Ray Casting Through Volume– Summation (Simulated X-Ray)– MIP (Maxiumum Intensity Projection)– SSD (Shaded Surface Display)

• Color Transfer Function and Opacity Transfer Function

• Pseudocolor + Gradient Lighting

• Less Applicable for MRI than CT

• Reference Labels for Standard Views– Left/Right, Anterior/Posterior,

Inferior/Superior

National Alliance for Medical Image Computing http://na-mic.org

MRI Contrast Enhancement

• Vascular / Oral: Gadolinium, Iron Oxide, Manganese– Change MR Properties of Tissues

• Dynamic Contrast Tissue Enhancement (4D)– Vascularized Tumor– Stroke Related Ischemia

• Perfusion, Diffusion, Bloodflow• Mismatch Indicates Brain Tissue that

May Recover

Gilbert et al University of Wisconsin

Gonzalez et al MGH

National Alliance for Medical Image Computing http://na-mic.org

Longitudinal Imaging (4D)

• Volumes Acquired Over Multiple Visits– “Watchful Waiting” Prior to

Intervention– Monitor Treatment (Multiple

Sclerosis, Cancer, Lupus…)

• Comparison View– Linked Cursors– Subtraction Imaging and

Quantification

Guttmann, Meier, Fedorov – BWHMiller - GE

National Alliance for Medical Image Computing http://na-mic.org

Registration

• Intra-subject– Pre-Intra-Post Procedure– Longitudinal Tracking of Disease Progression

• Inter-subject– Support Group Comparison (fMRI)– Map Anatomical Atlas to Individual

• Degrees of Freedom (DOFs) – Rigid (Rotation + Translation) – Similarity (Rigid + Uniform Scale)– Affine (Rigid + Nonuniform Scale and Shear)– Polyaffine (Locally Affine Interpolation)– B-Spline (Cubic Displacement) – Vector Field

Sota - BWH

National Alliance for Medical Image Computing http://na-mic.org

Registration Considerations

• Registration is Typically an Ill-Posed Problem– Requires Nonlinear Optimization– Distortions, Anatomy, Pathology Means No

Exact Correspondence– Multimodality Registration Requires Statistical

Metric (e.g. Mutual Information)– When No “Right” Answer

• Only What “Looks Right”

or

• What is Reproducible and Statistically Significant

• Insight Toolkit (ikt.org) for Details and Software

FixedImage

MovingImage

Metric

Transform

Interpolator

Optimizer

Registration Method

Ibanez - Kitware

National Alliance for Medical Image Computing http://na-mic.org

Registration Visualization

• Cross Fade / Toggle

• Color / Skeleton Overlay

• Checkerboard• Vector Fields

National Alliance for Medical Image Computing http://na-mic.org

Segmentation

• Definition: Assignment of Anatomical Labels to Image Regions– Not an Exact Science

• Anatomists Disagree• Definition Depends on Scale

• Techniques– Intensity Driven: Function of Image Measurements

• Thresholding is Most Common (Typically Bad for MRI)

– Rule Based: • E.g. “Skin is always on the outside”• Obviously not always the case in clinical scans

– Atlas Driven: Registration of Manually Labeled Data• Also difficult for clinical scans

– Hybrid Approaches Typically Required• E.g. Expectation Maximization (EM)

Pohl – IBMKikinis, Shenton - BWH

National Alliance for Medical Image Computing http://na-mic.org

Segmentation Visualization

• Label Map Overlay– Cross Fade / Toggle– Solid or Outline

• 3D Surface Models– Leverage Commodity Graphics

Cards– Material Properties, Lighting,

Transparency…

National Alliance for Medical Image Computing http://na-mic.org

Parcellation

• Functional or Anatomical Subdivisions (e.g. Cerebral Hemisphere Surface)

• Obtained from Curvature and Landmarks on Surface– Flattened to Plane or Sphere for Tractability– Displayed Inflated to Show Sulci (Valleys) and Gyri (Ridges)

Fischl et al - MGH

National Alliance for Medical Image Computing http://na-mic.org

Functional MRI

• Blood Oxygen Level Dependent (BOLD)• Volume Time Series ~2mm Resolution• Activation Statistics Volume: Typically

Generalized Linear Model (GLM) of– Hemodynamic Response Function (HRF)– Stimulus / Paradigm– Intensity Trends– Physiology– Motion…

• Statistics Become Difficult– Noisy Data– Multiple Comparisons / Multiple Regressors– Detects Metabolism, not Neural Activity Directly

Plesniak, Liu - BWH

Finger Tapping

National Alliance for Medical Image Computing http://na-mic.org

fMRI Visualization• Statistics Volume• Multi Volume Rendering• Cortical Surface Map• Group Comparisons in Atlas Space (Spatial

Normalization)– Group Contrasts (Left/Right, Schizophrenics/Normal Control,

Active/Resting…)– Positive and Negative Correlations

Plesniak, Liu – BWHHernell – Linköpings U.

National Alliance for Medical Image Computing http://na-mic.org

Multi-Modality Imaging

• Integrated Visualization of What is Known About the Subject– Anatomical Space as

Common Coordinate System

– Segmented Anatomy and Volume Rendering for Context

– Statistics Volumes

– Interactive Visualization (View, Visibility, Cropping, Slicing… )

• Image Guided Therapies

Plesniak, Aucoin et al - BWHJakab and Berenyi - University of Debrecen

National Alliance for Medical Image Computing http://na-mic.org

Group Comparisons

• Visualization for MRI Data Mining– Images, Clinical Data, Demographics

on Web Database– Target Population and Hypothesis

Specify Batch Computation– Group Statistics Overlay on Inflated

Cortical Surface of Atlas– Click to Get Scatter Plot of Subjects

• Supports Alzheimer’s Disease Research

• Anatomy Labels Linked to Web Resources and Journal Papers

Morphometry BIRN Consortium

National Alliance for Medical Image Computing http://na-mic.org

Open Challenges in MRI Visualization

• Stereo / Virtual Reality (VR) / Augmented Reality (AR) / Telepresence– No real traction in spite of significant

investment– Probably due to lack of mainstream hardware

• Dynamic multimodal volume rendering– Large volumes – CUDA vs. Cluster– Encode segmentation in transfer function

• General Information Overload• Display of Uncertainty in Analysis Results

– Many techniques require discretization