1
Multivariate Textures/Information Theory
in Brain Imaging
Karl Young
University of California, San Francisco
Department of Radiology and Biomedical Imaging
ADVANCED STATISTICAL CONCEPTS FOR MULTIMODAL MRI: THEORY AND APPLICATIONS
Center for Imaging of Neurodegenerative Diseases
June 19, 2010
2
The Challenge
Simultaneous availability of large data sets from a large number of medical imaging modalities (i.e. high dimensional feature space)– structural images (structural MRI, DTI,...)
– functional images (PET, fMRI, pMRI,…)
– metabolite images (MRSI)
Combined analysis is complex and hard to interpret
3
Proposed Approach Use texture/complexity analysis methods derived from
computer vision, machine learning, information theory, and nonlinear time series analysis.
Advantages:
– Automated (“objective” in distinction to region based methods)
– Global (in distinction to VBM, TBM)
– Multimodal (multivariate – simultaneous use of co-registered images is straightforward)
– Multiresolution (Generalizes texture measures -Haralick measures,… - and measures from nonlinear dynamics - fractal dimension, multifractal specrum, dynamical entropies, statistical complexity,… - )
4
Proposed Approach Use texture/complexity analysis methods derived from
computer vision, machine learning, information theory, and nonlinear time series analysis.
Disadvantages: – Texture/complexity measures are abstract and though
statistically efficacious can be difficult to causally associate with interpretable clinical outcomes.
– Sensitivity of texture/complexity measures can vary based on quality of preprocessing steps such as registration, interpolation, and segmentation of images
5
Background
Texture/complexity analysis in images is based on analysis of “higher order statistics” (i.e. relies on multiple image values rather than single image values as in VBM):
– Autocorrelation function: is 2nd order statistic
– Third order moment: is 3rd order statistic
6
Background
Related methods from nonlinear time series analysis (and generalized to image analysis) are based on analysis of correlation structure in time series
In both standard texture analysis and nonlinear time series, use of higher order statistics and correlation structure are attempts to account for nonlinear relationships
7
Background – Texture Analysis
Early work (1970’s) in texture analysis via computer vision research was done by Robert Haralick et al, e.g.:
R. M. Haralick, K. Shanmugam, and I. Dinstein, "Textural features for image classification,” IEEE Trans. Syst., Man, Cybern, vol. SMC-3, pp. 610-621, Nov. 1973.
Haralick defined a heuristically based set of texture measures that was first used to provide automatic region classification in remotely sensed (satellite) images
Long delay between original development of methods and application to medical imaging (maybe due to the fact that some of the early research was classified)
A number of recent applications in areas such as tumor detection from images
8
Background – Nonlinear Time Series Analysis
Early work (1970’s) demonstrated utility of multiresolution analysis (e.g. measuring fractal dimension and multifractal spectrum of chaotic systems)
Attempts (1990’s) to reconstruct nonlinear models from chaotic time series for optimal prediction led to rigorous classification of typical nonlinear dynamical systems in terms of information and computational complexity theories
Generalization of the application of these methods from time series to the analysis of higher dimensional data (e.g. images) proved difficult
9
Background – Combined Approach
Though not possible to directly apply rigorous time series methods to images, application of Haralick style heuristic methods led to hybrid approach for sensitive, interpretable diagnostic classification via image complexity measures:
– K. Young, Y. Chen, J. Kornak, G. B. Matson, N. Schuff. Summarizing Complexity in High Dimensions. Physical Review Letters 94:098701:1-4 (2005).
– K. Young, N. Schuff. Measuring Structural Complexity in Brain Images. Neuroimage. 39(4):1721-1730 (2008).
– K. Young, A. Du, J. Kramer, H. Rosen, B. Miller, M. Weiner, N. Schuff. Patterns of Structural Complexity in Alzheimer’s Disease and Frontotemporal Dementia. Human Brain Mapping. 30(5):1667-77 (2009).
10
Fundamental Construct Fundamental to the generation of texture and complexity
measures is the generation of a co-occurrence matrix from an image or set of co-registered images
Co-occurrence matrices can be thought of as joint histograms
As joint histograms the co-occurrence matrices are density estimates of joint probability density functions associated with the correlation structure of the multivariate, co-registered images
Given the density estimates the texture/complexity measures can be generated using information and computational complexity theories
11
“Future” Conditioned on “Past” - P(q2|q1)
“Past” Conditioned on “future” - P(q1|q2)
Co-occurrence matrix <->Joint Histogram P(q2,q1)
q1q2
12
Use of Co-occurrence Matrices Image co-registration - joint entropy or mutual
information of joint intensity histogram is used as optimization measure for co-registration algorithm
Texture analysis – generated for base and offset pixels over image or region and used to estimate texture measures
Nonlinear time series analysis – estimated joint conditional distribution over past and future sequences
Hybrid method - generated for two regions (arbitrary but with fixed size and offset) over image or region of segmented multivariate image and used to estimate texture/complexity measures
13
For image registration:
14
15
16
For standard texture analysis:
d
17
0
0 1
1
0 1
2525 75 75
q1
q2q1
q2
counts
For time series:…000110101…
18
For Generation of Multivariate Texture/Complexity Measures
q1
q2
Question: How does P(q2|q1) vary as function of q1,q2, and shape ?
19
Multivariate Texture/Complexity Analysis Proceeds in 4 Stages
I. Choice of appropriate Feature Space (e.g. structural MRI, DTI, MRSI, or combinations)
II. Segmentation (Clustering) of Feature Space
III. Generation of Texture/Complexity Estimates
IV. Classification Based on Texture/Complexity Estimates (e.g. supervised or unsupervised)
20
Stage I – Choice of Appropriate Feature Space
Co-registered, warped, interpolated/smoothed
images
Structural
DTI
MRSI
Voxel Grid
Structural
MRSI
DTI
Feature Space
21
Features from all voxels
Stage II - Segment Feature Space (I.e. Find Clusters) and Map Cluster
Values Back to Voxel Grid
MRSI
Structural
DTI
Map clustered features
Back to voxel grid
22
Stage III - Generation of Complexity Estimates
Generate co-occurrence matrix/joint distribution by parsing labeled image (just as in standard texture analysis)
Calculate texture/complexity measures from co-occurrence matrix/joint distribution
23
Stage IV – Classification Based on Texture/Complexity Estimates
Classification using texture/complexity estimates in a supervised or unsuprevised learning algorithm (pick your favorite -LDA, SVM, RVM, Bayes nets, Random Forest,…)
24
3 Class LDA Results For AD, FTD, CN Using Entropy, Statistical Complexity,
Excess Entropy on structural MRI
25
3 Class LDA Results For AD, FTD, CN Using Entropy, Statistical Complexity,
Excess Entropy on structural MRI
26
Conclusion
Texture/Complexity Measures Provide Sensitive/Specific Statistics For Studying Subtle Global, Nonlinear Effects From Images
Collaborators – Yue Chen, John Kornak, Gerald Matson, Michael Weiner, Wang Zhan, Grant Gauger, and Norbert Schuff
