SI: GENETIC NEUROIMAGING IN AGING AND AGE-RELATED DISEASES

Graphical neuroimaging informatics: Applicationto Alzheimer’s disease

John Darrell Van Horn & Ian Bowman &

Shantanu H. Joshi & Vaughan Greer

# Springer Science+Business Media New York 2013

Abstract The Informatics Visualization for Neuroimaging(INVIZIAN) framework allows one to graphically displayimage and meta-data information from sizeable collectionsof neuroimaging data as a whole using a dynamic and com-pelling user interface. Users can fluidly interact with an entirecollection of cortical surfaces using only their mouse. Inaddition, users can cluster and group brains according inmultiple ways for subsequent comparison using graphical datamining tools. In this article, we illustrate the utility ofINVIZIAN for simultaneous exploration and mining a largecollection of extracted cortical surface data arising in clinicalneuroimaging studies of patients with Alzheimer’s Disease,mild cognitive impairment, as well as healthy control subjects.Alzheimer’s Disease is particularly interesting due to thewide-spread effects on cortical architecture and alterations ofvolume in specific brain areas associated with memory. Wedemonstrate INVIZIAN’s ability to render multiple brain sur-faces from multiple diagnostic groups of subjects, showcasethe interactivity of the system, and showcase how INVIZIANcan be employed to generate hypotheses about the collectionof data which would be suitable for direct access to theunderlying raw data and subsequent formal statistical analysis.Specifically, we use INVIZIAN show how cortical thicknessand hippocampal volume differences between group are evi-dent even in the absence of more formal hypothesis testing. In

the context of neurological diseases linked to brain aging suchas AD, INVIZIAN provides a unique means for consideringthe entirety of whole brain datasets, look for interesting rela-tionships among them, and thereby derive new ideas forfurther research and study.

Keywords Neuroimaging . Graphical informatics . Aging .

Alzheimer’s Disease . GPU processing . OpenGL

Introduction

Recent advancements in imaging protocols combined with areduction in storage costs have led to an upsurge of neuroim-aging data in both clinical as well as research settings. Increas-ingly, neuroimaging databases are capable of containing im-age volumes in excess of hundreds or thousands of subjectsVan Horn et al. 2005, Van Horn and Toga 2009a, b, Biswalet al. 2010.Moreover, brain databases are often limited to text-based metadata searches of their contents thus limiting theuser interaction considerably. Once the complete set of imagedata files has been downloaded, a user is obliged to conduct aformal analysis of the data simply to discover if the datacontain any particular effects of interest. Finally, many com-monly available visualization tools ideally designed for neu-roimaging focus on single subject data and are not conduciveto plotting such multi-subject relationships. In the context ofdata mining and exploratory inspection of database content,this process is inefficient and time consuming.

Many neuroscience database uses follow a typical process-ing and analytical approach which begins with a text-basedsearch for relevant data and then downloading the raw imag-ing files for analysis. The commonly employed neuroimagingprocessing framework involves the fitting of the imaging datafrom each subject to a common spatial frame of reference inthe form of a standardized brain atlas (Collins et al. 1994,

Submitted to: Brain Imaging and Behavior Special Issue onNeuroimaging and Genetics in Aging and Age-related Disease

J. D. Van Horn (*) : I. Bowman :V. GreerThe Institute for Neuroimaging and Informatics, Keck School ofMedicine, University of Southern California, 2001 North Soto Street– SSB1-102, Los Angeles, CA 90032, USAe-mail: jvanhorn@usc.edu

S. H. JoshiDepartment of Neurology, UCLA School of Medicine, 635 CharlesE. Young Drive, NRB #225, Los Angeles, CA 90095, USA

Brain Imaging and BehaviorDOI 10.1007/s11682-013-9273-9

Nowinski and Belov 2003). The individual brain volumes arethereby subjected to warping with respect to specific templatebrain volume in order to remove the anatomical differences inbrain structure (Lancaster et al. 1999, Nowinski andThirunavuukarasuu 2001), permitting population-level aver-aging (Van Horn and Toga 2009a, b), and comparisons ofanatomy between groups of subjects irrespective of overallbrain size (Mega et al. 2005). Final steps involve the specifi-cation of an experimental design and inferential statisticalmodeling, used to assess morphological differences accordingto phenotypic observations of interest, such as cortical thick-ness in relation to patient diagnosis (Thompson et al. 2001).

Such analysis-driven techniques can provide useful aver-age summaries of a collection of scans from a database (VanHorn and Toga 2009a, b). However, these mappings often donot allow a user to view the structure and variation of multipledata sets across different subject-based categories. For in-stance, there are hippocampal volume deficits (Carmichaelet al. 2005) as well as white matter alterations (Carmichaelet al. 2010) in the case of Alzheimer’s disease which maycorrelate with disease duration, severity, genetic susceptibility,and other factors (Stein et al. 2012). While correlationalanalyses representing this result could be obtained by mergingdata from multiple subjects into an atlas space, extracting thehippocampus, and conducting a formal statistical analysis, thiswould be time consuming, computationally expensive, andmight prohibit independent visualization of individual sub-jects with respect to one another, per se. Consequently, suchan approach tends to prohibit or severely limit a moreexploratory examination of database contents. In otherwords, data mining or exploratory analyses are not easilyperformed and a user must commit to a more formal imageprocessing and statistical analysis. As a means for browsingdatabase contents, this approach is costly and counter to thenotion of exploring databases from a content-driven point ofview.

Thus, to rapidly understand and explore the image-centricdata contained within these repositories can often becomeboth computationally demanding as well as operationallyoverwhelming. What is needed is a means by which usersmay explore relationships in these archives through interac-tion with the data themselves in contrast to only text-basedsearches and then inspection of data one subject at a time.Methods for data mining and exploratory analysis have beenproposed (Megalooikonomou et al. 2000) and would seemideally suited for examining database contents. For instance,in the context of neuroanatomical imaging data, it wouldgreatly assist database users if they could directly view repre-sentations of the subject’s data in an interactive way, discoverthe basic relationships between neuroanatomical features andsubject metadata, as a prelude to downloading the data for aformal statistical examination. Dimension reduction, graphi-cal display, and data-mining approaches form a useful

combination of tools which, when used collectively, can beemployed to interactively examine potentially large amountsof neuroimaging data at once. This has clear advantages fordatabase interactivity, the discovery of interesting effects pres-ent therein, and hypothesis generation (Mennes et al. 2012,Voytek and Voytek 2012).

We recognized that the ability to display multiple brain datasets simultaneously represents an essential step for data ex-ploration and mining, allowing users to examine patterns ofsimilar brain anatomy in advance of subsequent processing,atlas fitting, averaging, and computational analysis (Joshiet al. 2009). Through the process of exploring the similarityspace of brain anatomy from across a large collection ofpreprocessed MR data, a user might be ideally positioned toidentify interesting patterns in the data which would not beappreciated by averaging or by inspection of the metadataalone.

With this idea in mind, we developed a novel frameworkfor the interactive discovery of brain morphological featureand meta-data attribute relationships drawn from a large-scalearchive of structural MRI data. The goals of our work in thearea are as follows: 1) To demonstrate a visually compellinginterface for displaying multiple cortical surfaces computedfrom a repository of raw brain imaging data in which usersmay dynamically interact with the complete collection ofsurfaces and rapidly examine feature relatedness; 2) Illustratea novel’s text query-based framework which provides for theimmediate interactive investigation of how subject metadatavaries with regard to extracted morphological metrics; and 3)showcase graphical meta-analytics for visualizing the basicrelationships between morphological metric values and sub-ject metadata which may help to set the stage for subsequentformal statistical modeling.

In what follows, we illustrate the use of the InformaticsVisualization for Neuroimaging (INVIZIAN) framework(Bowman et al. 2012) in the use-case context of neuroimagingdata relevant to human brain aging and Alzheimer’s Disease.The INVIZIAN software can be accessed via our projectwebsite, http://invizian.loni.usc.edu, with easy-to-use installa-tion packages available to deploy the software on PC andLinux platforms, and links to the source code repository(http://bitbucket.org/uscini/invizian). INVIZIAN is alsorepresented on the Neuroimaging Tools and ResourcesClearinghouse (NITRC) website (http://www.nitrc.org/projects/invizian).

Background and related work

Graphical visualization, dimension reduction, and querybased database searches are widely available individuallybut have not been used together under a single frameworkpreviously. In this section we review several of these

Brain Imaging and Behavior

elemental approaches for data exploration noting theirstrengths and weaknesses:

Visualization in neuroscience The prevailing graphical brainimaging tools such as FreeSurfer (Dale et al. 1999),BrainVoyager (Goebel et al. 2006), and BrainSuite (Shattuckand Leahy 2002) are specifically designed to display brainsurface models from a single subject at a time. This is becausetheir focus tends to be on analysis and visualization of subject-specific cortical data along with the feature values such asthickness, volume, curvature etc. In these approaches, whileimportant for processing and visualization of brain surface-based results at the individual subject level, the examination ofmultiple subjects at a time is not possible.

Dimensionality reduction Dimension reduction is widelyused as a primary step to reduce the overall complexity oflarge volumes of data while minimizing information-loss.There are a number of well-known approaches. PrincipalComponent Analysis (PCA) (Rencher 2002) transforms thedata to a reduced dimensional representation using a set oforthogonal vectors which seek to explain variance compo-nents. Multidimensional scaling (MDS) (Kruskal and Wish1978) is a strategy aiming to replicate the similarity betweendata in native representation using a lower dimensional repre-sentation. Kohonen's Self OrganizingMaps (SOM) (Kohonen1998) are a form of artificial neural network that produce atopology-preserving two-dimensional representation of theinput data. While dimension reduction successfully reducesstructural complexity of the input, in neuroimaging the rela-tionship between the restructured output and original data issometimes unclear.

Prior work has implemented an interactive component ontop of the dimension reduction to improve the focus andintuition of the calculations. An MDS system is proposed in(Williams and Munzner 2004) which allows a user to itera-tively reduce the dimensionality of data, while targeting re-duction computation toward regions of interest. Visual Hier-archical Dimension Reduction (VHDR) (Yang et al. 2003a)constructs and arranges dimensions into a hierarchy, presentedas a radial interface facilitating user exploration of dimension-al configuration. Dimension Ordering Spacing and Filtering(DOSFA) (Yang et al. 2003a, b) extends VHDR by addingautomated components seeking to identify lower dimensionalsubspace structure. Johansson and Johansson (2009) proposean interactive dimension reduction system driven by user-defined quality metrics. Any dimension reduction approachcan lead to information loss, but in this interactive system theuser is in effect able to decide what dimensions are importantby appropriately weighting correlation, clustering and outliersignificance. Once the user assigns weights and a dimension-ality threshold, the system provides a graph illustrating infor-mation loss per variable reduced. In another compelling recent

paper Endert et al. (2011) consider a means for users tographically manipulate various dimension reduction modelparameters through clicking and dragging of a two dimen-sional data plot. The users of the observation-level systemoperate entirely with the spatial interface and do not need tomodify numerical parameters directly to alter the visualiza-tion. In addition to dimension reduction, visualization frame-works such as Ggobi (Cook and Swayne 2007) provides theability to link associated graphical displays together using acoordinated multiple view approach.

Query-based search A query-based search system can bethought of as one facilitating guided discovery of significantdata values through user query-result interaction. This can beperformed through text-based phrases or via a user interfacefor selecting various criteria upon which to search. Resultsfrom such searches can be displayed as a list or table, but moreinterestingly as a graphical rendering. One approach for searchvisualization was pioneered by Keim and Kriegel (1994) intheir method entitled VisDB for building a multidimensionalview of a relational database repository. The authors definedtype-specific distance metrics which were employed againstall data to rank relevance to each user query. Feedback wasthen presented as a pixel map, with a carefully chosen colorscheme depicting distance of query result from significantnear matches. In a related approach, the program called Scout(McCormick et al. 2004), uses GPU hardware acceleration toenable the guidance of visualization processing via mathemat-ical evaluation. Users interact with the system through acustomizable API which manipulates pixel parameters duringvisualization to identify statistically interesting combined pa-rameter values.

INVIZIAN system architecture

In designing the INVIZIAN program, we sought to develop agraphically compelling system in which users could easilyvisualize a set of cortical surfaces as an entire collection andinteract with them in a fluid manner using only their computermouse (Bowman et al. 2012). By combining tools for viewingrelationships between clusters of similarly featured corticalsurfaces, enabling searches over those data, permitting group-ing and meta-analytic assessment, the INVIZIAN interface isspecifically developed for users to explore and discover rela-tionships that might not be appreciated in a strictly text-basedsearch as a prelude to more formal statistical treatment. Usinghigh-density surface meshes in an OpenGL-based (http://www.opengl.org) interface, users may zoom in, rotate, andexamine in detail the regional cortical patterns of within theclusters of similar brains. A summary of the INVIZIAN visual

Brain Imaging and Behavior

analytics system is illustrated in Fig. 1 and briefly consists ofthe following basic elements:

Feature analytics, extraction, and representation Image pre-processing for T1-weighted anatomical volumes and subsequentfeature extraction takes advantage of FreeSurfer’s ReconAllcortical surface reconstruction and cortical parcellation algorithm(Dale et al. 1999, Fischl et al. 1999). These steps are implement-ed in the LONI Pipeline workflow design and execution envi-ronment (Dinov et al. 2010) to take advantage of efficient parallelcomputing of surfaces and other results. Feature metrics, such asthickness, volume, etc. are extracted for each brain parcellation.Such features have previously served as useful biomarkers whenevaluating the effect of disease (Narr et al. 2005), but are alsorecently being investigated as potential genetic phenotypes(Winkler et al. 2010) that influence heritability of the brain. Asdescribed previously (Joshi et al. 2011), using these regionalfeature vectors, subjects are systematically compared to all othersubjects in the collection using a probabilistic “distance” metric,and then this set of pairwise comparisons is decomposed usingmulti-dimensional scaling (MDS) to reduce its dimensionalityand create a new set of coordinates in which brains which aremost similar are positioned close to one another while thosemostdissimilar are positioned furthest from each other (Fig. 2). Thesenew coordinates form the basis for the simultaneous rendering ofthe entire set of extracted brain surfaces in the INVIZIAN userinterface (Bowman et al. 2012).

Multiresolution brain surface attribute mapping Fluid inter-action with the displayed cortical surfaces is an importantdesign characteristic of INVIZIAN which employs a multi-resolution technique to facilitate interactivity when viewingdata. This has several advantages for when displaying multiple

brain surfaces at once. For instance, a fully zoomed out view ofthe entire dataset collection loads surface meshes decimated by95 %. The triangle count of such data is approximately 30 k,and the disk size of such a dataset is slightly less than 1 MB.However, when INVIZIAN determines that the user is viewingless than six surfaces in the field of view it seamlessly replacesthem with meshes that have been decimated by 80 %. By andlarge, the decimated cortical surface models tend to be visuallyindistinguishable from the non-decimated versions, but affordconsiderable reduction in load-times by having triangle countsof around 100 k at ~3.5 MB in size while providing a finerresolution surface for the user to inspect. When a single brainsurface is selected, it is seamlessly replaced by its full resolutionversion, thereby enabling detailed inspection of its surfacecharacteristics. Additional advantage is taken of GPU-basedprocessing to interpolate and color the brain surfaces to presentan aesthetically compelling experience for the user.

Query-based surface identification, clustering, and coordinatedviewing Within INVIZIAN, a simple metadata dialogue is avail-able if the user right-clicks on a surface with their mouse (Fig. 3).The dialogue box lists subject data at the time the subject scanwas conducted and typical information might include age, sex,weight, disease diagnosis information, as well as MRI scanningparameters. In addition to simply viewing the cortical neuroanat-omy and meta-data for individual subjects, a user might beinterested in how cortical feature values relate to clinical subjectmetadata. The user enters expressions such as ``Subject Age>70" into the query text box and INVIZIAN renders a bubblearound matching subjects (Fig. 4, top). Clustered matches indi-cate potential relationships between subject metadata and theanatomical feature similarity. The user can then have INVIZIANplace a circular glyph around the surfaces to indicate group

Fig. 1 A schematic overview of INVIZIAN system architecture operat-ing on MRI cortical surface data. The filled top boxes represent fixedprogram functionality, while the outlined bottom boxes represent variable

inputs defined by user-driven interactivity. Any of these user-definedvalues may be saved to an INVIZIAN scene file for use across visualanalysis sessions

Brain Imaging and Behavior

membership (Fig. 4, bottom) and store these groupings for laterrecall (Fig. 5).

INVIZIAN also provides a “top-down” or coordi-nated viewing window for the entire set of data whichis linked to user mouse movements. When the userpans, zooms, or rotates, the projection renders a rect-angle on top of a “dot plot”, indicating the region ofdisplay currently occupying the view. The coordinatedprojection frees the user of either memorizing subjectplacements, or having to continually zoom out to re-orient the view, subject-to-subject relationships. Examples

of coordinated projection sub-plots can be seen in Figs. 5and 9.

For the interested reader, an example data set for use in explor-ing INVIZIAN is available by contacting the corresponding author.

Use case: Graphical informatics applied to examiningAlzheimer’s disease

In order to illustrate how the INVIZIAN framework can beutilized to conduct exploratory analyses on neuroimaging

Fig. 2 The initialized INVIZIAN similarity-based display of corticalfeature information. The cortical surfaces are positioned according tofeature-descriptive offsets in three dimensions, so that degree of proxim-ity indicates similarity of neuroanatomical feature. Other arrangementsare also possible. Feature values on the cortical surfaces are color mapped

which, when viewed across subjects, highlights neuroanatomical trends.For instance, the surfaces pictured display a color gradient from blue toyellow along a central axis. This gradient indicates that generally surfaceson the left exhibit proportionally lower cortical feature values than thoseon the right

Fig. 3 Subject metadata list(right) for surface shown (left).Clicking with the mouse on anycortical surface within INVIZIANdisplays this metadata dialoguecontaining subject-specificinformation including thesubject's age, sex, weight and (byscrolling down in the windowshown) diagnostic MRIinformation. Having themaximum resolution corticalsurface rendering available allowsthe user to dynamically interactwith and precisely examine thecortical detail for any selectedsubject

Brain Imaging and Behavior

data, we describe an example of the interaction with neuroim-aging data from patients diagnosed with Alzheimer’s Disease(AD), those havingMild Cognitive Impairment (MCI), and anotherwise set of older healthy subjects. We specifically chosethis demonstration due to the well-known observation that ADhas significant alterations in brain cortical morphometry, in

particular, the atrophic effects of AD on hippocampal anatomy(Jack et al. 2008).

Subjects The neuroimaging datasets used consisted of a set of120 MRI volumetric images (MPRAGE) from theAlzheimer's Disease Neuroimaging Initiative (ADNI; http://

Fig. 4 Highlighting the subjectsmatching a search expression: Asthe subject-specific corticalsurfaces are arranged in a feature-similarity dependent manner, theimmediate graphical indication ofresults from a user-providedsearch illustrates a putativeassociation betweenneuroanatomy and the subjectmetadata. (Top) Based on queryresults the user may define namedgroups, also assigning a groupcolor, and optional description.(Bottom) INVIZIAN indicatesgroup membership with a circularglyph which allows the user toview surface characteristics andgroup membershipsimultaneously

Fig. 5 (Left) The initialized view of one coordinated projection is a two-dimensional dot plot of the feature-respective offsets, colored by group.(Middle) The coordinated projection renders a small rectangle that up-dates with zooming and panning to represent what region of the overallenvironment currently occupies the view. (Right) The user may turn off

the group indicator glyphs within the display, but determine group mem-bership by observing group colored plot. All in all, coordinated projec-tions such as this allow the user to investigate cortical regional surfacedetail and global feature relationships, simultaneously

Brain Imaging and Behavior

www.adni-info.org/) cohort maintained at the Laboratory ofNeuro Imaging (LONI) at the University of California LosAngeles. Data belonging to 40 Alzheimer's disease (AD)patients, 40 mild cognitively impaired (MCI) subjects, and40 healthy, otherwise normal control (NC) subjects wereidentified. Each group had a mean age of 76 years.

Data pre-processing All the MRI images underwent skullstripping and further processing for cortical surface extractionusing FreeSurfer (http://surfer.nmr.mgh.harvard.edu) via aLONI Pipeline (Dinov et al. 2009) data processing workflowfrom which the cortex was parcellated into 34 regions, andvolumetric feature values such as gray matter thickness, vol-ume and surface area were determined. Using these metrics asregional feature vectors, we computed pair-wise probabilisticdifference measurements - the Jensen-Shannon divergence -between each pair of concordant regions, integrated over allregion-pairs, to compute subject-by-subject “distances” (Joshiet al. 2011). The Jensen-Shannon divergence is an informationtheoretical metric of (dis)similarity which has been used inmedical image processing (Iglesias et al. 2011, Lu et al. 2012),biomarker discovery for cancer (Berretta and Moscato 2010),genetics (Chen et al. 2013), as well as text mining (Sasaharaet al. 2013). It has been used successfully in morphologicalanalysis of neuroimaging data (Guo et al. 2005). However,other measures of relative distance between subjects would bepossible to reflect different aspects of data (dis)similaritysuitable for various pattern recognition techniques (see forinstance, Kuriakose et al. 2004, Xu et al. 2006).

Once this matrix of NxN subject-by-subject distances wasobtained, we utilized classical MDS (Kruskal and Wish 1978,Chen et al. 2008)to project the data in a lower dimensionalspace and determination of new subject-specific coordinates asdiscussed above. Each subject’s brain surface was then posi-tioned in the INVIZIAN interface at these newly derived coor-dinates. Incidentally, to enable other spatial arrangements of

brain surfaces, any set of 2D or 3D sets of coordinates can beutilized for surface placement within INVIZIAN permittingplanar, scatter cloud, or in any other configuration (e.g. helical,spherical, etc.) depending on the interests of the researcher.

Visualization of diagnostic trends using cortical features

In the following subsections we illustrate several examples forvisualizing differences between AD patients and NC subjectsand making predictive inferences that might merit furtherexamination of the data under more formal analyticframeworks.

Cortical gray matter thickness Gray matter thickness wasmeasured as the mean distance from the gray matter/cerebrospinal fluid interface to the gray/white matter surface,and vice versa. Figure 6 (left) shows the color map subdividedinto quintiles used to map colors to the (0 to 5 mm) thicknessvalued features on the cortical surface. Figure 6 (middle)shows all 120 subjects colored according to thickness. Simplevisual examination indicates that there is a general trend ofcolors from more blue on the left, to more red and yellow onthe right. This describes the subjects having large thicknessdeficits on the left that gradually improve toward the right.Using the query input dialogue box, it can be quickly seen thatthat the subjects towards the left are the AD patients, while thesubjects on the right are NC subjects. Utilizing the groupingfunctionality of INVIZIAN, Fig. 6 (right) shows the subjectsgrouped according to diagnosis meta-data value, with redglyphs representing patients with an AD value, and blueglyphs representing subjects with an NC value.

Cortical gray matter volume Graymatter volume was definedas the product of the thickness and the area of the surface layermidway between the gray/CSF and white/gray matter

Fig. 6 Visualization of cortical thickness: (Left) Quintile-based colormap legend representing 0 to 5 mm of thickness. Quintile or other sub-divisioning helps the user assess the position of features in the overalldistribution of measured values. (Middle) The observed feature trend isan increase in thickness from left to right. (Right) Each brain is annotated

by a glyph according to diagnosis, red representing AD (Alzheimer'sDisease) and blue representing NC (Normal Control). The clusteredgroupings of AD and NC subjects on the left and right respectivelyindicate that cortical thickness is a strong predictor of Alzheimer’s disease

Brain Imaging and Behavior

boundaries. Following identical steps discussed above for therendering and display of cortical thickness we find a similardistribution of the subjects based on cortical volume. Figure 7(left) illustrates the color mapping of volumetric feature valueson the brain surfaces. Based upon the feature distribution andthe query and grouping tool, Fig. 7 (middle) shows the sub-jects grouped according to the categories AD (red glyphs) andNC (blue glyphs). However, from observing the group clus-tering, it can be noted that many subjects with differingdiagnosis are in overlapping clusters in contrast to the clearerseparation seen when considering the distributions of corticalthickness. This clustering hints that there is a slightly de-creased discriminatory capability of gray matter volume as afeature as compared to cortical thickness.

Cortical gray matter surface area Lastly, an average surfacearea measurement was calculated at each point on the corticalsurface mesh through a processing of averaging over the areasof those adjacent triangles including that point. Displayingthese color mapped brain surfaces illustrates a further reduced

discriminating ability as illustrated in Fig. 8. This is alsoevident observing the color-coded distribution of the area onthe individual cortical surfaces. Little variation of surface areais present locally on the individual cortical surfaces as well asglobally throughout the population.

Visualization of local structural features The diagnostictrend findings illustrated above focus on global featurepatterns in the population. To gain a better understandingof the local structural changes due to the disease, we usethe coordinated projection to zoom in on a neighborhoodof brains that share similar feature characteristics.Figure 9 (left) shows an image of the zoomed-in viewof the brain surfaces from AD patients (red dots in thecoordinated projection plot), while Figures 9 (right) is azoomed-in view of the NC subjects (blue dots in thecoordinated projection plot). From the two side-by-sideviews one can compare the detailed cortical structure andimmediately appreciate the localized thickness deficitsalong the cortex. In the case of AD patients, there is a

Fig. 7 Cortical surface area: (Left) Quintile-based color map legendrepresenting 0 to 7 mm2 surface area values. The small quintile distancein the color map indicates small area variation across the surfaces.(Middle) The area remains nearly same across the population. (Right)

Each brain is annotated by a glyph according to diagnosis, redrepresenting AD (Alzheimer's Disease) and blue representing NC (Nor-mal Control). There is little separation between groupings, indicating thatcortical area is not a predictor for Alzheimer's disease

Fig. 8 Gray matter volume: (Left) Here, the quintile-based color maplegend represents 0 to 10 mm3 gray matter volume values. (Middle) Theobserved trend in features shows an increase in volume from left to right.(Right) Each brain is annotated by a glyph according to diagnosis, red

representing AD (Alzheimer’s Disease) and blue representing NC (Nor-mal Control). The slightly overlapped groupings indicate that gray mattervolume has a reduced predictive power relative to cortical thickness forAlzheimer’s disease

Brain Imaging and Behavior

noticeable decrease in cortical thickness overall, andparticularly in the frontal, temporal, and some portionsof the parietal cortex.

Implications of results

From visualizing this collection of cortical surface data inINVIZIAN, a user can rapidly examine the properties of thedata itself to derive potentially valuable neuroscientific infer-ences. In regard to the use-case example illustrated here, welist them succinctly as follows:

& Cortical thickness is likely to be the best predictor ofAlzheimer's Disease in the observed population sample.

& Although thickness and volume are proportionally relatedin the cortex, the spatial variation of thickness appears tofollow a different distribution compared to regional vol-ume. This may be due to the fact that regional boundariesvary widely from subject to subject.

& Consequently, volume has a lower predictive effect fordisease compared to thickness.

& Surface area values do not provide a trend one way or theother with regard to AD.

& While it is understood that the human cortex is thinner inthe granular post-central gyrus region, which is apparentgraphically for both AD and NC subjects, the AD subjectsalso show region-specific atrophy particularly in the fron-tal, temporal and to some extent parietal regions.

Although the data processing was computationally the mostexpensive task (in excess of a few hours using an 800 CPUcluster grid), followed by feature display and analysis, theentire visualization process consisting of loading the processeddata to reaching the above conclusions can be performed in amatter of a few minutes. The case studies demonstrate thatINVIZIAN can be successfully used by researchers for visu-alizing database con exploratory analyses using visual tools.

Discussion

In this paper we sought to illustrate how the INVIZIANsystem can be used for exploring distributions of neuroana-tomical feature and subject metadata values across a collectionof brain surfaces drawn from databases containing patientswith age-related disease. This graphical informatics systememploys a dynamic and interactive approach for mining large-scale neuroimaging data, allowing the user to discover sug-gestive relationships between cortical surface feature andmeta-data attribute values. This approach builds on pairwisedistance calculations and decomposition of dataset fea-tures and uses the extracted brain surfaces to create avisualization environment intuitive to users of existingneuroimaging applications. Utilizing a multi-resolutionmesh display, the system provides an exploratory visualoverview of the data. The display environment anchorsquery-based grouping of clusters and feature-wise color-ing for visual exploration of neuroanatomically similargroups. A collection of coordinated projections and graphicalexploratory data analysis tools help the user keep track ofglobal relationships with the data collection while they inspectand examine detailed local cortical structure. All in all,INVIZIAN provides a uniqueway to visually inspect, explore,and assess collections of data from large-scale databases.

We wish to emphasize that INVIZIAN is not meant to com-pete with nor replace a more formal statistical analytic treatmentof the data. Though this may appear to be a limitation of thisapproach when, rather, the opposite is the case. INVIZIAN canbe used as a useful precursor to explore, mine, and perform basicmeta-analyses of neuroimaging archives as a way to generatenew hypotheses about the data that might not have been apparentthrough narrative description of their collection, the samples inquestion, or previously described effects. Image-centric data-bases for neuroscience (Bug and Nissanov 2003) will requiresuch tools to move beyond the limits of text-only searches andmore readily take advantage of image content. The presentsituation with neuroimaging archives is not too dissimilar that

Fig. 9 A comparison of subjects cortical attributes. Left , the quintile based color mapping of cortical thickness is coded in example AD patients, while,on the right, healthy control subjects are displayed using the same color scaling

Brain Imaging and Behavior

of satellite imagery and space science telescope databases whichfound particular utility when their content was re-factored from acatalogue-based system to one driven specifically by what theimages contain. Noteworthy examples of this includeMicrosoft’sWorld Wide Telescope (http://www.worldwidetelescope.org; seealso Szalay and Gray 2001) and, of course, Google Earth (http://www.google.com/earth).

As we develop the INVIZIAN further we will examinemethods for visualizing multiple brain surface features perscene. For example, it would be of benefit to color the meshdata based on combinations of feature values. We will makethe dimension reduction and offset generation system avail-able interactively during runtime. We will also generalize themethods used to calculate the cortical surface coordinates byincorporating user-selected metadata attributes. While entirelyfeasible, there is a challenge in maintaining a suitable interfacewhich is intuitive and not overly abstract to the user. Furtherevaluations will include what distance metrics lend them-selves to effective user-based visual comparison of MRI cor-tical surface data.

In conclusion, the Informatics Visualization for Neuroim-aging (INVIZIAN) framework graphically displays image andmeta-data information from sizeable collections of neuroim-aging data as a whole using a dynamic and compelling userinterface. While not a replacement for more formal statisticalanalysis, INVIZIAN provides a means for quickly assessingbrain and meta-data relationships as a precursor to further dataaccess and modeling. In our paper, we illustrate the utility ofINVIZIAN for simultaneous exploration and mining a largecollection of extracted cortical surface data arising in clinicalneuroimaging studies of patients with Alzheimer’s Disease,mild cognitive impairment, as well as healthy control subjects.Specifically, we show how cortical thickness and hippocam-pal volume differences between groups are evident usingprobabilistic distance and multivariate decomposition evenin the absence of more formal hypothesis testing. For neuro-logical diseases linked to brain aging such as AD, as well asacross the entire lifespan, INVIZIAN provides a uniquemeansfor considering the entirety of whole brain datasets, exploringinteresting relationships among them, and, in so doing, deriv-ing new ideas for further research and study.

Acknowledgements The authors wish to thank the faculty and staff ofthe Institute of Neuroimaging and Informatics of the University of South-ern California and the Laboratory of Neuro Imaging (LONI) at theUniversity of California Los Angeles. This work was supported by RC1MH088194 and R01 MH100028 (sub-award) grants to JVH.

References

Berretta, R., & Moscato, P. (2010). Cancer biomarker discovery: theentropic hallmark. PLoS ONE, 5(8), e12262.

Biswal, B. B., Mennes, M., Zuo, X. N., et al. (2010). Toward discoveryscience of human brain function. Proceedings of the NationalAcademy of Sciences of the United States of America, 107(10),4734–4739.

Bowman, I., Joshi, S. H. and Van Horn, J. (2012). "Visual Systems forInteractive Exploration and Mining of Large-Scale NeuroimagingData Archives." Frontiers in Neuroinformatics 6 .

Bug, W., & Nissanov, J. (2003). A guide to building image-centricdatabases. Neuroinformatics, 1(4), 359–377.

Carmichael, O. T., Aizenstein, H. A., Davis, S. W., et al. (2005). Atlas-based hippocampus segmentation in Alzheimer's disease and mildcognitive impairment. NeuroImage, 27(4), 979–990.

Carmichael, O., Schwarz, C., Drucker, D., et al. (2010). Longitudinalchanges in white matter disease and cognition in the first year of theAlzheimer disease neuroimaging initiative. Archives of Neurology,67(11), 1370–1378.

Chen, C.-h., Härdle, W., Unwin, A., et al. (2008). MultidimensionalScaling. Handbook of Data Visualization, Springer BerlinHeidelberg: 315–347.

Chen, R. L., Guo, W., Shi, Y., et al. (2013). Computational identificationof specific splicing regulatory elements from RNA-seq in lungcancer. European Review for Medical and PharmacologicalSciences, 17(13), 1716–1721.

Collins, D. L., Neelin, P., Peters, T. M., et al. (1994). Automatic 3Dintersubject registration of MR volumetric data in standardizedTalairach space. Journal of Computer Assisted Tomography,18(2), 192–205.

Cook, D., & Swayne, D. F. (2007). Interactive and Dynamic Graphics forData Analysis: With R and GGobi . New Yourk: Springer.

Dale, A. M., Fischl, B., & Sereno, M. I. (1999). Cortical surface-basedanalysis. I. Segmentation and surface reconstruction. NeuroImage,9(2), 179–194.

Dinov, I. D., Van Horn, J. D., Lozev, K. M., et al. (2009). "Efficient,distributed and interactive neuroimaging data analysis using theLONI pipeline." Frontiers in Neuroinformatics 3 .

Dinov, I., Van Horn, J., Lozev, K., et al. (2010). Efficient, distributed andinteractive neuroimaging data analysis using the LONI pipeline.Frontiers in Neuroinformatics, 3(22), 1–10.

Endert, A., Han, C., Maiti, D., et al. (2011). Observation-levelinteraction with statistical models for visual analytics. IEEESymposium on Visual Analytics Science and Technology -VAST. S. a. W. Miksch, M. Providence, RI, IEEE ComputerSociety.

Fischl, B., Sereno, M. I., & Dale, A. M. (1999). Cortical surface-basedanalysis. II: Inflation, flattening, and a surface-based coordinatesystem. NeuroImage, 9(2), 195–207.

Goebel, R., Esposito, F., & Formisano, E. (2006). Analysis of functionalimage analysis contest (FIAC) data with brainvoyager QX: Fromsingle-subject to cortically aligned group general linear model anal-ysis and self-organizing group independent component analysis.Human Brain Mapping, 27(5), 392–401.

Guo, H., Rangarajan, A., & Joshi, S. C. (2005). 3-D diffeomorphic shaperegistration on hippocampal data sets. Med Image Comput ComputAssist Interv Int Conf Med Image Comput Comput Assist Interv, 8(Pt2), 984–991.

Iglesias, J. E., Konukoglu, E., Montillo, A., et al. (2011). Combininggenerative and discriminative models for semantic segmentation ofCT scans via active learning. Inf Process Med Imaging, 22, 25–36.

Jack, C. R., Jr., Bernstein, M. A., Fox, N. C., et al. (2008). TheAlzheimer's Disease Neuroimaging Initiative (ADNI): MRImethods. Journal of Magnetic Resonance Imaging, 27(4), 685–691.

Johansson, S., & Johansson, J. (2009). Interactive dimensionalityreduction through user-defined combinations of quality met-rics. IEEE Transactions on Visualization and ComputerGraphics, 15 (993–1000).

Brain Imaging and Behavior

Joshi, S. H., Van Horn, J. D., & Toga, A.W. (2009). Interactive explorationof neuroanatomical meta-spaces. Front Neuroinformatics, 3 , 38.

Joshi, S. H., Bowman, I., Toga, A. W., et al. (2011). "Brain PatternAnalysis of Cortical Valued Distributions." Proc IEEE Int SympBiomed Imaging: 1117–1120.

Keim, D. A., & Kriegel, H. P. (1994). VisDB: database exploration usingmulti-dimensional visualization. IEEE Transactions on ComputerGraphics and Applications, 14(5), 40–49.

Kohonen, T. (1998). Teh self-organizingmap.Neurocomputing, 21(1–3), 1–6.Kruskal, J. B., &Wish, M. (1978).Multidimensional Scaling . New York:

Sage Publications.Kuriakose, J., Ghosh, A., Ravi Kumar, V., et al. (2004). Isometric

graphing and multidimensional scaling for reaction–diffusionmodeling on regular and fractal surfaces with spatiotemporal patternrecognition. Journal of Chemical Physics, 120(11), 5432–5443.

Lancaster, J. L., Fox, P. T., Downs, H., et al. (1999). Global spatialnormalization of human brain using convex hulls. Journal ofNuclear Medicine, 40(6), 942–955.

Lerch, J. P., Pruessner, J., Zijdenbos, A. P., et al. (2008). Automatedcortical thickness measurements from MRI can accurately separateAlzheimer's patients from normal elderly controls. Neurobiology ofAging, 29(1), 23–30.

Lu, C., Zheng, Y., Birkbeck, N., et al. (2012). Precise segmentation ofmultiple organs in CT volumes using learning-based approach andinformation theory.Med Image Comput Comput Assist Interv, 15(Pt2), 462–469.

McCormick, P. S., Inman, J. M., Ahrens, J. P., et al. (2004). Scout: AHardware-Accelerated System for Quantitatively DrivenVisualization and Analysis. Visualization '04 (VIS '04) .Washington, DC, USA: IEEE Computer Society.

Mega, M., Dinov, I., Mazziotta, J., et al. (2005). Automated brain tissueassessment in the elderly and demented population: construction andvalidation of a sub-volume probabilistic brain atlas. NeuroImage,26(4), 1009–1018.

Megalooikonomou,V., Ford, J., Shen, L., et al. (2000). Datamining in brainimaging. Statistical Methods in Medical Research, 9(4), 359–394.

Mennes, M., Biswal, B. B., Castellanos, F. X., et al. (2012). "Making datasharing work: The FCP/INDI experience." Neuroimage.

Narr, K. L., Bilder, R. M., Toga, A. W., et al. (2005). Mapping corticalthickness and gray matter concentration in first episode schizophre-nia. Cerebral Cortex, 15(6), 708–719.

Nowinski, W. L., & Belov, D. (2003). The Cerefy Neuroradiology Atlas:a Talairach-Tournoux atlas-based tool for analysis of neuroimagesavailable over the internet. NeuroImage, 20(1), 50–57.

Nowinski, W. L., & Thirunavuukarasuu, A. (2001). Atlas-assisted local-ization analysis of functional images.Medical Image Analysis, 5(3),207–220.

Rencher, A. C. (2002).Methods of Multivariate Analysis. NewYork: NY,John Wiley & Sons, Inc.

Sasahara, K., Hirata, Y., Toyoda, M., et al. (2013). Quantifying collectiveattention from tweet stream. PLoS ONE, 8(4), e61823.

Shattuck, D. W., & Leahy, R. M. (2002). BrainSuite: an automatedcortical surface identification tool. Medical Image Analysis, 6(2),129–142.

Stein, J. L., Medland, S. E., Vasquez, A. A., et al. (2012).Identification of common variants associated with humanhippocampal and intracranial volumes. Nature Genetics,44 (5), 552–561.

Szalay, A., & Gray, J. (2001). The World-Wide Telescope. Science,293(5537), 2037–2040.

Thompson, P. M., Mega, M. S., Woods, R. P., et al. (2001). Corticalchange in Alzheimer's disease detected with a disease-specificpopulation-based brain atlas. Cerebral Cortex, 11 (1), 1–16.

Van Horn, J. D., & Toga, A.W. (2009a). Brain Atlases: Their Developmentand Role in Functional Inference. In M. Filippi (Ed.), Functional MRITechniques and Protocols . New York: Humana Press.

Van Horn, J. D., & Toga, A. W. (2009b). Multisite neuroimaging trials.Current Opinion in Neurology, 22(4), 370–378.

Van Horn, J. D., Wolfe, J., Agnoli, A., et al. (2005). Neuroimagingdatabases as a resource for scientific discovery. InternationalReview of Neurobiology, 66 , 55–87.

Voytek, J. B., &Voytek, B. (2012). Automated cognome construction andsemi-automated hypothesis generation. Journal of NeuroscienceMethods, 208(1), 92–100.

Williams,M. andMunzner, T. (2004). Steerable, progressive multidimen-sional scaling. IEEE Symposium on Information Vizualization,Washington, D.C., IEEE.

Winkler, A. M., Kochunov, P., Blangero, J., et al. (2010). Cortical thicknessor grey matter volume? The importance of selecting the phenotype forimaging genetics studies. NeuroImage, 53(3), 1135–1146.

Xu, M., Thompson, P. M., & Toga, A. W. (2006). Adaptivereproducing kernel particle method for extraction of the cor-tical surface. IEEE Transactions on Medical Imaging, 25 (6),755–767.

Yang, J., Peng, W., Ward, M. O., et al. (2003). Interactive hierarchicaldimension ordering, spacing, and filtering for exploration of highdimensional datasets. Ninth Annual IEEE Conference onInformation Visualization (INFOVIZ'03). Washington, D.C., IEEEComputer Society: 105–112.

Yang, J., Ward, M. O., Rundensteiner, E. A., et al. (2003).Visual hierarchical dimension reduction for exploration ofh i gh d imens i ona l da t a s e t s . Sympos i um on Da t aVisualization, Aire-la-Ville, Switzerland, Euro-GraphicsAssociation.

Brain Imaging and Behavior