NeuroImage 127 (2016) 186–195

Contents lists available at ScienceDirect

NeuroImage

j ourna l homepage: www.e lsev ie r .com/ locate /yn img

MUSE: MUlti-atlas region Segmentation utilizing Ensembles ofregistration algorithms and parameters, and locally optimalatlas selection

Jimit Doshi a,1, Guray Erus a,⁎,1, Yangming Ou a,b, Susan M. Resnick c, Ruben C. Gur d, Raquel E. Gur d,Theodore D. Satterthwaite d, Susan Furth e, Christos Davatzikos a, for the Alzheimer's Neuroimaging Initiative 2:a Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USAb Martinos Biomedical Imaging Center, Massachusetts General Hospital, Harvard Medical School, Boston, MA, 02129, USAc Laboratory of Behavioral Neuroscience, National Institute on Aging, Baltimore, MD, USAd Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia PA, USAe Division of Nephrology, Children's Hospital of Philadelphia, 34th and Civic Center Boulevard, Philadelphia PA, USA

⁎ Corresponding author at: University of PennsylvaHamilton Walk, 7th Floor, Philadelphia, PA 19104, USA.

E-mail address: jimit.doshi@uphs.upenn.edu (G. Erus)1 Equally contributing authors.2 Part of the data used in preparation of this article were

Disease Neuroimaging Initiative (ADNI) database (adni.lotigators within the ADNI contributed to the design and improvideddata but did not participate in analysis orwritingof ADNI investigators can be found at http://adni.loni.usc.to_apply/ADNI_Acknowledgement_List.pdf.

http://dx.doi.org/10.1016/j.neuroimage.2015.11.0731053-8119/© 2015 Elsevier Inc. All rights reserved.

a b s t r a c t

a r t i c l e i n f o

Article history:Received 1 August 2015Accepted 30 November 2015Available online 8 December 2015

Atlas-based automated anatomical labeling is a fundamental tool inmedical image segmentation, as it defines re-gions of interest for subsequent analysis of structural and functional image data. The extensive investigation ofmulti-atlaswarping and fusion techniques over the past 5 ormore years has clearly demonstrated the advantagesof consensus-based segmentation. However, the common approach is to usemultiple atlases with a single regis-tration method and parameter set, which is not necessarily optimal for every individual scan, anatomical region,and problem/data-type. Different registration criteria and parameter sets yield different solutions, each providingcomplementary information. Herein, we present a consensus labeling framework that generates a broad ensem-ble of labeled atlases in target image space via the use of several warping algorithms, regularization parameters,and atlases. The label fusion integrates two complementary sources of information: a local similarity ranking toselect locally optimal atlases and a boundary modulation term to refine the segmentation consistently with thetarget image's intensity profile. The ensemble approach consistently outperforms segmentations using individualwarping methods alone, achieving high accuracy on several benchmark datasets. The MUSE methodology hasbeen used for processing thousands of scans from various datasets, producing robust and consistent results.MUSE is publicly available both as a downloadable software package, and as an application that can be run onthe CBICA Image Processing Portal (https://ipp.cbica.upenn.edu), a web based platform for remote processingof medical images.

© 2015 Elsevier Inc. All rights reserved.

Keywords:MRIMulti-atlas segmentationRegistrationLabel fusionROI

Introduction

Automated segmentation of anatomical structures, i.e. delineation ofregions of interest (ROIs), onMR images is an extremely important taskfor quantitative analysis of structural and functional brain changes, par-ticularly in studies with large datasets (Good et al., 2002; Poldrack,2007). The rapidly increasing amount of imaging data creates an urgent

nia, Richards Building, 3700

.

obtained from the Alzheimer'sni.ucla.edu). As such, the inves-plementation of ADNI and/or

of this report. A complete listingedu/wp-content/uploads/how_

need for accurate and consistent phenotyping of brain structures in tensof thousands of images acquired from multiple institutions, and of sub-jects in various age groups (Medland et al., 2014; Hibar et al., 2015).During thepast 5 years,multi-atlas segmentation (MAS)has increasing-ly gained attention as a potential solution to this problem (Iglesias andSabuncu, 2015). Themain principle of MAS is to use a priori knowledge,provided by ensembles of segmented atlases, i.e. images with manuallyor semi-automatically created reference segmentation labels, to infersegmentation in a target image via multiple atlas-to-target image regis-trations. After being warped individually to the target image, multipleatlases provide various representations of the anatomy and correcteach other's errors in a process known as label fusion. MAS has shownremarkable improvement over single-atlas-based segmentation, andhas now been considered as the standard framework for segmentationof biomedical images. A multitude of algorithms have been proposedin recent years to improve various facets of the MAS framework, withparticular emphasis on atlas selection and robust and accurate fusion

187J. Doshi et al. / NeuroImage 127 (2016) 186–195

of the warped atlas labels (Aljabar et al., 2009; Lötjönen et al., 2010;Sabuncu et al., 2010; Landman et al., 2011; Leung et al., 2011; Asmanand Landman, 2013; Cardoso et al., 2013; Zikic et al., 2014; Wu et al.,2015).

With the exception of patch-based approaches that use affine regis-tration to align atlas images (Coupé et al., 2011; Konukoglu et al., 2013),deformable image registration (Sotiras et al., 2013) is a core componentof allMASmethods, and the quality of individual registrations has a veryhigh impact on the accuracy of the final segmentation. However,anatomical correspondence may not be uniquely determined fromintensity-based image attributes, which drive deformable registrationalgorithms. Furthermore, exact anatomical correspondence may notexist at all due to anatomical variability across subjects. Anatomies clos-er to an atlas are well represented by a diffeomorphism. However, largedifferences between an individual and the atlas lead to residual infor-mation that the transformation does not capture. Techniques such asatlas selection from a larger atlas dataset (Aljabar et al., 2009; Wuet al., 2007; Gousias et al., 2010; Hoang Duc et al., 2013; Sanromaet al., 2014), or local similarity based weighting in label fusion(Artaechevarria et al., 2009; Isgum et al., 2009; Khan et al., 2011;Wang et al., 2013a) were proposed to address these challenges, byselecting, either locally or globally, warps most similar to the targetimage. However, not only the choice of the atlas, but also the warpingalgorithm and the parameters of the algorithm, particularly regulariza-tion, play an important role in the accuracy of the registration. Compar-ative evaluations on multiple datasets have shown that registrationalgorithms differed greatly in performance, when facing diversedatabases or challenges, globally as well as in individual regions of thebrain (Ou et al., 2014). Fig. 1 shows an illustrative example of suchdifferences in registration accuracy for two registration algorithmsthat have reported high accuracies.

Fig. 1. Illustration of differences resulting from the application of two different registration algoobtained using ANTS. The red circles point to areas where one of the methods is locally more a

The importance of variability in algorithm performance is oftenneglected in current MAS methods, where the general practice is to se-lect a single warping algorithmand to use it with a single set of registra-tion parameters. There have been few methods that specificallyinvestigated the effect of registration algorithms and parameters onmulti-atlas segmentation. In Bai et al. (2012), the authors investigatedthe roles of image registration and segmentation model complexityfor mouse brain segmentation using 4 different registration algorithms,and concluded that image registration plays a more crucial role insegmentation compared to the complexity of the segmentation model.Interestingly, in early days of atlas based segmentation, work reportedin Rohlfing andMaurer (2005) investigated the effects of various atlasesand parameterizations of the registration algorithm, casting thesegmentation problem as a “multi-classifier” framework. This analysiswas limited to a comparison between single atlas with 3 parametersand 3 atlases with a single parameter, using a free-form deformationalgorithm with simple label fusion, and was validated on 7 subjectsonly. Despite this limitation, the authors observed that in all casesclassifier combinations consistently improved classification accuracy,and that improvements in accuracy were possible with various param-eterizations of the non-rigid registration technique, even using a singleatlas.

In this paper, we propose a new method, MUlti-atlas regionSegmentation utilizing Ensembles (MUSE), a generalization of theMAS framework to include a broad representation of a given anatomythat reflects variations due to the choice of the atlas, as well as thewarping method and warping parameters. In this way, we obtain alarge ensemble of tentative label maps that are generated by applyinga multitude of transformations on multiple atlases, and we use the en-semble for deriving final labels for each voxel. The general concept ofgenerating a larger ensemble of labelmapswas explored in a few recent

rithms; a) The target image, b) warped image obtained using DRAMMS, c) warped imageccurate than the other method (DRAMMS on the top, and ANTS in the bottom).

188 J. Doshi et al. / NeuroImage 127 (2016) 186–195

papers: inWang et al. (2013b)multiplewarps from the same atlasweregenerated by composing inter-atlas registrations and atlas-target regis-trations; in Pipitone et al. (2014) segmentations from a small number ofatlaseswere propagated to a subset of target images and the newatlaseswere used for segmenting all target images. However, these methodsused a different approach than ours, by following an “atlas propagation”strategy.

MUSE utilizes a spatially adaptive strategy for the label fusion. A localsimilarity ranking score is calculated and used for selecting warpedatlases that are locally most similar to the target image. For similaritycalculation, we define a rich attribute descriptor as in Ou et al. (2011)that renders each voxel more distinctive than intensity informationalone. The local similarity ranking is particularly essential for theensemble approach: the ensemble construction covers a range ofdeformation parameter values (as well as atlases and algorithms). Thismay result in a high variation within the ensemble, which is desir-able to be able to better capture the target anatomy. However thisalso necessitates a reliable selection of best warps in order to guar-antee that suboptimal or failed registrations don't affect the deci-sion in final label assignment. The label fusion also incorporates anintensity term that modulates the segmentations in the boundariesof the ROIs. The main purpose of the intensity term is to make thefinal segmentation consistent with the intensity profile of the targetimage.

In the current paper, we validated our method using several publicbenchmark datasets with expert defined reference labels, and we con-firmed that the ensemble approach consistently outperforms segmen-tations obtained using individual warping methods/parameters alone.Also, in an independent comparative evaluation done as part of the“MICCAI 2013 Challenge Workshop on Segmentation” MUSE obtainedthe highest average Dice score (d = 0.8686) in the mid-brain segmen-tation category, and it maintains the first rank as of 11/02/2015.3

As an attempt towards the ambitious goal of quantitative anatomicalphenotyping of the human brain using big data, we applied MUSE onthousands of images from several large-scale neuro-imaging studies,and showed the robustness of our method and consistency ofsegmentations for datasets with significant differences in scannercharacteristics and sample demographics, by accurately estimatingbrain age from segmented ROIs. Finally, we performed experimentsthat investigated the contribution of various ensemble combinations,and individual components of our method, to the final segmentationaccuracy.

Our method is publicly available and can be downloaded from ourweb page.4 Alternatively the MUSE software can be run remotely onthe CBICA Image Processing Portal,5 a new web platform that allowsusers to upload their data and run software developed in our lab.The web client version of MUSE will also include a new multi-studyatlas dataset with a very large sample size, and a wide range of ageand scanner characteristics. This atlas dataset was constructed by auto-matically selecting a subset of the most representative subjects fromseveral datasets that include scans of healthy individuals. The ROI labelsfor atlas images were automatically created using MUSE and werecarefully controlled for quality using automated andmanual verificationprocedures. We believe that the software package and the new atlasdataset with large sample size will be valuable resources for thecommunity.

Methods

MUSE generates a large ensemble of candidate labels in the targetimage space using multiple atlases, registration algorithms and

3 MICCAI 2013 SATA Challenge Leaderboard, retrieved 11/02/2015 from URL http://masi.vuse.vanderbilt.edu/submission/leaderboard.html.

4 http://www.cbica.upenn.edu/sbia/software.5 The beta release is accessible from http://ipp.cbica.upenn.edu.

smoothness values for these algorithms. The ensemble is then fusedinto a final segmentation. An illustration of the MUSE algorithm isgiven in Fig. 2. Individual components of MUSE are explained in detailin the following subsections.

Construction of the ensemble of warps

Given the target image S to be segmented, and n atlas images withthe corresponding reference label maps each having l ROI labels, includ-ing the background, the aim of our method is to segment S by assigningan ROI label to each voxel in the image. Inspired by the manifold repre-sentation that was introduced in Baloch and Davatzikos (2008), we de-fine the anatomic equivalence class of S as a set of all possible ways ofrepresenting the morphology of that individual via a transformation ofan atlas and a respective residual, obtained by varying transformationparameters, i.e.

Q ¼ Qhθ xð Þ� �

: T hθ xð Þð Þ ¼ S xð Þ−Rhθ xð Þ;∀x ∈ ΩS� �

;∀θ ∈ Θ� �

ð1Þ

where h :ΩS→ΩT, x→ h(x) is a transformation that maps the subjectspace ΩS to atlas space ΩT, Rhθ is the residual of the transformation,and θ is the parameter vector, which herein combines three impor-tant parameters for variations of h : hτ,μ,λ, the atlas, the deformationmethod, and the amount of regularization. Varying Θ = {θ1, …, θk}effectively allows every individual representation to slide along itsown manifold, thereby leading to multiple ways of representing

each individual as an ensembleQ ¼ fQhθigki¼1

. By applying each trans-

formation hθi ; i ¼ f1…kg on the corresponding atlas image and labelmap we obtain an ensemble of k atlas images and label maps regis-tered to subject space, which we denote here as T S ¼ fTS

1;…; TSkg

and LS ¼ fLS1;…; LSkg.

Spatially varying similarity weighting

We calculate a local similarity score at each voxel of S against eachwarp Ti

S, such that a higher score is given to warps that are locallymore similar to the target image. For the local similarity calculation,we define a rich attribute descriptor as in Ou et al. (2011). A voxel x isdescribed by a d-dimensional attribute vector A(x), which encodes thegeometric context of this voxel. For computing A(x) an image isconvolvedwith a set of Gabor filter banks, which capture the texture in-formation at multiple scales and orientation, and the responses of thesefilters at voxel x are concatenated into a vector. The local similaritybetween two voxels x and y is then defined as

sim x; yð Þ ¼ 1

1þ 1d

A xð Þ−A yð Þk k2∈ 0;1½ � ð2Þ

After the similarity sim(S(x), TiS(x)) between the target image andeach warp at each voxel x ∈ ΩS is calculated, local similarity scores areranked to assign a rank s(x, i), with a higher rank representing highermorphological similarity between S and the warp Ti

S at voxel x. The cal-culated spatially varying rank score s(x, i) is assigned to the selectedtransformations as a weight for subsequent label fusion.

Boundary modulation

The intensity based boundary modulation term indicates the proba-bility of the observed intensity at a voxel x to belong to the tissue inten-sity distribution of the specific ROI. Intuitively, themain objective of theboundarymodulation term is to refine the ROI boundaries by penalizinglarge variations in the intensity distributionwithin anROI. It's importantto note that in contrast to methods where the intensity model is esti-mated from the atlas images for which the segmentation labels are

Fig. 2. A schematic illustration of the MUSE algorithm.

189J. Doshi et al. / NeuroImage 127 (2016) 186–195

known, we estimate the intensity distribution of ROIs from the targetimage, similar to Wolz et al. (2009). Such estimation is more robust toglobal and local intensity variations between the atlases and the targetimage. The intensity distribution of each ROI ismodeled as a normal dis-tribution with the assumption that the ROIs belong to a single tissuetype with a smooth intensity variation. As the ROI segmentation onthe target image is not known, the parameters of the distribution are es-timated from a consensus segmentation, using intensities of voxels forwhich 90% of the warps agree on the segmentation label. An ROImembership score b1(x, p) is calculated for each voxel x ∈ ΩS and eachROI p ∈ {1,…, l}.

In addition to the ROImembership function, we also calculate a seg-mentation based term in order to achieve more accurate delineation ofthe brain boundary.We assign to each ROI in the reference dataset a tis-sue type categorizing it as “brain”, if the ROI is on the gray matter (GM)or the white matter (WM), or “non-brain”, if the ROI is on the cerebro-spinal fluid (CSF) or the background. The new term, b2(x, p), quantifiesthe agreement between the tissue type observed at voxel x and theexpected tissue type of ROI p. Tissue probabilities of target image voxelsare computed for the three tissue types GM, WM and CSF using a fuzzysegmentation, and are converted into tissue probability maps for“brain” and “non-brain”. These two maps are then used to set thevalue of b2(x, p), by assigning to it the value at voxel x from the proba-bility map that corresponds to the tissue type of ROI p.

Weighted label voting

The label fusion incorporates the local similarity ranking and theintensity based boundary modulation term. These two terms areeffectively representing two complementary sources of information:

1. An ensemble constructed by atlas transformations, which is used totransfer segmentation labels from atlases to the target space;

2. The intensity information from the target image, which modulatesthe segmentation, particularly at the ROI boundaries.

The weighted vote of voxel x ∈ ΩS for being labeled as ROI p iscalculated as:

w label xð Þ ¼ pð Þ ¼

Xk

j¼1s x; jð Þ � δ LSj xð Þ ¼ p

� �� �

Xk

j¼1j

þ α1 � b1 x; pð Þ þ α2 � b2 x; pð Þ ð3Þ

where k is the number of selected warps, δ(⋅) is an indicator functionused for selecting warps with the ROI label p at voxel x, and α1 and α2

are coefficients that modulate the effect of each term to the final fusion.Note that the rank score is normalized by the sum of all ranks to obtain avalue bounded between 0 and 1. A voxel is assigned to the most likelylabel p*, i.e.:

label xð Þ ¼ p� s:t: p� ¼ arg maxp

w label xð Þ ¼ pð Þ ð4Þ

Experiments

In this section, we present experimental results that were obtainedby applying our method on a large number of datasets.

The first set of experiments aimed to evaluate the contribution ofvarious ensemble combinations and individual components of ourmethod to the final segmentation. A second set of experiments wereperformed to validate segmentation performance in comparison toother multi-atlas label fusion methods. We also present the results ofan independent comparative evaluation that was done as part of theMICCAI 2013 segmentation challenge. The validation experimentswere performed on various publicly available datasets with referencelabels for diverse brain regions. A single fold cross-validation was

aracteristicsof

valid

ationda

tasets.

Dataset

name

#of

subjects

#of

timep

oints

Imag

edimen

sion

sVox

eldimen

sion

s#of

ROIs

Age

rang

e(m

ean)

#of

males

Scan

ner

Scan

protoc

olTR

TEFA

ithreferenc

eBrainW

eb11

125

6×25

6×18

11.0×1.0×1.0

324

–37(3

0)n/a

Siem

ensSo

nata

(1.5

T)SP

GR

22ms

9.2ms

30°

IBSR

181

256×12

8×25

60.84

×1.5×0.84

(n=

4)0.94

×1.5×

0.94

(n=

8)1.0×1.5×1.0(n

=6)

327–

71(3

8)*

14GESign

a(1

.5T)

SPGR

40ms

5ms

40°

NIREP

161

256×30

0×25

61.0×1.0×1.0

3324

–48(3

1)n/a

GESign

a(1

.5T)

SPGR

24ms

7ms

50°

OASIS

351

256×(2

61–3

34)×25

61.0×1.0×1.0

1415

–96(5

3)n/a

Siem

ensVision(1

.5T)

MP-RA

GE

9.7ms

4ms

10°

itho

utelabe

lsBL

SA46

81

256×25

6×12

4(n

=92

)17

0×25

6×25

6(n

=37

6)0.94

×0.94

×1.5(n

=92

)1.2×

1.2×1.0(n

=37

6)29

–94(6

7)22

0GESign

a(1

.5T),

PHILIPS(3

T)SP

GR,

MP-RA

GE

Varies

bysite

PNC

201

119

2×25

6×16

00.94

×0.94

×1.0

8–22

(15)

101

Siem

ensTIM

Trio

(3T)

MP-RA

GE

1.81

s3.5ms

9°BB

L-NC

791

256×25

6×19

20.94

×0.94

×1.0

18–4

9(2

8)41

NiCK

661

256×25

6×16

00.98

×0.98

×1.0

9–25

(16)

35Siem

ensVerio

(3T)

MP-RA

GE

1.79

s3.06

ms

10°

ADNI-1

215

1(1

60–1

84)×

(192

–256

)×

(192

–256

)(1

.18–

1.21

)×

(0.91–

1.35

)×

(0.93–

1.36

)60

–90(7

6)10

9Variesacross

57sites

MP-RA

GE

Varies

bysite

190 J. Doshi et al. / NeuroImage 127 (2016) 186–195

applied for each dataset to segment ROIs in each image. This was pre-ferred in order not to over-tweak the parameters for a specific set of ref-erence atlas dataset and thus to keep ourmethodmore generalizable tounseen datasets. Besides, differently from learning-based approachesthat rely onmulti-fold cross-validation for the construction of the train-ing model, our method does not require any training on reference atlaslabels. The segmentations were applied independently on each dataset,as the reference label definitions were not consistent between variousatlas datasets. The Dice similarity coefficient, or Dice Score, a standardmetric that is widely used for measuring the degree of overlap betweenthe target and the reference segmentations, was calculated for quantita-tive evaluation. The global Dice score for a subject was calculated as theaverage of the Dice scores for all individual ROIs for this subject.

A third set of experiments investigated the segmentation perfor-mance of our method on multi-site data. Imaging variability due to dif-ferences in scanner manufacturers, scan protocols and parameters is amajor challenge for cross-study analysis of MRI data. The robustnessof any segmentation method is of critical importance for addressingthe challenges of cross-study analyses using “big data”, which is a direc-tion of research that has been recently necessitated by the explosivegrowth of neuroimaging data. Since common reference labels for multi-ple datasets were not available, a direct quantitative evaluation of thesegmentation accuracy in multi-site settings was not possible. Thus,we evaluated a higher-level outcome obtained by using the segmentedROIs as features in a support vector regression, in order to estimate“brain age”. In recent years the concept of usingmachine learning to de-termine brain age has gained popularity as ameans for defining norma-tive trajectories of brain development and aging (Dosenbach et al.,2010; Franke et al., 2012; Erus et al., 2014). A good brain age index offershigh specificity, thereby enabling the detection of subtle deviationsfrom normative trajectories much better. We created a large dataset ofclinically normal subjects (n = 1029, age range 8 to 94) by poolingdata from several studies. We segmented ROIs and we calculated thebrain agewith cross validation using ROI volumes as input to an ensem-ble learner. We measured the cross-validated brain age predictionaccuracy, as an indicator of robustness and precision of the derivedbrain age index.

Data description

The internal validation experiments were performed on four bench-mark datasets, BrainWeb (Aubert-Broche et al., 2006), IBSR,6 NIREP(Christensen et al., 2006) and OASIS (Marcus et al., 2010), for which ex-pert defined reference ROI labelswere publicly available. The evaluationin the MICCAI 2013 segmentation challenge was done using the OASISdataset. An illustration of the reference labels in each dataset is providedin supplementary Fig. 1.

In order to evaluate themulti-site segmentation performance of ourmethod we created a large multi-study dataset of clinically normal sub-jects by pooling data from studies including BLSA (Resnick et al., 2000,2003), ADNI-1 (Jack et al., 2008), PNC (Satterthwaite et al., 2014),BBL-NC,7 and NiCK (Hartung et al., 2015). The general characteristicsof each of these datasets are summarized in Table 1.

Choice of registration methods and parameters

We used two relatively recent and extensively validated deformableregistration methods, DRAMMS (Ou et al., 2011, 2014) and ANTS(Avants et al., 2008; Klein et al., 2009), for transferring atlas labels to tar-get space. For both methods, the main parameter that regulates thesmoothness of the deformation field was sampled at two operationalpoints, specifically g = {0.1, 0.2} for DRAMMS and s = {0.25, 0.5} for

6 National Institute of Health supported Internet Brain Segmentation Repository (IBSR),http://www.cma.mgh.harvard.edu/ibsr.

7 Brain Behavior Laboratory, UPENN, http://www.med.upenn.edu/bbl/bbl.shtml. Table1

Summarych

Datasetsw

labe

ls

Datasetsw

referenc

Table 2Dice scores obtained for the labeling of images in 4 public data sets using various combinations of warps and label fusion strategies.

DRAMMS ANTS DRAMMS ANTS DRAMMS + ANTS

0.1 0.2 0.25 0.5 0.1 + 0.2 0.5 + 0.25 All parameters

A. MV BrainWeb 86.38 ± 0.59 85.8 ± 0.63 84.23 ± 1.21 85.75 ± 1.15 86.35 ± 0.61 85.18 ± 1.18 86.91 ± 0.84IBSR 82.45 ± 1.31 82.34 ± 1.3 82.44 ± 1.44 82.95 ± 1.44 82.51 ± 1.32 82.79 ± 1.46 83.23 ± 1.36NIREP 79.09 ± 1.67 78.62 ± 1.69 77.77 ± 1.74 78.7 ± 01.72 79.18 ± 1.7 78.42 ± 1.73 79.73 ± 1.7OASIS 86.98 ± 2.63 86.99 ± 2.58 86.85 ± 2.6 87.05 ± 2.46 87.06 ± 2.61 87.01 ± 2.51 87.53 ± 2.34

B. SimRank BrainWeb 88.06 ± 0.58 87.65 ± 0.61 86.94 ± 1.14 88.16 ± 1.08 88.24 ± 0.58 87.94 ± 1.1 89.26 ± 0.76IBSR 83.21 ± 1.3 83.15 ± 1.23 83.38 ± 1.31 83.79 ± 1.36 83.35 ± 1.28 83.76 ± 1.31 84.14 ± 1.3NIREP 80.03 ± 1.74 79.75 ± 1.78 79.36 ± 1.79 79.99 ± 1.79 80.23 ± 1.79 79.91 ± 1.8 80.95 ± 1.79OASIS 87.43 ± 2.47 87.44 ± 2.36 87.46 ± 2.29 87.57 ± 2.18 87.53 ± 2.41 87.62 ± 2.21 88.07 ± 2.06

C. SimRank + BM BrainWeb 90.54 ± 0.57 90.27 ± 0.59 89.73 ± 0.97 90.46 ± 0.91 90.78 ± 0.59 90.41 ± 0.92 91.61 ± 0.7IBSR 84.15 ± 1.3 84.14 ± 1.21 84.27 ± 1.38 84.65 ± 1.35 84.27 ± 1.25 84.63 ± 1.35 84.96 ± 1.3NIREP 79.91 ± 1.69 79.61 ± 1.78 78.59 ± 1.9 79.24 ± 1.83 79.97 ± 1.77 79.04 ± 1.88 79.98 ± 1.88OASIS 87.68 ± 2.2 87.72 ± 2.11 87.99 ± 1.93 88.12 ± 1.77 87.76 ± 2.14 88.13 ± 1.82 88.27 ± 1.81

191J. Doshi et al. / NeuroImage 127 (2016) 186–195

ANTS. The parameters g=0.2 for DRAMMS and s=0.5 for ANTs are thedefault weighting parameters in these registration algorithms. Severalindependent studies have reported that these default parameters gen-erated reasonable to very accurate registration results for multipledatasets. We further included g = 0.1 for DRAMMS and s = 0.25 forANTs, which means smaller weights for smoothness. This results in amore aggressive registration, trading the deformation smoothnesswith higher registration-based voxel-/region-wise matching, whichmay be needed especially when the atlas and target subjects bear largerinter-subject variations (e.g., different cortical folding patterns may re-quire less smooth deformation to match across patients). In compara-tive survey papers (Ou et al., 2014; Klein et al., 2009) and in manyother studies, it was reported that more aggressive deformations mayresult in higher atlas-to-target regional overlaps, which is not ideal forregistration but preferable for atlas-based segmentation. In our experi-ments we used these two parameter values to explore the potentialcomplementary information provided by the default and more aggres-sive values for smoothness of the deformation field. However ourmeth-od is generic and it can be run with other values of these and otherparameters, as well as with other registration algorithms.

Results on public datasets

MUSE was applied on 4 public datasets for which reference labelmasks for diverse sets of ROIs were provided. All experimentswere per-formed using leave-one-out cross validation, applying it independentlyfor each dataset. In each fold, for segmenting the left-out subject all

Fig. 3. Box plots of average Dice scores for each benchmark dataset obtained using majoritymodulation. In all experiments DRAMMS and ANTS registrations of 7 atlases for two set of pabetween datasets is not comparable, since each dataset has a different reference ROI definition

remaining images were used as the atlas pool, from which a subset ofatlases were selected. In these experiments, we appliedMUSEwith var-ious combinations of warps as input to the label fusion, i.e. by varyingthe number of atlases, the registration algorithms and registrationparameters used in the registration step, in order to analyze thecontribution of the ensemble to the labeling performance.

Comparison of various combinations of warpsTable 2.A below shows the contribution of combining various regis-

trationmethods and various regularization parameters in segmentationperformance. In all experiments, the number of atlases was set to 7. Theselection of the atlases that were used in the segmentation was done byranking the atlas pool based on global similarity to the target image afterlinearly aligning all atlases to an average atlas. In order to emphasize theeffect of registration algorithms and parameters, we used simplemajor-ity voting in label fusion,where each voxelwas assigned to the ROIwiththe highest number of votes from all warped label maps. The first fourcolumns show the average Dice scores for single method/parameter,while the last column includes the ensemble of all warps in the fusion.We observe that, for each dataset, the complete ensemble of warps con-sistently outperforms ensembles of warps from a single registrationalgorithm, with significant differences (p b 0.01 with a paired t-test).

MUSE with similarity rankingTable 2.B shows the results when the label fusion was donewith the

adaptive weighting using the local similarity ranking term. The additionof the similarity based weighting term significantly increases the

voting, MUSE with similarity ranking and MUSE with similarity ranking and boundaryrameters were used as input to label fusion. Please note that the segmentation accuracy.

Fig. 4.Anexample from the BrainWebdataset highlighting the effects and the improvements resulting from individual components of themethod. Thefigure shows the original image (a),and theGMprobabilitymaps resulting from:majority voting of DRAMMSwarps (b),majority voting of ANTSwarps (c), majority voting of DRAMMS+ANTSwarps (d) and thefinal resultemploying thewarp ensemble, similarity ranking and boundarymodulation (e). Themarked circle indicates the area of improvement as a result of the combination of warps aswell as theproposed label fusion method.

192 J. Doshi et al. / NeuroImage 127 (2016) 186–195

performance compared to simple majority voting (p b 0.01 for alldatasets). Differences between using the complete ensemble of warpsversus any set of warps from a single registration algorithm are signifi-cant as well.

MUSE with similarity ranking and boundary modulationTable 2.C shows the results obtained using the complete MUSE

method. The additional boundary modulation term results in highersegmentation accuracy for all datasets, except NIREP. The differenceswere significant, except for OASIS (p = 0.06185). Box plots of averageDice scores for each dataset using the complete ensemble of warpsand the three different label fusion strategies are shown in Fig. 3. An ex-ample segmentation on one of the BrainWeb subjects that highlightsthe improvements as a result of ensemble construction and theproposed label fusion strategy are shown in Fig. 4.

The effect of number of atlases on the segmentationIdeally, it would be preferable to run any multi atlas label fusion

method with the maximum number of available atlases. However, aseach atlas should be non-linearly warped to the target space, this maynot be feasible or preferable in terms of available computational re-sources. Thus it's important to select the appropriate atlas set that willgive accurate results with the least number of atlases. For evaluatingthe effect of number of atlases on the segmentation performance, we

Fig. 5. Dice scores obtained for label fusion using varying number of atlases.

applied MUSE on all 4 datasets with varying number of atlases withina wide range. We observe that the segmentation accuracy consistentlyincreases with more atlases until it reaches a stable value around 7atlases (Fig. 5).

Segmentation of midbrain structures in the OASIS dataset

As part of the MICCAI 2013 segmentation challenge, a dataset withtraining and testing images from OASIS project with reference labelmaps was created for evaluating multi-atlas label fusion algorithms insegmentation of mid-brain structures. The reference labels for thefinal test set were kept undisclosed and the segmentation accuracy ofparticipating methods was calculated by the organizers. Table 3 belowpresents a summary of the challenge results. The challenge attracted awide range of methods, including MUSE (named in the challenge asUPENN-SBIA-MAM), the joint label fusion approach described in(Wang et al., 2013a) (PICSL), a levelset-based label fusion and cor-rection method (SBIA-LevelSet), a new label fusion method thatuses a modality independent neighborhood descriptor (Heinrichet al., 2012) (deedsMIND), and a label propagation method usingrandom forests (Zikic et al., 2014) (MSRC). Consistently with ourcross-validated segmentation accuracy on the training dataset, MUSEperformed with high accuracy on the testing dataset and obtained thehighest average Dice score. However, the scores of the three methodswith the highest ranks were similar and comparable.

Table 3Segmentation of mid-brain structures.

Average Dicecoefficient

Hausdorff distance

Team name Mean Median Mean Median

UPENN-SBIA-MAM 0.8686 0.8772 3.3043 3.1006PICSL 0.8663 0.8786 3.5381 3.2369SBIA-LevelSet 0.8654 0.8775 3.4237 3.1429deedsMIND 0.8402 0.8573 4.1027 3.8983PICSL-SV-MRF 0.8343 0.8421 4.1749 3.9972MSRC-AtlasForest-Stage2 0.8282 0.8484 3.7411 3.5231DL-UB 0.8243 0.8346 5.0941 4.5566PICSL-SV-MRF-baseline_3 0.8186 0.822 5.3519 5.1216MSRC-AF-NEW-STAPLE 0.8063 0.8169 4.6494 4.376deedsMIND-no-marginals 0.7216 0.7539 6.1614 5.512

Retrieved from http://masi.vuse.vanderbilt.edu/submission/leaderboard.html on 11/02/2015. Unidentified or undocumented submissions, and repeat submissions are notshown.For detailed method descriptions of listed submissions please see the challengeproceedings at https://masi.vuse.vanderbilt.edu/workshop2013/images/1/1b/SATA-2013-Proceedings.pdf

Table 4Brain age prediction from ROI volumes computed using three alternative label fusionmethods. The brain age is calculated with cross-validation using across study data.Pearson's correlation coefficient (r), concordance correlation coefficient (ccc) and meanabsolute error (MAE) were calculated as quantitative metrics of prediction accuracy.

STAPLE JLF MUSE

r ccc mae r ccc mae r ccc mae

ADNI-1 0.34 0.21 7.37 0.45 0.31 6.13 0.47 0.40 4.91PNC 0.40 0.21 6.37 0.51 0.28 5.86 0.59 0.47 3.47BBL-NC 0.46 0.30 9.32 0.60 0.43 8.49 0.70 0.62 5.99BLSA 0.79 0.77 6.85 0.81 0.79 6.79 0.80 0.78 6.59NiCK 0.40 0.20 6.82 0.46 0.22 7.16 0.53 0.34 5.11All studies 0.94 0.93 7.05 0.95 0.94 6.62 0.96 0.96 5.49

193J. Doshi et al. / NeuroImage 127 (2016) 186–195

Application of MUSE on a multi-site dataset of healthy control subjects

For each subject of the pooled multi-site dataset ROI segmentationwas performed by independently applying three different methods,MUSE, STAPLE (Warfield et al., 2004) and Joint Label Fusion (Wanget al., 2013a). In each experiment we used 11 atlases selected fromthe set of 35 OASIS atlases with reference labels, and we used the com-plete ensemble of warps computed by applying DRAMMS and ANTSwith the two smoothness values for each algorithm. Scatter plots ofROI volumes of all subjects for lateral ventricles, hippocampus, posteriorcingulate gyri and superior frontal gyri, as well as for total GM andWMvolumes, calculated usingMUSE and the two other label fusionmethodsare shown in supplementary Figs. 2 and 3.

We used ROI volumes as input to a supervised learning frameworkfor the prediction of brain age. We trained an ensemble of regressorson the ROI volumes using the complete dataset (n = 1029) withleave-one-out cross-validation, and predicted the age of each subjectusing the trained models. We calculated Pearson's correlation coeffi-cient (r), concordance correlation coefficient (ccc) (Lin, 1989) andmean absolute error (MAE) as quantitativemetrics of prediction accura-cy. The quantitative evaluation results for the three methods are givenin Table 4. MUSE obtained the highest accuracy in terms of all threemetrics. A plot of actual ages and predicted brain ages of all subjectsare shown in Fig. 6.

Discussion

We presented a new method for ensemble-based brain parcelation.The main contribution of the proposed framework is that it representseach anatomy with a rich ensemble of warps that incorporates choiceof the atlas, deformation algorithm and deformation parameters.

Fig. 6. Scatter plot of actual and predicted ages for the multi-study data of normal controls. Timaging features obtained by applying three different label fusion methods.

Different registration methods, which generally use notably differentimage features, energy formulations and optimization algorithms, pro-vide complementary information about the anatomy. Each methodand parameter set can be relatively less or more accurate in certainareas of the brain, thereby rendering an ensemble-based segmentationadvantageous. Moreover, our approach is effectively a patient- andregionally-specific application of MAS, as for each individual and eachbrain region the most suitable set of labeling estimates was used inthe ensemble approach.

We demonstrated that the ensemble of a multitude of warps,particularly using appropriate techniques for fusing them together,has significantly improved the segmentation accuracy, and provided ro-bust segmentations. The fact that ensemble fusion consistently outper-forms segmentations using single registration algorithm/parametercombinations is particularly important. Selection of optimal algorithm/parameter value for MAS is an open question, as we don't have a prioricorrect value for it. As stated in Rohlfing andMaurer (2005), the ensem-ble approach efficiently solves this problem by covering a range of pos-sible values without having to pick one, and in this way provides arobust segmentation tool. This approach is more and more required inthe big data era where multi-site data with diverse scanner characteris-tics and subject demographics are increasingly used.

MUSE achieved consistently highDice scores for the segmentation ofimportant deep brain structures, such as hippocampus, thalamus andcaudate, which have been previously shown to be associated withvarious neurodegenerative diseases (Laakso et al., 1996; Konick andFriedman, 2001; Levitt et al., 2002), and for which accurate segmenta-tion is very important for the quantification of disease related changes.Importantly, MUSE achieved an accuracy comparable to a recent learn-ing based approach (Wang et al., 2013a), which incorporated massivetraining for each ROI using image patches and corresponding labelsfrom the training set. A learning based approach is expected to improvethe segmentation accuracy within a single dataset, however it may alsooverfit to a specific set of reference labels and thusmay have lower gen-eralizability to new datasets, compared to a pure label fusion based ap-proachwithout learning.We tested the robustness ofMUSE as a genericsegmentation tool in our experiments usingmulti-site datasets. Imagingfeatures derived usingMUSE segmentation could accurately predict thebrain age, which is promising for the exploration of large quantities ofneuroimaging data from various studies with the aim of phenotypingthe human brain.

In our internal validation experiments we demonstrated the contri-bution of the similarity ranking and the boundary modulation terms tosegmentation performance. For one of the datasets, NIREP, the bound-ary modulation term did not improve the quantitative results. With acloser inspection, we observed that the decrease in the Dice score is

he age prediction was performed using an ensemble regressor using as input volumetric

194 J. Doshi et al. / NeuroImage 127 (2016) 186–195

mainly due to the low accuracy of the reference ROI masks. Specificallyin the deep brain structures, the reference ROIs were under-segmented.The boundary modulation term thus tends to compensate for theunder-segmented areas that have a similar intensity profile.

One of the limitations of MUSE is the high computational require-ment, which is a general problem for MAS methods. While the use ofmultiple registration methods and atlases would linearly increase thecomputation time, this is not a hard constraint for the practicalapplication of MUSE, because the registrations are very efficientlyparallelizable, and notably, multi-scale implementation of registrationalgorithms can effectively generate warps of various smoothness levelsfor no additional cost. Furthermore, depending on the available compu-tational resources, the user can easily limit the number of required reg-istrations using the command line parameters.

In summary, we have presented a methodological framework forensemble-based segmentation of brain MRI using a rich representationof brain anatomy viamultiple atlases, warps and parameter sets, and viaan adaptive and subject-specific ensemble-based segmentation. Our re-sults showed that this approach outperformsmethods that are based onsingle parameter sets and registration algorithms, and can thereforeprovide a foundation for robust segmentation.

We provideMUSE software both as a downloadable package, and asan application that can be run remotely on our web based platform.Webelieve that this would allow users with diverse needs, datasets,expertise and computational resources to be able to use MUSE bothconveniently and efficiently. The web client will also allow users touse a very large atlas dataset for the segmentation. This dataset will in-corporate datasets with considerable diversity in scanner and subjectcharacteristics, and will be regularly expanded with new atlases in thefuture.

Acknowledgments

This work was partially supported by the Intramural ResearchProgram, National Institute on Aging, NIH. This work is also supportedin part by the National Institutes of Health grant number R01-AG014971, and by contract HHSN271201300284.

Appendix A. Supplementary data

Supplementary data to this article can be found online at http://dx.doi.org/10.1016/j.neuroimage.2015.11.073.

References

Aljabar, P., Heckemann, R., Hammers, A., Hajnal, J., Rueckert, D., 2009. Multi-atlas basedsegmentation of brain images: atlas selection and its effect on accuracy. NeuroImage46 (3), 726–738 (Jul.).

Artaechevarria, X., Munoz-Barrutia, A., Ortiz-de Solorzano, C., 2009. Combination strate-gies in multi-atlas image segmentation: application to brain MR data. IEEE Trans.Med. Imaging 28 (8), 1266–1277 (Aug).

Asman, A.J., Landman, B.A., 2013. Non-local statistical label fusion for multi-atlas segmen-tation. Med. Image Anal. 17 (2), 194–208 (Feb.).

Aubert-Broche, B., Evans, A.C., Collins, L., 2006. A new improved version of the realisticdigital brain phantom. NeuroImage 32 (1), 138–145 (Aug).

Avants, B.B., Epstein, C.L., Grossman, M., Gee, J.C., 2008. Symmetric diffeomorphic imageregistration with cross-correlation: evaluating automated labeling of elderly andneurodegenerative brain. Med. Image Anal. 12 (1), 26–41 (Feb).

Bai, J., Trinh, T.L.H., Chuang, K.-H., Qiu, A., 2012. Atlas-based automatic mouse brain imagesegmentation revisited: model complexity vs. image registration. Magn. Reson. Imag-ing 30 (6), 789–798 (Jul).

Baloch, S., Davatzikos, C., 2008. Morphological appearance manifolds in computationalanatomy: groupwise registration and morphological analysis. NeuroImage 45(Suppl. 1), S73–S85 (Nov.).

Cardoso, M.J., Leung, K., Modat, M., Keihaninejad, S., Cash, D., Barnes, J., Fox, N.C., Ourselin,S., 2013. STEPS: Similarity and Truth Estimation for Propagated Segmentations and itsapplication to hippocampal segmentation and brain parcelation. Med. Image Anal. 17(6), 671–684 (Aug).

Christensen, G.E., Geng, X., Kuhl, J.G., Bruss, J., Grabowski, T.J., Pirwani, I.A., Vannier, M.W.,Allen, J.S., Damasio, H., 2006. Introduction to the non-rigid image registration evalu-ation project (NIREP). Proceedings of the Third International Conference on Biomed-ical Image Registration. WBIR'06. Springer-Verlag, Berlin, Heidelberg, pp. 128–135.

Coupé, P., Manjón, J.V., Fonov, V., Pruessner, J., Robles, M., Collins, D.L., 2011. Patch-basedsegmentation using expert priors: application to hippocampus and ventricle segmen-tation. NeuroImage 54 (2), 940–954 (Jan.).

Dosenbach, N.U., Nardos, B., Cohen, A.L., Fair, D.A., Power, J.D., Church, J.A., Nelson, S.M.,Wig, G.S., Vogel, A.C., Lessov-Schlaggar, C.N., Barnes, K.A., Dubis, J.W., Feczko, E.,Coalson, R.S., Pruett, J.R., Barch, D.M., Petersen, S.E., Schlaggar, B.L., 2010. Predictionof individual brain maturity using fMRI. Science 329 (5997), 1358–1361 (Sep).

Erus, G., Battapady, H., Satterthwaite, T.D., Hakonarson, H., Gur, R.E., Davatzikos, C., Gur,R.C., 2014. Imaging patterns of brain development and their relationship to cognition.Cereb. Cortex 25 (6), 1676–1684.

Franke, K., Luders, E., May, A., Wilke, M., Gaser, C., 2012. Brain maturation: predicting in-dividual BrainAGE in children and adolescents using structural MRI. NeuroImage 63(3), 1305–1312.

Good, C.D., Scahill, R.I., Fox, N.C., Ashburner, J., Friston, K.J., Chan, D., Crum, W.R., Rossor,M.N., Frackowiak, R.S., 2002. Automatic differentiation of anatomical patterns in thehuman brain: validation with studies of degenerative dementias. NeuroImage 17(1), 29–46 (Sep.).

Gousias, I., Hammers, A., Heckemann, R., Counsell, S., Dyet, L., Boardman, J., Edwards, A.,Rueckert, D., 2010. Atlas selection strategy for automatic segmentation of pediatricbrain MRIs into 83 ROIs. Imaging Systems and Techniques (IST), 2010 IEEE Interna-tional Conference on, pp. 290–293 (July).

Hartung, E.A., Laney, N., Kim, J.Y., Ruebner, R.L., Detre, J.A., Liu, H.-S., Davatzikos, C., Erus,G., Doshi, J.J., Schultz, R.T., Herrington, J.D., Jawad, A.F., Moodalbail, D.G., Gur, R.C.,Port, A.M., Radcliffe, J., Hooper, S.R., Furth, S.L., 2015. Design and methods of theNiCK study: neurocognitive assessment and magnetic resonance imaging analysisof children and young adults with chronic kidney disease. BMC Nephrol. 16, 66.

Heinrich, M.P., Jenkinson, M., Bhushan, M., Matin, T., Gleeson, F.V., Brady, S.M.,Schnabel, J.A., 2012. Mind: modality independent neighbourhood descriptorfor multi-modal deformable registration. Med. Image Anal. 16 (7), 1423–1435(special Issue on the 2011 Conference on Medical Image Computing andComputer Assisted Intervention).

Hibar, D.P., Stein, J.L., Renteria, M.E.E.A., 2015. Common genetic variants influence humansubcortical brain structures. Nature 520 (7546), 224–229 (Apr.).

Hoang Duc, A.K., Modat, M., Leung, K.K., Cardoso, M.J., Barnes, J., Kadir, T., Ourselin, S., forthe Alzheimer's Disease Neuroimaging Initiative, 2013. Using manifold learning foratlas selection in multi-atlas segmentation. PLoS ONE 8 (8), e70059, e70059 (Aug.).

Iglesias, J.E., Sabuncu, M.R., 2015. Multi-atlas segmentation of biomedical images: a sur-vey. Med. Image Anal. 24 (1), 205–219.

Isgum, I., Staring, M., Rutten, A., Prokop, M., Viergever, M.A., van Ginneken, B., 2009.Multi-atlas-based segmentation with local decision fusion—application to cardiacand aortic segmentation in CT scans. IEEE Trans. Med. Imaging 28 (7), 1000–1010(Jul).

Jack Jr., C.R., Bernstein, M.A., Fox, N.C., Thompson, P., Alexander, G., Harvey, D., Borowski,B., Britson, P.J., LWhitwell, J., Ward, C., Dale, A.M., Felmlee, J.P., Gunter, J.L., Hill, D.L.G.,Killiany, R., Schuff, N., Fox-Bosetti, S., Lin, C., Studholme, C., DeCarli, C.S., Krueger, G.,Ward, H.A., Metzger, G.J., Scott, K.T., Mallozzi, R., Blezek, D., Levy, J., Debbins, J.P.,Fleisher, A.S., Albert, M., Green, R., Bartzokis, G., Glover, G., Mugler, J., Weiner, M.W.,2008. The alzheimer's disease neuroimaging initiative (ADNI): MRI methods.J. Magn. Reson. Imaging 27 (4), 685–691 (Apr).

Khan, A.R., Cherbuin, N., Wen, W., Anstey, K.J., Sachdev, P., Beg, M.F., 2011. Optimalweights for local multi-atlas fusion using supervised learning and dynamic informa-tion (SuperDyn): validation on hippocampus segmentation. NeuroImage 56 (1),126–139 (May).

Klein, A., Andersson, J., Ardekani, B.A., Ashburner, J., Avants, B., Chiang, M.-C., Christensen,G.E., Collins, D.L., Gee, J., Hellier, P., Song, J.H., Jenkinson, M., Lepage, C., Rueckert, D.,Thompson, P., Vercauteren, T., Woods, R.P., Mann, J.J., Parsey, R.V., 2009. Evaluationof 14 nonlinear deformation algorithms applied to human brain MRI registration.NeuroImage 46 (3), 786–802 (Jul).

Konick, L.C., Friedman, L., 2001. Meta-analysis of thalamic size in schizophrenia. Biol. Psy-chiatry 49 (1), 28–38 (Jan.).

Konukoglu, E., Glocker, B., Zikic, D., Criminisi, A., 2013. Neighbourhood approximationusing randomized forests. Med. Image Anal. 17 (7), 790–804 (Oct.).

Laakso, M.P., Partanen, K., Riekkinen, P., Lehtovirta, M., Helkala, E.L., Hallikainen, M.,Hanninen, T., Vainio, P., Soininen, H., 1996. Hippocampal volumes in Alzheimer's dis-ease, Parkinson's disease with and without dementia, and in vascular dementia: anMRI study. Neurology 46 (3), 678–681 (Mar).

Landman, B.A., Asman, A.J., Scoggins, A.G., Bogovic, J.A., Xing, F., Prince, J.L., 2011. Robuststatistical fusion of image labels. IEEE Trans. Med. Imaging 31 (2), 512–522 (Oct.).

Leung, K.K., Barnes, J., Modat, M., Ridgway, G.R., Bartlett, J.W., Fox, N.C., Ourselin, S., 2011.Brain MAPS: an automated, accurate and robust brain extraction technique using atemplate library. NeuroImage 55 (3), 1091–1108 (Apr.).

Levitt, J.J., McCarley, R.W., Dickey, C.C., Voglmaier, M.M., Niznikiewicz, M.A., Seidman, L.J.,Hirayasu, Y., Ciszewski, A.A., Kikinis, R., Jolesz, F.A., Shenton, M.E., 2002. MRI study ofcaudate nucleus volume and its cognitive correlates in neuroleptic-naive patientswith schizotypal personality disorder. Am. J. Psychiatry 159 (7), 1190–1197 (Jul).

Lin, L.I., 1989. A concordance correlation coefficient to evaluate reproducibility. Biometrics45 (1), 255–268 (Mar).

Lötjönen, J.M., Wolz, R., Koikkalainen, J.R., Thurfjell, L., Waldemar, G., Soininen, H.,Rueckert, D., 2010. Fast and robust multi-atlas segmentation of brain magneticresonance images. NeuroImage 49 (3), 2352–2365 (Feb.).

Marcus, D.S., Fotenos, A.F., Csernansky, J.G., Morris, J.C., Buckner, R.L., 2010. Open AccessSeries of Imaging Studies (OASIS): longitudinal MRI data in nondemented anddemented older adults. J. Cogn. Neurosci. 22 (12), 2677–2684 (Dec.).

Medland, S.E., Jahanshad, N., Neale, B.M., Thompson, P.M., 2014. Whole-genome analysesof whole-brain data: working within an expanded search space. Nat. Neurosci. 17 (6),791–800 (Jun.).

195J. Doshi et al. / NeuroImage 127 (2016) 186–195

Ou, Y., Sotiras, A., Paragios, N., Davatzikos, C., 2011. DRAMMS: deformable registration viaattributematching andmutual-saliency weighting.Med. Image Anal. 15 (4), 622–639(Aug).

Ou, Y., Akbari, H., Bilello, M., Da, X., Davatzikos, C., 2014. Comparative evaluation ofregistration algorithms in different brain databases with varying difficulty: resultsand insights. IEEE Trans. Med. Imaging 33 (10), 2039–2065 (Oct).

Pipitone, J., Park, M.T., Winterburn, J., Lett, T.A., Lerch, J.P., Pruessner, J.C., Lepage, M.,Voineskos, A.N., Chakravarty, M.M., Initiative, A.D., et al., 2014. Multi-atlas segmenta-tion of the whole hippocampus and subfields using multiple automatically generatedtemplates. NeuroImage 101, 494–512.

Poldrack, R.A., Feb. 2007. Region of interest analysis for fMRI. Soc. Cogn. Affect. Neurosci. 2(1), 67–70.

Resnick, S.M., Goldszal, A.F., Davatzikos, C., Golski, S., Kraut, M.A., Metter, E.J., Bryan, R.N.,Zonderman, A.B., 2000. One-year age changes in MRI brain volumes in older adults.Cereb. Cortex 10 (5), 464–472 (May).

Resnick, S.M., Pham, D.L., Kraut, M.A., Zonderman, A.B., Davatzikos, C., 2003. Longitudinalmagnetic resonance imaging studies of older adults: a shrinking brain. J. Neurosci. 23(8), 3295–3301 (Apr).

Rohlfing, T., Maurer Jr., C.R., 2005. Multi-classifier framework for atlas-based image seg-mentation. Pattern Recogn. Lett. 26 (13), 2070–2079 (Oct.).

Sabuncu, M.R., Yeo, B.T.T., Van Leemput, K., Fischl, B., Golland, P., Oct 2010. A generativemodel for image segmentation based on label fusion. IEEE Trans. Med. Imaging 29(10), 1714–1729.

Sanroma, G., Wu, G., Gao, Y., Shen, D., 2014. Learning to rank atlases for multiple-atlassegmentation. IEEE Trans. Med. Imaging 33 (10), 1939–1953 (Oct).

Satterthwaite, T.D., Elliott, M.A., Ruparel, K., Loughead, J., Prabhakaran, K., Calkins, M.E.,Hopson, R., Jackson, C., Keefe, J., Riley, M., Mentch, F.D., Sleiman, P., Verma, R.,Davatzikos, C., Hakonarson, H., Gur, R.C., Gur, R.E., 2014. Neuroimaging of the Phila-delphia neurodevelopmental cohort. NeuroImage 86, 544–553 (Feb).

Sotiras, A., Davatzikos, C., Paragios, N., 2013. Deformable medical image registration: asurvey. IEEE Trans. Med. Imaging 32 (7), 1153–1190 (May).

Wang, H., Suh, J.W., Das, S.R., Pluta, J., Craige, C., Yushkevich, P.A., 2013a. Multi-atlas segmen-tation with joint label fusion. IEEE Trans. Pattern Anal. Mach. Intell. 35 (3), 611–623.

Wang, H., Pouch, A., Takabe, M., Jackson, B., Gorman, J., Gorman, R., Yushkevich, P.A.,2013b. Multi-atlas segmentation with robust label transfer and label fusion. Informa-tion processing in medical imaging 23 pp. 548–559.

Warfield, S.K., Zou, K.H., Wells, W.M., 2004. Simultaneous truth and performance level es-timation (STAPLE): an algorithm for the validation of image segmentation. IEEETrans. Med. Imaging 23 (7), 903–921 (Jul).

Wolz, R., Aljabar, P., Rueckert, D., Heckemann, R., Hammers, A., 2009. Segmentation ofsubcortical structures and the hippocampus in brain MRI using graph-cuts andsubject-specific a-priori information. Biomedical Imaging: From Nano to Macro,2009. ISBI'09. IEEE International Symposium on, pp. 470–473 (June).

Wu, M., Rosano, C., Lopez-Garcia, P., Carter, C.S., Aizenstein, H.J., 2007. Optimum templateselection for atlas-based segmentation. NeuroImage 34 (4), 1612–1618 (Feb).

Wu, G., Kim, M., Sanroma, G., Wang, Q., Munsell, B.C., Shen, D., A. D. N. I., 2015. Hierarchi-cal multi-atlas label fusion with multi-scale feature representation and label-specificpatch partition. NeuroImage 106, 34–46 (Feb).

Zikic, D., Glocker, B., Criminisi, A., 2014. Encoding atlases by randomized classification forestsfor efficient multi-atlas label propagation. Med. Image Anal. 18 (8), 1262–1273 (Dec).