Computer-aided diagnosis for dynamic contrast-enhanced breast MRI ofmass-like lesions using a multiparametric model combining a selection ofmorphological, kinetic, and spatiotemporal features
S. Agliozzoa)
im3D, Research and Development Department, Turin 10153, Italy
M. De LucaUnit of Radiology, Institute for Cancer Research and Treatment, IRCC, Turin 10060, Italy
C. BraccoUnit of Radiation Therapy, Institute for Cancer Research and Treatment, IRCC, Turin 10060, Italy
A. Vignati, V. Giannini, and L. MartincichUnit of Radiology, Institute for Cancer Research and Treatment, IRCC, Turin 10060, Italy
L. A. CarbonaroUnita di Radiologia, IRCCS Policlinico San Donato, San Donato Milanese, Milano 20097, Italy
A. Bertim3D, Research and Development Department, Turin 10153, Italy
F. SardanelliUnita di Radiologia, IRCCS Policlinico San Donato, San Donato Milanese, Milano 20097, Italyand Dipartimento di Scienze Medico-Chirurgiche, Universita degli Studi di Milano,San Donato Milanese, Milano 20097, Italy
D. ReggeUnit of Radiology, Institute for Cancer Research and Treatment, IRCC, Turin, Italy
(Received 1 August 2011; revised 11 December 2011; accepted for publication 12 February 2012;
published 8 March 2012)
Purpose: Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is a radiological
tool for the detection and discrimination of breast lesions. The aim of this study is to evaluate a
computer-aided diagnosis (CAD) system for discriminating malignant from benign breast lesions at
DCE-MRI by the combined use of morphological, kinetic, and spatiotemporal lesion features.
Methods: Fifty-four malignant and 19 benign breast lesions in 51 patients were retrospectively eval-
uated. Images were acquired at two centers at 1.5 T. Mass-like lesions were automatically segmented af-
ter image normalization and elastic coregistration of contrast-enhanced frames. For each lesion, a set of
28 3D features were extracted: ten morphological (related to shape, margins, and internal enhancement
distribution); nine kinetic (computed from signal-to-time curves); and nine spatiotemporal (related to
the variation of the signal between adjacent frames). A support vector machine (SVM) was trained with
feature subsets selected by a genetic search. Best subsets were composed of the most frequent features
selected by majority rule. The performance was measured by receiver operator characteristics analysis
with a stratified tenfold cross-validation and bootstrap method for confidence intervals.
Results: SVM training by the three separated classes of features resulted in an area under the curve
(AUC) of 0.90 6 0.04 (mean 6 standard deviation), 0.87 6 0.06, and 0.86 6 0.06 for morphologi-
cal, kinetic, and spatiotemporal feature, respectively. Combined training with all 28 features
resulted in AUC of 0.96 6 0.02 obtained with a selected feature subset composed by two morpho-
logical, one kinetic, and two spatiotemporal features.
Conclusions: Quantitative combination of morphological, kinetic, and spatiotemporal
features is feasible and provides a higher discriminating power than using the three differ-
ent classes of features separately. VC 2012 American Association of Physicists in Medicine.
[http://dx.doi.org/10.1118/1.3691178]
Key words: support vector machine, genetic feature selection, dynamic contrast-enhanced breast MRI,
computer-aided diagnosis
I. INTRODUCTION
Breast cancer is one of the leading causes of death in
women.1,2 Scientific evidence indicates that early detection
and treatment of breast cancer can reduce the mortality and
morbidity.3,4 Dynamic contrast-enhanced magnetic reso-
nance imaging (DCE-MRI) has evolved into an established
clinical imaging modality for detection and diagnosis of
1704 Med. Phys. 39 (4), April 2012 0094-2405/12/39(4)/1704/12/$30.00 VC 2012 Am. Assoc. Phys. Med. 1704
breast lesions. The American Cancer Society has issued a
guideline recommending annual breast MRI as an adjunct to
mammography for screening women with lifetime breast
cancer risk greater than 20%–25%,5 an indication confirmed
by recent studies.6,7 Moreover, indications for breast MRI
now include not only screening high-risk women but also
presurgical staging, therapy response monitoring, and
searching for occult primary breast cancer, as recently dis-
cussed by a multidisciplinary group.8
Breast MRI currently demonstrates a high sensitivity
(ranging 93%–100%) and a more variable specificity.9–13 A
robust estimate for specificity has been produced by a large
meta-analysis by Peters et al.14 who reported a value of 72%
(combine with a sensitivity of 92%). Moreover, the diagnos-
tic performance is subjected to the level of experience of the
radiologist.16–18 In an attempt to address this issue, the BI-
RADS (breast imaging reporting and data system) MRI lexi-
con19 was developed to provide standardized criteria and
descriptors for reporting breast MRI findings. In particular,
the BI-RADS lexicon provides terminology for describing
enhancement kinetic behavior and morphology of a lesion.
Computer-aided detection (CADe) and diagnosis systems
(CADx) could provide an accurate and time efficient support
to the interpretation of breast MR images by improving
lesion detection and differentiation between malignant and
benign nodules.15
Research has been undertaken on developing computer-
aided systems for DCE-MRI for detection and diagnosis of
breast tumors. Typically, CAD systems characterize mor-
phological and contrast enhancement kinetics features of
lesions, in order to depict differences between malignant and
benign lesions.20–27 Indeed, spatial attributes such as spicu-
lated margins and irregular shapes are important predictors
of malignancy, whereas smooth border and round shapes are
more often associated with benignancy;28 heterogeneous and
peripheral internal enhancement is an important indicator of
malignancy, whereas homogenous enhancement is often
associated with benignancy.28,30 Temporal attributes such as
early strong enhancement with rapid wash out are more fre-
quently found in malignant lesions, whereas benign lesions
have typically slow persistent enhancement increase.28,31
Recent reports have introduced another class of features
related to the spatial variations of temporal signal enhance-
ments at a pixel scale to capture the complexity of the spatio-
temporal association of tumor enhancement patterns,
proving differentiation of malignant and benign lesions.32–35
The concept of studying spatiotemporal properties is related
to the lesion internal enhancement, but it is extended to cap-
ture a measure of the complex change in this feature as a
function of contrast enhancement with more rigorous mathe-
matical frameworks. In most of previous studies, however,
the combination of all three classes of morphological, ki-
netic, and spatiotemporal features was rarely investigated.
Moreover, most of proposed CAD schemes are not auto-
matic, since they are based on either manual lesion detection
or manual lesion segmentation.
The aim of this study is to present a full scheme for the
discrimination of malignant from benign breast lesions
detected and segmented automatically. Discrimination is
based on quantitative extraction of features, diagnostic fea-
ture selection, and lesion classification. In particular, quanti-
tative feature extraction includes the calculation of a pool of
three classes of features: morphological, contrast enhance-
ment kinetic, and spatiotemporal features. New morphologi-
cal and contrast enhancement kinetic features are introduced
together with those already reported in the literature. Spatio-
temporal properties are analyzed with features extracted
from new dedicated maps. Genetic algorithms (GAs) are
used to choose the most performing feature subset from the
feature pool. The optimal feature subset is used to train a
classifier based on support vector machines (SVMs) classify-
ing lesions into malignant or benign.
II. MATERIALS AND METHODS
II.A. MRI protocols
Breast MRI studies were acquired at two centers. The first
group contains 26 studies acquired with a 1.5-T scanner
(Sigma Exite Hdx, General Electric Healthcare, Milwakee,
USA) using an eight-channel breast RF coil and a fat-
saturated three dimensional (3D) axial fast spoiled gradient-
echo sequence [VIBRANT(TM), general electric] with the
following technical parameters: TR/TE¼ 4.5/2.2 ms, flip
angle¼ 15�, reconstructed matrix 512� 512, field of view
32 cm, slice thickness 2.6 mm, and pixel size 0.39 mm2. The
3D sequence was acquired once before and 5 times after in-
travenous injection with time resolution 50 or 90 s; a late ac-
quisition frame was obtained 7 min after contrast injection.
Gadopentetate dimeglumine (Gd-DPTA, Magnevist, Bayer-
Schering, Berlin, Germany) was administered at a dose of
0.1 mmol/kg at 2 ml/s, followed by 20 ml of saline solution
at the same rate. The second group comprised 25 studies
performed on a different 1.5-T scanner (Sonata Maestro
Class, Siemens, Erlangen, Germany), using a standard bilat-
eral breast coil (four-element; two-channel) and a dynamic
3D axial fast low angle shot sequence with the following
technical parameters: TR/TE¼ 11/4.9 ms, flip angle 25�,reconstructed matrix 512� 512, field of view 38 cm, slice
thickness 1.3 mm, and pixel size 0.56 mm2. Gadobenate
dimeglumine (MultiHance, Bracco, Milan, Italy) was used
as contrast material, administered at a dose of 0.1 mmol/kg
at 2 ml/s, followed by 20 ml of saline solution at the same
rate. One baseline scan was acquired prior to contrast injec-
tion, followed by five postcontrast frames taken 118 s apart.
The Local Ethical Committees approved the retrospective
use of the database for scientific purposes and waived from
informed consent. The study was conducted in accordance
with national legislation and the declaration of Helsinki.
II.B. Lesion dataset
Overall, the dataset contained 73 mass-like lesions collected
from small consecutive series and confirmed by histopathology
(core-biopsy and/or surgical specimen): 54 malignant and 19
benign lesions (Table I). Figure 1 shows examples of malig-
nant and benign lesions.
1705 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1705
Medical Physics, Vol. 39, No. 4, April 2012
Lesion size was 13 6 8.4 mm (mean 6 standard devia-
tion) for benign lesions and 16.1 6 14.7 mm) for malignant
lesions, with lesion size determined as the longest diameter
measured by radiologists. Thirty three lesions had a size
smaller than 10 mm (22 malignant, 11 benign), whereas
40 lesions had a size larger than 10 mm (32 malignant, 8
benign). Table I summarizes lesions histology.
II.C. Lesion detection and segmentation
Automatic lesion detection and segmentation were carried
out as already described in detail in Refs. 36 and 37. To sum-
marize the method, unenhanced and contrast-enhanced
frames are aligned with an elastic coregistration algorithm,
in order to correct for possible misalignments in the dynamic
sequence; then, subtracted images (contrast-enhanced minus
unenhanced) are calculated. Exploiting a priori knowledge,
the breast area is segmented to discard enhanced anatomical
structures located outside the region of interest (e.g., heart).
After breast segmentation, images are normalized to the con-
trast enhancement of mammary vessels, in order to correct
for variations of acquisition parameters and contrast mate-
rial. Mammary vessels are segmented with a method pro-
posed by Sato et al.38 Finally, a histogram based threshold is
applied to select enhanced areas, and filters based on mor-
phology and kinetic characteristics are applied, to reduce the
number of false positive findings. Figure 1 shows examples
of segmentation masks for a malignant and benign lesion.
The detection and segmentation algorithm generated a
number of candidate lesions. The candidates corresponding to
true lesions were manually confirmed by expert radiologists
and associated to histology. Morphological, enhancement
kinetic, and spatiotemporal features were extracted from these
confirmed detections. These features are described in the
following sections and summarized in Table II.
II.D. Morphological features
Two morphological features related to the lesion shape
are calculated on the binary mask: circularity and convex
index. Circularity20 is defined as
lesion volume within sphere of effective diameter
lesion volume; (1)
where the effective diameter is defined by
2 �ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi3 � lesion volume
4p3
r; (2)
and coindex is given by
volume of convex hull
lesion volume; (3)
with the convex hull computed using the algorithm described
in Ref. 39.
Three features are used to describe the margin of a lesion:
irregularity, mean, and standard deviation of angles between
surface normals40 (ABSN).
Irregularity20 is given by
1� p � effective diameter2
surface of lesion: (4)
The ABSN is a measure of tortuosity, computed at every
voxel in a surrounding layer around the lesion border,
defined as the sum of angles between ni and its neighbors nij
hi ¼X
j
arccosðni � nijÞ; (5)
where the vectors ni are approximated by normalized gradient
vectors. The average ABSN will be zero on a regular, flat bor-
der as the neighboring normal vectors are nearly parallel, while
TABLE I. Histological types of the 73 lesions included in the study.
Tumor type Number Percentage (%)
Malignant lesions 54 74
Invasive ductal carcinoma (IDC) 36 67
Invasive lobular carcinoma (ILC) 4 7
Ductal carcinoma in situ (DCIS) 4 7
Mixed invasive carcinoma 10 18
Benign lesions 19 26
Fibroadenoma (FAD) 9 47
Papilloma 4 21
Other benign lesions 6 32
FIG. 1. Examples of one malignant IDC (top row) and one benign FAD (bottom row) breast lesions. For both lesions from left to right images recorded are
shown as acquired before (first image) and progressively after contrast injection, with 90 s time resolution (from second to fifth image). The sixth image is a
late contrast-enhanced image recorded 560 s after contrast injection. The last image on the right is the resulting automatic segmentation mask.
1706 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1706
Medical Physics, Vol. 39, No. 4, April 2012
it will be within (0, p] on an irregular, curved surface. These
features, describing lesion margins, are computed on the maxi-
mum intensity projection image along time (MIPT), where
each pixel takes the maximum value among the corresponding
pixels in all time frames. Therefore, in the MIPT all parts of the
lesion uptaking contrast along time frames are visible.
Other six features characterizing the internal enhance-
ment pattern are extracted: the autocorrelation function
(evaluated at 2 mm displacement), two features related to the
peripheral uptake, and the mean and standard deviation of
the shape index41 computed inside the segmented mass. The
internal enhancement features are computed on the first
contrast-enhanced subtracted image in order to analyze the
early contrast enhancement pattern. The periphery and the
center of some malignant lesions show different enhance-
ment characteristics after injection of contrast material at
MRI.42,43 In this work, we have developed an algorithm to
make a quantitative analysis of the peripheral uptake. First, a
flood-fill operation is performed on the original binary mask
in order to fill segmentation holes, and the distance trans-
form is applied to the filled binary mask B1 (Fig. 2) to label
each lesion voxel with its distance from the lesion nearest
border. Then, a new binary image, B2, is created by setting
the voxels with intensities greater than the median contrast
value inside the lesion to foreground and the remaining
voxel to background. The function H(d) is extracted as
HðdÞ ¼ number foreground voxels in B1at distance d
number foreground voxels in B2 at distance d; (6)
where d is the distance from the lesion border normalized by
the maximum value of such distance within the lesion. A
third-degree polynomial is fitted on H(d) and the second and
third order fitting coefficients are used as peripheral uptake
quantitative features.
Another useful descriptor of the mass is the level of con-
trast homogeneity within lesion. This properties is described
by the local shape index computed for each lesion voxel to
characterize the shape of a local isosurface passing through a
voxel p. This isosurface can be represented in a parametric
2D form as P ¼ fðu; vÞ 2 R2; hðu; v;/ðu; vÞÞ ¼ ag, and the
principal curvatures k1 and k2 are obtained as the eigenval-
ues of the Weingarten endomorphism. Then the shape index
(SI) S(p) at the voxel p is defined as41
SðpÞ ¼ 2
p� arctan
kmaxðpÞ þ kminðpÞkmaxðpÞ � kminðpÞ
; (7)
where kmax ¼ maxðk1; k2Þ and kmin ¼ minðk1; k2Þ. Since
the SI is related to the curvature of the intensity
TABLE II. List of all features used according to the origin class (morphological, kinetic, spatiotemporal).
Feature type and no lesion feature
Morphological
1 Circularity
2 Convex hull
3 Irregularity
4 Mean of angles between surface normals, mean(ABSN)
5 Standard deviation of angles between surface normals, std(ABSN)
6 Autocorrelation, AutoCorr
7 First coefficient of peripheral uptake (peripheralUptake1)
8 Second coefficient of peripheral uptake (peripheralUptake2)
9 Mean of Shape Index, mean(SI)
10 Standard deviation of Shape Index, std(SI)
Kinetic
11 Mean of amplitude coefficients A of function (9), mean(A)
12 Standard deviation of amplitude A coefficients of function (9), std(A)
13 Entropy of amplitude coefficients A of function (9), entropy(A)
14 Mean of decay coefficients D of function (9), mean(D)
15 Standard deviation of decay coefficients D of function (9), std(D)
16 Entropy of decay coefficients D of function (9), entropy(D)
17 Mean of area under contrast enhancement curve, mean(AUCEC)
18 Standard deviation of area under contrast enhancement curve, std(AUCEC)
19 Entropy of area under contrast enhancement curve, entropy(AUCEC)
Spatiotemporal
20 Mean of mean eigenvalue-trace image, mean(MeanTr)
21 Standard deviation of mean eigenvalue-trace image, std(MeanTr)
22 Entropy of mean eigenvalue-trace image, entropy(MeanTr)
23 Mean of standard deviation eigenvalue-trace image, mean(StdTr)
24 Standard deviation of standard deviation eigenvalue-trace image, std(StdTr)
25 Entropy of standard deviation eigenvalue-trace image, entropy(StdTr)
26 Mean of range eigenvalue-trace image, mean(RangeTr)
27 Standard deviation of range eigenvalue-trace image, std(RangeTr)
28 Entropy of range eigenvalue-trace image, entropy(RangeTr)
1707 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1707
Medical Physics, Vol. 39, No. 4, April 2012
isosurface, it is independent of the intensity values of the
lesion. Indeed, lesion intensity values can vary according
to the contrast material and to the acquisition setting.
Lesions with heterogeneous internal enhancement have a
SI map with more different values than the homogeneous
lesions, as shown in Fig. 3; therefore, SI mean and stand-
ard deviation can be used as internal enhancement
features.
FIG. 3. From left to right: the MIPT image, the SI map
for a malignant heterogeneous lesion (DCIS) (top row)
and for a benign homogeneous lesion (FAD) (bottom
row).
FIG. 2. From left to right: the first contrast-enhanced subtracted frame, the binary mask B1, the binary mask B2, and the function H(d) for a malignant lesion
(IDC) with a peripheral wash-in (on top) and for an heterogeneous lesion (DCIS) (bottom).
1708 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1708
Medical Physics, Vol. 39, No. 4, April 2012
II.E. Enhancement kinetic features
Enhancement kinetics is related to the time course of sig-
nal intensity over time. We denote Sðr; iÞ as the voxel value
at location r in the lesion at time frame i, where i runs from 0
(precontrast frame) to N (last postcontrast frame). The inten-
sity Sðr; iÞ is normalized to the contrast enhancement of
mammary vessels, in order to correct for variations of acqui-
sition parameters and contrast material. For each lesion
voxel, the contrast enhancement is computed as
Cðr; iÞ ¼ Sðr; iÞ � Sðr; 0ÞSðr; 0Þ ; i ¼ 1;…;N; (8)
where the quantity Cðr; iÞ is related to the contrast material
concentration in the extracellular space of breast tissue in
r.44 Note that at i¼ 0, Cðr; iÞ¼ 0.
Two types of features are derived from the enhancement
kinetics. The first type is related to the fitting of the contrast
enhancement to the following analytical exponential
function:
CðtÞ ¼ A � t � e�tD ; (9)
where A controls the function amplitude and therefore the
contrast uptake, whereas D captures the function decay and
thus the contrast washout. Coefficients A and D were used as
features. Figure 4 shows examples of A and D maps for the
malignant and the benign lesion shown in Fig. 1. The malig-
nant lesion has heterogeneous maps, whereas the benign
lesion has homogeneous maps. Moreover, the malignant
lesion shows a relative large uptake and large washout in the
central lesion zone, whereas the benign lesion shows a
weaker washout all over the volume. We characterized the
lesion contrast uptake and washout by fitting the contrast
enhancement Cðr; iÞ with an analytical function rather than
using a two-compartmental pharmacokinetic model.45 The
use of a pharmacokinetic model implies strict constrains on
temporal resolution of MRI technique to measure contrast
material concentration in an artery (arterial input function)
and in the tissue (tissue residue function).46 These constrains
are not fulfilled in DCE-MRI acquisition protocols typically
used in clinical practice. Although the analytical function
proposed [Eq. (9)] cannot model physiologically the lesion,
its simple form allows for relaxing constrains on the acquisi-
tion protocols still characterizing the kinetic behavior of the
lesion. The second type of feature computes the area under
the contrast enhancement curve Cðr; iÞ. This feature is
related to the total amount of contrast material in the lesion
tissue. For each of the three features, mean, standard devia-
tion, and entropy were computed, yielding a total of nine
contrast enhancement kinetic features.
II.F. Spatiotemporal features
Spatiotemporal features are related to the spatial variation
of contrast enhancement over time. The lesion spatial varia-
tion of the contrast is represented as a signal intensity flow.
At voxel-scale, the flow can be studied looking at voxel sig-
nal intensity changes from frame to frame in relation to
neighboring voxels. For example, if a voxel appears brighter
than its neighbors in a given frame, and appears darker in the
subsequent frame, we may say that the signal intensity is
flowing out of the voxel. We denote Dðr; jÞ as the voxel
FIG. 4. Examples of contrast enhancement kinetic
maps for a malignant (IDC) (top row) and a benign
(FAD) (bottom row) lesion. For both lesions, from left
to right maps of the coefficients A and D of the analyti-
cal function [Eq. (9)] are shown.
1709 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1709
Medical Physics, Vol. 39, No. 4, April 2012
signal intensity difference between adjacent frames at loca-
tion r, j running from 0 (precontrast) to N� 1, where N is the
number of frames. For each Dðr; jÞ, the vector field Gðr; jÞ is
computed as
Gðr; jÞ ¼ r � Dðr; jÞ: (10)
Gðr; jÞ represents the signal intensity change between adja-
cent frames and quantifies the amount of intensity that has
flowed from a voxel to its neighbors. Gðr; jÞ is topologically
characterized using the Jacobian matrix Jðr; jÞ
Jðr; jÞ ¼rxGx ryGx rzGx
rxGy ryGy rzGy
rxGz ryGz rzGz
24
35; (11)
and calculating the three eigenvalues of the characteristic
equation
k2 þ Pkþ Q ¼ 0; (12)
where P ¼ �traceðJÞ and Q ¼ jJj. Each voxel of the vector
field Gðr; jÞ is therefore topologically characterized by three
eigenvalues ki (i¼ 1, 2, 3). A voxel represents a source of
signal if all three eigenvalues are positive, whereas it is a
sink of signal if the eigenvalues are negatives.47 The voxel
topological information is summarized by the calculation of
the eigenvalues trace trðrÞ
trðrÞ ¼X
i
ki; i ¼ 1; 2; 3: (13)
Sinks and sources are characterized by negative and positive
traces, respectively. Mean, standard deviation, and range
(max–min) of the N� 1 traces are calculated for each voxel,
and three maps are generated: MeanTr(r), StdTr(r), and
RangeTr(r). Out of each map, lesion mean, standard devia-
tion, and entropy are computed, to get a total of nine spatio-
temporal features. Figure 5 shows examples of MeanTr and
StdTr maps for the malignant and the benign lesion shown in
Fig. 1. Values are reported in arbitrary units. The MeanTr
map of the malignant lesion shows smaller values at the
boundary than in the center, whereas the benign lesion shows
an opposite behavior. This can be explained by the fact that
the malignant lesion shows over the time a strong washout
from the center to the boundary as shown in Fig. 1. Accord-
ingly, the trace values in the center, immediately after the
contrast injection, are lower than those at the boundary, the
lesion center behaves as a sink. These trace values at center
in the late dynamic series decrease their negative values
since the center acts as a source. This results, on average, in
less negative trace values at the center than at the boundary.
Considering the StdTr maps, the malignant lesion shows a
pattern more heterogeneous than that of the benign lesion.
II.G. Feature selection
A feature selection method based on the wrapper
approach48 was used in our study in order to select an opti-
mal subset of the original features. Wrapper approach uses
the performance of the learning algorithm as feature evalua-
tion function; therefore, the selected feature subset (FS) is
dependent on the chosen classifier. The performance mea-
sure is the area under the receiver operator characteristics
(ROC) curve (AUC) calculated with a tenfold cross-
validation. Feature selection has been carried out through a
genetic approach. Genetic algorithms are search and optimi-
zation algorithms based on natural evolution and selection as
FIG. 5. Examples of spatiotemporal maps for a malig-
nant (IDC) (top row) and a benign (FAD) (bottom row)
lesion. For both lesions from left to right the MeanTr
and the StdTr maps are shown.
1710 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1710
Medical Physics, Vol. 39, No. 4, April 2012
means of determining an optimal solution to feature selec-
tion problem. A cross generational elitist selection, heteroge-
neous recombination, and cataclysm mutation genetic
algorithm49 is used to search through candidate subsets of
features. This GA starts with an initial random population of
dimension N, each individual in the population represents a
candidate solution to the feature subset selection problem
and is represented by a binary string, called chromosome,
having a length equal to the total number of features, each
bit representing the presence or absence of a feature. The fit-
ness of an individual is determined by the following
equation:
Fitness ¼ ð1� AUCÞ þ k
nMalLesions� nSelectFeat; (14)
where the AUC is the area under the ROC curve, k is a con-
stant (k¼ 1), nMalLesions is the number of malignant lesions
used in the training, and nSelectFeat is the number of selected
features in a given individual. The fitness is composed of two
competing terms. The first term ð1� AUCÞ is formed by indi-
viduals with high classification performances that tend to
have a larger number of features. The smaller this part is, the
more discriminating the individuals are. The second term acts
as a penalty term, discouraging selection of individuals having
large number of active features. The weight of the penalty
term is controlled by the constant k, with large k values pro-
moting individuals with a small number of features.
Each feature subset selection experiment is composed of
ten experiments of tenfolds cross-validation. Each cross-
validation generates ten feature subsets, keeping iteratively
one of the ten folds out of the GA search and using the
remaining nine folds for the selection of an optimal candi-
date feature subset with a further tenfold cross-validation.
Repeating this procedure for the remaining nine folds, ten
subsets are selected. After ten experiments of cross-
validation, 100 candidates feature subsets are generated. The
optimal feature subset is selected by the majority rule out of
the 100 candidate subsets. Cross-validation is carried out
keeping constant the proportion between malignant and be-
nign lesions in order to reduce validation bias. Moreover, in
the training, the lesions of the minority class (i.e., benign
class) are duplicated to balance the number of lesions of the
majority class and reduce the classification bias.
II.H. Classification
Support vector regressors �-SVRs are used because of
their good generalization and ability to solve many practical
problems, such as small sample, non linearity, and high
dimensionality.50,51 SVRs map the input space into the high-
dimensional feature space and determine the optimal hyper-
plane given by
f ðxÞ ¼ wTgðxÞ þ b; (15)
where w is the one-dimensional weight vector, gðxÞ is the
mapping function that maps the input feature vector x into
the one-dimensional space, and b is the bias term. In �-SVR,
a piecewise error function E(r) is defined as
EðrÞ ¼ 0 if jrj < �;jrj � � otherwise;
�
where the residual r is the difference between the expected
response y and the predicted value f ðxÞ. The solution is
determined in order to minimize the structural risk, i.e., the
probability of testing patterns to be classified correctly for a
fixed but unknown probability of the data. The solution of
the constrained minimization problem is52
w ¼XM
i¼1
ðai � a�i ÞgðxiÞ; (16)
f ðxÞ ¼XM
i¼1
ðai � a�i ÞgTðxiÞgðxiÞ þ b; (17)
where ai and a�i are the Lagrange multipliers and only the
vectors xi in the training set with both ai and a�i 6¼ 0 contrib-
ute in constructing the function f ðxÞ and are called support
vectors.
Hðxi; xÞ ¼ gTðxiÞgðxÞ is the kernel which maps the fea-
ture vectors into higher dimensional spaces to achieve bet-
ter class separation translating nonlinear boundaries in the
original space into linear boundaries. For our classification
problem, the radial basis kernel proved to yield the best
result
Hðxi; xÞ ¼ e�cjjx�xijj2 (18)
where c is the kernel parameter and the support vectors xi
are the center of the radial basis functions.
II.I. Experiments and performance evaluation
A series of experiments were carried out to evaluate the
performance of the discrimination system using different
classes of features. Three experiments were performed
using the three classes of features separately; three other
experiments used paired classes of features; the last experi-
ment used all features. For all experiments, AUC was cal-
culated to quantitatively estimate the performance. ROC
curves are calculated from the feature subsets selected by
the genetic search with a tenfold cross-validation strategy.
Moreover, performances were reported for each selected
feature subset in terms of sensitivity, specificity, positive
predictive value, negative predictive value, and accuracy.
Performances were calculated at the ROC point with the
highest accuracy, the point closest to the top-left corner
(0,1) of the ROC.
Finally, classification performances were also evaluated
according to the lesion size, grouping lesions based on their
size being smaller than 10 mm or larger or equal to 10 mm.
Two separate training procedures were carried out using the
most discriminating feature subset found in the previous
experiments. AUC values and p-value were also calculated.
II.J. Statistical analysis
The bootstrap technique was used in order to estimate the
confidence interval of AUC and to compare the classification
performances of the different selected features subsets.53 To
1711 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1711
Medical Physics, Vol. 39, No. 4, April 2012
obtain the statistics of AUC values, 200 sets of malignant
and benign lesions were formed by sampling with replace-
ment from the whole dataset. The ROC and AUC for each
set was then calculated with a tenfold cross-validation strat-
egy and the 95% confidence interval for AUC was derived
from this collection of measurements.
A Wilcoxon matched pairs one-tailed test was also per-
formed to determine the significance level of the perform-
ance improvement, evaluating the p-value between each
features subsets selected in the experiments with separated
and paired classes of features and the feature subset obtained
using all classes of features, under the null hypothesis that
there exists no AUC value difference between them against
the alternative hypothesis that the performance of combined
morphologic, kinetic, and spatiotemporal features is better
than the performance of the other features subsets. (p-values
were considered significant when lower than 0.05). Wilko-
nox test was chosen as statistical test since the assumption of
normality was not valid on the distribution of AUC values.
III. RESULTS
Figure 6 shows the ROC curves related to the feature sub-
sets selected in separated genetic searches for each class of
features (morphological, kinetic, and spatiotemporal) and to
the features subset selected by the genetic search using all
three classes of features. Each figure contains mean ROC
curves.
The Table III reports the selected feature subsets and the
mean and standard deviation of the AUC values calculated
with 200 bootstrap replicates, as described above. Table III,
also reports the performances of feature subsets searched in
the three possible pairing the features classes. The AUC values
for the selected subsets associated to spatiotemporal FS1, ki-
netic FS2, and morphological FS3 features are (0.86 6 0.06),
(0.87 6 0.06), and (0.90 6 0.04), respectively. Similar per-
formances were found for feature subsets FS4, FS5, and FS6
obtained from pairing the classes of features. The largest AUC
(0.96 6 0.02) was obtained with the FS7 resulting from the
combined use of all classes of features. This AUC was signifi-
cantly higher (p¼ 0.0127) than those obtained with all other
selected FSs. Table IV shows performances for each selected
feature subset in terms of sensitivity, specificity, positive pre-
dictive value (PPV), negative predictive value (NPV) and
accuracy.
The AUC values for the small lesion and the large lesion
groups were 0.92 6 0.05 and 0.96 6 0.04, respectively
(p < 0.001). Table V reports performances in terms of sen-
sitivity, specificity, positive predictive value, negative pre-
dictive value, and accuracy.
IV. DISCUSSION
We developed and analyzed an automatic CADx system to
discriminate malignant from benign breast mass-like lesions
at DCE-MRI. The system is based on the combination of three
groups of features (morphological, kinetic, and spatiotempo-
ral), each of them working on different properties of breast
lesions. FSs were selected from each single group and from
their combinations. The selected feature subsets were used to
evaluate the diagnostic performance in distinguishing between
benign and malignant lesions by ROC analysis. The AUC for
the FS resulting from the combination of all three feature
groups, AUC(FS7), was significantly higher than those
obtained with all other selected FSs, showing that the combi-
nation of features increases the classification performances.
The morphological feature group resulted the most discrimi-
nating (AUC¼ 0.90), followed by the kinetic (AUC¼ 0.87)
and spatiotemporal feature groups (AUC¼ 0.86). This result
could be considered as a CADx prevalence of morphologic fea-
tures in comparison with kinetic features and in some way cor-
responds to clinical practice: a spiculated lesions should be
evaluated with needle-biopsy, independently from kinetics.19
Nevertheless, the combination of the spatial-temporal features
with kinetic ones or with morphological and kinetic features
resulted in a high classification performances (respectively,
AUC¼ 0.93 and 0.96), suggesting that they provide an inde-
pendent information to the other two features classes.
From the entire pool of 28 features, 16 features were
selected in the 7 classification experiments. Among these 16
features, we focused on 7 features that were chosen more
than once in the selected FSs, thus being more significant.
This small set of features was composed of four morphologi-
cal, one kinetic, and two spatiotemporal features. Morpho-
logical features are mean of angles between surface normals,
mean(ABSN), standard deviation of angles between surface
normals, std(ABSN), standard deviation of shape index,
std(SI), and peripheral uptake. We can say that these features
describe lesion margin and internal enhancement pattern.
The selected kinetic feature is the mean of voxels decay rates
of the analytical function [Eq. (9)] mean(D). They character-
ize the lesion contrast washout. These morphological and ki-
netic features should be considered as the main lesion
properties used by radiologists for identifying malignant
tumors, in agreement with clinical observations28 and
FIG. 6. ROC curves associated to the feature subsets selected in separated
genetic searches for each class of features [morphological (dashed), kinetic
(dotted), and spatiotemporal (dashed–dotted)] and to the features subset
selected by the genetic search using all three classes of features (solid).
1712 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1712
Medical Physics, Vol. 39, No. 4, April 2012
predictive models.29 The selected spatiotemporal features
are the mean of mean eigenvalue-trace image, mean(-
MeanTr), and the standard deviation of mean eigenvalue-
trace image std (MeanTr). Qualitatively, these features
describe whether the signal intensity flows in or out of the
voxels, on average over time. The selection of these features
is reasonable, since they provide an independent information
to that provided by morphological and kinetic features.
Lesions larger or equal to 10 mm were classified signifi-
cantly better (AUC¼ 0.96) than those smaller than 10 mm
(AUC¼ 0.92). Notably, morphological features showed a
lower ability to discriminate between malignant and benign
findings in the case of smaller lesions. This can be due to the
low spatial resolution used in the MRI clinical studies.
Patient motion during acquisitions of different MR data-
sets may introduce inaccuracies in the discrimination of
lesion properties. To this end, a nonrigid image coregistra-
tion was applied to dynamic series to correct for such motion
before lesion segmentation and discrimination.36
Different contrast materials were used in the two acquisi-
tion protocols. This could have determined variations in
lesion kinetics and introduced potential misinterpretations in
quantitative evaluation of kinetic and morphologic lesion
properties. In order to avoid this misleading effect, kinetic
curves were normalized by the mean values intensity meas-
ured at the mammary arteries, and morphological descriptors
were designed to be robust to variations of acquisition pa-
rameters and contrast material.
A pool of features produces a feature space to be searched
for optimal feature subsets. The larger the number of initial
features, the larger is the feature space to be spanned and the
larger can be the number of selected feature subsets. In this
work, a genetic algorithm was used to select feature subsets,
in order to prevent unnecessary computation, overfitting, and
to ensure a reliable classifier. The genetic algorithm was
driven by the fitness function in order to search for perform-
ing feature subsets with a reasonable number of features.
Indeed, the selected feature subsets were composed of a lim-
ited number of features that ranged from 3 to 5.
The total number of features was limited to 28 to avoid
potential overfitting of the small dataset. Moreover, the three
groups of features were limited to have a similar number of
features to prevent class overweighing and potential unbal-
anced comparison among feature classes. Indeed, the
obtained selected feature subsets were composed of a bal-
anced number of features of the different feature classes,
e.g., FS4, FS6, and FS7. Finally, the performances obtained
were evaluated with a stratified tenfold cross-validation
method that prevents optimistically biased evaluations due
to overfitting.54
Three groups of features, morphological, kinetic, and spa-
tiotemporal, were used and combined to select a classifier.
These groups of features are composed of features both orig-
inal and already reported in literature, with the aim of trying
a different approach. Specifically, morphological circularity,
convexity, irregularity, and shape index were used to
describe breast lesions in DCE-MRI in Refs. 20, 33, and 55
Shape index is a well known measure which was used in
many applications, and in this work is first introduced as
quantitative descriptor for the internal enhancement pattern.
ABSN was originally proposed in Ref. 40 for surface inspec-
tion on 3D binary images and is adapted in this work for
grayscale images to quantify lesion border irregularity.
ABSN gives similar information to the gradient histogram
proposed in Ref. 20 which, however, characterizes both mar-
gin and shape. To the authors’ knowledge, peripherals
uptake coefficients are first introduced in this work as quanti-
tative descriptors for the rim enhancement sign. Similarly
TABLE III. Mean and standard deviation of AUC for each selected feature subset.
Classes of features Selected feature subset (FS) AUC (mean 6 std)
Spatiotemporal FS1[mean(MeanTr), std(MeanTr), mean(StdTr)] 0.86 6 0.06
Kinetic FS2[mean(D), entropy(D), entropy(A)] 0.87 6 0.06
Morphological FS3[mean(ABSN), std(ABSN), peripheralUptake2] 0.90 6 0.04
Spatiotemporalþ kinetic FS4[mean(D), mean(A), std(AUCEC), std(MeanTr), entropy(StdTr)] 0.93 6 0.04
Spatiotemporalþmorphological FS5[mean(ABSN), std(ABSN), peripheralUptake2] 0.90 6 0.04
Kineticþmorphological FS6[mean(ABSN), std(SI), mean(D), mean(AUCEC)] 0.94 6 0.03
Spatiotemporalþ kineticþmorphological FS7[mean(ABSN), std(SI), mean(D), mean(MeanTr), entropy(Range)] 0.96 6 0.02
TABLE IV. Performances at the highest accuracy for each selected feature subset.
Classes of features
Feature
subset
Sensitivity
(mean 6 std)
Specificity
(mean 6 std)
PPV
(mean 6 std)
NPV
(mean 6 std)
Accuracy
(mean 6 std)
Spatiotemporal FS1 0.82 6 0.08 0.82 6 0.0 0.64 6 0.12 0.93 6 0.03 0.82 6 0.06
Kinetic FS2 0.83 6 0.08 0.85 6 0.09 0.68 6 0.13 0.93 6 0.3 0.84 6 0.07
Morphological FS3 0.9 6 0.06 0.87 6 0.05 0.71 6 0.08 0.96 6 0.02 0.88 6 0.04
Spatiotemporalþ kinetic features FS4 0.89 6 0.05 0.91 6 0.06 0.80 6 0.12 0.96 6 0.02 0.91 6 0.05
Spatiotemporalþmorphological FS5 0.9 6 0.06 0.87 6 0.05 0.71 6 0.08 0.96 6 0.02 0.88 6 0.04
Kineticþmorphological FS6 0.92 6 0.06 0.9 6 0.05 0.77 6 0.09 0.97 6 0.02 0.90 6 0.04
Spatiotemporalþ kineticþmorphological FS7 0.92 6 0.04 0.91 6 0.04 0.8 6 0.08 0.97 6 0.02 0.92 6 0.03
1713 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1713
Medical Physics, Vol. 39, No. 4, April 2012
for kinetic features, the area under the contrast enhancement
curve (AUCEC) and its initial part were used in kinetic anal-
ysis of breast lesion in Ref. 56, whereas uptake and washout
coefficients of the analytical function reported in Eq. (9) are
originally proposed in this work. The enhancement kinetic
featured were calculated in the whole lesion volume rather
than in a lesion hot spot. The measure of these features for a
hot spot at subjective evaluation has been demonstrated to
have a high diagnostic value for mass lesion in reports with
only ROI-based manual lesion analysis, so that the worst
scenario approach was used. However, this generally
resulted in a high sensitivity but a suboptimal specificity, not
considering the entire pattern of lesion enhancement.
Finally, all spatiotemporal features are first suggested in
this work. The developed spatiotemporal features although
were originally inspired by the internal enhancement BI-
RADS descriptors, they actually widely extend them (higher
complexity, quantitative, full use of all time frames,…), and
they can not be closely related to the BI-RADS any more.
The proposed spatiotemporal features are aimed to extract
information from the change over time of the tumor contrast
pattern for the purposes of a CADx system, they are not
designed to be used as a separate diagnostic tool.
The entire CADx system from segmentation to discrimi-
nation is fully automatic, increasing the objectivity of breast
MR image interpretation, since the interobserver variability
in lesion outlining and discrimination is avoided. Recently,
in literature, similar automatic systems, which combine dif-
ferent classes of features, have been proposed with varying
performances.32–35 Notably, these studies were mainly
focused on the development of spatiotemporal features. In
particular, Woods et al. computed a four-dimensional co-
occurrence matrix to calculate texture features in a pixel-
wise fashion, Zheng et al. used 2D discrete Fourier transfor-
mation (DFT) and Hu’s moment invariants computed on a
selected set of images, Lee et al. employed singular value
decomposition and 3D moment descriptors, Agner et al.studied kinetic texture by gradient filters and co-occurrence
Haralick’s features. In the present work, we focus on the
combination of three classes of features by genetic search
and to the development of some new features for each class
of feature. Considering the spatiotemporal features, we pro-
pose a different approach based on new dedicated maps,
instead of using kinetic parametric maps as in Zheng et al.,33
Lee et al.,34 and Agner et al.35 The performances obtained
are comparable to those reported in these previous studies.
There are some limitations in this study. First, the dataset
is composed of a limited number of lesions. This can pro-
duce an overfitting during training and limit the evaluation
of performances. This problem was addressed by the use of
limited number of features and a stratified tenfold cross-
validation. Second, the number of malignant lesions is larger
than that of benign lesions, leading to a possible bias in the
discrimination of malignancy. This effect was limited by
presenting at training the same number of malignant and be-
nign lesions. The benign minority class was filled in of cop-
ies of benign lesions belonging to the N� 1 folds used for
training until to reach the same number of malignant lesions
used for training. Nevertheless, the benign class is only par-
tially described in the feature space because of the limited
number of available lesions. A validation test is needed with
larger number of cases. Third, this study analyzed only
mass-like lesions. Detection, segmentation, and characteriza-
tion of lesion presenting nonmass-like enhancement by auto-
matic software is a different challenge due to different
morphology and low diagnostic value of dynamics for these
lesions, especially considering the relatively high probability
of DCIS with continuous increase.28,31
In conclusion, we showed that automatic combinations of
morphological, kinetic, and spatiotemporal features of mass-
like lesion at breast MRI allow for a better diagnostic per-
formance of each individual group of features. For accurate
diagnosis, an effective feature subset selection based on a
genetic algorithm was applied with stratified cross-validation
in order to select optimal subset of features. Our proposed
CADx framework has a potential for improving diagnostic
performance in breast DCE-MRI. A further analysis has to
be carried to validated these conclusions on a larger dataset.
ACKNOWLEDGMENTS
S. Agliozzo and A. Bert are researchers at im3D. L. A.
Carbonaro and D. Regge are research consultants for im3D.
F. Sardanelli has received research grants from and is on the
speakers’ bureau for Bracco Group and Bayer Pharma AG.
All other authors have nothing to disclose.
a)Author to whom correspondence should be addressed. Electronic mail:
[email protected]. Jemal et al., “Cancer statistics, 2007,” Ca-Cancer J. Clin. 57, 43–66
(2007).2J. Ferlay, D. M. Parkin, and E. Steliarova-Foucher, “Estimates of cancer
incidence and mortality in Europe in 2008,” Eur. J. Cancer 46, 765–781
(2010).3S. A. Feig et al., “American College of Radiology guidelines for breast
cancer screening,” AJR, Am. J. Roentgenol. 171, 29–33 (1998).4B. Cady and J. S. Michaelson, “The life-sparing potential of mammo-
graphic screening,” Cancer 91 1699–1703 (2001).5D. Saslow et al., “American Cancer Society guidelines for breast screen-
ing with MRI as an adjunct to mammography,” Ca-Cancer J. Clin. 57,
75–89 (2007).6C. K. Kuhl et al., “Prospective multicenter cohort study to refine manage-
ment reccomandations for woman at elevated familial risk of breast can-
cer: The Eva Trial,” J. Clin. Oncol. 20, 1450–1457 (2010).
TABLE V. Performances at the optimal cut off corresponding to the highest accuracy for lesions having different sizes.
Lesion size
Sensitivity
(mean 6 std)
Specificity
(mean 6 std)
PPV
(mean 6 std)
NPV
(mean 6 std)
Accuracy
(mean 6 std)
Size < 10 mm 0.89 6 0.07 0.88 6 0.08 0.78 6 0.16 0.94 6 0.04 0.88 6 0.05
Size > 10 mm 0.93 6 0.06 0.92 6 0.06 0.81 6 0.1 0.98 6 0.04 0.93 6 0.05
1714 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1714
Medical Physics, Vol. 39, No. 4, April 2012
7F. Sardanelli et al, “Multicenter surveillance of woman at high genetic
breast risk using mammography, ultrasonography, and contrast-enhanced
magnetic resonance imaging (the High Breast Cancer Risk Italian 1 Studt):
Final results,” Invest. Radiol. 46, 94–105 (2011).8F. Sardanelli et al., “Magnetic resonance imaging of the breast: Recco-
mendations from the EUSOMA working group,” Eur. J. Cancer 46,
1296–1316 (2010).9G. F. Tillman, S. G. Orel, and M. D. Schnall, “Effect of breast magnetic
resonance imaging on the clinical management of women with early-stage
breast carcinoma,” J. Clin. Oncol. 20, 3413–3423 (2002).10C. K. Kuhl, S. Schrading, and C. C. Leutner “Mammography, breast ultra-
sound, and magnetic resonance imaging for surveillance of women at high
familial risk for breast cancer,” J. Clin. Oncol. 23 8469–8476 (2005).11K. Schelfout et al., “Contrast-enhanced MR imaging of breast lesions and
effect on treatment,” Eur. J. Surg. Oncol. 30, 501–507 (2004).12L. W. Bassett et al., “National trends and practices in breast MRI,” Am. J.
Roentgenol. 191, 332–339 (2008).13C. K. Kuhl et al. “MRI for diagnosis of pure ductal carcinoma in situ: A
prospective observational study,” Lancet 370, 485–492 (2007).14N. Peters et al., “Meta-analysis of MR imaging in the diagnosis of breast
lesions,” Radiology 246, 116–124 (2008).15M. D. Dorrius et al., “Computer-aided detection in breast MRI: A system-
atic review and meta-analysis,” Eur. Radiol. 3, 1449–1460 (2011).16S. J. Kim et al., “Observer variability and applicability of BI-RADS termi-
nology for breast MR imaging: Invasive carcinomas as focal masses,”
Am. J. Roentgenol. 177, 551–557 (2001).17D. M. Ikeda et al., “Development, standardization, and testing of a lexicon
for reporting contrast-enhanced breast magnetic resonance imaging,” J.
Magn. Reson. Imaging 13, 889–895 (2001).18J. Stoutjesdijk et al., “Variability in the description of morphologic and
contrast enhancement characteristics of breast lesions on magnetic reso-
nance imaging,” Invest. Radiol. 40, 355–362 (2005).19Breast imaging reporting and data system (BI-RADS),” www.arc.org:
American College of Radiology, 2006.20K. G. Gilhuijs, M. L. Giger, and U. Bick. “Computerized analysis of breast
lesions in three dimensions using dynamic magnetic-resonance imaging,”
Med. Phys. 9, 1647–1654 (1998).21L. Arbach, A. Stolpen, and J. M. Reinhardt “Classification of breast MRI
lesions using a backpropagation neural network (BNN),” IEEE Interna-tional Symposium on Biomedical Imaging: Macro to Nano (IEEE, Arling-
ton, VA, 2004), Vol. 1, pp. 253–256.22W. Chen et al., “Computerized interpretation of breast MRI: Investigation
of enhancement-variance dynamics,” Med. Phys. 31, 1076–1082 (2004).23B. K. Szabo et al. “Neural network apporach to the segmentation and clas-
sification of dynamic magnetic resonance images of the breast: Compari-
son with empiric and quantitative kinetic parameters,” Acad. Radiol. 11,
1344–1354 (2004).24T. W. Nattkemper et al., “Evaluation of radiological features for breast
tumour classification in clinical screening with machine learning meth-
ods,” Artif. Intell. Med. 34, 129–139 (2005).25W. Chen et al., “Automatic identification and classification of characteris-
tic kinetic curves of breast lesions on DCE-MRI,” Med. Phys. 33,
2878–2887 (2006).26L. A. Meinel et al., “Breast MRI lesion classification: Improved perform-
ance of human readers with a backpropagation neural network computer-
aided diagnosis CAD system,” J. Magn. Reson. Imaging 25, 89–95 (2007).27D. Newell et al., “Selection of diagnostic features on breast MRI to differ-
entiate between malignant and benign lesions using computer-aided diag-
nosis: Differences in lesions presenting as mass and non-mass-like
enhancement,” Eur. Radiol. 20, 771–781 (2010).28M. D. Schnall et al., “Diagnostic architectural and dynamic features at
breast mr imaging: Multicenter study,” Radiology 238, 42–53 (2006).29W. B. DeMartini et al., “Probability of malignancy for lesions detected on
breast MRI: A predictive model incorporating BI-RADS imaging features
and patient characteristics,” Eur. Radiol. 21, 1609–1617 (2011)30C. K. Kuhl “The current status of breast MR imaging,” Radiology 244,
356–378 (2007).
31C. K. Kuhl et al., “Dynamic breast MR imaging: Are signal intensity time
course data useful for differential diagnosis of enhancing lesions,” Radiol-
ogy 211, 101–110 (1999).32B. J. Woods et al., “Malignant-lesion segmentation using 4D co-
occurrence texture analysis applied to dynamic contast-enhanced magnetic
resonance breast image data,” J. Magn. Reson. Imaging 25, 495–501
(2007).33Y. Zheng et al., “STEP: Spatiotemporal enhancement pattern for MR-
based breast tumor diagnosis,” Med. Phys. 36, 3192–3204 (2009).34S. H. Lee et al., “Multilevel analysis of spatiotemporal association features
for differentiation of tumor enhancement patterns in breast DCE-MRI,”
Med. Phys. 37, 3940–3956 (2010).35S. C. Agner et al., “Textural kinetics: A novel dynamic contrast-enhanced
(DCE)-MRI feature for breast lesion classification,” J. Digit. Imaging.
24(3), 446–463 (2010).36A. Vignati, V. Giannini, and A. Bert, “A fully automatic lesion detection
method for DCE-MRI fat-suppressed breast images,” Proc. SPIE 7260,
726026 (2009).37A. Vignati, V. Giannini, and L. Morra, “Performance of a fully automatic
lesion detection system for breast DCE-MRI,” J. Magn. Reson. Imaging
34(6), 1341–1351 (2011).38Y. Sato et al., “Three-dimensional multi-scale line filter for segmentation
and visualization of curvilinear structures in medical images,” Med. Image
Anal. 2, 143–168 (1998)39C. B. Barber, D. P. Dobkin, and H. T. Huhdanpaa, “The Quickhull
algorithm for convex hulls,” ACM Trans. Math. Softw., 22, 469–483
(1996).40T. Zhang and G. Nagy, “Surface turtuosity and its application to analyzing
cracks in concrete,” Proceedings of the IAPR International Conference onPattern Recognition (Elsevier, 2004), pp. 851–854.
41J. Koenderink, Solid Shape (MIT, Cambridge, MA, 1990).42L. D. Buadu et al., “Breast lesions: Correlation of contrast medium
enhancement patterns in MR images with histopathologic findings and tu-
mor angiogenesis,” Radiology 200, 639–649 (1996).43H. Sherif, “Peripheral washout sign on contrast-enhanced MR images of
the breast,” Radiology 205, 209–213 (1997).44U. Hoffmann et al., “Pharmacokinetic mapping of the breast: A new
method for dynamic MR mammography,” Magn. Reson. Med. 33,
506–514 (1995)45P. Tofts, “Modelling tracer kinetics in dynamic Gd-DTPA MR imaging,”
J. Magn. Res. Imaging 7, 91–101 (1997)46E. Henderson, B. K. Rutt, and T. Y. Lee, “Temporal sampling require-
ments for the tracer kinetics modeling of breast disease,” Magn. Reson.
Imaging 16, 1057–1073 (1998).47J. L. Helman and L. Hesselink, “Visualizing vector field topology in fluid
flows,” IEEE Comput. Graphics. Appl. 11, 36–46 (1991).48R. Kohavi and G. John, “Wrappers for feature subset selection,” Artif.
Intell. 97, 273–324 (1997).49L. J. Eshelman, “The CHC Adaptive Search Algorithm,” Foundations of
Genetic Algorithms (Ed San Mateo, CA 1991), pp. 265–283.50B. E. Boser, I. Guyon, and V. Vapnik, “A training algorithm for optimal
margin classifiers,” 5th Annual ACM Workshop Computational LearningTheory (Pittsburgh, PA, 1992), pp. 144–152.
51D. Nai-Yang and T. Ying-Jie, New Method in Data miing: Support VectorMachine (Science, Beijing, 2004).
52A. Shigeo, Support Vector Machines For Pattern Classification (Springer,
New York, 2005).53S. Gefen et al., “ROC analysis of ultrasound tissue characterization classi-
fiers for breast cancer diagnosis,” IEEE Trans. Med. Imaging 22, 170–177
(2003).54R. Kohavi, “A study of cross-validation and bootstrap for accuracy estima-
tion and model selection,” IJCAI, 1995.55N. Bhooshan et al., “Cancerous breast lesions on dynamic contrast-
enhanced MR images,” Radiology 254, 680–690 (2010).56J. A. Jesberger et al., “Model-free parameters from dynamic contrast
enhanced-MRI: Sensitivity to EES volume fraction and bolus timing,” J.
Magn. Res. Imaging, 24, 586–594 (2006).
1715 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1715
Medical Physics, Vol. 39, No. 4, April 2012