+ All Categories
Home > Documents > Trainable Immunohistochemical HER2/ neu Image Analysis

Trainable Immunohistochemical HER2/ neu Image Analysis

Date post: 25-Nov-2023
Category:
Upload: mayoclinic
View: 0 times
Download: 0 times
Share this document with a friend
7
Trainable Immunohistochemical HER2/neu Image Analysis A Multisite Performance Study Using 260 Breast Tissue Specimens Aziza Nassar, MD, MPH; Cynthia Cohen, MD; Sally S. Agersborg, MD, PhD; Weidong Zhou, MD; Kathleen A. Lynch, MD; Maher Albitar, MD; Edward A. Barker, MD; Burton L. Vanderbilt, MD; Jim Thompson, PhD; Eugene R. Heyman, PhD; Holger Lange, PhD; Allen Olson, PhD; Momin T. Siddiqui, MD N Context.—Aperio Technologies, Inc (Vista, California) provides a new immunohistochemistry (IHC) HER2 Image Analysis (IA) system that allows tuning of the intensity thresholds of the HER2/neu scoring scheme to adapt to the staining characteristics of different reagents. Objective.—To compare the trainable IHC HER2 IA system for different reagents to conventional manual microscopy (MM) in a multisite study. Design.—Two hundred sixty formalin-fixed, paraffin- embedded breast cancer specimens from 3 clinical sites were assayed: 180 specimens stained with Dako’s Her- cepTest (Carpinteria, California), and 80 specimens stained with Ventana’s PATHWAY HER-2/neu (Tucson, Califor- nia). At each site, 3 pathologists performed a blinded reading of the glass slides with the use of a light microscope. The glass slides were then scanned and after a wash-out period and randomization, the same patholo- gists outlined a representative set of tumor regions to be analyzed by IHC HER2 IA. Each of the methods, MM and IA, was evaluated separately and comparatively by using k statistics of negative HER2/neu scores (0, 1+) versus equivocal HER2/neu scores (2+) versus positive HER2/ neu scores (3+) among the different pathologists. Results.—k Values for IA and MM were obtained across all sites. MM: 0.565–0.864; IA: 0.895–0.947; MM versus IA: 0.683–0.892 for site 1; MM: 0.771–0.837; IA: 0.726– 0.917; MM versus IA: 0.687–0.877 for site 2; MM: 0.463– 0.674; IA: 0.864–0.918; MM versus IA: 0.497–0.626 for site 3. Conclusion.—Aperio’s trainable IHC HER2 IA system shows substantial equivalence to MM for Dako’s HercepT- est and Ventana’s PATHWAY HER-2/neu at 3 clinical sites. Image analysis improved interpathologist agreement in the different clinical sites. (Arch Pathol Lab Med. 2011;135:896–902) H ER2/neu is a proto-oncogene located on the long arm of human chromosome 17 (17q11.2–q12). 1–3 It is amplified in 15% to 20% of breast cancers, leading to increased expression of the protein gene product. 4,5 In 90% to 95% of tumors, HER2/neu protein expression is the result of gene amplification detectable by fluorescence in situ hybridization (FISH). 2,6 HER2/neu overexpression in the absence of amplification is rare (3%). 7 HER2/neu status in the primary breast carcinoma is usually maintained in metastatic lesions. 8 Overexpression of HER2/neu is associated with poor prognosis, response to trastuzumab treatment, resistance to several chemotherapeutic agents, early recurrence, and decreased disease-free and overall survival in lymph node–positive patients. 2 In addition to its prognostic role, HER2/neu has now become more important as a predictive marker of treatment response to trastuzumab, a human murine monoclonal antibody to the HER2/neu protein. 9 In 1998, trastuzumab (marketed as Herceptin, Genentech Inc, South San Francisco, California) was approved for the targeted therapy of patients with HER2/neu-overexpressing metastatic breast cancer by the US Food and Drug Administration (FDA). It has also recently been shown to be very effective in the adjuvant setting. 2,9 The 2 methodologies in current clinical use to assess HER2/neu status in breast cancer are FISH to evaluate HER2/neu gene amplification and immunohistochemistry (IHC) to detect protein overexpression. 1 According to the College of American Pathologists (CAP) and American Accepted for publication October 4, 2010. From the Department of Pathology and Laboratory Medicine, Emory University School of Medicine, Atlanta, Georgia (Drs Nassar and Cohen); the Hematology Oncology Department (Drs Agersborg, Zhou, and Lynch) and Hematology Department (Dr Albitar), Quest Diagnos- tics Nichols Institute, San Juan Capistrano, California; the Pathology Department, Medical Laboratory Associates, Seattle, Washington (Dr Barker); the Pathology Department, Vanderbilt Medical Services PC, Bellingham, Washington (Dr Vanderbilt); the Pathology Department, Pathogenesys LLC, San Juan Capistrano, California (Dr Thompson); the Department of Health Sciences, Biostatistics, University of Maryland, College Park (Dr Heyman); Aperio, Vista, California (Drs Lange and Olson); and the Department of Pathology and Laboratory Medicine, Emory University Hospital, Atlanta, Georgia (Dr Siddiqui). Dr Vander- bilt is now with the Department of Pathology, Bartlett Regional Hospital, Juneau, Alaska. Dr Nassar is now with the Department of Pathology, Mayo Clinic, Rochester, Minnesota. All pathologists (Drs Nassar, Cohen, Siddiqui, Agersborg, Zhou, Lynch, Barker, Vanderbilt, Albitar, and Thompson) who participated in the study received a monetary fund of $3000 and a laptop computer for implementing the research. All pathologists were paid by Aperio Inc. Dr Heyman, the biostatistician, was paid on an hourly basis as a biostatistics consultant. Drs Olson and Lange are employees of Aperio. Reprints: Aziza Nassar, MD, Department of Laboratory Medicine and Pathology, Mayo Clinic, 200 First Street SW, Rochester, MN 55905 (e-mail: [email protected]). 896 Arch Pathol Lab Med—Vol 135, July 2011 Aperio and HER2 Image Analysis—Nassar et al
Transcript

Trainable Immunohistochemical HER2/neu Image Analysis

A Multisite Performance Study Using 260 Breast Tissue Specimens

Aziza Nassar, MD, MPH; Cynthia Cohen, MD; Sally S. Agersborg, MD, PhD; Weidong Zhou, MD; Kathleen A. Lynch, MD;Maher Albitar, MD; Edward A. Barker, MD; Burton L. Vanderbilt, MD; Jim Thompson, PhD; Eugene R. Heyman, PhD;

Holger Lange, PhD; Allen Olson, PhD; Momin T. Siddiqui, MD

N Context.—Aperio Technologies, Inc (Vista, California)provides a new immunohistochemistry (IHC) HER2 ImageAnalysis (IA) system that allows tuning of the intensitythresholds of the HER2/neu scoring scheme to adapt to thestaining characteristics of different reagents.

Objective.—To compare the trainable IHC HER2 IAsystem for different reagents to conventional manualmicroscopy (MM) in a multisite study.

Design.—Two hundred sixty formalin-fixed, paraffin-embedded breast cancer specimens from 3 clinical siteswere assayed: 180 specimens stained with Dako’s Her-cepTest (Carpinteria, California), and 80 specimens stainedwith Ventana’s PATHWAY HER-2/neu (Tucson, Califor-nia). At each site, 3 pathologists performed a blindedreading of the glass slides with the use of a lightmicroscope. The glass slides were then scanned and aftera wash-out period and randomization, the same patholo-gists outlined a representative set of tumor regions to be

analyzed by IHC HER2 IA. Each of the methods, MM andIA, was evaluated separately and comparatively by using kstatistics of negative HER2/neu scores (0, 1+) versusequivocal HER2/neu scores (2+) versus positive HER2/neu scores (3+) among the different pathologists.

Results.—k Values for IA and MM were obtained acrossall sites. MM: 0.565–0.864; IA: 0.895–0.947; MM versusIA: 0.683–0.892 for site 1; MM: 0.771–0.837; IA: 0.726–0.917; MM versus IA: 0.687–0.877 for site 2; MM: 0.463–0.674; IA: 0.864–0.918; MM versus IA: 0.497–0.626 forsite 3.

Conclusion.—Aperio’s trainable IHC HER2 IA systemshows substantial equivalence to MM for Dako’s HercepT-est and Ventana’s PATHWAY HER-2/neu at 3 clinical sites.Image analysis improved interpathologist agreement in thedifferent clinical sites.

(Arch Pathol Lab Med. 2011;135:896–902)

HER2/neu is a proto-oncogene located on the long armof human chromosome 17 (17q11.2–q12).1–3 It is

amplified in 15% to 20% of breast cancers, leading to

increased expression of the protein gene product.4,5 In 90%to 95% of tumors, HER2/neu protein expression is theresult of gene amplification detectable by fluorescence insitu hybridization (FISH).2,6 HER2/neu overexpression inthe absence of amplification is rare (3%).7 HER2/neu statusin the primary breast carcinoma is usually maintained inmetastatic lesions.8

Overexpression of HER2/neu is associated with poorprognosis, response to trastuzumab treatment, resistanceto several chemotherapeutic agents, early recurrence, anddecreased disease-free and overall survival in lymphnode–positive patients.2 In addition to its prognostic role,HER2/neu has now become more important as apredictive marker of treatment response to trastuzumab,a human murine monoclonal antibody to the HER2/neuprotein.9 In 1998, trastuzumab (marketed as Herceptin,Genentech Inc, South San Francisco, California) wasapproved for the targeted therapy of patients withHER2/neu-overexpressing metastatic breast cancer bythe US Food and Drug Administration (FDA). It has alsorecently been shown to be very effective in the adjuvantsetting.2,9

The 2 methodologies in current clinical use to assessHER2/neu status in breast cancer are FISH to evaluateHER2/neu gene amplification and immunohistochemistry(IHC) to detect protein overexpression.1 According to theCollege of American Pathologists (CAP) and American

Accepted for publication October 4, 2010.From the Department of Pathology and Laboratory Medicine, Emory

University School of Medicine, Atlanta, Georgia (Drs Nassar andCohen); the Hematology Oncology Department (Drs Agersborg, Zhou,and Lynch) and Hematology Department (Dr Albitar), Quest Diagnos-tics Nichols Institute, San Juan Capistrano, California; the PathologyDepartment, Medical Laboratory Associates, Seattle, Washington (DrBarker); the Pathology Department, Vanderbilt Medical Services PC,Bellingham, Washington (Dr Vanderbilt); the Pathology Department,Pathogenesys LLC, San Juan Capistrano, California (Dr Thompson); theDepartment of Health Sciences, Biostatistics, University of Maryland,College Park (Dr Heyman); Aperio, Vista, California (Drs Lange andOlson); and the Department of Pathology and Laboratory Medicine,Emory University Hospital, Atlanta, Georgia (Dr Siddiqui). Dr Vander-bilt is now with the Department of Pathology, Bartlett Regional Hospital,Juneau, Alaska. Dr Nassar is now with the Department of Pathology,Mayo Clinic, Rochester, Minnesota.

All pathologists (Drs Nassar, Cohen, Siddiqui, Agersborg, Zhou,Lynch, Barker, Vanderbilt, Albitar, and Thompson) who participated inthe study received a monetary fund of $3000 and a laptop computer forimplementing the research. All pathologists were paid by Aperio Inc. DrHeyman, the biostatistician, was paid on an hourly basis as abiostatistics consultant. Drs Olson and Lange are employees of Aperio.

Reprints: Aziza Nassar, MD, Department of Laboratory Medicine andPathology, Mayo Clinic, 200 First Street SW, Rochester, MN 55905(e-mail: [email protected]).

896 Arch Pathol Lab Med—Vol 135, July 2011 Aperio and HER2 Image Analysis—Nassar et al

Society of Clinical Oncology (ASCO) guidelines, HER2/neu IHC 2+ is regarded as equivocal and should befollowed by testing for HER2/neu gene amplification byFISH.4 A pathologist and/or technologist visually quan-tifies FISH results. The results are thus vulnerable tosubjective interpretation, and the interpreter’s experienceplays a role in result reproducibility. Results of IHC andFISH can be influenced by duration and method of tissuefixation, decalcification, antigen retrieval method, anti-body/probe type, or by differences in the interpretation ofstaining intensity by the observer.4

Several different HER2/neu antibodies are available: theFDA-approved Dako (Dako, Glostrup, Denmark) HercepTest10 and the Ventana (Tucson, California) PATHWAYanti-HER2/neu (4B5) rabbit monoclonal antibody aremostly used and provide excellent sensitivity, specificity,and interlaboratory reproducibility.1

It has been suggested that the use of digital microscopyimproves the accuracy and interobserver reproducibilityof HER2/neu IHC analysis. Digital measurement ofstaining intensity is more accurate than measurementwith the human eye because it is not influenced by factorssuch as the ambient light, pathologist fatigue, or interob-server and intraobserver variation.3 Consistent objectiveand reproducible results of HER2/neu assessment can begenerated by a number of available automated scoringsystems such as the automated cellular imaging system(ACIS) (ChromaVision, Inc, San Juan Capistrano, Califor-nia)11,12 optimized for use with Dako HercepTest, Micro-metastasis Detection System (MDS, Applied Imaging, SanJose, California),13 Extended Slide Wizard (Tripath Imag-ing, Inc, Burlington, North Carolina), and others.14–16

To be widely accepted for clinical use, digital pathologysystems need to be cleared/approved by the FDA, andperformance studies must demonstrate this new technol-ogy to the clinical community. This study evaluates theperformance of Aperio Technologies’ (Vista, California)Digital Pathology Platform and compares it with that ofmanual microscopy (MM), while using 2 different FDA-approved antibodies, Dako HercepTest and VentanaPATHWAY anti-HER2-neu (clone CB11) mouse monoclo-nal antibody, for evaluating IHC in 3 different clinicalcenters.

MATERIALS AND METHODS

Three Clinical Laboratory Improvement Amendments of 1988–qualified clinical sites participated in the study. Before theirparticipation in the study, each clinical site obtained exemptionstatus from an institutional review board.

A multisite performance study was conducted at an academiccenter (site 1), a reference laboratory (site 2), and a private grouppractice (site 3). Aperio Technologies’ trainable IHC HER2 ImageAnalysis (IA) system was compared to conventional MM, withdifferent reagents. Each institution performed its own study,with no exchange of slides between the different centers.

Two hundred and sixty formalin-fixed, paraffin-embeddedinvasive breast carcinoma specimens from these 3 clinical siteswere assayed: 80 specimens with almost equal HER2/neu scoredistribution from clinical site 1; 100 routine specimens fromclinical site 2 all stained with Dako HercepTest; and 80 specimenswith almost equal HER2/neu score distribution from clinical site3, stained with Ventana PATHWAY HER2-neu.

The specimens at the first clinical site were selected retrospec-tively by their clinical scores on file to provide an equaldistribution of slides for the different HER2/neu scores. Thespecimens at the second clinical site were prospective routinespecimens taken from its clinical operation, representing the true

target population of cases in a typical clinical setting. Thespecimens at the third clinical site were selected by their clinicalscores on file to provide an equal distribution of slides for thedifferent HER2/neu scores.

All specimens at the first and second clinical site wereimmunohistochemically stained with Dako in vitro diagnostic,FDA-approved HerceptTest (A085). All specimens at the thirdclinical site were immunohistochemically stained with Ventanain vitro diagnostic, FDA-approved PATHWAY HER2-neu (CloneCB11).

At each site, a 20-slide representative training set with scoresfrom 3 pathologists was used to automatically train the IAalgorithm. At each site, 3 pathologists, using their microscopes,performed a blinded reading of the glass slides and reported theHER2/neu score (0, 1+, 2+, or 3+) for each slide. The glass slideswere then scanned with an 320 objective and randomized. Aftera wash-out period exceeding 1 week, the same 3 pathologistsoutlined a representative set of tumor regions to be analyzed byIA (Figure 1). The IA system reported the HER2/neu score foreach of the 3 pathologists for each of the slides. The IA itself wasrun in batch mode and in blinded fashion to avoid influencingthe pathologists in their choice of the tumor regions.

The whole process was repeated, this time with the differentclinical samples (80 specimens from each of clinical sites 1 and 3;and 100 specimens from clinical site 2). The same 3 pathologistsat each clinical site used Aperio’s ScanScope Systems’ remoteediting capability to outline a representative set of tumor regionsfor each digital slide. The pathologists were blinded to eachother’s annotations of tumor region outlines. Image Analysis wasrun for each slide, resulting in a separate IA score for each of the 3pathologists. The IA algorithm reported the HER2/neu score as 0,1+, 2+, or 3+ for each of the digital slides.

For HER2/neu tissue scoring applications, the ScanScopeAperio digital microscope locates tissues on a slide and generatesa scanned image of the entire tissue (Figure 2). At review, thepathologist views this image on a computer monitor and selectsmultiple tissue regions of appropriately stained invasive carci-noma for scoring (Figure 3). The system then generates a series ofscores for these areas, including both individual region scoresand the average score of selected regions. In HER2/neu IHCevaluation, the score is the average brown pixel intensity,determined by using a color threshold designed to measure thebrown color associated with the 3, 39-diaminobenzidine tetra-hydrochloride (DAB) stain. The specific algorithm selects thosepixels within the selected tissue regions that meet the brownDAB color threshold. This color selection chooses pixels that areassociated with extracellular domains of the plasma membrane.The ScanScope Aperio digital microscope calculates the HER2/neu IHC intensity score on a scale of 0 to 255, and also provides abinned score that relates to the HercepTest scoring system (1+,2+, and 3+). Owing to the more quantitative data of ScanScopeAperio digital microscopy, this score can be reported in tenths ofinteger units (eg, 1.3, 2.1). Cases with an average score of at least2.2 are considered to have HER2/neu protein overexpression. Aseparate/smaller set of digital slides was used to establish the cutpoints. Then, the algorithm (with the determined cut points) wasapplied independently of the study data.

Other details of the hardware and software for the ScanScopeAperio digital microscopy system are available online (http://www.aperio.com; accessed August 8, 2010).

Immunohistochemistry

From each tissue block, 4-mm sections were cut, deparaffinizedin xylene, and dehydrated through alcohol changes. For DakoHercepTest, processing was according to the manufacturers’instructions with water-bath antigen retrieval as follows: slideswere immersed in 10 mmol of citrate buffer per liter in acalibrated water bath (required temperature 95uC–99uC). Theywere then incubated for 40 (61) minutes at 95uC to 99uC. Afterdecanting the epitope retrieval solution, the sections were rinsedin the wash buffer, and later soaked in the buffer for 5 to

Arch Pathol Lab Med—Vol 135, July 2011 Aperio and HER2 Image Analysis—Nassar et al 897

Figure 1. Schematic diagram of the study design.

Figure 2. Captured image of a HER2/neu-immunostained slide by Aperio’s ImageScope(Vista, California) on a computer monitor.

898 Arch Pathol Lab Med—Vol 135, July 2011 Aperio and HER2 Image Analysis—Nassar et al

20 minutes before staining. The slides were loaded onto the Dakoautostainer using the HercepTest. In the autostainer, the slideswere rinsed, followed by 200 mL of peroxidase-blocking reagentfor 5 minutes rinsing, then 200 mL of primary anti-HER2/neuprotein (or negative control reagent) for 30 minutes, rinsed twice,and finally immersed in 200 mL of substrate-chromogen solution(DAB) for 10 minutes. The slides were then removed from theautostainer, counterstained with hematoxylin, and finally cover-slipped. Known positive, negative, and intermediate valuecontrols were run with each patient set.

For immunostaining with the Ventana Benchmark XT stainingsystem, sections were transferred to Ventana wash solution.Endogenous peroxidase activity was blocked in 3% hydrogenperoxide. Slides were then incubated with Ventana PATHWAYanti-HER2/neu (CB11) mouse monoclonal antibody at 37uC for32 minutes and developed in DAB for 10 minutes. Finally,sections were counterstained with hematoxylin and mounted.

Per ASCO/CAP guidelines,4 HER2/neu IHC protein expres-sion status was classified by applying the following criteria.

. Negative for HER2/neu protein: IHC staining of 0 or 1+, withabsence of or weak, incomplete membrane staining in anyproportion of tumor cells.

. Equivocal for HER2/neu protein: IHC staining of 2+, withcomplete membrane staining that is either nonuniform orweak in intensity but with obvious circumferential distribu-tion in at least 10% of cells.

. Positive for HER2/neu protein: IHC staining of 3+, withuniform intense chicken-wire circumferential membranestaining of more than 30% of invasive tumor cells.

. The Aperio HER2 IA scoring system was as follows: negative,below 1.8; equivocal, from 1.8 to less than 2.2; positive, greaterthan or equal to 2.2.

Statistical Analysis

The statistical analyses for all 3 sites, for each of the methods,included analysis of MM, IA, and comparative analysis betweenthe 2 methods (MM and IA).

For each of the methods, MM and IA were evaluatedseparately and comparatively by using percentage agreement(PA), with an exact 95% confidence interval (CI), of atrichotomous categorization of the HER2/neu scores thatcombined 0 and 1+ as negative cases, 2+ as equivocal, and 3+as positive cases. Simple k statistics with 95% CI were estimatedfor each agreement (interobserver and intraobserver) table,which provides the degree of agreement above that expectedby chance alone. Statistical analyses were performed with SASsoftware version 9.2 (Chicago, Illinois). The interpretation of thek statistics is as follows: below 0, less than chance agreement; 0.01to 0.20, slight agreement; 0.21 to 0.40, fair agreement; 0.41 to 0.60,moderate agreement; 0.61 to 0.80, substantial agreement; and 0.81to 0.99, almost perfect agreement.

RESULTS

Comparable PA values were obtained for MM and IAfor digital HER2/neu slides. Tables 1 through 6 show PAand k statistics along with an exact 95% CI for theclinically relevant trichotomous outcome of negative (0and 1+) versus equivocal (2+) versus positive (3+) HER2/neu scores. Each table presents the results for each of themethods—MM, IA, and comparatively between MM andIA—for the 3 different clinical sites, each with its 3different pathologists.

Percentage agreement values between pathologistsusing MM (65.0%–91.3%) (Table 1) were comparable toPA values between MM and IA (65.0%–90.0%) (Table 5),as based on the tumor regions outlined by the pathologist.The k statistics show that there is moderate to almostperfect agreement (0.481–0.832) between the differentpathologists using manual microscopy in the differentclinical sites (Table 2). The interpathologist agreement forclinical site 2 is better than that for the other clinical sites(Table 2). The PA values for IA between pathologists,based on the tumor regions outlined by the pathologists(85.0%–94.0%) (Table 3), were more or less comparable to

Figure 3. Captured image of a HER2/neu-immunostained slide on a computer monitorwith annotations by the pathologist.

Arch Pathol Lab Med—Vol 135, July 2011 Aperio and HER2 Image Analysis—Nassar et al 899

the PA values between pathologists using MM (65.0%–91.3%) (Table 1). The k statistics for interpathologistagreement improved with the use of image analysis(Table 4). In fact, the k statistics show that there issubstantial to almost perfect agreement (0.725–0.892)between the different pathologists in the 3 clinical sites

(Table 4). The interpathologist agreement improved dra-matically for clinical sites 1 and 3 (Table 4) with the use ofimage analysis.

These data show that if pathologists rely on IA resultsinstead of consulting another pathologist, IA wouldprovide them with better agreement (agreement between

Table 1. Manual Microscopy: Interpathologist Agreement

Pathologist 1 Versus 2 Pathologist 1 Versus 3 Pathologist 2 Versus 3

Percentage Agreement (95%% CI), %%

Clinical site 1 91.3 (82.8–96.4) 77.5 (66.8–86.1) 76.3 (65.4–85.1)Clinical site 2 84.0 (75.3–90.6) 82.0 (73.1–89.0) 90.0 (82.4–95.1)Clinical site 3 65.0 (53.5–75.3) 75.0 (64.1–84.0) 72.5 (61.4–81.9)

Abbreviation: CI, confidence interval.

Table 2. k Statistics for Interpathologist Agreement for Manual Microscopy

Pathologist 1 Versus 2 Pathologist 1 Versus 3 Pathologist 2 Versus 3

k (95%% CI)

Clinical site 1 0.832 (0.716–0.949) 0.638 (0.496–0.781) 0.615 (0.471–0.759)Clinical site 2 0.723 (0.603–0.842) 0.672 (0.544–0.801) 0.814 (0.705–0.923)Clinical site 3 0.481 (0.343–0.618) 0.626 (0.488–0.765) 0.570 (0.428–0.713)

Abbreviation: CI, confidence interval.

Table 3. Image Analysis: Interpathologist Agreement

Pathologist 1 Versus 2 Pathologist 1 Versus 3 Pathologist 2 Versus 3

Percentage Agreement (95%% CI), %%

Clinical site 1 91.3 (82.8–96.4) 92.5 (84.4–97.2) 88.8 (79.7–94.7)Clinical site 2 85.0 (76.5–91.4) 94.0 (87.4–97.8) 87.0 (78.8–92.9)Clinical site 3 86.3 (76.7–92.9) 87.5 (78.2–93.8) 88.8 (79.7–94.7)

Abbreviation: CI, confidence interval.

Table 4. k Statistics for Interpathologist Agreement for Image Analysis

Pathologist 1 Versus 2 Pathologist 1 Versus 3 Pathologist 2 Versus 3

k (95%% CI)

Clinical site 1 0.841 (0.736–0.947) 0.866 (0.764–0.968) 0.796 (0.676–0.915)Clinical site 2 0.725 (0.596–0.853) 0.892 (0.809–0.975) 0.761 (0.640–0.882)Clinical site 3 0.789 (0.675–0.902) 0.806 (0.695–0.917) 0.826 (0.720–0.931)

Abbreviation: CI, confidence interval.

Table 5. Manual Microscopy Versus Image Analysis: Individual Pathologist’s Agreement

Pathologist 1 Pathologist 2 Pathologist 3

Percentage Agreement (95%% CI), %%

Clinical site 1 87.5 (78.2–93.8) 87.5 (78.2–93.8) 80.0 (69.6–88.1)Clinical site 2 90.0 (82.4–95.1) 79.0 (69.7–86.5) 88.0 (80.0–93.6)Clinical site 3 78.8 (68.2–87.1) 66.3 (54.8–76.4) 65.0 (53.5–75.3)

Abbreviation: CI, confidence interval.

Table 6. k Statistics for Individual Pathologist’s Agreement for Manual Microscopy Versus Image Analysis

Pathologist 1 Pathologist 2 Pathologist 3

k (95%% CI)

Clinical site 1 0.771 (0.639–0.903) 0.763 (0.637–0.890) 0.680 (0.545–0.815)Clinical site 2 0.825 (0.725–0.926) 0.619 (0.478–0.759) 0.773 (0.658–0.889)Clinical site 3 0.677 (0.545–0.809) 0.473 (0.328–0.619) 0.472 (0.324–0.620)

Abbreviation: CI, confidence interval.

900 Arch Pathol Lab Med—Vol 135, July 2011 Aperio and HER2 Image Analysis—Nassar et al

MM and IA is in most cases higher than the agreementbetween pathologists) and a more reliable score (theagreement between IA, when used by different patholo-gists, is in most cases higher than the agreement betweenpathologists). The highest IA variations are introduced byinterpathologist variations, which still yield an excellentPA, ranging from 85.0% to 94.0% in the comparison study,in terms of the clinically relevant negative (0 and 1+)versus equivocal (2+) versus positive (3+) HER2/neuscores (Table 3).

The intrapathologist agreement was moderate to almostperfect (0.472–0.825) (Table 6).

COMMENT

Our results support the hypothesis that automationimproves interobserver IHC quantitation results ofHER2/neu when compared to MM. We demonstrate thatreading digital HER2/neu slides on a computer monitor,using Aperio Technologies’ Digital Pathology Platform,is a substantial equivalence to conventional MM andtherefore can be used as an alternative to the conventionalmicroscope. This is even the case with multiple manufac-turers’ HER2/neu IHC reagents, each expressing differentstaining characteristics, and with different pathologists.This study was done in 3 different institutions, and eachinstitution had 3 different pathologists, with variablelevels of experience for evaluating HER2/neu immuno-stain. These differences in experience have accounted forsome of the variations that are seen in the percentageagreement, specifically in Table 1. The other factor thataccounts for these variations is the different hot spot thateach pathologist chose for evaluating the staining. Thehuman eye is not perfect in detecting subtle differences inintensity, whereas the image analysis does have thiscapability.

Digital microscopy improves the accuracy and reliabil-ity of HER2/neu IHC analysis, when compared tosemiquantitative scoring by a pathologist.3 As the samestained slides were used, this most likely reflects errors inmanual interpretation and not IHC reagent limitations, asdemonstrated by our study. We have shown that IAimproved interobserver agreement between the differentpathologists. Furthermore, the variability in intrapathol-ogist agreement could be attributed to the experience levelof the pathologist, familiarity with the use of digitalizedimages, and the choice of the different fields used forannotations.

False-positive HercepTest results have been reported in12% to 23% of cases.13,17 HercepTest has been shown togive false-negative results in up to 28% of HER2/neuFISH-positive cases.13 The CAP/ASCO HER2/neu guide-lines recommend that laboratories show 95% concordancewith another validated test, such as FISH, for HER2/neuIHC negative results and 3+ staining results.4

Another automated IA system that is successful inHER2/neu testing is the automated cellular imagingsystem (ACIS) (Dako). ACIS is known to be more accuratethan visual scoring and is reported to have 91% to 95%concordance with FISH results when evaluating HER2/neu overexpression in whole tissue sections.3,11,12,18–21 UsingACIS for HER2/neu expression helps pathologists estab-lish consistency, especially among several pathologists,and provides clinicians with objective results for triagingpatients for directed trastuzumab (Herceptin) therapy,specifically following the new CAP/ASCO guidelines for

HER2/neu analysis.4 Since the evaluation of stainingintensity and percentage of cells with complete membranepositivity is subjective, the interobserver variability tendsto be higher for scoring HER2/neu 2+ cases, and fordiscriminating 1+ and 2+, or 2+ and 3+ cases.3 Thepercentage of disagreement in intraobserver reproducibil-ity ranges from 0.9% to 3.7%.3 The interobserver agree-ment is excellent for categorized variables (0, 1+ versus 2+versus 3+) between 2 pathologists (k 5 0.929; 95% CI,0.909–0.946).3

Using the ACIS system, Bloom and Harrington18 wereable to eliminate interobserver variability in HER2/neuscoring by IHC.18 They showed that 10 pathologistsscoring the same IHC slides (129 cases of invasive breastcarcinoma) with the assistance of digital microscopyimproved concordance with FISH to 93% (k 5 0.86; P ,.001), compared with 71% (k 5 0.51) for manual IHCanalysis.18 Similarly, using the ACIS system, Wang et al20

found improved concordance of IHC with FISH, as didTawfik et al12 (94% concordance between IHC-ACIS [$2.2]and FISH [$2.0]) and Luftner et al22 (88.5% concordance, k5 0.68, category ‘‘good’’). Luftner et al22 also showed‘‘very good’’ concordance (95.1%, k 5 0.85) betweenmanual interpretations and the automated IHC assay.

The use of computer-aided microscopy has beensuggested as a way to improve interobserver reproduc-ibility in immunohistochemical evaluation.23–26 Computer-aided microscopy involves the digitization of stainedtissue and the automated evaluation of immunohisto-chemistry with image analysis. This makes for a moreobjective assay than subjective manual microscopy,eliminating lack of reader experience. In the case ofHER2/neu, the CAP/ASCO guidelines recognize imageanalysis as an effective tool for achieving consistentinterpretation of immunohistochemical staining, providedthat a pathologist confirms the result.4 The latter ensuresthat invasive cancer is assessed and not benign breasttissue or ductal carcinoma in situ. Computer-aidedquantitative assessment of immunohistochemical staininghas potentially several benefits. It can provide a truecontinuous and reproducible assessment of staining. Thehuman eye has difficulty distinguishing subtle differencesin staining intensity using a continuous scale.26 Conse-quently, scoring systems tend to be nominal (ie, 0, 1+, 2+,and 3+). Studies for HER2/neu have shown that accuratedistinction between nominal categories is difficult andoften arbitrary, and this difficulty contributes to asignificant lack of reproducibility.27 Automated systemscan consistently preselect stained areas and extract a scorefrom them or point the same areas to different observers;the selection of different areas to be assessed by differentobservers has been identified as a source of interobservervariability.28 Those hot spots are identified on the digitalslides. Automated systems could be used for screening ofstrongly positive or strongly negative slides, such that theever-increasing reading load of a pathologist will berelieved from obvious cases. In addition to increasedreproducibility, automated systems have the potential toincrease prognostic accuracy by revealing differences inbiomarker expression that are not discernible to thepathologist owing to their inability to distinguish betweenfine levels of expression.26 Digital image analysis candetect subtle changes in the intensity of biomarkerexpression that are indiscernible to the human eye. Thisaccounts for the improvement that is seen in this study

Arch Pathol Lab Med—Vol 135, July 2011 Aperio and HER2 Image Analysis—Nassar et al 901

using image analysis. Those hot spots that are identifiedby the IA system are always the same despite the differentreaders using the system.

Aperio Technologies Inc provides trainable IHC HER2IA, which allows adaptation to the staining characteristicsof different reagents (Dako HercepTest, Ventana PATH-WAY) and correlation of the analysis results to a scoringstandard, and is of substantial equivalence to convention-al MM. Compared to manual microscopy, Image Analysisby Aperio ScanScope has improved the concordanceamong 3 pathologists in each of 3 different institutions.

References

1. Powell WC, Hicks DG, Prescott N, et al. A new rabbit monoclonal antibody(4B5) for the immunohistochemical (IHC) determination of the HER2 status inbreast cancer: comparison with CB11, fluorescence in situ hybridization (FISH)and interlaboratory reproducibility. Appl Immunohistochem Mol Morphol. 2007;15(1):94–102.

2. Slamon DJ, Leyland-Jones B, Shak S, et al. Use of chemotherapy plus amonoclonal antibody against HER2 for metastatic breast cancer that overex-presses HER2. N Engl J Med. 2001;344(11):783–792.

3. Turashvili G, Leung S, Turbin D, et al. Inter-observer reproducibility ofHER2 immunohistochemical assessment and concordance with fluorescent insitu hybridization (FISH): pathologist assessment compared to quantitative imageanalysis. BMC Cancer. 2009;9:165.

4. Wolff AC, Hammond ME, Schwartz JN, et al. American Society of ClinicalOncology/College of American Pathologists guideline recommendations forhuman epidermal growth factor receptor 2 testing in breast cancer. J Clin Oncol.2007;25(1):118–145.

5. Carlson RW, Moench SJ, Hammond ME, et al. HER2 testing in breastcancer: NCCN Task Force report and recommendations [quiz in J Natl ComprCanc Netw. 2006;4(suppl 3):S23–S24]. J Natl Compr Canc Netw. 2006;4(suppl3):S1–S22.

6. Pauletti G, Dandekar S, Rong H, et al. Assessment of methods for tissue-based detection of the HER-2/neu alteration in human breast cancer: a directcomparison of fluorescence in situ hybridization and immunohistochemistry. JClin Oncol. 2000;18(21):3651–3664.

7. Persons DL, Bui MM, Lowery MC, et al. Fluorescence in situ hybridization(FISH) for detection of HER-2/neu amplification in breast cancer: a multicenterportability study. Ann Clin Lab Sci. 2000;30(1):41–48.

8. Park K, Han S, Kim HJ, Kim J, Shin E. HER2 status in pure ductal carcinomain situ and in the intraductal and invasive components of invasive ductalcarcinoma determined by fluorescence in situ hybridization and immunohisto-chemistry. Histopathology. 2006;48(6):702–707.

9. Laudadio J, Quigley DI, Tubbs R, Wolff DJ. HER2 testing: a review ofdetection methodologies and their clinical performance. Expert Rev Mol Diagn.2007;7(1):53–64.

10. Rhodes A, Jasani B, Anderson E, Dodson AR, Balaton AJ. Evaluation ofHer-2/neu immunohistochemical assay sensitivity and scoring on formalin-fixedand paraffin-processed cell lines and breast tumors: a comparative studyinvolving results from laboratories in 21 countries. Am J Clin Pathol. 2002;118(93):408–417.

11. Ciampa A, Xu B, Ayata G, et al. HER-2 status in breast cancer: correlationof gene amplification by FISH with immunohistochemistry expression usingadvanced cellular imaging system. Appl Immunohistochem Mol Morphol. 2006;14(2):132–137.

12. Tawfik OW, Kimler BF, Davis M, et al. Comparison of immunohisto-chemistry by automated cellular imaging system (ACIS) versus fluorescence in-situ hybridization in the evaluation of HER2 expression in primary breastcarcinoma. Histopathology. 2006;48 (3):258–267.

13. Ellis CM, Dyson MJ, Stephenson TJ, Maltby EL. HER2 amplification statusin breast cancer: a comparison between immunohistochemical staining andfluorescence in situ hybridisation using manual and automated quantitativeimage analysis scoring techniques. J Clin Pathol. 2005;58(7):710–714.

14. Hatanaka Y, Hashizume K, Nitta K, Kato T, Itoh I, Tani Y. Cytometricalimage analysis for immunohistochemical hormone receptor status in breastcarcinomas. Pathol Int. 2003;53(10):693–699.

15. Joshi AS, Sharangpani GM, Porter K, et al. Semi-automated imaging systemto quantitate Her-2/neu membrane receptor immunoreactivity in human breastcancer. Cytometry A. 2007;71(5):273–285.

16. Skaland I, Øvestad I, Janssen EA, et al. Comparing subjective and digitalimage analysis HER2/neu expression scores with conventional and modified FISHscores in breast cancer. J Clin Pathol. 2008;61(1):68–71.

17. Lebeau A, Deimling D, Kaltz C, et al. Her-2/neu analysis in archival tissuesamples of human breast cancer: comparison of immunohistochemistry andfluorescence in situ hybridization. J Clin Oncol. 2001;19(2):354–363.

18. Bloom K, Harrington D. Enhanced accuracy and reliability of HER-2/neuimmunohistochemical scoring using digital microscopy. Am J Clin Pathol. 2004;121(5):620–630.

19. Anderson JM, Ariga R, Govil H, et al. Assessment of Her-2/Neu status byimmunohistochemistry and fluorescence in situ hybridization in mammary Pagetdisease and underlying carcinoma. Appl Immunohistochem Mol Morphol. 2003;11(2):120–124.

20. Wang S, Hossein Saboorian M, Frenkel EP, et al. Aneusomy 17 in breastcancer: its role in HER-2/neu protein expression and implication for clinicalassessment of HER-2/neu status. Mod Pathol. 2002;15(2):137–145.

21. Minot DM, Kipp BR, Root RM, et al. Automated cellular imaging system IIIfor assessing HER2 status in breast cancer specimens—development of astandardized scoring method that correlates with FISH. Am J Clin Pathol. 2009;132(1):133–138.

22. Luftner D, Henschke P, Kafka A, et al. Discordant results obtained fordifferent methods of HER-2/neu testing in breast cancer—a question ofstandardization, automation and timing. Int J Biol Markers. 2004;19(1):1–13.

23. Seidal T, Balaton AJ, Battifora H. Interpretation and quantification ofimmunostains. Am J Surg Pathol. 2001;25(9):1208–1210.

24. Braunschweig T, Chung JY, Hewitt SM. Perspectives in tissue microarrays.Combinatorial Chem High Throughput Screening. 2004;7(6):575–585.

25. Braunschweig T, Chung JY, Hewitt SM. Tissue microarrays: bridging thegap between research and the clinic. Expert Rev Proteomics. 2005;2(3):325–326.

26. Camp RL, Chung GG, Rimm DL. Automated subcellular localization andquantification of protein expression in tissue microarray. Nature Med. 2002;8(11):1323–1327.

27. Paik S, Bryant J, Tan-Chiu E, et al. Real-world performance of Her2testing—National Surgical Adjuvant Breast and Bowel Project experience. J NatCancer Inst. 2002;94(11):852–854.

28. van Diest PJ, van Dam P, Henzen-Logmans SC, et al. A scoring system forimmunohistochemical staining: consensus report of the task force for basicresearch of the EORTC-GCCG. Clin Pathol. 1997;50(10):801–804.

902 Arch Pathol Lab Med—Vol 135, July 2011 Aperio and HER2 Image Analysis—Nassar et al


Recommended