Comparative Effectiveness Review: Prostate Cancer Antigen 3Testing for the Diagnosis and Management of Prostate Cancer
Linda A. Bradley,*,† Glenn E. Palomaki,† Steven Gutman,† David Samson†and Naomi Aronson‡From the Department of Pathology and Laboratory Medicine, Women & Infants Hospital/Alpert Medical School of BrownUniversity, Providence, Rhode Island (LAB, GEP), and BlueCross BlueShield Association TEC Evidence-based PracticeCenter, Chicago, Illinois (SG, DS, NA)
Purpose: We compared the effectiveness of PCA3 (prostate cancer antigen 3) andselect comparators for improving initial or repeat biopsy decision making in menat risk for prostate cancer, or treatment choices in men with prostate cancer.Materials and Methods: MEDLINE®, EMBASE®, Cochrane Database and grayliterature were searched from January 1990 through May 2012. Included studieswere matched, and measured PCA3 and comparator(s) within a cohort. Nomatched analyses were possible. Differences in independent performance esti-mates between PCA3 and comparators were computed within studies. Studieswere assessed for quality using QUADAS (Quality Assessment of DiagnosticAccuracy Studies) and for strength of evidence using GRADE (Grading of Rec-ommendations Assessment, Development and Evaluation) criteria.Results: Among 1,556 publications identified, 34 observational studies were an-alyzed (24 addressed diagnostic accuracy and 13 addressed treatment decisions).Most studies were conducted in opportunistic cohorts of men referred for proce-dures and were not designed to answer key questions. Two study biases (partialverification and sampling) were addressed by analyses, allowing some conclu-sions to be drawn. PCA3 was more discriminatory than total prostate specificantigen increases (eg at an observed 50% specificity, summary sensitivities were77% and 57%, respectively). Analyses indicated that this finding holds for initialand repeat biopsies, and that the markers were independent predictors. For allother biopsy decision making comparisons and associated health outcomes,strength of evidence was insufficient. For treatment decision making, strength ofevidence was insufficient for all outcomes and comparators.Conclusions: PCA3 had a higher diagnostic accuracy than total prostate specificantigen increases, but strength of evidence was low (limited confidence in effectestimates). Strength of evidence was insufficient to conclude that PCA3 testingleads to improved health outcomes. For all other outcomes and comparators,strength of evidence was insufficient.
Key Words: prostatic neoplasms; prostate cancer antigen 3, human; biopsy;
Abbreviations
and Acronyms
CER � comparative effectivenessreviewcPSA � complexed prostatespecific antigenEVN � externally validatednomogramFPR � false-positive ratefPSA � free prostate specificantigenKQ � key questionPCA3 � prostate cancer antigen 3PSA � prostate specific antigenPSAD � prostate specific antigendensitySOE � strength of evidenceTEP � Technical Expert PaneltPSA � total prostate specificantigen
Accepted for publication February 1, 2013.Supported by Evidence-based Practice Center
under Contract No. 290-02-0009. AHRQ Publica-tion No. 06-E017. Rockville, MD: Agency forHealthcare Research and Quality, August 2006.Available at www.effectivehealthcare.ahrq.gov/reports/final.cfm.
* Correspondence: Women & Infants Hospi-tal, Division of Medical Screening and SpecialTesting, 70 Elm St., 2nd Floor, Providence, RhodeIsland 02903 (telephone: 207-894-6616; FAX:
predictive value of tests; comparative effectiveness research
PROSTATE cancer is the second leadingcause of cancer death in the UnitedStates.1 The rationale for tPSA screen-ing was to reduce the prevalence of ad-vanced prostate cancer and relatedmortality through early detection and
cancer related mortality has been ob-served,1,2 studies in screened popu-lations have raised concerns aboutthe test’s low specificity and therisks of treatment in men with earlystage disease that may never prog-
ress to symptoms.3 The balance of
http://dx.doi.org/10.1016/j.juro.2013.02.005Vol. 190, 389-398, August 2013
EARCH, INC. Printed in U.S.A.
207-892-8620; e-mail: [email protected]).† Nothing to disclose.‡ Financial interest and/or other relationship
COMPARATIVE EFFECTIVENESS REVIEW FOR PCA3 AND PROSTATE CANCER390
benefits and harms of tPSA screening remainscontroversial,3,4 and the search continues for ef-fective markers.
In 1999 researchers reported that the prostatecancer antigen 3 gene (PCA3) was highly over ex-pressed in prostate cancer tissue compared to nor-mal prostate or benign prostatic hyperplasia tissue.5
PCA3 tests measuring mRNA expression in urinewere developed and evaluated for analytical perfor-mance.6,7 Data supported the stability of PCA3 assayreagents, the insensitivity of the assay to pre-analyti-cal factors and the acceptability of the analytical per-formance.6,7 Adding stabilization buffer and shippingurine frozen improved the stability of samples.
In February 2012 the U.S. Food and Drug Admin-istration (FDA) approved the PROGENSA® PCA3assay to inform decision making about biopsy inmen age 50 years or older who had 1 or more negativebiopsies (no atypical small acinar proliferation) and forwhom “. . . repeat biopsy would be recommended . . .based on current standard of care”.7,8 No data on ini-tial biopsies were submitted.7 Used in conjunctionwith other clinical and laboratory information, the testcould save some patients from “. . . unnecessary biop-sies and related complications.”8
The comparative effectiveness review summa-rized here investigated the performance of PCA3and other test comparators for 2 intended uses.9 Thefirst use was the prediction of positive biopsy in menat risk for prostate cancer to inform decision makingabout initial or repeat biopsy. The second use wasthe distinction of men with aggressive disease need-ing immediate therapy from those with insignificanttumors who may be candidates for active surveil-lance.10 In this article we summarize the key dataand conclusions of the CER, discuss their potentialrelevance to clinical practice and identify key gapsin knowledge.
MATERIALS AND METHODS
This summary article is based on the 2012 CER of thesame title.9 Detailed methods11,12 can be found in thereport.9 With guidance from the TEP we developed ana-lytical frameworks and key questions to frame the review.KQs 1 and 2 investigated the comparative effectiveness ofPCA3 and other biomarker testing to predict a positiveprostate biopsy in at risk men. KQ 1 focused on initialbiopsy and KQ 2 focused on repeat biopsy. KQ 3 investi-gated the comparative effectiveness of PCA3 and otherbiomarkers or pathological tests to categorize tumor riskin biopsy positive men.
MEDLINE®, EMBASE® and the Cochrane CentralRegister were searched from January 2012 through May2012 for English language studies addressing the KQs,with additional information sought through bibliographyhand searches and targeted web searches. Only matched
studies were included in analysis, defined as measuring
PCA3 and comparator tests in the same cohort of men.The TEP and other clinical advisors informed the selectionof comparators. Review and extraction of study datawere performed by multiple reviewers, and a seniorreviewer confirmed the accuracy. The QUADAS tool wasused to assess the quality of studies13 graded as good,fair or poor.11
For KQs 1 and 2 the selection criteria included appro-priate study design; study subjects at increased prostatecancer risk based on increased tPSA and/or abnormaldigital rectal examination; and reported results for PCA3,comparator test(s) and prostate biopsy. There were 6 bio-marker comparators including 3 PSA isoforms (tPSA,fPSA and cPSA); PSAD; total PSA velocity or doublingtime; and EVNs (risk assessment tools that combine clin-ical and laboratory risk factors).14 Clinical outcomes werediagnostic accuracy (clinical validity); impact on biopsydecision making; harms of biopsy; and health outcomes(clinical utility) of morbidity/mortality, quality of life andtreatment related harms.
For KQ 3 the selection criteria included appropriatestudy design; study subjects having biopsy confirmedprostate cancer; reported results for PCA3, other biomark-ers, and pathological markers from biopsy and/or prosta-tectomy; and followup of health outcomes to determine thediagnostic accuracy of risk categorization. The selectedcomparators for defining categories of tumor aggressive-ness or risk included biomarkers and other clinical andpathological markers (eg Gleason score, tumor volumeand clinical stage). Clinical outcomes of interest were di-agnostic accuracy for improved health outcomes based onrisk categorization; impact on treatment decisions; andlong-term health outcomes of morbidity and mortality,quality of life and harms of misclassification or treatment.
For KQs 1 and 2 the ideal analysis for these studieswould be a comparison of within person results (iematched analysis). For example, a 2�2 table would beconstructed for men with a positive biopsy, showing whenboth tests predicted positive or negative biopsy for thesame men, or when test predictions disagreed. Another2�2 table would be constructed for subjects with anegative biopsy and analyzed similarly. In the absenceof such data we would perform unmatched analyseswithin studies for PCA3 and each comparator, including1) AUC, 2) parameters (eg median, IQR) that define thedistribution of test results for men with positive vsnegative biopsies, 3) estimated clinical sensitivity andspecificity of PCA3 at a cutoff score of 35 compared withthe sensitivity of the comparator at the same specificity,4) estimated clinical sensitivities (from published ROCcurves) over a range of specificities and 5) results fromregression analyses (eg independence of PCA3 and com-parators, regressed odds ratios).
For KQ 3 the variety of potential PCA3 comparatorsmade it unlikely that multiple matched studies wouldaddress the risk classification of cancers using similarstudy designs, comparators and outcomes. If quantitativedata were insufficient, qualitative analysis would be per-formed to include descriptive narrative, summary tablesdescribing the studies and findings, and identification of
any content themes.
COMPARATIVE EFFECTIVENESS REVIEW FOR PCA3 AND PROSTATE CANCER 391
Each outcome for each comparator would be evaluatedfor overall SOE using the GRADE system.15 This systeminvolves the assessment of 4 domains (risk of bias, consis-tency of results, directness of evidence and precision) toestablish a rating of high, moderate, low or insuffi-cient.11,15 Risk of bias is primarily related to study de-sign limitations. GRADE initially rates all randomizedcontrolled trials as high (ie confident that the true effect isclose to the effect estimate) unless they are downgradedbased on domain results (eg inconsistency, methodologicalflaws).15 Observational studies are considered inher-ently biased and are initially rated low (ie limited con-fidence in the effect estimate and the true effect may bedifferent), unless they are upgraded based on large andconsistent effect size, good study design and executionor other factors.15
RESULTS
Of the 1,556 citations identified, 42 relevant peerreviewed articles and a regulatory document wereincluded (fig. 1). Thirteen studies addressed riskcategorization of prostate cancer as described in KQ3 (supplementary table 1, http://jurology.com/). Af-ter excluding articles from analysis (eg duplicatedata), 24 studies addressed diagnostic accuracy forbiopsy but no other KQ 1 and KQ 2 outcomes (table1). No studies performed or provided data to sup-port matched analyses. Studies did provide indepen-dent estimates of clinical sensitivity and specificity (orrelated measures) for PCA3 and a comparator. Thus,
Figure 1. Flow chart detailing selection of pub
analyses relied on comparing the differences betweenwithin study paired summary estimates.
Analyses of tPSA and related comparators werecomplicated by the presence of partial verificationbias. All included studies of diagnostic accuracywere performed in opportunistic cohorts of menchoosing biopsy. Partial verification bias was causedby the use of tPSA in biopsy decision making. Menwith higher tPSA increases were more likely to un-dergo biopsy, so men with a lower tPSA would beunderrepresented in the study cohort. We relied onrelevant publications and in-house modeling in anattempt to address the extent to which this biasimpacted results.16 Analyses were also subject to asampling bias as some studies restricted enrollmentto tPSA results in the gray zone (eg 2.5 to less than10 ng/ml). This bias was addressed by stratifyinganalyses by enrollment criteria.
Table 1 shows the 24 included studies, 20 ofwhich reported the proportion of men with initialbiopsy (eg 20% repeat biopsy indicated as 80%initial).7,17–35 Table 1 also summarizes the typesof analyses reported for each of the 6 compara-tors.36 –39 For example, Ankerst et al reported 3 dif-ferent analyses for the PCA3 and tPSA comparison,and 2 for PCA3 and EVNs.17 There were 6 studies thatrestricted enrollment to men with tPSA in the grayzone associated with sampling bias.18,22–24,28,29 Themost common comparator reported was tPSA and the
COMPARATIVE EFFECTIVENESS REVIEW FOR PCA3 AND PROSTATE CANCER392
most common analysis reported was AUC. All 5 anal-yses were reported for 3 comparators (tPSA, fPSA andPSAD), but never more than 2 for any given com-parator/analysis pair. The QUADAS rating of allstudies was poor (ie flaws such as partial verifica-tion bias that may impact performance estimates).No studies were identified that provided evidenceon the outcomes of patient decision making, biopsyrelated harms or health outcomes. SOE for theseoutcomes was insufficient (ie evidence not avail-able or did not permit a conclusion to be reached)for all comparators (supplementary table 2, http://jurology.com/).
Diagnostic Accuracy
for Predicting Positive Biopsy
The absence of studies, or the number of studiesbeing too limited to allow assessment of GRADEdomains, led to a SOE rating of insufficient for di-agnostic accuracy in 5 of the 6 comparators (fPSA,tPSA velocity/doubling time, PSAD, cPSA andEVNs). Only tPSA had sufficient studies and re-ported data to allow for analyses.
Two studies were identified that compared PCA3and tPSA, restricted enrollment to men having onlyinitial biopsies and were tPSA gray zone stud-ies.28,29 Seven studies restricted enrollment to onlyrepeat biopsies,7,17–19,22,24,25 and of these 3 were
Table 1. Data available for comparing PCA3 scores with compa
References% with
Initial Biopsy tPSA
Ankerst et al17 0 A, B, DAubin et al18* 0 A, C, DAuprich et al19 0 A, BFDA Summary7 0 —Pepe and Aragona22* 0 —Ploussard et al24* 0 A, DWu et al25 0 A, DDeras et al20 51 A, C, DNyberg et al21 55 A, B, DBollito et al27 59 A, B, DPerdonà et al23* 61 A, B, D, EGoode et al30 63 A, C, DRoobol et al33 71 AWang et al35 73 CRigau et al32 74 A, DOchiai et al31 81 A, B, C, DAdam et al26 82 A, C, DSchilling et al34 86 A, Dde la Taille et al28* 100 A, C, D, EFerro et al29* 100 BCao et al36 Not reported AHessels et al37 Not reported AMearini et al38 Not reported AOuyang et al39 Not reported A
A—AUC, B—population parameters, C—performance at a PCA3 of 35 or greate* Study focused on the gray zone of tPSA when enrolling patients.
gray zone studies.18,22,24 When stratified by gray
zone, no comparator had more than 2 analyses incommon. All studies were rated poor for the ques-tions posed. Based on the limited number of compa-rable studies, the SOE was considered insufficientfor diagnostic accuracy comparisons of PCA3 andtPSA in only initial biopsy (KQ 1) or only repeatbiopsy (KQ 2) study groupings.
Restriction of Analyses to Studies
on Only Initial or Only Repeat Biopsies
Since only 9 of 24 included studies (37%) focused oncohorts with only initial or only repeat biopsies, allstudies reporting PCA3 and tPSA comparisons wereexamined to determine whether this stratificationwas necessary (table 1). Overall 20 of 24 studies(83%) reported the proportion of men having initialbiopsy and AUC. Figure 2 shows the initial biopsyrate vs the matched difference between the PCA3and tPSA AUCs. Among the 11 studies with a mix-ture of initial and repeat biopsies, most only re-ported results for the entire cohort. Three studiesseparately reported data for initial and/or repeatbiopsies,24,27,30 resulting in more observations at 0%and 100% initial biopsy than listed in table 1. Theassociation between AUC difference and initial bi-opsy rate (regression line) showed a nonsignificant(p � 0.81) slope (�0.01307) (fig. 2). This slope wouldtranslate into the PCA3-tPSA AUC difference de-
and the percentage of men having initial biopsy
tched Analyses Available for Each PCA3 Comparator
PSA Velocity PSAD cPSA EVN
— — — A, B— — — —
A, B — — —— — — E— — — —— — — —— A, C, D — —— — — —— — — —— — — —— — — A, B, D— D — —— — — —— — — —— — — —— A, B, D — D— — — —— — — —— A, C, D — —— — — —— — — —— — — —— — — —— — — —
ensitivity/specificity from ROC curves and E—regression analysis.
rators
Ma
%fPSA
—A, D, EA, B
—BA, D
———
A, B, DB, E
———————
C, D, EA, D
————
r, D—s
creasing from an 11% advantage for PCA3 (left side)
COMPARATIVE EFFECTIVENESS REVIEW FOR PCA3 AND PROSTATE CANCER 393
to an approximately 10% advantage (right side) asthe biopsy mixture ranged from all repeat to allinitial. Because this change is statistically nonsig-nificant and clinically unimportant, we concludedthat data could be combined regardless of biopsystatus. Figure 2 also showed that restricting theregression to those not focused on the gray zone(closed symbols) or to those using the PROGENSAassay (circles) did not change the finding. The 4studies not reporting the initial biopsy rate areshown on the far right of figure 2 and all wereconsistent with previous AUC differences.
Diagnostic Accuracy of
PCA3 Scores vs tPSA Increases
Combining information from all of the studies intable 1 without regard to proportion with initialbiopsy resulted in several findings.
Area under ROC curve. Among the 20 includedstudies (table 1, with AUC available for tPSA), 18(90%) showed PCA3 AUC to be higher by a mediandifference of 0.1055. Data from the 4 gray zone stud-ies tended to find larger differences in favor of PCA3(0.1595).18,23,24,28
Population parameters. Eight studies reported
Figure 2. Difference in AUC for PCA3 scores minus AUC for tPSAincreases vs proportion of men having initial biopsy. Analysis wasperformed to maximize observations at 0% (all repeat biopsies)and 100% (all initial biopsies). AUC differences for 4 studies notreporting biopsy status of population are shown at far right. Eachsymbol represents computed difference from included study, as 1study may have reported AUCs separately for men with initial vsrepeat biopsies. Closed symbols indicate gray zone studies inwhich tPSA upper limit was part of inclusion criteria (eg tPSAgreater than 2.5 but less than 10 ng/ml). Circles indicate PCA3scores were obtained using PROGENSA technology while squaresindicate predecessor or alternative methodologies (supplementarytable 1). Regression line is fitted to all 19 data points for whichproportion of men with initial biopsies was available (results withunknown proportion not included).
combinations of medians, interquartile ranges and
ranges.17,19,21,23,27,29,31,36 All found the distributionsto be right skewed but none used transformations.The difference in marker levels between the biopsypositive and negative men was expressed as az-score. The difference in the 2 z-scores was used tocompare markers. In 7 of 8 studies the z-score dif-ference was in favor of PCA3 (median 0.45, range�0.22 to 1.05). The 2 gray zone studies had signifi-cantly smaller tPSA standard deviations.23,29 Thedistribution parameters from the remaining 6 stud-ies formed the basis of later modeling (table 2).
Performance using PCA3 score cutoff of 35. Ninestudies reported the sensitivity and specificity of PCA3at this cutoff.18,20,25–28,30,31,35 The difference in thePCA3 and tPSA sensitivities (at the set PCA3 speci-ficity) was positive for 8 of 9 studies (median difference16.3%). The PCA3 sensitivity and false-positive ratescentered around 53% and 20%, respectively.
ROC curve. Fourteen studies provided 16 data setsfor analysis,17,18,20,21,23–28,30–32,34 as some studiesprovided ROC curves stratified by initial and repeatbiopsy.27,30 Sensitivities were estimated at FPRs(1-specificity) of 20% through 80%. Within each studythe PCA3-tPSA sensitivity differences were com-puted and the median difference summarized foreach of the 7 false-positive rates. At a 50% FPR themedian difference in sensitivities was 20% (range�9% to 39%). When the 4 gray zone studies18,23,24,28
and the study in south African men26 were removed,the median difference in the 9 remaining studieswas unchanged. The corresponding median sensitiv-ities for PCA3 scores and tPSA increases were 77%and 57%, respectively. Figure 3 displays the mediansensitivities for the 9 studies at the 7 false-positiverates for PCA3 scores (open circles) and tPSA in-creases (closed circles). At all specificities the me-dian sensitivity for PCA3 was higher.
Regression analyses. Five studies reported datathat could be used to determine the independence ofPCA3 scores and tPSA increases to predict biopsyoutcome or to compare the addition of PCA3 scoresto a regression model (usually logistic) that alreadyincluded tPSA measurements.18,23,28,31,37 Threestudies reported the correlations to be low.23,28,37 All5 studies reported PCA3 and tPSA odds ratios thatwere statistically significant as a single marker, andremained so after combining PCA3 with tPSA (andsometimes with other demographic markers as well).These findings indicate that the 2 markers providerelatively independent information, and that thecombination of the 2 should be more predictive thaneither marker alone.
The initial GRADE rank for SOE was low becauseall studies were observational.15 Additional domains
were considered when assessing potential revision of
iduals a
COMPARATIVE EFFECTIVENESS REVIEW FOR PCA3 AND PROSTATE CANCER394
the SOE (upward to moderate, or downward to in-sufficient). Modeling suggested biases had little im-pact on the ROC curves (and the AUC) or on thePCA3 measurements. Among the 5 analyses with
Table 2. Modeled performance including sensitivity, false-posit
% 1-Specificity (FPR) % Saved Biopsy* % Se
PCA3 score:10 or Greater 77.9 2215 or Greater 63.6 3620 or Greater 52.0 4825 or Greater 42.8 5730 or Greater 35.6 6435 or Greater 29.9 7040 or Greater 25.3 7545 or Greater 21.5 78
tPSA (ng/ml):2 or Greater 77.1 233 or Greater 62.7 374 or Greater 51.0 495 or Greater 41.9 586 or Greater 34.7 657 or Greater 29.0 718 or Greater 24.5 769 or Greater 20.8 79
* Proportion of biopsies avoided is equal to the specificity of the test.† Proportion of prostate cancer missed is equal to 100 [�] sensitivity.‡ The population likelihood ratio is the increase (or decrease) in the prevalencegreater). The individual likelihood ratio is the increase (or decrease) for the indiv
Figure 3. Summary observed sensitivities at 7 selected FPRs(1-specificity) with fitted ROC curves. Median of 11 sensitivity es-timates (from 9 studies) are plotted for PCA3 scores (open circles)and tPSA increases (closed circles) at selected FPRs of 20% to 80%.Data were derived from published figures. Fitted ROC curves are
based on overlapping logarithmic Gaussian curves.
sufficient data, the matched differences were consis-tent. Since raw data were not available, the usualmethods of calculating confidence intervals of differ-ences could not be applied. However, the consistentlysmaller matched differences observed for gray zonestudies suggested the precision was sufficient. Therewas a direct link to the health outcome (ie biopsyresults). However, because none of the individual stud-ies provided a matched analysis, directness could notbe confirmed (ie even with equal test performance, the2 markers might identify groups of men with differentlevels of severity). There was no indication of publica-tion bias and the overall strength of association wasweak (ie small effect size). These domain results didnot provide a sufficient rationale for downgrading orupgrading the SOE grade of low.
Knowledge Synthesis of
PCA3 Scores and tPSA Increases
This review directly addressed the impact of verifi-cation and sampling bias by developing a mathemat-ical model based on overlapping Gaussian curves(CER Appendix J).9 The parameters defining thesecurves were informed by the data generated in thepopulation parameter analysis. Figure 4, A showsthe overlapping curves for PCA3 scores, and thesecould be used to estimate sensitivity and specificityat various PCA3 cutoffs. For example, at a PCA3score cutoff of 20, the modeled false-positive (1-spec-ificity) rate of 52.0% and sensitivity rate of 80.5%were consistent with the observed summary results
se in the group identified at or above the indicated cutoff (eg PCA3 score 10 ort the indicated cutoff (eg PCA3 score of 10).
ive rat
nsitivity
98876655
87654332
of disea
of 50% and 77% provided earlier. By changing the
COMPARATIVE EFFECTIVENESS REVIEW FOR PCA3 AND PROSTATE CANCER 395
PCA3 cutoff across the range of scores (x-axis of fig.4, A), selected sensitivity/specificity pairs could begenerated. These pairs were displayed as a smoothcurve (ROC) in figure 3, superimposed on the sum-mary PCA3 ROC point estimates (open circles) fromthe 14 included studies. The close fit indicated thatthe model was acceptable over the range examined.
The modeled performance of PCA3 scores couldthen be computed at specific cutoffs (table 2). Forexample, at scores of 35 or greater, the FPR wasapproximately 30%, indicating that 70% of the highrisk men would not need a biopsy (potential benefit).The corresponding sensitivity was approximately61%, indicating that 39% of prostate cancers wouldnot be detected compared to all at risk men having abiopsy (potential harm). Table 2 also presents like-lihood ratios (relative increase or decrease in priorrisk) for 1) those with values at or above the selectedcutoff (labeled population) and 2) those at the se-lected cutoff (labeled individual). Most of these like-lihood ratios were relatively close to 1.0, indicatingthat the risks do not change dramatically as a resultof testing (small effect size).
Creating a model for tPSA was more complicated,but followed the same procedures (CER AppendixJ).9 Studies focused on the gray zone were excluded.The parameters used to draw the overlapping curvesfor tPSA must be viewed as approximate and wouldneed additional validation before clinical use (fig. 4,B). For example, at a tPSA cutoff of 4 ng/ml orgreater, a sensitivity of 61.4% was estimated to oc-cur at a false-positive rate of 51%. This was in linewith the summary 57% sensitivity at a false-positive
Figure 4. Overlapping Gaussian curves representing PCA3 scores(A) and tPSA (B) increases in men with negative (solid curve) andpositive (broken curve) biopsies. At PCA3 cutoff score of approxi-mately 45, FPR or 1-specificity is approximately 21% (crosshatchedarea) while corresponding sensitivity is approximately 50%(hatched area). By moving cutoff to higher and lower levels, ROCcurve can be generated (fig. 3). Curves can be generated using logmean/standard deviations for negative and positive biopsy group(1.322, 0.420; 1.663, 0.420, respectively). Analysis for tPSA in-creases shows more overlap in measurements. Correspondingparameters for tPSA are 0.613, 0.420; 0.724, 0.420, respectively.
rate of 50% reported earlier under the ROC analyses.
Additional modeled specificity, sensitivity and like-lihood ratios were provided for select tPSA increases(table 2).
Diagnostic Accuracy of Tumor Risk
Categorization to Inform Treatment Decision Making
Thirteen identified studies investigated the perfor-mance of PCA3 and comparators in placing menwith biopsy confirmed prostate cancer into catego-ries of cancer risk (aggressive or insignificant can-cer).28,36,37,40–49 Ten studies reported prostatectomyresults as the end point. Prostatectomy was not con-sidered a validated intermediate or surrogate endpoint for determining clinical sensitivity and speci-ficity of risk classification. For example, an associa-tion might be observed between a pattern of bio-marker/biopsy results that categorize cancer asinsignificant and subsequent pathological findingsat prostatectomy. However, a formal evidentiarylink would still be needed between the specific pros-tatectomy findings and health outcomes of patientsin this category (eg measures of progression, func-tion, metastasis, morbidity/mortality). Only 2 stud-ies addressed health outcomes related to PCA3 andtPSA.42,46 We were unable to compare or combinethese 2 studies as they reported on different sampletypes, compared different performance parametersand addressed different outcomes. No studies re-ported on other outcomes. All studies were observa-tional, with a QUADAS rating of poor. The SOE forall outcomes and comparators was insufficient.
DISCUSSION
To our knowledge this is the first CER to examinethe role of PCA3 in biopsy and treatment/surveil-lance decision making. Sufficient data were foundto address only 1 of the many comparisonsplanned, and no study performed or provided datafor matched analyses. Only matched studies wereincluded to ensure that demographic and otherfactors (eg age, race, family history) were at leastequivalent for comparisons performed within agiven study. This appears to have been successful.However, this restriction may also explain the lim-ited data for analyses. Future reviews could deter-mine if informative unmatched studies were ex-cluded from study, and whether their inclusionwould affect the heterogeneity of results.
Three important conclusions were drawn fromthese analyses. 1) PCA3 scores had greater diagnos-tic accuracy for positive biopsy than tPSA increases.2) The relative performance of PCA3 vs tPSA did notappear to be dependent on whether it was a repeatbiopsy. 3) The information provided by PCA3 scorewas essentially independent from that provided by
tPSA increases. The possibility that PCA3 testing may
COMPARATIVE EFFECTIVENESS REVIEW FOR PCA3 AND PROSTATE CANCER396
be as useful in men making decisions about an initialbiopsy could broaden the test’s potential use beyonddecision making about repeat biopsies. However, thisnew finding was documented using only AUC analysisand should be independently confirmed.
In general, the included studies were opportunis-tic cohorts of men opting for biopsy or prostatec-tomy. These study designs produced the describedpartial verification bias, but none of the studies ac-knowledged the existence of this bias or the role itmight have in interpreting the results. Our analysessuggested that this bias had little impact on ROCcurves (and the AUC) or PCA3 measurements. How-ever, it had an important impact on the sensitivityand specificity estimates at given tPSA cutoffs. Weused simple Gaussian modeling to fit the evidencefor PCA3 and tPSA that attempted to account forthe verification bias (table 2). This allowed for amore comprehensive comparison of the 2 markersthat included specifying performance at selectedcutoffs. Although more difficult to execute, a studydesign with less bias would be one in which theentire cohort of at risk men was investigated (egtPSA greater than 4.0 ng/ml with and without ab-normal digital rectal examination), with the decisionmaking steps made explicit. The men not choosingbiopsy could then be followed for subsequent bi-opsy and/or evidence of future disease. In addi-tion, a reexamination of included studies that re-lies on matched analyses of individual data is stillpossible, and should be encouraged.
Future clinical use of PCA3 will likely depend onseveral factors. The FDA approval of a PCA3 testhas raised awareness and could accelerate adoptioninto practice. However, many unknowns remain.Uptake by physicians could depend on issues suchas cost and availability, how the test could be inte-grated into current protocols for management, pro-fessional practice guidelines, and the availability ofadditional information regarding the impact of test-ing on intermediate and long-term health outcomes.For example, it is unclear if the test should be of-fered to all tPSA screen positive men, only thosewith a tPSA less than 10 ng/ml or only those makingdecisions about repeat biopsy. It would be importantto know how much improvement in diagnostic accu-racy would be required for PCA3 to impact physicianand patient decisions about biopsy, and if those de-cisions translated into fewer biopsies being per-formed in men without prostate cancer. Neither
PCA3 nor tPSA alone demonstrated high screening
REFERENCES
2010. CA Cancer J Clin 2010; 60: 277. fying the role of PSA screen
performance. However, methods to combine theseand other biomarker results with clinical factors tobetter inform decision making should be pursuedand validated in clinical settings.
The importance of effective schemes for risk strati-fication was reemphasized by the recent Prostate Can-cer Intervention Versus Observation Trial (PIVOT),which reported on a 12-year followup of men withhistologically confirmed localized prostate cancer.50
The authors found no difference in all cause or pros-tate cancer specific mortality between men assignedto observation (watchful waiting) vs those randomlyassigned to radical prostatectomy. This provides ev-idence that the markers used for informing decisionswere not adequate. Newer markers like PCA3 needto be further explored in well designed, and appro-priately powered and analyzed prospective trialsthat determine intermediate and long-term out-comes.10 Long-term observational studies of healthoutcomes are also subject to biases. Although difficultand expensive, randomized trials may be needed toanswer these questions.
ACKNOWLEDGMENTS
The TEP provided insight, suggestions and com-ments, and included Dr. Peter Albertsen, Chief, Di-rector of Urology, University of Connecticut HealthCenter, Farmington, CT; Dr. Todd Alonzo, AssociateProfessor of Research, Department of PreventiveMedicine, University of Southern California, Arca-dia, CA; Dr. William Dotson, Senior CoordinatingScientist, Public Health Genomics, Centers for Dis-ease Control and Prevention, Atlanta, GA; Dr. PeterGann, Professor of Pathology, College of Medicine,University of Illinois at Chicago, Chicago, IL; Dr.Roger Klein, Affiliate Associate Professor, Center ofEvidence-Based Medicine and Health Outcomes Re-search, Clinical and Translational Science Institute,University of South Florida, Morsani College ofMedicine, Tampa, FL; Dr. Stephen Spann, Professorand Chair, Family/Community Medicine, BaylorCollege of Medicine, Houston, TX; and Dr. ThomasTrikalinos, Associate Professor, Director, Center forEvidence-Based Medicine, Program in Public Health,Brown University, Providence, RI.
The other BlueCross BlueShield Association Tech-nical Evaluation Center, Evidence-based PracticeCenter staff who contributed to this project, ClaudiaJ. Bonnell, Jennifer Brock, Ryan Chopra, MichaelDouglas, Lisa Garofalo, Denise Oliansky, Lisa Sar-
sany and Kathleen M. Ziegler.
1. Jemal A, Siegel R, Xu J et al: Cancer statistics,
2. Etzioni R, Tsodikov A, Mariotto A et al: Quanti- ing in the US prostate
cancer mortality decline. Cancer Causes Control
2008; 19: 175.
COMPARATIVE EFFECTIVENESS REVIEW FOR PCA3 AND PROSTATE CANCER 397
3. Chou R, Croswell JM, Dana T et al: Screening forprostate cancer: a review of the evidence for theU.S. Preventive Services Task Force. Ann InternMed 2011; 155: 762.
4. Schröder FH, Hugosson J, Roobol MJ et al: Pros-tate-cancer mortality at 11 years of follow-up.N Engl J Med 2012; 366: 981.
5. Bussemakers MJ, van Bokhoven A, Verhaegh GWet al: DD3: a new prostate-specific gene, highlyoverexpressed in prostate cancer. Cancer Res1999; 59: 5975.
6. Sokoll LJ, Ellis W, Lange P et al: A multicenterevaluation of the PCA3 molecular urine test:pre-analytical effects, analytical performance,and diagnostic accuracy. Clin Chim Acta 2008;389: 1.
7. U.S. Food and Drug Administration: Summary ofSafety and Effectiveness Data: PROGENSA PCA3Assay, 2012. Available at www.accessdata.fda.gov/cdrh_docs/pdf10/P100033b.pdf. Accessed July16, 2012.
8. U.S. Food and Drug Administration: Medical De-vices: PROGENSA PCA3 Assay, 2012. Availableat www.fda.gov/MedicalDevices/ProductsandMedicalProcedures/DeviceApprovalsandClearances/Recently-ApprovedDevices/ucm294907.htm. Acces-sed January 11, 2013.
9. Bradley LA, Palomaki GE, Gutman S et al: Com-parative Effectiveness of PCA3 and ComparatorTesting for the Diagnosis and Management ofProstate Cancer. Comparative Effectiveness Re-view No. 98 (Prepared by the BCBSA TechnicalEvaluation Center Evidence-based Practice Centerunder Contract No. 290–2007-10058-I.) AHRQ Pub-lication No. 13-EHC030-EF. Rockville, MD: Agencyfor Healthcare Research and Quality; December2012. Available at www.effectivehealthcare.ahrq.gov/reports/final.cfm. Accessed October 19, 2012.
10. Cooperberg MR, Carroll PR and Klotz L: Activesurveillance for prostate cancer: progress andpromise. J Clin Oncol 2011; 29: 3669.
11. Methods Guide for Effectiveness and Compar-ative Effectiveness Reviews. Rockville, MD:Agency for Healthcare Research and Quality; Au-gust 2011. AHRQ Publication No. 10(11)-EHC063-EF. Available at www.effectivehealthcare.ahrq.gov. Accessed October 19, 2012.
12. Methods Guide for Medical Test Reviews. Rock-ville, MD: Agency for Healthcare Research andQuality; Draft Posted November 2010, vol. 2012.
13. Whiting PF, Weswood ME, Rutjes AW et al:Evaluation of QUADAS, a tool for the qualityassessment of diagnostic accuracy studies. BMCMed Res Methodol 2006; 6: 9.
14. Schröder F and Kattan MW: The comparability ofmodels for predicting the risk of a positive pros-tate biopsy with prostate-specific antigen alone:a systematic review. Eur Urol 2008; 54: 274.
15. Owens DK, Lohr KN, Atkins D et al: AHRQ seriespaper 5: grading the strength of a body of evi-
dence when comparing medical interventions–
Agency for Healthcare Research and Qualityand the effective health-care program. J ClinEpidemiol 2010; 63: 513.
16. Punglia RS, D’Amico AV, Catalona WJ et al:Effect of verification bias on screening for pros-tate cancer by measurement of prostate-specificantigen. N Engl J Med 2003; 349: 335.
17. Ankerst DP, Groskopf J, Day JR et al: Predictingprostate cancer risk through incorporation ofprostate cancer gene 3. J Urol 2008; 180: 1303.
18. Aubin SM, Reid J, Sarno MJ et al: PCA3 molec-ular urine test for predicting repeat prostate bi-opsy outcome in populations at risk: validation inthe placebo arm of the dutasteride REDUCE trial.J Urol 2010; 184: 1947.
19. Auprich M, Augustin H, Budäus L et al: A com-parative performance analysis of total prostate-specific antigen, percentage free prostate-spe-cific antigen, prostate-specific antigen velocityand urinary prostate cancer gene 3 in the first,second and third repeat prostate biopsy. BJU Int2012; 109: 1627.
20. Deras IL, Aubin SM, Blase A et al: PCA3: amolecular urine assay for predicting prostate bi-opsy outcome. J Urol 2008; 179: 1587.
21. Nyberg M, Ulmert D, Lindgren A et al: PCA3 as adiagnostic marker for prostate cancer: a valida-tion study on a Swedish patient population.Scand J Urol Nephrol 2010; 44: 378.
22. Pepe P and Aragona F: PCA3 score vs PSA free/total accuracy in prostate cancer diagnosis atrepeat saturation biopsy. Anticancer Res 2011;31: 4445.
23. Perdonà S, Cavadas V, Di Lorenzo G et al: Pros-tate cancer detection in the “grey area” of pros-tate-specific antigen below 10 ng/ml: head-to-head comparison of the updated PCPT calculatorand Chun’s nomogram, two risk estimators incor-porating prostate cancer antigen 3. Eur Urol 2011;59: 81.
24. Ploussard G, Haese A, Van Poppel H et al: Theprostate cancer gene 3 (PCA3) urine test in menwith previous negative biopsies: does free-to-total prostate-specific antigen ratio influence theperformance of the PCA3 score in predicting pos-itive biopsies? BJU Int 2010; 106: 1143.
25. Wu AK, Reese AC, Cooperberg MR et al: Utility ofPCA3 in patients undergoing repeat biopsy forprostate cancer. Prostate Cancer Prostatic Dis2012; 15: 100.
26. Adam A, Engelbrecht MJ, Bornman MS et al: Therole of the PCA3 assay in predicting prostatebiopsy outcome in a South African setting. BJUInt 2011; 108: 1728.
27. Bollito E, De Luca S, Cicilano M et al: Prostatecancer gene 3 urine assay cutoff in diagnosis ofprostate cancer: a validation study on an Italianpatient population undergoing first and repeat
biopsy. Anal Quant Cytol Histol 2012; 34: 96.
28. de la Taille A, Irani J, Graefen M et al: Clinicalevaluation of the PCA3 assay in guiding initialbiopsy decisions. J Urol 2011; 185: 2119.
29. Ferro M, Bruzzese D, Perdonà S et al: Predictingprostate biopsy outcome: prostate health index(phi) and prostate cancer antigen 3 (PCA3) areuseful biomarkers. Clin Chim Acta 2012; 413:1274.
30. Goode RR, Marshall SJ, Duff M et al: Use ofPCA3 in detecting prostate cancer in initial andrepeat prostate biopsy patients. Prostate 2013;73: 48.
31. Ochiai A, Okihara K, Kamoi K et al: Prostatecancer gene 3 urine assay for prostate cancer inJapanese men undergoing prostate biopsy. IntJ Urol 2011; 18: 200.
32. Rigau M, Morote J, Mir MC et al: PSGR andPCA3 as biomarkers for the detection of prostatecancer in urine. Prostate 2010; 70: 1760.
33. Roobol MJ, Schröder FH, van Leeuwen P et al:Performance of the prostate cancer antigen 3(PCA3) gene and prostate-specific antigen in pre-screened men: exploring the value of PCA3 for afirst-line diagnostic test. Eur Urol 2010; 58: 475.
34. Schilling D, Hennenlotter J, Munz M et al: Inter-pretation of the prostate cancer gene 3 in refer-ence to the individual clinical background: impli-cations for daily practice. Urol Int 2010; 85: 159.
35. Wang Y, Sun G, Pan JG et al: Performance oftPSA and f/tPSA for prostate cancer in Chinese. Asystematic review and meta-analysis. ProstateCancer Prostatic Dis 2006; 9: 374.
36. Hessels D, van Gils MP, van Hooij O et al:Predictive value of PCA3 in urinary sediments indetermining clinico-pathological characteristicsof prostate cancer. Prostate 2010; 70: 10.
37. Cao DL, Ye DW, Zhang HL et al: A multiplexmodel of combining gene-based, protein-based,and metabolite-based with positive and negativemarkers in urine for the early diagnosis of pros-tate cancer. Prostate 2011; 71: 700.
38. Mearini E, Antognelli C, Del Buono C et al: Thecombination of urine DD3(PCA3) mRNA and PSAmRNA as molecular markers of prostate cancer.Biomarkers 2009; 14: 235.
39. Ouyang B, Bracken B, Burke B et al: A duplexquantitative polymerase chain reaction assaybased on quantification of alpha-methylacyl-CoAracemase transcripts and prostate cancer antigen3 in urine sediments improved diagnostic accu-racy for prostate cancer. J Urol 2009; 181: 2508.
40. Auprich M, Chun FK, Ward JF et al: Criticalassessment of preoperative urinary prostate can-cer antigen 3 on the accuracy of prostate cancerstaging. Eur Urol 2011; 59: 96.
41. Durand X, Xylinas E, Radulescu C et al: The valueof urinary prostate cancer gene 3 (PCA3) scores inpredicting pathological features at radical pros-
COMPARATIVE EFFECTIVENESS REVIEW FOR PCA3 AND PROSTATE CANCER398
42. Kusuda Y, Miyake H, Kurahashi T et al: Assess-ment of optimal target genes for detecting mi-crometastases in pelvic lymph nodes in patientswith prostate cancer undergoing radical prosta-tectomy by real-time reverse transcriptase-poly-merase chain reaction. Urol Oncol 2011; Epubahead of print.
43. Liss MA, Santos R, Osann K et al: PCA3 molec-ular urine assay for prostate cancer: associationwith pathologic features and impact of collectionprotocols. World J Urol 2011; 29: 683.
44. Nakanishi H, Groskopf J, Fritsche HA et al: PCA3molecular urine assay correlates with prostate
EDITORIAL COMMENT
perior to total PSA in accurately diagn
candidates for active surveillance. J Urol 2008;179: 1804.
45. Ploussard G, Durand X, Xylinas E et al: Prostatecancer antigen 3 score accurately predicts tumourvolume and might help in selecting prostate can-cer patients for active surveillance. Eur Urol 2011;59: 422.
46. Tosoian JJ, Loeb S, Kettermann A et al: Accuracyof PCA3 measurement in predicting short-termbiopsy progression in an active surveillance pro-gram. J Urol 2010; 183: 534.
47. van Poppel H, Haese A, Graefen M et al: The
osing prostate
(PCA3) and prostate cancer significance. BJU Int2012; 109: 360.
48. Vlaeminck-Guillem V, Devonec M, Colombel M etal: Urinary PCA3 score predicts prostate cancermultifocality. J Urol 2011; 185: 1234.
49. Whitman EJ, Groskopf J, Ali A et al: PCA3 scorebefore radical prostatectomy predicts extracapsu-lar extension and tumor volume. J Urol 2008; 180:1975.
50. Wilt TJ, Brawer MK, Jones KM et al: Radicalprostatectomy versus observation for localized
cancer tumor volume: implication in selecting relationship between Prostate CAncer gene 3 prostate cancer. N Engl J Med 2012; 367: 203.
This report summarizes a comparative effectivenessreview of PCA3 commissioned by the U.S. Agencyfor Healthcare Quality and Research. I suspect thatmany readers will not be familiar with the rigorousmethods used in these reviews and will find thewriting style somewhat foreign. Acknowledging this,it is important that we not dismiss these findings.Specifically, using an explicit and structured ap-proach to rate study quality, the researchers foundthat all reviewed studies of PCA3 were of poor qual-ity. This, in turn, restricted their ability to makepositive statements regarding the comparative effec-tiveness of PCA3 in improving health outcomes oraiding in clinical decision making. In fact, the onlyconclusion they could draw was that PCA3 was su-
cancer, and even here, the strength of evidence waslow.
What are we to take away from this report, be-yond the fact that PCA3 may add some additionalinformation beyond total PSA in diagnosing prostatecancer? This study shows us that we must designbetter, more rigorous comparative effectivenessstudies of prostate cancer biomarkers. It is incum-bent on the urological community to adopt the high-est quality standards in study design if we are to bethe stewards of prostate cancer research. This studyshows us we need to do better.
David F. Penson
Department of UrologyVanderbilt University Medical Center
