Interlaboratory Comparison of Real-Time PCR Protocols forQuantification of General Fecal Indicator BacteriaOrin C. Shanks,*,† Mano Sivaganesan,† Lindsay Peed,† Catherine A. Kelty,† A. Denene Blackwood,‡
Monica R. Greene,‡ Rachel T. Noble,‡ Rebecca N. Bushon,§ Erin A. Stelzer,§ Julie Kinzelman,∥
Tamara Anan’eva,∥ Christopher Sinigalliano,⊥ David Wanless,⊥ John Griffith,¶ Yiping Cao,¶
Steve Weisberg,¶ Valarie J. Harwood,▲ Christopher Staley,▲ Kevin H. Oshima,# Manju Varma,#
and Richard A. Haugland#
†U.S. Environmental Protection Agency, Office of Research and Development, National Risk Management Research Laboratory,Cincinnati, Ohio, United States‡University of North Carolina at Chapel Hill, Institute of Marine Sciences, Morehead City, North Carolina, United States§U.S. Geological Survey, Columbus, Ohio, United States∥City of Racine Health Department, Racine, Wisconsin, United States⊥National Oceanic and Atmospheric Administration, Atlantic Oceanographic and Meteorological Laboratories, Ocean ChemistryDivision, Miami, Florida, United States¶Southern California Coastal Water Research Project, Costa Mesa, California, United States▲Department of Biology, University of South Florida, Tampa, Florida, United States#U.S. Environmental Protection Agency, Office of Research and Development, National Exposure Research Laboratory, Cincinnati,Ohio, United States
*S Supporting Information
ABSTRACT: The application of quantitative real-time PCR (qPCR)technologies for the rapid identification of fecal bacteria in environmentalwaters is being considered for use as a national water quality metric in theUnited States. The transition from research tool to a standardizedprotocol requires information on the reproducibility and sources ofvariation associated with qPCR methodology across laboratories. Thisstudy examines interlaboratory variability in the measurement ofenterococci and Bacteroidales concentrations from standardized, spiked,and environmental sources of DNA using the Entero1a and GenBac3qPCR methods, respectively. Comparisons are based on data generatedfrom eight different research facilities. Special attention was placed on theinfluence of the DNA isolation step and effect of simplex and multiplexamplification approaches on interlaboratory variability. Results suggestthat a crude lysate is sufficient for DNA isolation unless environmental samples contain substances that can inhibit qPCRamplification. No appreciable difference was observed between simplex and multiplex amplification approaches. Overall,interlaboratory variability levels remained low (<10% coefficient of variation) regardless of qPCR protocol.
■ INTRODUCTIONThe application of quantitative real-time PCR (qPCR)technologies for the rapid identification of fecal indicatorbacteria (FIB) in environmental waters is being considered foruse as a national water quality metric in the United States.Unlike cultivation techniques that require 18 or more hours togenerate test results, qPCR methods can provide the necessaryinformation to open or close a beach in less than four hours.Shorter sample processing times represent a significantimprovement over conventional culture-based methods, asthey would allow watershed and beach managers to assess waterquality on the same day. As a result, qPCR methods have been
developed to detect and estimate the concentration of key fecalFIB such as enterococci1−3 and Bacteroidales.2,4−6 Thesemethods have been the subject of many research studiesfocusing on the density and distribution of genetic markers inprimary sources of fecal pollution,5,7,8 the detection and decayof DNA targets in fresh and marine water matrices,9−12 andcomparisons between culture and qPCR measurement in paired
Received: September 9, 2011Revised: November 29, 2011Accepted: December 1, 2011Published: December 1, 2011
samples,13−16 as well as the identification of correlationsbetween genetic marker concentrations and associated publichealth risk.17,18 However, little is known about the reproduci-bility and sources of variation in these FIB qPCR methodsacross laboratories.The lack of information available on interlaboratory
performance is, in part, due to the complexity of these qPCRmethods and the lack of consensus among researchers onstandardization of protocols. Some areas of contention amongresearchers include DNA isolation protocols and the use ofsimplex (single gene target) or multiplex (multiple genetargets) amplification approaches. Before a FIB qPCR methodprotocol can be considered for regulatory use, studies must beperformed to characterize the advantages and disadvantages ofthese different protocol options, especially in the context ofinterlaboratory variability. For example, the advantages of amultiplex amplification approach become irrelevant if theprotocol is too complicated to be performed across laboratorieswith a high level of reproducibility.In this study, interlaboratory variability of two qPCR
methods designed to estimate the concentration of enterococciand Bacteroidales in ambient water samples is reported based ondata generated from eight facilities including federal, state, city,and academic laboratories. Each laboratory followed apredetermined series of protocols to generate comparabledata. In addition to using standardized protocols, participatinglaboratories used the same lots of reference DNA sources, DNAisolation kits, and amplification reagents, as well as the sameqPCR thermal cycler instrument model to generate estimates ofFIB on replicate filters from spiked and environmental watersamples. The above instruments, reagents, and protocols werestandardized to help isolate the impact of the DNA isolationstep and simplex or multiplex amplification approaches oninterlaboratory variability.
■ MATERIALS AND METHODSParticipants. Eight laboratories were selected for partic-
ipation including the U.S. EPA National Risk ManagementResearch Laboratory (Cincinnati, OH), U.S. EPA NationalExposure Research Laboratory (Cincinnati, OH), University ofSouth Florida (Tampa, FL), University of North Carolina atChapel Hill (Morehead City, NC), U.S. Geological Survey(Columbus, OH), City of Racine Health Department (Racine,WI), National Oceanic and Atmospheric Administration(Miami, FL), and Southern California Coastal Water ResearchProject (Costa Mesa, CA) and were randomly assignednumbers from one to eight.Assay Selection. Three qPCR assays were included in the
study including Entero1a, GenBac3, and Sketa22 as describedin EPA Method A for Enterococci and EPA Method B forBacteroidales (http://water.epa.gov/scitech/methods/cwa/bioindicators/biological_index.cfm#rapid). Primer and hydrol-ysis probe sequences are listed in Supporting Information TableS1.Scheme Design and Reagent Sets. Participants received
detailed protocols including instructions to complete themultiple laboratory study. All participants were required touse the following: (1) 50 μL of calibration curve DNA plasmidconstruct for Entero1a and GenBac3 containing 5 × 104 copiesper μL prepared by a central laboratory; (2) 500 μL of internalamplification control (IAC) DNA plasmid construct forEntero1a and GenBac3 containing 2 × 102 copies per μLprepared by a central laboratory; (3) BioBall Multishot 550
Enteroccocus faecalis (ATCC 29212; BTF, North Ryde,Australia); (4) BioBall Custom HighDose 10K Bacteroidesthetaiotaomicron (ATCC 29741; BTF, North Ryde, Australia)supplied by a central laboratory; 5) Salmon testis DNA (Sigma-Aldrich; Catalog D7656 or D1626) prepared by individuallaboratories; (6) DNA-EZ DNA purification kit (GeneRite,North Brunswick, NJ; Catalog K102-02-50); (7) TaqManUniversal PCR Master Mix (Applied Biosystems; Catalog4304437); (8) Applied Biosystems StepOne Plus real-timePCR instrument; and (9) test sample filters M, D, Z, G, andS1−S9 provided in triplicate prepared by central laboratory.Test samples were blinded to all participants except one lab(see test sample preparation). Using the required supplies,participants were instructed to (1) generate six individualcalibration curves for Entero1a and GenBac3 qPCR assays, (2)carry out DNA isolation and qPCR amplification protocols forall supplied test samples, and (3) submit raw data to statisticsexpert for analysis.
Preparation of Reference DNA Sources. Five differentreference DNA sources were used in this study. DNA sourcesincluded two plasmid constructs (Integrated DNA Technolo-gies), two BioBall preparations, and salmon testis DNA.Calibration curve and IAC DNA plasmid constructs wereprepared by a single laboratory. Plasmids were linearized byNotI restriction digestion (New England BioLabs, Beverly,MA), quantified with a NanoDrop ND-1000 UV spectropho-tometer (NanoDrop Technologies), and diluted in 10 mM Trisand 0.1 mM EDTA (pH 8.0) to generate 5 × 104 copies/μLand 2 × 102 copies/μL, respectively. Participating laboratorieswere responsible for preparing dilutions of 3.17 × 103, 1 × 103,3.17 × 102, 1 × 102, 3.17 × 101, and 1 × 101 copies/5 μL forthe calibration curve standards, as well as a 50 copy/2 μL IACworking stock solution. BioBall DNA sources were suppliedfrom the same lot and commercial laboratory. Salmon DNAworking stocks containing 10 μg/mL were prepared by eachlaboratory either by dilution of a commercially available 10 mg/mL solution (Sigma-Aldrich D7656) or from lyophilizedmaterial (Sigma-Aldrich D1626). All stock and working stocksolutions were stored in low-retention microtubes at 4 °C untiltime of analysis.
Preparation of Stock Solutions and Test Filters. Stocksolutions of ambient water samples were prepared from fourdifferent marine beach locations along the southern coast ofCalifornia (Supporting Information Table S2) by a centralizedlaboratory. Some water preparations were spiked with E. faecalis(ATCC 29212) and B. thetaiotaomicron (ATCC 29741)cultured cells (samples D, Z, and G) or primary effluentcollected from a local wastewater treatment facility (samplesS4−S9). The number of Enterococcus spp. colony forming units(CFU) per 100 mL was estimated for each water test samplepreparation prior to filtering using the U.S. EPA Method 1600mEI agar approach.19 Water test samples ranged in Enterococcusspp. concentrations from 4 to 1545 CFU/100 mL (SupportingInformation Table S2). Test filters were prepared in triplicatefor each participating group. For each test filter, 100 mL ofwater was filtered through a 47-mm, 0.4-μm pore sizepolycarbonate filter (Osmonics Inc., Catalog K04CP04700).Filters were then placed in sterile 2-mL screw-cap tubescontaining a silica bead mill matrix (GeneRite, Catalog S0205-50), immediately frozen at −80 °C, and then shipped within 24h on dry ice to each participating laboratory.
DNA Isolation. Two different DNA isolation protocolswere investigated in this study. Prior to DNA isolation, 10 μg/
mL stocks of salmon testis DNA were diluted to 0.2 μg/mL inAE buffer. Six hundred microliters of 0.2 μg/mL salmon testisDNA was added to each bead mill tube. Each tube was thensealed, bead milled at 5000 reciprocations/minute for 60 s, andcentrifuged at 12 000g for 1 min to pellet silica beads anddebris. The supernatant was then transferred to a freshmicrotube and centrifuged for an additional 5 min. For thecrude extract (CE) approach, 40 μL of crude lysate was dilutedin 160 μL of 10 mM Tris and 0.1 mM EDTA (pH 8.0) andstored at 4 °C until DNA amplification. For the purified extract(PE) approach, DNA from the remaining bead mill lysate wasisolated using the DNA-EZ kit (GeneRite) according tomanufacturer’s instructions. Purified DNA was eluted off theDNA-Sure columns with 100 μL of EZ elution buffer andstored at 4 °C until DNA amplification.qPCR Amplification. The Entero1a and GenBac3 qPCR
assays were performed in simplex (Enterococcus or BacteroidalesDNA target) and multiplex (Enterococcus or Bacteroidales DNAtarget + IAC target) amplification formats as previouslydescribed.20 The Sketa22 qPCR assay was run in the simplexformat only for use as a DNA isolation efficiency control aspreviously described.20 All reactions were performed intriplicate on a StepOne Plus real-time PCR sequence detector(Applied Biosystems) with a 25-μL reaction volume inMicroAmp Optical 96-well reaction plates with MicroAmp96-well Optical Adhesive Film (Applied Biosystems). Datawere initially viewed with Sequence Detector Software (Version2.3), and quantification cycle (Cq) values (0.03 threshold for allassays) were exported to Microsoft Excel.Identification of Amplification Inhibition and Com-
petition. An internal amplification control (IAC) designed toevaluate the suitability of isolated DNA for qPCR-basedamplification was performed on each test sample DNA extractwith the Entero1a and GenBac3 multiplex IAC qPCR assays aspreviously described.20 The amplification interference criterionfor each assay and laboratory was based on repeatedexperiments measuring the mean Cq of a 50-copy IAC spikein buffer only. Evidence of amplification interference wasdefined as any observed IAC Cq value in a test sample DNAextract greater than the respective mean Cq + 1.5 for a givenassay and laboratory. DNA extracts exhibiting amplificationinterference were further classified into groups affected byinhibition or competition based on observed competitionthresholds, which were calculated separately for the Entero1aand GenBac3 assays, and for each laboratory (see SupportingInformation for details).
Evaluation of DNA Isolation Efficiency. For each testsample filter, the efficiency of DNA isolation was estimatedusing a salmon testis DNA control spike and subsequentamplification with the Sketa22 qPCR assay. A DNA isolationacceptance threshold for each participating laboratory wasestablished based on repeated control experiments wherelaboratory grade water was substituted for ambient water. Anytest sample filter DNA isolation with a Sketa22 Cq measure-ment that differed from the laboratory-specific control mean Cq
± 3 threshold was discarded from the study.Monitoring for Extraneous DNA. To monitor for
potential sources of extraneous DNA during DNA isolationand qPCR amplification, extraction blanks where laboratorygrade water was substituted for ambient water were performedin each participating laboratory over the course of the study.
Calculations and Statistics. Outliers were removed fromdata sets for calibration curve plasmid and BioBall cell calibratorstandardized sources of DNA. The coefficient of determination(R2) and amplification efficiency (E = (10(1/‑slope) − 1) werecalculated for each Entero1a and GenBac3 individual fittedcalibration curve (n = 24 per laboratory). Individual fittedcalibration curves with R2 < 0.90 and/or E values outside therange of 0.70−1.30 were discarded from the study. Theremaining acceptable quality calibration curve data were thenpooled to generate a master fitted calibration curve for eachrespective laboratory and method variant (Entero1a simplex,Entero1a multiplex, GenBac3 simplex, and GenBac3 multiplex).All fitted curves were constructed using a simple linearregression model. The mean log10 number of cell equivalentsper filter for each test sample was estimated using the ΔΔCq
model where cell calibrator and Sketa22 values were allowed tovary by laboratory and instrument run. The Enteroccocus spp.and Bacteroidales reference numbers are assumed to be constantvalues of 583 and 104 cells respectively, based on manufacturerreports. Test sample Cq values below an assay-specific lowerlimit of quantification (LLOQ), defined as the Lab #1 (sourceof centralized reference plasmid DNA) ten-copy calibrationcurve plasmid standard mean Cq, were not included inquantitative analyses. ANOVA was used to identify statisticallysignificant differences in Cq and cell equivalent estimate datasets. Analysis of covariance was done to compare the slopeparameter between fitted master calibration curves. All reportedstatistical analyses were performed by a single laboratory withSAS (Version 9.2; Cary, NC).
Table 1. Summary of Quality Control Metrics and Data Trimminga
metric acceptance criteria total % removed ref
reference DNA outliers absolute value of Cq − mean/SD 7240 1.1% 32calibration curve outliers absolute value of studentized residual >3 3580 1.2% 33
E 0.7−1.3 384 5.5% 22R2 ≥0.90 384 3.4% 22
DNA isolation Sketa22 Cq mean control Sketa22 Cq ± 3 Cq 624 10.1% 20amplification inhibition IAC Cq see Table S4 448 3.8% 21,34range of quantification LLOQ Lab #1 10-copy plasmid mean Cq 2688 3.9% 21extraneous DNA field blanks within ROQ 1408 3.7% 21
aMetric indicates specific quality control utilized. Acceptance criteria lists criterion used for a specific metric to determine inclusion or exclusion of aparticular data point. Total denotes the number of measurements subject to respective quality metric. % removed reports the frequency of failedmeasurements. Ref provides previously published documents used to define quality control metric and acceptance criteria. SD denotes standarddeviation.
■ RESULTSQuality Control Metrics and Data Trimming. In this
study, data generation and analysis were performed inaccordance with the Minimum Information for Publication ofQuantitative Real-Time PCR Experiments (MIQE) guide-lines.21 Quality control metrics and acceptance criteriaemployed for data validation and analysis are summarized inTable 1. E and R2 performance metrics were expanded beyondrecommendations by experts and manufacturers to allow theinclusion of a wider range of results to estimate interlaboratoryvariability. For example, the recommended E acceptance rangeof 0.9−1.122 was expanded to 0.7−1.3. Field blank controlsindicated the absence of extraneous DNA molecules in 96.3%of all amplifications. Forty-two instances (81%) were fromcontrols using the PE DNA isolation protocol.Variability in Reference DNA Sources. Variation in Cq
values are reported for reference DNA sources utilized in thisstudy (Table 2). Reference DNA sources were classified ascentralized (initial stock originated from single lab) ornoncentralized (initial stocks prepared by each lab). TheBioBall and salmon DNA sources were subdivided into CE andPE DNA isolation groups.Generation of Pooled Calibration Curves. Six fitted
calibration curves were generated for each assay andparticipating laboratory using both simplex and multiplexamplification approaches. Each fitted curve was evaluated basedon R2 and E values (Table S3) and low-quality individual fittedcurves were discarded from the study. Pooled calibration curveswere calculated for each laboratory and qPCR methodconsisting of high-quality individual calibration curves to derivethe slope parameter for the ΔΔCq model and associated qualitymetrics (Table S4). For pooled fitted curves, E and R2 qualitymetrics ranged across laboratories from 0.88 to 1.17 and 0.91 to0.98 for Entero1a and from 0.99 to 1.20 and 0.91 to 0.97 forGenBac3, respectively. ANCOVA comparison of pooled fittedcurve slope parameters identified a significant difference (p <0.03) between respective simplex and multiplex slopes forlaboratories #1, #2, and #3 with Entero1a and lab #1 forGenBac3 (see Figure S1 for example).Detection of Amplification Interference with Multi-
plex qPCR. To identify amplification interference, aninterference Cq threshold was calculated for each laboratory,assay, and DNA isolation strategy combination (Table S5). Anested-ANOVA to compare interference Cq thresholds acrosslaboratories indicated significant differences between CE and
PE DNA isolation preparations for laboratories 1, 5, 7, and 8 (p< 0.05) with the Entero1a IAC multiplex qPCR assay and lab 7(p < 0.05) with the GenBac3 IAC multiplex qPCR assay. AnANOVA indicated significant interlaboratory variability (p <0.0001) between IAC ROQ values for Entero1a and GenBac3.A total of 448 test sample DNA extracts were screened for
IAC amplification interference, and interference was detected in173 (38.6%) DNA extracts. IAC assay interference can arisefrom two possible sources including inhibition or competitionbetween the IAC and respective FIB DNA target. Based onlaboratory-specific competition Cq thresholds (Table S5),90.2% of the DNA extracts (n = 156) exhibiting amplificationinterference were attributed to competition of which 73.9%were from the GenBac3 method. Overall, only 17 of the 448(3.8%) test sample DNA extracts showed evidence ofamplification inhibition. All but three of the inhibited DNAextracts were from the CE DNA isolation protocol. Fifty eightpercent of inhibited DNA extracts (n = 10) were from S2 testsample DNA extracts. Based on amplification interferencescreening, all data associated with test sample S2 (alllaboratories) and CE Entero1a data from lab #4 were discardedfrom quantification analysis due to evidence of inhibition.
DNA Isolation Efficiency. Acceptance criteria (meancontrol Sketa22 Cq ± 3 Cq) were allowed to vary by laboratoryand DNA isolation protocol. Test sample Sketa22 Cqmeasurements ranged from 21.6 to 28.9 Cq. Acceptancethresholds ranged from <22.5 to <30.2 Cq across laboratories.Sixty-three test sample DNA extracts (10.1%) failed the DNAisolation efficiency screen and were discarded from quantifica-tion analysis. All failed DNA extracts were prepared by lab #1(PE test samples M, G, D, and Z) and #3 (PE and CE testsamples S1−S9) and attributed to laboratory personnelexperimental error based on deviations from standardizedprotocols.
Lower Limit of Quantification (LLOQ). The LLOQthreshold for each method variant was 36.4 Cq (Entero1a smlx),35.7 Cq (Entero1a mplx), 36.1 Cq (GenBac3 smlx), and 35.7 Cq(GenBac3 mplx). A total of 107 (3.9%)Cq values from testsamples were greater than the respective LLOQ and discardedfrom quantification analysis. All instances occurred withsamples D, M, and S2. The majority of these instances werefrom Entero1a measurements (92.5%).
Estimation of Target Concentration in Spiked andEnvironmental Samples. All spiked and environmentalsamples were analyzed in triplicate by each participating
Table 2. Summary of Mean Cq and Standard Deviation Ranges for Reference DNA Sourcesa
centralized non-centralized
assay type plasmid standard IAC BioBall PE BioBall CE salmon PE salmon CE
mean range Entero1a smlx 32.6−34.9 ND 29.9−30.8 32.4−33.6 ND NDmplx 32.5−35.3 33.0−35.9 30.0−31.7 32.4−33.7
aType denotes multiplex (mplx) or simplex (smlx) approach. Plasmid standard represents 100 copy test quantity. IAC denotes 50 copy test quantityof internal amplification control. PE and CE represent crude and purified extractions, respectively. SD indicates standard deviation. ND depicts nodata available.
laboratory to estimate the mean log10 cell equivalents per filterof Enterococcus spp. and Bacteroidales using the Entero1a andGenBac3 qPCR methods, respectively (Figure 1). When datafrom all test samples were pooled by assay, DNA isolationprotocol, and smlx/mplx approach, an ANOVA indicatedsignificant variability across laboratories for each protocolcombination (p > 0.05). Thus, potential trends in DNAisolation protocols, smlx/mplx approaches, and lab-to-labvariability were analyzed on a sample by sample basis. Tovisualize the degree of interlaboratory variability for eachsample and protocol combination, a percent coefficient ofvariation (%CV) was determined and plotted (Figure 2).Overall, %CV values exceeded 10% across laboratories only 3times and all instances were from the same test sample (S9). Itis interesting to note that the ambient water used to prepare theS9 stock solution was collected from the same location (see
Table S2) as the S2 sample, which showed evidence ofamplification inhibition by multiple laboratories. %CV valueswere less than 5% 56 times (58%).A comparison of mean log10 cell equivalents per filter and
variability from CE and PE DNA isolation protocols is shownin Table 3. Results indicate that CE and PE mean log10estimates were not significantly different across laboratories90.9% of the time and that variability in CE estimates is lessthan or equal to PE 79.5% of the time. A similar pattern wasobserved for mean log10 estimates from mplx and smlxapproaches (Table 4) where values were not significantlydifferent across laboratories 88.6% of the time. In addition,variability in mplx estimates was less than or equal to smlxacross laboratories 88.6% of the time.
Figure 1. Quantification of Enterococcus spp. (Entero1a; panel A) and Bacteroidales (GenBac3; panel B) by simplex approach in spiked andenvironmental replicate samples with CE DNA isolation. Concentration estimates reported as mean log10 cell equivalents per sample. Shapes andcolors indicate respective participating laboratory data.
Quality Control Metrics and Data Trimming. Theimportance of quality control metrics and data trimming forqPCR applications is well recognized.21 Parameters such as E
calculated from a calibration curve slope, R2 of a calibrationcurve, evidence for LLOQ, identification of outliers, and resultsof extraneous DNA controls, as well as evidence of acceptableDNA isolation efficiency and absence of amplification
Figure 2. Plot of interlaboratory percent coefficient of variation (%CV) of mean log10 cell equivalent estimates for each test sample acrosslaboratories for each Entero1a (panel A) and GenBac3 (panel B) method variant. The horizontal solid black line indicates the 10%CV threshold.Symbol shape, color, and lines indicate respective protocol combinations.
Table 3. Comparison of Impact of Crude (CE) and Purified Extraction (PE) Protocols on Mean log10 Estimates and Variabilityby Samplea
mean variability
assay type CE ≠ PE CE < PE CE > PE CE = PE
Entero1a mplx Z G,Z,S9 S1,S4,S6 D,S3,S5,S7,S8smlx G,Z,S5,S9 S1,S4,S6,S7 D,S3,S8
aType indicates mplx or smlx approach. Mean denotes comparison of estimated log10 cell equivalents per 100 mL for spiked and environmentalsamples. Variability denotes comparison of among laboratory variability. CE and PE represent crude extraction and purified extraction protocols. =indicates no significant difference, < indicates significantly lower, and > indicates significantly higher.
interference are required by many peer reviewed journals fordata validation and publication.21 All of these parameters weredefined, measured, and accounted for with each method andassay protocol variant across laboratories in this study andrevealed information about the performance of the Entero1aand GenBac3 protocols. For example, only 7% of individualfitted curves failed the E and R2 criteria (Table S3). A closerexamination of these data shows that 82% of failed curvesoriginated from two laboratories suggesting that calibrationcurves can be consistently reproduced within an accepted rangeof variability among the majority of the participatinglaboratories. The IAC also provides a good example of theutility of quality control metrics where the S2 sample DNAextract accounted for almost 60% of the inhibition interferenceobservations. Presence of inhibition in this sample wascorroborated by five of the participating laboratories illustratingthe value of the IAC approach. It is also worth noting that allinstances of test sample mean estimates below the LLOQ werefrom only three samples (D, M, and S2), which included thetwo lowest concentrations of Enterococcus spp. measured bymembrane filtration Method 1600 (D and M) along with thesample with evidence of amplification inhibition (S2). Theinclusion of quality control metrics in this study provides vitalinformation to validate interlaboratory qPCR data and offers aclear demonstration of how individual control metrics fordifferent steps in a qPCR protocol are combined to increase theconfidence in results.Reliability of Reference DNA Sources. Accurate and
reproducible measurement of reference DNA sources is criticalfor the successful application of any qPCR method includingEntero1a and GenBac3. Reference DNA sources are partic-ularly important for water quality qPCR methods because thereare currently no certified reference DNA materials available,making it challenging to standardize quantification across alarge number of laboratories. The preparation of referenceDNA sources for qPCR begins with the determination of theinitial reference DNA target concentration. Typically, spec-trophotometry or intercalating dyes such as PicoGreen, both ofwhich are reported to introduce variation.23 In addition, thedilution of reference DNA sources for calibration curvegeneration is reported to increase variability, especially atlower concentrations.24
The ΔΔCq calibration model employed in this study relieson the ability to reliably measure reference DNA from plasmidDNA constructs (slope determinant and IAC), DNA isolatedfrom BioBall preparations (y-intercept determinant), andsalmon DNA (DNA isolation control). To help minimizevariability across laboratories, initial concentrations of plasmidDNA constructs were determined by a single laboratory andthen shipped to participating laboratories. Each laboratory then
prepared dilutions to appropriate test concentrations. Althoughthere are numerous reasons to expect some variation inreference DNA source measurements (minor deteriorationduring shipment, freeze/thaw steps, and dilution preparation),these data represent the best opportunity to characterizeinterlaboratory variability related to individual laboratorytechnical ability. A comparison of Cq standard deviationsamong participating laboratories indicated that individuallaboratory values exceeded 1.5 Cq only 1.6% of the time (2of 128 instances) and that values were less than 1.0 Cq 84.4% ofthe time regardless of assay, laboratory, or reference DNAsource. In addition, lab-to-lab Cq variances ranged from 0.02 to1.01 in centralized reference DNA sources. The consistentlylow variability within and among laboratories observed in thisstudy suggests that reference DNA sources can be highlyreproducible, especially when provided by a centralizedlaboratory in a stable form that requires minimal handling.
Importance of the DNA Isolation Protocol. The DNAisolation protocol determines the concentration and quality ofDNA recovered from an environmental sample. Two generalstrategies are used in water quality testing including beadmilling, dilution, and amplification of the resultant crude lysate(CE) or protocols designed to purify and concentrate the DNAtarget (PE). The CE approach can be performed in as little as 5min, where PE techniques require anywhere from 30 to 60 minto complete. Thus, the CE approach would allow water qualitymanagers to report beach water quality almost an hour faster. ACE protocol also requires less technical training and fewersample manipulations, which should reduce the frequency oflaboratory error and potential cross-contamination of valuablewater samples. However, the presence of qPCR inhibitors inthe sample matrix can hamper or even prevent amplification.25
Interlaboratory data in this study illustrates this conundrumwhere 81% of false positives in field blanks originate from thePE protocol, but 82.4% of DNA extracts exhibiting inhibitionwere from CE prepared DNA extracts. CE and PE preparationsappear to have a minimal impact on estimating Entero1a andGenBac3 genetic marker concentrations in spiked andenvironmental samples in this study (>90% agreement amonglaboratories). In addition, variance estimates between CE andPE BioBall reference DNA preparations never exceeded 0.51 Cq
suggesting that neither approach offers a decisive advantage interms of variability. Ultimately, the decision to employ a CEapproach hinges on the anticipated absence of qPCR inhibitorsin the water body of interest. It is also worth noting that bothqPCR methods in this study include BSA and salmon testisDNA, two additives reported to combat inhibitory effects.26,27
The occurrence of inhibition will most likely be reduced furtherin future studies as researchers explore the use of engineered
Table 4. Comparison of Impact of Multiplex (mplx) and Simplex (smlx) Approaches on Mean Estimates and Variability bySamplea
Entero1a CE Z S9 D,G,Z,S1,S3−S8PE S7,S8 S9 G,S6 D,Z,S1,S3−S5,S7,S8
GenBac3 CE Z,S6 D,S1,S4,S7−9 G,Z,S3,S5,S6PE G,S9 S4,S6 D,Z,S1,S3,S5,S7,S8
aType indicates CE or PE DNA isolation protocol. Mean denotes comparison of estimated log10 cell equivalents per 100 mL for spiked andenvironmental samples. Variability denotes comparison of among laboratory variability. Mplx and smlx represent multiplex and simplex amplificationapproaches. = indicates no significant difference, < indicates significantly lower, and > indicates significantly higher.
polymerases selected to be resistant to common environmentalinhibitor effects.28
Effect of Multiplexing. Multiplex qPCR amplificationentails the simultaneous detection of two more target DNAmolecules in the same reaction. A multiplex approach canreduce the cost of sample analysis by combining more than oneassay into a single reaction, and it also allows for the inclusionof an internal control to test for amplification interference, anecessary data validation parameter.21 Variations in mplx testsample mean estimates were not significantly different (p >0.05) from parallel smlx measurements more than 88% of thetime and between laboratory variance for reference DNAsources measured by a mplx approach never exceeded 1.0 Cq.High levels of agreement among laboratories combined withlow levels of variability suggest that mplx applications arereliable across laboratories if potential competition betweenFIB target DNA and the IAC is identified and accounted for.There is always the potential for competition between assays
in mplx reactions, especially when both DNA targets use acommon set of primers. Competition can result in changes in E,LLOQ, and partial or even complete amplification inhibition.Thus, it becomes critical to assess the influence of DNA targetsacross the range of concentrations used in a particular multiplexapplication. Detailed experiments were employed to estimate Efrom mplx amplifications with FIB genetic marker concen-trations ranging from 10 to 3.17 × 103 copies per reaction inthe presence of an IAC 50 copy spike. A comparison of smlxand mplx slopes and competition thresholds (Figure 1 andTable S4) indicated that there can be a significant difference (p< 0.05) among smlx/mplx approaches, assays, and laboratoriessuggesting that these parameters should not be fixed acrosslaboratories or assays during implementation.Implications for Implementing qPCR-Based Water
Quality Monitoring Method. This study characterizedinterlaboratory variability of the Entero1a and GenBac3qPCR methods and we found that mean log10 estimates rarelyexceeded 10% CV regardless of method or assay variant (Figure2). This value is similar in magnitude to differences previouslyreported for single laboratory replicated qPCR experiments16
suggesting that the two DNA isolation protocols oramplification approaches (simplex and multiplex) investigatedin this study do not dramatically increase variability in meanlog10 estimates of genetic marker concentration in test samples.Although interlaboratory variability was relatively low in thisstudy, we still observed that two laboratories performed poorlyin comparison to other participating groups. Decreasedperformance appears to be more associated with incorrectimplementation of the qPCR protocols rather than inherentmethod interlaboratory variability. For example, one of thelaboratories was unable to finish sample analysis in a single dayand deviated from the standardized protocol by refrigeratingcrude extracts overnight before completing sample processing.This might not be a concern for purified DNA extracts, butcrude lysates can contain nucleases released during bead millingthat can rapidly degrade DNA template and potentially reducesample processing efficiency. Similarly, another laboratory usedreconstituted BioBall preparations after storing them overnightat 4 °C, rather than preparing a fresh stock daily as directed inthe standardized protocol, which may increase variability in cellcalibrator measurements. Neither of the above practices arerecommended, meaning that the actual interlaboratory methodvariability may even be lower than we present. On the otherhand, interlaboratory variability may be underestimated because
the eight participating laboratories in this study are professionalresearch groups with more extensive qPCR experience than atypical regional, state, or city facility. In addition, this studyfocuses on variability associated with technical implementation,using reagents from the same lot and identical thermal cyclinginstruments. Before any of these protocols can be adopted by aregulatory agency, a single FIB qPCR method protocol must beselected and tested across multiple laboratories. Futureinterlaboratory studies must also measure the impact of usingdifferent instruments, reagents, and consumables (plasticware),factors that have been shown to dramatically influence qPCRresults among some manufacturers.29−31
Interlaboratory experiments also illustrated a number offactors that should be considered in the technology transferprocess. First, detailed written procedures need to bedeveloped, as we discovered some differences in qPCRprotocol implementation among the eight participatinglaboratories. Second, there is a need for more training thanjust publishing a detailed written protocol, as illustrated by theperformance differences even among experienced laboratories.Novice users are likely to require a combination of classroomand hands-on training, even if they already have somefamiliarity with qPCR. Perhaps most important, experimentsreinforce the need for a centralized source of reference DNAmaterials and the importance of establishing laboratoryproficiency benchmarks prior to implementation, raising thequestion of what is an acceptable amount of variability amonglaboratories. To include as much interlaboratory data aspossible, broad quality control metric acceptance ranges (i.e.,outlier definitions, E, R2, amplification interference threshold,and DNA isolation acceptance range) were employed in thisstudy. Inflated acceptance ranges undoubtedly increasedinterlaboratory variability and contributed to the observationthat different laboratories can yield significantly different meanlog10 cell equivalent estimates from replicate filters. Futureinvestigations designed to characterize the balance among morestringent quality control metrics, interlaboratory variability, andprotocol feasibility are warranted.This study examined interlaboratory variability based on the
measurement of enterococci and Bacteroidales concentrationsfrom standardized, spiked, and environmental sources of DNAusing the Entero1a and GenBac3 qPCR methods, respectively.Special attention was placed on the influence of the DNAisolation step and effect of simplex and multiplex amplificationapproaches on interlaboratory variability. Results indicated thatinterlaboratory variability differences among protocols tested inthis study are relatively low and that major differences amonglaboratories were attributed to experimental error due todeviations in the execution of standardized protocols. Thesefindings should help regulatory agencies decide on a singleqPCR protocol, and recognize the importance of standardizedprotocols, quality control metrics, and proficiency of laboratorypersonnel.
■ ASSOCIATED CONTENT*S Supporting InformationThis material is available free of charge via the Internet athttp://pubs.acs.org.
We thank Rick Naher and Applied Biosystems for the use ofreal-time PCR instruments.
■ REFERENCES(1) Ludwig, W.; Schleifer, K. H. How quantitative is quantitativePCR with respect to cell counts? Syst. Appl. Microbiol. 2000, 23 (4),556−562.(2) Siefring, S. C.; Varma, M.; Atikovic, E.; Wymer, L. J.; Haugland,R. A. Improved real-time PCR assays for the detection of fecalindicator bacteria in surface waters with different instrument andreagent systems. J. Water Health 2008, 6, 225−237.(3) He, J. W.; Jiang, S. Quantification of enterococci and humanadenoviruses in environmental samples by real-time PCR. Appl.Environ. Microbiol. 2005, 71 (5), 2250−2255.(4) Dick, L. K.; Field, K. G. Rapid Estimation of Numbers of FecalBacteroidetes by Use of a Quantitative PCR Assay for 16S rRNA Genes.Appl. Environ. Microbiol. 2004, 70 (9), 5695−5697.(5) Kildare, B. J.; Leutenegger, C. M.; McSwain, B. S.; Bambic, D. G.;Rajal, V. B.; Wuertz, S. 16S rRNA-based assays for quantitativedetection of universal, human-, cow-, and dog-specific fecalBacteroidales: A Bayesian approach. Water Res. 2007, 41, 3701−3715.(6) Layton, A.; McKay, L.; Williams, D.; Garrett, V.; Gentry, R.;Sayler, G. Development of Bacteroides 16S rRNA gene TaqMan-basedreal-time PCR assays for estimation of total, human, and bovine fecalpollution in water. Appl. Environ. Microbiol. 2006, 72 (6), 4214−4224.(7) Shanks, O. C.; White, K.; Kelty, C. A.; Sivaganesan, M.; Blannon,J.; Meckes, M.; Varma, M.; Haugland, R. A. Performance of PCR-based assays targeting Bacteroidales genetic markers of human fecalpollution in sewage and fecal samples. Environ. Sci. Technol. 2010, 44(16), 6281−6288.(8) Silkie, S. S.; Nelson, K. L. Concentrations of host-specific andgeneric fecal markers measured by quantitative PCR in raw sewage andfresh animal feces. Water Res. 2009, 43, 4860−4871.(9) Bell, A.; Layton, A. C.; McKay, L.; Williams, D.; Gentry, R.;Sayler, G. S. Factors influencing the persistence of fecal Bacteroides instream water. J. Environ. Qual. 2009, 38 (3), 1224−1232.(10) Green, H.; Shanks, O. C.; Sivaganesan, M.; Haugland, R.; Field,K. Extended survival of human fecal Bacteroides in marine water.Environ. Microbiol. 2011, 13 (12), 3235−3249.(11) Schulz, C. J.; Childers, G. W. Fecal Bacteroidales diversity anddecay in response to variations in temperature and salinity. Appl.Environ. Microbiol. 2011, 77 (8), 2563−2571.(12) Walters, S. P.; Yamahara, K. M.; Boehm, A. B. Persistence ofnucleic acid markers of health-relevant organisms in seawatermicrocosms: Implications for their use in assessing risk in recreationalwaters. Water Res. 2009, 43, 4929−4939.(13) Viau, E.; Peccia, J. Evaluation of the enterococci indicator inbiosolids using culture-based and quantitative PCR assays. Water Res.2009, 43 (19), 4878−4887.(14) Frahm, E.; Obst, U. Application of fluorogenic probe technique(TaqMan PCR) to the detection of Enterococcus spp. and Escherichiacoli in water samples. J. Microbiol. Methods 2003, 52, 123−131.(15) Whitman, R. L.; Ge, Z.; Nevers, M. B.; Chern, E. C.; Haugland,R.; Lukasik, A. M.; Molina, M.; Przybyla-Kelly, K.; Shively, D. A.;White, E. M.; Zepp, R. G.; Byappanahalli, M. N. Relationship andvariation of qPCR and culturable enterococci estimates in ambientsurface waters are predictable. Environ. Sci. Technol. 2010, 44, 5049−5054.(16) Haugland, R. A.; Siefring, S. C.; Wymer, L. J.; Brenner, K. P.;Dufour, A. P. Comparison of Enterococcus measurements in freshwaterat two recreational beaches by quantitative polymerase chain reactionand membrane filter culture analysis. Water Res. 2005, 39, 559−568.(17) Wade, T. J.; Calderon, R. L.; Brenner, K. P.; Sams, E.; Beach,M.; Haugland, R.; Wymer, L.; Dufour, A. P. High sensitivity ofchildren to swimming-associated gastrointestingal illness: Results usinga rapid assay of recreational water quality. Epidemiology 2008, 19 (3),375−383.
(18) Wade, T. J.; Calderon, R. L.; Sams, E.; Beach, M.; Brenner, K.P.; Williams, A. H.; Dufour, A. P. Rapidly measured indicators ofrecreational water quality are predictive of swimming-associatedgastrointestinal illness. Environ. Health Perspect. 2006, 114, 24−28.(19) USEPA.Method 1600: Enterococci in water by membrane filtrationusing membrane-enteroccucus indoxyl-D-gludoside agar (mEI); U.S.Environmental Protection Agency, Office of Water: Washington,DC, 2006.(20) Haugland, R. A.; Varma, M.; Kelty, C. A.; Peed, L.; Sivaganesan,M.; Shanks, O. C. Evaluation of genetic markers from the 16S rRNAgene V2 region for use in quantitative detection of selectedBacteroidales species and human fecal waste by real-time PCR. Syst.Appl. Microbiol. 2010, 33, 348−357.(21) Bustin, S. A.; Benes, V.; Garson, J. A.; Hellemans, J.; Huggett, J.;Kubista, M.; Mueller, R.; Nolan, T.; Pfaffl, M. W.; Shipley, G. L.;Vandesompele, J.; Wittwer, C. T. The MIQE Guidelines: Minimuminformation for publication of quantitative real-time PCR experiments.Clin. Chem. 2009, 55 (4), 611−622.(22) AppliedBiosystems. Real-time PCR: Understanding Ct; LifeTechnologies: Carlsbad, CA, 2011.(23) Wiseman, G. State of the art and lmitation of quantitativepolymerase chain reaction. J. AOAC Int. 2002, 85, 792−796.(24) Singer, V. L.; Jones, L. J.; Yue, S. T.; Haugland, R. P.Characterization of PicoGreen reagent and development of afluorescence-based solution assay for double-stranded DNA quantita-tion. Anal. Biochem. 1997, 249, 228−238.(25) Wilson, I. G. Inhibition and facilitation of nucleic acidamplification. Appl. Environ. Microbiol. 1997, 63, 3741−3751.(26) Al-Soud, W. A.; Radstrom, P. Effects of amplification facilitatorson diagnostic PCR in the presence of blood, feces, and meat. J. Clin.Microbiol. 2000, 38, 4463−4470.(27) Kreader, C. A. Relief of amplification inhibition in PCR withbovine serum albumin or T4 gene 32 protein. Appl. Environ. Microbiol.1996, 62, 1102−1106.(28) Baar, C.; d’Abbadie, M.; Vaisman, A.; Arana, M. E.; Hofreiter,M.; Woodgate, R.; Kunkel, T. A.; Holliger, P. Molecular breeding ofpolymerases for resistance to environmental inhibitors. Nucleic AcidsRes. 2011, 39 (8), e51.(29) Reiter, M.; Pfaffl, M. W. Effects of plate position, plate type andsealing systems on real-time PCR results. Biotechnol. Biotechnol. Equip.2008, 22, 824−828.(30) Kim, Y. H.; Yang, I.; Bae, Y. S.; Park, S. R. Performanceevaluation of thermal cyclers for PCR in a rapid cycling condition.Biotechniques 2008, 44, 495−500.(31) Saunders, G. C.; Dukes, J.; Parkes, H. C.; Cornett, J. H.Interlaboratory study on thermal cycle performance in controlled PCRand random amplified polymorphic DNA analyses. Clin. Chem. 2001,47, 47−55.(32) Smirnov, N. V.; Dunin-Barkovskii, I. V. Matehmatische Statiskikin der Technik; Verlag Wissenschaft: Deutsch, 1969.(33) Clark, R. M.; Sivaganesan, M. Predicting chlorine residuals indrinking water: second order model. J. Water Resour. Plan. Manage.2002, 128 (2), 152−161.(34) Shanks, O. C.; Atikovic, E.; Blackwood, A. D.; Lu, J.; Noble, R.T.; Santo Domingo, J.; Siefring, S.; Sivaganesan, M.; Haugland, R. P.Quantitative PCR for Detection and Enumeration of Genetic Markersof Bovine Fecal Pollution. Appl. Environ. Microbiol. 2008, 74 (3), 745−752.
■ NOTE ADDED AFTER ASAP PUBLICATIONThis article was published ASAP on December 16, 2011, withminor text errors in the Results section and in the caption ofFigure 1. The correct version was published ASAP on January6, 2012.
