Computer Sciences Corporation www.csc.com Federal Sector Civil Systems Development Division 6101 Stevenson Avenue Alexandria, Virginia 22304 703.461.2000 Fax 703.461.2020 MEMORANDUM DATE: October 17, 2011 TO: Marion Kelly, EAD and Alice Yeh, Region 2 FROM: Lynn Walters, Harry McCarty, and Ken Miller, CSC, and Dale Rushneck, Interface SUBJECT: Assessment of the Proposed Correction Factor for Dioxin/Furan Results from Lower Passaic River Sediment Split Samples in Light of CPG Comments As requested, CSC has performed the following additional evaluation of the split sample results from the Lower Passaic River Sediment Samples: • Validating the AXYS and CAS split sample data pairs that were used to determine the dioxin/furan correction factors. The validation was performed in accordance with the validation level specified in the CPG QAPP • Responding to the issues in the CPG memo dated 6/6/11 • Re-evaluating the dioxin/furan correction factors that we proposed be used to correct the CAS data in light of our validation of the AXYS and CAS data. Data Validation CSC originally used a total of 30 split samples to develop the correction factor cited in CSC’s January 2011 report. a The results from AXYS were originally provided in two data packages. The results from CAS were spread across 18 data packages. As requested by EAD and Region 2, we reviewed and validated the results from both laboratories as described in the CPG QAPP provided to us. We also used the EPA Region 2 SOP and checklist for review of PCDD/PCDF results, and examined the QAPP from Malcolm Pirnie in reviewing the AXYS results. Finally, we compared many of the QAPP and Region 2 specifications to the details of EPA Method 1613B, which both laboratories used for sample analysis (with previously noted modifications). During the course of our validation, we identified aspects of the results and the analyses by both laboratories that were not in exact concurrence with the CPG QAPP or the Region 2 SOP. However, we caution that: • AXYS was bound by the Malcolm Pirnie QAPP, and was not bound by (and may not have even been aware of) the CPG QAPP a The Effect of Application of a Correction Factor on Chlorinated Dibenzo-p-Dioxins and Dibenzofuran Results Produced by Columbia Analytical Services for Lower Passaic River Sediment Samples
Transcript
Computer Sciences Corporation www.csc.com
Federal Sector Civil Systems Development Division 6101 Stevenson Avenue Alexandria, Virginia 22304 703.461.2000 Fax 703.461.2020
MEMORANDUM DATE: October 17, 2011 TO: Marion Kelly, EAD and Alice Yeh, Region 2 FROM: Lynn Walters, Harry McCarty, and Ken Miller, CSC, and Dale Rushneck, Interface SUBJECT: Assessment of the Proposed Correction Factor for Dioxin/Furan Results from Lower
Passaic River Sediment Split Samples in Light of CPG Comments As requested, CSC has performed the following additional evaluation of the split sample results from the Lower Passaic River Sediment Samples: • Validating the AXYS and CAS split sample data pairs that were used to determine the dioxin/furan
correction factors. The validation was performed in accordance with the validation level specified in the CPG QAPP
• Responding to the issues in the CPG memo dated 6/6/11 • Re-evaluating the dioxin/furan correction factors that we proposed be used to correct the CAS data in
light of our validation of the AXYS and CAS data. Data Validation CSC originally used a total of 30 split samples to develop the correction factor cited in CSC’s January 2011 report.a The results from AXYS were originally provided in two data packages. The results from CAS were spread across 18 data packages. As requested by EAD and Region 2, we reviewed and validated the results from both laboratories as described in the CPG QAPP provided to us. We also used the EPA Region 2 SOP and checklist for review of PCDD/PCDF results, and examined the QAPP from Malcolm Pirnie in reviewing the AXYS results. Finally, we compared many of the QAPP and Region 2 specifications to the details of EPA Method 1613B, which both laboratories used for sample analysis (with previously noted modifications). During the course of our validation, we identified aspects of the results and the analyses by both laboratories that were not in exact concurrence with the CPG QAPP or the Region 2 SOP. However, we caution that:
• AXYS was bound by the Malcolm Pirnie QAPP, and was not bound by (and may not have even been aware of) the CPG QAPP
a The Effect of Application of a Correction Factor on Chlorinated Dibenzo-p-Dioxins and Dibenzofuran Results Produced by Columbia Analytical Services for Lower Passaic River Sediment Samples
Correction Factor Assessment 2
• Both laboratories reported results that did not meet every specific item in the Region 2 SOP and checklist. In part, this is because that SOP is largely directed at analyses of soil and water samples by CLP laboratories, and the samples in question were very wet sediments. In other cases, the issues identified based on the Region SOP and checklist are trivial and do not affect data quality (e.g., an administrative requirement that every data package have a CLP Case Number and SDG Number)
Details of our data validation findings are provided data review narratives we prepared for laboratory’s data set. Copies of these narratives are being sent to you via email. Based on our validation of the 30 pairs of samples, our overall conclusion is that minor qualifiers should be applied to some results from both laboratories. We did identify some AXYS results that did not meet a strict interpretation of the method blank requirements of the Region 2 SOP and could be rejected based on the blank level alone. However, we also noted that for 8 of the 9 associated AXYS samples, the sample results were 20 to 900 times the level found in the method blank, and it seems unlikely that the sample results were driven by the blank itself. Likewise, one sample from AXYS might be removed from the calculations based on concerns about a laboratory duplicate analysis that was not required by the method, but performed by AXYS. Although we concluded that few of the split sample results from either lab had major data quality issues that warranted rejection and/or precluded their use, we considered the effects of removing those results from the correction factor calculations, as illustrated later in this memorandum. Issues Raised by CPG The memo from the CPG dated 6/6/11 summarizes concerns raised by CPG after they subjected the AXYS data to formal validation by an outside party, Laboratory Data Consultants (LDC). Their concerns ranged from incomplete data packages to questions about individual analyte results in specific samples that LDC believed should be flagged as estimates. The overall premise of the CPG memo was that EPA (and CSC) must reconsider using the AXYS results for decision making, and they questioned the effects of the alleged data quality issues on CSC proposed correction factor approach. CSC examined the LDC concerns in great detail. We prepared a response to each of the 22 points in the CPG memo, and we are sending that to you along with this memorandum. Our overall response is that few of the issues raised by the LDC validation prevent the use of the specific split sample results we considered when we developed the proposed correction factor. We agreed with LDC that results for some analytes in some samples should be considered as estimated values because of minor QC concerns. As noted above, we have considered the impacts of removing those results from the correction factor calculations. Re-evaluation of the Correction Factors General information. As noted in our original report in March 2010, we calculated the ratios of the CAS and AXYS results only for those split sample pairs for which both laboratories’ results were greater than their corresponding quantitation limits, to assure that the ratios reflect the systematic differences between laboratories’ results, rather than a comparison of quantitation limits. That approach means that values flagged as estimates (J flagged) to advise the data user that the value is below the quantitation limit, but above the detection limit, were not used in the correction factors. While results qualified as estimated for other reasons, such as the minor QC issues noted by LDC, may be less certain than similar values that are not flagged, these values were present in the data from both laboratories. In developing the correction factors, we used the geometric mean of the sample-specific
Correction Factor Assessment 3
ratios between the CAS and AXYS results for a given congener as the best available means to adjust the CAS data to address the disparity, to prevent the correction factor(s) from being overly affected by a few high or low concentrations, and which also addresses concerns that some of those values may be less exact than others. EMPCs. Both laboratories reported a few results as estimated maximum possible concentrations, or EMPCs. These occurred when chromatographic peaks at the retention time of a dioxin or furan is present, but the ratio of their areas does not meet the method identification criteria. The concern is that an interference may be present for one of the two ions being monitored that its present increases the area of that peak, which is ultimately reflected in the reported concentration as a high bias. In all cases, both laboratories flagged those EMPCs to warn the data user of the concern. During validation, those results were also flagged as estimated values (J flagged). CSC did not use any EMPC data reported by AXYS when we initially developed the correction factor recommended in our January 2011 report. This was because each of the AXYS EMPC values were either less than the quantitation limit, or were associated with a CAS result that was less than their quantitation limit. However, CSC did use some of the EMPC data reported by CAS when we initially developed our correction factors. The impacts of these data on our calculated correction factor are described below. Moisture Content. Finally, the Region 2 SOP describes qualifying results based on the moisture content of the sample. Specifically, it states:
As per Region II requirements, if any sample analyzed as a soil, contains 50% to 90% water, all data shall be flagged as estimated “J.” If a soil sample contains more than 90% water, then qualify positive hits “J”, and nondetects “UJ.”
The SOP never provides a rationale for this approach, and as we discussed at length with EPA and the CPG in the earlier stages of this effort, it is not based on anything in Method 1613B, which specifies the use of the SDS extractor to minimize the effects of moisture in solid samples. If one accepts the SOP’s approach of flagging sample results for moisture content, it calls into question many of the results from either laboratory, as well as the decision to analyze wet sediment samples at all. In our original evaluation of the results from both laboratories, we looked for potential correlations between analyte results and the moisture content of the samples, both within each laboratory and across both laboratories. We found no relationship between the results and moisture content at that time. (It may also be worth noting that none of the 30 pairs contained anywhere near 90% moisture; the highest moisture content reported by CAS was 65.6% and the highest reported by AXYS was 62.3%.) Therefore, given our earlier findings and the lack of a clear rationale in the Region 2 SOP, CSC did not consider any J flags applied because of moisture content as a reason to eliminate split sample results from the correction factor calculations. Impact of flagged results on CSC’s correction factors. To address concerns raised by CPG, we considered the impact of the J-flagged results on our original correction factor calculations. Overall, we noted 12 results, for 7 analytes, from the 30 splits samples where the CAS data were J-flagged for QC concerns other than moisture content or results reported below the QL (e.g., EMPCs and lab control sample results that were outside method-specified criteria). While there were no similarly flagged AXYS results, we identified ten 2,3,7,8-TCDD results associated with blank contamination or laboratory duplicate results that did not meet criteria. After eliminating those 12 CAS and 10 AXYS results, we recalculated the correction factors. The table below is patterned after Table 1 in our January 2011 report to EPA. This version shows the original number of split sample results for each analyte, the original correction factors. We added new columns for the revised number of the split samples results (after
Correction Factor Assessment 4
removing CAS J-flagged data, and the AXYS results associated with the one contaminated method blank and one laboratory duplicate failure), the new correction factors, and the percent difference between the original and new correction factors. The new data for the affected analytes are shown in red.
PCDD/PCDF January 2011 Version October 2011 Re-evaluation
Overall, the effect of removing those 22 results was minor. For five of the analytes, the change in the correction factor was less than 5%, with two correction factors going down and three going up. For a sixth analyte, 2,3,7,8-TCDD, the correction factor increased by 8.4%. The changes were much more pronounced for 1,2,3,4,7,8,9-HpCDF (-20.1%) and 1,2,3,7,8-PeCDF (+92.9%). For both these analytes, there were far fewer paired results to begin with, so the removal of one CAS result for each analyte had a much greater effect. At this time, we have not recalculated the correlations between the split sample results, or estimated the increased variability of corrected results, as we had in our January 2011 report, because we do not believe that the removal of all 22 of these results is necessarily appropriate, and we wish to give Region 2 the opportunity to consider the effects of the removal. In particular, as we note above, there are eight AXYS results for 2,3,7,8-TCDD associated with one contaminated method blank where the sample results are 20 to 900 times the blank level. Perhaps ironically, removal of those AXYS results actually increased the correction factor that 2,3,7,8-TCDD, as shown in the table above. Conclusions No data set is perfect. Our review of the AXYS and CAS results for the 30 split samples identified minor data quality concerns with some analytes, a few non-substantive administrative concerns that have no demonstrable affects on data quality, and a few more significant concerns that may warrant removal of some data from the correction factor calculations. We also considered the 22 points raised in the CPG memo and believe that few represent issues that affect the actual split sample results used in our correction factor calculations. Our rationale for rebutting those concerns in provided in an attached document. Finally, we removed a small number of J-flagged CAS results and AXYS results associated with identified issues from the data originally used to develop the correction factors and recalculated those factors. For six analytes, no data were removed. For five other analytes, removal changed the factors by less than 5%. For 2,3,7,8-TCDD, removal increased the factor by 8.4%. Only two analytes exhibited changes greater than 20%, due to very small numbers of split sample results for those analytes.
Correction Factor Assessment 5
Given those levels of changes, we do not believe it is necessary or productive to remove the data that were J-flagged only due to the percent moisture content of the samples. Moreover, we presume that there are many other J-flagged results among the hundreds of other samples analyzed for CPG at this site. The CPG QAPP does not state that J-flagged data would not be used, and it is common practice to use J-flagged data. The implications of removal of all J-flagged results could be more significant for the project as a whole than the potential changes to the proposed correction factors. Attachments • Responses to Specific Comments Raised by the CPG in “Attachment 2 - LRC Split Data Detailed
Summary” • Summary of CSC Review of AXYS Data from the 30 Sample Splits used to Generate Correction
Factors • Summary of CSC Review of CAS Data from the 30 Sample Splits used to Generate Correction
Factors
AXYS Review Summary 1 10/2011
Summary of CSC Review of AXYS Data from the 30 Sample Splits Used to Generate Correction Factors
CSC reviewed results from 30 samples analyzed by AXYS Analytical Services for dioxins and furans using EPA Method 1613B. Results of CSC’s data quality assessment are described below. SAMPLES REVIEWED
Sample IDs for the 30 sample reviewed by CSC, along with corresponding lab identification numbers, are provided below. Sample ID # Laboratory ID # Collection Date Receipt Date Comments 08A-0038-C1ES-MP L11879-1 10/21/2008 10/23/2008 08A-0020-C2CS-MP L11938-1 11/4/2008 11/4/2008 08A-0020-C2DS-MP L11938-2 11/4/2008 11/4/2008 08A-0020-C2ES-MP L11938-3 11/4/2008 11/4/2008 Lab duplicate sample
(WG26923-103) 08A-0051-C2CS-MP L11651-2 8/26/2008 8/26/2008 08A-0050-C1BS-MP L11681-1 9/3/2008 9/3/2008 08A-0050-C1CS-MP L11681-2 9/3/2008 9/3/2008 08A-0089-C2BS-MP L11750-1 9/17/2008 9/18/2008 08A-0089-C2CS-MP L11750-2 9/17/2008 9/18/2008 08A-0019-C3DS-MP L11781-3 9/24/2008 9/25/2008 08A-0098-C1ES-MP L11810-1 10/1/2008 10/4/2008 08A-0098-C1FS-MP L11810-2 10/1/2008 10/4/2008 08A-0019-C3BS-MP L11781-1 9/24/2008 9/25/2008 08A-0019-C3CS-MP L11781-2 9/24/2008 9/25/2008 According to the Chain of Custody (COC), some samples arrived at the laboratory above the recommended temperature; however, the cooler temperature was not recorded on the COCs. Upon arrival at the laboratory, all samples were stored at -20 °C. DATA REVIEW PROCESS CSC reviewed and qualified the data in accordance with both the USEPA Region II SOP No. HW-25 Revision 3 and selected worksheets (#12, #15, #19, and #24) in the Quality Assurance Project Plan: RI Low Resolution Coring/Sediment Sampling, Lower Passaic River Restoration Project RI/FS.
AXYS Review Summary 2 10/2011
According to the Region 2 SOP, a matrix spike/matrix spike duplicate (MS/MSD) should be performed. Because Method 1613 is an isotope dilution method, however, MS/MSD analyses are not included as a required QC procedure in the method itself. According to the Final Quality Assurance Project Plan, CPG Oversight, Lower Passaic River Restoration Project (LPRRP) 2008 Sediment Coring, prepared by Malcolm Pirnie and dated July 2008, AXYS was not required to analyze MS/MSD samples. This is noted in item 7 below. The following abbreviated notations have been used through this data review summary to identify target compounds: Abbreviated Notation Full Target Analyte Name 2,3,7,8-TCDD 2,3,7,8-Tetrachlorordibenzo-p-dioxin 2,3,7,8-TCDF 2,3,7,8-Tetrachlorodibenzofuran 1,2,3,7,8-PeCDD 1,2,3,7,8-Pentachlorodibenzo-p-dioxin 1,2,3,7,8-PeCDF 1,2,3,7,8-Pentachlorodibenzofuran 2,3,4,7,8-PeCDF 2,3,4,7,8-Pentachlorodibenzofuran 1,2,3,4,7,8-HxCDD 1,2,3,4,7,8-Hexachlorodibenzo-p-dioxin 1,2,3,6,7,8-HxCDD 1,2,3,6,7,8-Hexachlorodibenzo-p-dioxin 1,2,3,7,8,9-HxCDD 1,2,3,7,8,9-Hexachlorodibenzo-p-dioxin 1,2,3,4,7,8-HxCDF 1,2,3,4,7,8-Hexachlorodibenzofuran 1,2,3,6,7,8-HxCDF 1,2,3,6,7,8-Hexachlorodibenzofuran 1,2,3,7,8,9-HxCDF 1,2,3,7,8,9-Hexachlorodibenzofuran 2,3,4,6,7,8-HxCDF 2,3,4,6,7,8-Hexachlorodibenzofuran 1,2,3,4,6,7,8-HpCDD 1,2,3,4,6,7,8-Heptachlorodibenzo-p-dioxin 1,2,3,4,6,7,8-HpCDF 1,2,3,4,6,7,8-Heptachlorodibenzofuran 1,2,3,4,7,8,9-HpCDF 1,2,3,4,7,8,9-Heptachlorodibenzofuran 1,2,3,4,6,7,8,9-OCDD 1,2,3,4,6,7,8,9-Octachlorodibenzo-p-dioxin 1,2,3,4,6,7,8,9-OCDF 1,2,3,4,6,7,8,9-Octachlorodibenzofuran In the following sections, QC failures, resulting qualifiers, and associated results are described for each deviation from specified requirements. However, if a reviewer has chosen a different qualifier for a failure because of best professional judgment, then the most severe qualifier will be applied. Qualifiers from most severe to least severe are: “R,” “NJ,” “UJ,” “U,” “J.”
1. DATA COMPLETENESS No major problems were found. All relevant data reporting forms, and raw data were present. There was no SDG number as the Region 2 SOP requires; however, the laboratory did provide a case number, data package identifiers, DPWG27692 and DPW27405, and a contract number, 4331, so the results are readily and unambiguously identifiable.
2. HOLDING TIME
All samples met the technical holding specified in Method 1613 and in the QAPP. The following samples did not meet the verified time of sample receipt (VTSR) contractual holding time for extraction (30 days) and analysis (45 days) specified in the Region 2 SOP:
CSC believes, however, that qualification of the data is not necessary since the holding times specified in Method 1613 were met.
3. INSTRUMENT PERFORMANCE
The PFK tune, Window Defining Mix, and Isomer Specificity standards were present in the data package. The only issue appeared to be that, per the Region 2 SOP, a Window Defining Mix/Isomer Specificity standard was not analyzed within the 12-hour period with the following samples: 08A-0026-C1ES-MP and 08A-0002-C1ES-MP All other criteria associated with the Window Defining Mix/Isomer Specificity standard were met. Since there were no problems with the PFK tunes, CSC believes that results for the samples above do not require qualification.
4. CALIBRATION
No major problems were found; however the following issues should be noted. 1) Due to interference, ions 354/356 and 366/368 were used for native and labeled penta-
substituted furans. The theoretical ratio for the P5CDD M/M+2 ions is 0.61; therefore, the ±15% acceptance range is 0.52 - 0.70. All calibration data met this requirement.
2) For sample 008A-0020-C2DS-MP, the last eluting tetra-substituted dioxin and furan and penta-substituted dioxin were slightly outside the retention time window established in the Window Defining Solution. Since these isomers were not target compounds, no data qualification is necessary.
5. BLANKS
Some analytes were reported in method blanks above the AXYS detection limits but below their quantitation limits. Because these analytes were present in samples at levels greater than 5 times their level in blanks, CSC believes that no qualification of the data is necessary. There were 9 samples where 2,3,7,8-TCDD was detected in the associated method blank above the minimum level. Per the Region 2 SOP, acceptable method blanks must not contain 2,3,7,8-TCDD above the minimum level in Method 1613B, and any positive results in the associated samples are rejected. The table below illustrates the relative magnitudes of the TCDD results for these 9 samples.
Although the blank result of 6.44 ng/kg exceeds the ML of 1 ng/kg, the results for 8 of the 9 samples are 20 to 900 times higher than the blank, and its seems highly unlikely that there is an actual effect of the blank on the sample results. For one sample, 08A-0084-C1FS, the sample/blank ratio is just above 5, suggesting a potential blank contribution. In addition, there were 12 samples associated with a method blank where OCDD was detected above the minimum level. The 12 affected samples all had concentrations 50 – 800 times higher than the blank. Therefore, in accordance with Action 3B in Section 10 of the Region 2 SOP, if the concentration in the sample is higher than five times the contamination concentration in the blank, no action is needed. CSC does not believe that rejection of the 2,3,7,8-TCDD results for those 8 of the 9 samples is warranted. However, we will consider the impact of rejecting them on the correction factors calculated from the split sample data. Another minor issue occurred in several samples with results below the ML and less than five times the result in the associated method blank as per Region 2 SOP (Section 10.4 Action 3.A). This does not impact the correction factor since none of these results were used in the calculation. The following sample/analyte results should be qualified non-detect “U.” Sample ID Analyte 08A-0098-C1ES, 08A-0098-C1FS, 08A-0051-C2CS, 08A-0051-C1BS, 08A-0067-C2ES, 08A-0026-C1ES, 08A-0002-C1ES, 08A-0002-C1FS
There is no method criterion for the internal standard area response. However, the Region 2 SOP requires that the internal standard area responses for labeled 1,2,3,4-TCDD and 1,2,3,7,8,9-HxCDD should be checked for every sample. The Region 2 SOP recommends that:
• Samples having internal standard area responses less than 50% but greater than 25% of the response in the CS3 standard in the corresponding initial calibration have both detects and non-detects qualified “J.”
• Samples having internal standard area responses less than 25% should have non-detects be qualified as unusable “R” and detects be qualified “J.”
The following samples had the internal standard labeled 1,2,3,7,8,9-HxCDD less than 50% but greater than 25% % of the CS3 standard in the corresponding initial calibration.
CSC notes that Method 1613B does not contain any criteria for internal standard area response, nor do the CPG or Malcolm Pirnie QAPPs. CSC also notes that the method-specified recovery limits were met for all labeled standards for the samples cited above. Given these findings, CSC has noted the deviations from Region 2’s SOP, but does not believe that the internal standard responses listed above adversely impact the usability of the data.
7. MATRIX SPIKE/MATRIX SPIKE DUPLICATE (MS/MSD)
As noted earlier, the Region 2 SOP requires analysis of MS/MSD samples. However, Method 1613B does not specify the use of MS/MSD samples because it involves isotope dilution techniques in which the recoveries of the labeled compounds spiked into every sample provide more information than the recoveries of native analytes spiked into only 2 of 20 samples in the batch. In addition, CSC’s review of the Malcolm Pirnie QAPP indicates that AXYS was not required to analyze MS/MSD samples. Accordingly, CSC does not believe any qualification of AXYS results, based on a lack of MS/MSD analyses, is appropriate.
8. LABORATORY CONTROL SAMPLE (LCS)
Method 1613B calls an LCS an Ongoing Precision and Recovery (OPR) sample. Both the QAPP and the Region 2 SOP specify the same recovery limits for all native compounds in the LCS (i.e., OPR), except 1,2,3,4,6,7,8-HpCDF. The QAPP specifies recovery limits 82 – 132% for this compound, while the SOP specifies limits of 82 – 122%. The QAPP and Region 2 specify different LCS (i.e., OPR) recovery limits for all of labeled compounds. The recovery limit windows are tighter in the QAPP for all labeled compounds.
AXYS analyzed an OPR with each batch as specified in the method, and no significant problems were observed with these OPRs. Recoveries for all native and labeled compounds were within limits listed in Appendix C-2 of the QAPP and the Region 2 SOP.
CSC did note a minor deviation from the LCS requirements cited in the Region 2 SOP (Section 15.2), which requires that an Ongoing Precision and Recovery standard (OPR) be analyzed periodically, at the beginning of 12-hour shift after the analysis of the CS3 calibration verification (VER), and before the analysis of any sample in each set. However, this is not a requirement of the method, which simply requires that an OPR be prepared with each extraction batch and analyzed once.
AXYS Review Summary 6 10/2011
The following samples did not have an OPR analyzed in the same 12-hour period.
In some cases, samples were analyzed 100 hours after the OPR prepared with the samples; however, an OPR was extracted with each extraction batch. The LCS/OPR sample is designed ensure that laboratory performance is in control beginning with preparation (e..g., extraction) of field samples, whereas calibration verification is designed to ensure that instrument performance is in control during each shift in which samples are analyzed. Because the results for calibration verifications met the method requirements, and because OPRs were prepared at the required frequency, CSC does not believe the OPR analysis frequency noted above adversely impacts the usability of the data.
9. FIELD BLANKS/FIELD DUPLICATES
Field blanks and field duplicates were not collected; therefore, their analyses were not performed. However, the laboratory prepared a laboratory duplicate for each sample batch using samples, 08A-0020-C2CS-MP and 08A-0051-C1BS-MP.
Neither the Region 2 SOP, nor the CPG QAPP, cites criteria for laboratory duplicates. However, the Region 2 SOP provides criteria for field duplicates (which may include additional variability from the field sampling component). In addition, the CPG QAPP provides criteria for matrix spike duplicates (which are similar to lab duplicates in that they address only the lab variability component). CSC examined the AXYS lab duplicates against both criteria. • The Region 2 field duplicate criteria specify that the RPD for 2,3,7,8-substituted analytes
must be within 25%, and that the RPD criteria for the remaining analytes must be within 50%.
• The CPG QAPP for field duplicate criteria specifies that the RPD must be ≤ 50% if both samples are > 5x QL
• The CPG QAPP for MSD criteria specifies that the RPD must be ≤50% for all analytes.
The sample/lab duplicate pair for sample 08A-0020-C2CS-MP met all of the above criteria.
The sample/lab duplicate pair for samples 08A-0051-C1BS-MP met all of the above criteria except for 2 of the native compounds (2,3,7,8-TCDD and 1,2,3,4,7,8,9-HpCDF), which exceeded the field duplicate limits specified in the Region 2 SOP. This means that approximately 6% (2/34) native compounds had failures. CSC believes that failure rate is within the margin of error expected when performing Method 1613B, and based on our assessment of other QC data, we do not believe that these failures adversely impact data usability.
10. COMPOUND IDENTIFICATION
Several issues with compound identification were observed with samples.
1) In the following samples, 2,3,7,8-TCDF RRTs were slightly outside the RRT limit established
in the window defining solution (WDS); however, the labeled compounds were within 2 seconds of the natives, therefore no qualification of the data is necessary, as per the Region 2 SOP.
2) The following sample/analytes did not meet ion abundance criteria and are reported by the laboratory as an Estimated Maximum Possible Concentration (EMPC). CSC recommends qualifying these results with a “J,” as per the Region 2 SOP.
11. COMPOUND QUANTITATION AND REPORTED DETECTION LIMITS
Note: Due to the use of a lower concentration low-level calibration standard, the AXYS quantitation levels are lower than the minimum levels described in Method 1613B. While the data quality is not impacted, the lower quantitation levels will result in more target analytes being detected in the method blank.
AXYS Review Summary 8 10/2011
The following samples were reported with analyte concentrations below the AXYS-defined quantitation level. As per the Region 2 SOP, these detected compounds are qualified “J.”
Sample 08A-0020-C2ES-MP and its laboratory duplicate both had OCDD that exceeded the calibration range slightly. Diluted reanalyses were not performed for these samples; therefore, the results should be qualified with a “J.”
Data Qualifier Sheet Qualifier Definition U The analyte was analyzed for, but was not detected above the reported sample quantitation limit. J The analyte was positively identified; the associated numerical value is an approximate
concentration of the analyte in the sample. UJ The analyte was not detected above the reported sample quantitation limit. However, the
reported quantitation limit is approximate and may or may not represent the action limit of quantitation necessary to accurately and precisely measure the analyte in the sample.
N The analysis indicates the presence of an analyte for which there is presumptive evidence to make a tentative identification. NJ The analysis indicates the presence of an analyte for which there is presumptive evidence to make a tentative identification and the associated numerical value represents its approximate concentration. R The data are unusable. (The compound may or may not be present.)
CAS Review Summary 1 10/2011
Summary of CSC Review of CAS Data from the 30 Sample Splits Used to Generate Correction Factors
CSC reviewed results from 30 samples analyzed by Columbia Analytical Services (CAS) for dioxins and furans using EPA Method 1613B. Results of CSC’s data quality assessment are described below.
SAMPLES REVIEWED
Sample IDs for the 30 samples reviewed by CSC, along with corresponding lab identification numbers, are provided below.
Lab documentation indicates that all samples had a cooler temperature between 0 - 4 °C upon receipt at the lab, and all samples were subsequently stored at -20 °C. DATA REVIEW PROCESS
CSC reviewed and qualified the data in accordance with both the USEPA Region II SOP No. HW-25, Revision 3 and selected worksheets (#12, #15, #19, and #24) in the Quality Assurance Project Plan: RI Low Resolution Coring/Sediment Sampling, Lower Passaic River Restoration Project RI/FS.
According to the Region 2 SOP, a matrix spike/matrix spike duplicate should be performed. Because Method 1613B is an isotope dilution method, however, MS/MSD analyses are not included as a required QC procedure in the method itself. The CPG QAPP under which CAS performed their analyses did
CAS Review Summary 2 10/2011
specify performance of MS/MSD analyses, and these analyses were performed with some of the CAS sample batches. The following abbreviated notations have been used throughout this data review summary to identify target compounds: Abbreviated Notation Full Target Analyte Name 2,3,7,8-TCDD 2,3,7,8-Tetrachlorordibenzo-p-dioxin 2,3,7,8-TCDF 2,3,7,8-Tetrachlorodibenzofuran 1,2,3,7,8-PeCDD 1,2,3,7,8-Pentachlorodibenzo-p-dioxin 1,2,3,7,8-PeCDF 1,2,3,7,8-Pentachlorodibenzofuran 2,3,4,7,8-PeCDF 2,3,4,7,8-Pentachlorodibenzofuran 1,2,3,4,7,8-HxCDD 1,2,3,4,7,8-Hexachlorodibenzo-p-dioxin 1,2,3,6,7,8-HxCDD 1,2,3,6,7,8-Hexachlorodibenzo-p-dioxin 1,2,3,7,8,9-HxCDD 1,2,3,7,8,9-Hexachlorodibenzo-p-dioxin 1,2,3,4,7,8-HxCDF 1,2,3,4,7,8-Hexachlorodibenzofuran 1,2,3,6,7,8-HxCDF 1,2,3,6,7,8-Hexachlorodibenzofuran 1,2,3,7,8,9-HxCDF 1,2,3,7,8,9-Hexachlorodibenzofuran 2,3,4,6,7,8-HxCDF 2,3,4,6,7,8-Hexachlorodibenzofuran 1,2,3,4,6,7,8-HpCDD 1,2,3,4,6,7,8-Heptachlorodibenzo-p-dioxin 1,2,3,4,6,7,8-HpCDF 1,2,3,4,6,7,8-Heptachlorodibenzofuran 1,2,3,4,7,8,9-HpCDF 1,2,3,4,7,8,9-Heptachlorodibenzofuran 1,2,3,4,6,7,8,9-OCDD 1,2,3,4,6,7,8,9-Octachlorodibenzo-p-dioxin 1,2,3,4,6,7,8,9-OCDF 1,2,3,4,6,7,8,9-Octachlorodibenzofuran
In the following sections, QC failures, resulting qualifiers, and associated results are described for each deviation from specified requirements. However, if a reviewer has chosen a different qualifier for a failure because of best professional judgment, then the most severe qualifier will be applied. Qualifiers from most severe to least severe are: “R,” “NJ,” “UJ,” “U,” “J.”
1. DATA COMPLETENESS
No major problems were found. All relevant data reporting forms and raw data were present. There was no SDG number as the Region 2 SOP requires; however, the laboratory did provide a service request number for each data package, and a project number 1282-004, so the results are readily and unambiguously identifiable.
2. HOLDING TIME
The verified time of sample receipt (VTSR) contractual holding time for extraction (30 days) and analysis (45 days) specified in the Region 2 SOP were all met. All samples also were extracted and analyzed within the technical holding times specified in Method 1613B and in the QAPP. Therefore, qualification of the data based on holding times is not necessary or appropriate.
3. INSTRUMENT PERFORMANCE The PFK tune, Window Defining Mix, and Isomer Specificity standards were present in the data package, and all associated performance criteria were met. Since there were no problems, no qualification of the data is necessary.
4. CALIBRATION No major problems were found; however, the following issues should be noted. In a majority of samples, several compounds in the initial calibration and calibration verification are 1-3 seconds outside the window established by the window defining solution. This was primarily observed
CAS Review Summary 3 10/2011
for 1,2,3,7,8,9-HxCDD, but also occasionally occurred with 1,2,3,6,7,8-HpCDD. The sample/analytes affected by this issue are as follows: 08A-0050-C1BS, 08A-0050-C1CS, 08A-0098-C1ES, 08A-0098-C1FS, 08A-0038-C1ES, 08A-0002-C1ES, 08A-0002-C1FS, 08A-0043-C2BS, 08A-0043-C2CS, 08A-0026-C1ES, 08A-0019-C3CS, 08A-0019-C3DS, 08A-0020-C2CS, 08A-0020-C2DS, 08A-0020-C2ES, 08A-0062-C1ES, 08A-0062-C1FS 08A-0062-C1GS, 08A-0006-C1BS, 08A-0006-C1DS
CSC believes that this issue does not significantly impact the data because the relative retention times are always met. Neither the Region 2 SOP, nor the CPG QAPP provides explicit instructions for qualifying calibration data when compounds are outside the retention time windows established by the Window Defining Mixture. However, Section 7 of the Region 2 SOP provides guidance for a similar situation in field samples. This guidance indicates that, “if the analyte has a corresponding labeled analyte and is within 2 seconds of the labeled analyte, no action taken on positive data or nondetects.” CSC believes this a reasonable approach to apply to the calibration data, and therefore, believes that no further action is necessary.
Another recurring problem is that the absolute retention times of internal standards, 13C12-1,2,3,4-TCDD and 13C12-1,2,3,7,8,9-HxCDD differ by more than 15 seconds between the CS3 analysis and the analysis of the associated initial calibration standards. In some cases, the difference is as much as 5 minutes. This may be an indication of instrument drift or that significant changes to the instrument conditions or column have occurred. It is the CSC reviewer’s opinion that a new initial calibration should have been performed prior to sample analysis. The sample/analytes affected by this issue are as follows:
The Region 2 SOP recommends professional judgment regarding qualification of data when absolute retention times differ by more than 15 seconds between the CS3 analysis and the analysis of the associated initial. Since the native and labeled compounds maximized within two seconds of each other, CSC believes that no qualification of the data is necessary.
CAS Review Summary 4 10/2011
The following samples were associated with a calibration standard where the native compound did not maximize within 2 seconds of the labeled compound. These compounds did maximize within 2 seconds in the samples; therefore, no qualification of the data is necessary.
No issues were noted with the blank results reported by CAS for the 30 samples of interest.
6A. LABELED COMPOUND RECOVERIES
Labeled compound recovery of OCDD in Sample 08A-0050-C1CS was below the lower limit in Table 7 of Method 1613B. Accordingly, the detected result for OCDD should be qualified as estimated with a “J.” 08A-0050-C1CS OCDD
6B. INTERNAL STANDARD AREA RESPONSE
There is no method criterion for the internal standard area response. However, the Region 2 SOP requires that the internal standard area responses for labeled 1,2,3,4-TCDD and 1,2,3,7,8,9-HxCDD be checked for every sample. The Region 2 SOP recommends that:
• Samples having internal standard area responses less than 50%, but greater than 25%, of the response in the CS3 standard in the corresponding initial calibration have both detects and non-detects qualified “J.”
• Samples having internal standard area responses less than 25% should have non-detects be qualified as unusable “R” and detects be qualified “J.”
The following samples were associated with an initial calibration in which the 13C12-1,2,3,4-TCDD and 13C12-1,2,3,7,8,9-HxCDD internal standard area responses of the midpoint calibration (i.e., CS3) standard was less than 50% but greater than 25% % of the samples. 08A-0002-C1ES, 08A-0002-C1FS, 08A-0026-C1ES, 08A-0020-C2CS, 08A-0020-C2DS, 08A-0020-C2ES, 08A-0089-C2BS, 08A-0089-C2CS The following samples had the 13C12-1,2,3,4-TCDD internal standard area response less than 50%, but greater than 25% of the CS3 standard in the corresponding initial calibration. 08A-0084-C1ES and 08A-0084C1FS CSC notes that Method 1613B does not contain any criteria for internal standard area response, nor does the CPG QAPP. CSC also notes that the method-specified recovery limits were met for all labeled and internal standards for the samples cited above. Given these findings, CSC has noted the deviations from Region 2’s SOP, but does not believe that the internal standard responses noted above adversely impact the usability of the data.
CAS Review Summary 5 10/2011
7. MATRIX SPIKE/MATRIX SPIKE DUPLICATE (MS/MSD) The following samples were associated with MS/MSDs that had recoveries within the Region 2 limits of 60-140%, but below the limits in the QAPP (OPR limits in Table 6 of Method 1613B). 08A-0020-C2CS, 08A-0020-C2DS, 08A-0020-C2ES
The following samples were associated with MS/MSDs that had recoveries below both the Region 2 limits of 60% and the limits in the QAPP (OPR limits in Table 6 of Method 1613B). 08A-0062-C1ES, 08A-0062-C1FS, 08A-0062-C1GS
The following samples were associated with MS/MSDs that had recoveries above both the Region 2 limits of 140% and the limits in the QAPP (OPR limits in Table 6 of Method 1613B). 08A-0084-C1ES, 08A-0084-C1FS 1,2,3,4,6,7,8-HpCDD and OCDD 08A-0006-C1BS, 08A-0006-C1DS 2,3,7,8-TCDD, OCDD
The following samples were associated with MS/MSD analysis with an RPD > 50% 08A-0062-C1ES, 08A-0062-C1FS, 08A-0062-C1GS
08A-0084-C1ES, 08A-0084-C1FS 1,2,3,4,6,7,8-HpCDD , OCDD The Region 2 SOP recommends use of professional judgment to assess the quality of data associated with MS/MSD failures. Based on the quality of other QC data, CSC does not believe that the MS/MSD deviations noted above adversely impact the usability of associated sample data.
8. LABORATORY CONTROL SAMPLE (LCS)
Method 1613B calls an LCS an Ongoing Precision and Recovery (OPR) sample. Both the QAPP and the Region 2 SOP specify the same recovery limits for all native compounds in the LCS (i.e., OPR), except 1,2,3,4,6,7,8-HpCDF. The QAPP specifies recovery limits 82 – 132% for this compound, while the SOP specifies limits of 82 – 122%. The QAPP and Region 2 specify different LCS (i.e., OPR) recovery limits for all of labeled compounds. The recovery limit windows are tighter in the QAPP for all labeled compounds. The Region 2 SOP (Section 15.2) requires that an OPR be analyzed periodically, at the beginning of 12-hour shift after the analysis of the CS3 calibration verification (VER), and before the analysis of any sample in each set. All of the CAS OPRs met this 12-hour shift requirement.
CAS Review Summary 6 10/2011
The Region 2 SOP recommends that professional judgment be used to evaluate the impact of data quality when analytes are recovered in the OPR outside of method performance criteria, and that data should at a minimum be qualified as a “J.”
The following samples are associated with OPRs that had labeled compounds below the recovery limits for OPRs listed in the QAPP but within the limits listed in Table 6 of Method 1613B. The native compounds of the failed labeled analogues were within limits. In such cases, CSC does not believe that qualification of the data is necessary.
The following samples are associated with OPRs that had native compounds below the recovery limits for OPRs listed in Table 6. CSC believes that the detected results associated with such samples should be qualified with a “J” and that non-detects results should be qualified with a “UJ.”
The following samples are associated with an OPR that had labeled compounds below the recovery limits listed in Table 6 of Method 1613B. The native compounds of the failed labeled analogues were within limits; therefore, no qualification of the data is necessary. 08A-0026-C1ES, 08A-0019-C3CS, 08A-0019-C3DS, 08A-0019-C3ES
13C12-1,2,3,4,6,7,8-HpCDD, 13C12-OCDD
9. FIELD BLANKS/FIELD DUPLICATES
Field blanks and field duplicates were not collected; therefore, the results from their analysis could not be assessed.
10. COMPOUND IDENTIFICATION
Several issues with compound identification were observed with samples. In the following samples, the retention time of the internal standard in the sample differed by more than 15 seconds from the retention time in the CS3 initial calibration standard.
08A-0050-C1BS, 08A-0050-C1CS 13C12-1,2,3,4-TCDD
Since all labeled compounds using this internal standard for quantification all maximized within 2 seconds of the native analogue, no qualification of the data is necessary, as per the Region 2 SOP.
CAS Review Summary 7 10/2011
The following samples had analytes that did not meet ion abundance criteria and are reported by the laboratory as an Estimated Maximum Possible Concentration (EMPC). CSC recommends qualifying these results with a “J,” as per the Region 2 SOP.
The following sample had one analyte for which an interference was noted by the laboratory as coming from a diphenyl ether. CSC recommends that this result be qualified as estimated with a “J.” 08A-0038-C1ES 1,2,3,7,8-PeCDF The following samples/analytes were outside the retention time window established by the Window Defining Solution analyzed prior to sample analysis.
Since the native and labeled compounds all maximized within 2 seconds, no qualification of the data is necessary, as per the Region 2 SOP.
The following samples had native compounds that did not maximize within 2 seconds of its labeled analogue. All results were detects and are qualified as estimated “J,” as per the Region 2 SOP.
In the following samples, results of 2,3,7,8-TCDF in the primary and secondary columns differed by greater than 50%. The Region 2 SOP recommends use of professional judgment to determine which column result to report. Method 1613B specifies that the results of the secondary column are to be reported. CSC agrees with this recommendation, and notes that the laboratory reported the 2,3,7,8-TCDF results from the second column. Therefore no action is needed, and the results are considered of acceptable quality.
In the following samples, the integrated ion current for a given analyte was not at least 2.5 times the background level. All results were reported as non-detects and are qualified “UJ” as per the Region 2 SOP.
11. COMPOUND QUANTITATION AND REPORTED DETECTION LIMITS
Note: Due to the use of a smaller sample size and low percent solids, the minimum level (ML) reported by the laboratory is higher than what is described in Method 1613B. The higher quantitation levels will likely result in more target analytes qualified as estimated, “J.” The following samples have analyte concentrations below the minimum level. Detected compounds are qualified “J.” 08A-0017-C1BS 1,2,3,7,8-PeCDD, 1,2,3,4,7,8-HxCDD, 1,2,3,6,7,8-HxCDD, 1,2,3,7,8,9-HxCDD, 1,2,3,7,8-PeCDF, 2,3,4,7,8-PeCDF, 2,3,4,6,7,8-HxCDF, 1,2,3,4,7,8,9-HpCDF
CAS reported results for the following sample that exceeded the calibration range of the instrument. The laboratory reanalyzed this sample at a five-fold dilution, rather than a complete re-extraction and reanalysis of a smaller aliquot, as described in the method. The result of this dilution should be used for the following analyte since it was within the calibration range.
08A-0038-C1ES 2,3,7,8-TCDD
CAS Review Summary 12 10/2011
Data Qualifier Sheet Qualifier Definition U The analyte was analyzed for, but was not detected above the reported sample quantitation limit. J The analyte was positively identified; the associated numerical value is an approximate
concentration of the analyte in the sample. UJ The analyte was not detected above the reported sample quantitation limit. However, the
reported quantitation limit is approximate and may or may not represent the action limit of quantitation necessary to accurately and precisely measure the analyte in the sample.
N The analysis indicates the presence of an analyte for which there is presumptive evidence to make a tentative identification. NJ The analysis indicates the presence of an analyte for which there is presumptive evidence to make a tentative identification and the associated numerical value represents its approximate concentration. R The data are unusable. (The compound may or may not be present.)
CSC Response to CPG Comments 1
Responses to Specific Comments Raised by the CPG in “Attachment 2 - LRC Split Data Detailed Summary”
Attachment 2 to the June 6, 2011 letter from de maximis, inc., on behalf of the Cooperating Parties Group (CPG), provides a detailed discussion of the CPG concerns regarding the Malcolm Pirnie, Inc (MPI) split sample data set for dioxins/furans. These MPI sample splits were analyzed by AXYS Analytical. The concerns cited in Attachment 2 are based on the findings of a data validation report prepared for the CPG by Laboratory Data Consultants, Inc (LDC). Details of the CPG’s specific concerns about these data begin on Page 4 of Attachment 2. Each of these concerns presented below is followed by CSC’s response, and is based on our validation of both the AXYS and CAS split sample data sets. CPG: Overall Assessment of MPI Split Sample Data Quality - LDC concluded that it could not perform a full data validation of AXYS SDGs DPWG27405 and DPWG27692 (that provide the MPI LRC split samples PCDD/PCDF results) due to incomplete data packages. LDC believes that these data gaps and inconsistencies in the data packages put into question the validity and data integrity of the results. As a result, LDC concluded that the MPI Split Sample Data Set should not be used for final decision-making until data package completeness is verified and full data validation completed. LDC identified the following primary deficiencies and inconsistencies from the AXYS data packages, which include, but are not limited to: Issue 1: The case narrative did not discuss comprehensively why multiple extractions and analyses were performed on the samples. It appears there were no corrective actions in place to address laboratory analytical errors and oversights. CSC Response: In some cases, the case narrative described why multiple extractions were performed. For example, a failed OPR for sample batch WG26664 required the laboratory to re-extract and re-analyze samples. For some samples, up to 3 re-extractions and re-analyses were performed and the reason for doing this was not always specified. CSC’s review of the original data shows that the lab re-extracted and re-analyzed the samples due to multiple ion abundance ratio failures for the native compounds. This is a normal corrective action. CAS data had similar corrective actions based on ion abundance failures. Issue 2: Multiple extractions and analyses were performed on the samples; however, only one set of data was reported. CSC Response: This is true; however, it is no different than the reporting requirements of many other programs such as the CLP program. It is accepted practice to report only a single set of results for a given sample, especially when the laboratory has taken appropriate corrective actions in response to observed data quality concerns. Issue 3: The 1,2,3,8-TCDD peak was incorrectly labeled as 2,3,7,8-TCDD in the window resolution raw data. This error resulted in the incorrect calculation of the percent valley resolution. CSC Response: While the peak was incorrectly labeled for some of the window resolution data, the percent valley resolution was performed on the correct peak. None of the peak valleys were greater than 25%. Issue 4: The sample size for the method blanks was reported as three grams on the forms and raw data; however, the sample size was documented as 10 grams on the extraction logs.
CSC Response to CPG Comments 2
CSC Response: This is correct. CSC recommends that Region 2 request clarification from AXYS as to whether they used 3 or 10 grams for blanks. We recalculated the native compound concentrations in the blanks and compared those values to the results on the sample result forms, and it appears that 3.00 grams were used. Therefore, it is likely that the sample size listed in the extraction logs is an error. Issue 5: Area responses on the selected ion current profile (SICP) do not match the area responses on the quantitation reports for several samples and calibration standards. CSC Response: This is true for a small percentage of analytes. For example, in sample 08A-0067-C2ES, the following analytes had area responses on the SICP that did not match the area responses on the quantitation report: 2,3,7,8-TCDF, 2,3,4,6,7,8-HxCDF, 1,2,3,4,6,7,8-HpCDF and 1,2,3,7,8-PeCDD. It is important to note that in samples this occurs only for detects below the quantitation level, which were not used in calculation the correction factor. For standards, this issue seems to affect only a couple of labeled compounds, namely the two HxCDF compounds that partially coelute (i.e., 13C-1,2,3,4,7,8-HxCDF and 13C-1,2,3,6,7,8-HxCDF). The discrepancy in the standard areas in the SICP and quantitation report is less than 1%. Therefore, CSC believes that these minor SICP discrepancies have minimal impact on the overall data quality.
Issue 6: Inappropriate manual integration baselines were observed for 13C-1,2,3,4,7,8-HxCDF, 13C-1,2,3,6,7,8-HxCDF and 13C-2,3,7,8-TCDD in the initial and continuing calibration standards. CSC Response: It appears that for some of the standards, the manual integration was slightly off for the 3 labeled compounds mentioned, but upon visual inspection, a more appropriate integration would be different than the reported value by no more than 5%. This is well within the expected measurement error associated with Method 1613B, and therefore, this minor issue should not adversely affect use of the data. Issue 7: The laboratory provided revised SICPs for all samples and QC for the labeled HxCDDs, labeled HxCDFs and/or 13C-1,2,3,4-TCDD, but not for the associated initial and continuing calibrations. CSC Response: CSC did not review the revised SICPs; however, CSC believes that SICPs for associated initial and continuing calibration will have minimal impact on the data, given that the discrepancies between the responses on the SICP and quantitation reports were less than 1% for samples and less than 5% for calibrations. CPG: Specific Data Validation Results for the MPI Split Sample Data Set- The data validation reports and qualified Form 1s, provided as Attachment 2, should be considered as preliminary. If submission of this additional data cannot be provided to meet the requirements of the MPI QAPP, the data set would be considered as invalid and have limited use. In summary, the data validation report prepared by LDC qualifies the MPI LRC Split Sample Data Set as follows: Issue 8: The positive results for several PCDDs and PCDFs for samples 08A-0067-C2ES-MP, 08A-0062-C1GS-MP, 08A-0043-C2BS-MP, 08A-0051-C1BS-MP, 08A-0051-C1BS-MPDUP, 08AR. 0051-C2CS-MP, 08A-0050-C1BS-MP, 08A-0050-C1CS-MP, 08A-0089-C2BS-MP, 08A-0089-C2CS-MP, 08A-0098-C1ES-MP, 08A-0098-C1FS-MP and 08A-0019-C3BS-MP in SDG DPWG27405 and samples 08A-0026-C1ES-MP, 08A-0002-C1ES-MP, 08A-0002-C1FS-MP, and 08A-0084-C1ES-MP in SDG DPWG27692 were qualified as nondetect (U) due to method blank contamination. CSC Response: All sample results that were changed to nondetect (U) were for cases where the sample result and method blank result were below the quantitation limit and above the detection limit. CSC agrees that the results should be changed to nondetect (U). Changing this is consistent with the Region 2
CSC Response to CPG Comments 3
SOP. This does not affect the calculation for the correction factors because none of these results were used in CSC’s initial calculation. Issue 9: The positive results for 2,3,7,8-TCDD, Total TCDD, 1,2,3,4,6,7,8-HpCDF, and Total HpCDF in samples 08A-0051-C1BS-MP, and 08A-0051-C1BS-MPDUP in SDG DPWG27405 and the positive results for 1,2,3,4,7,8,9-HpCDF, and Total HpCDF in samples 08A-0020-C2ES-MP, and 08A-0020-C2ES-MPDUP in SDG DPWG27692 were qualified as estimated (J) due to high laboratory duplicate (DUP) relative percent differences (RPD). Possible causes of poor precision include sample matrix interference, improper sample collection or handling, inconsistent sample preparation, and poor instrument stability. CSC Response: For samples 08A-0020-C2ES-MP, and 08A-0020-C2ES-MPDUP in SDG DPWG27692, all RPDs are within 25% for the 2,3,7,8-substituted analytes. In the absence of laboratory duplicate criteria in the QAPP, RPD limits for field duplicates described in Method 1613B and the Region 2 SOP were used. For samples 08A-0051-C1BS-MP, and 08A-0051-C1BS-MPDUP, only 2 of the native compounds were outside the RPD limits required for environmental duplicates. This means that approximately 6% (2/34) native compounds failed to meet the laboratory duplicate 25% RPD criterion. This is within the margin of error expected when performing Method 1613B. CSC believes that qualifying the results for those 2 native compounds falling outside RPD limits as estimated (J) does not represent a significant data quality concern. The duplicate results for one of the failing compounds, 1,2,3,4,6,7,8-HpCDF, was not used in the original correction factor calculation because the CAS result was qualified as estimated “J”. For 2,3,7,8-TCDD, CSC recalculated the correction factor without the duplicate failure and the resulting values were higher than the original. These results are described in more detail in the October 17, 2011 memo.
Issue 10: The positive results for 2,3,7,8-TCDD, Total TCDD, 1,2,3,6,7,8-HxCDD and Total HxCDD and the positive and nondetect results for 1,2,3,7,8-PeCDD and Total PeCDD for samples 08A-0038-C1ES-MP, 08A-0200-C1ES-MP, 08A-0020-C2CS-MP, 08A-0020-C2DS-MP, 08A-0020-C2ES-MP, 08A-0201-C2ES-MP, 08A-0006-C1BS-MP, 08A-0006-C1DS-MP, 08A-0017-C1BS-MP, 08A-0017-C1DS-MP, 08A-0084-C1FS-MP, and 08A-0020-C2ES-MPDUP in SDG DPWG27692 were qualified as estimated (J/UJ) due to SRM concentrations falling outside the certified concentration range.
CSC Response: The CPG QAPP states that the corrective action for any result outside the supplier certified limits is to provide feedback to laboratory and request that the laboratory to review their data for problems. Neither the CPG QAPP nor the Region 2 SOP requires qualifying the data as estimated due to this issue. CSC agrees with this approach. AXYS did analyze OPRs with each batch to ensure their sample preparation and analytical procedures were under control. Issue 11: Both the positive detections and nondetect results for samples 08A-0043-C2BS-MP, 08A-0043-C2CS-MP, 08A-0019-C3DS-MP, and 08A-0098-C1FS-MP in SDG DPWG27405 and samples 08A-0020-C2CS-MP, 08A-0020-C2DS-MP, 08A-0020-C2ES-MP, 08A-0020-C2ESMPDUP, 08A-0201-C2ES-MP, 08A-0006-C1BS-MP, 08A-0017-C1DS-MP, and 08A-0017-C1BSMP in SDG DPWG27692 were qualified estimated (J/UJ) due to unacceptable moisture content (i.e., percent solids < 50%).
CSC Response: Neither Method 1613B nor the QAPP requires that data be qualified on the basis of percent solids being <50%. The Region 2 SOP does include such qualification. However, as noted during extensive discussions among all parties since December 2009, the source or rationale for this qualification is not clear. We note that some CAS samples also had percent solids <50% and imagine that other samples from this site that were not part of the split sample comparison may have similar issues. Given that all of the samples involved in this effort are wet sediments, it is unclear if there is a real basis for this qualification.
CSC Response to CPG Comments 4
Issue 12: Potential uncertainty exists for select results which were flagged K by the laboratory as estimated maximum possible concentration (EMPC). These results were qualified as estimated (EMPC-J). CSC Response: Both laboratories reported results at the EMPC value; however, none of these values were used in the calculation of the correction factors. CPG: Other Observations Regarding the MPI Split Sample Data Set - LDC identified the following additional observations of quality control anomalies during the data validation process: Issue 13: AXYS used three grams (dry) instead of the 10 grams (dry) specified by Method 1613B. Better precision and representativeness is attainable with a larger sample size. CSC Response: While a larger sample size would normally demonstrate better representativeness, better precision is not always achieved. Incidentally, CAS also used only 3 grams, however, AXYS calibrated down to a lower level in order to achieve lower MLs, thus offsetting the effect of the lower sample size. CAS did not achieve these lower MLs with their use of a 3-gram sample. Issue 14: MS/MSD analyses were not performed. MS/MSD samples were extracted in batch WG27393 in SDG DPWG27692; however, the laboratory noted that the MS extract was concentrated to dryness. The laboratory also indicated that MS/MSD samples for dioxin analyses were not contractually required by MPI. CSC Response: Although CPG required CAS to perform MS/MSD analyses, Malcolm Pirnie did not require AXYS to do so. CSC doesn’t believe that this is reason for concern. An MS/MSD analysis is not required in Method 1613B because each and every sample is spiked with the labeled compounds. The recoveries of the labeled compounds provide a sample-specific demonstration of method performance, whereas the MS/MSD pair would only be performed on one sample in the batch of 20, which may or may not be representative of the other 19 samples. Issue 15: SRM analyses were not performed in SDG DPWG27405 and with samples 08A-0026-C1ES-MP, 08A-0002-C1ES-MP, 08A-0002-C1FS-MP, and 08A-0084-C1ES-MP in SDG DPWG27692. Certified reference material or second source calibration standards are necessary to assure accuracy of calibration standards and overall reliability of the analytical process. The MPI QAPP states that:
“AXYS Analytical will also be requested to analyze NIST SRM 1944 with a batch of samples for PCB Congeners, PCDD/Fs, and Pesticides.”
A review of the EQA data validation report for Data Package DPWG27692 (Enclosure C) indicates that NIST SRM 1944 was analyzed as part of this data package. The results of the PT/certified reference material indicated the need to qualify some results. However, no PT sample was run with the first MPI split sample data package DPWG27405 (Enclosure B) which was received by AXYS between August 7 and October 4, 2008; the NIST sample was run with the samples that comprise the second data package DPWG27692, which was received by AXYS from October 22 through December 10, 2008.
The split sample program conducted by MPI appears to be inconsistent with, and not meet the requirements of, the UFP QA/QC Activities guidance to run an associated PT sample for a batch of split samples since a PT sample was not run with the Data Package DPWG27405. In addition, AXYS qualified six of the 16 NIST SRM 1644 results (i.e., 2,3,7,8-TCDD, Total TCDD, 1,2,3,6,7,8-HxCDD, Total HxCDD, 1,2,3,7,8-PeCDD and Total PeCDD) included in Data Package DPWG27692 with a “K” qualifier for peaks that did not meet quantitation requirements and as such are estimated maximum possible concentrations (EMPC). As a result of its validation, LDC qualified the results of the six PCDD congeners for 12 samples from SDG DPWG27692 with J or UJ.
CSC Response to CPG Comments 5
Thus, MPI’s LRC split sampling program suffers from at least two significant flaws (Issue 17 and 18) that severely compromise its ability to quantify the differences between the LRC samples and the MPI splits: CSC Response: While an SRM was not performed for every batch of samples, the QAPP specifies that an SRM will be analyzed with “a batch of samples” and not “every batch of samples”. For many programs, a PT sample (e.g., SRM, CRM) is analyzed on a quarterly basis. The ongoing performance of OPRs, method blanks, and CCVs by the laboratory is another indicator of performance. We note that these other performance checks indicate that the lab performed acceptably during the period in which the sample analyses were performed. Moreover, as discussed at length with all parties during earlier stages of our examination of the split sample results, the SRM in question is a dry powdered soil. The relevance of that dry matrix to the extraction and analysis of wet sediment samples is tenuous. We believe that the benefits of the SRM are no greater than that of the other QC checks mentioned above in demonstrating the performance of the laboratory. Issue 16: Six of the 16 SRM results were flagged “K” by the laboratory as EMPC. The laboratory indicated that area responses were low due to sample size (0.977 grams) and final volumes used. A larger sample size should have been used as specified in the method required by Method 1613B to attain better area responses.
CSC Response: This was the amount of SRM received by the laboratory. They analyzed the SRM with the material available. We agree that better sensitivity would have been possible with a larger sample size. However, the difficulty that lead the laboratory to call these 6 results EMPCs does not necessarily mean that they did not meet the acceptance limits for SRM itself, nor does it appear that LDC alleges that the numerical results (e.g., the EMPCs) failed to meet those criteria. Our review showed that the AXYS SRM results were in fact within the vendor-specified concentration limits for this SRM. AXYS appears to have met that requirement with the exception of 2,3,7,8-TCDD; however, this analyte was above the upper acceptance limit by only 1 ppt , or 0.7%.
Issue 17: Contrary to guidance in the UFP QA/QC Activities, a PT sample was not analyzed for all 17 samples that comprise Data Package DPWG27405 and for three of the 15 samples that comprise Data Package DPWG27692.
CSC Response: While a PT sample in the form of an SRM was not analyzed with every data package, the MPI QAPP specifies that “AXYS Analytical will also be requested to analyze NIST SRM 1944 with a batch of samples for PCB Congeners, PCDD/Fs, and Pesticides.” This statement does not mean an SRM is required for every sample batch. And again, we question the value of an SRM that so poorly represents the actual sample matrix of interest. Issue 18: For the remaining 12 samples in Data Package DPWG27692, the reported concentrations for six PCDD congeners were flagged by AXYS as not meeting quantification requirements. Following LDC’s full validation, the reported concentrations for the six congeners were qualified as estimated values (J) for both positive and non-detect results. CSC Response: The results that did not meet quantification requirements, and therefore were qualified as estimated by the validator, were not used in the calculation of correction factors. Issue 19: The MPI split results cannot be used to support the quantification of differences between the LRC Data Set and the MPI Split Sample Data Set because critical PT sample results are either missing or unreliable.
CSC Response to CPG Comments 6
CSC Response: The absence of supporting data does not invalidate results or their use, particularly when all method-specified QC elements are present. CPG and LDC do not present data showing the results are not valid, missing, or unreliable, PT results notwithstanding. We continue to voice our concerns about relevance of a dry powdered SRM for this project. CPG Summary CSC’s March 2010 report states: “In order to develop a correction factor for the Columbia Analytical Services [(CAS)] results, the results produced by AXYS must be presumed to be “true values (emphasis added).” The CPG requests that the EPA review the Oversight QAPP, EQA data validation reports, UFP QAPP guidance on split samples, and the LDC data validation report and, at minimum, discuss and consider the following questions with CSC regarding the proposed PCDD/PCDF adjustment factors: Issue 20: Would CSC come to the same conclusions and make the same recommendations if it had been aware of the actual validation status (i.e., partial, forms–only check inconsistent with the approved QAPP) of the MPI Split Sample Data Set? CSC Response: CSC does not believe any of the issues mentioned above would significantly impact the correction factors. We have examined and documented the impact of these issues in greater detail in an October 17, 2011 memo to EPA Region 2. Issue 21: Does the lack of performance testing sampling (e.g., CRMs, SRMs, etc.) for one of the MPI SDGs and for some sample results in the other SDG weaken the presumption that the MPI Split Sample Data Set is correct (i.e., accurate and precise) and can be relied upon?
CSC Response: The lack of SRMs for every sample batch does not weaken the presumption that the MPI Split Sample Data Set is correct. Most programs do not require an SRM with each sample batch, nor does Method 1613B. Method 1613B does not require the use of an SRM with each batch. It requires that a SRM or CRM be analyzed as a QC Check Sample periodically and suggests they be analyzed at least quarterly (Sections 7.16 and 9.6). AXYS analyzed one SRM for this project and while several compounds did not meet the qualitative identification criteria (e.g., ion abundance ratios), all compounds met the certified reference limits, with the exception of 2,3,7,8-TCDD, which was outside of the acceptance limit by 1 ppt, or 0.7%. SRMs are just one of many tools that are used to demonstrate that laboratory performance is in control. The absence of SRMs should not be a basis to discard results. AXYS did analyze all method-specified QC samples and standards. For example, AXYS extracted an OPR with each extraction batch and all recoveries were within the limits described in the Region 2 SOP. Also, all labeled compound recoveries in samples were within limits. Finally, all calibration verifications were within the limits specified in Method 1613B. In contrast, we believe that the more frequent analyses of SRMs by CAS do not necessarily strengthen the presumption that the CAS data are correct because we believe that the issue is difference in the abilities of the laboratories to extract wet sediment samples. and the SRM used by CAS is a dry powder. Issue 22: Would the qualification of a significant number of the data in the MPI Split Sample Data Set as estimated (J) concentrations by the data validator (in this case, LDC) result in different conclusions and recommendations regarding the application of adjustments to the LRC Data Set? CSC Response: No, because the majority of that estimated data was not used in the calculation of the correction factor. We describe our examination the effects of the estimated results from both laboratories in greater detail in our October 17, 2011 memo to EPA Region 2.
