1/9/18 Page 1Beyond CD Compliance, to Audit or Not?

David Gratson, CEAC

Rock Vitale, CEAC

4C Conference,

April, 2018

San Antonio, TX

Beyond CD Compliance, to Audit

or Not?

1/9/18 Page 2Beyond CD Compliance, to Audit or Not?

• Highest Risk Methods/Analytes

• Laboratory Accreditation

• Accrediting Body

• Proficiency Testing

• Reporting Indicators

• Types of Audits

• TNI/TCEQ

• BWON

• Why Audit if not Required and

Accredited?

Agenda

Prioritize Risk

Management

Measure Risk

Act to Minimize/Control Risk

Continuous Improvement

1/9/18 Page 3Beyond CD Compliance, to Audit or Not?

• NOV

• Cost of false positive

• Cost of yet to be discovered exceedance

• Remediation Human/Eco Risk Assessment

• Cost of false positive/false negative/inaccurate

• Cost to prove to a regulator your data are actually wrong

• Invoicing ≠ Requested/Contracted

Analytical Services Risks

1/9/18 Page 4Beyond CD Compliance, to Audit or Not?

• Methods with no Reference Materials/Proficiency Tests• Method 325B, Benzene Fenceline Monitoring

• Dissolved gases (e.g. methane) in water

• PFASs

• Methods/Analytes with know and common interference issues• Cyanide

• Alkylated PAHs

• DRO

• Selenium

• Methods where PTs

dissimilar to your matrix• All?

• Methods not undergone multi-lab

Validation (new ASTM)

Method/Analyte Risk Issues

1/9/18 Page 5Beyond CD Compliance, to Audit or Not?

• Shall, Should and RCRA Methods Innovation Rule• 8260C, Section 4.3: should prove no carry-over between samples…

• RCRA/CWA Method Defined Parameter• “The TCLP is a method-defined parameter and therefore, it must

be performed as written, which includes meeting all specifications …. The effect of failing to perform the method as written is that theresults are not valid for the purposes of determining whether the waste is hazardous based on the toxicity characteristic.

• RCRA: Total and Amendable Cyanide, many others…• CWA: Oil and Grease and others…

• Method Modification Allowance• Significant, requires validation vs reasonable change

• Instrumentation vs time/temperature/reagents

Method Language and Modification Issues

1/9/18 Page 6Beyond CD Compliance, to Audit or Not?

• Turn-over rate

• Ethics, historical knowledge, reasonable/unreasonable changes not

specified in SOP

• Alleged Falsification

• DEP Collects $220,000 Penalty from Ohio Lab for Violations (2017)

• • Failure to properly notify public water supplier and the Department of an MCL

exceedance

• Failure to control laboratory conditions, to identify and correct contamination

• Failure to generate accurate and valid data

• Failure to have proper staffing, management structure, quality assurance/quality

control to ensure Summit generated accurate, valid data

• U.S. Geological Survey lab worker faked test results, investigators say (2016).

• DEP fines Upper Southampton lab $600K for falsified water tests (3/2018).

Laboratory Personnel

1/9/18 Page 7Beyond CD Compliance, to Audit or Not?

• Quality System and Procedures per Accrediting

Body/Standard

• TNI, TCEQ, LA, AZ, OR, FL, MN, PA, NM …?

• Method, analyte, matrix-specific

• Audits ~2-3 years apart, Auditors vary

• PT component (single blind), how

representative?

• Concentration, matrix, interferences

• Not focused on you as client

Environmental Laboratory Accreditation

1/9/18 Page 8Beyond CD Compliance, to Audit or Not?

• Quality System and Technical Components

• Accreditation Standard (e.g., TNI 2009, 2016)

• Procedures match method/State requirements

• Big Picture - Say it, do it, prove it

• Analyte and Method Specific

• If not an accredited analyte, method, matrix not

included

• Large scope

• Checklist-based

Accreditation Type Audits

1/9/18 Page 9Beyond CD Compliance, to Audit or Not?

• May not account for multiple method versions for

which laboratory uses (e.g. 8260B, 8260C,

8260D)

• Method flexibility understanding varies

• May expect strict adherence to instrumentation

without understanding impact of substitution

• Ambiguity in TNI language• 2009 TNI Section V1M2 5.8.6.c: “use of appropriate sample

containers.”

Accreditation Type Audits

1/9/18 Page 10Beyond CD Compliance, to Audit or Not?

• Pre-audit scheduling, request for documents• Pre-Audit Questionnaire

• On-site Audit

• Sample Receiving and Storage

• Instrumentation

• Data Management, backup

• Record Keeping

• Reporting

• Audit Report with Findings, Recommended Cas

*bold indicates Client focus, may not be covered in accreditation audit

Client Focused Technical Audits (BWON)

1/9/18 Page 11Beyond CD Compliance, to Audit or Not?

• Quality System

• Quality Manual

• Staff turnover

• Staff Qualifications sufficient

• Training documented and complete

• Annual data integrity training and/or certification that

employee acknowledges and understands his/her

personal and legal responsibilities including potential

penalties for violations

• Demonstration of Capability documented and complete for

bench analysts

• Data management system ruggedness

Client Focused Technical Audits

1/9/18 Page 12Beyond CD Compliance, to Audit or Not?

• Do internal audits address all elements of the quality

systems including testing activities?

• Are the more stringent criteria between the TNI

standard and source method followed?

• Technical Component

• Login account for all conditions and records

sufficiently detailed

• PT recoveries acceptable, focus on clients

• Internal/external audit results

• Anything that measures: calibration/check,

traceable, maintenance schedule

Client Focused Technical Audits

1/9/18 Page 13Beyond CD Compliance, to Audit or Not?

• Sample handling impact integrity/accuracy: anything

that touches primary sample, aliquot, extract, residual

• Are volume checks of disposable sample containers

verified and/or documented by lot?

• QC correct order, all matrix types, all steps in

preparation, analysis, peer reviewed,

spreadsheets/LIMS validated

• Waste management and container disposal

• How well does analyst understand qualitative

analysis. Can they identify interferences and take

appropriate action

Client Focused Technical Audits

1/9/18 Page 14Beyond CD Compliance, to Audit or Not?

Time and Experience allows in-depth look at:• Bench analysts understanding and ability to

interpret gray areas of methods/SOPs

• Raw calibration data

• Calibration across medium and low-level samples

• Method blanks for low and medium level samples

• Inspection of sample, standards containers

• Recalculation from raw data

• Historic perspective on method flexibility

• Documentation and traceability from primary standard to associated samples

• We track issues with time and place

Client Focused Technical Audits

1/9/18 Page 15Beyond CD Compliance, to Audit or Not?

• Laboratories have become a revolving door

with high employee turnover rates

• We regularly identify critical findings while

assessing accredited laboratories

• Published Methods lack detail and teeth

• Client-specific, focus on known concerns

• Still cases of fraud

Why Audit beyond CD Requirements?

1/9/18 Page 16Beyond CD Compliance, to Audit or Not?

• When does not following a published method, or taking easy route when method is ambiguous, affect data defensibility?

• Non-accredited methods/matrices• I want an audit for PIANO,

Biomarkers, Stable Isotope analysis, Oil-specific Toxicology Protocols …

Why Audit beyond CD Requirements?

1/9/18 Page 17Beyond CD Compliance, to Audit or Not?

• Effluent Limit Guidelines under CWA (§308)• EPA is soliciting data from 9 petroleum refining

companies related to their wastewater

characteristics. 2018-2019 timeframe

• Use to set minimum technology requirements (BAT)

• B.9 and other sections: Required to submit 2016

discharge intake and effluent, “Follows 40 CFR Part

136 sampling, testing, QA/QC requirements?”

• TWPE, if high TWF (e.g. Hg), more scrutiny

Why Audit beyond CD Requirements?

1/9/18 Page 18Beyond CD Compliance, to Audit or Not?

• RCRA Updates, Hazardous Generator Waste Improvements (2017, 2018 state dependent)

• 2013 evaluation 34% of waste determinations non-compliant

• Benzene Fenceline Monitoring

• API Study showed high variability with two labs

• EPA’s equations included errors and ambiguity

• Calibration and quantification within method dissimilar to many water and waste methods.

• Data will be public on EPA CEDRI

• TSCA Reform

Why Audit beyond CD Requirements?

1/9/18 Page 19Beyond CD Compliance, to Audit or Not?

• Custom Technical Specification

• Customized, double blind PT samples

that better match matrix, concentrations

of interest

• Verification/validation

• Data management system to ID anomalous data asap

• Bat-phone: ~40 chemists available with extensive background in Organics, Inorganics, WET, wet chemistry, Radiochemistry, LEAF methods, fuels methods, method development

Beyond Auditing

1/9/18 Page 20Beyond CD Compliance, to Audit or Not?

What to Expect of Your Assessor? to Expect of

Your Assessor?

1/9/18 Page 21Beyond CD Compliance, to Audit or Not?

Thank You for your Time!

Headquarters 1140 Valley Forge Road | PO Box 810 | Valley Forge, PA 19482 | 610.935.5577

Virginia 1412 Sachem Place, Suite 100 | Charlottesville, VA 22901 | 434.293.4039

Tennessee 8331 East Walker Springs Lane, Suite 402 | Knoxville, TN 37923 | 865.376.7590

New Mexico PO Box 29432 | Santa Fe, NM 87592 | 505.660.8521

Illinois PO Box 335 | Geneva, IL 60134 | 630.262.3979

South Carolina PO Box 14315 | Charleston, SC 29422 | 843.469.5867

New Jersey 1175 Marlkress Rd. #4222 | Cherry Hill, NJ 08034 | 304.552.1442

West Virginia 2501 Chapline St. PO Box 6562 | Wheeling, WV 26003 | 304.312.7830

Web www.envstd.com | E-mail solutions@envstd.com

“Setting the Standards for Innovative Environmental Solutions”