EPA Precursor Gas Training Workshop Precursor Gas Quality Assurance Implementation Dennis K. Mikel...

EPA Precursor Gas Training Workshop

Precursor Gas Quality Assurance Implementation

Dennis K. Mikel

EPA Office of Air Quality Planning and

Standards


Outline

• QA System Development • QA Requirements for the NCore Program

– Changes to the 40 CFR Appendix A – Overview of the New Calculations

• Data Assessment Statistical Calculator (DASC)• Comparing the DQO/MQOs to the Burden’s

Creek data • National Performance Audit Program


OR

GA

NIZ

AT

ION

/PR

OG

RA

M

PR

OJ

EC

T

PO

LIC

Y/R

EG

UL

AT

ION

S

Defensible Products and Decisions

EPA Program &Regional Policy

External PoliciesContracts - 48 CFR 46

Assistance Agreements - 40 CFR 30, 31, and 35

Internal EPA PoliciesEPA Order 5360.1EPA Manual 5360

Consensus StandardsANSI/ASQC E4ISO 9000 Series

Annual Review and Planning

(e.g., QAARWP)

SystemsAssessments(e.g., QSAs)

Quality SystemDocumentation

(e.g., QMP)

Training/Communication(e.g., Training Plan,

Conferences)

Supporting System Elements(e.g., Procurements,

Computer Hardware/Software)

TechnicalAssessments

IMPLEMENTATIONPLANNING ASSESSMENT

Conduct Study/Experiment

QA Project Plan

SystematicPlanning

(e.g., DQO Process)

StandardOperating

Procedures

Data QualityAssessment

Data Verification& Validation

A structured and documented management system that describes how and by whom an organization assures quality in its work.

What is a Quality System ?

– Make correct decisions

– Optimize resource use

Quality System Goals

EPA

YourProgram

The Project!


Reporting

Where are we?


Changes in the QA Regulations(40 CFR Part 58 Appendix A)

• Ensured regs reflected current EPA QA policies and requirements

• Combined Appendix A and B (PSD)

• Institute the Primary Quality Assurance Organization (PQAO) to help lower the QA burden on SLT agencies and provide adequate level of independence

• Measurement quality objectives (MQOs) changing for gaseous pollutants


Gas Instrument QA Requirements*

* Table A-2 is from 40 CFR 58, Appendix A


MQOs for Gas Pollutants Has Changed

• Ozone: – Coefficient of Variation (CV): 7% – Bias: +/- 7%

• SO2, NO2, CO: – CV: 10%– Bias: +/-10%

• For PM: – No Changes

Ozone (2004-2005) Upper Bound CV

0.00

7.00

14.00

1 42 83 124 165 206 247 288 329 370 411 452 493 534 575 616 657 698 739 780 821

Individual Monitors

90%

Up

per

Bo

un

d C

V (

%)

Nationally, 97% of all ozone monitors reportCoefficient Variance of less than 7%.


“Reporting Organization” vs. “Primary Quality Assurance Organization”

• What’s the Difference?

– RO: “an entity, such as State, local or Tribal monitoring organization that collects and reports air quality data to EPA.”

– PQAO: Monitoring Organization or “other” organization that is responsible for a set of stations that monitor the same pollutant and which data quality assessments can be pooled.”

• PQAOs are responsible for implementing QA


“Reporting Organization” vs. “Primary Quality Assurance Organization” (cont.)

• PQAO: – Measurement uncertainty “reasonably homogeneous”– Common team of field operators– Use of a common QAPP and SOPs– Oversight of QA functions– In some cases, RO will equal PQAO

• Must demonstrate both Adequacy and Independence


Adequacy

• Performing audits at 20% of monitoring sites/instruments

• Audit data submission to AQS• TTP Delivery system• Follow NPAP field/lab SOP critical

performance criteria• Audit gases that are NIST certified and

validated at least once a year• Validation/certification with the EPA NPAP

program • Incorporated in QAPP


Independence

• No part of the organization is directly performing and accountable for the work being assessed (i.e., PQAO)

• Management structure that allow for the separation of its routine sampling personnel from its auditing personnel by two levels of management

• Submission of a plan demonstrating independence to the EPA Regional Office.


New Statistics in 40 CFR 58 Appendix A

• Focus Workgroup of the QA Strategy team formed to review and revise the old statistics in Appendix A

• Proposed that the stats be based on confidence intervals to meet the MQOs

• This approach allows flexibility for analyst and agencies to tighten or loosen their test if they meet criteria

• Allows analysts to look at “pooled” results rather than individual or small sets of data for which the old stats were designed

• For gases, both the precision and bias calculations use the bi-weekly QC 1-point check (i.e., Precision checks)


Precision and Bias

100

X

XYd ii

: Monitor value

: Target concentration (standard)

: Percent Difference (individual bias) id

XiY

Estimates start the same way & both use the 1-point QC check


Precision

* Where is the 10th percentile

of a Chi-Squared Distribution

2

1,10.0 n

CV

n d d

n n

n

X

i

i

n

i

i

n

n

2

1 1

2

20 1 11

1

. ,

This represents a 90% upper confidence limit on the CV estimate

“A measure of mutual agreement among individual measurements of the same property, usually under prescribed similar conditions”


Bias

Absolute Bias Point Estimate:

n

iiabs d

nm

1

1

n

stmbias absd

nabs_

)1(,95.0

Where t 0.95,(n-1) is the 95th quantile of a Student’s t distribution with n-1 df and sd_abs is the standard deviation of the absolute value of the relative percent differences

This represents a 95% upper confidence limit on the Bias estimate

Absolute Bias Upper Bound:

“A systematic or persistent distortion of a measurement process which causes errors in one direction”


P&B Guidance and Data Assessment Statistical Calculator (DASC) Software

Site:

Automated Methods

Manual Methods

{Enter Site ID or Name Here}

Step 3

DASC (Data Assessment Statistical Calculator)

Step 2

Pick a Pollutant

Step 1

Pick a Statistic to Calculate

SO2

Go To Worksheet

Precision Estimate

Bias Estimate

Absolute Bias EstimateAbsolute Bias Estimate

Semi-Annual Flow RateSemi-Annual Flow Rate

PM 2.5

PM 10

PM 10-2.5

NO2

O3

CO

Lead

One-Point Flow RateOne-Point Flow RatePM 2.5

PM10

PM 10-2.5

CV_ub (%) Bias (%)Meas Val

(Y)Audit Val

(X) d (Eqn. 1) 25th Percentile d_sqrd d_abs d_abs ^219.9 20 -0.500 -6.500 0.250 0.500 0.25020 20 0.000 75th Percentile 0.000 0.000 0.000 n st dev(d) st dev (d^2) sum(d_abs) "AB" (Eqn 3a)19.8 20 -1.000 -4.000 1.000 1.000 1.000 162 2.544 27.349 863.500 5.33019.9 20 -0.500 0.250 0.500 0.250 n-1 sum(d) sum(d^2) sum(d_abs^2) "AS" (Eqn 3b)20 20 0.000 0.000 0.000 0.000 161 -850.500 5507.250 5507.250 2.37020.1 20 0.500 0.250 0.500 0.25019.9 20 -0.500 0.250 0.500 0.250 Bias (%) (Eqn 3) Both Signs Positive19.9 20 -0.500 0.250 0.500 0.250 5.64 FALSE19.6 20 -2.000 4.000 2.000 4.000 CV (%) (Eqn 2) Signed Bias (%) Both Signs Negative19.6 20 -2.000 4.000 2.000 4.000 2.74 -5.64 TRUE19.7 20 -1.500 2.250 1.500 2.25019.6 20 -2.000 4.000 2.000 4.000 Upper Probability Limit Lower Probability Limit19.8 20 -1.000 1.000 1.000 1.000 -0.26 -10.2419.6 20 -2.000 4.000 2.000 4.00019.5 20 -2.500 6.250 2.500 6.25019.7 20 -1.500 2.250 1.500 2.25019.6 20 -2.000 4.000 2.000 4.00019.6 20 -2.000 4.000 2.000 4.00019.1 20 -4.500 20.250 4.500 20.25019.5 20 -2.500 6.250 2.500 6.25019.4 20 -3.000 9.000 3.000 9.00019.6 20 -2.000 4.000 2.000 4.00019.5 20 -2.500 6.250 2.500 6.25019.5 20 -2.500 6.250 2.500 6.25019.4 20 -3.000 9.000 3.000 9.00019.5 20 -2.500 6.250 2.500 6.25019.3 20 -3.500 12.250 3.500 12.25019.1 20 -4.500 20.250 4.500 20.25019.1 20 -4.500 20.250 4.500 20.25019.3 20 -3.500 12.250 3.500 12.25019.2 20 -4.000 16.000 4.000 16.00019.2 20 -4.000 16.000 4.000 16.00019.1 20 -4.500 20.250 4.500 20.25019.2 20 -4.000 16.000 4.000 16.000

Gaseous Assessments Site ID: Burdens Pollutant type: NOy API

NOy API Percent Difference

-14.000

-12.000

-10.000

-8.000

-6.000

-4.000

-2.000

0.000

2.000

4.000

6.000

8.000

Observations

Per

cen

t D

iffe

ren

ce


AMP 255 Report

– Monitoring Orgs. Can run this as often as they wish

– OAQPS will run annually– Box–and-whisker plots

included in annual summary

http://www.epa.gov/ttn/amtic/parslist.html


Data Quality Objectives for the Precursor Gases

•DQOs are used to create quality parameters so that the data collected is of sufficient quality for its use

•OAQPS staff began the DQO Process 1.5 years ago

•OAQPS staff employed Battelle Inc. to run simulation models to examine how the different QA indicators would affect data quality

•From these the Measurement Quality Objectives are defined


DQO Table for Precursor Gas ParametersCarbon Monoxide Urban Site

AnnualCompleteness

HourlyMeasurementPrecision(CV)

Site-SpecificBias

Power to Detect Annual Trend at a Single Site

Power to Detect Annual Trend Across 50 Sites

Multiplicative1/2 Width for 95% PredictionInterval at anIndividual Site

Estimated Fraction of Data Falling in Prediction Interval

5% Reduction

10% Reduction

5% Reduction

10% Reduction

LowerBound

UpperBound

90% 7%-20% 5%-20% 75% 100% 100% 100% 9.6 27% 70%

75% 7%-20% 5%-20% 71% 100% 100% 100% 16.1 40% 75%


Burden’s Creek Station

• AAMG staff have been operating PG/trace gas instruments at BC for 2 years

•We have been operating like an NCore Station

• Since we perform 1-point QC tests, as described in 40 CFR 58, we can assess the data

•How do our results compare to the Battelle model?


Burden’s Creek MQOs (May 06 – April 07)

Coefficient of Variance

Bias Data Completeness

O3 API 2.1 +2.1 91.1%

CO–API 6.9 +9.4 76.7%

CO-TEI 5.8 +/-5.4 90.1%

NO-API 2.8 -4.7 84.1%

NOy-API 3.3 -5.7 84.1%

SO2-TEI 3.6 +/-3.0 92.5%


NPAP for Precursor Gas Sites

• Precursor monitoring will be part of NPAP • Monitoring organization are responsible for implementing adequate

and independent audits – Allows for continued Federal implementation with STAG funds

• NPAP program testing TTP capabilities for precursor gasses at Burdens Creek– Some equipment still being purchased– Expected testing in 1st quarter 2007


Expansion of Audit Levels

Concentration range, ppm Audit level O3 SO2, NO2 CO

1...... 0.02-0.05 0.0003-0.005 0.0002-0.002 0.08-0.10

2...... 0.06-0.10 0.006-0.01 0.003-0.005 0.50-1.00

3...... 0.11-0.20 0.02-0.10 0.006-0.10 1.50-4.00

4...... 0.21-0.30 0.11-0.40 0.11-0.30 5-15

5...... 0.31-0.90 0.41-0.90 0.31-0.60 20-50

• Low levels added to accommodate Trace Level Instruments

• At Least three consecutive audit levels

• The audit levels selected should represent or bracket 80 percent of ambient concentrations


Summary

• 40 CFR 58 Appendix A regulations have changed

• PQAOs vs. ROs• There are new statistical calculations

– New Tools to assist EPA and stakeholders• EPA has performed a DQO modeling exercise• The BC data illustrates that we can collect data

with the MQOs• NPAP TTP audit vehicles are now gearing up

Date post:	26-Dec-2015
Category:	Documents
Upload:	pauline-dalton
View:	219 times
Download:	0 times

EPA Precursor Gas Training Workshop Precursor Gas Quality Assurance Implementation Dennis K. Mikel...

Documents