Data Quality and Measurement Quality Objectives for the National Air Toxics Trends j
Program
Dennis K MikelDennis K. MikelEPA – OAQPS
AT Workshop 2011
•Urban Sites •Rural
•E. Providence, RI•Boston (Roxbury), MA•New York, NY•Rochester, NYW hi t DC
•Chicago, IL•Houston (Deer Park), TX•St. Louis, MO•Bountiful, UTS J CA
•Underhill, VT• Grayson Lake, KY•Chesterfield, SC•Grand Junction, CO•Washington, DC
•Decatur, GA•Tampa, FL (2)•Detroit, MI•Los Angeles, CA•Rubidoux, CA
•San Jose. CA•Phoenix, AZ•Seattle WA•Richmond, VA•Portland, OR
CO•La Grande, OR•Harrison County, TX•Mayville, WI
NATTS QA ObjectiveNATTS QA Objective
Data Quality Objective (DQO) is in support of the GPRA goal of reduction of Air Toxics by 75% (1993 levels) by 2010:
“To be able to detect a 15% difference (trend) between two successive 3-year annual mean concentrations within acceptable levels of decision error.”
To meet this DQO we need:
• 1-in-6 day sampling frequency with at least an 85% quarterly completeness
• Precision controlled to a Coefficient of Variance (CV) of no more than 15%
• Detectability based on 2001 Pilot Study Minimum Detection Limits (MDLs)
• Bias for the data set of less than 25%• Bias for the data set of less than 25%
These are our Measurement Quality Objectives (MQOs)!
Measurement Quality ObjectivesMeasurement Quality Objectives
How are these determined?
• Precision: Determined by analyzing collocated/duel samples or analyzing sample twice (replicate)
• Bias: EPA sends out blind PT samples to the labs – report back results
• Completeness: Calculated from number of samples reported to AQS divided by possible samples
• Detectability: Determined by laboratories using EPA Method of 7 duplicate samples near the level of detection
NATTS QA ProgramNATTS QA Program
Measurement Quality Objectives (MQOs)Measurement Quality Objectives (MQOs)
Compound Precision Bias Detectability Completenessp(CV) (Lab)
y p
Arsenic < 15% < 25% 0 217 ng/m3 > 85%Arsenic < 15% < 25% 0.217 ng/m3 > 85%
Benzene < 15% < 25% 0.016 ug/m3 > 85%
1 3-Butadiene < 15% < 25% 0 013 ug/m3 > 85%1,3-Butadiene < 15% < 25% 0.013 ug/m3 > 85%
Formaldehyde < 15% < 25% 0.0074 ug/m3 > 85%
Meeting Objectives: Precision Results 2004 - 2008Five Year Average:
Benzene: 25%NATTS - Method Precision 2004 - 2008
1,3 Butadiene 19%
Formaldehyde: 16%
Arsenic: 12% 50
60
Only Arsenic is meeting our Precision MQO
30
40
% C
V
Benzene
1,3 Butadiene
10
20
30%
Formaldehyde
Arsenic
0
10
2004 2005 2006 2007 2008
Years
M ti Obj ti O ll P i i A l ti l P i i 2008Meeting Objectives: Overall Precision vs. Analytical Precision -2008
Arsenic
Analytical Precision t f
1 3 Butadiene
Formaldehyde
Method
Analytical
accounts for laboratory practices and instrumentation
M th d P i i
Benzene
1,3 Butadiene y Method Precision accounts for analytical precision, sample handling and fi ld ti
0 5 10 15 20 25 30 35 40 45 50 55 60
Coefficient of Variance (%)
field operations
Meeting Objectives: Overall Precision of 1,3 butadiene + Benzeneg j ,
Benzene 1,3, Butadiene
Meeting Objectives: Method Detection Limits 2004 2008
2.00
Benzene 1,3 BD Formaldehyde Arsenic
Meeting Objectives: Method Detection Limits 2004 - 2008
0.50
1.00
0.13
0.25
/m3
/m3
2004
2005
2006
2007
0.03
0.06
ug
/n
g/ 007
2008
MQO
0.01
0.02
0.00
Meeting Objectives: Method Detection Limits 2004 – 2008 eet g Object es et od etect o ts 00 008
1.2
8
1
0.6
0.8
ug
/m3
ng
/m3
Benzene
1,3, BD
0.4
u n Formaldehyde
Arsenic
0.2
0
2004 2005 2006 2007 2008
Meeting Objectives: Data Completeness 2004 – 2008
120
80
100
enes
s
60
benzene
1,3 Butadiene
Formaldehydennua
l com
plet
e
40
Arsenic
% a
n
20
0
2004 2005 2006 2007 2008
Meeting Objectives: Proficiency Testing: Arsenic 2004- 2009
5
Meeting Objectives: Proficiency Testing: Arsenic 2004- 2009
4
3
s (n
g/fil
ter)
2
Lab
Res
pons
e
0
1
L
00 1 2 3 4 5
True (ng/filter)
12
Meeting Objectives: Proficiency Testing: Benzene 2004- 2009
10
12
6
8
e pp
b-v)
4
6
Lab
Res
pons
2
00 2 4 6 8 10 12
True (ppb-v)
M ti Obj ti P fi i T ti 1 3 B t di 2004 2009
14
Meeting Objectives: Proficiency Testing: 1, 3 Butadiene 2004- 2009
10
12
v)
6
8
spon
ses
(ppb
-v
4
6
Lab
Res
0
2
0 2 4 6 8 10 120 2 4 6 8 10 12
True (ppb-v)
Meeting Objectives: Proficiency Testing: Formaldehyde 2004- 2009
4.5
5
g j y g y
3.5
4
2.5
3
es (u
g/fil
ter)
1.5
2
Lab
Res
pons
e
0.5
1
0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
True (ug/filter)
Meeting Objectives: Acrolein PT Results 2006 - 2009
10
7
8
9
5
6
7
pons
es (p
pb-v
)
3
4
Lab
Res
p
1
2
00 1 2 3 4 5 6 7 8 9 10
True (ppb-v)
Summary: Is the Program Able to Meet the DQOs?
Yes and NoYes and No
The data completeness (2007-2008) is above the required 85% good job everyone!
Overall precision data illustrates that we are not meeting a CV of less than 15% with the exception of Arsenic
Th MDL d t ill t t th t t ti The mean MDLs data illustrates that we are not meeting our MQO for detectability with the exception of Arsenic
The laboratories are meeting the 25% bias requirement
Take Home Message: We now have a realistic view of the quality of Air Toxics data - This will guide us in the formulating new Data Quality Objectives