Uncertainties in measurement and modelling : an overview

Laurence Rouïl

In-situ Measurement data : main sources

Regulatory observation sites (in compliance with the Air quality directives) • “Selected” air pollutants and parameters measured• Obligations related to the choice of the observation site and the standards used for the

measurement devices • Commitment of the Member States to comply with the directives in term of station

number, location, quality assurance, reporting (EIONET network)• Data reported to the European Environment Agency and made routinely available

Research networks grow up in Europe : • New parameters measured : non regulatory pollutants, aerosol speciation, size

distribution, physico-chemical parameters vertical profiles (Lidars, Radiosondes), aircraft measurements …

• Wide range of methods can be tested and compared • Continuous measurement and/or fields campaigns (EUSAAR, EARLINET, GALION,

EUCAARI, MOZAIC/IAGOS….)• Data compiled by the project partners and made available under certain constraints

(publication, restrictive use…)

Uncertainties in measurement : Data quality objectives (DQOs) specified in particular in AQ Directives :

• Measurement uncertainty• Minimum data capture• Minimum time coverage

Metrological uncertainty : from the measurement devices; rather well managed for regulatory pollutants • Appropriate standards are developed by normalisation Committees (CEN, ISO)

according to the requirements of the Air quality Directives (e.g. measurement uncertainty lower than 30% in most of the cases)

• Definition of reference methods and inter-laboratory tests• Definition of common statistical procedures for uncertainty estimations

Metrological uncertainty : a field of investigation for research networks • Intercalibration campaigns (see the EUSAAR project) : EC/OC measurement,

optical properties, size distribution (SPMS)....

Intercalibration experiments (from P. Laj) :OC/EC (J. –P. Putaud, JRC): Round-Robin intercomparison and development of artefact free

sampler• Intercomparison of identical filters from several EUSAAR sites operating operating with

similar thermo-optical methods• Need for homogeneizing methods -> Converging towards a EUSAAR

method for thermal-optical methods and EMEP references

Size distribution (A. Wiedensohler, IFT): intercalibration and improvement of SMPS

• 34 CPCs (12 different models) and 16 SMPS were checked and calibrated• Intercalibration clearly needed. High

variability in terms of total number and size • Improvement when using standard

retrieval procedures

Uncertainties in measurement (ii)

Uncertainties in measurement interpretation• Which parameters are measured?• Artefacts in the measurement?• How to retrieve the expected data (concentration level) from the available measurement

(AOD for instance)? Non validated and validated data : role of the human expertise

• Reporting chains (EMEP, EEA) include data flagging to qualify the status and the quality of the data

• Time release of validated data must be improved in most cases (EMEP)• Access to Near Real Time (NRT) unvalidated data offers new opportunities (monitoring of air

pollution episodes, air quality forecasting and short term analysis, NRT model evaluation....) but can increase uncertainties.

Uncertainty due to the measurement strategy :• Representativeness of the observations : to reduce uncertainties in maps production and air

quality assessment• Performance in terms of data capture and time coverage

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2400 2400

2500 2500

2600 2600

Y (km)

Y (km)

Jeu de données copmlet

Isatis

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2400 2400

2500 2500

2600 2600

Y (km)

Y (km)

Sélection 15km 1

Isatis 700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

Sélection 15km 2

Isatis 700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km) Y

(km)

Sélection 20km 1

Isatis 700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km) Y

(km)

Sélection 20km 2

Isatis

Δx= 15 km (1)

Δx= 15 km (2)

Δx= 20 km (1) Δx= 20 km (2)

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2400 2400

2500 2500

2600 2600

Y (km)

Y (km)

Sélection 25km 1

Isatis 700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

Sélection 25km 2

Isatis 700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)Sélection 30km 1

Isatis 700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (k

m)

Sélection 30km 2

Isatis

Δx= 25 km (1) Δx= 25 km (2) Δx= 30 km (1) Δx= 30 km (2)

Example : sensitivity to the spatial sampling strategy

x : mesh size

Initial data set (source: ATMO Champagne-Ardenne, 2005)

Example :

NO2 background concentrations over the region Champagne-Ardenne (France) – winter 2005

Spatial sampling strategy

Ordinary kriging – Estimated maps

Isatis

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH KO pep=0 >=403938373635343332313029282726252423222120191817161514131211109<8

N/A

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH KO pep=0 S15km1 ss10i

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH KO pep=0 S15km2

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH KO pep=0 S20km1

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH KO pep=0 S20km2

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH KO pep=0 S25km1 ss10i

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH KO pep=0 S25km2

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH KO pep=0 S30km1

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH KO pep=0 S30km2

Isatis

Sensitivity of the estimated map to sampling density. The sampling mesh should not be larger than 15 km.

Spatial sampling strategy

Kriging of the residuals using population and NOx emissions density– Estimated maps

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH Kres

N/A

40.0000039.0000038.0000037.0000036.0000035.0000034.0000033.0000032.0000031.0000030.0000029.0000028.0000027.0000026.0000025.0000024.0000023.0000022.0000021.0000020.0000019.0000018.0000017.0000016.0000015.0000014.0000013.0000012.0000011.0000010.00000 9.00000 8.00000

Isatis

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH Kres S15km1 ss10i

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH Kres S15km2

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH Kres S20km1

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH Kres S20km2

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH Kres S25km1 ss10i

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH Kres S25km2

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH Kres S30km1

700

700

750

750

800

800

850

850

X (km)

X (km)

2300 2300

2350 2350

2400 2400

2450 2450

2500 2500

2550 2550

Y (km)

Y (km)

MoyenneH Kres S30km2

Isatis

With auxiliary variables, the sampling mesh can be extended to 25-30 km..

Uncertainties in modelling

Estimated by comparison with measurement :• Statistical scores (bias, root mean square error, gross error, correlation) • Graphical indicators (Taylors diagrams)• Contingency tables assessing the ability of the model to capture situations where

thresholds are exceeded or not Various sources of uncertainties :

• input data: emissions and meteorological fields (V, temperature, . . .) ;• physical parameterizations (ci , K, . . .) ;• numerical schemes• Model resolution

Sensitivity to input data : propagating input uncertainty in the models with Monte-Carlo approaches

Méthodology

• Probability Distribution Function (PDF) for input parameters

PDFs propagates in the CTMs with a Monte Carlo approach Hanna et al. [1998, 2001], Beekmann and Derognat [2003]

Sources :• Parole d'expert• Erreur de mesure• Ecart aux observations

AQ model

PDFs parametersPDF concentrations

Standard deviation : measure of the outputconcentration uncertainty.

Parameter PDF Factor

Wind speed LN 1.5

Temperature N 1%

PM emissions LN 4

Example : CHIMERE – France results

Ozone august 2009500 simulations

PM10 winter 2009300 simulations

• Standard deviation : 19% for ozone daily peak et 33% PM10 daily average• Lower for highest concentrations • Uncertainty can be underestimated for PM model concentrations, the bias being also underestimated

Identification of the sensitive variables for ozone concentrations

• Temperature• Lateral boundary conditions• Deposition speed

The ensemble approach to assess model uncertainty

14

From the individual model verification....

… to the multi-model analysis : range of variability = a kind of model uncertainty measurement

Biais RMS

Model intercomparison and evaluation exercises : a promising approach to assess model uncertainty

The AQMEII initiative : JRC (S. Galmarini), USEPA (S.T. Rao)

The Eurodelta initiative : with JRC, CONCAWE,Next phase under the TFMM umbrella

Emissions, modelling and measurement ….. Close relationship : missing sources (natural) , inaccurate approximation (diffusive

emissions, wood combustion...) can explain a part of uncertainty in model results High temporal resolution for emissions can be crucial for forecasting or NRT monitoring

applications Observation should help in improving emission events ; new opportunities with earth

observation Modelling should help in assessing emission inventories Inverse modelling : considering “reduced” uncertainties of observations to constrain

models and to improve emission inventories next operational step?

Impact of high resolution emission inventory MACC/TNO) on NO2 daily peak simulated by CHIMERE (RMS)