8th EIONET Workshop AQ,Oslo 1 06 Nov ‘03 EoI 2003 EoI Report Upload process DEMv7.

8th EIONET Workshop AQ,Oslo 106 Nov ‘03

EoI 2003EoI 2003

• EoI Report

• Upload process

• DEMv7


EoI 2003 : EoI reportEoI 2003 : EoI report

• Status

• Use of DEM

• Transmission


EoI Report : StatusEoI Report : Status

• 28 countries reported

• 2 more countries expected

• Oct.1st : 75 % received


EoI Report : Use of DEMEoI Report : Use of DEM

• DEM reports: 85 % (EMEP: 4 MS)• Problems using DEM:

– adding/modifying meta information– adding results of new measurements– ...

• Reasons not using DEM:– problems using DEM– lack of time


EoI Report : TransmissionEoI Report : Transmission

• ETC ftp-site

• CD

• ReportNet/CDR


EoI 2003 : Upload processEoI 2003 : Upload process

• Procedure

• Report checks

• Communication

• Time schedule


Upload process: ProcedureUpload process: Procedure

• Report/quality checks • Upload DEM into AirBase

• Report results checks to Member States

• Process reply Member States

• Preparing for Report


Upload process: Report checksUpload process: Report checks

• Re-submitted raw data

• Deleted meta data

• Missing essential meta data

• Outliers



• Re-submitted data (general report)

COUNTRY COMP NMBR_STATIONS STARTDATE ENDDATE NMBR_RECORDSGB CO 40 01.04.1989 31.12.1996 25589GB NO2 54 01.01.1985 31.12.2001 35879GB NOX 53 01.01.1996 31.12.2001 15987GB O3 52 01.01.1985 31.12.1996 41173GB PM10 33 01.01.1993 31.12.1996 17897GB SO2 44 01.01.1993 31.12.1996 27520



• Re-submitted data (detail report)COUNTRY COMP STATIONNR STATIONNAME STARTDATE ENDDATE NMBR_RECORDSGB CO GB0040R STEVENAGE 01.04.1989 31.12.1992 1371GB CO GB0206A LONDON CROMWELL ROAD 01.01.1993 12.09.1996 986GB CO GB0420A WEST LONDON 01.01.1993 31.12.1996 1096GB CO GB0452A GLASGOW CITY CHAMBERS 01.01.1993 31.12.1996 1096GB CO GB0453A MANCHESTER TOWN HALL 01.01.1993 31.12.1996 1096GB CO GB0536A LONDON BRIDGE PLACE 01.01.1993 31.12.1996 1096GB CO GB0538A SHEFFIELD TINSLEY 01.01.1993 31.12.1996 1096GB CO GB0566A LONDON BLOOMSBURY 01.01.1993 31.12.1996 1096GB CO GB0567A BELFAST CENTRE 01.01.1993 31.12.1996 1096GB CO GB0568A NEWCASTLE CENTRE 01.01.1993 31.12.1996 1096GB CO GB0569A BIRMINGHAM CENTRE 01.01.1993 31.12.1996 1096GB CO GB0580A CARDIFF CENTRE 01.01.1993 31.12.1996 1096GB CO GB0581A EDINBURGH CENTRE 01.01.1993 31.12.1996 1096GB CO GB0583A MIDDLESBROUGH 01.01.1996 31.12.1996 366GB CO GB0584A LEEDS CENTRE 04.01.1993 31.12.1996 1093GB CO GB0585A BRISTOL CENTRE 04.01.1993 31.12.1996 1093GB CO GB0594A LIVERPOOL CENTRE 23.04.1993 31.12.1996 984GB CO GB0595A BIRMINGHAM EAST 01.12.1993 31.12.1996 762GB CO GB0596A HULL CENTRE 04.01.1994 31.12.1996 728



• Deleted meta data (stations)Overview deleted stations (EoI2002) for which data are stored in Airbase

COUNTRYNW_CODE NW_NAME SN_CODE SN_EU_CODE STATION_NAMEBE 123 FLANDERS N 4751 BE0138A 200VOID BE 147 CELINAIR 84 BE0114A LIEGE : 2 202 BE 147 CELINAIR 90 BE0115A LIEGE : 2 218 BE 147 CELINAIR 94 BE0160A LIEGE : 2 227 BE 147 CELINAIR 104 BE0116A CHARLEROI : 2 504 BE 147 CELINAIR 105 BE0117A CHARLEROI : 2 505 BE 147 CELINAIR 107 BE0118A CHARLEROI : 2 509 BE 147 CELINAIR 109 BE0119A CHARLEROI : 2 514 BE 229 WALLONIA N 84 BE0114A LIEGE : 2 202 BE 229 WALLONIA N 90 BE0115A LIEGE : 2 218 BE 229 WALLONIA N 94 BE0160A LIEGE : 2 227 BE 229 WALLONIA N 104 BE0116A CHARLEROI : 2 504 BE 229 WALLONIA N 105 BE0117A CHARLEROI : 2 505 BE 229 WALLONIA N 107 BE0118A CHARLEROI : 2 509 BE 229 WALLONIA N 109 BE0119A CHARLEROI : 2 514 BE 235 BRUSSELS N 6085 BE0342A 01R012B - IRM-KMI



• Deleted meta data (measurements)Overview deleted meas_configurations (EoI2002) for which data are stored in Airbase

COUNTRY NW_CODE NW_NAME SN_CODE SN_EU_CODE STATION_NAME COMP_ID COMP_NAME MC_CODE RD_TYPE DC_STARTDATE DC_ENDDATE AUTOMATICGR 256 ATHENS 2371 GR0031A Nea Smyrni 10 CO 110 day 01.07.1983 31.12.1995 Y GR 256 ATHENS 2371 GR0031A Nea Smyrni 10 CO 110 hour 01.07.1983 31.12.1995 Y GR 256 ATHENS 2371 GR0031A Nea Smyrni 38 NO 120 day 01.01.1987 31.12.1989 Y GR 256 ATHENS 2371 GR0031A Nea Smyrni 38 NO 120 hour 01.01.1987 31.12.1989 Y GR 256 ATHENS 2372 GR0032A Patission 8 NO2 100 day 01.07.1983 31.12.1995 Y GR 256 ATHENS 2372 GR0032A Patission 8 NO2 100 hour 01.07.1983 31.12.2000 Y GR 256 ATHENS 2372 GR0032A Patission 10 CO 110 day 01.07.1983 30.11.1984 Y GR 256 ATHENS 2372 GR0032A Patission 10 CO 110 hour 01.07.1983 30.11.1984 Y



• Missing essential meta data Overview stations with missing essential meta dataCOUNTRY SN_EU_CODE STATION_NAME LATITUDE(DEC) LONGITUDE(DEC) ALTITUDE STATION_TYPE TYPE_OF_AREA NETWORK_NAMENL NL0105A Breduisbad 52.39361111 4.86944444 0 Unknown urban Other network NL NL0106A Vegastraat 52.41166667 4.88944444 0 Unknown urban Other network NL NL0107A Einsteinweg 52.38527778 4.84333333 0 Unknown urban Other network NL NL0108A OZ Voorbrugwal 52.37222222 4.89555556 0 Unknown urban Other network NL NL0109A Museumstraat 0 0 0 Unknown unknown Other network NL NL0110A Kamerlingh-Onnesl 52.35111111 4.92388889 0 Unknown urban Other network NL NL0111A Buitenveldert 52.33055556 4.87722222 0 Unknown urban Other network NL NL0112A Beethovenlaan 52.04194444 4.22777778 0 Unknown urban Other network NL NL0113A Langenhorst 51.87555556 4.45333333 0 Unknown urban Other network NL NL0114A Achter Langest. 41 52.21972222 6.89694444 0 Unknown urban Other network NL NL0134A Ringweg - Zaandam west 52.42222222 4.79388889 -2 Unknown suburban Landelijk Meetnet Luchtkwaliteit NL NL0147A Geefweersterweg 53.32527778 6.90027778 0 Unknown rural Landelijk Meetnet Luchtkwaliteit NL NL0165A Axel 51.28527778 3.91611111 1 Unknown suburban Landelijk Meetnet Luchtkwaliteit NL NL0167A Louisenburgweg 51.37916667 6.22305556 40 Unknown urban Landelijk Meetnet Luchtkwaliteit NL NL0168A Papenlaan 52.13527778 4.42 -1 Unknown suburban Landelijk Meetnet Luchtkwaliteit NL NL0183A Wilgenstraat 52.46194444 4.63555556 2 Unknown urban Landelijk Meetnet Luchtkwaliteit NL NL0186A Biddinghuizen Kuilweg 52.425 5.59416667 -4 Unknown rural Landelijk Meetnet Luchtkwaliteit NL NL0204A Amsterdam-Florapark 52.39194444 4.91777778 0 Background urban Landelijk Meetnet Luchtkwaliteit



• Outliers (general report)Quality Assurance / Quality Control

Component(s) checked:----------------------------------- -------- ---------- ------------ ----------- Checked on value Component Avg.Time Unit Smaller than Larger than----------------------------------- -------- ---------- ------------ -----------CO (air) hour mg/m3 -10 20NO2 (air) hour ug/m3 -10 450NO (air) hour ug/m3 -10 900O3 (air) hour ug/m3 -10 400PM10 (aerosol) day ug/m3 -10 525PM2.5 (aerosol) day ug/m3 -10 525SO2 (air) hour ug/m3 -10 450TSP (=SPM?) (aerosol) hour ug/m3 -10 850----------------------------------- -------- ---------- ------------ -----------

Results:------- ------------------------------------------- -------------------Country Component Avg.Time Status ------- ------------------------------------------- -------------------AT CO (air) hour OK AT NO2 (air) hour OK AT NO (air) hour OUTLIER(S) DETECTEDAT O3 (air) hour OK AT PM10 (aerosol) day OK AT PM2.5 (aerosol) day OK AT SO2 (air) hour OK AT TSP (=SPM?) (aerosol) hour OUTLIER(S) DETECTED------- ------------------------------------------- -------------------



• Outliers (detail report)Component: NO (air)Unit: ug/m3Startyear: 2002Average time: hourChecked on values smaller than: -10 or larger than: 900------- ----------------------------------- ------- -------------------------------------------------- ---------------------------- Station Station Country Component EU code Name Hour Outlier ------- ----------------------------------- ------- -------------------------------------------------- ---------------- ------------AT NO (air) AT0182A Vomp A12 Inntal-Highway 01-09-2002 09:00 1033AT NO (air) AT0182A Vomp A12 Inntal-Highway 01-09-2002 10:00 948AT NO (air) AT0182A Vomp A12 Inntal-Highway 01-10-2002 07:00 974AT NO (air) AT0182A Vomp A12 Inntal-Highway 01-10-2002 08:00 1006AT NO (air) AT0182A Vomp A12 Inntal-Highway 01-10-2002 10:00 1092AT NO (air) AT0182A Vomp A12 Inntal-Highway 01-11-2002 09:00 1051AT NO (air) AT0182A Vomp A12 Inntal-Highway 01-15-2002 22:00 956AT NO (air) AT0182A Vomp A12 Inntal-Highway 01-16-2002 09:00 903AT NO (air) AT0182A Vomp A12 Inntal-Highway 02-13-2002 07:00 919AT NO (air) AT0182A Vomp A12 Inntal-Highway 04-03-2002 07:00 905AT NO (air) AT0182A Vomp A12 Inntal-Highway 12-12-2002 08:00 941------- ----------------------------------- ------- -------------------------------------------------- ---------------- ------------


Upload process: CommunicationUpload process: Communication

• Status upload EoI 2003 on website

• reply results upload reports:– re-submission raw data -> confirmation– outliers -> re-

submission– deleted meta data

with data stored in AirBase -> confirmation– missing essential meta data -> updated DEM

• Technical/AQ report 2002


Upload process: Time scheduleUpload process: Time schedule

Before:

1st Jan’04 results checks upload EoI 2003

1st Feb’04 reply member states

1st Mar’04 final report upload EoI 2003


EoI 2003 : DEMv7EoI 2003 : DEMv7

• Integration with New Ozone Directive (2002/3/EC) dataflow

• Integration with Reportnet (Upload to CDR)

• Solving bugs and other problems in DEMv6


Data flows Ozone Directives Data flows Ozone Directives 92/72/EEC and 2002/3/EC92/72/EEC and 2002/3/EC

What Format old new

monthly <1/n+2 ex1 ex1

summer 31/10 - ex 8, aot40, 1hmax

annual30/9 ex1/8/24 ex1/8, [stat],

stat [prec], aot40


Experiences with Directive Experiences with Directive 92/72/EEC data flows92/72/EEC data flows

Major problems:

• Station meta information not complete

• Linking info between files (which data belongs to which station)


ProposalProposal

• Spreadsheet based

• Link between files based on EU station code

• New stations: station sheet

• Existing stations: meta info from AirBase

• Annual data combining with EoI (DEM)


Sheets:Sheets:

• Exceedance sheet (monthly, summer, year)• Summer statistics• Annual statistics (aot40, [mean, percentiles,

max])• Precursor sheet [ann mean, percentiles,

max]• Station sheet (new stations) [station info]


Sheets-import in DEM:Sheets-import in DEM:

• Only in the annual (30/9) delivery• Annual exceedance, statistics and precursor

sheets can also be imported in DEMv7• The annual statistics and precursor data can

also be imported in DEM with standard DEM import formats

• At the annual report new station information can be processed directly in the DEM.


DataflowsDataflows

Sheet

monthly Exceedance (Station)

summer Exceedance (Station)

Statistics

annualExceedance

[Statistics] [DEM]

[Precursor] [DEM]


AQ-DEM and ReportNetAQ-DEM and ReportNet

AQ-DEM1.Supports dataflow from

MS national database to European data warehouse's AirBase

2.QC on meta information (networks, stations, meas. Conf.)

3.QA/QC on measurement data

Data flow1.ETC-ACC receives AQ-DEM’s

(MS Access mdb files) on CDrom or by FTP

2.The MS load the AQ-DEM also in the CDR

Availability of data1.Airbase (available via

Internet (AirView))2.XML-dumps of complete

AirBase on ETC/ACC-ftp3.Factsheets4.Meta-data in techn. report

on national data reporting

Data Dictionary1.AQ-DEM can support the

development of a data dictionary format derived from the data model of Airbase according the requirements of the decisions and directives.

Legal basis• EoI decision• Ozone directive• EuroAirnet• EMEP/CCC• AQ reporting

requirements, formats and guidances

Indicator production• Air pollution indicators by

using air quality data from Airbase


Transmitting data from EU-countries to Commission, Transmitting data from EU-countries to Commission, EEA, ETC/ACC and EMEP/CCCEEA, ETC/ACC and EMEP/CCC

• EoI/EuroAirnet (+ EMEP) data – CDROM to EU-Commission (Inform ETC/ACC and EEA)

or:

– CDROM to ETC/ACC (Inform EU-Commission and EEA)

or:

– Upload to EEA’s CDR (Inform EU-Commission and ETC/ACC)

if CDR failed then:• Upload to ETC/ACC FTP (Inform EU-Commission And EEA)

• Only EMEP data in DEM:– Upload to ETC/ACC FTP (Inform EMEP/CCC)


Transmitting data from non-EU-countries to EEA, Transmitting data from non-EU-countries to EEA, ETC/ACC and EMEP/CCCETC/ACC and EMEP/CCC

• EoI/EuroAirnet (+ EMEP) data – CDROM to ETC/ACC (Inform EEA)

or:

– Upload to EEA’s CDR (Inform ETC/ACC)

if CDR failed then:• Upload to ETC/ACC FTP (Inform EEA)

• Only EMEP data in DEM:– Upload to ETC/ACC FTP (Inform EMEP/CCC)


Bugs Demv6Bugs Demv6

• It is not possible to add new measurement configurations and to modify existing measurement configurations ( NL, GR, BU, RO, SE )

• The station identification in the mark-file should be not only the station name but also the station code (DE)


Other problems (1)Other problems (1)

• The generation of the NASA Ames header file gives problems when importing EMEP-info by means of DEM-file (NL). When exporting the NASA Ames files the quality flags are not correct.

• Comboboxes of the Station and Measurement configuration screens do not work properly.

• The Meta Information, Measurement Configuration report does not work.

• Visualisation of weekly and annual means is not ok.

• Timing day samples can start at another time than 0:00 hour. (When importing the samples the time is default set at 0:00 hr) (NL)

• Missing some components in the component list (iso-hexane, benzo(a)pyrene, nitrogen monoxide). Benzene in the main list. (UK, NL, AT).


Other problems (2)Other problems (2)

• When on a station a component is measured with two measurement methods, only one method is in the Mark-file (NL).

• When a station is shut down during a year, this is not identified in the Mark-file (NL).

• Some small problems with QA/QC module (NL).• Unable to enter 2-week benzene means since some of the data is 13, 14 or 15-

day samples. (UK)• In the MeteoPar. tab also the possibility: “no meteo measured”. (NL).• The .mdb DEM-file can be very big: > 1 Gb. Add a warning and the access

option “compact database” (=standard access tool) (UK, NL). • Problems with decimal symbol in the coordinates (AT).

• Changing the manager info changes automatically the data supplier info (NL).

Date post:	01-Apr-2015
Category:	Documents
Upload:	lilly-aspinwall
View:	221 times
Download:	1 times

8th EIONET Workshop AQ,Oslo 1 06 Nov ‘03 EoI 2003 EoI Report Upload process DEMv7.

Documents