Slide 1 WMO BUFR training course Langen 17-20 April 2007 Slide 1 Ecmwf observations data handling...

WMO BUFR training course Langen 17-20 April 2007

Slide 1

Slide 1

Ecmwf observations data handling

Milan Dragosavac


Slide 2

Slide 2

MDB RDB PDBFDB

ODB

IFS

MARS

General applications

Analysis Forecast

DisplayPrintPlot

Data transport

Datatransport

Data acquisition


Slide 3

Slide 3

Data acquisitionAcquire dataProcess dataArchive data ( not MARS)Outgoing ( send data

outside Ecmwf)


Slide 4

Slide 4

Acquisition processGTS data ftp Other data


Slide 5

Slide 5

Acquisition outgoingQscat satellite data in BUFR

formSSMI brightness

temperaturesTOVS 1B calibrated and

converted into BUFR format


Slide 6

Slide 6

MEB feedRestores any old dataJoin segmented bulletinsWrite bulletins in MEBstatistics


Slide 7

Slide 7

Message data base“STREAM”_YYYYMMDDAIRC_20070216AIRS_20070216AMAP_20070216GRIB_2007021675 streams in totalOne stream per day


Slide 8

Slide 8

Message data base attributesTIME character( 4,1)OBSTYPE character( 4,1)TTAAii character( 6,1)BBB character( 3,1)CCCC character( 4,1)_SOURCE character(80,1)_CHECKSUM longinteger_TIMESTAMP character(20,1)


Slide 9

Slide 9

Message data base attributes continued_data_size longinteger_bulk_size longinteger_owner character( 3,1)_last_access time(0)_access_number longinteger_compress character( 1,1)_data bulk(20,0,512,1)


Slide 10

Slide 10

Message data base indicesUNIQUE I_timestamp_SYNO_20070215 on (_TIMESTAMP)UNIQUE I_comp_SYNO_20070215 on (TIME, TTAAii, BBB,

_CHECKSUM)NORMAL (2,62) I_OBSTYPE_SYNO_20070215 on

(OBSTYPE)


Slide 11

Slide 11

Message data base100 Gbytes allocated + 150 Gbytes for METOP dataOver 22 Gbytes / daySaved on ECFS according to OBSTYPEAll data can be reprocessed if Data base is lostHP cluster 4 CPU 64 bit processors4 operational machines + 1 for developmentRaid disks 1.2 Tbytes


Slide 12

Slide 12

MEB data arrivalMonitoring

arrival times of different data types

Analysis cycleAverage number

of days


Slide 13

Slide 13

Message data base data availabilityData availability

according to synoptic hour


Slide 14

Slide 14

MEB x-interfaceAllows MEB

searchFind

corresponding BUFR observation in REB

Expand BUFR and GRIB data

Write data into file


Slide 15

Slide 15

MEB x-interfaceREB BUFR observations

corresponding to MEB bulletin


Slide 16

Slide 16

Preprocessing suiteMEB – message data base

feed taskdeqc – decoding, quality

control and BUFR creation, writing into REB

ex00 – extraction for analysis

bc00 – extraction for boundary condition suite

wmo – ftp WMO station list


Slide 17

Slide 17

DecodingDecodingQuality controlBUFR creationGlobal grib fields creationWriting into REB


Slide 18

Slide 18

REB extraction for analysis14 files createdConventional observationsSatellite dataSea surface temperatures Ice analysis


Slide 19

Slide 19

REB – report data baseOne table per observation type per dayAIRC_20070215ATOV_20070215B002_20070215CREX_20070215TEMP_20070215SSBT_20070215QSCA_20070215statistic


Slide 20

Slide 20

REB attributesTIME character(6,1)IDENT character(9,1)_BUFR_TYPE integer_BUFR_SUBTYPE integer_LATITUDE1 float(6)_LONGITUDE1 float(6)_COR1 integer_QC integer


Slide 21

Slide 21

REB attributes continued_TIMESTAMP character(20,1)NR_SUBSETS integer_data_size longinteger_bulk_size longinteger_owner character(3,1)_last_access time(0)_access_number longinteger_data bulk(20,0,512,1)


Slide 22

Slide 22

REB indicesUNIQUE I_ts_SYNO_20070215 on (_TIMESTAMP)UNIQUE I_comp_SYNO_20070215 on (TIME, IDENT,

_BUFR_SUBTYPE, _LATITUDE1, _LONGITUDE1NORMAL (2,15) I_extract_SYNO_20070215 on (TIME,

_BUFR_SUBTYPE, IDENT)


Slide 23

Slide 23

REB reports data baseJust bellow 22 Gbytes / day100 Gbytes allocated + 150 Gbytes for METOP dataAll data can be re-decoded at any timeData can be accessed through MARS request and

MetviewArchived in MARS


Slide 24

Slide 24

REB x-interfaceSearch REBFind original bulletin in

MEBExpand BUFR and GRIB

dataWrite data to fileAccess WMO station list

data base


Slide 25

Slide 25

BUFR x-interfaceExpands BUFR messageAny subset can be viewedFiltering features


Slide 26

Slide 26

WMO Data BaseWMO_a_masterWMO_a_master_manualWMO_a_master_tape


Slide 27

Slide 27

WMO_a_master attributesRegion_country integer not nullBlock smallint not nullStation_index integer not nullStation_subindex smallint not nullPrincipal _station character(1,1)Station_name text(20,20,30,1)Latitude realLongitude real


Slide 28

Slide 28

WMO_a_master attributes continuedPressure_elevation integerPressure_elevation_flag character(1,1)Ground_elevation integerGround_elevation_flag character(1,1)Pressure_level smallintSurf_prog_00 character(2,1)Surf_prog_03 character(2,1)Surf_prog_06 character(2,1)Surf_prog_09 character(2,1)Surf_prog_12 character(2,1)


Slide 29

Slide 29

WMO_a_master attributes continuedSurf_prog_15 character(2,1)Surf_prog_18 character(2,1)Surf_prog_21 character(2,1)Ua_type_00 smallintUa_type_06 smallintUa_type_12 smallintUa_type_18 smallintModify_flag character(1,1) not null


Slide 30

Slide 30

WMO_a_master indicesUNIQUE I_wmo_a_master ON (block,station_index,

station_subindex)


Slide 31

Slide 31

WMO station list


Slide 32

Slide 32

ODB Observational data baseInterface to analysisData base contains unpacked bufr data and original bufr

dataLoading done in parallel with load balance achieved on

supercomputer due to bufr featuresAnalysis updates ODBAnalysis bufr feedback files createdArchiving of feedback files in MARS


Slide 33

Slide 33

Loading ODB with observationsTasks running in

parallel2-16 processors

used per taskLoad balancing

achieved


Slide 34

Slide 34

ODB to BUFRodb2bufr family


Slide 35

Slide 35

Odbviewer


Slide 36

Slide 36

Odbviewer


Slide 37

Slide 37

Odbviewer


Slide 38

Slide 38

Odbviewer


Slide 39

Slide 39

Odbviewer


Slide 40

Slide 40

Odbviewer


Slide 41

Slide 41

Odbviewer


Slide 42

Slide 42

FutureSharp increase in volume of observations expected IASI data with 8641 channels per observationData bases have crucial role in observations handlingBinary data representation allows easy handling of

observations as bulk objects

Date post:	20-Jan-2018
Category:	Documents
Upload:	coral-fox
View:	217 times
Download:	0 times

Slide 1 WMO BUFR training course Langen 17-20 April 2007 Slide 1 Ecmwf observations data handling...

Documents