+ All Categories
Home > Documents > Introduction to BUFR

Introduction to BUFR

Date post: 02-Jan-2016
Category:
Upload: shea-rowland
View: 36 times
Download: 1 times
Share this document with a friend
Description:
Introduction to BUFR. Simon Elliott EUMETSAT [email protected]. What is BUFR?. B inary U niversal F orm for the R epresentation of Meteorological Data Used for data that are not on a regular grid, such as observations Conceptually equivalent to CREX, but format is - PowerPoint PPT Presentation
63
WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, 17 - 20 April, 2007 Introduction to BUFR Simon Elliott EUMETSAT [email protected]
Transcript
Page 1: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Introduction to BUFRSimon Elliott

EUMETSAT

[email protected]

Page 2: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

What is BUFR?

• Binary Universal Form for the Representation

of Meteorological Data• Used for data that are not on a regular grid, such as

observations• Conceptually equivalent to CREX, but format is

binary rather than alphanumeric

Page 3: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

What does a BUFR message look like? 01000010010101010100011001010010000000000000000000110100000000110000000000000000

00010010000000000000000000111000000000000000000000000000000000000000100100000001

00000001000001000001110100001100000000000000000000000000000000000000111000000000

00000000000000011000000000000001000000010000000100000010000011000000010000000000

00000000000000000000100000000000100100001111010111011100010000000011011100110111

0011011100110111

(In other words, just an apparently random string of 0’s and 1’s!)

Page 4: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Sections of a BUFR message

• 0 Indicator section

• 1 Identification section

• 2 Optional local use section

• 3 Data description section

• 4 Data section

• 5 End of message

Page 5: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Section 0 – Indicator section

• The character string “BUFR” indicating the start of the message

• The total length of the message

• The BUFR edition number

This section contains:

Page 6: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Section 0 - Details

• Length always 8

• Octets 1-4 “BUFR” (in CCITT IA5)

• Octets 5-7 Total length of message

(including Section 0)

• Octet 8 Edition number

(currently 4, but 3 is still used)

Page 7: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Now, let’s go back and look at that BUFR message again…

‘B’ ‘U’ ‘F’ ‘R’ end of section 0 +

octet number 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |

binary string 01000010010101010100011001010010000000000000000000110100000000110000000000000000

00010010000000000000000000111000000000000000000000000000000000000000100100000001

 

00000001000001000001110100001100000000000000000000000000000000000000111000000000

 

00000000000000011000000000000001000000010000000100000010000011000000010000000000

00000000000000000000100000000000100100001111010111011100010000000011011100110111

0011011100110111

Page 8: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Section 1 – Identification section

• The table versions referred to by this message• An overall description of the message contents,

including:– The originating centre and sub-centre

– The data category and sub-category

– A representative date and time

• Whether or not the optional section is included

This section contains:

Page 9: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Section 1 – Details(example based upon BUFR edition 3)

• Length at least 18

• Octets 1-3 Length of section

• Octet 4 Master table (0 for WMO, 10 for IOC, etc.)

• Octet 5-6 Originating sub-centre and centre

• Octet 7 Update sequence number

• Octet 8 Flag (Optional section?)

• Octets 9-10 Data category and sub-category

• Octets 11-12 Master and local table version numbers

• Octets 13-17 Date and time typical of message contents

• Octets 18-?? Reserved for local use

Page 10: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Now, let’s go back and look at that BUFR message again…

‘B’ ‘U’ ‘F’ ‘R’ end of section 0 +

octet number 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 1 | 2 |

binary string 01000010010101010100011001010010000000000000000000110100000000110000000000000000

octet number 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |

binary string 00010010000000000000000000111000000000000000000000000000000000000000100100000001

  end of section 1 +

octet number 13 | 14 | 15 | 16 | 17 | 18 |

binary string 00000001000001000001110100001100000000000000000000000000000000000000111000000000

 

00000000000000011000000000000001000000010000000100000010000011000000010000000000

00000000000000000000100000000000100100001111010111011100010000000011011100110111

0011011100110111

Page 11: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Section 2 – Optional section

• It typically contains additional information of use to the ADP centre, such as– Database keys to aid searching for specific data

without decoding the message– Anything else a processing centre may find useful

This section is defined by the ADP (Automated Data Processing) centre generating or using the message

Page 12: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Section 3 – Data description section

• A count of the number of data subsets (typically individual observations)

• Flags indicating whether or not the data are compressed or uncompressed and observed or forecast

• A list of the data elements (fields) that are contained in each data subset

This section contains:

Page 13: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Section 3 - Details

• Length at least 10• Octets 1-3 Length of section• Octet 4 Set to zero• Octets 5-6 Number of subsets• Octet 7 Flag (Obs?, Compressed?)• Octets 8-?? List of descriptors• Each descriptor 2 bits F, 6 bits X, 8 bits Y

Page 14: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Now, let’s go back and look at that BUFR message again…

‘B’ ‘U’ ‘F’ ‘R’ end of section 0 +

octet number 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 1 | 2 |

binary string 01000010010101010100011001010010000000000000000000110100000000110000000000000000

octet number 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |

binary string 00010010000000000000000000111000000000000000000000000000000000000000100100000001

  end of section 1 +

octet number 13 | 14 | 15 | 16 | 17 | 18 | 1 | 2 | 3 | 4 |

binary string 00000001000001000001110100001100000000000000000000000000000000000000111000000000

  end of section 3 +

octet number 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 |

binary string 00000000000000011000000000000001000000010000000100000010000011000000010000000000

00000000000000000000100000000000100100001111010111011100010000000011011100110111

0011011100110111

Page 15: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Section 4 – Data section

• The actual data as specified by Section 3

• One of two formats is used– Compressed– Uncompressed

• Such data are still packed, but not as efficiently as compressed data usually are

This section contains:

Page 16: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Section 4 - Details

• Octets 1-3 Length of section

• Octet 4 Set to zero

• Octets 5-?? Binary data as specified

by Section 3

Page 17: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Now, let’s go back and look at that BUFR message again…

‘B’ ‘U’ ‘F’ ‘R’ end of section 0 +

octet number 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 1 | 2 |

binary string 01000010010101010100011001010010000000000000000000110100000000110000000000000000

octet number 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |

binary string 00010010000000000000000000111000000000000000000000000000000000000000100100000001

  end of section 1 +

octet number 13 | 14 | 15 | 16 | 17 | 18 | 1 | 2 | 3 | 4 |

binary string 00000001000001000001110100001100000000000000000000000000000000000000111000000000

  end of section 3 +

octet number 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 |

binary string 00000000000000011000000000000001000000010000000100000010000011000000010000000000

end of section 4 +

octet number 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |

binary string 00000000000000000000100000000000100100001111010111011100010000000011011100110111

0011011100110111

Page 18: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Section 5 – End section

• The character string “7777” indicating the end of the message

• Checking for this indicator can be useful to detect some types of data corruption (especially missing bytes in the rest of the message) since the total length of the message is known from Section 0

This section contains:

Page 19: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Now, let’s go back and look at that BUFR message one last time!

‘B’ ‘U’ ‘F’ ‘R’ end of section 0 +

octet number 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 1 | 2 |

binary string 01000010010101010100011001010010000000000000000000110100000000110000000000000000

octet number 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |

binary string 00010010000000000000000000111000000000000000000000000000000000000000100100000001

  end of section 1 +

octet number 13 | 14 | 15 | 16 | 17 | 18 | 1 | 2 | 3 | 4 |

binary string 00000001000001000001110100001100000000000000000000000000000000000000111000000000

  end of section 3 +

octet number 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 |

binary string 00000000000000011000000000000001000000010000000100000010000011000000010000000000

end of section 4 + ‘7’ ‘7’

octet number 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 1 | 2 |

binary string 00000000000000000000100000000000100100001111010111011100010000000011011100110111

‘7’ ‘7’ + end of section 5

octet number 3 | 4 |

binary string 0011011100110111

Page 20: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

BUFR Descriptors

• Section 3 contains a list of BUFR descriptors• These describe the data elements that are

contained in Section 4• Most descriptors are references to BUFR Tables

B, C and D• Using the list of descriptors in Section 3, together

with the tables, it is possible to unpack the data in Section 4

Page 21: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Types of BUFR descriptors

• Element descriptors (Table B)• Replication descriptors• Operator descriptors (Table C)• Sequence descriptors (Table D)• Specified by 3 numbers in 16 bits (2 octets)

– F: 2 bits 0-3

– X: 6 bits 0-63

– Y: 8 bits 0-255

Page 22: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Element descriptors

• Defined by entries in Table B• F is 0• Each element descriptor describes an encoded

value, such as:– The value of a meteorological parameter (e.g. mean sea

level pressure, temperature, wind speed)– Instrument details– Location or date and time information– Quality control information

Page 23: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Replication Descriptors

• Describe the repetition of one or more element, operator, sequence or other replication descriptors

• Used for repetitive data such as the individual levels in vertical soundings or temperature profiles

• Can be:– Fixed - the number of repetitions is pre-determined and

the same for all data subsets– Variable - the number of repetitions can differ from one

subset to the next (ie. delayed replication)

Page 24: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Replication descriptors - continued• Replication descriptors are defined by three numbers

F X Y• F is 1• X is an integer between 1 and 63

– Defines the number of descriptors to be repeated

• Y is an integer between 0 and 255– Defines how many times the X descriptors are to be

repeated– A count of zero indicates delayed replication, where the

repeat count is stored in the data section and can change from one data subset to another.

Page 25: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Operator descriptors• Defined by entries in Table C• F is 2• Describe changes to be made to other descriptors• Operators exist for applications such as:

– Changing the scaling and packing of data– Adding quality control or other associated fields– Describing the descriptors to which quality control

information applies– Substituting a better value for an element, while

retaining the original value

Page 26: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Sequence descriptors

• Defined by entries in Table D• F is 3• Shorthand notations for pre-defined lists of other

element, replication, sequence and operator descriptors

• Not really necessary, but useful in reducing the overhead involved in transmitting data in BUFR:– Replace a commonly-used sequence of descriptors with

a single descriptor, and thereby reduce the overall length of Section 3

Page 27: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

BUFR tables

• Table A– Data Categories, used in Section 1

• Table B– Element descriptors, used in Section 3

• Table C– Operator descriptors, used in Section 3

• Table D– Sequence descriptors, used in Section 3

• Code and Flag tables– Numerical values to be encoded where the element values are

qualitative, used in Section 4

There are many different tables involved in BUFR:

Page 28: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table A• Defines the general category of the data contained

in the BUFR message• Encoded in Section 1• Examples of typical entries:

Code figure Meaning 0 Surface data – land 1 Surface data – sea 2 Vertical soundings (non-satellite) 3 Vertical soundings (satellite) … … 6 Radar data … … 10 Radiological data 12 Surface data (satellite) … … 31 Oceanographic data

Page 29: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table B• Describes the individual values that are encoded• Element descriptors are grouped according to

classes (i.e. X value)

Class Number Class Name Class Number Class Name

01 Identification 12 Temperature

02 Instrumentation 13 Hydrological

… … 14 Radiation and radiance

04 Location (time) … …

05 Location (horizontal-1) 19 Synoptic features

06 Location (horizontal-2) 20 Observed phenomena

07 Location (vertical) 21 Radar data

… … … …

11 Wind and turbulence 33 Quality information

Page 30: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Class 01 – Identification(excerpt)

TABLE

REFERENCE

F X Y

TABLE

ELEMENT

NAME

UNIT SCALEREFERENCE

VALUE

DATA

WIDTH

(BITS)

0 01 001 WMO block number Numeric 0 0 7

0 01 002 WMO station number Numeric 0 0 10

0 01 003 WMO region number Code table 0 0 3

0 01 005 Buoy/platform identifier

Numeric 0 0 17

0 01 006 Aircraft flight number

CCITT IA5 0 0 64

0 01 007 Satellite identifier Code table 0 0 10

0 01 011 Ship or mobile land station identifier

CCITT IA5 0 0 72

0 01 015 Station or site name CCITT IA5 0 0 160

0 01 063 ICAO location indicator

CCITT IA5 0 0 64

Page 31: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Class 11 – Wind and turbulence(excerpt)

TABLE

REFERENCE

F X Y

TABLE

ELEMENT

NAME

UNIT SCALEREFERENCE

VALUE

DATA

WIDTH

(BITS)

0 11 001 Wind direction Degree true 0 0 9

0 11 002 Wind speed m s-1 1 0 12

0 11 003 U-component m s-1 1 -4096 13

0 11 004 V-component m s-1 1 -4096 13

0 11 021 Relative vorticity s-1 9 -65536 17

0 11 031 Degree of turbulence Code table 0 0 4

0 11 032 Height of base of turbulence

m -1 -40 16

0 11 033 Height of top of turbulence

m -1 -40 16

0 11 034 Vertical gust velocity m s-1 1 -1024 11

Page 32: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table B• Columns are:

– Table reference– Element name– Unit– Scale– Reference value– Data width (in bits)

• Scale, reference value, and bit width are chosen so that the desired range of possible data values can be stored in BUFR as positive integers– Preserves the machine-independence of BUFR

Page 33: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table B reference• Expressed as 3 small numbers F X Y• Used to refer to this descriptor• F is always 0 for an element descriptor• X is in the range 0 to 63 and refers to a broad

class of elements– Classes 48 to 63 are reserved for local use

• Y is in the range 0 to 255 and refers to the individual descriptor in the class– Within all classes, descriptors 192 to 255 are reserved

for local use

Page 34: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table B element name

• Natural language description of the meaning of the value

• Usually English but could be translated to other languages

• For example:– Brightness temperature– Total precipitation past 3 hours– Wind speed at 10m

Page 35: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table B unit• The units used for the value

– Normally SI units are used– “CCITT IA5” (the international version of ASCII) is used

for character data such as identifiers– “Code Table” is used for qualitative data where only one of

a set of possible values can be applicable in a given data subset

– “Flag Table” is used for qualitative data where more than one of a set of possible values may be applicable in a given data subset

– For qualitative data, the coded values are references to the Code and Flag tables

Page 36: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table B scale

• Scale– Power of 10 by which to multiply the data value before

packing

– Determines the precision with which the data are encoded

– A scale of 2 means 2 decimal places of precision (eg. 273.16)

– A scale of –1 means that the data values are rounded to the nearest multiple of 10

Page 37: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table B reference value• Used to subtract an offset where negative data

have to be encoded

• Table B contains the value (multiplied by the scale) of the offset to be subtracted

• For example, scale=2, reference value -9000 means that -90.00 is to be subtracted before scaling (ie. -9000 after scaling), allowing values as negative as -90.00 to be represented

Page 38: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table B data width

• The number of bits to be used to encode the value• If all bits are set to ones when encoding (ie. a

value of (2n-1) when n is the data width), then this denotes a “missing” value.

• If the scale is s, the reference value is r, and the data width is n, then the representable range of values is:– Minimum (10-s r)– Maximum (10-s (2n-2+r))

and (10-s (2n-1+r)) denotes the “missing” value.

Page 39: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table B examples

Table reference

F X Y

Element name Unit Scale Reference value

Data width

0 11 012 Wind speed at 10m m s-1 1 0 12

0 13 020 Total precipitation past 3 hours kg m-2 1 -1 14

0 20 003 Present weather Code table 0 0 9

0 08 001 Vertical sounding significance Flag table 0 0 7

Page 40: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table B examples - continued• 0 11 012 - Wind speed at 10m• Scale=1, Reference value=0, Data width=12• Precision is one decimal place (ie. 0.1 m s-1)• Minimum representable value is:

(10-1×0) = 0.0 m s-1

• Maximum representable value is:

(10-1×(212-2+0)) = 409.4 m s-1

• “Missing” value is:

(10-1×(212-1+0)) = 409.5 m s-1

Page 41: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table B examples - continued• 0 13 020 - Total precipitation past 3 hours• Scale=1, Reference value=-1, Data width=14• Precision is one decimal place (ie. 0.1 kg m-2)• For this descriptor, -0.1 kg m-2 is a special value for trace,

according to a specific note in Table B• Minimum representable value is:

(10-1×-1) = -0.1 kg m-2 (= trace)• Maximum representable value is:

(10-1×(214-2-1)) = 1638.1 kg m-2

• “Missing” value is:(10-1×(214-1-1)) = 1638.2 kg m-2

Page 42: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table B examples - continued• 0 20 003 - Present weather• Scale=0, Reference value=0, Data width=9• Coded values are integers since Scale=0• Minimum representable value is:

(100×0) = 0• Maximum representable value is:

(100×(29-2+0)) = 510• “Missing” value is:

(100×(29-1+0)) = 511• One must refer to Code Table 0 20 003 in order to

discover the actual meaning of each coded value

Page 43: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

0 20 003 – Present WeatherCode Table (excerpted)

Code figure Meaning

0 Cloud development not observed or not observable

1 Clouds generally dissolving or becoming less developed

10 Mist

11 Patches of shallow fog or ice fog

13 Lightning visible, but no thunder heard

171 Snow, slight

172 Snow, moderate

173 Snow, heavy

511 Missing

Page 44: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Code tables vs. Flag tables(choice of one vs. choice of more than one)

0-01-003 0-08-001

WMO region number Vertical sounding significanceCode figure Meaning Bit number Meaning

0 Antarctica 1 Surface

1 Region I 2 Standard level

2 Region II 3 Tropopause level

3 Region III 4 Maximum wind level

4 Region IV 5 Significant

5 Region V temperature level

6 Region VI 6 Significant wind level

7 Missing value All 7 Missing value

For a Code table, the value stored in Section 4 is the code figure corresponding to the applicable meaning.

For a Flag table of N bits, the value stored in Section 4 is (2(N-bit#) + 2(N-bit#) + …) for the bit(s) corresponding to each applicable meaning. An extra bit is always present in order to be able to distinguish “all meanings applicable” from “missing”.

Page 45: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Some other important regulations pertaining to Table B

• Elements in classes 01 – 09 are “coordinate” descriptors which

remain in effect until redefined or until the end of the subsetException: when two identical descriptors from classes 04 – 07 are listed consecutively, they define the boundaries of a range

• Similar descriptors exist in “coordinate” vs. “non-coordinate” classesExample: 0 07 004 and 0 10 004 are both “Pressure” with identical scale, reference value and bit width; however, the former is a “coordinate” for use when pressure is the main defining coordinate measured in the vertical direction (e.g. in radiosondes) vs. the latter which is a “non-coordinate” for use when pressure is a derived value (e.g. an aircraft calculating pressure as a function of an observed or measured height)

• Class 08 contains significance qualifiers which can be used to report

qualitative information and which can be explicitly “cancelled”Example: 0 08 011 with value 12 can indicate that we are talking about a “cloud”

Page 46: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table C• Describes the various operators• Columns are:

– Table reference F X• F is 2• X is an integer between 0 and 63• There is no sub-range of X values reserved for local use

– Operand• A number between 0 and 255

– Operator name• A short name describing the operation

– Operator definition• A detailed description of the operation and its effects

Page 47: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table CTABLE

REFERENCE

F X

OPERAND OPERATOR NAME OPERATOR DEFINITION

2 01 Y Change data width Add (Y-128) bits to the data width given for each data element in Table B, other than CCITT IA5, code or flag tables

2 02 Y Change scale Add (Y-128) to scale in Table B for elements which are not CCITT IA5, code or flag tables

2 03 Y Change reference value Subsequent element descriptors define new reference values for corresponding Table B entries. Each new reference value is represented by Y bits in Section 4…

2 04 Y Add associated field Precede each element with Y bits of information (e.g. quality marker).

2 05 Y Signify character Y characters from CCITT IA5 are inserted as a field of (Y*8) bits in length.

2 06 Y Signify data width for following local descriptor

Y bits of data are described by the immediately following local descriptor from Table B

• This is just an excerpt – there are many other (even more complicated!) operators in Table C.• There are also many important notes to Table C describing, e.g. how to cancel an operator.

Page 48: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table C example• Table reference F=2 X=01• Operand, in this case represented as Y• Operator name “Change data width”• Operator description:

“Add Y-128 bits to the data width given for each data element in Table B, other than CCITT IA5 (character) data, code or flag tables”

• According to a note under Table C, this operator is cancelled (ie. effect is turned off) by repeating the operator with Y=0, or at the end of each data subset

Page 49: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table C example - continuedThe “Change data width” operator causes the data width to be changed for subsequent elements, in effect giving them a larger (or smaller) range than is otherwise prescribed within Table B. Thus, it can be used to:

• encode values that exceed the usual representable range for a descriptor, instead of having to introduce a new Table B descriptor (note: in such cases, Y > 128)• reduce the size of the data (and thus the overall encoded message as well!) if the required data range can be encoded using a smaller data width than provided within Table B (note: in such cases, Y < 128)

Page 50: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table C example -continued

As an example, one of the standard descriptors for the height coordinate of an observation is 0 07 007 with unit=m, scale=0, reference=-1000, data width=17, giving a representable range of –1000 m to 130070 m. If one needed to encode a value larger than this, then the 2 01 operator could be used to increase the data width. For example, use of the operator 2 01 130 before the 0 07 007 descriptor would increase its data width to 19 bits and therefore allow values up to 523286 m.

Page 51: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table D• Describes sequences of descriptors• Columns are:

– Table reference F X Y• F is 3• X is in the range of 0 to 63 and refers to a broad category of sequences

– Categories 48 to 63 are reserved for local use

• Y is in the range of 0 to 255 and refers to the individual sequence in the category

– Within all categories, entries 192 to 255 are reserved for local use

– Table references• List of other descriptors, including element descriptors (Table B),

replication descriptors, operator descriptors (Table C) and other sequence descriptors (Table D)

– Element name• Not really necessary, but makes reading the tables easier

Page 52: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table D categories

• Categories correspond to the X value of the underlying sequence descriptor.

Category number Category name

01 Location and identification sequences

02 Sequences common to surface data

03 Sequences common to vertical soundings data

04 Sequences common to satellite observations

05 Sequences common to hydrological observations

06 Sequences common to oceanographic observations

07 Surface report sequences (land)

08 Surface report sequences (sea)

… …

18 Radiological report sequences

21 Radar report sequences

Page 53: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Category 01 – Location and Identification Sequences (excerpt)TABLE REFERENCE TABLE REFERENCES ELEMENT NAME

3 01 001 0 01 001

0 01 002

WMO block number

WMO station number

3 01 011 0 04 001

0 04 002

0 04 003

Year

Month

Day

3 01 012 0 04 004

0 04 005

Hour

Minute

3 01 024 0 05 002

0 06 002

0 07 001

Latitude (coarse accuracy)

Longitude (coarse accuracy)

Height of station

3 01 038 3 01 001

0 02 011

0 02 012

3 01 011

3 01 012

3 01 024

(Land station for vertical soundings)

WMO block and station number

Radiosonde type

Radiosonde computational method

Date

Time

Lat/long (coarse accuracy), height of station

Page 54: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table D example

Table reference F X Y

Table References

Element name

(Buoy/platform – fixed)

3 01 033 0 01 005 Buoy/platform identifier

0 02 001 Type of station

3 01 011 Date

3 01 012 Time

3 01 021 Latitude and longitude (high accuracy)

Page 55: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Table D example - continued• In this example, the sequence consists of five

descriptors• Two of these are Table B element descriptors• The other three are Table D sequence descriptors,

which in turn represent other sequences of descriptors

• Obviously the repeated replacement of Table D descriptors with lists of descriptors must not continue forever, and a Table D descriptor cannot include itself, directly or indirectly

Page 56: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Replication descriptors - example• The 4 descriptors:

– 1 03 002 Repeat three descriptors twice– 0 22 003 Direction of swell waves– 0 22 013 Period of swell waves– 0 22 023 Height of swell waves

• Are equivalent to the 6 descriptors:– 0 22 003 Direction of swell waves– 0 22 013 Period of swell waves– 0 22 023 Height of swell waves– 0 22 003 Direction of swell waves– 0 22 013 Period of swell waves– 0 22 023 Height of swell waves

Page 57: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Delayed replication descriptor - example

• The 3 descriptors:– 1 01 000 Delayed replication of one descriptor– 0 31 001 Replication factor (8 bit)– 3 03 012 Winds at pressure levels

• Are equivalent to:– 3 03 012 Winds at pressure levels– 3 03 012 Winds at pressure levels– … for as many times as specified by the 0 31 001 replication factor

• The number of repeats for each data subset is stored in the data. Each data subset has its own repetition count.

• In addition to 8-bit delayed replication, there is also 1-bit and 16-bit delayed replication

Page 58: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

A caveat…The previous example is worth a closer look. Specifically, when replicating a sequence, the replication takes place before the sequence expansion!

For example, note that:

1 01 004

3 03 012 ( = 0 07 004, 0 08 001, 0 11 001, 0 11 002 )

is equivalent to: and not to:

3 03 012 0 07 004

3 03 012 0 07 004

3 03 012 0 07 004

3 03 012 0 07 004

0 08 001

0 11 001

0 11 002

Page 59: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Section 4 – Detailed description• The form of the data in Section 4 varies

depending upon the descriptors and flags in Section 3– One of the main differences concerns whether

the data are uncompressed or compressed– Delayed replication and compression may be

combined, but only if the number of repetitions is the same for each subset within a particular BUFR message

Page 60: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Section 4 – Uncompressed data• Each data item takes up the number of bits

specified by the data width in Table B, as adjusted by operators or as otherwise specified.

• The order for N data subsets, each with M data values is:

Set 1 Value 1, Set 1 Value 2, Set 1 Value 3, … Set 1 Value M

Set 2 Value 1, Set 2 Value 2, Set 2 Value 3, … Set 2 Value M

Set N Value 1, Set N Value 2, Set N Value 3, … Set N Value M

Page 61: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Section 4 – Compressed data• Each data item is encoded by its minimum over all

the data subsets (using the data width specified in Table B or otherwise), the number of bits used to encode the increments (using 6 bits), and a list of increments to be added to the minimum.

• The order for N data subsets, each with M data values is:

Min 1, Nbits 1, Set 1 Inc 1, Set 2 Inc 1, … Set N Inc 1Min 2, Nbits 2, Set 1 Inc 2, Set 2 Inc 2, … Set N Inc 2….Min M, Nbits M, Set 1 Inc M, Set 2 Inc M, … Set N Inc M

Page 62: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Summary• BUFR is flexible

– It can represent a wide range of data types

• BUFR is table-driven– Tables A, B, C and D (and code and flag tables) contain

majority of information needed to encode and decode data

• BUFR is self-describing– The data description section describes the data, using

element (Table B), replication, operator (Table C) and sequence (Table D) descriptors

• Replication descriptors and sequence descriptors can be used to reduce the size of the data description

• Compression can be used to reduce the size of the data themselves

Page 63: Introduction to BUFR

WORLD METEOROLOGICAL ORGANIZATION

RA-VI Regional Training on BUFR and Migration to Table Driven Code FormsLangen, Germany, 17 - 20 April, 2007

Acknowledgements

Based on:• WMO No. 306 Manual on Codes, Volume I.2

(http://www.wmo.int/web/www/WMOCodes/ManualCodes/WMO306vol-I-2PartB.pdf)

• Guide to WMO Table-Driven Code Forms FM94 BUFR and FM95 CREX (http://www.wmo.int/web/www/WMOCodes/Guides/BUFRCREXPreface_en.html)

Special thanks to:• Jeff Ator NOAA, United States of America• Charles Sanders BOM, Australia (retd.)• Joël Martellet WMO


Recommended