NCAR/TN-408NCAR TECHNICAL NOTE
November 1994
ELDORA Data User's Guidefor TOGA COARE
Peter H. HildebrandWen-Chau LeeRobert RillingNCAR/ATD/Remote Sensing Facility
Richard OyeNCAR/ATD/Research Data Program
ATMOSPHERIC TECHNOLOGY DIVISION
NATIONAL CENTER FOR ATMOSPHERIC RESEARCHBOULDER, COLORADO
II
__
ELDORA Data User's Guide for TOGA COARE
Peter H. Hildebrand Wen-Chau Lee Robert RillingNCAR/ATD/Remote Sensing Facility
Richard OyeNCAR/ATD/Research Data Program
23 November, 1994
Table of Contents
1. Introd uction ..... ....................................................... ......................................... 1
2. ELDORA Operations in TOGA COARE ........................................................... 1
3. D ata P rocessing ................................................................................................ 3
3.1 Error Sources and Corrections .................................................................... 5
3.1.1 Corrections Applied to Data ............................................................... 6
3.1.2 Half-Nyquist Folding Correction............................................................7
3.1.3 Aircraft Ground Speed Removal and Unfolding Doppler Velocities......7
3.1.4 UNIX File Size Problem for Copies of ELDORA Data Made in Honiara........................................................................................................................ 9
3.1.5 Bias and Uncertainty in INS and Antenna Positioning .......................... 9
3.2 Converting Data to DORADE and UF ....................................................... 10
3.3 Data Compression........................................... 12
4. Analysis Software......................................................................................... 12
4.1 Scan-plane Data Visualization and Editing .................... ....................... 12
4.2 Interpolation, Multiple-Doppler Analysis, and Display................................13
5. Obtaining Data, Software, and Answers ......................................................... 14
Appendix A: DORADE Format Overview ................. .......................... 16
Appendix B: Radar Coordinate Transformation .................................................. 20
i
List of Tables
Table 1. ELDORA Radar Characteristics as Operated in TOGA COARE...........2
Table 2. Summary of ELDORA Operations in TOGA COARE .............................. 4
Table 3. Definitions of Five Classes of TOGA COARE Convections .................... 5
Table 4. ELDORA Parameters Provided for TOGA COARE ......... 6.......... ... 6
Table 5. ELDORA Correction Factor Evaluation. The Correction Factors AreEvaluated for Two or More Groups of Scans on Five Flights in February1993. Each Group of Scans Contains 25 or More Individual Scans............ 1 1
Table 6. List of Persons to Contact Regarding ELDORA Data ........................... 15
List of Figures
Figure 1. Schematic Diagram Showing How the Half-Nyquist Folding ProblemOccurred During TOGA COARE due to Improper Averaging of Velocitiesfrom Two Frequencies as Described in the Text .......................................... 8
ii
1. Introduction
The purpose of this technical note is to provide a concise User's Guide for theNational Center for Atmospheric Research (NCAR) ELectra DOppler RAdar (ELDORA)data collected during the Tropical Ocean and Global Atmosphere Program CoupledOcean-Atmosphere Response Experiment (TOGA COARE). This guide includes 1) adescription of the radar system configuration, 2) characteristics of the sampling strategy,and 3) the ELDORA operations in TOGA COARE. The major portion of this User'sGuide discusses the corrections applied to the data, the ELDORA data formats available,the data processing steps used, analysis routines available, and the means of obtainingdata. Instructions for obtaining additional information are provided, including how toobtain assistance and how to access on-line documents concerning updates to data and/orcalibrations. The appendices present a brief description of the data format and a summaryof the coordinate transformation matrix.
The ELDORA radar as configured for TOGA COARE and mounted on NCAR'sElectra aircraft can sample storm hydrometeor motions and reflectivities over a domainextending 30-60 km from the flight track. The ELDORA radar (Hildebrand et al. 1994) isan X-band Doppler radar with 35 kW peak power, a pulsed dual-frequency waveform, abeamwidth of 1.8°, = 40 dB antenna gain, and a horizontal polarization when the antennasare pointed horizontally (see specifications in Table 1). The radar employs dual flat-plateantennas with fore- and aft-pointing radar beams. These beams are scanned in coneswhich are tilted by ±+18.50 fore and aft of a plane which is perpendicular to the aircraftheading. This technique was developed to enable the radar to collect dual-Dopplerinformation as the aircraft passes by or through atmospheric storms. The radarobservations are then used to reconstruct storm structure and kinematics.
Data from the ELDORA operations in TOGA COARE have been carefullyreviewed and are now available for research use. Although some signal processorproblems limited operations to two transmit frequencies during TOGA COARE, the datastill meet the ELDORA design goals for the absence of statistical sampling noise. The datagenerally have smooth, noise-free reflectivity and velocity fields which are much more likeground based radar data than previously obtained airborne radar data. Achieving this levelof data quality was a major design goal for the ELDORA development and the primarymotivation for using a multiple frequency transmitted waveform.
2. ELDORA Operations in TOGA COARE
During January and February 1993 the ELDORA airborne Doppler radar wasoperated on the NCAR Electra aircraft in support of TOGA COARE (Table 2). The radarwas installed on the aircraft in December 1992 during the break between the second andthird Intensive Operational Periods (IOPs). Thereafter, ELDORA operated on 13 out of
1
Characteristic TOGA COARE Design GoalWavelength (cm) 3.2 3.2Transmit Frequency (GHz) 9.35, 9.45 9.2-9.8Beamwidth (deg, circular) 1.8 1.8Antenna Gain (dB: fore/aft antennas) 38.75/38.70 40Polarization (antenna horizontal horizontal horizontalSidelobes (d) -23 to -30 -40Beam tilt angle (fore or aft, deg) +18.5 +15-19Antenna spin axis parallel to: heading headingAntenna rotation rate (deg/s) 66 0-144Dwell time (ms) 15 7-50Rotational sampling interval (deg) 1 1Peak Transmit Power (kw) 35-50 35-50Minimum Detectable Signal at 10 km (dBZ) -12 -12Receiver Bandwidth (MHz) 2 2Receiver Temperature at Antenna (°K) <600 <600Pulse Repetition Frequency (pps) 2000 2000-5000Unambiguous Range (km) 55 20-90Unambiguous Velocity (m/s) +16 +13-20Number of Radars, 1 (switched) 2 (fore/aft)Number of Xmit Frequencies per Radar 2 3 3.Pulse Chip Length (pis) 1 0.1-3.0Range Averaging (chips) 1 -4Total Gate Length (km) 0.15 0.035-1.2Elevation Step (deg) 1 1Along-Track Beam Spacing (km) 0.65-1.3 0.3-1.0
Table 1. ELDORA Radar Characteristics as Operated in TOGA COARE.
the 16 NCAR Electra flights during IOP's #3 and #4. The first 7 flights served asELDORA check-out missions during TOGA COARE boundary layer studies. Theremaining 6 flights provided ELDORA observations of TOGA COARE Class 1-3convection. The TOGA COARE convection was categorized according to the areacoverage of the 208K IR cloud top temperature (see Table 3). Four of these 6 flightsincluded coordinated data collection with the National Oceanic and AtmosphericAdministration (NOAA) P-3s and other TOGA COARE aircraft.
The start of ELDORA support of TOGA COARE was delayed due to severereliability problems with the transmitter high power amplifiers (HPAs). Partial repairs ofthe HPA units were completed in time for the start of IOP #3 and produced the neededhigh quality radar transmissions, but only with the reliability to operate for about 20 hoursbetween major HPA failures. Since just two HPA units were available and the TOGA
2
COARE research was expected to extend considerably longer than 20 hours, the radarwas operated with only a single transmitter and a waveguide switch. In this configuration,the single transmitter switched between the fore and aft antennas on a scan-by-scan basis.(The dual-transmitter ELDORA design is intended to provide simultaneous 360°observations from fore and aft antennas.) In the early flights (missions 23-31) theswitching was primarily performed on a complete 360° scan basis. In the latter flights(missions 33-38) the radar was principally switched every 180° to provide observations on
one side of the aircraft track only but with higher resolution. In TOGA COARE,ELDORA operated with 650 m along-track resolution for the 180° scans and 1300 malong-track resolution for the full 360° scans.
The best ELDORA radar data were collected on TOGA COARE missions 33-38,the last six Electra flights. These missions included Class 1-3 convection, and includedflights with the NOAA P-3s and other TOGA COARE aircraft. The earlier Electra flights(missions 23-31) occurred during boundary layer missions with Class 0-1 convection andprovided the opportunity for ELDORA's initial test flights, producing only a smallquantity of ELDORA data. During this period three problems in the data system and theoperational procedures were solved. First, a calculation problem which producedincorrect velocity values at Nyquist folds was discovered between the first and secondELDORA flights (flights 23 and 24). (See Section 3.1 on page 5 for a completedescription of the problem and its solution.) Second, a disabling intermittence developedin the high bandwidth data cable between the digital signal processor and the rest of thedata system (displays, recording, etc.). This problem was finally located and repaired aftermission 31. The data system then ran reliably for the rest of TOGA COARE. Finally,there were the normal variety of operational quirks which any new system has on its first
outing. These problems were principally corrected during missions 23-26.
3. Data Processing
Each beam of ELDORA data has the following attributes: aircraft location(latitude, longitude, altitude), aircraft velocity (with respect to both the air and theground), time, and azimuth and elevation angles (see Lee et al. 1994a for definitions and
descriptions of the airborne Doppler radar geometry and terminology).. Similar to ground-based systems, azimuth is the horizontal pointing angle of the radar beam measuredclockwise looking down from north, and elevation is the vertical angle above thehorizontal plane on which azimuth is measured. The rotation angle of the antenna about
the spin axis (positive in a clockwise sense looking from the tail to the nose of the aircraft)and the tilt angle of the scan cones fore and aft of a plane normal to the aircraft headingare recorded, as is the aircraft attitude information needed for calculating all angles. In thedelivered data set, the small errors inherent in these measurements have been corrected byusing radar measurements of ground reflectivity and velocity (see Section 3.1.5).
3
TC Date Mission Electra ELDORA Other AcftOps
Flt# GMT Class' Flt# Times2 Date h Times h N42RF N43RF DC-8 ER-2 C130 C340
1-20 Nov-Dec1992
21 930109 1 Conv 17 2053-0327 8 Jan 2054-0555 2056-061122 930111 0/1 BL 2105-0342 x x
23 930112 1BL/Conv 18 2051-0321 12/13 Jan 6.6 2219-0013 1.924 930113 0 BL 19 2052-0338 13/14 Jan 6.6 2207-2235 0.5 x
25 930114 0 BL 20 2052-0332 14/15 Jan 6.6 2308-0309 1.6 x
26 930116 I1BL 21 2100-0322 16/17 Jan 6.2 2256-2358 1.0 2106-0442 2105-0031 x x
27 930117 1 BL 22 2242-0521 17/18 Jan 6.5 2253-0743 2242-0754 x x x
28 930118 1BL 23 2237-0517 18/19 Jan 6.5 2318-0245 1.7 2244-0735 2242-0745 x x x x
29 930126 OBL 24 2052-0332 26/27 Jan 6.5 2248-2316 0.5 x
30 930127 OBL 25 2058-0358 27/28 Jan 6.9 2230-0302 1.1 x
31 930128 OBL 26 2056-0326 28/29 Jan 6.5 0.0 x
32 930201 OBL IFeb 1918-0408 1916-0416
33 930204 3Conv 27 2112-0358 4/5 Feb 6.8 2149-0030 2.7
34 930206 2Conv 28 1505-2124 6Feb 6.3 1545-1935 3.8 1302-2204 1418-2306 x
35 930209 3 Conv 29 1501-2219 9 Feb 7.3 1605-1949 3.7 1318-2204 1420-2318
36 930210 2/3 Conv 30 1805-0059 10/11 Feb 6.9 1846-0026 5.7 1805-0108 1802-0102 x
37 930217 lConv 31 1857-0121 17/18Feb 6.3 2008-2328 3.3 1809-0307 x
38 930218 2Conv 32 2047-0307 18/19 Feb 6.2 2128-0204 4.6
39 930219 O BL 19 Feb 1905-035140 930220 3Conv 20 Feb 1834-0352 1931-0412 x x
41 930222 4 Conv 22 Feb 1800-0110 1917-0243 x x
1 From TOGA COARE Turboprop Mission Summary At-A-Glance (Marks and Smull 1993, personal communication)2 From NCAR/RAF TOGA COARE Project Documentation Summary
Table 2. Summary of ELDORA Operations in TOGA COARE.
4
CLASS AREA OF TEMPERATURE < 208K0 INone-I <6000km2
II 6000 - 20000 km2
III 20000 - 60000 km2
IV >60000 km2
Table 3. Definitions of Five Classes of TOGA COARE Convections.
The ELDORA data are written in ELDORA FIELD format (Walther and Lee 1994),which is an extension of the DOppler RAdar Data Exchange (DORADE) format (see Appendix Aand Lee et al. 1994b). DORADE format (version 1) is the preferred exchange format for TOGACOARE ELDORA data. The DORADE format supports data compression, reducing the dataamount by 60%, compared with the data amount on the Universal Format (UF) (Barnes 1980).Note that there is some loss of information (particularly concerning correction terms) in theconversion to UF (see section 4.1). It is, therefore, hoped that DORADE is the exchange formatof choice for the future. Users are advised to start considering ways to modify their software tomake use of DORADE. Since new software to handle DORADE format is still underdevelopment, TOGA COARE ELDORA data are available in both DORADE and UF. DORADEand UF format data have been staged to the NCAR Mass Store System (MSS), and are availablefor remote accessing of the data. These data are also available directly from the AtmosphericTechnology Division (ATD). Users will still be able to use the existing post-analysis software (i.e.Perusal, Editor, REORDER, and CEDRIC) to analyze ELDORA data. To date, both REORDERand SPRINT software have been enhanced to ingest DORADE data. A UNIX X-Window basedradar perusal and editor package (SOLO) is currently being developed at NCAR. This softwarewill make extensive use of DORADE. Fortran programs with C I/O modules are available fromthe Remote Sensing Facility (RSF) at NCAR/ATD which can demonstrate methods of access toDORADE format data. These programs can be requested, used and modified freely by the users,but are NOT intended to be an ATD-supported software.
The balance of this section presents the data processing steps which were used to convertdata from ELDORA FIELD format into DORADE and UF. In addition, the procedures to obtainELDORA data from the NCAR MSS or from NCAR/ATD are discussed.
3.1 Error Sources and Corrections
The detailed descriptions of the known error sources will be discussed in this subsection.The solutions and the corresponding corrections are also provided.
5
DORADE NAME UF NAME Parameter DescriptionDBZ DB Reflectivity factorNCP NC Normalized Coherent PowerSW SW Spectral WidthVR VR Raw Velocity as recorded by the ELDORA data systemVN VN Raw Velocity with a correction for the error at velocity
folds introduced by averaging multi-frequency velocitiesVG VG Velocity corrected for aircraft ground speed and other
aircraft motionsVU VU Unfolded Velocity, derived during objective batch
processing from VG
Table 4. ELDORA Parameters Provided for TOGA COARE.
3.1.1 Corrections Applied to Data
The following errors in ELDORA FIELD tape housekeeping have been identified. Thesenew housekeeping values have been used in producing the final data set (DORADE or UF), andshould be transparent to the users. However, any users who access data from ELDORA fieldtapes (or who obtained UF data from RSF in Honiara) should be aware of these problems.
1. The transmitted frequencies in the field tape header were incorrect. The actual transmittedfrequencies were 9.35 and 9.45 GHz. The average frequency was 9.40 GHz. The effect isthat the Nyquist velocity in the header and the Doppler velocity values on the tape wereoff approximately 1%. The corrected scale factor for the velocity is 7.80.
2. The scale factors for normalized coherent power (NCP) and spectral width (SW) in theheader were incorrect in the field tape. The corrected values are 100.0 for NCP and 7.95for SW. NOTE that the UF and DORADE data contain incorrect SW scale factor (4.42instead of 7.95). The SW value is about 1.8 times larger than its actual value.
3. The reflectivities for the fore and aft radar beams were computed using only one radarequation and radar constant during TOGA COARE. The reflectivity fields have beenrecalibrated to account for different antenna gains and different transmitted power fromthe two HPAs used. The corrected antenna gains are 38.75 dB for the fore antenna and38.70 dB for the aft antenna.
4. The Julian day recorded in the field tape ray header was sometimes incorrect. The date inthe volume header was correct; however, month = 0 and month = 1 both indicatedJanuary.
6
5. The horizontal winds (u and v) in the platform info block were incorrect and have beencorrected.
6. The antenna H-plane angle used in the calculations of rotation angle was incorrect. Thisangle affects rotation angle, azimuth, and elevation. The rotation angle for the aft antennawas corrected by -0.88°. The correct tilt angle for the fore and aft antennas was + and -18.5°, respectively.
7. The roll angle recorded in the platform info block was actually the roll angle rate. Thesevalues are generally within 1° of the Research Aviation Facility (RAF) Aircraft DataSystem (ADS) data; however, the difference can become large when the aircraft turns.The correct roll angles from the ADS data tape have been merged into the final ELDORAdata.
3.1.2 Half-Nyquist Folding Correction
During TOGA COARE ELDORA operated at the two transmit frequencies of 9.35 and9.45 GHz, for which the Nyquist velocities are at 16.04 and 15.87 m/s, respectively. DuringTOGA COARE the Doppler velocity was calculated by averaging pulse-pair mean velocityestimates for each separate frequency, rather than by averaging the real and imaginarycomponents of the vector and then taking the arctangent to obtain the Doppler velocity estimate.Due to the different Nyquist velocities of 15.87 and 16.04 m/s, erroneous near-zero velocitiesoccurred after the Doppler velocity folds at 9.35 GHz Nyquist velocity, but not folds at 9.45 GHzNyquist velocity. A schematic diagram showing how the problem occurred near the Nyquistvelocity is illustrated above in Figure 1. These near-zero velocities always occurred betweenpositive and negative Nyquist velocity.
Using the above characteristic, the data have been corrected by identifying the problematicvalues and then adding or subtracting half a Nyquist interval (depending on the sign of the originalDoppler velocities, VR). A new velocity field, VN, has been created after unfolding these half-Nyquist-folded velocities. In creating VN, VR was thresholded by normalized coherent power of0.33 to remove background noise -- a critical step for the success of this scheme. Very fewoutliers escape this scheme undetected. These few erroneous values remain in the data and mustbe deleted interactively using a data editor.
3.1.3 Aircraft Ground Speed Removal and Unfolding Doppler Velocities
The ELDORA antennas scan about an axis parallel to the aircraft heading, at tilt angles+18.5° fore and aft of the plane perpendicular to the fuselage. As a result, the observed Dopplervelocities contain the velocity components not only from hydrometeors but also from theaircraft's flight speed. The equation to compute the component of the aircraft's flight speed in the
7
Averaging Problems Near Nyquist Velocities
20
15
10
5
0
-5
-10
-15
-20
Gate Number
Figure 1. Schematic Diagram Showing How the Half-Nyquist Folding Problem OccurredDuring TOGA COARE due to Improper Averaging of Velocities from Two Frequenciesas Described in the Text.
beam-viewing direction has been documented in Lee et al. (1994a). At a ±18.5° tilt angle anda120 m/s aircraft ground speed, the component of the aircraft ground speed along the radar beamis about ±38 m/s. Multiple folding in Doppler velocities thus occurs, considering that the Nyquistvelocity for the ELDORA system was =16 m/s during TOGA COARE. A new velocity field, VGhas been created by subtracting the aircraft-ground speed component from VN.
Finally, the VG velocity values were unfolded using the Bargen and Brown (1980) "B"-algorithm. This process unfolds each gate separately, using the along-beam component of the in-situ wind as a reference to unfold the first gate. The following gates are then unfolded, using theunfolded Doppler velocity from a running average of (up to) the 7 previous gates as a template.The resultant unfolded Doppler velocity has a field name VU. Normally, the unfolding andaircraft ground speed removal are both accomplished in the same step. All velocity fields (VR,VN, VG, and VU) are provided to the users of ELDORA data (Table 4).
8
o0E5
>
0Q0I
3.1.4 UNIX File Size Problem for Copies of ELDORA Data Made in Honiara
The tape copying routine that the RSF distributed in Honiara had a default file size limit of2 Gbytes. This problem caused the tape copying to stop at the 2 Gbyte mark and exitprematurely. All ELDORA field format data past this point which were copied in Honiara weretherefore not copied, and are missing from "copy" tapes. Due to this and the above-mentionedcorrections made to the ELDORA FIELD tape data, RSF strongly recommends that usersobtain a DORADE or UF tape which contains the most recent updates.
3.1.5 Bias and Uncertainty in INS and Antenna Positioning
The accuracy of the unfolded Doppler velocities, and their mapping into an earth-relativecoordinate system, is critically influenced by biases in the Inertial Navigation System (INS) and byother antenna positioning errors such as mounting errors, aircraft altitude error, and radar rangedelay error. The major INS error is the 90 min INS Schuler oscillation (Rodi et al. 1991) whichproduces an error in the aircraft location (longitude and latitude) as well as ground speed. Thisbias can be corrected by using the reliable but relatively infrequent GPS data as a template. Thelongitude and latitude information in both the DORADE and UF format have been adjustedaccordingly.
The stability of the raw recorded aircraft attitudes are within 'the original designexpectations. The drift angle has the largest uncertainty among all INS variables. If the INSvariables and the antenna positions were measured perfectly, and the aircraft ground speed andunfolding are performed correctly, the ocean surface should appear to be not moving (it isassumed that the ocean current is negligible in normal conditions). The intersection of the oceansurface and a conical helix of radar beams is nearly a hyperbola. The ocean surface would appearlevel if the hyperbola is projected to a plane normal to the fuselage properly. Given the expecteduncertainties in the raw aircraft attitude and positioning data it is common to find that even afterperforming all the data processing steps, the sea surface is not quite flat and level and the velocityof the sea surface is not quite zero.
Using the range and residual velocity of the ocean surface as constraints, a variationaltechnique has been derived (Testud et al. 1994) to correct systematic biases in drift, pitch,rotation angle, aircraft ground speed, aircraft pressure altitude, range delay, and tilt angle. Errorsin these values can be deduced by evaluating: 1) the predicted versus measured range error of thesurface range gate for each beam in the sweep, and 2) the bias of the surface velocities away fromzero. The biases produced by different errors result in different signatures in the velocity and therange residuals within a sweep. The analytical technique of Testud et al. (1994) can therebyseparate the biases contributed by the different parameters. An iterative process combines theunfolding, aircraft ground speed removal, and the INS bias removal to calculate a set ofcorrection factors which -- in least squares sense-- minimizes the velocity and range residualsalong a sweep.
9
The correction factors were evaluated using from 14 periods encompassing the ELDORAflights during TOGA COARE IOP #4. Error Sources and Corrections summarizes the findings,giving the average correction factor for a number of important variables: range to the first gate(fore and aft radar), pitch, drift, rotation angle, aircraft altitude and aircraft horizontal velocity.
The correction factors in the range to the center of the first gate (range delay, RO) for thefore and aft radars were nearly constant for all time periods reviewed. The variability in the rangedelay error is considerably less than a gate length, which is the upper limit of the uncertainty inrange residual. These values agree with values estimated from measurements in the field. Thepitch and rotation angle have small, stable biases which are within expectations. The mostunstable parameter is the drift bias, which, due to the measurement error in the heading, can varywithin about ±+1. The measured drift bias variability of ±-0.7° is consistent with this expectedmeasurement error. This drift bias variability should be expected to introduce variable errors inthe aircraft ground speed of =1 m/s. The flight-to-flight deviations among all other parametersare negligible except for the aircraft pressure altitude, where day-to-day variations are expecteddue to the changes in surface pressure.
Our analysis indicates that a single set of corrections for the entire TOGA COAREELDORA data set should account for most errors in navigation of the data. These correctionshave been derived and applied to the aircraft's pitch, roll, and radar range delay. Small additionalcorrections should be added by users in analysis to correct for drift and subsequent aircraft groundvelocity error. Users desiring to correct for these residual errors should expect to fine-tune theirdata set with help from the FORTRAN code provided by RSF for analyzing the above biases.
The final data are generated using the average correction factors obtained in the iterativeprocess. For TOGA COARE data the correction factors are:
* 2.3° for rotation angle* -1.45° for pitch* -10 m for the aircraft altitude* 10 m for the range delay
3.2 Converting Data to DORADE and UF
The conversion of data from ELDORA FIELD Format to DORADE or UF involvescorrections to both the tape-header and the data. In addition to the standard ELDORA variablessuch as reflectivity factor (DBZ or DB), normalized coherent power (NCP or NC), spectral width(SW), and raw velocity (VR), three new velocity fields are presented on the tape. Theseparameters are defined in Table 4. By providing all the fields, users can easily reformulate any ofthe corrections which have been applied to the data, should they wish to make their owncorrections, or if the correction factors are updated by RSF.
10
1. Variable corrections between 19:52-19:56.2. Aircraft turn and variable corrections around 19:43.3. First file of tape: subsequent files have different corrections.4. Aircraft climbing. Heading changes smoothly from 360° - 180°.
Table 5. ELDORA Correction Factor Evaluation. The Correction Factors AreEvaluated for Two or More Groups of Scans on Five Flights in February 1993.Each Group of Scans Contains 17 or More Individual Scans.
All the above corrections have been added to the corresponding data values in thedistributed data set. Due to the structural differences between DORADE and UF, thesecorrection factors are treated differently in the two formats. For DORADE, the correction factorsare recorded in the CORRECTION FACTOR DESCRIPTOR with the expectation that programsutilizing the data will apply the correction factors appropriately in the data analysis process.Therefore, the original INS attitudes are not modified except for the roll angles, which have beencorrected by merging in data from the Electra's ADS data. For the UF data, all corrections havebeen applied because there is no place-holder for these correction factors.
11
Date Time #Scans RO fore RO aft Pitch Drift Rotation Altitude HorizontalAngle Velocity
(GMT) (m) (m) (deg) (deg) (deg) (m) (m/s)
06 Feb 93 18:47-18:57 50 5 20 -1.4 -0.3 2.3 -10.0 -0.506 Feb 93' 19:48-19:58 50 10 25 -1.4 0.6 1.3 -10.0 1.0
09 Feb 93 16:08-16:18 25 10 25 -1.4 -1.1 2.3 -10.0 -0.409 Feb 93 18:34-18:43 25 10 25 -1.4 -2.4 2.3 -10.0 -0.909 Feb 932 19:40-19:50 25 0 15 -1.4 0.0 2.3 -10.0 -1.5
10 Feb 93 18:46-19:01 80 20 35 -1.4 -1.1 2.3 10.0 0.310 Feb 933 20:37-20:52 25 30 45 -1.4 -0.4 1.3 180.0 0.510 Feb 93 20:53-20:58 25 5 20 -1.4 -0.4 2.3 0.0 0.210 Feb 934 23:06-23:12 25 20 35 -1.3 -0.5 2.4 0.0 -0.1
17 Feb 93 20:08-20:14 26 20 35 -1.4 -0.8 2.4 0.0 -0.417 Feb 93 20:30-20:40 29 20 35 -1.4 -0.2 2.4 -10.0 -0.417 Feb 93 22:19-22:29 50 10 25 -1.4 -1.2 2.3 0.0 -0.4
18 Feb 93 21:28-21:40 68 20 35 -1.4 -1.0 2.4 -10.0 0.218 Feb 93 01:00-01:03 17 25 40 -1.4 -1.4 2.3 10.0 -0.4
Mean Values 15 30 -1.4 -0.7 2.2 9.3 -0.2
3.3 Data Compression
The file size for TOGA COARE ELDORA data is too large for efficient data access. Thedata are therefore broken into files of approximately 30 Mbytes. The data have been compressedusing the data compression scheme described below:
1. Data are truncated below z = -5 km and above z = 25 km using the NOAA P-3 datacompression scheme. This will reduce the data volume by about 25%.
2. To ease data handling, an artificial file mark has been inserted at the end of a sweep atabout 5 minute (30 Mbytes) intervals.
4. Analysis Software
This section summarizes analysis software packages which are available from NCAR forprocessing TOGA COARE ELDORA data. Additional analysis packages are available fromCentre de Recherche en Physique de l'Environnement (France), NOAA/EnvironmentalTechnology Laboratory/National Severe Storms Laboratory (Boulder), and NOAA/HurricaneResearch Division (Miami).
All TOGA COARE ELDORA data are available in DORADE and UF on the NCARMSS. Additionally, DORADE and UF data are available from ATD on Exabyte tapes. The datainterpolation programs (REORDER and SPRINT) have the capability to access DORADE as wellas UF data. Multiple Doppler analysis and display programs (CEDRIC) make use of interpolatoroutput and are therefore independent of radar format.
4.1 Scan-plane Data Visualization and Editing
The tasks of data visualization and editing for UF data have been supported by theNCAR/ATD/RDP's PERUSAL and EDITOR. These programs continue to be available andsupported, but only for UF data.
A new visualization and editing software (SOLO) is being developed to replace andimprove upon PERUSAL and EDITOR capabilities. The SOLO program is currently available forP-testing for interested users. A conversion routine to convert DORADE, UF, NOAA P3 dataand the ELDORA FIELD format data into 'sweep files' as the input data to SOLO is available.This conversion routine will also perform aircraft ground speed removal and Bargen and Brown(1980) unfolding. In the interim users have three primary options: 1) use UF data, 2) write aninterface for their own software using Fortran access subroutines which can be obtained fromRSF, or 3) skip the traditional visualization/editing step and move directly into use of an
12
interpolator for display and analysis purposes. All three approaches are viable and are currentlybeing used in RSF.
Option 1, the use of UF data, while somewhat laborious, is certainly tractable and thenecessary software packages are all available. Option 2, writing a new DORADE interface toexisting software using sample programs provided by NCAR, CRPE or elsewhere, presupposesthe user has alternative data visualization, editing, and analysis capabilities. This is probably thepreferable choice for those users having their own personal software, and should bestraightforward. Option 3, eliminating the pre-analysis visualization and editing step, whilepossibly a surprising suggestion, appears to be workable. (This approach has for some time beensuccessful at NCAR/Mesoscale Microscale Meteorology Division (MMM) and ResearchApplication Program for analysis of ground-based radar data.) Test analyses performed in RSFindicate that since much of the ELDORA data are sufficiently free from noise, simple batchediting approaches are capable of producing excellent analyses. The edit steps can occur in theradar scan space or in the process of griding, and then on the grided data. The batch editingincludes elimination of data from the sea surface and below, use of NCP and local variability toeliminate residual noise, and other appropriate techniques. Our experience indicates that simplebatch removal of the residual sea surface motion is also possible in this batch mode.
Users of the NCAR RDSS EDITOR program are warned of a problem which was onlyencountered with analysis of ELDORA TOGA COARE data. Prior to TOGA COARE, theEDITOR has seldom been used for southern or eastern hemisphere (e.g. TAMEX) data analysis.An undiscovered programming error reversed the sign of the longitude and latitude for southernand eastern hemispheres. This problem has been corrected at NCAR but will affect users whoobtained the software before October 1993. Contact Dick Oye or Michele Case for updateinformation (see Table 6 for a complete listing of telephone and e-mail information).
4.2 Interpolation, Multiple-Doppler Analysis, and Display
Implementation of the DORADE format data input capability for most interpolators hasbeen accomplished at NCAR and many other institutions. Persons wishing to implement thiscapability on other programs can make use of sample FORTRAN or C code obtained fromNCAR, CRPE, or elsewhere to ease the programming task. Our experience shows this as astraightforward programming task. At NCAR, the data interpolator REORDER (from ATD) andSPRINT (from MMM) can access UF and DORADE data.
Most multiple Doppler analysis and display programs (e.g. the MMM software CEDRIC)makes use of an interpolator output format as its input. Therefore, such programs are generallynot affected by the radar data format.
13
5. Obtaining Data, Software, and Answers
To facilitate responding to requests for data, software, and assistance, a special e-mailalias ([email protected]) has been established at ATD. Messages received through thisaddress will be directed to staff who can best handle the requests contained in the message. Youmay send e-mail to other ELDORA staff, but please copy the message to this ELDORA addressas well. All e-mail sent to "eldora" will be centrally logged (including the ATD replies). Thecompiled messages, with any necessary editing, will be redistributed to all interested parties. Thecompiled messages will also be maintained for anonymous ftp under the name ELDORA.notes, onthe machine "ftp.atd.ucar.edu" (128.117.78.19), in the directory -ftp/pub. It is hoped that thismechanism for compiling messages will serve to get any and all relevant information toresearchers in as short a time frame as possible.
Users are encouraged to obtain their own copies of the data directly from the NCARMSS. In order for ATD to best serve ELDORA users with data from the MSS, we request an e-mail be sent to [email protected] when you access the data from the MSS. The pathname on the MSS is /FOFDMG/DATA/TOGACOARE/ELDORA. Individual files are namedwith date and time keys as well as the data format.
Requests for data should indicate the required media for distribution. The preferredmedium of distribution is 8 mm (Exabyte) tape (either high or low density, no internal Exabytedata compression). Due to the large data volume, only a very limited quantity of data will beprovided for any user on 9-track tapes. Any such requests will be filled on a "background"priority basis because only a single 9-track drive is available. Hardware is available to facilitatebatch copying of 8 mm tapes; it is likely that the quickest and easiest way to obtain data will beon 8 mm high density tapes. There is a copy charge of $25/tape for data distributed byNCAR/ATD, which covers the cost of blank tapes, shipping, and some maintenance costs of thecopy equipment. The data requests should be sent to Robert Rilling at ATD.
For recovery of data from the NCAR MSS, contact the Scientific Computing Division's(SCD) consulting office for techniques and details.
For general questions on ELDORA data, please contact Peter Hildebrand or Wen-ChauLee at NCAR/ATD/RSF.
For the REORDER, SOLO, and the TRANSLATOR, please contact Richard Oye orMichele Case at NCAR/ATD/RDP.
For the SPRINT and CEDRIC software, please contact L. Jay Miller or William Andersonat NCARIMMM.
If you wish to be on the e-mail distribution list for updates concerning ELDORA, pleasesend a short message to the e-mail address listed above; include your name, your e-mail address(internet preferred when more than one option is available), your postal address, and your phonenumber.
14
Name E-mail address Phone Number Inquiry About:William Anderson [email protected] (303)497-8973 SPRINT, CEDRICMichele Case [email protected] (303)497-8756 Distribution on Translator, SOLO,
REORDERPeter Hildebrand [email protected] (303)497-2050 ELDORA DataWen-Chau Lee [email protected] (303)497-8814 ELDORA DataL. Jay Miller [email protected] (303)497-8975 SPRINT, CEDRICSCD Consultant consultl @ncar.ucar.edu (303)497-1278 NCAR MSSRichard Oye [email protected] (303)497-8809 Translator, SOLO, REORDERRobert Rilling rilling @stout.atd.ucar.edu (303)497-8842 ELDORA Data Request
Table 6. List of Persons to Contact Regarding ELDORA data.
15
Appendix A: DORADE Format Overview
Al. Introduction
The Common Doppler Radar Exchange Format, generally referred to as "UniversalFormat", (Barnes 1980) has been used extensively for the exchange of Doppler radar data. Themajor goal of this "Universal Format" was ease of access to ground based Doppler radar data byproviding a standard for exchanging radar data. With this goal, the efficiency of the format wasnot a primary consideration. The tape structure was therefore designed to include all headerinformation in every ray even though this information only changed rarely. While inefficient, thisstructure has allowed easy access to the data and has served the meteorology community well.
A new area of scientific research opened in the early 1980s with the advent of the tail-mounted Doppler radar aboard the NOAA WP-3D aircraft (Jorgensen et al. 1983; Parrish 1989).From inception the data from the NOAA P-3 airborne Doppler radars have been recorded in aNOAA field format because of the different geometry and data characteristics between airborneand ground-based radars. For subsequent analysis and combination with ground based radar datathese data frequently have been translated into Universal Format. Due to the need to recordnavigation information unique to the moving platform, additional entries were introduced into theUF local use header.
The new NCAR ELDORA airborne Doppler radar (Hildebrand et al. 1994) adds theadditional complication of having two radars operating on the same platform. This capability isalso emulated through beam switching or beam steering on the NOAA P-3s (Jorgensen andDuGranrut 1991). The result is that for any of these airborne radar systems, the data from tworadars are recorded simultaneously. Unlike some dual radar, ground based systems which haveco-located beams, these airborne radars have different beam positions. Airborne radar data arethus sufficiently complicated that Universal Format is a poor solution to the data recording needs.A new common format therefore needed to be developed.
The specifications for the new DOppler RAdar Data Exchange format, DORADE, weredeveloped by representatives from the primary producers of airborne Doppler radar data: theNCAR Atmospheric Technology Division (ATD), the NOAA AOML/Hurricane ResearchDivision (HRD) and NOAA National Severe Storms Laboratory (NSSL), and the Centre deRecherche en Physique de l'Environnement (CRPE, France). This group gathered in Miami, FL(NOAA/AOML/HRD) in April 1991 to discuss the proposed DORADE format. In addition todiscussing the structure of the format, airborne radar coordinate systems and terminologies to beused in the DORADE format were defined. This group met again at the 25th Radar Conferencein Paris, France, to continue the discussion on the structure and contents of the DORADE format.The initial draft of the proposed format was distributed in July 1991 to an expanded group,including scientists and programmers who will use the data, to solicit their comments. Numerousresponses were received. The revised draft was then distributed in November 1991 and the mostrecent version of DORADE format (Lee et al. 1994b) was distributed in June 1992.
16
The DORADE format will be used to exchange data collected by theELDORA/ASTRAIA and NOAA P-3 airborne Doppler radars and ground-based radars.Exchanged data should always be corrected as best as the facility making the tape can do, e.g.aircraft motion removed, range delay corrected, etc.
A2. Design Goals for DORADE
A2.1. Planning for Multiple Remote Sensing Systems
The Universal Format assumed radars would be ground based radars, operated at a singlePRF, with constant and uniform gate spacing and multiple variables per beam. The DORADEformat is designed to meet a new, broader span of possibilities. DORADE is designed to handle amoving platform, multiple radars or instruments in the same data set, different beam positions foreach radar, variable PRF, and variable gate spacing. The data structure is designed to be flexibleenough to enable a reasonable lifetime of upgrades, and should not be dependent on the recordingmedium.
Due to the scanning geometry of airborne Doppler radar, the data are collected with fore-and aft-pointed radar beams which are processed as independent data sets. Whether the radarconsists actually on one radar which scans fore and aft as on the NOAA P-3s (Parrish 1989,Jorgensen and DuGranrut 1991), or of two complete Doppler radar systems as in the NCARELDORA system (Hildebrand et al. 1994), the data are treated as if from two separate radars inthe subsequent multiple Doppler analysis. On the NOAA P-3 aircraft there is also a PPI-scanningbelly radar which is recorded along with the tail radar data. Other remote sensing systems arelikely additions to the NOAA P-3s and the NCAR Electra. It is therefore necessary to enable therecording of data from multiple remote sensing systems, each having its own operatingcharacteristics: scanning, range gating, data recording frequency, data types, etc.
A2.2. Measurement Conventions
There have been differences in the definitions of radar pointing angle, e.g. elevation, andazimuth, between the ground-based and airborne Doppler radars. The initial airborne radarsystems used an elevation-over-azimuth antenna pedestal, mounted vertically on the tail of theaircraft. The "azimuth" reading from this antenna was thus the rotation angle and the "elevation"was the fore-aft tilt angle. While there is a clear logic to this installation, this application of theterms "elevation" and "azimuth" has been extremely confusing to users of the data. TheDORADE specification thus includes definition of the radar beam pointing angles and all otherradar data navigation parameters.
17
A2.3. Data Compression
Due to the volume of data collected by the airborne Doppler radar systems, datacompression is an important topic. The NOAA P-3 radars routinely suppress the recording ofrange gates with no data in order to reduce the number of tapes which must be utilized on boardthe aircraft. From a meteorological point of view, 30% to 70% of the radar's range bins arelocated below the earth's surface or far above the weather. These data can easily be deleted fromthe recorded data.
Common forms of data compression include suppression of range gates with no data,range averaging of data at long range, and elimination of range bins below the earth surface orabove some altitude. Other approaches can also be utilized.
Airborne radars such as ELDORA can record at 500 Kbytes/s or more. For an 8 hourflight, about 14 Gbytes could thus be recorded. Without the use of some form of datacompression this volume of data can easily overwhelm the I/O, data storage, and computationalcapabilities of typical workstations. Planning for flexible implementation of real-time, i.e. prior torecording, data compression is therefore a requirement for DORADE.
A3. Description of the DORADE Format
A3.1. Recording Media
The DORADE format is not tied to a certain type of recording media, but instead shouldbe enabled on any type of recording media. The optimal recording medium is likely to be differentfrom the point of view of different radar operators or data analysts, and will change with time.The following rules are designed to assist with the goal of recording medium flexibility.
A3.2. Overview of the DORADE Data Structure
The DORADE data will be organized around "volumes", "sweeps", and "rays" of radardata. The customary high level logical construct for radar data is the volume scan. The volumescan consists of a series of radar beam sweeps which scan completely through some volume ofinterest. This concept works equally well for ground based or airborne radar and therefore formsthe basic data block for DORADE. The sub-element of the volume scan is the radar sweep whichoccurs as the radar scans once between scan limits or scans through 360° in azimuth or elevation.The sub-element of the sweep is the ray which is the collection of data along one beam pointingangle, during one signal processing dwell interval.
The data for each "volume" are written into one or multiple files, with a descriptivevolume header written at the beginning and end of the file. The volume header includes a sensor
18
descriptor for each of the different radars. This volume header is followed by data from theindividual radars, grouped into "sweeps". For the airborne radar the sweeps consist of complete360° scans in antenna rotation angle. Each sweep record contains data from a complete sweepfor one radar and is preceded by a sweep information block. The data from each sweep of oneradar are organized by "rays", with all data for a given parameter grouped together.
This logical structure is somewhat different than the field formats for the NOAA P-3 orfor the NCAR ELDORA. The field formats have a version of the volume header which issomewhat larger than the normal DORADE header to include various engineering data. Thus, theELDORA FIELD format volume headers are only written when something is changed or amaximum length of time has passed. The field headers are followed by data rays. In order toeliminate the need for sorting and buffering of data into sweeps in real time, the field format dataare not organized by radar. Thus the fore and aft beams are interleaved as on the NOAA P-3aircraft where the belly radar data are also interleaved. Consequently, any programs making useof the data must check the radar descriptors at the beginning of each ray.
19
Appendix B: Radar Coordinate Transformation
The coordinate transformation matrix and the expression of the Doppler velocity aresummarized in this appendix (For details, see Lee et al. 1994a). Following is a summary list of allsymbols used in this appendix.
x,y,z Leveled Cartesian coordinate systems relative to the radaru,v,w Velocity components in the Cartesian coordinate systemr,K,0 Airframe relative spherical coordinate systemsr Distance from the radar to a pulse volume0 Rotation (spin) angle
Elevation angleD Drift angleP Pitch angleR Roll angleH HeadingT Tilt anglek Azimuth angleVt Terminal velocityL Distance from the INS to the radarVG Aircraft horizontal ground speedWG Aircraft vertical ground speed
x x cos(a + R) sin H cos a sin P+ cos Hsin(0 a + R) costa + sin Hcos Psinty = r cos(Oa + R) cos H costa sin P - sin H sin(O a + R) cost + cos H cos P sin I
sJ r^ cos P c os ( a C Ocs( + R)costa + sin Psinta )
= tan -
y-1 Z
0 =sin-' -r
V, = ( cos0 cos + vcosO sin ) + (w -v, - WG)sin - VGsinT,
dH+L{ (1 + cos P)(cos 0 cos k cos H - cos sin k sin H)-
dtdP
-[sin P(cos ( cos X sin H + cos 0 sin X cos H) - sin ( cos P]-}.dt
20
References:
Bargen, D. and Brown, 1980: Interactive radar velocity unfolding. Preprints, 19th Conf on RadarMeteorology, Miami Beach, FL, Amer. Meteor. Soc., 278-285.
Barnes, S. L., 1980: Report on a meeting to establish a common Doppler radar exchange format.Bull. Amer. Meteor. Soc., 61, 1401-1404.
Hildebrand, P. H., C. W. Walther, C. L. Frush, J. Testud and F. Baudin, 1994: TheELDORA/ASTRAIA airborne Doppler weather radar: Goals, design and first field tests.IEEE Proceedings, (In Press).
Jorgensen, D. P., and J. D. DuGranrut, 1991: A dual-beam technique for deriving wind fieldsfrom airborne Doppler radar. Preprints, 25th Int. Conf. on Radar Meteorology, Paris,France, Amer. Meteor. Soc., 458-461.
Jorgensen, D. P., and P. H. Hildebrand, and C. L. Frush, 1983: Feasibility test of an airbornepulse-Doppler meteorological radar. J. Climate Appl. Meteor., 22, 744-757.
Lee, W.-C., P. Dodge, F. D. Marks, and P. H. Hildebrand, 1994a: Mapping of airborne Dopplerradar data. J. Atmos. Oceanic Technol., 11, 572-578.
Lee, W.-C., C. Walther, and R. Oye, 1994b: The Doppler Radar Exchange Format: DORADE.NCAR Tech. Note. NCAR/TN 403+IA. 18pp. (Also available on internet: Mosaichttp://www.atd.ucar.edu under Recent RSF Technical Reports).
Parrish, J. R., 1989: New NOAA OAO WP-3D Doppler radar system. Preprints, 24th Conf. onRadar Meteorology, Tallahassee, FL, Amer. Meteor. Soc., 615-618.
Rodi, A. R., J. C. Fankhauser, and R. L. Vaughan, 1991: Use of distance-measuring equipment(DME) for correcting errors in position, velocity and wind measurements from aircraftInertial Navigation Systems. J. Atmos. Oceanic Technol., 8, 827-834.
Testud, J., P. H. Hildebrand and W.-C. Lee, 1994 : A procedure to correct airborne Dopplerradar data for navigation, using the echo returned from the earth's surface. J. Atmos.Oceanic Technol. (Accepted).
Walther, C., and W.-C. Lee, 1994: The ELDORA field data format. NCAR Tech. Note (Inpreparation).
21
