+ All Categories
Home > Documents > Validation and Monitoring of the OSI SAF Low Resolution...

Validation and Monitoring of the OSI SAF Low Resolution...

Date post: 06-Jul-2020
Category:
Upload: others
View: 0 times
Download: 0 times
Share this document with a friend
30
Ocean & Sea Ice SAF Validation and Monitoring of the OSI SAF Low Resolution Sea Ice Drift Product OSI-405 Figure1: Example ice drift vector fields as processed at the EUMETSAT OSI SAF. Left panel displays displacement vectors and length (color shades) in the Arctic Ocean in April 2010, while right panel displays vectors and dis- placement length in the Ross Sea in August 2010. Both examples are from the multi-sensor merged product. Version 3 — Jan 2012 Thomas Lavergne
Transcript
Page 1: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

Ocean & Sea Ice SAF

Validation and Monitoring of the OSI SAFLow Resolution Sea Ice Drift Product

OSI-405

Figure 1: Example ice drift vector fields as processed at the EUMETSAT OSI SAF.Left panel displays displacement vectors and length (color shades) in theArctic Ocean in April 2010, while right panel displays vectors and dis-placement length in the Ross Sea in August 2010. Both examples arefrom the multi-sensor merged product.

Version 3 — Jan 2012Thomas Lavergne

Page 2: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays
Page 3: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

Documentation Change Record:

Version Date Author Descriptionv0.9 03.12.2008 TL Initial version, before reviewv1 05.02.2009 TL Amended after PCR commentsv2 15.03.2010 TL Include more in-situ data sources,

change the (spatial) collocationmethod and extend validation timeperiod.

v3 25.01.2012 TL Include more in-situ data sources, in-troduce SAR ice drift as ground truthover SH sea ice, change (temporal)collocation method.

Page 4: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131

Table of contents

Table of contents

List of Figures

1 Introduction 11.1 The EUMETSAT Ocean and Sea Ice SAF . . . . . . . . . . . . . . . . . . . . 11.2 Scope and Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Validation dataset 42.1 In-situ trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.2 High-resolution SAR-based sea ice drift product . . . . . . . . . . . . . . . . 62.3 Remarks on the validation dataset . . . . . . . . . . . . . . . . . . . . . . . . 62.4 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.5 Geographical overview of the validation dataset . . . . . . . . . . . . . . . . . 7

3 Validation methodology 93.1 Variables to be validated . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.2 Reformatting of the validation dataset . . . . . . . . . . . . . . . . . . . . . . 93.3 Collocation strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.4 Representativity error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.5 Graphs and statistical measures . . . . . . . . . . . . . . . . . . . . . . . . . 12

4 Results of validation 134.1 Validation over the NH area . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.2 Validation over the SH area . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

5 Conclusion 22

References 24

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 5: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131

List of Figures

1 Examples of NH and SH ice drift vector fields . . . . . . . . . . . . . . . . . . 1

2 Trajectories of NH validation drifters . . . . . . . . . . . . . . . . . . . . . . . 8

3 NH validation graphs for ice drift products . . . . . . . . . . . . . . . . . . . . 144 NH validation graphs for ice drift products — continued . . . . . . . . . . . . . 155 SH validation graphs against SAMS Icebell buoys . . . . . . . . . . . . . . . . 196 SH validation graphs against DTU SAR product . . . . . . . . . . . . . . . . . 21

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 6: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 1

1. Introduction

1.1 The EUMETSAT Ocean and Sea Ice SAF

For complementing its Central Facilities capability in Darmstadt and taking more benefit fromspecialized expertise in Member States, EUMETSAT created Satellite Application Facilities(SAFs), based on co-operation between several institutes and hosted by a National Meteo-rological Service. More on SAFs can be read from www.eumetsat.int.

The Ocean and Sea Ice Satellite Application Facility (OSI SAF) is producing on an op-erational basis a range of air-sea interface products, namely: wind, sea ice characteristics,Sea Surface Temperatures (SST), Surface Solar Irradiance (SSI) and Downward LongwaveIrradiance (DLI). The sea ice products include sea ice concentration, the sea ice emissivity,sea ice edge, sea ice type and sea ice drift and sea ice surface temperature (from mid 2013).

The OSI SAF consortium is hosted by Meteo-France. The sea ice processing is per-formed at the High Latitude processing facility (HL centre), operated jointly by the Norwegianand Danish Meteorological Institutes.

Note: The ownership and copyrights of the data set belong to EUMETSAT. The datais distributed freely, but EUMETSAT must be acknowledged when using the data. EUMET-SAT’s copyright credit must be shown by displaying the words ”copyright (year) EUMETSAT”on each of the products used. User feedback to the OSI SAF project team is highly valued.The comments we get from our users is important argumentation when defining develop-ment activities and updates. We welcome anyone to use the data and provide feedback.

1.2 Scope and Overview

Sea ice drift products for Northern and Southern Hemisphere are processed at the HighLatitude center of the Ocean & Sea Ice Satellite Application Facility (EUMETSAT OSI SAF).Those datasets are introduced and documented in a dedicated Product User’s Manual(PUM, Lavergne and Eastwood 2011) and in an Algorithm Theoretical Basis Document(ATBD, Lavergne et al. 2011) that can both be found on http://osisaf.met.no. The HighLatitude processing facility (HL centre) is jointly operated by the Norwegian and Danish Me-teorological Institutes.

See http://osisaf.met.no for real time examples of the products as well as updated infor-mation. The latest version of this document can also be found there. General informationabout the OSI SAF is given at http://www.osi-saf.org.

This Validation and Monitoring report only deals with the OSI-405 low resolutionsea ice drift product. The medium resolution ice drift product based on AVHRR

imagery, OSI-407, is documented in a dedicated report.

The aims of this report are several:

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 7: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 2

1. To document the level of agreement between the OSI SAF low resolution sea ice driftproduct and ground-based truth. Various graphs displaying the match between thesatellite product and the in-situ datasets should give (qualitative) confidence in usingthe product.

2. To report quantitative estimates of errors and uncertainties in the product. Particularly,the bias and uncertainty covariance matrix is computed. It is important that eachsingle-sensor and the multi-sensor products are validated separately so that users canhave error estimates for the product they choose to use.

Chapter 2 presents the datasets used as validation data while chapter 3 documentsthe validation strategy and, particularly, the way collocation is handled. Chapter 4 providesdetailed, graphical and quantitative analysis of the validation results. We conclude in chap-ter 5.

Note that the OSI SAF low resolution sea ice drift product will not be introduced at anydepth in this report. Refer to the PUM and http://osisaf.met.no for information on the algo-rithms, processing schemes and data format.

Let us nonetheless remind that the OSI SAF low resolution ice drift product comes asdaily vector fields obtained by processing low-resolution satellite signal from, among others,AMSR-E, SSM/I and ASCAT. It is computed on a Northern Hemisphere grid, delivered fromOctober, 1st to April, 30th every year, and on a Southern Hemisphere grid, available from 1st

April to 31st October. Summer ice drift is indeed not as straightforward from those sensors.It is a 2 days (48 hours) ice drift product on a 62.5 km resolution polar stereographic grid.Both single and multi sensor products are distributed.

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 8: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 3

1.3 Glossary

AARI Arctic and Antarctic Research InstituteASCAT Advanced SCATterometerAVHRR Advanced Very High Resolution RadiometerAMSR-E Advanced Microwave Scanning Radiometer for EOSAWI Alfred Wegener InstituteCDOP Continuous Development and Operations PhaseDMI Danish Meteorological InstituteDMSP Defense Meteorological Satellite ProgramDTU Danish Technical UniversityIABP International Arctic Buoy ProgramITP Ice Tethered ProfilerGCOM-W Global Change Observation Mission – WaterGPS Global Positioning SystemHL High Latitudesmet.no Norwegian Meteorological InstituteNetCDF network Common Data FormatNH Northern HemisphereNP North PoleSAF Satellite Application FacilitySAR Synthetic Apperture RadarSH Sourthern HemisphereSSM/I Special Sensor Microwave/ImagerWSM Wide Swath Mode

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 9: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 4

2. Validation dataset

In this section, we introduce the ice motion datasets that constitute our best estimate of theground truth and that is used as reference to validate the OSI SAF low resolution sea icedrift dataset.

Several data sources are available for validating an ice drift product and they can besorted into three groups:

1. Trajectories of in-situ ice drifters. Historically, this is the main validation data source.A fair number of buoys are indeed deployed in the ice covered ocean to measureatmospheric, cryospheric or oceanic variables (e.g. Mean Sea Level Pressure, icethickness or temperature and salinity profiles of the ocean). Of interest to us is the factthat they regularly and automatically report their position via the Argos system or bytransmitting GPS positions as part as their data stream. Drifting ships (like the Tara)or manned stations (NP-35, NP-36, etc...) also constitute good opportunities to get icetrajectory data, sometimes in near-real-time.

2. High resolution satellite based ice drift datasets. Those are processed from highresolution satellite images (e.g. ENVISAT SAR or AVHRR). Those products are not’ground truth’ but are assumed to present much less deviations to truth than the lowresolution ice drift datasets.

3. Moored Doppler-based velocity measures from under the ice. This source of datapresents three major disavantages. Firstly, they are Eulerian measures of instanta-neous velocity, a quantity that is not directly comparable to satellite-based ice dis-placement vectors. Second, they do not transmit data in near-real-time and are thusnot suitable for daily monitoring of a product. Finally, they are often located in coastalareas where the retrieval of sea ice drift from low resolution sensors is challenged bythe proximity to land.

For the validation exercise reported in this document, both in-situ trajectories and a high-accuracy SAR-based ice drift dataset are used.

2.1 In-situ trajectories

2.1.1 Ice Tethered Profilers

The Ice Tethered Profilers (ITP) platforms are advanced autonomous drifting instrumentthat are designed to measure temperature and salinity profiles in the ocean under sea ice.As part of its daily data stream, each ITP transfers hourly unfiltered GPS locations. As ofsummer 2011, there are about 25 active (plus additional 30 completed) ITPs in the ArcticOcean that form a high quality validation dataset, especially for the Beaufort Sea, CanadianBasin and Fram Strait.

It is noteworthy that the primary objective of ITPs is to sample the cwater column and,thus, require to be deplyed in location with enough water depth. This excludes all shelf arealike Laptev or Chuckchi Sea and limits the spatial sampling of the Arctic Ocean.

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 10: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 5

The ASCII formatted level 1 raw data position files (itpNrawlocs.dat) for all ITPs weredownloaded from the FTP server at Woods Hole Oceanographic Institution and processedto extract ice drift vectors.

2.1.2 Tara

The Tara schooner (http://www.taraexpeditions.org) has been one of the flagships of the De-veloping Arctic Modelling and Observing Capabilities for Long-term Environmental Studies(DAMOCLES, www.damocles-eu.org) project. Its now famous drift started in September2006. It followed the transpolar drift motion and was released from the ice in the first daysof January 2008. All along its 500 days drift, the ship’s position was recorded via Argos andin a sub-hourly GPS log.

Although it provides only one vector per day, moreover only until January 3rd 2008, thisdataset is relevant since it samples a geographical region (Transpolar Drift and Fram Strait)that is not very often sampled by the other drifters.

2.1.3 Russian manned polar stations

GPS trajectory logs for the Russian manned stations NP-35, NP-36, NP-37 and NP-38 weremade available by the Arctic and Antarctic Research Institute (AARI). The drift of NP-35lasted from September 2007 to July 2008. Its GPS trajectory was not available in near-real-time. NP-36 was deployed in September 2008 and recovered August 2009. NP-37 wasdeployed September 2009 and recovered May 2010. NP-38 was deployed October 2010.Positions for NP-36, NP-37 and NP-38 were available in near-real-time from the AARI website http://www.aari.nw.ru.

As for the Tara, the NP stations provides ice drift vectors in a region not often sampledby the ITPs, namely the Nansen Basin.

The trajectory of NP-39 is not used in this version of the report (deployed 2nd October2011).

2.1.4 Buoy array deployed by Brummer et al.

In the context of the DAMOCLES project, an array of 16 CALIB buoys was deployed in April2007 in the central Arctic. They were dropped from an airplane within a square pattern of400 × 400 km centred on the Tara (see section 2.1.2). Trajectories are originaly recorded viathe Argos doppler positioning system, but the dataset used in the present report have beenpre-processed and quality checked at the University of Hamburg. They are thus expected ofmuch higher quality than raw Argos (Dr. Gerd Muller, personal communication).

2.1.5 SAMS Icebell buoys

The Scottish Associastion for Marine Science (SAMS) designs robust thermistor-based IceMass Balance buoys for monitoring the snow and sea ice thickness. Part of the data streamsent via Iridium are the sub-hourly GPS locations of the platforms. In total, about 20 Ice-bell buoys were deployed between November and December 2010. However, most of theplatforms drifted too close to land, or stopped reporting their positions after 2-3 months.

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 11: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 6

Moreover, the SH region is only covered by the OSI SAF ice drift product from 1st Aprilto 31th October. Surface melting and optically thicker atmosphere indeed prevent the opera-tional, basin scale retrieval of sea ice motion from the satellite sensors at use during australsummer.

As a result, only few days at the end of the Icebell 10 drift (Admunsen Sea) and about4 months of the Icebell 18 drift (Ross Sea) are available for this validation exercise in theSouthern Hemisphere.

2.2 High-resolution SAR-based sea ice drift product

High-resolution (10 km), 24 hours sea ice motion vectors are processed in near-real-timeat the Danish Technical University (DTU, www.seaice.dk). They are obtained by a patternmatching method applied on Wide Swath Mode (WSM) images of the Envisat ASAR instru-ment. These images are first re-sampled at a 300 m resolution, then processed througha Maximum Cross-Correlation algorithm, and finally distributed on an FTP server as ASCIIfiles.

As part of the MyOcean project, these ice motion vectors are also made available innetCDF format. Still in the frame of this project, a validation is conducted at DTU (againstthe Ice Tethered Profilers trajectories) that document a very high accuracy of the WSM driftvectors. Some results can be accessed from http://www.seaice.dk/MyOcean/validation.

The error statistics are max 0.7 km standard deviation of error in X/Y components of thedrift, and some tens of meters in bias. The high accuracy of these vectors make it possibleto use them as comparison data when validating the OSI SAF low resolution sea ice driftdataset.

2.3 Remarks on the validation dataset

Although we tried to use as many good quality drifters as possible, entire regions coveredby the OSI SAF ice drift product grid are not sampled by our validation dataset. See, forreference, Figure (2).

Two such regions are the Baffin and Hudson Bay for which it was not possible to obtaintrajectories in the validation period we have been covering. Some buoys are released everyyear in the Nares Strait. However, most of them sink due to unstable ice conditions in thestrait or Baffin Bay at that period.

A major validation data gap is also over the Antarctic sea ice during winter (April toOctober). The only available GPS trajectories we could access are those from SAMS (seesection 2.1.5). Other short-lived installations exist in earlier years, but they are often not inthe appropriate time period, or not easily accessible to us.

2.4 Acknowledgements

The Ice-Tethered Profiler data were collected and made available by the Ice-Tethered Pro-filer Program based at the Woods Hole Oceanographic Institution http://www.whoi.edu/itp.

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 12: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 7

GPS-located data from Russsian stations were kindly provided by the Arctic and Antarc-tic Research Institute (AARI, http://www.aari.ru) of Roshydromet, PIs Vladimir Sokolov andVasily Smolyanitsky.

The GPS trajectory of the Tara was kindly made available by the DAMOCLES project.

The preprocessed trajectories of the 16 CALIB buoys array were kindly offered by Dr. BurghardBrummer and Dr. Gerd Muller, both from the Meteorological Institute at University of Ham-burg.The quality checked GPS trajectories of the Icebell buoys were provided by Phil Hwang fromthe Scottish Association for Maring Science (SAMS).

The archive of global SAR-based product in Ascii format was made available by RobertoSaldo (DTU)

2.5 Geographical overview of the validation dataset

Figure (2) displays a graphical overview of the in-situ trajectories that were used in thevalidation period for NH area. All the color-coded position records are obtained with GPSaccuracy, that is the ITPs (violet), the NP stations (blue) and the Tara (orange). The light-grey records are mostly from the IABP, coming with Argos positions and are thus not usedin this validation report.

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 13: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 8

Figure 2: Trajectories of some validation drifters (ITPs, Tara, NP-35 and NP-36) inthe period 1st October to 30th April in 2006-2007 to 2010-2011. See textfor a description of the colors used.

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 14: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 9

3. Validation methodology

The validation strategy is introduced in this chapter. It covers the re-formatting of the tra-jectories and the collocation with the sea ice drift product. We also present the graphs andstatistical properties that will be displayed and commented upon in chapter 4.

3.1 Variables to be validated

As introduced in the sea ice drift PUM, an ice drift vector is fully described with 6 values:the geographical position of the start point (lat0 and lon0), the start time of the drift (t0), theposition of the end point of the drift (lat1 and lon1) as well as the end time of the drift (t1).

However, the primary variables the ice drift processing software optimizes are dX anddY , the components of the displacement vector along the X and Y axes of the product grid.Those are thus the two variables we are aiming at validating.

3.2 Reformatting of the validation dataset

3.2.1 Trajectories of in-situ drifters

In any validation exercise, especially if making use of a broad range of data sources, one isconfronted to new and varying formats. Most of the times, trajectories from in-situ drifters areavailable in an ASCII format, proposing one position and time stamp per line. The variousformats for the time information, in particular, as well as the ordering of columns make itchallenging to design a unique software package to read all those files.

A first step of this validation effort has thus been the design of dedicated software rou-tines to read the observation files, extract the portion of the trajectory that fits the time spanof the OSI SAF ice drift product file and dump the validation data in a NetCDF formatted file.

3.2.2 Pre-processing of the DTU SAR-based product

The WSM DTU SAR product is available as 24 h drift vectors. The exact start and stoptime however varies over the basin due to the orbit configuration of Envisat and revisit timeof the platform. As a result, the duration of the DTU SAR product is anywhere between 7and 47 hours (Roberto Saldo, personnal communication). Since the OSI SAF is a 48 h driftproduct, it cannot directly be compared with the SAR-based product.

It is noteworthy that the SAR images used as end image for a drift vector are re-usedas start image for a subsequent drift period. In order to extract 48 h drift vectors, we thusfirst concatenate over time any 2 (sometimes 3) SAR drift scenes. The concatenated vectorshave, on average, a duration of 48 h, and quite some variation in the individual start and stoptime. These concatenated vectors, stored in the same NetCDF format as the trajectories ofin-situ drifters, are those entering the collocation routines described below.

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 15: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 10

3.3 Collocation strategies

In order to compare the OSI SAF sea ice drift product with the validation trajectories, theyneed to be collocated one with the other. Collocation is the act of selecting or transformingone or both datasets so that they represent the same quantity, at the same time and at thesame geographical location.

Because the OSI SAF ice drift product comes with two flavours of time information (referto PUM, section Time information), two validation exercises are conducted:

• One using a 2D collocation, in which the satellite product is considered representing adrift from day D@1200 UTC to (D+2)@1200 UTC, uniformly over the grid.

• One with a 3D collocation, in which the position dependent start and end times areused (found in datasets dt0 and dt1, in the product file).

The reason for having those two validation strategies is that some users might wish (orhave to) ignore the accurate timing information provided with each vector. Using only thecentral times, these users need to know if the uncertainty estimates are to be enlarged and,if so, by which amount.

3.3.1 2D collocation

The in-situ drift vector is defined by first selecting the start and end position record in theirtrajectory. Those are the ones closest (in time) to 1200 UTC, on both dates. From those 2positions, dX ref and dY ref are computed. The product dXprod and dY prod are those of thenearest-neighbour in the product grid.

3.3.2 3D collocation

The in-situ start (end) point is searched for along the trajectory: each record is remappedinto the product grid where a product start (end) time is computed by bilinear interpolationfrom the 4 encompassing grid points. Because the records are ordered chronologically, itis possible to stop searching as soon as both start and end in-situ records are selected.As in the 2D collocation, they define the ’truth’ displacement components: dX ref and dY ref.The components for the product (dXprod and dY prod) are selected as those of the nearest-neighbour in the product grid.

3.3.3 Additional remarks

In version v1 of this validation exercise, the spatial collocation was achieved by bi-linearinterpolation of the 4 neighbouring vectors from the product grid to the position of the ref-erence vector. Further investigations confirmed that this method could lead to artificiallygood validation statistics, since part of the noise in the product was averaged out in thisprocess. From v2 of this report onwards, spatial collocation relies on nearest-neighbour se-lection. The distance to the nearest neighbour must be inside 40 km radius from the start ofreference vector.

In versions v1 and v2 of this validation report, matchup pairs were allowed in the val-idation dataset if the time difference of both the start and stop records were less than 3hours. This choice conducted to potentially allowing differences in drift durations of up to 6

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 16: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 11

hours. Further investigation of the v1 and v2 validation pairs, however, demonstrated thatthe difference in drift durations was mostly in the range [-1 h:+1 h]. This was thanks tothe hourly (sometimes sub-hourly) sampling of most of the buoy trajectories, allowing foraccurate temporal registration. This excellent temporal sampling is also why changing thetime collocation criteria did not lead to significant changes in results in Hwang and Lavergne2010b, section 4.1.

In versions v3 of the validation excercise, the temporal collocation was changed to allowcollocation pairs only if the time difference at the start data record is within [-3 h:+ 3h], andif the difference in drift duration is within [-1 h:+1 h]. As noted above, this change has verylittle impact on the selection of buoy matchups since they mostly report hourly sampledtrajectories. This change does not invalidate earlier validation results documented in v2 ofthis report, nor those in Lavergne et al. 2010 or Hwang and Lavergne 2010b. This change ishowever required to ingest high-resolution SAR-based drift vectors (section 2.2) which mighthave strongly varying temporal sampling characteristics due to orbit configuration and revisittime.

To clear the temporal and spatial collocation criteria above is not enough for entering thevalidation dataset. Additional constraints are imposed to obtain a more controlled validation:

• A validation vector must be surrounded by 4 valid OSI SAF vectors. Although only thenearest of these 4 is considered in the statistics, this constraint is introduced to avoidvalidation data in the outer edges of the vector field, like in the marginal ice zone, orin coastal regions (land-fast ice). Since some of the ice-tethered buoys we use (e.g.ITPs) are designed to continue floating when sea ice melts, this constraint is also aneffective way of not collocating ocean drift measurements with our product.

• Any two validation vectors for a given product date must be separated by at least3 × 62.5 = 187.5 km. This constraint is introduced so that all data pairs enteringthe validation dataset are independent from each others. Two neighbouring OSI SAFsea ice drift vectors indeed exhibit correlated uncertainties due to the overlap of theirimage-matching windows (e.g. Lavergne et al. 2010, section 3.2). This correlation ofneighbouring vector should ideally be taken into account when computing validationstatistics. However, thanks to the excellent coverage of the OSI SAF product, we canbe strict about the correlations and steer away from them by rejecting validation pairsthat are too close to each others on a daily grid.

3.4 Representativity error

Although we only use high quality buoy position data and although the collocation metodsand parameters are quite sringents, a possibly high and mostly uncontrolled source of errorresides in the representativity mismatch between the scales sampled by the buoy and thesatellite product.

A buoy indeed samples the motion of the ice floe it was deployed over. Although in-vestigators in field campaigns tend to choose rather large floes for limiting the risk of thebuoy disappearing too rapidly, the size and shape of the floe can change with time throughcollision or breaking events.

On the other hand, the satellite ice drift product samples the motion of a much largerarea of the sea ice surface that is close to 12000 km2. The mismatch between the twoscales of motion contributes to part of the error budget and it is not possible to separate this

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 17: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 12

representativity error from the measurement error of the satellite product with such a simpletwo-way statistical analysis. See Hwang and Lavergne 2010a, section 3.3 for a discussionon the representativity error budget of comparing a satellite drift vector to one recorded by abuoy.

3.5 Graphs and statistical measures

As introduced in section 3.1, this report is mostly interested in validating drift componentsdX and dY . We concentrate on two comparison exercises for reporting validation resultsfor those variables.

3.5.1 Product vs Reference

In this type of graph, the x-axis is the truth and the y-axis is the estimate given by theproduct. In an ideal comparison, all (truth,product) pairs are aligned on the 1-to-1 line.The spread around this ideal line can be expressed by the statistical correlation coefficientρ(Reference,Product), noted ρ(R,P ). If, at the same time, this correlation is close to 1 andthe parameters of the regression line are close to 1 (α, slope) and 0 (β, intercept), then thematch between the truth and the product is satisfactory.

In this report, a unique graph (and statistical values) is produced for dX and dY at thesame time. This means that the pairs appearing on the graphs are both (dX ref,dXprod)and (dY ref,dY prod). This also implies that errors in dX and dY are considered globallyindependent, an assumption that is validated using the graphs introduced in the next section.

3.5.2 Error in dY vs error in dX

In this type of graph, the x-axis is the error in dX, that is dXprod − dX ref (noted ε(dX))and the y-axis is the error in dY , that is dY prod − dY ref, noted ε(dY ). This graph is amore interesting approach for presenting the validation data than the one presented in theprevious section.

Indeed, such a graph permits giving quantitative estimates for:

• the statistical bias1 in both components: 〈ε(dX)〉 and 〈ε(dY )〉;

• the statistical standard deviation of the errors in both components: σ(dX) and σ(dY );

• the statistical correlation between the errors in both components: ρ(εdX, εdY ).

The last three quantities enter the error covariance matrix Cobs which is of prime importanceto any data assimilation scheme. It is important to note the difference between the corre-lation coefficient introduced in this section and the one from section 3.5.1. ρ(εdX, εdY )assesses if the errors in the two components of the drift vector are correlated or not. ρ(R,P )assesses if the product (seen as a sample) is close to a linear combination of the referencedataset (seen as a sample too).

In any case, those are statistical measures of the errors. They can only give averageuncertainties estimates and result in a unique set of numbers (those populating Cobs) to beused for an extended period of time (all distribution year round) and for the whole extent ofthe Northern or Southern Hemisphere grid.

1〈x〉 is the average of x.

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 18: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 13

4. Results of validation

Validation of the NH and SH product grids are addressed separately in this section. A 3year long validation of the NH region is first documented and analysed. In a later section,we document the quality over the SH area against the few buoy datasets available in thisregion. Additionally, we introduce a comparison against a high-accuracy SAR-based icedrift product available from DTU.

4.1 Validation over the NH area

Note: The following results are duplicated from v2 of the report. Since no algorithm changewas implemented that could affect the NH region processing, they are still valid.

4.1.1 Graphs and analysis

Figure (3) and Figure (4) introduce selected validation graphs for various single-sensorOSI SAF sea ice drift products as well as for the multi-sensors dataset. For all plots, thegeographical region being validated is the Northern Hemisphere and the validation pe-riod includes all product files whose start date is between October, 1st and April, 30th in2006-2007, 2007-2008 and 2008-2009.

On ”Error(dY ) vs Error(dX)” graphs (left column in Figure (3) and all in Figure (4)), thered (green) thick line encompasses a region of the 2D frequency distribution of the errorsrepresenting 0.68 (0.95) probability of occurrence. Corresponding dashed lines are drawn forthe 1.5σ and 2.5σ ellipses, which are known to delineate 0.68 and 0.95 probability regions inthe case of a bivariate normal distribution with parameters 〈ε(dX)〉, 〈ε(dY )〉, σ(dX), σ(dY )and ρ(εdX, εdY ) (Lavergne et al. 2006, Appendix G). A black plus (+) symbol is located atthe centre point of the PDF, namely (〈ε(dX)〉, 〈ε(dY )〉).

On ”Product vs Reference” graphs (right column in Figure (3)) each validation pair (onefor dX and one for dY ) are plotted in a 1-to-1 scatterplot. The solid line is the regressionline (whose coefficients are entered as labels in the plot area).

Figure (3) and Figure (4) are a simple and effective way of presenting the validationresults and get a good impression of the quality of each product. First, it can be noted thatall products are mostly non biased. The magnitudes of 〈ε(dX)〉, 〈ε(dY )〉 are indeed small (acouple of 100 metres) in comparison to the standard deviations (a couple of 1000 metres).This is an important result when it comes to using the products in assimilation exercises.The bias in the Y component of the drift is usually larger than for the X component. Besides,it is quite consistently negative which indicates that the satellite product has a smaller driftmagnitude than the reference dataset. Kwok et al. (1998) (section 3.2, p. 8203) wrote adetailed investigation of a similar bias in their ice drift product. Such a thorough analysishas not yet been performed for the OSI SAF product and we are left with referring to Kwok’sanalysis and to following versions of this report.

It also clearly appears from an analysis of Figure (3) that the method implemented in theOSI SAF chain results in a limited uncertainty. Displacement errors (in terms of standard

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 19: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]

Error in drift along the processing grid (dX) [km]

N = 3977

<ε> = (-0.10 , -0.10) [km]

σ = (2.70 , 2.77) [km]

ρ = -0.03

amsr-aqua

NN3D

ε = x|yprod - x|yref

Validation period:

20061001-20070430 + 20071001-20080430 + 20081001-20090430

pdf regions

68%95%

-40

-20

0

20

40

-40 -20 0 20 40

am

sr-

aq

ua

d

rift

(d

X a

nd

dY

) a

lon

g t

he

pro

ce

ssin

g g

rid

[km

]

Reference drift (dX and dY) along the processing grid [km]

Validation period:

20061001-20070430 + 20071001-20080430 + 20081001-20090430

N = 7954

α = 0.95

β = -0.02

ρ = 0.97

amsr-aqua

NN3D

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]

Error in drift along the processing grid (dX) [km]

N = 4218

<ε> = (-0.07 , -0.02) [km]

σ = (4.10 , 4.01) [km]

ρ = 0.02

ssmi-f15

NN3D

ε = x|yprod - x|yref

Validation period:

20061001-20070430 + 20071001-20080430 + 20081001-20090430

pdf regions

68%95%

-40

-20

0

20

40

-40 -20 0 20 40

ssm

i-f1

5

drift

(d

X a

nd

dY

) a

lon

g t

he

pro

ce

ssin

g g

rid

[km

]

Reference drift (dX and dY) along the processing grid [km]

Validation period:

20061001-20070430 + 20071001-20080430 + 20081001-20090430

N = 8436

α = 0.93

β = 0.08

ρ = 0.94

ssmi-f15

NN3D

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]

Error in drift along the processing grid (dX) [km]

N = 4707

<ε> = (-0.12 , -0.27) [km]

σ = (3.65 , 4.05) [km]

ρ = 0.04

multi-oi

NN3D

ε = x|yprod - x|yref

Validation period:

20061001-20070430 + 20071001-20080430 + 20081001-20090430

pdf regions

68%95%

-40

-20

0

20

40

-40 -20 0 20 40

mu

lti-o

i d

rift

(d

X a

nd

dY

) a

lon

g t

he

pro

ce

ssin

g g

rid

[km

]

Reference drift (dX and dY) along the processing grid [km]

Validation period:

20061001-20070430 + 20071001-20080430 + 20081001-20090430

N = 9414

α = 0.89

β = 0.04

ρ = 0.95

multi-oi

NN3D

Figure 3: Selected validation graphs for AMSR-E (top row), SSM/I ’F15’ (middlerow) and merged (multi-oi) (bottom row) products. All pertain to NH area,to the 3D collocation setup and October 1st to April 30th in 2006-2007,2007-2008 and 2008-2009 period. Left (right) column presents ”error(dY )vs error(dX)” (”product vs reference”) types of graphs. N is the number ofvalidation pairs.

Page 20: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 15

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]

Error in drift along the processing grid (dX) [km]

N = 3668

<ε> = (-0.17 , -0.24) [km]

σ = (4.71 , 4.47) [km]

ρ = 0.01

ascat-metopA

NN3D

ε = x|yprod - x|yref

Validation period:

20061001-20070430 + 20071001-20080430 + 20081001-20090430

pdf regions

68%95%

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]

Error in drift along the processing grid (dX) [km]

N = 3673

<ε> = (-0.20 , -0.23) [km]

σ = (4.88 , 4.62) [km]

ρ = 0.01

ascat-metopA

NN2D

ε = x|yprod - x|yref

Validation period:

20061001-20070430 + 20071001-20080430 + 20081001-20090430

pdf regions

68%95%

Figure 4: Validation graph for ASCAT product. The left (right) panel contains resultsfor the 3D (2D) collocation methods.

deviation) are small (maximum 4.5 km, 1/3 of image pixel size). Those errors are smalldespite no special treatment has been implemented for correcting the satellite geolocationuncertainty which might contribute to a fair amount for sensors like SSM/I and AMSR-E (seefor example Wiebe et al. 2008).

Another interesting result is that errors in dX and dY are mostly uncorrelated. Thistranslates into having the red and green ellipses of the bivariate PDF aligned with the carte-sian axes of the graphs. The observation error covariance matrix Cobs can most probably beconsidered a diagonal matrix as a first approach. Note however that all necessary informa-tion is provided in this report to use a non-diagonal Cobs.

Most assimilation techniques imply (or are used in) a Gaussian model for the error distri-bution. The close match between the solid and dashed red and green curves on Figure (3)and Figure (4) are a qualitative assessment that the statistical error distribution is not farfrom a perfect bivariate Gaussian model. A quantitative assessment would require comput-ing bivariate tail and Kurtosis statistics which was not performed in this report.

The analysis conducted so far indicates that the error distribution (when spatially andtemporally averaged) can be quite safely approximated by a 0 mean, uncorrelated, bivari-ate Gaussian probability model. Only the standard deviations σ(dX) and σ(dY ) are to beadapted when choosing from the set of single- and multi- sensor ice drift products.

Indeed, when it comes to ranking the products, one of them seems to compare muchbetter with the validation dataset. The sea ice drift product retrieved from AMSR-E (37GHzchannels) presents, by far, the smallest values for both σ(dX) and σ(dY ). This limited rangeof errors also translates in the high correlation coefficient (ρ = 0.97) and good regressionline for this product (right column, first row in Figure (3)). This can also be visualized bylooking at the vector field itself which, most of the times, looks less noisy than the ones fromother instruments.

This higher quality might be explained by several factors, including the smaller foot-print/spacing of the two 37 GHz channels on board AMSR-E (see the PUM) and the bettertemporal stability of their intensity patterns (compared to, e.g., those at 85GHz on SSM/I).In any case, the ice drift product from AMSR-E allows statistical standard deviations of 2.70

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 21: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 16

km (2.77 km) and is the product comparing best to the reference dataset. The main draw-back of the AMSR-E product, however, is the average stability of the Aqua satellite platformwhich causes quite frequent delays or interruptions in the reception of input swath data at theOSI SAF HL processing centre. As a consequence, it is not rare that the grid is incompletelyfilled or that the product is missing for one or more days. The loss of AMSR-E instrumentin October 2011 halts the processing and distribution of the AMSR-E single-sensor productuntil its follow-up mission flied on board GCOM-W, scheduled for late 2012.

Ice drift datasets from other sources (SSM/I and ASCAT) have approximatively all equiv-alent quality, with statistical standard deviations in the range 4.0 – 4.5 km.

4.1.2 Comparison to other datasets

Kwok et al. (1998), for example, report standard deviations of 8.9 km (10.8) and 9.9 (11.2)for SSM/I1 85 GHz V. pol. dX (dY ) and H. pol. dX (dY ) products, respectively. Thisis for a 3 days product in the central Arctic. For their 1 day dataset in the Fram Straitsand Baffin Bay, those values are 5.3 km (4.3) and 6.0 (4.7) respectively. Those values areextracted from Table 2, p. 8196. To be fair, one should mention that the validation exercisein Kwok et al. (1998) was performed against IABP buoys and using a 2D-type collocation(see our section 3.3.1). IABP buoys are mainly tracked with Argos positioning, which areless accurate. Those statistics would thus better be compared with those in our Table (2).This being said, Kwok et al. do not state clearly either that they provide the accurate t0 andt1 time information which are needed for using their product in a 3D collocation strategy.

Ezraty et al. (2007) propose a theoretical derivation of the variance induced by thepixel length in the Maximum Cross Correlation technique. They deduce the value of δ2/6for the variance in dX and dY , where δ is the pixel’s length. Although the OSI SAF icedrift products are not derived using the MCC (see PUM), it is comforting to note that theequivalent standard deviation for the 12.5 km resolution pixels we are processing is 5.1 km.This is only a theoretical estimate which does not include other uncertainty sources such asany atmospheric contamination, satellite geolocation errors or non accuracy of the start andend time of the drift vectors. It is even more so satisfactory to document standard deviationsfor the OSI SAF products below this theoretical value.

Error statistics reported for the various IFREMER datasets (QuikSCAT-SSM/I mergedand AMSR-E 89 Ghz) as well as by Haarpaintner (2006) are not obviously compared withour values as they are computed for the North-South and East-West components of thedrift vectors. Those components exhibit non linear, latitude dependent relationships to thedX and dY we are validating. Note, however, that only the AMSR-E (89 GHz) from IFRE-MER and the QuikSCAT product of Haarpaintner (2006) have a time span of 2 days like theOSI SAF product. The merged SSM/I and QuikSCAT dataset delivered by IFREMER is a 3days ice drift product.

An intercomparison of the IFREMER/Cersat and OSI-405 products was conducted anddocumented in (Hwang and Lavergne 2010b), which addressed to some extent the caveatsof intercomparing drift products with varying time spans.

4.1.3 Discussion and conclusion

The validation statistics for all the OSI SAF low resolution ice drift products over NH areaare summarized in Table (1) (3D collocation) and Table (2) (2D collocation).

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 22: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 17

Product 〈ε(dX)〉 〈ε(dY )〉 σ(dX) σ(dY ) ρ(dX,dY ) α β ρ(R,P )

amsr-aqua -0.10 -0.10 2.70 2.77 -0.03 0.95 -0.02 0.97ssmi-f15 -0.07 -0.02 4.10 4.01 +0.02 0.93 +0.08 0.94ascat-metopA -0.17 -0.24 4.71 4.47 +0.01 0.92 -0.05 0.92multi-oi -0.12 -0.27 3.65 4.05 +0.04 0.89 +0.04 0.95

Table 1: Statistical results for validation of the ice drift product in Northern Hemi-sphere for period October, 1st to April, 30th in 2006-2007, 2007-2008 and2008-2009. Those results pertain to the 3D collocation. In the 6th column,ρ(dX,dY ) is a shortened notation of ρ(ε(dX), ε(dY )).

Product 〈ε(dX)〉 〈ε(dY )〉 σ(dX) σ(dY ) ρ(dX,dY ) α β ρ(R,P )

amsr-aqua -0.09 -0.12 3.11 3.05 -0.06 0.95 -0.02 0.96ssmi-f15 -0.07 -0.01 4.41 4.24 +0.02 0.92 +0.09 0.93ascat-metopA -0.20 -0.23 4.88 4.62 +0.01 0.92 -0.05 0.92multi-oi -0.12 -0.27 3.65 4.05 +0.04 0.89 +0.04 0.95

Table 2: Statistical results for validation of the ice drift product in Northern Hemi-sphere for period October, 1st to April, 30th in 2006-2007, 2007-2008 and2008-2009. Those results pertain to the 2D collocation. In the 6th column,ρ(dX,dY ) is a shortened notation of ρ(ε(dX), ε(dY )).

On top of the analysis conducted so far from Figure (3) and Figure (4), the most inter-esting comparison between those two tables is the slight degradation of the statistics fromthe 3D to the 2D collocation. For example, the AMSR-E standard deviations grow from 2.70(2.77) km to 3.11 (3.05) km. Neglecting the information on t0 and t1 thus led to enlargingthe uncertainties in each drift component by roughly 350 meters. The other products showa similar (although more limited) pattern. The multi-sensor product exhibits no sensitivityto the use (or not) of the extra temporal information. This is not surprising as the mergingprocedure implemented in the OSI SAF chain does not allow for book-keeping the time infor-mation of each vector (see PUM). As a result are the statistics for the 2D and 3D collocationsidentical.

This enlargement of the error statistics is, however, dampered by the high level of aver-aging occuring in our validation exercise, spatially and temporally. On a single case basis,like when a circular motion pattern is induced by a moving atmopsheric low pressure, thet0 and t1 are quite significant and should not be neglected, as illustrated in Lavergne et al.2008.

Finally, it should be noted that the statistical results and particularly the standard devi-ations and bias are quite depending on the collocation methodology and on the choice ofreference data to enter the validation database. Indeed, the standard deviation for the amsr-aqua product (3D collocation) over the same period is only 2.56 (2.59) km when the Argostrajectories from the buoy array (section 2.1.4) are not included.

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 23: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 18

4.1.4 Monthly validation results

An analysis of monthly time series of validation results was performed and are reported inLavergne et al. 2010.

As expected, monthly validation results show better validation statistics in the core ofwinter (Dec-Jan-Feb) than at the edges of the summer period. This somehow confirmsour a-priori feeling that the product should be more accurate when the sea ice emissivitypatterns are well pronounced and stable in time (core of winter).

The simultaneous temporal variation of the representativity error challenges howeverthis conclusion. Indeed, since the ice in the Arctic Ocean is more packed and rigid duringthese very same winter months, we expect that the representativity error that is inducedby comparing an area averaged drift (satellite product) against that of a single floe (driftingbuoy) is reduced as well.

A thorough discussion of these aspects is in Lavergne et al. 2010, section 4.4.

4.2 Validation over the SH area

Due to the lack of buoy trajectories over the SH sea ice (section 2.3), two separate sectionsare dedicated to validation in the Souther Hemisphere. First, we present validation againsttwo buoys in (austral) winter 2011. Second, we document a comparison obtained by col-location of the OSI SAF product against a high-resolution SAR-based sea ice drift productover winters 2010 and 2011.

4.2.1 Validation against SAMS Icebell buoy

Figure (5) present validation results obtained when collocating the OSI SAF product withthe buoy trajectories of Icebell 10 and Icebell 18 from SAMS. The validation period israther short since it spans from mid April to late August 2011.

Analysis of the statistics embedded in these graphs is necessarily limited by the amountof validation data. The total number of matchups ranges between 20 to 30 depending on thesensor and collocation method. This is to be compared to the NH case with its almost 5000matchups (section 4.1).

These SH validation graphs are thus only used to confirm that the ice motion trackingmethod seemlingly works over SH sea ice as in the Arctic. Users might want to use slightlylarger uncertainty values in SH than in NH, but the lack of buoy validation data does notallow to conclude on a quantitative estimate.

4.2.2 Comparison against DTU SAR-based ice drift product

As mentionned in section 2.2, accurate sea ice drift information is available via the MyOceanproject from Envisat ASAR WSM images. The product is processed and validated at DTUagainst ITPs, in the Arctic only. It exibits limited bias and standard devitations of the drifterrors of about 300 to 600 m. Although not negligible, these uncertainy are less than thosedocumented for the OSI SAF low resolution product we want to validate (see previous sec-tion).

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 24: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]

Error in drift along the processing grid (dX) [km]

SH Validation:

20110401-20110901

N = 21

<ε> = (-0.82 , -2.18) [km]

σ = (3.39 , 5.00) [km]

ρ = -0.22

amsr-aqua

NN3D

ε = x|yprod - x|yref

amsr-aqua CMCC

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]

Error in drift along the processing grid (dX) [km]

SH Validation:

20110401-20110901

N = 29

<ε> = (-0.48 , -0.18) [km]

σ = (4.48 , 5.24) [km]

ρ = -0.36

ssmi-f15

NN3D

ε = x|yprod - x|yref

ssmi-f15 CMCC

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]

Error in drift along the processing grid (dX) [km]

SH Validation:

20110401-20110901

N = 19

<ε> = (-0.53 , -0.08) [km]

σ = (3.32 , 5.02) [km]

ρ = -0.19

ascat-metopA

NN3D

ε = x|yprod - x|yref

ascat-metopA CMCC

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]

Error in drift along the processing grid (dX) [km]

SH Validation:

20110401-20110901

N = 25

<ε> = (-0.20 , -1.09) [km]

σ = (2.82 , 4.17) [km]

ρ = -0.18

multi-oi

NN3D

ε = x|yprod - x|yref

multi-oi CMCC

Figure 5: Selected validation graphs for AMSR-E (top-left), SSM/I ’F15’ (top-right),ASCAT (bottom-left) and merged (multi-oi) (bottom-right) products. Allpertain to SH area, to the 3D collocation setup and period from April 1st

to September 1st 2011. N is the number of validation pairs.

Page 25: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 20

It is thus of interest to try the comparison of the OSI SAF against the SAR-based andthus extend the limited validation against buoys we introduced in section 4.2.1. Severallimitations to this approach should however be kept in mind.

First, the SAR-based ice drift product from DTU is not validated over Antarctic sea ice.We can assume that the SAR-based product quality is not degraded over Antarctic sea ice,but this does not allow us to refer to documented validation statistics as over the ArcticOcean.

Second, the validation of DTU SAR product against ITPs (in the Arctic) shows non-negligible uncertainty amounting for about 1/4th to 1/3rd of those obtained with the OSI SAFproduct. Taking these into account is however not trivial since a possibly large part of theerror budget of each satellite product is strongly correlated with each other. Indeed, andfollowing the discussion from Hwang and Lavergne (2010a), the representativity error madewhen comparing a point measurement (buoy) against the average drift of a 5 km radius disc(DTU product) is not independent from reprensentativity error to bridge the gap between thebuoy and the averaging scales of the OSI SAF (60 km radius).

Finally, Antarctic sea ice is generally found at lower latitudes than in the Northern Hemi-sphere. Orbital parameters and fewer ordering of Envisat SAR scenes in the SouthernHemisphere induce that there is much less SAR-drift coverage in SH than in NH. Anotherconsequence is that the temporal sampling (start time and duration) are less varying than inthe Arctic Ocean, leading to less matchups falling into the collocation criteria.

For all the reasons above, the results we present in this section are to be considered withcare, and are more documenting a comparison than a proper validation. This is nonethelessa usefull exercise to gain confidence in the accuracy of the product over Antarctic sea ice.

The first impression from Figure (6) is that both the OSI SAF and the DTU productsmostly agree about the magnitude and direction of the drift vectors. This is quite confortingand encouraging although it does not constitute an absolute proof.

Statistics appearing on Figure (6) differ from those presented above for NH area in thatthe spread around the 1:1 line is generally larger. This translates into slightly lower linearcorrelation coefficients (ρ = 0.90 to ρ = 0.95 while the NH values were generally above 0.95).

The spread in σ(dX) and σ(dY ) is also larger in this comparison against DTU SARproduct, with values around 5 km for all instruments and collocation setup (3D and 2D).Over the SH area, we do not observe a better match for one of the instrument wrt to theothers. The merged dataset (multi-oi) exhibit slightly better statistics.

We note that the biases 〈ε(dX)〉 and 〈ε(dY )〉 are larger than in the NH against buoys,but still an order of magnitude less than the standard deviations. The underestimation of theOSI SAF product with respect to the reference dataset also translates into larger deviationfrom the 1:1 line.

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 26: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]

Error in drift along the processing grid (dX) [km]

N = 2580

<ε> = (-0.15 , -0.49) [km]

σ = (4.70 , 4.56) [km]

ρ = 0.02

ascat-metopA

NN3D

CMCC

ε = x|yprod - x|yref

SH Validation:

20100401-20101101 + 20110401-20111101

pdf regions

68%95%

-40

-20

0

20

40

-40 -20 0 20 40

CM

CC

drift

(d

X a

nd

dY

) a

lon

g t

he

pro

ce

ssin

g g

rid

[km

]

Reference drift (dX and dY) along the processing grid [km]

SH Validation:

20100401-20101101 + 20110401-20111101

N = 5160

α = 0.93

β = -0.08

ρ = 0.92

ascat-metopA

NN3D

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]

Error in drift along the processing grid (dX) [km]

N = 2886

<ε> = (-0.27 , -0.58) [km]

σ = (5.05 , 4.70) [km]

ρ = 0.14

ssmi-f15

NN3D

CMCC

ε = x|yprod - x|yref

SH Validation:

20100401-20101101 + 20110401-20111101

pdf regions

68%95%

-40

-20

0

20

40

-40 -20 0 20 40

CM

CC

drift

(d

X a

nd

dY

) a

lon

g t

he

pro

ce

ssin

g g

rid

[km

]

Reference drift (dX and dY) along the processing grid [km]

SH Validation:

20100401-20101101 + 20110401-20111101

N = 5772

α = 0.91

β = -0.12

ρ = 0.91

ssmi-f15

NN3D

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]

Error in drift along the processing grid (dX) [km]

N = 3017

<ε> = (-0.70 , -0.47) [km]

σ = (4.52 , 4.23) [km]

ρ = 0.04

multi-oi

NN3D

CMCC

ε = x|yprod - x|yref

SH Validation:

20100401-20101101 + 20110401-20111101

pdf regions

68%95%

-40

-20

0

20

40

-40 -20 0 20 40

CM

CC

drift

(d

X a

nd

dY

) a

lon

g t

he

pro

ce

ssin

g g

rid

[km

]

Reference drift (dX and dY) along the processing grid [km]

SH Validation:

20100401-20101101 + 20110401-20111101

N = 6034

α = 0.91

β = -0.16

ρ = 0.93

multi-oi

NN3D

Figure 6: Selected validation graphs for ASCAT (top row), SSM/I ’F15’ (middle row)and merged (multi-oi) (bottom row) products. All pertain to SH area, tothe 3D collocation setup and period from April 1st to September 1st 2010and 2011. Left (right) column presents ”error(dY ) vs error(dX)” (”productvs reference”) types of graphs. N is the number of validation pairs.

Page 27: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 22

5. Conclusion

This report deals with the validation of the OSI SAF low-resolution sea ice drift product frompassive and active microwave satellite data (OSI-405). Three sets of validation exercisesare presented:

1. Validation in the Northern Hemisphere (Arctic Ocean) over 3 years (2006-2007, 2007-2008and 2008-2009: from October, 1st to April, 30th);

2. Validation in the Southern Hemisphere (Ross Sea) in (austral) winter 2011 against apair of GPS buoys;

3. Comparison in the Southern Hemisphere in (austral) winters 2010 and 2011 against ahighly accurate SAR-based ice drift product available from DTU.

After having presented the data that constitute our reference dataset (mainly GPS in-situtrajectories) we introduced the validation strategy and, particularly, two ways of collocatingthe datasets: 2D which considers uniform start and end time at 1200 UTC for all vectors and3D which makes use of the extra time information available in the product file.

Results are analyzed from various graphs and statistics and conclude that the OSI SAFice drift parameters dX and dY are mostly unbiased and that their statistical error probabilitydistributions can be well represented by uncorrelated bivariate normal PDFs. Quantitativeestimates of the terms entering Cobs, the error covariance matrix are provided in separatetables for the 2D and 3D collocation/usage for the Northern Hemisphere grid.

The tables confirm that the AMSR-E instrument gives the best ice drift vectors in theArctic Ocean, with standard deviations (on dX and dY ) only slightly less than 3 km. Thedatasets from other sensors (SSM/I and ASCAT) have uncertainties level of around 4 – 4.5km. SSM/I ’F15’ seems to provide better results than the other DMSP platforms, F13 andF14 (not shown). Bias levels are very low in comparison.

As expected, the uncertainty level is slightly raised when the pixel varying time informa-tion is ignored (2D collocation). However, the standard deviations are only enhanced by fewhundred meters.

In the Southern Hemisphere, the lack of GPS buoy trajectories prevent us from conclud-ing on quantitative estimates for the uncertainties. Although limited in scope and hazardousto compare against NH results, we thus introduce results from a comparison of the OSI SAFproduct against the SAR-based product from DTU (which itself is not validated in the SH).The obtained statistics show an overal and encouraging agreement between the two prod-ucts, but also larger biases and spread.

It is worth mentioning that, in the Northern Hemisphere, the validation statistics arematching the threshold and target accuracy requirements as specified in the Product Re-quirement Document CDOP PRD for OSI-405 product. The threshold (target) accuracy is 10(5) km standard deviation. The optimal accuracy is defined as 2 km. This latter value mighthave been reached by the AMSR-E product during selected periods of time (e.g. during thecore of the winter season) but not on a yearly average validation exercise.

The failure of the AMSR-E instrument in early October 2011 prevents us from usingit, until its follow-up mission is launched on GCOM-W (planned 2012). The loss of this

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 28: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 23

data source undoubtly reduces the accuracy of the merged/multi-sensor distributed sincethat date (not documented in this report). It is however noted that since the single-sensorproducts from ASCAT and SSM/I are independently fulfilling the target accuracy, the mergedproduct using only these two will obviously also fulfil this requirement. This is because themerging algorithm of the OSI SAF uses an optimal (in the sense of inverse of variance)combination of the available single-sensor products.

In the Southern Hemisphere, one can expect that both the threshold and target accuracyrequirements are met on a season averaged basis. The comparison against SAR-basedproducts show statistics of the order of the target accuracy, but the DTU product itself surelyexhibits uncertainties with respect to ground truth (not accessible).

This report is a living document that will be updated when new sensors are introducedin the OSI SAF ice drift production chain as well as when the validation period is extendedor when new data sources are available for inclusion in the reference dataset. The latestversion of the present report and Product User Manuals are always available from the OSISAF Ice web portal: http://osisaf.met.no or by contacting the author.

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 29: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 24

References

CDOP PRD (2009, January). Ocean and Sea Ice SAF CDOP Product Requirement Docu-ment.

Ezraty, R., F. Girard-Ardhuin, and J.-F. Piolle (2007, February). Sea ice drift in the cen-tral Arctic estimated from SeaWinds/QuikSCAT backscatter maps – User’s manual. v2.2,CERSAT, IFREMER, France.

Haarpaintner, J. (2006, January). Arctic-wide operational sea ice drift from enhanced-resolution Quikscat/SeaWinds scatterometry and its validation. IEEE Transactions on Geo-science and Remote Sensing 44(1), 102–107.

Hwang, P. and T. Lavergne (2010a, September). Triple comparison of OSI SAF low reso-lution sea ice drift products – v1.3. Technical Report SAF/OSI/CDOP/met.no/SCI/RP/152,EUMETSAT OSI SAF – Ocean and Sea Ice Sattelite Application Facility.

Hwang, P. and T. Lavergne (2010b, September). Validation and comparison of OSI SAFlow and medium resolution and IFREMER/Cersat sea ice drift products – v1.2. TechnicalReport SAF/OSI/CDOP/met.no/SCI/RP/151, EUMETSAT OSI SAF – Ocean and Sea IceSattelite Application Facility.

Kwok, R., A. Schweiger, D. A. Rothrock, S. Pang, and C. Kottmeier (1998, April). Seaice motion from satellite passive microwave imagery assessed with ERS SAR and buoymotions. Journal of Geophysical Research 103, 8191–8214.

Lavergne, T. and S. Eastwood (2011, October). Low resolution sea ice drift Product User’sManual – v1.5. Technical Report SAF/OSI/CDOP/met.no/TEC/MA/128, EUMETSAT OSISAF – Ocean and Sea Ice Sattelite Application Facility.

Lavergne, T., S. Eastwood, H. Schyberg, and L.-A. Breivik (2008, October). Ice drift mon-itoring from low resolving sensors: an alternative method and its validation against in-situdata. Report MERSEA WP02 METNO STR 005 1A, MERSEA – Marine EnviRonmentand Security for the European Area.

Lavergne, T., S. Eastwood, H. Schyberg, and L.-A. Breivik (2011, April). Algorithm Theoret-ical Basis Document for the OSI SAF low resolution sea ice drift product – v1.0. TechnicalReport SAF/OSI/CDOP/met.no/SCI/MA/130, EUMETSAT OSI SAF – Ocean and Sea IceSattelite Application Facility.

Lavergne, T., S. Eastwood, Z. Teffah, H. Schyberg, and L.-A. Breivik (2010). Sea ice motionfrom low resolution satellite sensors: an alternative method and its validation in the Arctic.Journal of Geophysical Research 115, C10032.

EUMETSAT OSI SAF Version 3 — Jan 2012

Page 30: Validation and Monitoring of the OSI SAF Low Resolution ...osisaf.met.no/docs/osisaf_ss2_valrep_sea-ice-drift-lr_v3p0.pdf · Arctic Ocean in April 2010, while right panel displays

SAF/OSI/CDOP/Met.no/T&V/RP/131 25

Lavergne, T., M. Voßbeck, B. Pinty, T. Kaminski, and R. Giering (2006). Evaluation ofthe two-stream inversion package. EUR 22467 EN, European Commission – DG JointResearch Centre, Institute for Environment and Sustainability.

Wiebe, H., G. Heygster, and L. Meyer-Lerbs (2008, October). Geolocation of AMSR-Edata. IEEE Transactions on Geoscience and Remote Sensing 46, 3098–3103.

EUMETSAT OSI SAF Version 3 — Jan 2012


Recommended