Validation and Monitoring of the OSI SAF Low Resolution...

Ocean & Sea Ice SAF

Validation and Monitoring of the OSI SAFLow Resolution Sea Ice Drift Product

GBL LR SID – OSI-405-b

Figure 1: Example ice drift vector fields as processed at the EUMETSAT OSI SAF.Left panel displays displacement vectors and length (color shades) in theArctic Ocean in April 2010, while right panel displays vectors and dis-placement length in the Ross Sea in August 2010. Both examples arefrom the multi-sensor merged product.

Version 4 — March 2015Thomas Lavergne

Documentation Change Record:

Version Date Author Descriptionv0.9 03.12.2008 TL Initial version, before reviewv1 05.02.2009 TL Amended after PCR commentsv2 15.03.2010 TL Include more in-situ data sources,

change the (spatial) collocationmethod and extend validation timeperiod.

v3 25.01.2012 TL Include more in-situ data sources, in-troduce SAR ice drift as ground truthover SH sea ice, change (temporal)collocation method.

v4 15.03.2015 TL Include more in-situ data sources.Validation of SSMIS (F17) andAMSR2 (GCOM-W1) products. Noupdate of SH validation (not enoughvalidation data).

SAF/OSI/CDOP/Met.no/T&V/RP/131

Table of contents

Table of contents

List of Figures

1 Introduction 11.1 The EUMETSAT Ocean and Sea Ice SAF . . . . . . . . . . . . . . . . . . . . 11.2 Scope and Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Validation Excercise Version History . . . . . . . . . . . . . . . . . . . . . . . 21.4 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Validation dataset 52.1 In-situ trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 High-resolution SAR-based sea ice drift product . . . . . . . . . . . . . . . . 62.3 Remarks on the validation dataset . . . . . . . . . . . . . . . . . . . . . . . . 72.4 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.5 Spatial and Temporal overview of the validation dataset . . . . . . . . . . . . 7

3 Validation methodology 103.1 Variables to be validated . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.2 Reformatting of the validation dataset . . . . . . . . . . . . . . . . . . . . . . 103.3 Collocation strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.4 Representativity error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.5 Graphs and statistical measures . . . . . . . . . . . . . . . . . . . . . . . . . 13

4 Results of validation 144.1 Validation over the NH area . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.2 Validation over the SH area . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

5 Conclusion 27

References 29

EUMETSAT OSI SAF Version 4 — March 2015

SAF/OSI/CDOP/Met.no/T&V/RP/131

List of Figures

1 Examples of NH and SH ice drift vector fields . . . . . . . . . . . . . . . . . . 1

2 Trajectories of NH validation drifters (buoy type) . . . . . . . . . . . . . . . . . 83 Trajectories of NH validation drifters (timestamp) . . . . . . . . . . . . . . . . 9

4 NH validation graphs for ice drift products . . . . . . . . . . . . . . . . . . . . 155 NH validation graphs for ice drift products — continued . . . . . . . . . . . . . 166 NH map of mismatch between selected products and validation data . . . . . 197 Example validation map of multi-oi product in Beaufort Sea . . . . . . . . . . 208 SH validation graphs against SAMS Icebell buoys . . . . . . . . . . . . . . . . 249 SH validation graphs against DTU SAR product . . . . . . . . . . . . . . . . . 26


SAF/OSI/CDOP/Met.no/T&V/RP/131 1

1. Introduction

1.1 The EUMETSAT Ocean and Sea Ice SAF

For complementing its Central Facilities capability in Darmstadt and taking more benefit fromspecialized expertise in Member States, EUMETSAT created Satellite Application Facilities(SAFs), based on co-operation between several institutes and hosted by a National Meteo-rological Service. More on SAFs can be read from www.eumetsat.int.

The Ocean and Sea Ice Satellite Application Facility (OSI SAF) is producing on an op-erational basis a range of air-sea interface products, namely: wind, sea ice characteristics,Sea Surface Temperatures (SST), Surface Solar Irradiance (SSI) and Downward LongwaveIrradiance (DLI). The sea ice products include sea ice concentration, the sea ice emissivity,sea ice edge, sea ice type and sea ice drift and sea ice surface temperature (from mid 2013).

The OSI SAF consortium is hosted by Meteo-France. The sea ice processing is per-formed at the High Latitude processing facility (HL centre), operated jointly by the Norwegianand Danish Meteorological Institutes.

Note: The ownership and copyrights of the data set belong to EUMETSAT. The datais distributed freely, but EUMETSAT must be acknowledged when using the data. EUMET-SAT’s copyright credit must be shown by displaying the words ”copyright (year) EUMETSAT”on each of the products used. User feedback to the OSI SAF project team is highly valued.The comments we get from our users is important argumentation when defining develop-ment activities and updates. We welcome anyone to use the data and provide feedback.

1.2 Scope and Overview

Sea ice drift products for Northern and Southern Hemisphere are processed at the HighLatitude center of the Ocean & Sea Ice Satellite Application Facility (EUMETSAT OSI SAF).Those datasets are introduced and documented in a dedicated Product User’s Manual(PUM, Lavergne and Eastwood 2015) and in an Algorithm Theoretical Basis Document(ATBD, Lavergne 2015) that can both be found on http://osisaf.met.no.

See http://osisaf.met.no for real time examples of the products as well as updated infor-mation. The latest version of this document can also be found there. General informationabout the OSI SAF is given at http://www.osi-saf.org.

This Validation and Monitoring report only deals with the OSI-405 low resolutionsea ice drift product. The medium resolution ice drift product based on AVHRR

imagery, OSI-407, is documented in a dedicated report.

The aims of this report are several:

1. To document the level of agreement between the OSI SAF low resolution sea ice driftproduct and ground-based truth. Various graphs displaying the match between the


www.eumetsat.int

http://osisaf.met.no


http://www.osi-saf.org


satellite product and the in-situ datasets should give (qualitative) confidence in usingthe product.

2. To report quantitative estimates of errors and uncertainties in the product. Particularly,the bias and uncertainty covariance matrix is computed. It is important that eachsingle-sensor and the multi-sensor products are validated separately so that users canhave error estimates for the product they choose to use.

3. To match these quantitative estimates with the accuracy thresholds defined in theProduct Requirements Document (PRD, CDOP2 PRD 2015).

Chapter 2 presents the datasets used as validation data while chapter 3 documentsthe validation strategy and, particularly, the way collocation is handled. Chapter 4 providesdetailed, graphical and quantitative analysis of the validation results. We conclude in chap-ter 5.

Note that the OSI SAF low resolution sea ice drift product will not be introduced at anydepth in this report. Refer to the PUM ATBD, and http://osisaf.met.no for information on thealgorithms, processing schemes and data format.

Let us nonetheless remind that the OSI SAF low resolution ice drift product comes asdaily vector fields obtained by processing low-resolution satellite signal from, among others,AMSR2, SSMIS and ASCAT. It is computed on a Northern Hemisphere grid, delivered fromOctober, 1st to April, 30th every year, and on a Southern Hemisphere grid, available from1st April to 31st October. It is a 2 days (48 hours) ice drift product on a 62.5 km resolutionpolar stereographic grid. Both single and multi sensor products are distributed.

1.3 Validation Excercise Version History

This report is updated regularly when new validation results need to be documented, par-ticularly when the OSI SAF low resolution ice drift product undergoes reviews in view ofoperational upgrades. This section aims at shortly introducing the rational for each newversion.

1.3.1 Version 4 – Spring 2015

Version 4 of this validation exercise is prompted by the need to document the accuracyobtained when processing ice drift vectors from JAXA’s AMSR2 on-board GCOM-W1. Thereis no change to the processing algorithm. At the same time, the opportunity is taken to alsoupdate validation results for SSMIS (F17) and ASCAT (Metop-A).

1.3.2 Version 3 – Winter 2012

Version 3 of the validation exercise is prompted by the need to document the accuracy ofthe product in the Southern Hemisphere (SH), in view of starting operational distribution ofSH drift product. Due to the insufficient number of buoys available, this is mainly achievedby an comparison to high-accuracy ice motion vectors processed from Synthetic AppertureRadar images available from DTU/PolarView/MyOcean.




1.3.3 Version 2 – Spring 2010

Version 2 of the validation exercise extends previous version by introducing more buoy data.Only the Northern Hemisphere (NH) product is available (and validated here).

1.3.4 Version 1 – Fall 2009

Version 1 of the validation exercise takes place during the Product Consolidation Review,where the potential accuracy of the ice motion algorithms is assessed.



1.4 Glossary

AARI Arctic and Antarctic Research InstituteASCAT Advanced SCATterometerAVHRR Advanced Very High Resolution RadiometerAMSR2 Advanced Microwave Scanning Radiometer - 2AMSR-E Advanced Microwave Scanning Radiometer for EOSAWI Alfred Wegener InstituteCDOP Continuous Development and Operations PhaseCF Climate and ForecastCRREL Cold Regions Research & Engineering Laboratory (US Army)DMI Danish Meteorological InstituteDMSP Defense Meteorological Satellite ProgramDTU Danish Technical UniversityIABP International Arctic Buoy ProgramITP Ice Tethered ProfilerIMB Ice Mass BalanceGCOM-W Global Change Observation Mission for Water (”Shizuku”)GPS Global Positioning SystemHL High LatitudesJAXA Japan Aerospace Exploration AgencyMET Norway Norwegian Meteorological InstituteNetCDF network Common Data FormatNH Northern HemisphereNP North PoleSAF Satellite Application FacilitySAR Synthetic Apperture RadarSAMS Scottish Association for Marine ResearchSH Sourthern HemisphereSIMBA Sea-Ice Mass BAlanceSSM/I Special Sensor Microwave/ImagerSSMIS Special Sensor Microwave Imager SounderTb Brightness TemperatureTOA Top Of AtmosphereWSM Wide Swath Mode



2. Validation dataset

In this section, we introduce the ice motion datasets that constitute our best estimate of theground truth and that is used as reference to validate the OSI SAF low resolution sea icedrift product.

Several data sources are available for validating an ice drift product and they can besorted into three groups:

1. Trajectories of in-situ ice drifters. Historically, this is the main validation data source.A fair number of buoys are indeed deployed in the ice covered ocean to measureatmospheric, cryospheric or oceanic variables (e.g. Mean Sea Level Pressure, icethickness or temperature and salinity profiles of the ocean). Of interest to us is the factthat they regularly and automatically report their position via the Argos system or bytransmitting GPS positions as part as their data stream. Drifting ships (like the Tara)or manned stations (NP-35, NP-36, etc...) also constitute good opportunities to get icetrajectory data, sometimes in near-real-time.

2. High resolution satellite based ice drift datasets. Those are processed from highresolution satellite images (e.g. ENVISAT SAR or AVHRR). Those products are not’ground truth’ but are assumed to present much less deviations to truth than this lowresolution ice drift datasets.

3. Moored Doppler-based velocity measures from under the ice. This source of datapresents three major disavantages. Firstly, they are Eulerian measures of instanta-neous velocity, a quantity that is not directly comparable to satellite-based ice dis-placement vectors. Second, they do not transmit data in near-real-time and are thusnot suitable for daily monitoring of a product. Finally, they are often located in coastalareas where the retrieval of sea ice drift from low resolution sensors is challenged bythe proximity to land.

For the validation exercise reported in this document, both in-situ trajectories and a high-accuracy SAR-based ice drift dataset are used.

2.1 In-situ trajectories

2.1.1 Ice Tethered Profilers

The Ice Tethered Profilers (ITP) platforms are advanced autonomous drifting instrument thatare designed to measure temperature and salinity profiles in the ocean under sea ice. Aspart of its daily data stream, each ITP transfers hourly unfiltered GPS locations. As of early2015, there are about 9 active (plus additional 80 completed) ITPs in the Arctic Ocean thatform a high quality validation dataset, especially for the Beaufort Sea, Canadian Basin andFram Strait.

It is noteworthy that the primary objective of ITPs is to sample the water column and,thus, require to be deployed in location with enough water depth. This excludes all shelfarea like Laptev or Chuckchi Sea and limits the spatial sampling of the Arctic Ocean.



The ASCII formatted level 1 raw data position files (itpNrawlocs.dat) for all ITPs weredownloaded from the FTP server at Woods Hole Oceanographic Institution and processedto extract ice drift vectors.

2.1.2 Russian manned polar stations

GPS trajectory logs for the Russian manned stations NP-35 to NP-40 were made availableby the Arctic and Antarctic Research Institute (AARI, http://www.aari.nw.ru). The drift startand end dates are in Table (1).

Station Begin End Duration [days]NP-35 2007.10.27 2008.07.13 260.54NP-36 2008-09-07 2009-08-24 350.42NP-37 2009.09.07 2010.05.31 266.25NP-38 2010-10-14 2011-09-20 341.00NP-39 2011-10-02 2012-09-15 348.88NP-40 2012-10-01 2013-06-07 248.25

Table 1: Drift trajectories of AARI’s North Pole stations.

2.1.3 SAMS SIMBA buoys

The Scottish Association for Marine Science (SAMS) designs robust thermistor-based Sea-Ice Mass Balance (SIMBA) buoys for monitoring the snow and sea ice thickness. Part of thedata stream sent via Iridium are the sub-hourly GPS locations of the platforms. In recentyears, SIMBA buoys were deployed by several teams, in various research projects suchas EU-ACCESS, etc... We access the unfiltered Iridium messages direcly from the SAMSservers, and apply our own quality control.

2.1.4 CRREL IMB buoys

The Cold Regions Research & Engineering Laboratory (CRREL) of the US Army also de-signs thermistor-based Ice Mass Balance (IMB). The earliest versions were already de-ployed in 1993. As of early 2015, there are 6 active platforms, and 97 archived trajectories,available from http://imb.erdc.dren.mil/.

2.2 High-resolution SAR-based sea ice drift product

High-resolution (10 km), 24 hours sea ice motion vectors are processed in near-real-timeat the Danish Technical University (DTU, www.seaice.dk). They are obtained by a patternmatching method applied on Wide Swath Mode (WSM) images of the Envisat ASAR instru-ment, Radarsat-2, and recently Copernicus Sentinel-1. These images are first re-sampledat a 300 m resolution, then processed through a Maximum Cross-Correlation algorithm, andfinally distributed on an FTP server as ASCII files.

As part of the EU MyOcean project, these ice motion vectors are also made available innetCDF format. Still in the frame of this project, a validation is conducted at DTU (against


http://www.aari.nw.ru

http://imb.erdc.dren.mil/

www.seaice.dk


the Ice Tethered Profilers trajectories) that document a very high accuracy of the WSM driftvectors. Some results can be accessed from http://www.seaice.dk/MyOcean/validation.

The error statistics are off max 0.7 km standard deviation of error in X/Y components ofa 24 h drift, and some tens of meters in bias. The high accuracy of these vectors make itpossible to use them as comparison data when validating the OSI SAF low resolution seaice drift dataset.

2.3 Remarks on the validation dataset

Although we tried to use as many good quality drifters as possible, entire regions coveredby the OSI SAF ice drift product grid are not sampled by our validation dataset. See, forreference, Figure (2).

Two such regions are the Baffin and Hudson Bay for which it was not possible to obtaintrajectories in the validation period we have been covering. Some buoys are released everyyear in the Nares Strait. However, most of them sink due to unstable ice conditions in thestrait or Baffin Bay at that period.

A major validation data gap is also over the Antarctic sea ice during winter (April toOctober). The only available GPS trajectories we could access are some SIMBA buoys (seesection 2.1.3). Other short-lived installations exist in earlier years, but they are often not inthe appropriate time period, or not easily accessible to us.

2.4 Acknowledgements

The Ice-Tethered Profiler data were collected and made available by the Ice-Tethered Pro-filer Program based at the Woods Hole Oceanographic Institution http://www.whoi.edu/itp.

GPS-located data from Russsian stations were kindly provided by the Arctic and Antarc-tic Research Institute (AARI, http://www.aari.ru) of Roshydromet, PIs Vladimir Sokolov andVasily Smolyanitsky.

The trajectories for SIMBA buoys are provided by Phil Hwang from the Scottish Associationfor Maring Science (SAMS).

The trajectories of CRREL IMBs are from Perovich, Richter-Menge, Elder, Arbetter, Claffey,and Polashenski 2014

The archive of global SAR-based product in Ascii format was made available by RobertoSaldo (DTU)

2.5 Spatial and Temporal overview of the validation dataset

Figure (2) displays a graphical overview of the in-situ trajectories that were used in thevalidation period for NH area. All the color-coded position records are obtained with GPSaccuracy. It is not seldom that, e.g., both an ITP and a CRREL buoys are deployed by the


http://www.seaice.dk/MyOcean/validation

http://www.whoi.edu/itp

http://www.aari.ru


Figure 2: Trajectories of validation drifters between 1st January 2013 and 31st De-cember 2014. The colour codes the buoy programme or data provider.

same crew, on the same ice flow. This is the reason why some trajectories in Figure (2)switch color. But note that we do not mix positions of an ITP and CRREL buoys whenextracting the validation drift vector.

Figure (3) displays the same positions as in Figure (2), but the color now varies with thetime of the position record from 1st January 2013 (dark blue) to 31st December 2014 (darkred). Figure (3) helps visualizing the direction of the drift, but also the time of the year wheneach area was mostly validated. The discontinuities in the colors correspond to the summerperiod (May 1st to September 30th) when the product is not available (PUM). Thus, dark blueis for Jan–Apr 2013, cyan to yellow for Oct 2013 to Apr 2014, and red for Oct–Dec 2014.



Figure 3: Trajectories of validation drifters between 1st January 2013 and 31st De-cember 2014. The colour codes the timestamp of the position record.



3. Validation methodology

The validation strategy is introduced in this chapter. It covers the re-formatting of the tra-jectories and the collocation with the sea ice drift product. We also present the graphs andstatistical metrics that will be displayed and commented upon in chapter 4.

3.1 Variables to be validated

As introduced in the sea ice drift PUM, an ice drift vector is fully described with 6 values:the geographical position of the start point (lat0 and lon0), the start time of the drift (t0), theposition of the end point of the drift (lat1 and lon1) as well as the end time of the drift (t1).

However, the primary variables the ice drift processing software optimizes are dX anddY , the components of the displacement vector along the X and Y axes of the product grid.Those are thus the two variables we are aiming at validating.

3.2 Reformatting of the validation dataset

3.2.1 Trajectories of in-situ drifters

In any validation exercise, especially if making use of a broad range of data sources, one isconfronted to new and varying formats. Most of the times, trajectories from in-situ drifters areavailable in an ASCII format, proposing one position and time stamp per line. The variousformats for the time information, in particular, as well as the ordering of columns make itchallenging to design a unique software package to read all those files.

A first step of this validation effort has thus been the design of dedicated software rou-tines to read the observation files, extract the portion of the trajectory that fits the time spanof the OSI SAF ice drift product file and dump the validation data in a NetCDF formatted file.

3.2.2 Pre-processing of the DTU SAR-based product

The WSM DTU SAR product is available as 24 h drift vectors. The exact start and stoptime however varies over the basin due to the orbit configuration of Envisat and revisit timeof the platform. As a result, the duration of the DTU SAR product is anywhere between 7and 47 hours (Roberto Saldo, personnal communication). Since the OSI SAF is a 48 h driftproduct, it cannot directly be compared with the SAR-based product.

It is noteworthy that the SAR images used as end image for a drift vector are re-usedas start image for a subsequent drift period. In order to extract 48 h drift vectors, we thusfirst concatenate over time any 2 (sometimes 3) SAR drift scenes. The concatenated vectorshave, on average, a duration of 48 h, and quite some variation in the individual start and stoptime. These concatenated vectors, stored in the same NetCDF format as the trajectories ofin-situ drifters, are those entering the collocation routines described below.



3.3 Collocation strategies

In order to compare the OSI SAF sea ice drift product with the validation trajectories, theyneed to be collocated one with the other. Collocation is the act of selecting or transformingone or both datasets so that they represent the same quantity, at the same time and at thesame geographical location.

Because the OSI SAF ice drift product comes with two flavours of time information (referto PUM, section Time information), two validation exercises are conducted:

• One using a 2D collocation, in which the satellite product is considered representing adrift from day D@1200 UTC to (D+2)@1200 UTC, uniformly over the grid.

• One with a 3D collocation, in which the position dependent start and end times areused (found in datasets dt0 and dt1, in the product file).

The reason for having those two validation strategies is that some users might wish (orhave to) ignore the accurate timing information provided with each vector. Using only thecentral times, these users need to know if the uncertainty estimates are to be enlarged and,if so, by which amount.

3.3.1 2D collocation

The in-situ drift vector is defined by first selecting the start and end position record in theirtrajectory. Those are the ones closest (in time) to 1200 UTC, on both dates. From those 2positions, dX ref and dY ref are computed. The product dXprod and dY prod are those of thenearest-neighbour in the product grid.

3.3.2 3D collocation

The in-situ start (end) point is searched for along the trajectory: each record is remappedinto the product grid where a product start (end) time is computed by bilinear interpolationfrom the 4 encompassing grid points. Because the records are ordered chronologically, itis possible to stop searching as soon as both start and end in-situ records are selected.As in the 2D collocation, they define the ’truth’ displacement components: dX ref and dY ref.The components for the product (dXprod and dY prod) are selected as those of the nearest-neighbour in the product grid.

3.3.3 Additional remarks

In version v1 of this validation exercise, the spatial collocation was achieved by bi-linearinterpolation of the 4 neighbouring vectors from the product grid to the position of the ref-erence vector. Further investigations confirmed that this method could lead to artificiallygood validation statistics, since part of the noise in the product was averaged out in thisprocess. From v2 of this report onwards, spatial collocation relies on nearest-neighbour se-lection. The distance to the nearest neighbour must be inside 40 km radius from the start ofreference vector.

In versions v1 and v2 of this validation report, matchup pairs were allowed in the val-idation dataset if the time difference of both the start and stop records were less than 3hours. This choice conducted to potentially allowing differences in drift durations of up to 6



hours. Further investigation of the v1 and v2 validation pairs, however, demonstrated thatthe difference in drift durations was mostly in the range [-1 h:+1 h]. This was thanks tothe hourly (sometimes sub-hourly) sampling of most of the buoy trajectories, allowing foraccurate temporal registration. This excellent temporal sampling is also why changing thetime collocation criteria did not lead to significant changes in results in Hwang and Lavergne2010b, section 4.1.

In versions v3 of the validation excercise, the temporal collocation was changed to allowcollocation pairs only if the time difference at the start data record is within [-3 h:+ 3h], andif the difference in drift duration is within [-1 h:+1 h]. As noted above, this change has verylittle impact on the selection of buoy matchups since they mostly report hourly sampledtrajectories. This change does not invalidate earlier validation results documented in v2 ofthis report, nor those in Lavergne et al. 2010 or Hwang and Lavergne 2010b. This change ishowever required to ingest high-resolution SAR-based drift vectors (section 2.2) which mighthave strongly varying temporal sampling characteristics due to orbit configuration and revisittime.

To clear the temporal and spatial collocation criteria above is not enough for entering thevalidation dataset. Additional constraints are imposed to obtain a more controlled validation:

• A validation vector must be surrounded by 4 valid OSI SAF vectors. Although only thenearest of these 4 is considered in the statistics, this constraint is introduced to avoidvalidation data in the outer edges of the vector field, like in the marginal ice zone, orin coastal regions (land-fast ice). Since some of the ice-tethered buoys we use (e.g.ITPs) are designed to continue floating when sea ice melts, this constraint is also aneffective way of not collocating ocean drift measurements with our product.

• Any two validation vectors for a given product date must be separated by at least3 × 62.5 = 187.5 km. This constraint is introduced so that all data pairs enteringthe validation dataset are independent from each others. Two neighbouring OSI SAFsea ice drift vectors indeed exhibit correlated uncertainties due to the overlap of theirimage-matching windows (e.g. Lavergne et al. 2010, section 3.2). This correlation ofneighbouring vector should ideally be taken into account when computing validationstatistics. However, thanks to the excellent coverage of the OSI SAF product, we canbe strict about the correlations and steer away from them by rejecting validation pairsthat are too close to each others on a daily grid.

3.4 Representativity error

Although we only use high quality buoy position data and although the collocation metodsand parameters are quite stringents, a possibly high and mostly uncontrolled source of errorresides in the representativity mismatch between the scales sampled by the buoy and thesatellite product.

A buoy indeed samples the motion of the ice floe it was deployed over. Although in-vestigators in field campaigns tend to choose rather large floes for limiting the risk of thebuoy disappearing too rapidly, the size and shape of the floe will change with time throughcollision or breaking events.

On the other hand, the satellite ice drift product samples the motion of a much largerarea of the sea ice surface that is close to 12000 km2. The mismatch between the twoscales of motion contributes to part of the error budget and it is not possible to separate this



representativity error from the measurement error of the satellite product with such a simpletwo-way statistical analysis. See Hwang and Lavergne 2010a, section 3.3 for a discussionon the representativity error budget of comparing a satellite drift vector to one recorded by abuoy.

3.5 Graphs and statistical measures

As introduced in section 3.1, this report is mostly interested in validating drift componentsdX and dY . We concentrate on two comparison exercises for reporting validation resultsfor those variables.

3.5.1 Product vs Reference

In this type of graph, the x-axis is the truth and the y-axis is the estimate given by theproduct. In an ideal comparison, all (truth,product) pairs are aligned on the 1-to-1 line.The spread around this ideal line can be expressed by the statistical correlation coefficientρ(Reference,Product), noted ρ(R,P ). If, at the same time, this correlation is close to 1 andthe parameters of the regression line are close to 1 (α, slope) and 0 (β, intercept), then thematch between the truth and the product is satisfactory.

In this report, a unique graph (and statistical values) is produced for dX and dY at thesame time. This means that the pairs appearing on the graphs are both (dX ref,dXprod)and (dY ref,dY prod). This also implies that errors in dX and dY are considered globallyindependent, an assumption that is validated using the graphs introduced in the next section.

3.5.2 Error in dY vs error in dX

In this type of graph, the x-axis is the error in dX, that is dXprod − dX ref (noted ε(dX))and the y-axis is the error in dY , that is dY prod − dY ref, noted ε(dY ). This graph is a moreinteresting approach for presenting the validation data than the one in the previous section.

Indeed, such a graph permits giving quantitative estimates for:

• the statistical bias1 in both components: 〈ε(dX)〉 and 〈ε(dY )〉;

• the statistical standard deviation of the errors in both components: σ(dX) and σ(dY );

• the statistical correlation between the errors in both components: ρ(εdX, εdY ).

The last three quantities enter the error covariance matrix Cobs which is of prime importanceto any data assimilation scheme. It is important to note the difference between the corre-lation coefficient introduced in this section and the one from section 3.5.1. ρ(εdX, εdY )assesses if the errors in the two components of the drift vector are correlated or not. ρ(R,P )assesses if the product (seen as a sample) is close to a linear scaling of the referencedataset (seen as a sample too).

In any case, those are statistical measures of the errors. They can only give averageuncertainties estimates and result in a unique set of numbers (those populating Cobs) to beused for an extended period of time (all distribution year round) and for the whole extent ofthe Northern or Southern Hemisphere grid. In addition, the statistics include an unknownamount of representativity error (section 3.4).

1〈x〉 is the average of x.



4. Results of validation

Validation of the NH and SH product grids are addressed separately in this section. A 2years long validation of the NH region is first documented and analysed. In a later section,we document the quality over the SH area against the few buoy datasets available in thisregion. Additionally, we introduce a comparison against a high-accuracy SAR-based icedrift product available from DTU.

The NH validation in this report is for years 2013 and 2014, and document the accuracyof ice motion fields retrievec from AMSR2 (GCOM-W1) and SSMIS (F17), in addition toASCAT (METOP-A). Readers are referred to earlier versions of this report (e.g. v2) for asimilar validation covering years 2006-2009 with instruments AMSR-E (NASA Aqua), andSSM/I (F15).

4.1 Validation over the NH area

The validation results presented in sections section 4.1.1 and section 4.1.2exclude the Fram Strait and East Greenland Sea regions. These are addressed

in section 4.1.4

4.1.1 Graphs and analysis

Figure (4) and Figure (5) introduce selected validation graphs for various single-sensorOSI SAF sea ice drift products, as well as for the multi-sensor dataset. For all plots, the ge-ographical region being validated is the Northern Hemisphere (excluding Fram Strait southof 82N) and the validation period includes all product files whose start date is between 1st

January 2013 and 31st December 2014.On ”Product vs Reference” graphs (right column in Figure (4)) each validation pair (one

for dX and one for dY ) are plotted in a 1-to-1 scatterplot. The solid line is the regressionline (whose coefficients are entered as labels in the plot area).

Figure (4) and Figure (5) are a simple and effective way of presenting the validationresults and get a good impression of the quality of each product. First, it can be noted thatall products are mostly non biased. The magnitudes of 〈ε(dX)〉, 〈ε(dY )〉 are indeed small (acouple of 100 metres) in comparison to the standard deviations (a couple of 1000 metres).This is an important result when it comes to using the products in assimilation exercises.

It also clearly appears from an analysis of Figure (4) that the method implemented in theOSI SAF chain results in a limited uncertainty. Displacement errors (in terms of standarddeviation) are small (maximum 4.5 km, 1/3 of image pixel size).

Those errors are small despite no special treatment has been implemented for correctingthe satellite geolocation uncertainty, which might contribute with some fair amount for theearlier sensors like SSM/I and AMSR-E (see for example Wiebe et al. 2008).


Figure 4: Selected validation graphs for AMSR2 ’W1’ (top row), SSMIS ’F17’ (mid-dle row) and MULTI ’multi-oi’ (bottom row) products. All pertain to NHarea, to the 3D collocation setup and 1st January 2013 and 31st Decem-ber 2014 period. Left (right) column presents ”error(dY ) vs error(dX)”(”product vs reference”) types of graphs. N is the number of validationpairs.


Figure 5: Validation graph for ASCAT product. The left (right) panel contains resultsfor the 3D (2D) collocation methods.

Another interesting result is that errors in dX and dY are mostly uncorrelated. Theobservation error covariance matrix Cobs can be considered a diagonal matrix as a for allpractical purposes. Note however that all necessary information is provided in this report touse a non-diagonal Cobs.

The analysis conducted so far indicates that the error distribution (when spatially andtemporally averaged) can be approximated by a 0 mean, uncorrelated, bivariate Gaussianprobability model. Only the standard deviations σ(dX) and σ(dY ) are to be adapted whenchoosing from the set of single-sensor and multi-sensor ice drift products.

Indeed, when it comes to ranking the products, one of them seems to compare muchbetter with the validation dataset. The sea ice drift product retrieved from AMSR2 ’GCOM-W1’ (37GHz channels) presents, by far, the smallest values for both σ(dX) and σ(dY ). Thislimited range of errors also translates in the high correlation coefficient (ρ = 0.987) andgood regression line for this product (right column, first row in Figure (4)). This can also bevisualized by looking at the vector field itself which, most of the times, looks less noisy thanthe ones from other instruments.

This higher quality might be explained by several factors, including the smaller foot-print/spacing of the two 37 GHz channels of AMSR2 (see the PUM) and the better temporalstability of their intensity patterns (compared to, e.g., those at 91GHz on SSMIS). In anycase, the ice drift product from AMSR2 allows statistical standard deviations of 2.04 km (2.07km) and is the product comparing best to the reference dataset.

This is very much in line with the results obtained with the AMSR-E (Aqua) sensor (June2002 -October 2011) as documented in earlier version of this report. The AMSR2 (GCOM-W1) instrument is in many respect the follow-on mission from JAXA, and the good accuracywas expected. In addition, the data aquisition and downlink stream seems more reliablewith AMSR2 than it was with AMSR-E, so that we can expect excellent ice motion coverage(fewer days with missing data).

Ice drift estimates from SSMIS (F17) have roughly the same accuracy as was docu-mented earlier for SSM/I (F15), with standard deviation of mismatch between satellite prod-uct and validation data of 3.58 km (3.29 km) and correlation coefficient of 0.96 (second row



in Figure (4)).As documented in earlier version of the report, ASCAT (Metop-A) estimates are those

with relatively worse accuracy with standard deviations of 3.95 km (4.23 km) (left panel inFigure (5)).

As is expected from a multi-sensor analysis, the multi-oi product (bottom row in Fig-ure (4)) achieves a good (but not optimal) accuracy (2.70 km and 2.67 km) and more im-portantly a better spatial coverage. The later is hinted by the number of collocation pairsachieved in this 2 years period (6007 for MULTI-OI). The purpose of the MULTI-OI is exactlyto provide a more consistent spatial coverage, with least possible missing vectors. In thisreport we document this comes to the cost of a slightly larger uncertainties wrt to the bestof the single-sensor products (namely AMSR2-GW1).

The impact of the collocation scheme adopted is documented on Figure (5), where AS-CAT-METOPA validation results with the 3D (2D) collocation are in left (right) panel (seesection 3.3). The statistical uncertainties slightly increase from the 3D to 2D case whichconfirms that the start and stop time information provided with the product file is relevantto use when comparing (or assimilating) the motion vectors. This is although the motionvectors are computed from daily averaged maps in this OSI SAF product. Users of theMULTI-OI product miss this opportunity since all start and stop time in this product are atD@1200 UTC.

The same slight degradation of validation statistics is observed from 3D to 2D collocationfor all single-sensor products. This enlargement of the error statistics is, however, damperedby the high level of averaging occuring in our validation exercise, spatially and temporally. Ona single case basis, like when a circular motion pattern is induced by a moving atmopshericlow pressure, the t0 and t1 are quite significant and should not be neglected, as illustrated inLavergne et al. 2008.

4.1.2 Impact of the product flags

The graphs and statistical results presented above are when restricting the matchup pairs tocases when the product status flag indicates nominal quality (value 30, see PUM).For single-sensor products, it corresponds to vectors directly processed by the CMCC al-gorithm, e.g. without looking at the neighbouring vectors. For the multi-sensor product,it excludes the vectors computed by spatial averaging (in case of missing vectors in all 3single-sensor products). These nominal quality vectors are - by far - the most frequentin all products, although other flag values can appear early and late in the drift season (e.g.in October or November), especially for SSMIS-F17 and ASCAT-METOPA.

As expected, the validation statistics worthen when including matchup pairs that are notnominal quality.

In Table (2), the validation statistics of single-sensor products are reported for 3 dif-ferent values of status flag (columns 1-3) and gathering all flags together (column 4).For each flag category, both dX and dY standard deviation of mismatch is reported. Inthe first column (nominal quality), the same values as in section 4.1.1 are reported.These validation statistics are much worse when considering only the status flag 21(corrected by neighbours, in column 2). The status flag 20 (smaller pattern, incolumn 3) does not induce such high uncertainties. Since the status flag 30 (nominal quality)dominates in the matchup dataset (and in the product files), the statistics when taking allflags together (column 4) do not differ much.



ProductNominal (30) Corr. neighb. (21) Small pattern (20) All Flagsσ(dX) σ(dY ) σ(dX) σ(dY ) σ(dX) σ(dY ) σ(dX) σ(dY )

amsr2-gw1 2.037 2.073 9.229 9.244 5.017 3.875 2.261 2.249ssmis-f17 3.580 3.286 8.339 7.486 6.618 5.834 3.863 3.528ascat-A 3.945 4.229 7.696 8.127 5.841 7.906 4.393 4.734

Table 2: Validation statistics of single-sensor products for different values of status flagas well as for all flags together. The columns report the standard deviation of the mismatchbetween satellite and validation datasets for dX and dY components of the drift vector (unitkm).

The multi-sensor product comes with only two flavours of status flag. Value 30 isnominal quality, and value 22 is interpolated. Interpolated vectors do not resultfrom any motion tracking algorithms but are just an attempt to fill data gaps through spa-tial interpolation. The validation statistics of the multi-sensor are: 2.708 km (2.668 km) fornominal quality vectors, 9.903 km (9.060 km) for the interpolated vectors, and 2.955km (2.882 km) when considering all vectors in the validation dataset.

It seems from the analysis above that the different status flag values indeed corre-spond to different level of accuracies. This might be usefull when associating uncertaintyvalues to ice motion vectors, e.g. when performing Data Assimilation.

4.1.3 Geographical analysis

Figure (6) plot the location and magnitude of mismatch in length of 48 hour drift (all status flagvalues). The colorbar spans [−15 : +15] km, and is thus selected to show outliers.

As expected from the statistics documented above, the mismatch are generally larger forthe ssmis-f17 and ascat-metopA maps than for amsr2-gw1 or multi-oi.

No clear geographical pattern is seen from the maps corresponding to the single-sensorproducts, although the strongest colors seem to reside in the outskirt of the basin, for ex-ample along the northern coast of Canada (multi-oi product), and the Fram Strait and EastGreenland Sea (most products).

4.1.4 Challenges in highly dynamic and peripheral regions

Case study: Beaufort Sea in early March 2013

As an example, a series of strong underestimations by the multi-oi product (dark blue sym-bols along the northern coast of Canada) are all against the same ITP41, in early March2013. The situation for 5th to 7th March is illustrated on Figure (7). A weak Beaufort Gyreis visible in the drift field which is otherwise characterized by rapid westerly drift along thecanadian coast. This strong drift holds in several days and results in openings between theice pack and the coast (shear deformation), and more open ice cover (shades of blue in thebackground of Figure (7) are for ice concentration classes on March 7th).

ITP41 (pink arrow) is in this highly dynamic region and drifts at 0.5 m/s (averaged overthese two days). On the other hand, the satellite retrievals are challenged by the proximity tocoast and an open ice cover, and do not allow nominal quality (black vectors) retrievalsclose to ITP41 (minimum distance of 70 km towards the ocean). The collocation was actually


amsr2-gw1 ssmis-f17

ascat-metopA multi-oi

Figure 6: Map showing the location and magnitude of mismatch of 48 h drift length.Blue (red) shades indicate that the satellite estimates shorter (longer) dis-placement than observed by the drifter.


performed in that case against a interpolation estimate (violet vector closest to ITP41).The simple spatial interpolation scheme implemented in the multi-sensor algorithm does notallow to reproduce the higher coastal speeds, since in that case the interpolation really isan extrapolation. The degradation of multi-oi validation statistics with status flag wasalready presented in section 4.1.3.

Figure (7) is also an opportunity to illustrate how well the OSI SAF multi-oi productmatches with the drift vectors from all other drifters when these are located farther fromcoastal dynamical drift regions.

Figure 7: Ice motion vectors (multi-oi) from 5th to 7th March 2013 superimposedwith validation vectors of several drifters in the Beaufort Sea. Satellitevectors are in black (nominal quality) and violet (interpolation).In-situ vectors use the same colors as in Figure (2). The ice mask usedin background is that of March 7th. ITP41 is the pink vector close to theCanadian coast. Note the other six in-situ vectors.



Yearly statistics in the Fram Strait and East Greenland Sea

Another region of the Northern Hemisphere with very dynamic sea ice drift is the Fram Straitand East Greenland Sea, the main outflow gateway of the Arctic Ocean. In this region, therelatively coarse resolution OSI SAF drift product can be challenged by strong deformationsthat disable pattern recognition in the satellite images, vicinity of land, vicinity of the ice edge,strong longitudinal gradients in the motion field (bathymetry-driven East Greenland current).

With these challenges in mind, Table (3) summarizes the validation results obtainedwith the single-sensor and multi-sensor products in the Fram Strait and East Greenland Sea(south of 82N).

The statistical bias is in the first 2 columns is much larger (and negative) than in the restof the Arctic Ocean (Table (4)). The standard deviations are also larger (but note the quitelimited number of samples N ). The slope α of the linear fit is also much smaller than one,even as low as 0.60 for the multi-oi product. Note that there are about 50 additional matchupsfor the multi-oi product than for the single-sensor products, which points to a fair number ofinterpolated vectors.

Product 〈ε(dX)〉〈ε(dY )〉 σ(dX) σ(dY ) α β ρ(R,P ) N

amsr2-gw1 -1.604 -4.416 5.863 10.075 0.72 +2.451 0.83 109ssmis-f17 -0.385 -3.450 6.206 7.754 0.78 +2.369 0.86 95ascat-metopA -1.781 -6.794 9.580 11.226 0.65 +1.992 0.66 86multi-oi -3.447 -6.833 8.616 11.869 0.60 +3.333 0.78 150

Table 3: Statistical results for validation of the ice drift products in Fram Strait(south of 82N) for years 2013-2014.

4.1.5 Comparison to other datasets

Kwok et al. (1998), for example, report standard deviations of 8.9 km (10.8) and 9.9 (11.2)for SSM/I1 85 GHz V. pol. dX (dY ) and H. pol. dX (dY ) products, respectively. Thisis for a 3 days product in the central Arctic. For their 1 day dataset in the Fram Straitand Baffin Bay, those values are 5.3 km (4.3) and 6.0 (4.7) respectively. Those values areextracted from Table 2, p. 8196. To be fair, one should mention that the validation exercise inKwok et al. (1998) was performed against IABP buoys and using a 2D-type collocation (seeour section 3.3.1). IABP buoys were mainly tracked with Argos positioning, which are lessaccurate. Kwok et al. do not state that they provide the accurate t0 and t1 time informationwhich are needed for using their product in a 3D collocation strategy.

Error statistics reported for the various IFREMER datasets (QuikSCAT-SSM/I mergedand AMSR-E 89 Ghz) as well as by Haarpaintner (2006) are not obviously compared withour values as they are computed for the North-South and East-West components of thedrift vectors. Those components exhibit non linear, latitude dependent relationships to thedX and dY we are validating. Note, however, that only the AMSR-E (89 GHz) from IFRE-MER and the QuikSCAT product of Haarpaintner (2006) have a time span of 2 days like theOSI SAF product. The merged SSM/I and QuikSCAT dataset delivered by IFREMER is a 3days ice drift product.

An intercomparison of the IFREMER/Cersat, OSI-405 and other products was conducted



by Hwang 2013, which addressed to some extent the caveats of intercomparing drift prod-ucts with varying time spans.

4.1.6 Discussion and conclusion

The validation statistics for all the OSI SAF low resolution ice drift products over NH area(including Fram Strait and all status flag values) are summarized in Table (4).

Product 〈ε(dX)〉〈ε(dY )〉 σ(dX) σ(dY ) ρX,Y α β ρ(R,P ) N

amsr2-gw1 -0.07 -0.12 2.39 2.68 -0.06 0.96 -0.05 0.98 5930ssmis-f17 -0.05 -0.11 3.92 3.68 -0.03 0.95 +0.00 0.95 5061ascat-metopA +0.03 -0.18 4.52 4.96 +0.01 0.95 -0.00 0.93 5778multi-oi -0.16 -0.17 3.25 3.54 -0.08 0.92 -0.07 0.97 6246amsr-aqua (*) -0.10 -0.10 2.70 2.77 ? 0.95 -0.02 0.97 3977ssmi-f15 (*) -0.07 -0.02 4.10 4.01 ? 0.93 +0.08 0.94 4218

Table 4: Statistical results for validation of the ice drift products in the whole North-ern Hemisphere for years 2013-2014. All status flag values are in-cluded and the 3D collocation scheme is used. ρX,Y is a shortenednotation for ρ(σ(dX), σ(dY )), the averaged correlation between the un-certainties on dXand dY . (*) The last two lines (AMSR-E ’AQUA’ andSSM/I ’F15’) are reproduced from v3 of this report, and thus correspondto Oct-Apr 2006-2009. ρX,Y values was not available for these, but werealso tiny numbers.

In Table (4), the bias in the dY component of the drift is usually larger than for thedXcomponent. Besides, it is quite consistently negative. Kwok et al. (1998) (section 3.2, p.8203) wrote a detailed investigation of a similar bias in their ice drift product. Since such alimited negative bias was not observed when excluding Fram Strait matchups (section 4.1),we conclude that it is mostly driven by the more pronounced dY low bias in this region.

The last two lines in Table (4) are reproduced from v3 of the report (Table (1)) and corre-spond to Oct-April 2006-2009, and thus not the same validation dataset, although the ITPsfrom WHOI were already the prominent data source. These last two lines are reproducedhere for confirming that, since the satellite sensors are so similar, the new AMSR2 (SSMIS)and previous AMSR-E (SSM/I) ice motion products have very similar accuracies. Transitionfrom one to the other should be rather straightforward for users.

4.1.7 Monthly validation results

An analysis of monthly time series of validation results was performed and is reported inLavergne et al. 2010.

As expected, monthly validation results show better validation statistics in the core ofwinter (Dec-Jan-Feb) than at the edges of the summer period. This somehow confirmsour a-priori feeling that the product should be more accurate when the sea ice emissivitypatterns are well pronounced and stable in time (core of winter).

The simultaneous temporal variation of the representativity error (see section 3.4) chal-lenges however this conclusion. Indeed, since the ice in the Arctic Ocean is more packed



and rigid during these very same winter months, we expect that the representativity error thatis induced by comparing an area averaged drift (satellite product) against that of a single floe(drifting buoy) is reduced as well.

A thorough discussion of these aspects is in Lavergne et al. 2010, section 4.4.Interested users are also directed to http://osisaf.met.no/validation/val lrsid.shtml where

monthly validation statistics for the operational OSI-405 products are updated regularly.

4.2 Validation over the SH area

The SH validation results presented below are reproduced from v3 of the report.They do not cover AMSR2 (but AMSR-E) nor SSMIS (but SSM/I). However, the

validation results obtained for the Northern Hemisphere for these newinstruments are very similar to those obtained earlier with AMSR-E and SSM/I,so that we expect the conclusions presented below to hold. Lack of new, in-situ

validation data is the main reason for not updating the SH analysis now.

Due to the lack of buoy trajectories over the SH sea ice (section 2.3), two separatesections are dedicated to validation in the Souther Hemisphere. First, we present validationagainst two buoys in (austral) winter 2011. Second, we document a comparison obtained bycollocation of the OSI SAF product against a high-resolution SAR-based sea ice drift productover winters 2010 and 2011.

4.2.1 Validation against SAMS Icebell buoy

Figure (8) present validation results obtained when collocating the OSI SAF product withthe buoy trajectories of Icebell 10 and Icebell 18 from SAMS. The validation period israther short since it spans from mid April to late August 2011.

Analysis of the statistics embedded in these graphs is necessarily limited by the amountof validation data. The total number of matchups ranges between 20 to 30 depending on thesensor and collocation method. This is to be compared to the NH case with its almost 5000matchups (section 4.1).

These SH validation graphs are thus only used to confirm that the ice motion trackingmethod seemlingly works over SH sea ice as in the Arctic. Users might want to use slightlylarger uncertainty values in SH than in NH, but the lack of buoy validation data does notallow to conclude on a quantitative estimate.

4.2.2 Comparison against DTU SAR-based ice drift product

As mentionned in section 2.2, accurate sea ice drift information is available via the MyOceanproject from Envisat ASAR WSM images. The product is processed and validated at DTUagainst ITPs, in the Arctic only. It exibits limited bias and standard devitations of the drifterrors of about 300 to 600 m. Although not negligible, these uncertainy are less than thosedocumented for the OSI SAF low resolution product we want to validate (see previous sec-tion).


http://osisaf.met.no/validation/val_lrsid.shtml

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]

Error in drift along the processing grid (dX) [km]

SH Validation:

20110401-20110901

N = 21

<ε> = (-0.82 , -2.18) [km]

σ = (3.39 , 5.00) [km]

ρ = -0.22

amsr-aqua

NN3D

ε = x|yprod - x|yref

amsr-aqua CMCC

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]


SH Validation:

20110401-20110901

N = 29

<ε> = (-0.48 , -0.18) [km]

σ = (4.48 , 5.24) [km]

ρ = -0.36

ssmi-f15

NN3D


ssmi-f15 CMCC

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]


SH Validation:

20110401-20110901

N = 19

<ε> = (-0.53 , -0.08) [km]

σ = (3.32 , 5.02) [km]

ρ = -0.19

ascat-metopA

NN3D


ascat-metopA CMCC

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]


SH Validation:

20110401-20110901

N = 25

<ε> = (-0.20 , -1.09) [km]

σ = (2.82 , 4.17) [km]

ρ = -0.18

multi-oi

NN3D


multi-oi CMCC

Figure 8: Selected validation graphs for AMSR-E (top-left), SSM/I ’F15’ (top-right),ASCAT (bottom-left) and merged (multi-oi) (bottom-right) products. Allpertain to SH area, to the 3D collocation setup and period from April 1st

to September 1st 2011. N is the number of validation pairs.


It is thus of interest to try the comparison of the OSI SAF against the SAR-based andthus extend the limited validation against buoys we introduced in section 4.2.1. Severallimitations to this approach should however be kept in mind.

First, the SAR-based ice drift product from DTU is not validated over Antarctic sea ice.We can assume that the SAR-based product quality is not degraded over Antarctic sea ice,but this does not allow us to refer to documented validation statistics as over the ArcticOcean.

Second, the validation of DTU SAR product against ITPs (in the Arctic) shows non-negligible uncertainty amounting for about 1/4th to 1/3rd of those obtained with the OSI SAFproduct. Taking these into account is however not trivial since a possibly large part of theerror budget of each satellite product is strongly correlated with each other. Indeed, andfollowing the discussion from Hwang and Lavergne (2010a), the representativity error madewhen comparing a point measurement (buoy) against the average drift of a 5 km radius disc(DTU product) is not independent from reprensentativity error to bridge the gap between thebuoy and the averaging scales of the OSI SAF (60 km radius).

Finally, Antarctic sea ice is generally found at lower latitudes than in the Northern Hemi-sphere. Orbital parameters and fewer ordering of Envisat SAR scenes in the SouthernHemisphere induce that there is much less SAR-drift coverage in SH than in NH. Anotherconsequence is that the temporal sampling (start time and duration) are less varying than inthe Arctic Ocean, leading to less matchups falling into the collocation criteria.

For all the reasons above, the results we present in this section are to be considered withcare, and are more documenting a comparison than a proper validation. This is nonethelessa usefull exercise to gain confidence in the accuracy of the product over Antarctic sea ice.

The first impression from Figure (9) is that both the OSI SAF and the DTU productsmostly agree about the magnitude and direction of the drift vectors. This is quite confortingand encouraging although it does not constitute an absolute proof.

Statistics appearing on Figure (9) differ from those presented above for NH area in thatthe spread around the 1:1 line is generally larger. This translates into slightly lower linearcorrelation coefficients (ρ = 0.90 to ρ = 0.95 while the NH values were generally above 0.95).

The spread in σ(dX) and σ(dY ) is also larger in this comparison against DTU SARproduct, with values around 5 km for all instruments and collocation setup (3D and 2D).Over the SH area, we do not observe a better match for one of the instrument wrt to theothers. The merged dataset (multi-oi) exhibit slightly better statistics.

We note that the biases 〈ε(dX)〉 and 〈ε(dY )〉 are larger than in the NH against buoys,but still an order of magnitude less than the standard deviations. The underestimation of theOSI SAF product with respect to the reference dataset also translates into larger deviationfrom the 1:1 line.


-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]


N = 2580

<ε> = (-0.15 , -0.49) [km]

σ = (4.70 , 4.56) [km]

ρ = 0.02

ascat-metopA

NN3D

CMCC


SH Validation:

20100401-20101101 + 20110401-20111101

pdf regions

68%95%

-40

-20

0

20

40

-40 -20 0 20 40

CM

CC

drift

(d

X a

nd

dY

) a

lon

g t

he

pro

ce

ssin

g g

rid

[km

]

Reference drift (dX and dY) along the processing grid [km]

SH Validation:

20100401-20101101 + 20110401-20111101

N = 5160

α = 0.93

β = -0.08

ρ = 0.92

ascat-metopA

NN3D

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]


N = 2886

<ε> = (-0.27 , -0.58) [km]

σ = (5.05 , 4.70) [km]

ρ = 0.14

ssmi-f15

NN3D

CMCC


SH Validation:

20100401-20101101 + 20110401-20111101

pdf regions

68%95%

-40

-20

0

20

40

-40 -20 0 20 40

CM

CC

drift

(d

X a

nd

dY

) a

lon

g t

he

pro

ce

ssin

g g

rid

[km

]


SH Validation:

20100401-20101101 + 20110401-20111101

N = 5772

α = 0.91

β = -0.12

ρ = 0.91

ssmi-f15

NN3D

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

Err

or

in d

rift

alo

ng

th

e p

roce

ssin

g g

rid

(d

Y)

[km

]


N = 3017

<ε> = (-0.70 , -0.47) [km]

σ = (4.52 , 4.23) [km]

ρ = 0.04

multi-oi

NN3D

CMCC


SH Validation:

20100401-20101101 + 20110401-20111101

pdf regions

68%95%

-40

-20

0

20

40

-40 -20 0 20 40

CM

CC

drift

(d

X a

nd

dY

) a

lon

g t

he

pro

ce

ssin

g g

rid

[km

]


SH Validation:

20100401-20101101 + 20110401-20111101

N = 6034

α = 0.91

β = -0.16

ρ = 0.93

multi-oi

NN3D

Figure 9: Selected validation graphs for ASCAT (top row), SSM/I ’F15’ (middle row)and merged (multi-oi) (bottom row) products. All pertain to SH area, tothe 3D collocation setup and period from April 1st to September 1st 2010and 2011. Left (right) column presents ”error(dY ) vs error(dX)” (”productvs reference”) types of graphs. N is the number of validation pairs.


5. Conclusion

This report deals with the validation of the OSI SAF low-resolution sea ice drift product frompassive and active microwave satellite data (OSI-405). Three sets of validation exercisesare presented:

1. Validation in the Northern Hemisphere (Arctic Ocean) over 2 years (Jan 2013 to Dec2014);

2. Validation in the Southern Hemisphere (Ross Sea) in (austral) winter 2011 against apair of GPS buoys;

3. Comparison in the Southern Hemisphere in (austral) winters 2010 and 2011 against ahighly accurate SAR-based ice drift product available from DTU.

After having presented the data that constitute our reference dataset (mainly GPS in-situtrajectories) we introduced the validation strategy and, particularly, two ways of collocatingthe datasets: 2D which considers uniform start and end time at 1200 UTC for all vectors and3D which makes use of the extra time information available in the product file.

Results are analyzed from various graphs and statistics and conclude that the OSI SAFice drift parameters dX and dY are mostly unbiased and that their statistical error probabilitydistributions can be well represented by uncorrelated bivariate normal PDFs. Quantitativeestimates of the terms entering Cobs, the error covariance matrix are provided in separatetables for the 2D and 3D collocation/usage for the Northern Hemisphere grid.

The tables confirm that the AMSR2 instrument gives the best ice drift vectors in the ArcticOcean, with standard deviations (on dX and dY ) of about 2.5 km. The datasets from othersensors (SSMIS and ASCAT) have uncertainties level of around 3.5 – 4.5 km. The AMSR-2(SSMIS) results are very similar to those obtained eralier with AMSR-E and SSM/I. This setsthe scene for optimal operational sea ice drift mapping at the OSI SAF until 2025 (plannedend of life for GCOM-W3 and DMSP ’F20’).

As expected, the uncertainty level is slightly raised when the pixel varying time informa-tion is ignored (2D collocation). However, the standard deviations are only enhanced by fewhundred meters.

A dedicated analysis of the impact of status flag values on the validation statisticsconfirms (for the first time) that these are linked to different level of accuracies, and that thenominal quality vectors are by far the best in the products. The other flag values shouldbe associated with higher uncertainties or, if allowed by the use case, discarded.

A dedicated analysis of the quality of this low-resolution satellite product in peripheral re-gions with dynamic ice conditions reveals (for the first time) that the OSI-450 product can bechallenged and that the interpolation vectors of multi-oi product should not be trustedthere. All products showed low bias in the Fram Strait and East Greenland Sea regions.This will be investigated in later upgrades of the algorithms and processing software. Luck-ily, this region is well covered by other, higher resolution sea ice drift products such as theSAR-based operational estimates from MyOcean/PolarView/DTU.

In the Southern Hemisphere, the lack of GPS buoy trajectories prevent us from conclud-ing on quantitative estimates for the uncertainties. Although limited in scope and hazardous



to compare against NH results, we thus introduce results from a comparison of the OSI SAFproduct against the SAR-based product from DTU (which itself is not validated in the SH).The obtained statistics show an overal and encouraging agreement between the two prod-ucts, but also larger biases and spread.

The validation statistics are matching the threshold and target accuracy requirements asspecified in the Product Requirement Document (PRD) for OSI-405 product. The requiredthreshold (target) accuracy is 10 (5) km standard deviation.

All OSI-405 products (all 3 individual single-sensor and the multi-sensorproducts) clear the target accuracy (Table (4))

.Still in PRD, the optimal accuracy is defined as 2 km. This value might be reached by the

AMSR2 product during selected periods of time (e.g. during the core of the winter season)but not on a yearly and hemispheric average validation exercise.

The failure of the AMSR-E instrument in early October 2011 resulted in a clear reductionof the accuracy of the merged/multi-sensor distributed since that date. With this report, wedemonstrate that the AMSR2 instrument is an excellent follow-up mission, that will benefitthe OSI-405 as soon as it is routinely processed.

In the Southern Hemisphere, one can expect that both the threshold and target accuracyrequirements are met on a season averaged basis. The comparison against SAR-basedproducts show statistics of the order of the target accuracy, but the DTU product itself surelyexhibits uncertainties with respect to ground truth (not accessible).

This report is a living document that will be updated when new sensors are introducedin the OSI SAF ice drift production chain as well as when the validation period is extendedor when new data sources are available for inclusion in the reference dataset. The latestversion of the present report and Product User Manuals are always available from the OSISAF Ice web portal: http://osisaf.met.no or by contacting the author.

Users are also directed to http://osisaf.met.no/validation/val lrsid.shtml where monthlyvalidation statistics and graphs are updated regularly for the operational OSI-405 products.



http://osisaf.met.no/validation/val_lrsid.shtml


References

CDOP2 PRD (2015, January). Ocean and Sea Ice SAF CDOP-2 Product RequirementDocument - v3.0.

Haarpaintner, J. (2006, January). Arctic-wide operational sea ice drift from enhanced-resolution Quikscat/SeaWinds scatterometry and its validation. IEEE Transactions on Geo-science and Remote Sensing 44(1), 102–107.

Hwang, B. (2013). Inter-comparison of satellite sea ice motion with drifting buoy data.International Journal of Remote Sensing 34(24), 8741–8763.

Hwang, P. and T. Lavergne (2010a, September). Triple comparison of OSI SAF low reso-lution sea ice drift products – v1.3. Technical Report SAF/OSI/CDOP/met.no/SCI/RP/152,EUMETSAT OSI SAF – Ocean and Sea Ice Sattelite Application Facility.

Hwang, P. and T. Lavergne (2010b, September). Validation and comparison of OSI SAFlow and medium resolution and IFREMER/Cersat sea ice drift products – v1.2. TechnicalReport SAF/OSI/CDOP/met.no/SCI/RP/151, EUMETSAT OSI SAF – Ocean and Sea IceSattelite Application Facility.

Kwok, R., A. Schweiger, D. A. Rothrock, S. Pang, and C. Kottmeier (1998, April). Seaice motion from satellite passive microwave imagery assessed with ERS SAR and buoymotions. Journal of Geophysical Research 103, 8191–8214.

Lavergne, T. (2015, April). Algorithm Theoretical Basis Document for the OSI SAF low res-olution sea ice drift product – v1.1. Technical Report SAF/OSI/CDOP/met.no/SCI/MA/130,EUMETSAT OSI SAF – Ocean and Sea Ice Sattelite Application Facility.

Lavergne, T. and S. Eastwood (2015, March). Low resolution sea ice drift Product User’sManual – v1.7. Technical Report SAF/OSI/CDOP/met.no/TEC/MA/128, EUMETSAT OSISAF – Ocean and Sea Ice Sattelite Application Facility.

Lavergne, T., S. Eastwood, H. Schyberg, and L.-A. Breivik (2008, October). Ice drift mon-itoring from low resolving sensors: an alternative method and its validation against in-situdata. Report MERSEA WP02 METNO STR 005 1A, MERSEA – Marine EnviRonmentand Security for the European Area.

Lavergne, T., S. Eastwood, Z. Teffah, H. Schyberg, and L.-A. Breivik (2010). Sea ice motionfrom low resolution satellite sensors: an alternative method and its validation in the Arctic.Journal of Geophysical Research 115, C10032.



Perovich, D., J. Richter-Menge, B. Elder, T. Arbetter, K. Claffey, and C. Polashenski (2014).Observing and understanding climate change: Monitoring the mass balance, motion, andthickness of arctic sea ice. Technical report, CRREL – Cold Regions Research and Engi-neering Laboratory.

Wiebe, H., G. Heygster, and L. Meyer-Lerbs (2008, October). Geolocation of AMSR-Edata. IEEE Transactions on Geoscience and Remote Sensing 46, 3098–3103.


Date post:	14-Aug-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Validation and Monitoring of the OSI SAF Low Resolution...

Documents