Ensemble spread-based assessment of observation impact: application to a global ocean analysis...

Quarterly Journal of the Royal Meteorological Society Q. J. R. Meteorol. Soc. 139: 1842–1862, October 2013 A

Ensemble spread-based assessment of observationimpact: application to a global ocean analysis

system

Andrea Storto,a* Simona Masinaa,b and Srdjan Dobricica

aCentro Euro-Mediterraneo per i Cambiamenti Climatici, Bologna, ItalybIstituto Nazionale di Geofisica e Vulcanologia, Bologna, Italy

*Correspondence to: A. Storto, Department of Numerical Applications and Scenarios, Centro Euro-Mediterraneo per iCambiamenti Climatici, viale A. Moro 44, I-40127, Bologna, Italy. E-mail: [email protected]

This article explores an ensemble strategy for evaluating the impact of differentobserving networks. The impact is represented by the relative ensemble spreadincrease, in model space, of data-denial ensemble simulations with respect to an‘all-observation’ ensemble experiment, evaluated independently for each observingnetwork. The forecast-error covariance intercomparison reduces to the ensemblespread intercomparison; thus, the method can be applied to any assimilation systemand requires only the proper construction of an ensemble system, although theimpact assessment results depend on the specific configuration of the investigatedanalysis system. Our approach allows us to determine the impact of the observingnetworks in model space (unlike Observing System Experiments) and for differentforecast ranges of the ocean general circulation model. No tangent-linear and adjointcoding of the ocean model are required. The method is applied for demonstrationto a large-scale global ocean variational analysis system. The ensemble members aregenerated by (i) perturbing the observations within the 3D-Var assimilation scheme,(ii) perturbing the surface forcing, and (iii) stochastically perturbing the ocean modelparametrisation tendencies. The impact is calculated for CTDs, XBTs, moorings,Argo, sea-level anomaly observations and sea-surface temperature measurementsfrom space-borne microwave instruments within the three-year period from January2003 to December 2005. It turns out, on the global scale, that altimetry exhibitsthe largest impact on near-surface temperature and sea-surface height. In contrast,deep-ocean impacts are led by the Argo float network. As expected, space-borneobservations (sea-level anomaly and sea-surface temperature observations) increasetheir impact in the Southern Ocean, due to the lack of a robust network of in situobservations. The results of the impact on the salinity indicate the great importanceof Argo floats, especially in the northern Extratropics.

Key Words: 3D-Var; ensemble assimilation; observing networks

Received 8 December 2011; Revised 28 September 2012; Accepted 8 October 2012; Published online in WileyOnline Library 27 December 2012

Citation: Storto A, Masina S, Dobricic S. 2013. Ensemble spread-based assessment of observation impact:application to a global ocean analysis system. Q. J. R. Meteorol. Soc. 139: 1842–1862. DOI:10.1002/qj.2071

c© 2012 Royal Meteorological Society

Observation Impact on a Global Ocean Analysis System 1843

1. Introduction

The evaluation of the impact of observing systems isa key question to which both the oceanographic andmeteorological communities are devoting efforts, in orderto optimally plan and sustain the most valuable observationtypes and their coverage at both operational (i.e. short- andmedium- ranges) and climate (i.e. seasonal and decadal)scales. Over recent decades, many studies have demonstratedthat ocean observations have a great importance on oceanoperational analyses (Testut et al., 2003; Nilsson et al., 2011),ocean reanalyses (Ishii et al., 2003; Kohl et al., 2007; Lee et al.,2009; Carton and Santorelli, 2009; Masina et al., 2011) andinitial conditions for coupled ocean–atmosphere seasonaland decadal forecasts (Kleeman et al., 1995; Segschneideret al., 2001; Alves et al., 2004; Alessandri et al., 2010; Bellucciet al., 2012).

Traditional methods for assessing the value of observa-tions are Observing System Experiments (OSEs; e.g. Bouttierand Kelly, 2001), which compare verification skill scores ofdata-denial experiments. In general, the limitation of OSEsis that they do not provide any information of the obser-vation impact in model space but rely on the availabilityof verifying observations. This requires a dense networkof (preferably independent) verifying observations in orderto evaluate the impact of the observations over the globalocean, which is unlikely to be met with the current oceanobserving network.

On the contrary, adjoint-based methods can map anyforecast quality metrics back as a function of the dataassimilation system (e.g. Langland and Baker, 2004), thusproviding a way to evaluate the observation contribution tothe forecast quality. For instance, Cardinali (2009) foundthat short-range adjoint-based sensitivity computationsprovide qualitative results similar to OSEs. These methodsrequire the coding of the adjoint version of the oceanmodel, which is very demanding if not available. On theother hand, they are quite fast from a computationalpoint of view. Furthermore, adjoint-based methods wereusually developed with the primary goal of being usedwithin the backward steps of data assimilation systems (i.e.for four-dimensional variational data assimilation systems,4D-Var), and consequently they were not concernedabout the use of simplified dynamics and physics in thetangent-linear and adjoint codes with respect to the fullynonlinear model, which may jeopardise their applicationfor operational configuration impact studies. They use thetangent-linear approximation (i.e. the linear propagation ofthe observation impact on the analyses onto the forecasts),which might be a limiting assumption for long forecastranges, on–off processes (Zou, 1997) or rapidly developingevents. Although Tremolet (2008) recently extended theirapplicability in the case of several outer loops of a 4D-Vardata assimilation system, their use is questionable forobservation impact evaluations within long-range (seasonal)forecasts or in mesoscale-dominated areas.

A trade-off between model space based observationsresponse and affordability of computations may be obtainedby defining energy norms in model space and calculatingfrom that the impact with respect to ‘all-observation’experiments (Storto and Randriamampianina, 2010a).Nevertheless, a fully three-dimensional impact of theobservations cannot be assessed with the previous data-denial methodologies. Since ensemble simulations bear

information about uncertainty as spread across ensemblemembers, an interesting alternative is given by the possibilityof comparing ensemble spreads associated with differentobservation type configurations, provided that the ensemblespread can approximate the investigated error covariances.A similar idea has been exploited by Liu et al. (2009),although they limited their study to the self-sensitivity ofthe analyses to the observations (i.e. the sensitivity to theobservations at their location) derived from an ensembleKalman filter data assimilation system in a coarse-resolutionatmospheric model. The ensemble estimation of forecastsensitivity to observations may be applied to both realand simulated observing networks, as shown by Tan et al.(2007), who compared the ensemble spread associated withthe assimilation of space-borne wind profilers within anatmospheric variational data assimilation system. The resultsof this procedure are provided in model space, which is ofobvious advantage for a global ocean impact assessment.Furthermore, the impact of forthcoming or hypotheticalobservations can be established by simply simulating thosedata and assigning realistic observational error to them (e.g.Liu et al., 2007), thus without the difficulties of defininga true ocean state, as required in the Observing SystemSimulation Experiments (OSSEs; Arnold and Dey, 1986).

There are many studies which aim to depict theimportance, absolute or relative, of the different observingsystem components in the ocean. Vidard et al. (2007)highlighted the importance of the tropical Pacific mooringarray on both analyses and seasonal forecasts; Balmasedaet al. (2007) showed that Argo data are able to improvetemperature, salinity and circulation all over the globalocean, except in those areas where mooring arrays havebeen deployed and where the Argo impact is less important.Similar results were found by Huang et al. (2008). Balmasedaand Anderson (2009) found that the impact of moorings onseasonal forecast skill is dominant in the equatorial Pacificdue to their ability to correctly increase the heat content,while sea-level anomaly observations have a very importantimpact in the north subtropical Atlantic and in the easternPacific, and Argo floats have a larger effect in the central-western Pacific and in the Indian Ocean. Also Yan et al.(2007) found a leading role of the Pacific mooring array,although they concluded that the impact of Argo floats isin general very weak. This given list of impact studies is notexhaustive, and often it is possible to find some disagreementbetween the results of different authors, which confirms howdifficult is to provide an evaluation of observing networkimpacts independent from the analysis system used.

In this article, we describe a methodology for evaluatingthe impact of the observations from an ensemble ofdata assimilation experiments; the impact is quantifiedas analysis-error increase due to the withholding of acertain observation type with respect to the ‘all-observation’experiment. The methodology is rather general, as itstrictly requires an ensemble of data-denial assimilativeexperiments. However, the impact results depend on thespecific configuration of the analysis system such as, forinstance, the ocean model configuration (e.g. resolutionand boundary conditions), and the data assimilationcharacteristics (e.g. the model and observational-errorcharacterisation, the strategy for data selection and theobservation pre-processing in general). Therefore, while thevalidity of the general conclusions within other analysissystem is questionable, the method succeeds in providing

c© 2012 Royal Meteorological Society Q. J. R. Meteorol. Soc. 139: 1842–1862 (2013)

1844 A. Storto et al.

an insight into the observation usage and the impact ina specific assimilation system. The methodology is thenapplied as a proof-of-concept within a global ocean 3D-Varanalysis system, in order to demonstrate the feasibility of themethod in assessing the short-range (1 to 15 days) relativeimpact on global ocean analyses of sea-station reports, Argofloats, mooring reports, expendable bathythermographs(XBTs), sea-level anomaly (SLA) observations, and sea-surface temperature (SST) measurements from space-bornemicrowave instruments within the period from January 2003to December 2005.

Section 2 illustrates the methodology for evaluating theobservations impact, in Section 3 the assimilation system andthe observing networks are described, Section 4 outlines thestrategy for generating the ensemble experiments, Section 5shows the results of this proof-of-concept, Section 6 providesa validation of the ensemble approach impact study andSection 7 summarises and discusses the main achievements.

2. Assessment of observations impact

2.1. Derivation of background-error covariances in ensemblesimulations

Variational data assimilation systems usually do notexplicitly calculate the analysis-error covariance matrix,although some 4D-Var implementations, e.g. at theEuropean Centre for Medium-range Weather Forecast(ECMWF; Fisher and Courtier, 1995), calculate theanalysis-error covariance matrix through approximaterepresentation of the Hessian of the cost function.

Nevertheless, a Monte Carlo approach to the evaluationof the analysis-error covariance matrix can be applied byrunning ensemble variational assimilation and evaluatingthe ensemble spread. This approach has been used bymany authors (e.g. Belo Pereira and Berre, 2006; Stortoand Randriamampianina, 2010b) for assessing the forecastmodel error covariances and has proved superior to othermore empirical methods.

By defining the analysis error εa as the difference betweenthe analysis and the true state of the ocean xt, and likewisefor the background error εb and the observational errorεo, the analysis error and its covariance matrix A can beexpressed as (Talagrand, 1997):

εa = (I − KH) εb + K(εo),

A = (I − KH) B (I − KH)T + KRKT,

}(1)

where B and R are the background- and the observational-error covariance matrices, respectively, H is the linearisedobservation operator and K is the usual gain matrix, givenas K = BHT(HBHT + R)−1. The error εf and covariancematrix P associated with the forecast step at a given forecasttime range can be given as

εf = Mεa + εm,

P = MAMT + Q,

}(2)

where M is the tangent-linear version of the forecast model,εm is the forecast model error and Q is its covariance matrix.Equation (2) assumes that nonlinear terms can be neglectedin the forecast error evolution.

Background-error covariances are derived in the ensemblesimulations by representing the true state of the ocean

through the ensemble mean, which is the best estimate ofthe true state of the ocean, according to both MaximumLikelihood and Minimum Variance estimation, if theforecast errors have Gaussian distribution. In the followingderivation, the subscript k indicates a generic member ofthe ensemble simulations, while the operator < ... > refersto the ensemble mean (i.e. values averaged over all theensemble members). Using ε instead of ε to denote thesimulated errors, we have in the assimilation step for thebackground error

εbk = xb

k− < xb >, (3)

and for the observational error

εok = yk− < y >= y + δ

ok− < y + δ

ok >= δ

ok , (4)

where for each member the observation vector is perturbed

with an unbiased random error δo

k ∼ N (0, R). For theanalysis estimate

xak = xb

k + K[

yk − Hxbk

], (5)

< xa > = < xb > +K[< y > − H < xb >

]. (6)

Hence, subtracting the two previous expressions and usingthe former relation for εb

k and εok , we obtain the analysis

error as

εak = xa

k− < xa >= εbk + K

[εo

k − Hεbk

]= (I − KH) εb

k + Kεok . (7)

The error εak then evolves because of the ocean model

integration. For the next assimilation step, the backgroundstate can be expressed as

xbk = Mxa

k + εkm, (8)

where εkm is a forecast model perturbation intended to

represent model error contributions. Assuming that theforecast model perturbations are unbiased, the ensemblemean background can be given as

< xb >= M < xa >, (9)

hence

εbk = xb

k− < xb >= Mεak + εm

k . (10)

In terms of covariances, if R =< δo(δ

o)T > is the

observation-error covariance matrix and Q =<εm(εm)T >

is the model perturbation covariance matrix, the analysis-error covariance matrix after the first assimilation can bewritten as

A = (I − KH) B (I − KH)T + KRKT

, (11)

and after the first forecast step the forecast-error covariancesare given by

P = MAMT + Q. (12)



The background-error covariance matrix B is equal to Pt

for t equal to the time of the subsequent assimilation step.Finally, for the following analysis-error covariances (11) isreplaced by

A = (I − KH) B (I − KH)T + KRKT. (13)

Clearly, the ensemble spread propagation of (13) and (12)equals the covariance matrix propagation of (1) and (2).

The P matrix can be computed from the datasetof differences of background model fields between theensemble mean and the N members over M assimilationand forecast runs:

P= 1

(MN − 1)×

M∑j=1

N∑k=1

(xb

k,j−1

N

N∑k=1

xbk,j

)(xb

k,j−1

N

N∑k=1

xbk,j

)T

.

(14)

Equation (14) represents a diagnostic expression for (12)that can be computed from the ensemble simulation modeloutputs. Therefore, diagnosing the model errors by means of(14) requires a proper construction of an ensemble systemthat accounts for the perturbation of both the analysis state(A) and the ocean model integrations (Q). Neglecting Qreduces to the ‘perfect model’ case where the errors areassumed to be produced only by the data assimilationsystem and propagated in time by the forecast model; onthe other hand, neglecting A corresponds to the case wheredata assimilation has no error and the error starts fromzero at each analysis time, grows with time because ofthe forecast model errors, and then is back to zero afterthe next analysis step. Clearly, these two limiting cases areboth unrealistic, suggesting the importance of the propersimultaneous perturbation of the analyses and forecasts.

Furthermore, (14) is hereafter applied by using all theensemble anomalies (differences between the ensemblemembers and the ensemble mean valid at a specific time)within a period including M assimilation and forecaststeps. The statistics therefore represent the ensemble spreadaveraged in a period, rather than a time-evolving ensemblespread, although model-error covariances evolve in time.Doing this, with a large enough number of assimilation andforecast cases, the impact of the ensemble size is less crucialwith respect to the case of time-evolving error covariances,and we have verified that, for instance, the ratio between asix-member and a two-member ensemble spread is close tounity (not shown), especially for the longest forecast ranges.

In the impact metrics definition, we assume that theconstant for which P is proportional to P depends only onthe rank deficiency of the ensemble experiments, and not onthe observation configuration. To prove this assumption, wehave verified that the ratio between ensemble experimentswith different ensemble sizes is only slightly influenced bythe observation configuration (not shown). It is thereforepossible to compare the forecast-error variances betweendifferent experiments simply by comparing their ensemblespreads.

The ensemble spread can be thought of as an estimateof the analysis-error covariance matrix, whereas the rankdeficiency of the ensemble size underestimates its absolutevalue. However, we can reasonably assume that the rankdeficiency, i.e. the ratio between the true analysis-errorvariances and the analysis ensemble spread, does not dependon the data assimilation configuration, but uniquely relieson the ensemble size and model resolution.

2.2. Impact metrics

We define the impact of a certain observation type asthe forecast-error variances increase when that observationtype is not assimilated. For sake of simplicity, we will notconsider the full covariance matrix but only its diagonalpart, i.e. the impact of the observations on spatial correlationand cross-correlations between different parameters is notaccounted for. The total forecast errorEi

t for the ith ensembleexperiment at forecast range t is defined as

Eit = tr

(SPi

tST)

, (15)

where tr is the trace operator, Pit is the forecast error and

S is a localisation operator. Each element of S is given bySij = kij

√wij and S is used for

(i) isolating the variances to study (for instance onlytemperature variances in the first 200 m depth,namely kij is equal to 1 for elements of Pi

t which referto 0–200 m temperature, and 0 elsewhere), and

(ii) weighting the forecast-error matrix according to thedifferent horizontal and vertical resolution of thegridpoints (through the coefficients wij).

Note that the ith experiment is that where all the observationsexcept the ith group are assimilated. The impactLi

t of the ithobserving system is then found as the normalised increaseof total forecast error with respect to the ‘all-observation’experiment, formally:

Lit = Ei

t − Eallt

Eallt

, (16)

where the superscript all indicates that the full set ofobservations is used.

The impact metrics defined in (16) allow one to studythe forecast-error variance increase within the ensemblesimulation period. Li

t is a dimensionless quantity whichdefines the ratio of the error increase with respect to the‘all-observation’ experiment. Nevertheless, by limiting thecomputation of Pi

t to a number of individual cases ofinterest, its computation might then focus only on particularscenarios. It is important to recall that we have assumed that

(i) model errors are Gaussian or can be approximated toGaussian without significant loss of accuracy;

(ii) the ‘all-observation’ experiment acts as reference, as itis assumed to be the experiment that leads to the bestforecasts (however, this assumption has been usedalso within adjoint-based sensitivity computationsCardinali, 2009); and

(iii) the different configurations of the observations do notlead to different proportionality coefficients betweenthe ensemble spread and the true forecast-errorvariances, i.e. the rank deficiency of the ensemblespread does not depend on the number and type ofassimilated observations, which is reasonably true asthe rank deficiency primarily relies on the ensemblesize (Raynaud et al., 2008).



The localisation operator has been introduced for the twoimportant reasons mentioned earlier. Firstly, it allows thecomputation of the impact specifically for certain horizontalareas or vertical zones. Secondly, as the resolution of aglobal domain might be very irregular, the mere use ofthe trace operator may lead to area-averaged impact resultsdominated by the features of areas at higher resolution.Thus, S normalises the error variances according to the localresolution.

Compared to the ensemble-based sensitivity metricsproposed by Liu et al. (2009), our metrics focus on theforecast step of the analysis system and are therefore suitablefor both short-range and seasonal forecasts. Furthermore,the information provided by the application of our strategydoes not limit the results to the self-sensitivity of theobservations (i.e. the sensitivity of the analyses to theobservations at their locations), but aims at depicting theglobal impact of an observing system (local and remote),which is particularly interesting for those observationtypes which are concentrated only in a specific region(e.g. mooring arrays in the tropical regions). Note also thatwe have defined the impact in data-denial experiments, thusproviding information on the importance of an observingnetwork in synergy with the other observation types.A dual formulation, where the ‘reference experiment’ isinstead an assimilation-blind ensemble simulation and theimpact is accordingly represented by the relative ensemblespread diminution, is straightforward to set up, but it isnot considered in the current work. We believe that ourdefinition of impact is more interesting for realistic impactestimates.

The normalisation in the impact metrics is arbitrary inthe sense that our goal is to study the ratio of the augmentedforecast error with respect to a reference forecast error.These metrics thus allows us to understand the percentageforecast error gain and compare this dimensionless normfor different parameters, regions or ocean depth ranges.On the other hand, choosing a relative metric implies thatthe absolute observational impact is not captured, namelyareas with a small model error will amplify the observationimpact if present, and symmetrically areas characterised bya large model error will generally tend to underestimate theobservational impact. However this is consistent with theidea of studying the percentage gain in the analysis qualityborne by an observation type.

It should be stressed that the results obtained with theapplication of (15) and (16) rely on the specificity ofthe data assimilation system in terms of observation pre-processing, the actual observation operators implementedand the definition of the background- and observation-errorcovariance matrices, and should be interpreted in this way.However, the theoretical framework is strictly based onthe reproduction of the forecast errors by an ensemble ofassimilative experiments, which can be in general set up forany analysis system, provided that the ensemble is generatedby a proper perturbation of the analysis system inputs.Finally, it should be noted that this ensemble approach maybe applied also to study the impact of the observations interms of spatial correlations or cross-correlations, althoughthis possibility has not been explored in the present study.

3. Application to a global ocean variational analysissystem

The ensemble-based methodology for observational impactstudies is applied as a proof-of-concept to a global oceanvariational assimilation system. This section provides theformulation of the analysis system and a description of theobservations assimilated for which the impact is investigated.

3.1. Formulation of the data assimilation method

In this study we have used a global ocean implementation(Storto et al., 2011) of a 3D-Var data assimilation systemcalled OceanVar (Dobricic and Pinardi, 2008) developedat the Istituto Nazionale di Geofisica e Vulcanologia(INGV) and Centro Euro-Mediterraneo per i CambiamentiClimatici (CMCC). The OceanVar system uses a FirstGuess at Appropriate Time (FGAT) formulation with anassimilation time window of 10 days, which means thatobservation departures are computed with respect to thebackground fields corresponding to the observation time.The 3D-Var scheme in its incremental formulation (Courtieret al., 1994) consists in minimising a cost function givenby

J = 1

2vTv + 1

2(d − HVv)T R−1 (d − HVv) . (17)

In this cost function, v is the control variable defined ina way that x − xb = Vv, with xb and x the backgroundand the unknown analysed ocean state, respectively, wherethe background-error covariance matrix B is equal to VVT;R is the (diagonal) observation-error covariance matrix,d = y − H(xb) is the vector of observation departures(or misfits), whose computation takes advantage of thefully nonlinear observation operator H; finally, H is thetangent-linear version of the observation operator. Thecontrol variable transformation avoids the inversion of thecovariance matrix B and preconditions the minimisationproblem. The operator V is a sequence of linear operator V =VhVv. The operator Vv consists of 10-mode bivariate verticalempirical orthogonal functions (EOFs) with temperatureand salinity covariance errors, estimated from the modelclimatological anomalies (Bellucci et al., 2007). The useof the vertical bivariate eigendecomposition ensures thatthe vertical component of V is diagonal. The choice of10 as the number of leading EOFs was chosen as atrade-off between accuracy of covariance representationand computational cost, in accordance with previousexperiences (Bellucci et al., 2007; Storto et al., 2011). VerticalEOFs are spatially varying and defined at full horizontalmodel resolution. The operator Vh acts on the verticalprofiles of temperature and salinity (Vvv) and modelshorizontal Gaussian covariances by means of a four-iteration application of a first-order recursive filter. Theanalysis xa corresponds to xb + Vv at the minimum of vin J.

The resolution of OceanVar is the same of the oceanmodel, which is OPA 8.2 (Madec et al., 1998) in its free-surface configuration at 2◦×2◦ cos(lat)×31 vertical levels,with a meridional resolution increase (up to 0.5◦) inthe tropical band. The model domain does not includeclosed seas such as the Black and the Caspian Seas. Thisrelatively coarse resolution, which is still attractive for



seasonal ensemble prediction systems and climate variabilitystudies (Alessandri et al., 2010; Masina et al., 2011; Bellucciet al., 2012), allows us to perform at an affordable cost asmany ensemble simulations as the number of observingnetworks.

The observational data consist of (i) XBTs, (ii) buoyreports (BUOY), (iii) sea-station reports (TESAC), (iv)Argo floats, (v) SLA data, and (vi) SST from the AMSR-Eand TMI instruments.∗ On top of the 3D-Var minimisation,the data assimilation system also performs

(i) a background quality check to reject observationswhose departure from model-equivalents is too large(namely thrice the sum of the observational andbackground-error variances); and

(ii) a data thinning procedure, in the horizontal for allthe observation types and in the vertical also forin situ observations, to remove reports too close toeach other, provided that observations are assumedto be uncorrelated (Storto et al., 2011).

The experiments have been conducted from June 2002 toDecember 2005, although results will be assessed for theperiod from January 2003 to December 2005 due to aninitial 6-month ensemble system spin-up period.

3.2. In situ observations

The in situ observations were taken from the ENSEMBLESdataset (Ingleby and Huddleston, 2007), version 3. Theprofiles are obtained primarily from the World OceanDatabase 2005, supplemented using data from GTSPP from1990 onwards and from the USGODAE Argo Global DataAssembly Center (GDAC) for Argo data from 1999 onwards.The processing was performed for the EU-supported projectENSEMBLES. The observational errors have been set bylinearly interpolating the error values given by Ingleby andHuddleston (2007). In this study, in situ observations aredivided into four different subgroups, also in accordancewith the ENSEMBLES dataset. These are

1. buoys (abbreviated BUO), which are the mooredbuoys joining the TAO/TRITON, the PIRATA andthe RAMA arrays† in the Pacific, Atlantic and IndianOceans, respectively, plus a very few moored buoysin coastal areas. Buoy data were daily averaged. Onlybuoy measurements of temperature and salinity areassimilated in our analysis system;

2. expendable bathythermographs (XBTs) which mea-sure temperature only and are carried by ships ofopportunity (ferries, cargo vessels, etc.). A correctionon the XBT depth was performed by Wijffels et al.(2008), who provide further details;

3. temperature, salinity and current reports fromsea stations (abbreviated TESAC or TES), whichare essentially Conductivity–Temperature–Depth

∗Advanced Microwave Scanning Radiometer for Earth ObservingSystem; Tropical Rainfall Measuring Mission Microwave Imager.†Tropical Atmosphere Ocean/TRIangle Trans-Ocean buoy Network;PredIction and Research moored Array in The Atlantic; Research MooredArray for African-Asian-Australian Monsoon Analysis and Prediction.

(CTDs) profiles from ocean stations and measuretemperature and salinity very accurately. In somecases, vertical averages were performed on CTDreports to avoid very high vertical resolution (Inglebyand Huddleston, 2007);

4. Argo profiling floats (abbreviated ARG; Davis et al.,2001), which measure temperature and salinity duringtheir descent, parking depth and ascent phases.

Ingleby and Huddleston (2007) (their Table 3) providethe vertical profiles of observational errors that are usedin the assimilation system. All the in situ sub-types usethe same vertically varying error definition, assumingthat the instrumental accuracy is less important thanthe representativeness error for our coarse-resolutionconfiguration. For temperature, observational error valuesare equal to 0.78 K at the sea surface, peaking at 75 m depth(1.00 K) and slowly decreasing to 0.30 K at 1000 m depth.The peak roughly coincides with the larger uncertaintieslinked with the mixed-layer placement. For salinity, theobservational error is set to a maximum at the sea surface(0.20 psu) and decreases towards the sea bottom (0.15, 0.10,and 0.05 psu at 150, 400 and 1000 m depth, respectively),because the largest variability of salinity at the air–seainterface is due to freshwater flux uncertainties.

3.3. Sea-level anomaly observations

Sea-level anomaly observations were supplied by CLS-AVISO. Along-track data from Jason-1, Geosat-Follow-On(GFO), ERS-2, Envisat and Topex-Poseidon are assimilatedafter the usual geophysical removals and multi-satellitecross-correction (Le Traon et al., 1998). In our assimilationscheme, the observed sea-level increment is used to correctthe thermo- and halo-steric components according tothe bivariate definition of the background-error verticalcovariances. In practice, the observation minus backgrounddeparture in observation space is used to correct the columnintegral of the water density. A mean dynamic topographydeducted from a model long-term run and adjusted toaccount for SLA biases was used to compare the observationswith the model. More details about the assimilation strategyfor the SLA observations and the observational errorcharacterisation can be found in Storto et al. (2011), wherealtimetry is proved to importantly improve the analysisof tropical subsurface temperature and salinity as well asthe tropical and subtropical circulation. The observationalerror variance (Storto et al., 2011) is defined as the sum ofthe instrumental error variance (whose root square valuevaries between 3 and 5 cm depending on the satellite), therepresentativeness error variance (whose root square valuevaries between 1 and 9 cm in areas with strong mesoscalevariability), the error variance of the observation operator,whose root square value peaks at 6 cm near the Equator,and the error variance associated with the mean dynamictopography (between 0.8 and 5 cm depending on the region).

3.4. Sea-surface temperature observations from satellites

Space-borne SST observations are retrieved from themicrowave instruments AMSR-E and TMI aboard Aquaand TRMM, respectively, and optimally interpolated ontoa 0.25◦ resolution regular grid. They were produced byRemote Sensing Systems and sponsored by the National



Oceanographic Partnership Program (NOPP), the NASAEarth Science Physical Oceanography Program. A diurnalsea-surface overwarming estimation (Gentemann et al.,2003) was subtracted from the SST data. Observationalerrors are provided by Remote Sensing Systems as outputof their optimal interpolation analysis: their values are onthe average close to 0.65 K, with an increase in coastal areasand towards the polar regions. In our assimilation scheme,SST data are assumed to be located at the shallowest modellevel. Again, a background quality-check and a thinningprocedure (Section 3.5) are applied before SST data areingested into the assimilation system. The assimilationof microwave SST observations leads to a global area-averaged RMSE decrease with respect to the experimentwithout SST data assimilation of about 0.05 ◦C for the sea-surface temperature verified against monthly means of theReynolds SST daily analyses (Reynolds et al., 2007), withan error decrease which reaches 0.3 ◦C in the SouthernOcean. The information brought by the SST observation isspread vertically and to the salinity by means of the verticalcovariances represented by the bivariate vertical EOFs. Thevertical structure of the EOFs used in this study is such thatnon-negligible temperature vertical correlations (up to 0.8)and temperature–salinity vertical correlations (up to 0.5)occur between the surface and, for instance, 150 m depth,in correspondence with the Eastern Tropical Pacific and theAntarctic Circumpolar Current (ACC). Therefore, in theseregions the impact of the assimilation of SST is significantalso at many sub-surface levels. It is a subject of currentinvestigation whether to artificially avoid this downwardpropagation of SST increments by a proper modification ofthe EOFs (e.g. with a vertical localisation operator) or totrust the statistical structure of the EOFs themselves.

3.5. Data-thinning procedures

Since the number of space-borne observations (SLA andSST) is quite large and uniform, while the model resolutionis relatively low, a data-thinning procedure is appliedto these observations. The main motivation resides inthe formulation of the 3D-Var data assimilation system,which does not account for horizontal correlations ofthe observational errors. Therefore, as shown by Liu andRabier (2002), neglecting data thinning when observationalerrors are incorrectly assumed to be spatially uncorrelatedmay lead to inaccuracies in the analysis. This seemsparticularly important for SST data assimilation, since SSTdata come from a multi-instrument objectively analyseddataset; this might in turn strengthen spatial correlations ofthe observational errors. Furthermore, if many observationsare located within the same model gridbox, the formulationof the data assimilation problem is not well-conditioned andthe minimisation may require a large number of additionaliterations (Daley, 1991, p 111). The data assimilation systemused in this study thus accounts for a thinning procedure;after that all the independent quality checks (consistency

checks, background quality check) are performed on theobservations. The thinning procedure defines horizontalboxes whose size is equal to 1.5 times the model horizontalresolution, and for each thinning box retains only theobservation closest to the analysis time. The proceduremakes use of several quality flags, and so the observationsrejected in previous quality controls are not taken intoaccount. A similar procedure is also applied to hydrographicprofiles coming from the same platform, although the impactof such a rejection is negligible with respect to the SLAand SST data thinning. For in situ data, the data-thinningprocedure is performed also in the vertical, provided thatthe vertical resolution of hydrographic profiles is in generalhigher than the vertical meshing of the model, especiallyfor data from research vessels. Approximately 30%, 70%and 75% of in situ, SLA and SST observations, respectively,are rejected by means of the observational data thinning,suggesting again that in interpreting the impact results,especially of space-borne observations, one should bear inmind the low resolution of the model and the consequentdata-thinning procedure applied. Note that the rejection ofSST exceeds that of SLA because of the higher resolution ofSST observations.

3.6. Number and distribution of observations

In Table 1, we summarise the number of assimilated data peryear, while Figure 1 shows the average number of assimilatedsingle-level observations per assimilation cycle (i.e. every tendays) within 3◦×3◦ regular gridboxes.

The number of assimilated buoy data consisted ofabout 18 000 single-level measurements per year. Theirgeographical distribution is limited to the mooring arrays,whose elements are located between 20◦S and 20◦N in all thethree Tropical Oceans, with the Pacific Ocean showing thelargest number. The vertical extension of the observationsfrom buoys is limited to about the first 200 m depth.

XBT data are mostly located in the North Pacific (from40◦N, Figure 1(b)), along coastal areas of North Americaand Australia and corresponding to transoceanic shippingroutes between Australia and Central and North America,and between North America and Europe. XBT data are inpractice missing in the South Atlantic and in the non-coastalIndian Ocean. Their single-measurement number is of about70 400 per year.

The number of TESAC observations is the smallest, withabout 23 600 assimilated observations per year, concentratedin the North Atlantic and North Pacific, with a very fewplatforms providing data in the Southern Oceans.

The number of assimilated single-level measurementsfrom Argo floats is the largest, with yearly values from3 668 000 (2003) to 8 533 000 (2005). The Argo datadistribution is very dense in the North Atlantic, NorthPacific (especially at the western boundary), in the ArabianSea and along the Atlantic cost of Africa, but less dense in

Table 1. Summary of the numbers of assimilated observations.

Year SLA SST XBT TESAC ARGO BUOYS

2003 390 987 173 065 84 471 37 072 3 668 538 19 2962004 386 078 172 789 74 061 20 218 5 491 117 17 8412005 374 105 173 850 52 709 13 477 8 533 416 19 927



Figure 1. Number of single-level observations assimilated per assimilation cycle (i.e. every 10 days) during the period 2003–2005 by observation type: (a)buoys, (b) XBT, (c) TESAC, (d) Argo, (e) SLA and (f) SST. The numbers are computed as mean assimilated observations within gridboxes of resolution3◦ × 3◦ in zonal and meridional directions. In (d), note that the scale is different from those of the other panels.

the South Pacific and very poor in the South Atlantic andsouth of 55◦S.

The distribution of assimilated SLA and SST observationsis rather uniform but slightly increases towards the Equator(Figures 1(e) and (f), respectively), due to the thinningprocedure and the fact that the meridional resolution ofthe data assimilation computational domain increases inthe tropical band. Note that, due to the assumptions usedwithin the SLA assimilation, SLA observations near theEquator (3◦N–3◦S) are rejected (Storto et al., 2011). Theannual number of assimilated SLA observations is on average383 700, while that of SST observations is 173 000.

4. Ensemble generation

From the derivation provided in Section 2, it followsthat the correct evolution of error covariances maybe obtained by perturbing the observation vector andthe ocean model. The ensemble members have beengenerated as follows: a random perturbation is added tothe observation vector within the assimilation step, whileheat flux, wind stress and parametrisation tendencies areperturbed within the forecast step. In our implementation,the ensemble size was chosen equal to six members, asa compromise between computational speed and spreadstability. However, note that, although the specification ofthe optimal ensemble size is a topic largely debated for bothensemble prediction systems (Buizza and Palmer, 1998) andflow-dependent estimation of ensemble background-errorcovariances (Raynaud et al., 2008), here our final goal is toconstruct the ensemble spread by using all the differences

between the ensemble mean and the members for the studiedperiod. Figure 2 illustrates the ensemble assimilation system.Note that after the first assimilation and forecast cycle, thebackground fields are implicitly perturbed since they comefrom a perturbed forecast step.

4.1. Perturbation of observations

In practice, an unbiased random perturbation wasadded independently to each assimilated observation.Such a perturbation was generated by standard normaldistribution random realisations, which were multipliedby the observation-specific square-root observational errorvariance. The latter is the same specified within the OceanVaranalysis system, and the reader is referred to Sections 3.2, 3.3and 3.4. This approach has been largely used in ensemblesimulation techniques (e.g. Evensen, 1994).

4.2. Perturbation of surface forcing

The configuration of the OPA 8.2 ocean model (Madecet al., 1998) surface boundary conditions used in this studyconsists of the flux formulation, namely no bulk formulaeare used at the air–sea interface. Wind stress, net downwardwater flux and net downward heat flux are taken from theECMWF operational analyses and forecasts and used by theocean model as surface forcing. The net heat flux Qs is givenby

Qs = Q0 + α (T − T0) , (18)



Figure 2. Schematic illustration of the ensemble simulation.

where Q0 is the ECMWF operational system net heat flux,T is the model SST and T0 is an observed SST, providedby the National Oceanic and Atmospheric Administration(Reynolds and Smith, 1994), α is a relaxation coefficient,set to –200 W m−2K−1, which corresponds to a restoringtime-scale of about 12 days for a 50 m mixed-layer depth.

In the current configuration of the ensemble generationstrategy, wind stress and heat flux fields are perturbed.The net water flux is not perturbed for sake of simplicity.Although this choice may lead to an underestimatedvariability of near-surface salinity, especially where thevariability of the freshwater fluxes is high like in theIntertropical Convergence Zone, Daget et al. (2009) pointedout that the globally averaged ratio between water fluxvariances and the variance of ensemble simulations is clearlyless than that of the other surface forcing.

In particular, the net heat flux is perturbed by perturbingthe observed SST data in the restoring term of (18), whilewind stress forcing is perturbed by adding a perturbationdirectly in the wind stress fields.

The perturbation of SSTs and wind stress was obtainedin these simulations by sampling differences between twoindependent SST analyses and wind stress data, which is acommonly adopted procedure within seasonal forecastingsystems (e.g. Palmer et al., 2004; Vialard et al., 2005).

Uncertainties in the SSTs were modelled from thedifferences between the Reynolds weekly SSTs analyses(Reynolds and Smith, 1994) and the UK Met Office HadleyCentre monthly SSTs analyses (Rayner et al., 2003), thelatter interpolated in time to weekly fields, within the periodOctober 1992–January 2006. Most of the error amplitude(i.e. the standard deviation of the weekly differences ofthe two SST analyses, shown in Figure 3(a)) is found inmesoscale-dominated areas such as the Gulf Stream region,the Kuroshio Current region, corresponding to the ACC,and in the upwelling regions of both the eastern Pacificand Atlantic Oceans. In these areas, the standard deviationof the dataset differences reach value of 0.7 to 1 K. Theperturbations are sampled from the SST differences, anda 7-day running mean filter is applied to correlate theperturbations in time.

The daily CORE dataset (Large and Yeager, 2004) andthe ERA-Interim dataset (Simmons et al., 2007) for theperiod 1989–2006 provided the two wind stress data fromwhich perturbations were constructed. The errors are large

(Figure 3(b)) in the Southern Hemisphere Oceans (up to0.05 N m−2) and much smaller elsewhere (except for theeastern Equatorial Pacific), probably due to the massive useof scatterometer data by both datasets and which are notavailable at high latitudes. Again, we use a 7-day runningmean filter after having re-sampled the perturbations fromthe wind stress differences. The two wind stress perturbations(zonal and meridional) are sampled from the same differenceoccurrences for sake of physical coherency.

4.3. Perturbation of the ocean model

Equation (12) suggests that, in order to simulate theforecast-error covariances, a perturbation of the oceanmodel should be introduced. Furthermore, recent studies(e.g. Zanna et al., 2012) have highlighted that the long-termperturbation of the upper ocean alone (i.e. surface forcing),may lead to unrealistically underestimated error growths andoverestimated predictability results due to the importanceof deep density anomaly for the thermohaline circulation.Several methods may be used for perturbing an ocean model;they were originally developed in the context of numericalweather prediction, and are occasionally applied to oceanor coupled models. Optimal error growth constructionby means of singular vector calculation has recently beenemerging as a tool for studies at the climate time-scale(Zanna et al., 2010), but it requires the coding of the tangent-linear and adjoint version of the ocean model, and it has notbeen applied to short-range impact studies (to the authors’knowledge). Another possible strategy consists of formingan ensemble of experiments by perturbing ocean modelparameters (e.g. values for mixed-layer parametrisation,parameters of isopycnal diffusion, etc.) as proposed byCollins et al. (2007) for studying the transient climateresponse to doubled CO2 concentration. Nevertheless, sucha strategy requires an exhaustive prior knowledge of theprobability distributions of the perturbed parameters.

Since our goal here is to introduce a realistic model physicsperturbation for simulating model error growth typicalof a background state time-scale (rather than estimatinglong-term model uncertainties or exciting the mostunstable modes), the OPA 8.2 ocean model (Madec et al.,1998) was perturbed through the ‘Stochastically PerturbedParametrisation Tendencies Scheme’ (SPPT; Palmer et al.,



45°N

(a)

(b)

15°N

15°S

45°S

140°E100°E60°E20°E20°W60°W100°W140°W

140°E100°E60°E20°E20°W60°W100°W140°W

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

45°N

15°N

15°S

45°S

140°E100°E60°E20°E20°W60°W100°W140°W

140°E100°E60°E20°E20°W60°W100°W140°W

0

5

10

15

20

25

30

35

40

45

50

Figure 3. Standard deviation of differences (a) of sea-surface temperature (K) between the Reynolds and the UKMO Hadley Centre SST analyses and(b) of zonal wind stress (10−3N m−2) between the CORE and the ERA-Interim datasets.

2009). The idea behind stochastic parametrizations is toperturb the model consistently with the model physics andmodify the ensemble mean and spread accordingly. TheSPPT scheme is a revision of the scheme of Buizza et al.(1999), which was originally conceived for the ECMWFatmospheric Ensemble Prediction System. The tendencyof the ith ensemble member ocean state Wi can beseen as the sum of non-parametrised (i.e. advective, Di)and parametrised (Zi) processes tendencies. The latter areperturbed collinearly to the unperturbed tendencies:

Wi = Di + (1 + riµ) Zi, (19)

µ being a perturbation control factor and r being theperturbation amplitude which evolves according to thefirst-order auto-regressive process:

ri(t) = (1 − ψ) ri(t − �t) +√

ψζi, (20)

where ζi a random realisation, and ψ defines a decorrelationtime τ given as ψ = 1 − exp(−�t/τ ), �t being theocean model time step. This method is straightforwardto implement in an ocean model and relies by constructionon perturbations that respond to the ocean state variability,rather than on parametrisation uncertainty calibration orcomputation of the most unstable perturbations. For ourgoal, this seems to work well enough.

In our implementation of the SPPT scheme, ζi is aGaussian random realisation with variance equal to 1. Thetendencies that are perturbed are

(i) the horizontal viscosity/diffusion for zonal andmeridional currents and temperature and salinity;

(ii) vertical mixing for zonal and meridional currents andtemperature and salinity; and

(iii) solar light penetration terms for the temperature.

The decorrelation time-scale used in the simulations wasset equal to 10 days. The distribution of the perturbationsri is univariate, i.e. the same random realisation is used forall the ocean state parameters with the aim of generatingperturbations that are more consistent with the modelphysics. Furthermore, the random realisations are defineduniformly on the globe. The control factor µ has been setequal to 1, except for the two shallowest and the two deepestmodel levels, where a smoothing is applied in order to avoidadding large perturbations where the effect of the surfaceforcing is dominant, and to avoid modifying the bottomfriction, respectively.

The ensemble spread (vertically averaged in the first100 m depth) of temperature and salinity when only theparametrisation tendencies are perturbed is contoured inFigure 4. It turns out that within the SPPT scheme, asanticipated earlier, the spread follows the ocean variability,and it reaches its maximum values in the Tropicalregions, in mesoscale areas such as the Atlantic and thePacific western boundaries and the two eastern upwellingregions. The geographical distribution of the spread indeedemulates that of the ensemble experiments; temperature andsalinity exhibit similar features, although the latter shows a



45°N

15°N

15°S

45°S

140°E100°E60°E20°E20°W60°W100°W140°W

140°E100°E60°E20°E20°W60°W100°W140°W

0.00

0.01

0.02

0.05

0.10

0.20

0.50

1.00

45°N

15°N

15°S

45°S

140°E100°E60°E20°E20°W60°W100°W140°W

140°E100°E60°E20°E20°W60°W100°W140°W

0.000

0.001

0.002

0.005

0.010

0.020

0.050

0.100

0.200

(a)

(b)

Figure 4. Ensemble spread of 0–100 m (a) temperature (K) and (b) salinity (psu) when only the perturbation of the parametrisation tendencies (SPPTscheme) is applied. The contour intervals are on a logarithmic scale to better appreciate small spread signals.

reasonably more pronounced spread corresponding to theAmazon river outflow and in the western Tropical Pacific.Note that, because of the coarse resolution of our modelconfiguration, the spread corresponding to the ACC mayappear underestimated.

To quantify the impact of the different perturbation strate-gies (observations, surface forcing and model parametriza-tions), we performed ensemble simulations retaining onlyone perturbation component at a time. These tests are sum-marised in Figure 5, where the 0–700 m vertical profilesof globally averaged ensemble spreads are shown for eachperturbation strategy. The perturbation of the observationsis by far the most important component among the pertur-bation strategies for both temperature and salinity, and mostof the ensemble spread is found to come from that. Notethat this result may also be related to the analysis system res-olution, provided that representativeness errors increase therandom perturbations associated with each observation. Forthe temperature ensemble spread, the perturbation of theobservations is followed by that of the heat forcing (i.e. SST)in the first two model levels (up to 20 m depth). Below, thewind stress perturbation is the second most important, inaccordance with the results of Heimbach et al. (2010). Notethat all the perturbations, except that of the SST, present amaximum in the temperature ensemble spread correspond-ing to the mixed-layer depth; i.e. they display a qualitativelysimilar ensemble spread profile that responds to the variabil-ity of the temperature. With regards to the salinity spread,the wind stress perturbation is again second only to theobservation perturbation, whereas the perturbations of SST

0.00

5

0.01

0

0.02

0

0.05

0

0.10

0

0.20

0

600

400

200

0(a) (b)

Dep

th (

m)

OBSHEATTAUSPPT

OBSHEATTAUSPPT

5×10

−4

1×10

−3

2×10

−3

5×10

−3

1×10

−2

2×10

−2

600

400

200

0D

epth

(m

)

Figure 5. Globally averaged profiles of ensemble spread of (a) temperature(K) and (b) salinity (psu) for the different perturbation methods. OBS =perturbation of observations; HEAT = perturbation of heat forcing throughperturbation of sea-surface temperature; TAU = perturbation of wind stressforcing; and SPPT = perturbation of parametrisation tendencies within theocean model. The scale of the ensemble spread is logarithmic.

and parametrisation tendencies exhibit a very close verticalprofile.

In Figure 6, we show the time series of the 0–600 mglobally averaged ensemble spread for both temperatureand salinity, in the case where all the observation types areassimilated and perturbed and all perturbation methods are



0.07

0.08

0.09

0.10

0.11

(a)

(b)

Tem

pera

ture

(°C

)

2002

07

2003

01

2003

07

2004

01

2004

07

2005

01

2005

07

2006

01

0.60.81.01.21.41.6

Sal

inity

(10

–2 p

su)

2002

07

2003

01

2003

07

2004

01

2004

07

2005

01

2005

07

2006

01Figure 6. Globally averaged timeseries of ensemble spread for (a)temperature and (b) salinity in the first 600 m depth. The time axis isshown as yyyymm.

used. The figure proves that the ensemble system is able todevelop a rather stable spread after a six-month (July 2002 toDecember 2002) spin-up period, provided that all ensemblemembers started from the same initial conditions given bya deterministic reanalysis from Storto et al. (2011) valid forJune 2002. Because of that, the impact results presentedin the following section are computed for the period fromJanuary 2003 to December 2005.

5. Results

In applying the impact metrics to our reanalysis system, theocean state vector is given by temperature and salinity inthe 3D ocean domain and sea-surface height (SSH) in the2D domain. Note that only temperature and salinity arecorrected by the analysis system, so that the impact of theobservations on the SSH is induced only by the baroclinicadjustments in the ocean model caused by the analysisincrements. Note also that the impact on sea level is assessedon the SSH (absolute value) and not on its anomaly.

We have defined two vertical regions of the ocean, given by

(i) a near-surface sunlight zone from 0 to 100 mdepth, which includes the mixed layer except withinhigh-latitude regions and

(ii) a sub-surface zone between 100 and 300 m depth.

We have also defined three latitude-dependent areas:

(i) the southern Extratropics (ET) from 60◦S to 20◦S,

(ii) the Tropics from 20◦S to 20◦N, and

(iii) the northern Extratropics from 20◦N and 60◦N.

Within the latter definitions, which will be used in theremaining part of this section to summarise the resultsas area-averaged histograms, semi-enclosed seas like the

Mediterranean and the Red Seas have been masked out asthese areas are not of interest for our coarse-resolution studyand may jeopardise the area-averaged results. Note that allthe previous definitions should be intended as a subjectiveapplication of the localisation operator S (in (15)).

5.1. Impact on temperature

Basin-averaged impact values of temperature (Figure 7)indicate that, in the first 100 m depth, SLA observationsare the most responsible for forecast error increases whennot assimilated. This seems a very interesting result as SLAdata do not provide a direct measurement of temperature,unlike the other observing systems studied. Note also thatimpact metrics maxima are about two to three times largerin the Tropics than in the Extratropics, confirming thegreat importance of observing networks there. In Figure 8,we compare the map of temperature forecast impact forArgo and SLA data in the first 100 m depth. Such mapsshould be considered as a particular application of (15)and (16), where the impact is computed separately foreach horizontal gridpoint. It is possible to see that thesetwo observing systems are complementary, as altimetrymostly impacts the equatorial area, while the profilingfloats are very important in the Extratropics, especiallyin the Gulf Stream and Kuroshio current regions wherethe temperature standard deviations are rather high (notshown); a similar result was also found by Guinehutet al. (2004). It should be borne in mind that the impact isdefined with respect to the ‘all-observation’ model errors,thus this definition tends by construction to underestimatethe observational impact in areas of large model errors(e.g. dynamically intense areas, Tropics) and amplify theimpact in areas of small model variability (mostly polarregions). For instance, sea-level anomaly observations havean important absolute impact in some dynamically intenseareas corresponding to the Kuroshio Current or the GulfStream, but the gain in accuracy introduced by suchobservations is much smaller than the model error itself.To exemplify, Figure 8(c) reports the temperature ensemblespread in the first 100 m depth for the ‘all-observation’experiment. The very high impact close to Antarctica isclearly due to the fact that the ‘all-observation’ ensemblespread is very small there. Consistently, we have foundthat in some upwelling regions (eastern tropical Pacific andAtlantic) where the ‘all-observation’ experiment exhibitslarge errors (up to 1.3 K in the near-surface eastern TropicalPacific), the two main observing systems are not able tosignificantly impact the scores. This is likely to be due tothe fact that in those areas the main source of error israther the overestimated vertical mixing that enhances thedownward propagation of zonal momentum (Vialard et al.,2003) and the ERA-40 underestimation of the wind stressforcing (Chelton et al., 2004; Risien and Chelton, 2008).These two factors contribute to the systematically weakerEquatorial Undercurrent simulated by the coarse-resolutionocean model, which the data assimilation system is unableto significantly correct (Bellucci et al., 2007; Masina et al.,2011).

It is interesting to note that, for longer forecast ranges(i.e. 10- and 15-day), the impact of sea stations and XBTson near-surface temperature becomes much larger in theTropical region, within all the Oceans. In these areas the0–100 m vertically averaged relative forecast errors when



AR

G

BU

O

SLA

SS

T

TE

S

XB

T

0.00.20.40.60.81.01.21.4

(a)

(c)

(e)

(b)

(d)

(f)

AR

G

BU

O

SLA

SS

T

TE

S

XB

T

0.0

0.5

1.0

1.5

AR

G

BU

O

SLA

SS

T

TE

S

XB

T

01234

AR

G

BU

O

SLA

SS

T

TE

S

XB

T

0.0

0.5

1.0

1.5A

RG

BU

O

SLA

SS

T

TE

S

XB

T

0.00.51.01.52.0

AR

G

BU

O

SLA

SS

T

TE

S

XB

T

0.00.51.01.52.0

Figure 7. Impact on temperature forecasts (as defined in the text) of the main observing systems for three latitudinal bands and for two vertical regions.The latitudinal bands are (a, b) southern Extratropics, 60–20◦S; (c, d) Tropics, 20◦S–20◦N; and (e, f) northern Extratropics, 20–60◦N. The verticalregions are (a, c, e) 0–100 m and (b, d, f) 100–300 m. ARG denotes Argo floats, BUO buoys, SLA sea-level anomalies, SST remotely sensed sea-surfacetemperature, TES sea-station reports, and XBT expendable bathythermographs. For each observing system, the errors are reported for 1-day (white),5-day (light grey), 10-day (dark grey) and 15-day (black) forecasts.

TESAC are not assimilated are four times bigger than theerrors of the ‘all-observations’ experiment, and locally theyare even 16 times bigger (not shown). On average, SLA,TESAC and XBT become very critical in reducing theforecast errors in these regions. In particular, the TESACimpact grows from negligible values at the 1-day forecastrange to the very large values at the 15-day forecast rangenear the Equator. This feature is confined to the mixed layerand is not seen below 100 m depth.

For deeper zones, Argo floats clearly present the mostimportant impact, especially in the northern Extratropics,while buoys, SLA observations and SST measurements fromsatellite have comparable scores.

In general, space-borne SST observations do not havea large impact compared to the other observing systems,although their impact is higher in the 0–100 m SouthernOcean, due to the very small number of buoys, sea-stationreports and XBTs in this region. Also SSTs have an impactcomparable with the other observation types except Argofloats below 100 m. This feature relies on the rather broadsurface–subsurface vertical correlations, especially in thetropical eastern boundaries and in the ACC.

By looking at the area-averaged vertical profiles oftemperature impact for SLA and Argo floats (Figure 9),it can be noticed that the separation between the impacts ofthese two observing networks, i.e. the depth where the impactof Argo starts to exceed that of SLA, is located at around120 to 150 m depth, except for the northern Extratropicswhere the separation is shallower and where at day 1 of theforecast the two observation types share the same profilestructure within the first 100 m. Importantly, the relativeimpact of SLA observations grows with the forecast range(Figure 9 shows only the profiles for 1-day and 15-dayforecast ranges), unlike Argo floats, whose impacts seemconstant in time. Note also that the structures of the two

impact profiles differ significantly in the first 150 m depth;this is due to the fact that, while the observational errorof Argo temperature measurements reaches a maximum at40 m depth on average (Ingleby and Huddleston, 2007),the bivariate assimilation of SLA, driven by the verticalstructure of temperature and salinity background-errorcovariances, induces a maximum of the temperature analysisincrement caused by SLA observations corresponding to thethermocline (Storto et al., 2011). The penetration of the SLAimpact up to 300 m depth in the northern Extratropics canindeed be partly justified because of the deeper mixed-layerdepth at high latitudes.

5.2. Impact on salinity

The uniqueness of the Argo network in providingglobal-scale information on salinity has been repeatedlyacknowledged by many authors (e.g. Oke et al., 2009).From the salinity impact results, Argo floats clearly providethe greatest impact at all depths and regions (Figure 10).The impact of Argo becomes larger with depth. It isparticularly large in the North Atlantic and the North Pacific(Figure 11(a)), probably due to the high number of availableobservations. Due to the deployment of mooring arrays,the buoy observational group also impacts the salinity,noticeably in the Atlantic equatorial region between 10and 20◦S (Figure 11(b)). Both these observation typescontribute to the error decrease in the areas where thesalinity uncertainty is higher (not shown), which are theregion influenced by the Amazon river outflow, in theeastern equatorial upwelling areas, and corresponding tothe Indonesian Throughflow. Note also that the modelsalinity error decreases with depth (not shown), because ofthe dominant uncertainty in water fluxes exchanges at theair–sea interface.



45°N

(a)

(b)

(c)

15°N

15°S

45°S

140°E100°E60°E20°E20°W60°W100°W140°W

140°E100°E60°E20°E20°W60°W100°W140°W

0

1

2

3

4

5

6

7

8

45°N

15°N

15°S

45°S

140°E100°E60°E20°E20°W60°W100°W140°W

140°E100°E60°E20°E20°W60°W100°W140°W

0

1

2

3

4

5

6

7

8

45°N

15°N

15°S

45°S

140°E100°E60°E20°E20°W60°W100°W140°W

140°E100°E60°E20°E20°W60°W100°W140°W

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0.45

0.50

Figure 8. 0–100 m temperature impact maps for (a) Argo and (b) SLA at forecast range 5 days. (c) shows the ‘all-observation’ ensemble spread for the0–100 m temperature (K).

In the Southern Hemisphere, where the number ofArgo floats is relatively smaller for the period consideredhere, an important impact is also found from space-borneobservations (SLA and SST), whose coverage is obviouslymore regular. In the tropical regions, the non-negligiblevertical auto- and cross- correlations of surface parameterswith deeper levels, as found in the upwelling areas of theAtlantic and Pacific Oceans, increase the impact of SLAsand SST observations – further to the mooring arrays– in the range 100 to 300 m depth, bringing them to havean impact comparable to Argo floats. Note also that thelarge impact of SST data on near-surface salinity, as in theTropics and southern Extratropics, is non-trivial. The mainreason why SST data have an impact on near-surface salinityresides in the cross-correlation structure of the background-error covariances used in this study, thus suggesting that

such an impact strongly depends on the data assimilationconfiguration. In particular, the set of background-errorcovariances derived from climatological anomalies and usedhere exhibits a quite strong coupling between SST andsea-surface salinity (not shown), which has similar spatialpatterns to that of Chang et al. (2011), and can be explainedby baroclinic modes (vertical displacement of the entirewater column), air–sea interactions (mainly freshwaterfluxes) and other processes such as vertical mixing.

5.3. Impact on sea-surface height

In interpreting the impact metrics for the SSH, it shouldbe borne in mind that in the present configuration ofOceanVar, SSH corrections are only induced by those oftemperature and salinity within the forecast model step. The



0

Dep

th (

m)

500

450

400

350

300

250

200

150

100

50

(a) (b) (c)0

ARGSLA

Dep

th (

m)

500

450

400

350

300

250

200

150

100

50

0

ARGSLA

0.0 1.0 2.0 3.0

Dep

th (

m)

500

450

400

350

300

250

200

150

100

50

0

ARGSLA

1 2 3 0 1 32 54

Figure 9. Vertical profiles of temperature impact for (a) the southern Extratropics, (b) the Tropics, and (c) the northern Extratropics, for Argo floatsand sea-level anomalies. Solid (dashed) lines show the 1-day (15-day) forecast impact.

AR

G

BU

O

SLA

SS

T

TE

S

XB

T

0.00.20.40.60.81.0

(a) (b)

(c) (d)

(e) (f)

AR

G

BU

O

SLA

SS

T

TE

S

XB

T

0.00.20.40.60.81.01.2

AR

G

BU

O

SLA

SS

T

TE

S

XB

T

0.00.20.40.60.8

AR

G

BU

O

SLA

SS

T

TE

S

XB

T

0.0

0.5

1.0

1.5

AR

G

BU

O

SLA

SS

T

TE

S

XB

T

0.00.51.01.52.02.53.0

AR

G

BU

O

SLA

SS

T

TE

S

XB

T

0.00.51.01.52.02.53.0

Figure 10. As Figure 7, but for salinity.

impact results on the SSH indicate that SLA observationsare largely the most impacting observing system, followedby Argo floats (Figure 12). This result holds for all the threelatitudinal bands studied, although the impact of Argo floatsin the northern Extratropics has values very close to thoseof SLA observations. The ensemble spread of SSH reach itsmaxima in mesoscale areas and in the Tropical Pacific.

From the impact maps of Argo and SLA (Figure 13),it appears that most of the impact of SLA observations isin the 15◦S and 15◦N latitude bands in both the Pacificand the Atlantic Oceans, and in the 50 to 60◦N bandcorresponding to the ACC, the latter due to the veryregular availability of data in this eddy-dominated region,unlike the other observation types. On the contrary, Argofloats manifest their largest impact in the Gulf Stream andKuroshio regions, where the impact of SLA is weaker. It isinteresting to note that such regions are characterised by the

highest sea-level variability, so that both the mean dynamictopography uncertainty and the representativeness errorare much larger than in other regions (Storto et al., 2011).Although SLA observations are acknowledged to be theonly observing network capable of representing mesoscalevariability (e.g. Fu et al., 2010), this ability is hard to capturewith the resolution of our model and the consequent datathinning that retains one observation per altimetric satelliteevery 270 km approximately (e.g. at the latitudes of the GulfStream and Kuroshio Current regions). This explains whythe Argo impact is, in relative terms, higher than that of SLAobservations. Furthermore, the impact of SLA on SSH seemsdifferent from that on temperature (Figure 8(b)), althoughthe data assimilation system corrects only temperature andsalinity and thus the impact on SSH is induced by baroclinicadjustments. This might be explained by examining theimpact of SLA on temperature averaged in the first 1500 m



45°N

(a)

(b)

15°N

15°S

45°S

140°E100°E60°E20°E20°W60°W100°W140°W

140°E100°E60°E20°E20°W60°W100°W140°W

0

2

4

6

8

10

12

14

16

18

20

45°N

15°N

15°S

45°S

140°E100°E60°E20°E20°W60°W100°W140°W

140°E100°E60°E20°E20°W60°W100°W140°W

0

2

4

6

8

10

12

14

16

18

20

Figure 11. 100–300 m salinity impact maps for (a) Argo and (b) buoys at forecast range 10 days.

AR

G

BU

O

SLA

SS

T

TE

S

XB

T

0.00.20.40.60.81.0

(a)

(b)

(c)

AR

G

BU

O

SLA

SS

T

TE

S

XB

T

0.00.20.40.60.81.01.2

AR

G

BU

O

SLA

SS

T

TE

S

XB

T

0.00.20.40.60.81.01.2

Figure 12. As Figure 7, but for sea-surface height.

depth (not shown), which looks very similar to that onSSH (Figure 13(a)), suggesting that the impact in the first100 m depth is not necessarily representative of the column-integrated baroclinic corrections that produce variations inthe SSH.

Note that the impact of SLA is almost zero at the Equator;this is because, in the 3◦S–3◦N latitudinal band, SLAobservations are not assimilated, since our SLA assimilationstrategy relies on geostrophic equilibrium (Storto et al.,2011).

6. An a posteriori qualitative validation

The validation of the ensemble strategy to assess observa-tional impact is in general very difficult due to the lackof uniformly distributed and independent verifying obser-vations. Nevertheless, gridded altimetry data produced anddistributed by AVISO have been recently reprocessed at dailytemporal resolution. They therefore provide a set of veri-fying observations against which daily fields of SSH can beverified, and the ensemble assessment of observation impactcan be validated. The daily reprocessed altimetry observa-tions represent an independent verifying dataset when theverified forecast range is greater than the assimilation timewindow, namely, in our analysis configuration, for fore-cast ranges greater than 5 days, assuming that the temporalobservational correlation introduced by the satellite cross-calibration and bias-correction (Le Traon et al., 1998) isnegligible. In a way similar to (16), we show in Figure 14 therelative RMSE increase when SLA and Argo are not assimi-lated, which can be compared to Figure 13. The comparisonbetween the ensemble spread and the RMSE (not shown),both when all observations are assimilated, indicates thatthe RMSE signal is dominant in mesoscale areas (Kuroshio,Gulf Stream and ACC regions), while the ensemble spread



45°N

(a)

(b)

15°N

15°S

45°S

140°E100°E60°E20°E20°W60°W100°W140°W

140°E100°E60°E20°E20°W60°W100°W140°W

0

1

2

3

4

5

6

7

8

45°N

15°N

15°S

45°S

140°E100°E60°E20°E20°W60°W100°W140°W

140°E100°E60°E20°E20°W60°W100°W140°W

0

1

2

3

4

5

6

7

8

Figure 13. Sea-surface height impact maps for (a) SLA and (b) Argo at forecast range 15 days.

is non-negligible not only in these regions but also in theEastern Tropical Pacific, east of Australia and in the CentralIndian Ocean, thus suggesting that the signal in the ensemblespread is affected more than the RMSE by the ocean stateuncertainty. The latter is also strongly influenced by theuncertainty in the mean dynamic topography. In terms ofrelative RMSE increase, that from SLA presents very similarspatial features to those of our impact assessment, indicat-ing in particular that SLA observations are very importantin subtropical areas, especially in the Atlantic and PacificSouthern Oceans (15–40◦S), in the regions east of Brazil,Australia and Madagascar, and corresponding to the ACCand the edge of the Indian Ocean gyre. The RMSE increasefor Argo floats is particularly strong at high latitudes in theNorth Pacific and Atlantic Oceans (40–60◦N) and in theACC, especially in the Weddell Sea, which agrees with theimpact maps, although these show only a small impact inthe Indian and Kuroshio regions.

7. Summary and discussion

In order to assess the impact of different observingsystems assimilated within a global analysis system, wehave developed a method which is based on performingensemble data-denial assimilative experiments. The errorvariance is assumed to be Gaussian and proportional to theensemble spread. Thus, the impact of a certain group ofobservations can be quantified through the forecast-errorvariance increase when that group is withheld from theassimilation system. The method is rather simple, and

might be applied to any assimilation system, althoughwe stress that results are not independent of the dataassimilation system mainly in terms of observation operatorchoice, observation preprocessing (e.g. quality check,data thinning) and background- and observational-errorcovariance specification. Nevertheless, it does not requireany tangent-linear assumption or simplified physics packageand provides impact results in the full model space. Itcan also be seen as an extension of traditional OSEs,with respect to which the ensemble experiments providevaluable information on the error variances within thefull model space. However, compared to adjoint sensitivitymethods, our approach is considerably more expensive froma computational point of view, since it needs as many modelintegrations as the ensemble size, whilst adjoint methodsrequire just one backward model integration per sensitivitystudy. As a proof-of-concept, the method was appliedto a large-scale global ocean analysis system (horizontalresolution 0.5 to 2◦). The ensemble members are generatedby perturbing the observations along with surface forcingfields and model parametrisation tendencies. Results werediagnosed for the period from January 2003 to December2005. We found that SLA observations dominate the impactresults in the first 100 m depth for temperature, especiallyin the equatorial area, and SSH. In some upwelling areascharacterised by wind-driven circulations, the impact of theobservations is found smaller than in other areas, due tothe fact that the main sources of error there reside in thesurface forcing and model vertical mixing rather than aninadequacy of the observing system. The definition of the



45°N

(a)

(b)

15°N

15°S

45°S

140°E100°E60°E20°E20°W60°W100°W140°W

140°E100°E60°E20°E20°W60°W100°W140°W0.00.20.40.60.81.01.21.41.61.82.02.22.42.62.8

45°N

15°N

15°S

45°S

140°E100°E60°E20°E20°W60°W100°W140°W

140°E100°E60°E20°E20°W60°W100°W140°W0.00.20.40.60.81.01.21.41.61.82.02.22.42.62.8

Figure 14. Sea-surface height 10-day relative root mean square error increase when (a) SLA and (b) Argo are not assimilated.

impact metrics is indeed such that areas characterised bylarger variability (e.g. eastern Tropical Pacific, due to thecoarse vertical and horizontal meshing and to the difficultyin the vertical physics parametrizations) than that borne bythe observations, generally exhibit a very small impact ofthe various observing types. Salinity and tropical circulationimpact results are led by the network of Argo floats, as wellas for all parameters below the mixed layer. The impacton SSH has also been validated with partly independentobservations of relative RMSE increase when Argo or SLAare not assimilated, showing a good agreement in the spatialfeatures of the impact assessment.

Although the main goal of this study is the demonstrationof the ensemble strategy for evaluating the observationalimpact rather than the provision of guidelines for optimalobserving network planning and sustaining, we were ableto detect many general features of the ocean observingsystem and of the INGV/CMCC analysis system. However,our model design (given the horizontal resolution and,accordingly, the observation selection procedures) doesnot allow direct implications of our study for operationalpurposes.

As a general remark, we found that the impact definedby (16) is in practice always positive for all the observationtypes. This means that the analysis system optimally exploitsthe different sets of observations and all the observation typescontribute to the model error decrease. A negative value forthe impact would in fact correspond to the case where modelerror decreases with the withholding of some observations.The long-term averaged statistics never present such a case.

Although no information is here provided on individualcase-study impacts, which might highlight the negativeimpact of certain observations, this result demonstrates thatthe time-averaged impact of all the observations is positivewithin the period 2003–2005, proves the robustness of ourvariational assimilation system, and provides a posteriorconfirmation that the ‘all-observation’ experiment may beused as reference experiment.

The bivariate formulation of the assimilation of SLAobservations is proved dramatically important for theanalysis system. In particular for the Tropical region,altimetry has a very large impact on the temperature ieldswithin the mixed layer. This is in agreement with the findingsof Balmaseda and Anderson (2009) for an ocean system witha very similar resolution. On salinity fields, this feature is lessvisible and in situ platforms which directly provide salinitymeasurements (Argo floats, buoys and less importantlyTESAC) exhibit a greater impact.

In the Southern Ocean, where the availability of in situobservations is very small, we found that both SLA andSST observations have a larger impact on temperature, SSHand even salinity. This seems a reasonable result, providedthat the lack of other observations make their impact largerand, when the mixed layer is deep as in the ACC during theboreal summer, also SST provide information that penetrateto depth. However, with the model configuration used inthis study, it was not possible to appreciate the impact of SLAon mesoscale variability, which might also be important inthose regions (Oke and Schiller, 2007). This result is clearlyvalid within our model configuration only, provided that



0.0

0.5

1.0

1.5

2.0

2.5

3.0

3

(a) (b)

(c) (d)

(e) (f)

12 15

−40

−20

0

20

40

−50

−30

−10

10

30

50

Forecast days

Latit

ude

−40

−20

0

20

40

−50

−30

−10

10

30

50

Latit

ude

0

2

4

6

8

0 100 200 300 400

Depth (m)

6 9

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3 12 15

−40

−20

0

20

40

−50

−30

−10

10

30

50

Forecast days

Latit

ude

−40

−20

0

20

40

−50

−30

−10

10

30

50

Latit

ude

0

2

4

6

8

0 100 200 300 400

Depth (m)

6 9

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3 12 15

−40

−20

0

20

40

−50

−30

−10

10

30

50

Forecast days

Latit

ude

−40

−20

0

20

40

−50

−30

−10

10

30

50

Latit

ude

0

2

4

6

8

0 100 200 300 400

Depth (m)

6 9

Figure 15. Impact of (a) sea-level anomaly observations, (c) Argo floats and (e) TESAC versus latitude and forecast range for the 0–100 m subsurfacetemperature. (b, d, f) show impact versus latitude and depth.

the coarse resolution requires a massive data thinning ofspace-borne observations. Since the data thinning is muchless important for the Argo network, Argo observationsbecome the most numerous and, consequently, the mostimportant in some dynamically intense areas (Kuroshio andGulf Stream regions) where Argo floats are constantly andintensely deployed, while the model resolution does notallow us to appreciate the potential of SLA observationsin eddy-dominated regions. It is likely that with an eddy-resolving model this result will reverse because of the lessaggressive thinning of altimetry data; the latter observationswill then produce the largest impact in eddy-dominatedareas.

The impact has been studied without any indication ofseasonality, which is important for surface observations athigh latitudes due to the strong seasonality of the mixed-layer depth which in turn affects the vertical correlationstructure. However, the extension of the results to thisaspect of observation impact is very easy to achieve and willbe investigated in the future.

Another feature we found is the growth of impact withtime for XBTs and TESAC observations, which is particularlyvisible for temperature fields. In contrast to Argo floats,XBTs and TESAC exhibit an impact growing almost linearlyfrom 1 to 15 days of the forecast. Figure 15(a, c, e) comparethe impact, by latitude and forecast range, of Argo, SLAand TESAC for the temperature in the first 100 m. Clearly,the TESAC impact grows at all latitudes, while that ofArgo keeps almost constant over the forecast ranges. Alsoaltimetry shows an impact growing with time, especiallyin subtropical areas. Similar results were found for the

other parameters (salinity and SSH). This finding confirmsthe importance of the network of XBTs and TESAC forbringing the information to longer time-scales in our system,although it is questionable whether such a conclusion canbe generalised since it may depend on the model resolutionand ensemble size used in this study.

Unlike the other observing systems, Argo floats are theonly observation type capable of providing an impactwhich penetrates deep below the mixed layer, due totheir vertical coverage, whilst all the other observationsshow their strongest impact within the first 50 to 100 m(Figure 15(b, d, f) where the temperature impact versuslatitude and depth are compared for Argo, SLA and TESAC).Therefore, Argo profiling floats might seem to be themost important network for studies aimed at representingthe integrated ocean heat and freshwater budget and thethermohaline circulation. However, the ensemble size andthe resolution of the system are probably too limitedto simulate and reproduce many important horizontaland vertical dynamical features of e.g. mesoscale, eddy-dominated or shallow and coastal areas, from which someimpact results clearly suffer. The impact growing with timeof XBTs and TESAC, or the small impact of all in situobserving systems below the mixed layer except for the Argofloats should be therefore considered with caution.

We have also found that, in general, the impact of SLAand SST seems to penetrate rather deep corresponding tothe eastern upwelling regions of the Tropical Atlantic andPacific. As already mentioned, this seems to be directlyconnected with the particular structure of the verticalcovariances, and it was not clear whether this feature



was realistic or an artifact. By running an ensemble ofsimulations with the sole perturbation of the heat flux (notshown), we have been able to identify the background-errorvertical covariances as responsible for this behaviour, as thecorresponding ensemble spread does not penetrate as deepas the SST impact. It is then questionable if the currentconstruction of vertical EOFs, relying on climatologicalanomalies, and therefore on the model field variances atmonthly time-scales rather than model error variances,is still valid in these upwelling regions. We have indeedfound that the structure of vertical EOFs constructedfrom the differences between the ensemble mean and theensemble members for the ‘all-observation’ experiment doesnot reproduce such a vertical correlation in the upwellingregions, while remaining very close to the EOFs deductedfrom climatological anomalies in the rest of the global ocean.Obviously, this result suggests further investigation of anoptimal method for the definition of the vertical EOFs,and demonstrates that the ensemble-based observationimpact method provides a complete insight into the dataassimilation system.

In this study we have focused on forecast time-scalestypical of operational oceanography, detecting the relativeobservation impact up to two weeks of forecast range. Anatural extension of this work will be the study of theimpact at seasonal scale, namely by extending the forecasttime from the perturbed analysis up to several monthsin order to evaluate the potential benefits of the differentocean observing systems for (coupled) seasonal predictionsystems. Similarly, an increase of the horizontal resolutionup to eddy-permitting scales is planned in order to assessresults for an operational-like ocean model configuration.Finally, another natural extension will be the study of theimpact of the different observing systems on many crucialintegrated quantities such as the volume, freshwater andheat transports and contents, which is straightforward toevaluate with the ensemble-based method.

Acknowledgements

This work has been funded by the European Commis-sion project MyOcean and its follow-up MyOcean2 inthe framework of global ocean reanalysis production.The altimeter products were produced by Ssalto/Duacsand distributed by Aviso, with support from CNES(http://www.aviso.oceanobs.com/duacs). The EN3 subsur-face ocean temperature and salinity data were collected,quality-controlled and distributed by the UK Met OfficeHadley Centre. TMI and AMSR-E data were produced byRemote Sensing Systems and sponsored by the NASA EarthScience MEaSUREs DISCOVER Project and the AMSR-EScience Team. Data are available at http://www.remss.com.The authors want to thank the AVISO team for support inthe use of SLA data and Dr Simon Good (UK Met Office) forsupport in the use of the EN3 dataset. The authors are alsograteful to two anonymous reviewers and to the Editor forfruitful comments and suggestions that led to an improvedversion of the manuscript.

References

Alessandri A, Borrelli S, Masina S, Cherchi A, Gualdi S, Navarra A,Di Pietro P, Carril A. 2010. The INGV-CMCC Seasonal PredictionSystem: improved ocean initial conditions. Mon. Weather Rev. 138:2930–2952.

Alves O, Balmaseda M, Anderson D, Stockdale T. 2004. Sensitivity ofdynamical seasonal forecasts to ocean initial conditions. Q. J. R.Meteorol. Soc. 130: 647–668.

Arnold CP Jr, Dey CH. 1986. Observing-Systems SimulationExperiments: Past, present, and future. Bull. Amer. Meteorol. Soc.67: 687–695.

Balmaseda MA, Anderson D. 2009. Impact of initialization strategies andobservations on seasonal forecast skill. Geophys. Res. Lett. 36: L01701.

Balmaseda MA, Anderson D, Vidard A. 2007. Impact of Argo on analysesof the global ocean. Geophys. Res. Lett. 34: L16605.

Bellucci A, Masina S, Di Pietro P, Navarra A. 2007. Using temperaturesalinity relations in a global ocean implementation of a multivariatedata assimilation scheme. Mon. Weather Rev. 135: 3785–3807.

Bellucci A, Gualdi S, Masina S, Storto A, Scoccimarro E, Cagnazzo C,Fogli P, Manzini E, Navarra A. 2012. Decadal climate predictionswith a coupled OAGCM initialized with oceanic reanalyses. ClimateDyn. DOI:10.1007/s00382-012-1468-z.

Belo Pereira M, Berre L. 2006. The use of an ensemble approach to studythe background-error covariances in a global NWP model. Mon.Weather Rev. 134: 2466–2489.

Bouttier F, Kelly G. 2001. Observing system experiments in theECMWF 4D-Var data assimilation system. Q. J. R. Meteorol. Soc.127: 1431–1454.

Buizza R, Palmer TN. 1998. Impact of ensemble size on ensembleprediction. Mon. Weather Rev. 126: 2503–2518.

Buizza R, Miller M, Palmer TN. 1999. Stochastic representation of modeluncertainties in the ECMWF Ensemble Prediction System. Q. J. R.Meteorol. Soc. 125: 2887–2908.

Cardinali C. 2009. Monitoring the observation impact on the short-rangeforecast. Q. J. R. Meteorol. Soc. 135: 239–250.

Carton JA, Santorelli A. 2009. Global decadal upper-ocean heat contentas viewed in nine analyses. J. Climate 21: 6015–6035.

Chang Y-S, Rosati A, Zhang S. 2011. A construction of pseudo salinityprofiles for the global ocean: Method and evaluation. J. Geophys. Res.116: C02002, DOI: 10.1029/2010JC006386

Chelton D, Schlax M, Freilich M, Milliff R. 2004. Satellite measurementsreveal persistent small-scale features in ocean winds. Science 303:978–983.

Collins M, Brierley CM, MacVean M, Booth B, Harris GR. 2007. Thesensitivity of the rate of transient climate change to ocean physicsperturbations. J. Climate 20: 2315–2320.

Courtier P, Thepaut J-N, Hollingsworth A. 1994. A strategy foroperational implementation of 4D-Var, using an incrementalapproach. Q. J. R. Meteorol. Soc. 120: 1367–1387.

Daget N, Weaver AT, Balmaseda MA. 2009. Ensemble estimation ofbackground-error variances in a three-dimensional variational dataassimilation system for the global ocean. Q. J. R. Meteorol. Soc. 135:1071–1094.

Daley R. 1991. Atmospheric data analysis. Cambridge University Press:Cambridge, UK.

Davis RE, Sherman JT, Dufour J. 2001. Profiling ALACEs and otheradvances in autonomous subsurface floats. J. Atmos. Oceanic Technol.18: 982–993.

Dobricic S, Pinardi N. 2008. An oceanographic three-dimensionalvariational data assimilation scheme. Ocean Modelling 22: 89–105.

Evensen G. 1994. Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast errorstatistics. J. Geophys. Res. 99: 143–162.

Fisher M, Courtier P. 1995. ‘Estimating the covariance matrix of analysisand forecast error in variational data assimilation’. Tech. Memo. No.220. ECMWF: Reading, UK.

Fu LL, Chelton D, Le Traon P-Y, Morrow R. 2010. Eddy dynamics fromsatellite altimetry. Oceanography 23: 14–25.

Gentemann C, Donlon CJ, Stuart-Menteth A, Wentz FJ. 2003. Diurnalsignals in satellite sea surface temperature measurements. Geophys.Res. Lett. 30: 1140–1143.

Guinehut S, Le Traon P-Y, Larnicol G, Phillips S. 2004. Combining Argoand remote-sensing data to estimate the ocean three-dimensionaltemperature fields: A first approach based on simulated observations.J. Marine Syst. 46: 85–98.

Heimbach P, Forget G, Ponte R, Wunsch C. 2010. Observationalrequirements for global-scale ocean climate analysis: Lessons fromocean state estimation. Community White Paper. In Proceedingsof OceanObs’09: Sustained Ocean Observations and Information forSociety, Hall J, Harrison DE, Stammer D. (eds). Venice, Italy,21–25 September 2009. ESA Publication WPP-306, Vol. 2, DOI:10.5270/OceanObs09.cwp.42



Huang B, Xue Y, Behringer D. 2008. Impacts of Argo salinity in theNCEP Global Ocean Data Assimilation System: The tropical IndianOcean. J. Geophys. Res. 113: C08002.

Ingleby B, Huddleston M. 2007. Quality control of ocean temperatureand salinity profiles – Historical and real-time data. J. Marine Syst.65: 158–175.

Ishii M, Kimoto M, Kachi M. 2003. Historical ocean subsurfacetemperature analysis with error estimates. Mon. Weather Rev. 131:51–73.

Kleeman R, Moore AM, Smith NR. 1995. Assimilation of subsurfacethermal data into a simple ocean model for the initialization ofan intermediate tropical coupled ocean–atmosphere forecast model.Mon. Weather Rev. 123: 3103–3113.

Kohl A, Stammer D, Cornulle B. 2007. Interannual to decadal changesin the ECCO global synthesis. J. Phys. Oceanogr. 37: 313–337.

Langland RH, Baker NL. 2004. Estimation of observation impact usingthe NRL atmospheric variational data assimilation adjoint system.Tellus 56A: 189–201.

Large WG, Yeager SG. 2004. ‘Diurnal to decadal global forcing for oceanand sea-ice models: the data sets and flux climatologies’. Tech. NoteNo. 460. NCAR: Boulder, CO.

Lee T, Balmaseda MA, Greiner E, Stammer D. 2009. Ocean stateestimation for climate research. Oceanography 22: 160–167.

Le Traon PY, Nadal F, Ducet N. 1998. An improved mapping method ofmulti-satellite altimeter data. J. Atmos. Oceanic Technol. 15: 522–534.

Liu Z-Q, Rabier F. 2002. The interaction between model resolution,observation resolution and observation density in data assimilation:A one-dimensional study. Q. J. R. Meteorol. Soc. 128: 1367–1386.

Liu H, Anderson JL, Kuo Y-H, Raeder K. 2007. Importance of forecasterror multivariate correlations in idealized assimilations of GPS RadioOccultation data with the ensemble adjustment filter. Mon. WeatherRev. 135: 173–185.

Liu J, Kalnay E, Miyoshi T, Cardinali C. 2009. Analysis sensitivitycalculation in an ensemble Kalman filter. Q. J. R. Meteorol. Soc. 135:1842–1851.

Madec G, Delecluse P, Imbard M, Levy C. 1998. ‘OPA 8.1 Ocean GeneralCirculation Model reference manual’. Note du Pole de modelisationNo 11. Institut Pierre-Simon Laplace: Paris.

Masina S, Di Pietro P, Storto A, Navarra A. 2011. Global ocean re-analysesfor climate applications. Dyn. Atmos. Oceans 52: 341–366.

Nilsson J, Dobricic S, Pinardi N, Taillandier V, Poulain P-M. 2011. On theassessment of Argo float trajectory assimilation in the MediterraneanForecasting System. Ocean Dyn. 61: 1475–1490.

Oke P, Schiller A. 2007. Impact of Argo, SST and altimeter data on aneddy-resolving ocean reanalysis. Geophys. Res. Lett. 34: L19601.

Oke PR, Balmaseda MA, Benkiran M, Cummings JA, Dombrowsky E,Fujiim Y, Guinehut S, Larnicol G, Le Traon P-Y, Martin MJ. 2009.Observing system evaluations using GODAE systems. Oceanography22: 144–153.

Palmer TN, Alessandri A, Andersen U, Cantelaube P, Davey M,Delecluse P, Deque M, Dıez E, Doblas-Reyes FJ, Feddersen H,Graham R, Gualdi S, Gueremy J-F, Hagedorn R, Hoshen M,Keenlyside N, Latif M, Lazar A, Maisonnave E, Marletto V, Morse AP,Orfila B, Rogel P, Terres J-M, Thomson MC. 2004. Development of aEuropean multi-model ensemble system for seasonal to inter-annualprediction (DEMETER). Bull. Amer. Meteorol. Soc. 85: 853–872.

Palmer TN, Buizza R, Doblas-Reyes F, Jung T, Leutbecher M, Shutts GJ,Stenheimer M, Weisheimer A. 2009. ‘Stochastic parametrization andmodel uncertainty’. Tech. Memo. No. 598. ECMWF: Reading, UK.

Raynaud L, Berre L, Desroziers G. 2008. Spatial averaging ofensemble-based background-error variances. Q. J. R. Meteorol. Soc.134: 1003–1014.

Rayner NA, Parker DE, Horton EB, Folland CK, Alexander LV,Rowell DP, Kent EC, Kaplan A. 2003. Global analyses of sea surfacetemperature, sea ice, and night marine air temperature since the latenineteenth century. J. Geophys. Res. 108: 1059–1077.

Reynolds RW, Smith TM. 1994. Improved global sea surface temperatureanalyses using optimum interpolation. J. Climate 7: 929–948.

Reynolds RW, Smith TM, Liu C, Chelton DB, Casey KS, Schlax MG. 2007.Daily high-resolution blended analyses for sea surface temperature.J. Climate 20: 5473–5496.

Risien C, Chelton D. 2008. A global climatology of surface wind andwind stress fields from eight years of QuikSCAT scatterometer data.J. Phys. Oceanogr. 38: 2379–2413.

Segschneider J, Anderson DLT, Vialard J, Balmaseda MA, Stockdale TN,Troccoli A, Haines K. 2001. Initialization of seasonal forecastsassimilating sea level and temperature observations. J. Climate 14:4292–4307.

Simmons A, Uppala S, Dee DP, Kobayashi S. 2007. ‘ERA-Interim:New ECMWF reanalysis products from 1989 onwards’. ECMWFNewsletter 110: ECMWF: Reading, UK.

Storto A, Randriamampianina R. 2010a. The relative impact ofmeteorological observations in the Norwegian regional model asdetermined using an energy norm-based approach. Atmos. Sci. Lett.11: 51–58.

Storto A, Randriamampianina R. 2010b. Ensemble variationalassimilation for the representation of background-error covariancesin a high-latitude regional model. J. Geophys. Res. 115: D17204.

Storto A, Dobricic S, Masina S, Di Pietro P. 2011. Assimilating along-track altimetric observations through local hydrostatic adjustmentsin a global ocean reanalysis system. Mon. Weather Rev. 139: 738–754.

Talagrand O. 1997. Assimilation of observations; an introduction.J. Meteorol. Soc. Japan 75: 191–209.

Tan D, Andersson E, Fisher M, Isaksen L. 2007. Observing-system impactassessment using a data assimilation ensemble technique: applicationto the ADM-Aeolus wind profiling mission. Q. J. R. Meteorol. Soc.133: 381–390.

Testut CE, Brasseur P, Brankart JM, Verron J. 2003. Assimilation of seasurface temperature and altimetric observations during 1992–1993into an eddy-permitting primitive equation model of the NorthAtlantic Ocean. J. Marine Syst. 40–41: 291–316.

Tremolet Y. 2008. Computation of observation sensitivity andobservation impact in incremental variational data assimilation. Tellus60A: 964–978.

Vialard J, Weaver AT, Anderson DLT, Delecleusem P. 2003. Three- andfour-dimensional variational assimilation with a general circulationmodel of the tropical Pacific Ocean. Part II: Physical validation. Mon.Weather Rev. 131: 1379–1395.

Vialard J, Vitart F, Balmaseda MA, Stockdale TN, Anderson DLT. 2005.An ensemble generation method for seasonal forecasting with anocean–atmosphere coupled model. Mon. Weather Rev. 133: 441–453.

Vidard A, Anderson D, Balmaseda MA. 2007. Impact of oceanobservation system on ocean analysis and seasonal forecasts. Mon.Weather Rev. 135: 409–429.

Wijffels SE, Willis J, Domingues CM, Barker P, White NJ,Gronell A, Ridgway K, Church JA. 2008. Changing expendablebathythermograph fall rates and their impact on estimates ofthermosteric sea level rise. J. Climate 21: 5657–5672.

Yan C, Zhu J, Zhou G. 2007. Impacts of XBT, TAO, altimetry and ARGOobservations on the tropical Pacific Ocean data assimilation. Adv.Atmos. Sci. 24: 383–398.

Zanna L, Heimbach P, Moore AM, Tziperman E. 2010. The role of oceandynamics in the optimal growth of Tropical SST anomalies. J. Phys.Oceanogr. 40: 983–1003.

Zanna L, Heimbach P, Moore AM, Tziperman E. 2012. Upper-oceansingular vectors of the North Atlantic climate with implicationsfor linear predictability and variability. Q. J. R. Meteorol. Soc. 138:500–513.

Zou X. 1997. Tangent linear and adjoint of ‘on–off’ processes and theirfeasibility for use in four-dimensional variational data assimilation.Tellus 49A: 3–31.


Date post:	23-Dec-2016
Category:	Documents
Upload:	srdjan
View:	212 times
Download:	0 times

Ensemble spread-based assessment of observation impact: application to a global ocean analysis...

Documents