Seasonal Prediction Based on EOF Analyses of GCM Ensemble Means
Ruping Mo and David. M. Straus
Center for Ocean-Land-Atmosphere Studies Institute of Global Environment and Society, Inc.
4041 Powder Mill Road, Suite 302 Calverton, MD 20705
e-mail: [email protected]
COLA Technical Report 75 November 1999
Abstract
In this study, we construct a regression prediction scheme for seasonal-averaged anomalies based on the EOF analysis of GCM ensemble means. The predictors of the regression equation are the principal components associated with the EOFs of the ensemble mean. This scheme, when applied to a 17-year ensemble generated by the GCM of the Center for Ocean-Land-Atmosphere Studies (COLA), achieves significant seasonal-averaged forecast skill over the tropics and some extratropical regions during Northern Hemisphere winter. This skill, either deterministic or probabilistic, is generally comparable to the corresponding skill achieved from direct application of the ensemble. In some regions, such as the extratropical western Pacific and East Asia, the EOF-based scheme is capable of realizing some implicit skill associated with the GCM ensemble, leading to significant improvement of seasonal forecast skill. Hidden by the climate noise, this implicit skill cannot be utilized by direct application of the ensembles. It emerges after the noise is reduced or totally removed from the data through the EOF and regression analyses. Generally speaking, implicit skill may be associated with different principal components in different locations. Therefore it is possible to carefully select the principal components as the predictors in the regression equation to achieve the best regional forecast. Our application also suggests that the COLA GCM can identify an ENSO-independent signal over the extratropical North Pacific, which may be related to the local air-sea interaction.
Applications of the EOF-based regression prediction scheme to ensembles
generated by other GCMs also lead to some interesting results. The scheme is generally applicable when the dynamical signal and the climate noise contained in the ensemble means are clearly separable through the EOF analysis. The possibility of generalizing this single-model scheme to a multi-model version is discussed at the end of the paper.
1. Introduction
The predictability of seasonal climate variations has been studied a great deal
using the ensemble forecast approach, in which several deterministic forecasts are created
using a state-of-the-art general circulation model (GCM). This approach can be used to
forecast probabilities of a number of probable future states instead of a definite single
state. Some recent studies (e.g., Shukla 1998; Shukla et al. 2000a, 2000b; Krishnamurti et
al. 1999) have revealed that ensemble forecasting with current GCMs leads to significant
skill in predicting seasonal anomalies in the tropics and some extratropical regions,
especially over the Pacific/North American (PNA) sector. Although the existence of such
seasonal predictability has long been suggested by statistical and theoretical studies on the
world-wide impact of the El Niño/Southern Oscillation (ENSO) phenomenon (Bjerknes
1966, 1969; Hoskins and Karoly 1981; Horel and Wallace 1981; Wallace and Gutzler
1981; Sardeshmukh and Hoskins 1988; Chen and van den Dool 1997; Mo et al. 1998),
successfully predicting the ENSO-related large-scale anomalies with a GCM on seasonal
timescales is indeed a major breakthrough in climate research (Barnston et al. 1999;
Mason et al. 1999).
The simplest method of utilizing an ensemble forecast is to issue a forecast using
the ensemble mean. This should be considered as a pseudo-deterministic forecast, given
that the single value of the ensemble mean is always subject to a random error associated
with the ensemble. It has been argued that a probabilistic approach is more appropriate
than a definite approach for the seasonal forecast for the chaotic atmosphere. A probability
forecast tries to predict the possible future states and estimate their chances to occur. In
the context of ensemble forecasting, this usually involves dividing a meteorological
1
2
variable into several categories and calculating the percentage of ensemble members in
each category. For example, one may consider a cold event in a winter as the occurrence
of seasonal average surface temperature being one standard deviation less than normal.
The probability forecast at each gridpoint for this happening in the GCM is simply given
by the fraction of ensemble integrations meeting the event criterion. Alternatively, one
may consider the ensemble mean as the best estimate of the future state, and estimate its
chance to occur based on the dispersion of the ensemble (Déqué et al. 1994; Mo and
Straus 1999).
Pan and van den Dool (1998) pointed out that the prediction of the future state
using exclusively the corresponding ensemble could be too confident because the
ensemble spread is too small to catch future reality. They suggested that a combination of
the current forecast and some historical forecasts could help to increase the reliability of
forecast. This idea has inspired Mo and Straus (1999) to derive a forecast scheme based on
the regression of the historical observations on the corresponding ensemble means of
historical forecasts. Application of this scheme to the ensemble hindcasts generated by the
GCM of the Center for Ocean-Land-Atmosphere Studies (COLA) showed some
noticeable improvements of the probability forecast over the regions where the boundary-
forcing signal is relatively weak as compared to the internal variability (noise) of the
model atmosphere.
The regression forecast discussed in Mo and Straus (1999) was constructed based
on the observation and forecast data at each gridpoint. While treating each gridpoint
independently is simple and straightforward, it may not be the best way to exploit the
GCM-generated ensembles. It has been recognized that large-scale characteristic patterns
associated with both boundary forcing and with internal variability (unrelated to boundary
conditions) are simulated well in some GCMs (Straus and Shukla 2000). Thus it is
reasonable to expect our current GCM to be capable of capturing some of the information
of the dominant patterns in the atmosphere. Under such circumstances, a regression
scheme treating each gridpoint independently obviously can not optimally benefit from the
captured teleconnection information. To overcome this drawback and improve the
performance of the regression forecast, we try in this study to construct predictors based
on the empirical orthogonal functions (EOFs) of the ensemble mean. These EOFs are
essentially a different representation of the GCM covariance matrix derived from the
historical ensemble mean. If they also represent the best knowledge of the observed
teleconnection patterns, then a regression forecast based on their fluctuations could lead to
significant improvement of forecast skill in regions where the EOFs explain significant
variance.
We will apply this EOF-based regression scheme to ensembles generated by a
number of different GCMs (see Section 2 for detailed description). Note that our scheme
represents a statistical-dynamical approach to the seasonal forecast. It can be considered as
a generalization of the EOF-based prediction algorithm recently developed by Kim and
North (1998, 1999) from a purely statistical approach. The details of our scheme are
outlined in Section 3. The scheme is evaluated using the COLA ensemble in Sections 4
and 5. It is applied to ensembles generated by other GCMs in Section 6. Further discussion
and conclusions are presented in Section 7. Some verification techniques are described in
the appendix.
3
2. Data description
The principal ensemble used to evaluate the forecast schemes developed in this
study were generated from the COLA GCM, which has a moderately high resolution
(rhomboidal 40) with 18 discrete levels in the vertical (Kinter et al. 1997). As part of a
project of seasonal dynamical prediction (Shukla et al. 2000a), nine integrations were
carried out each year, starting from different observed initial conditions of the atmosphere,
12 hours apart, centered in mid-December for the winters 1981/82-1997/98. The observed
sea surface temperature (SST) and sea-ice were prescribed in the boundary conditions.
Therefore, any skill derived from these integrations should be considered as only potential
forecast skill of the COLA GCM in response to SST anomalies. The observational data
used to verify the model performance are obtained from the NCEP/NCAR reanalysis (see
Kalnay et al. 1996).
To test the general applicability of the EOF-based regression forecast scheme, we
shall also make use of four other DSP ensembles generated using the GCMs of the
National Centers for Environmental Prediction (NCEP, 5 members, 1982/83-1995/96), the
National Center for Atmospheric Research (NCAR, 10 members, 1981/82-1996/97), the
Geophysical Fluid Dynamics Laboratory (GFDL, 10 members, 1979/80-1994/95), and the
Data Assimilation Office (DAO, and its model is denoted as GSFC, 9 members, 1980/81-
1995/96). The initial conditions for all these integrations were derived from observations
in mid-December. We also use an ensemble generated from the GCM of the European
Centre for Medium-Range Weather Forecasts (ECMWF, 9 members, 1979/80-1992/93),
in which the initial conditions were obtained from observations in mid-November. Further
details of these models and their ensembles are available in Shukla et al. (2000a).
4
We shall apply the forecast scheme only to the 500-hPa geopotential height
interpolated to a global grid and averaged over the period between January 1
and March 31 of each year. Therefore, the numbers of years covered by the data are 17,
14, 16, 17, 15, and 14 for the ensembles of the COLA, NCEP, NCAR, GFDL, GSFC, and
ECMWF, respectively.
5.25.2
3. The EOF analysis and the EOF-based regression scheme
EOF analysis, also known as principal component (PC) analysis in other academic
communities, has become a widely used tool in analyzing both observations and numerical
simulation of the global circulation (Bretherton et al. 1992). To illustrate this
methodology, let denote the anomaly field of a variable measured at M
locations for N times
) ,( nm txQ
), M,1(m ),,1( Nn . The eigenvectors of its covariance
matrix, say form an orthogonal set of basis vectors or EOFs. When the data are
projected onto each EOF, the resulting time series are the PCs given as
),( mj xE
(1) ),,1( ,)(),()(1
M
mmjnmnj LjxEtxQtT
where L is the rank (i.e., the number of positive eigenvalues) of the covariance matrix. It
cannot be greater than neither M nor N, i.e.,
).,(min NML
jT shows how the jth EOF evolves in time. It is conventional to order the EOFs and the
associated PCs so that the first EOF explains the largest fraction of the variance of the
data, and the last EOF explains the smallest.
We can reconstruct the data from and , i.e., jE jT
5
(2) .)()() ,(1
L
jnjmjnm tTxEtxQ
Figure 1 shows the correlation patterns associated with the first six PCs derived
from the observed 500-hPa heights of the 17 NH winters (1982-1998). In order to ensure
equal areas get equal weights in the areal integration (sum over grid points in Eq.(1)), we
have multiplied the height at each gridpoint by the square root of the cosine of latitude of
that gridpoint before doing the EOF analysis. The fraction of total variance explained by
each mode is indicated in the title of the corresponding map. The variance distribution of
each mode is displayed using shaded contouring. We see that the first EOF, which
explains about 31% of the total variance, appears to be a combination of the well-known
PNA pattern and the North Atlantic oscillation (NAO) discussed in Wallace and Gutzler
(1981). The fact that they are summarily represented by the leading EOF suggests that
they may not always be totally independent of each other. The second EOF is also a
combination of two patterns. One is located in the PNA sector, but is distinctly different
from the PNA pattern seen in the first EOF (see Straus and Shukla 2000). The other
pattern is similar to the eastern Atlantic pattern defined in Wallace and Gutzler (1981).
The third, fourth, and fifth EOFs represent respectively the western Atlantic pattern, the
Eurasian pattern, and the western Pacific pattern defined in Wallace and Gutzler (1981).
The sixth EOF, which explains only 4.9% of the total variance, represents a stationary
planetary wave in middle latitudes.
The correlation patterns associated with the first six PCs of the ensemble means of
the COLA GCM for the same period are shown in Fig.2. Here the ENSO-related pattern
over the PNA sector emerges as the leading mode, which explains nearly 45% of the total
variance. This pattern is much stronger and more well-defined than its observed
6
counterpart in Fig.1b, because part of the internal variability is averaged out in taking the
ensemble mean. The pattern representing the seasonal mean internal variability over the
PNA sector is combined with the eastern Atlantic pattern in the second EOF. The third
EOF of the ensemble mean represents a combination of the western Atlantic pattern and
the western Pacific pattern (again, see Wallace and Gutzler 1981). The fourth, fifth, and
sixth EOFs could be considered as other planetary waves simulated by the COLA GCM.
We now proceed to outline the procedure of the EOF-based regression prediction.
Let and denote the anomalies of the ensemble mean and observation, respectively.
The jth EOF and PC of the ensemble mean derived from N historical records are denoted
as and . Then can be regressed on as follows,
eQ
ejE
oQ
ejT oQ e
jT
J
j
enmn
ejmjnm
o LJtxtTxbtxQ1
(3) )( ),()()(),(
where are the regression coefficients, jb is a normally distributed variable, and is
the rank of the covariance matrix of the ensemble mean. Because the PCs of the ensemble
mean are uncorrelated with each other, the least squares estimates of are easily
obtained as
eL
jb
(4) )()(),()(ˆ1
2
1
N
nn
ej
N
nn
ejnm
omj tTtTtxQxb
When the ensemble mean of the future state, , is available, the
predicted future state is given by
),( 1Nme txQ
(5) )( )()(ˆ),(ˆ1
11e
J
jN
ejmjNm
o LJtTxbtxQ
where
7
8
(6) .)(),()(1
11
M
mm
ejNm
eN
ej xEtxQtT
Alternatively, we can also predict the PCs of the observations based on their regressions
on the PCs of the ensemble means, and then reconstruct the predicted anomalies as
(7) )(ˆ)(),(ˆ1 1
11
oL
i
J
jN
ejijm
oiNm
o tTxEtxQ
where are the corresponding regression coefficients, is the ith EOF derived from N
historical observations, and is the rank of the of the covariance matrix of the
observations. Apart from some small computational errors, Eqs.(5) and (7) should lead to
the same result, and we confirmed that they do.
ij oiE
oL
In the following sections, the regression forecast will evaluated using a cross-
validation approach. For a data set of size 1N , the cross-validation is carried out by
withholding an observation each time, constructing the regression model with the
remaining developmental data sets of size N, and then using the model to predict the
withheld observation. The resulting 1N regression forecasts are then verified by the
corresponding observations. 1N
4. Deterministic forecasts based on the COLA ensembles
In this study, a deterministic forecast is defined as a single-value prediction
carrying no probability information. For example, the ensemble mean at a gridpoint of
several GCM runs, regardless of the ensemble spread, can be used to issue a deterministic
forecast for that gridpoint. An application of the EOF-based regression method described
in the previous section can also lead to a single-value prediction for each gridpoint. The
skill of these two forecast schemes is evaluated in this section within a deterministic
framework using the mean-square-error skill score (MSS), and will be further evaluated
within a probabilistic framework in next section using the ranked probability skill score
(RPSS). The definitions of these skill scores are given in the appendix. Suffice it to say
that both of them have a range of to 1, with positive value indicating a forecast better
than a zero skill climatological forecast based on the distribution.
For convenience, the scheme that uses directly the ensemble mean of each
gridpoint in a winter to issue the forecast for that gridpoint in the same winter will be
referred to as a purely-dynamical forecast (PDF), and the scheme with application of the
EOF-based regression will be referred to as a statistical-dynamical forecast (SDF). Figure
3 shows the MSS of the 17-year COLA ensembles for the PDF and the SDFs with various
PCs of the ensemble mean used as predictors. We see that almost all positive scores are
statistically significant at the 95% confidence level. The simple PDF (Fig.3a) achieves
remarkably high skill over the tropics and some extratropical regions. In particular, the
impressive scores over the PNA sector indicate that the COLA GCM is well capable of
capturing the ENSO response (Shukla 1998). This skill will be referred to as the explicit
skill of the GCM, since it exists in the original GCM ensembles. On the other hand,
unacceptably large negative scores of PDF can be seen over Alaska, Central Europe, and
near the Himalayas. These poor scores, as implied by Eq.(A1), result mainly from the
large variances of the ensemble mean relative to their observational counterparts in the
regions with weak or negative correlation coefficients. In other words, the model
atmosphere must be much noisier than the real atmosphere in these regions. In the SDF
with the first PC of the ensemble mean as the only predictor (Fig.3b), however, no large
9
negative scores are apparent. This is a consequence of the regression analysis, which is
insensitive to the error in variance. Theoretically, there should be no negative score at all
in the SDF. If all predictors in the regression model are pure noise, the corresponding
regression coefficients should all be zero, and the SDF reduces to the climatological
forecast. Under such circumstances, the MSS is zero. Therefore those negative scores in
Fig.3b-d should be understood as random errors of the correlation between the noise and
observation. Note that the magnitudes of these errors increase as the number of predictor
increases. The reason for this problem is that the addition of a variable to a regression
equation almost always increases (and never decreases) the variance of a predicted
response. Addition of a useless predictor then will only contribute to increase the error of
prediction. Note that the large errors seen in Fig.3d are not directly relevant to the original
model noise seen in Fig.3a.
The first PC of the ensemble mean in fact represents the major ENSO response
identified by the COLA model (Straus and Shukla 2000). The skill scores achieved by the
SDF with this leading PC as the only predictor are statistically significant over the tropics
and the extratropical PNA sector (Fig.3b). As compared with the straightforward PDF
(Fig.3a), this simple SDF is less skillful over the tropical and extratropical Pacific, equally
skillful over Eastern Canada and around the Gulf of Mexico, and more skillful over the
equatorial Indian Ocean, the eastern equatorial Atlantic and part of the extratropical North
Atlantic. The disadvantage of this SDF over the Pacific region indicates either that the first
PC of the ensemble mean is incapable of representing all ENSO effects, or that there is an
ENSO-independent signal over this region. Such a signal could be related to the strong
coupling of the extratropical atmosphere with the SST anomalies over the North Pacific
10
(e.g., Namias 1969; Zhang et al. 1996; Mo et al. 1998). On the other hand, the advantage
of the SDF over the PDF in some regions suggests that the GCM ensembles contain some
implicit skill that cannot be utilized by direct application of the ensembles. This skill is
hidden by the climate noise, and emerges only after the noise is reduced or totally
removed from the data.
Fig.3c shows that including the second PC into the regression model has little
effect on the forecast skill over the PNA sector. However, the contribution of this mode is
very significant over the extratropical western North Pacific, East Asia, the Arctic, and
West Europe. In particular, the significant positive scores over Japan and the surrounding
area are in sharp contrast to those negative scores either in Fig.3a or Fig.3b. This
improvement implies that, while inclusion of the ENSO-independent PNA pattern seen in
Fig.2b apparently does not improve the forecast skill, the eastern Atlantic pattern in the
second GCM EOF (Fig.2b) is useful in some regions. The COLA GCM has no explicit
skill over Japan, as seen in Fig.3a. However, the EOF analysis identifies implicit skill over
this region. In the EOF analysis, the correct fluctuation signals of various patterns are
identified from the ensemble mean, and the regression analysis then combines these
signals with the spatial structure of the observed patterns.
The loss of skill by the SDF over the tropical and extratropical eastern Pacific in
Fig.3b,c is reversed when the first six PCs are taken into account (Fig.3d). It can be shown
that in this regard the sixth PC makes the most significant contribution. However,
including up to six PCs in the regression equation introduces a large amount of error in
other regions. This kind of regression error is also responsible for the degradation of
forecast skill over East Canada. This implies that some globally insignificant modes are
11
locally important. They could be taken into account, but only at the expense of some other
regions. In practice, we could select different PCs as predictors for different regions to get
the best regional forecast.
5. Probabilistic forecasts based on the COLA ensembles
Because of the chaotic nature of the atmosphere, a seasonal forecast should be
looked upon as probabilistic rather than deterministic. For a single-model ensemble, a
simple probability forecast can be constructed by counting the ensemble members
associated with some pre-defined events. Here we consider a three-category forecast, in
which the standardized anomaly of the seasonal-mean 500-hPa height is classified as
above normal (larger than 1), normal (between -1 and 1), and below normal (less than -1).
The occurrence of each event in a specified year at a gridpoint is given by the fraction of
ensemble members of the same year within the corresponding category. Following Mo
and Straus (1999), we shall refer to this scheme as counting probability forecast.
In order to take the advantage of the EOF and regression analyses shown in the
preceding section, we may also construct a probability forecast based on the prediction of
the EOF-based regression model. As before, the predictors are the principal components
of the ensemble mean. We then assume that the predicted value of a variable is normally
distributed, with the variance of the distribution estimated from the mean-square error of
the regression model (Mo and Straus 1999). With the notation defined in Section 3, the
estimated variance is given by
(8) )()(ˆ),(1
),(ˆVar1
2
11
N
nn
ej
J
jmjnm
oNm
o tTxbtxQJN
txQ
12
13
where is the degrees of freedom of the variance, reduced from N in the presence
of J regression coefficients (Montgomery and Peck 1982). This scheme will be
called regressive probability forecast. It is different, however, from the gridpoint-based
regressive probability forecast discussed in Mo and Straus (1999). In this section, we shall
only consider the regression model based on the PCs of the ensemble mean discussed in
Section 4.
)( JN
Jbb ˆ,,1
Both the counting and regressive probability forecasts are evaluated using the
RPSS (defined in the appendix) for the above-mentioned three-category forecast (Fig.4).
In order to ensure statistical stability, the RPSS for a gridpoint is computed using gridpoint
data collected over the surrounding region within 600 km, with each forecast at each
gridpoint over the region considered as a separate, independent forecast. As shown in
Fig.4a, the counting probability forecast is significantly high over the tropics and the
extratropical PNA sector. Again we see that the basic response to the ENSO signal is
successfully captured by the simplest regression scheme (Fig.4b). With the first PC of the
ensemble mean as the only predictor, the regression scheme cannot fully recover the
significant skill of the counting scheme over the tropical and extratropical Pacific. But it is
more skillful over the tropical Atlantic and the tropical Indian Ocean. Including the second
PC into the regression equation leads to significant skill over the extratropical western
Pacific and East Asia (Fig.4c), where the skill scores of the counting probability forecast
are negative. The forecast skill over Canada is also slightly improved by considering the
second PC. As the first six PCs are used as predictors, the skill of the regression scheme
increases noticeably over the tropical Pacific, the western equatorial Atlantic, and North
Africa. Again, we see the degradation of skill over East Canada in Fig.4d due to the fact
that too many useless predictors are included in the regression equation. These results are,
in general, consistent with those derived from the deterministic framework in the
preceding section.
6. Application to ensembles of othe GCMs
In this section, the applicability of the EOF-based regression forecast is tested by
replacing the COLA ensemble with the ensembles from each of the other five GCMs
mentioned in Section 2. The result of each model is evaluated using the MSS. Note that,
since different models cover different periods, the MSS from one model is not strictly
comparable to the MSS from another model.
Figure 5 shows the results derived from the NCEP ensemble and can be compared
to Figure 3. As in the COLA model, the first EOF of the NCEP ensemble mean represents
the typical ENSO signal (not shown). Therefore when the first PC of the ensemble mean is
used as the only predictor in the SDF, the ENSO response is successfully predicted
(Fig.5b). This simple SDF appears to be less skillful over the tropics than the PDF
(Fig.5a), but it is more skillful over the extratropical North Pacific and Central Canada.
Including the second PC has very little effect on the forecast skill. However, when the first
seven PCs are included, the skill scores over Europe and the Far East are significantly
improved, while the skill over Canada is notably degraded. These features are very similar
to those seen in the COLA ensemble.
The results derived from the NCAR ensemble are shown in Fig.6. Unlike the
COLA and NCEP GCMs, the ENSO signal in the NCAR model is represented by the
second, instead of the first, PC of the ensemble mean. This is confirmed in Fig.7, which
14
shows the EOFs of the NCAR GCM (and can be compared to Figures 1 and 2). The first
EOF of this model (Fig.7a) is the typical PNA pattern (similar to Wallace and Gutzler
1981) that represents mainly the internal variability of the model atmosphere. Therefore it
is not surprising that no significant skill can be achieved over the extratropical eastern
Pacific and North America by the SDF with the first PC as its only predictor (Fig.6b).
When the second PC is included, the ENSO response is evident (Fig.6c). When the first
six PCs are included, significant skill can be seen over the Far East. Note that this
significant skill also occurs in the corresponding PDF (Fig.6a). Therefore it can be
considered an explicit skill of the NCAR GCM. In the COLA and NCEP GCMs, however,
the skill over the Far East is not explicit.
Results from the GFDL and GSFC ensembles are presented in Figures 8 and 9. We
see that in neither case is there an obvious advantage of the SDF over the PDF. In
particular, the ENSO response is too weak when the first PC is used as the only predictor.
The reason for this problem is that the first EOF of the ensemble mean is a combination of
the ENSO response and the PNA pattern of the internal variability (not shown). In other
words, the EOF analyses fail to separate the ENSO signal from the model noise. Under
such circumstances, the skill of the SDF could be seriously undermined. Nevertheless,
when the first six PCs are considered, we can still see some impressive performances of
the SDF over Canada and the North Pacific as well (Fig.9d); the significant pattern almost
looks like the ENSO-related pattern, but shifted westward.
Figure 10 shows the result of the ECMWF ensemble. In this model, the SDF with
the first PC as the only predictor (Fig.10b) is more skillful over the eastern equatorial
Pacific and Canada than the PDF (Fig.10a). The explicit skill over the Far East achieved
15
by the model (Fig.10a) is recovered in the SDF that uses the first six PCs of the ensemble
mean as predictors (Fig.10d). Significant implicit skills over Canada are also evident in
Fig.10d.
In summary, the EOF-based regression forecast is generally applicable when the
dynamical signal and model noise contained in the ensemble means are clearly separable
through the EOF analysis. A successful application of this scheme can lead to significant
improvement of the forecast skill over certain regions.
7. Discussions and conclusions
Successful prediction of seasonal anomalies in the atmosphere depends on our
understanding of the dynamics of large-scale, low-frequency teleconnection patterns. It is
generally believed that some important teleconnection patterns are forced by the SST
anomalies, which in turn arise from coherent air-sea interaction. Dynamical seasonal
predictions are carried out on the premise that the atmospheric responses to the low-
frequency boundary fluctuations are predictable using a state-of-the-art GCM. However,
there is also evidence that substantial low-frequency variability can also arise as a result of
internal nonlinear atmospheric dynamics. To a forecaster, this internal variability
represents the climate noise and is basically unpredictable. If this noise is too strong
compared to the boundary-forcing signal, it will seriously undermine the predictability of
seasonal anomalies. In this study we show that, although there is no guarantee, the internal
variability contained in many GCM ensembles can be separated from the boundary-
forcing signal through application of the EOF analysis, and at least part of its impact on
the seasonal prediction can be removed through further application of regression analysis.
16
When the EOF-based regression forecast scheme is applied to the ensemble of
seasonal integrations of the COLA GCM, significant skill is seen over the tropics and the
extratropical PNA sector. This skill is considered as an explicit skill of the GCM, because
it is also achievable from direct application of the original ensemble mean. The major
advantage of using the EOF-based regression scheme is that it can realize some implicit
skill of the GCM ensembles, leading to significant improvement of the forecast in some
regions, especially over the extratropical western Pacific and East Asia for the COLA
GCM. This implicit skill may be associated with different principal components in
different locations. Therefore it is possible to carefully select the principal components as
the predictors in the regression model to achieve the best regional forecast.
Application of the EOF-based regression forecast may also improve our
understanding of certain atmospheric anomalies and their GCM simulations. We have
shown that the predictability in the PNA sector derived from the COLA GCM can be
partitioned into an ENSO-related component and an ENSO-independent component. The
ENSO-related component is the atmospheric response to the SST anomalies over the
tropical Pacific. This component, usually associated with the first EOF of the ensemble
mean, contributes to significant skill of the GCM over the tropical Pacific, extratropical
North Pacific, Central and East Canada, and the Gulf of Mexico. The ENSO-independent
component, which is likely related to the ENSO-independent air-sea interaction in the
extratropical North Pacific, has a noticeable effect on the forecast skill over the
extratropical North Pacific and the tropical Pacific as well.
The performance of the EOF-based regression forecast depends on the "quality" of
the ensemble-mean EOFs. We tested the scheme using six ensembles generated from
17
different GCMs and found that it is generally applicable and usually leads to some local
improvement of the seasonal prediction. The scheme could fail, however, to offer any
useful information if the EOF analysis cannot lead to a clear separation between the
boundary-forcing signal and the internal noise in the ensemble. In particular,
distinguishing between the SST-forced variability and internal variability in the
extratropical North Pacific (Straus and Shukla 2000) is essential for a satisfactory
application. However, there is no guarantee that such a separation will always happen for
any ensemble through a regular EOF analysis. It might be possible to obtain a better
separation by performing a rotated EOF analysis (e.g., Mo et al. 1998). Either way, we
suggest that the EOF-based regression forecast scheme should be used to complement,
rather than complete with, other commonly used schemes, such as the purely-dynamical
forecast mentioned in this study.
Finally, we mention the possibility of generalizing the single-model EOF-based
regression forecast to a multi-model version. Some recent studies (e.g, Krishnamurti et al.
1999; Palmer et al. 2000) have shown certain improvement of the ensemble forecast
obtained by blending forecast skills of different GCMs. In principle, a multi-model
regression forecast can be constructed by using some or all principal components derived
from the ensemble of each model to build a super-multiple regression equation. In
practice, however, two potential problems may render this approach useless. One is the
overfitting problem. The final regression equation may contain too many predictors, which
will lead to large variance of the predicted response. Another potential problem is called
the multicollinearity, which occurs when some of the predictors are highly intercorrelated.
In the presence of multicollinearity, the least-squares estimates of the regression
18
19
coefficients are unstable and have large variances. The hazard associated with these
problems could be prevented if the data records for all participating GCMs are long
enough. We can also manually chose only the useful principal components of each model
to be the predictors of the multi-model regression equation.
Acknowledgements
The authors wish to thank J. Shukla, A. Schlosser, B. Kirtman and Y. Fan for their
helpful suggestions, L. Marx and D.A. Paolino for assistance in the collection of data, and
B. Doty and C. Steinmetz for technical support.
Appendix: Some skill scores for forecast verifications
Verification of an ensemble forecast can be assessed within either a deterministic
or a probabilistic framework. For a deterministic forecast, the correlation coefficient
between the forecasts and the observations, r say, may be used as a skill score to evaluate
the forecast scheme relative to a climatological forecast. Here the climatological forecast
is simply a forecast of the long-term average of the variable, and then has no correlation
with the observed variable. Therefore, a negative r implies that the climate mean is a
better predictor than the forecast value of the model, or the converse if r is positive.
Although the correlation coefficient can be used to identify some potential
information contained in the forecast, it is not considered as a good measure of the actual
forecast skill due to its insensitivity to the error magnitude. Murphy (1988) introduced a
more accurate measure called the mean-square-error skill score (MSS), which, when
applied directly to the anomaly fields of forecast and observation, can be written as (see
Murphy 1988; Livezey 1995)
(A1) 2MSS
o
f
o
f
S
Sr
S
S
where r is, as before, the correlation coefficient between the forecasts and the
observations, and are the standard deviations of the forecasts and the observations,
respectively. The MSS can be considered as the mean-square error of the forecast scaled
by the mean-square error of the climatological forecast. Eq.(A1) shows that a perfect
forecast ( ) corresponds to For
fS
MSS
oS
1 of SSr and 1 . 10 r , the possible
maximum value of MSS is 2r , which can be achieved when . The forecast is
worse than the climatological forecast when
of rSS
0MSS , i.e., . Note that even for orS2fS
20
21
a forecast with perfect correlation (i.e., 1r ), the MSS can still be negative when
. of SS 2
When a forecast is assessed within a probabilistic framework, the mean squared
error between the predicted and observed probabilities can be used as a measure of
forecast skill. Such a measure was introduced by Brier (1950) for a simple two-category
probability forecast. The so-called Brier score was further generalized by Epstein (1969)
into a ranked probability score (RPS) for general multiple-category forecasts. More
specifically, the RPS measures the mean squared distance between the cumulative
probabilities of the forecast and the observation, and then is capable of taking into account
the fact that an error in predicting a category close to the observed one is less important
than an error in predicting a category far away from the observed one (for detailed
definition, see Mo and Straus 1999). Scaling the RPS of the forecast by the RPS of the
climatological forecast leads to a ranked probability skill score (RPSS), i.e.,
(A2) gy)(climatolo RPS
RPS (forecast)1RPSS
In this study, we define the climatological probability of an event at a gridpoint for
a winter season as the frequency of occurrence of the event calculated from the
observations of all other winter seasons at the same gridpoint, and a climatological
forecast of the event as the forecast in which the climatological probability is predicted.
According to the above definition, a positive RPSS corresponds to a forecast better
than the climatological forecast. It could be possible, however, that such a positive value is
not significantly different from zero, and could be achieved simply by chance. In this
study, this statistical significance problem is addressed using a Monte Carlo approach. To
outline the method, we label a multi-year ensemble as "Data I" and the corresponding
observations as "Data II". The original RPSS is calculated from Eq.(A2), with Data I
verified by Data II in chronological order. We then create a randomized data by replacing
each field of Data I with a field randomly chosen from the other years (note that this is
different from shuffling Data I in the time domain). The corresponding RPSS is computed
again from Eq.(A2), with Data I replaced by the randomized data. The same procedure is
repeated 1000 times, each time storing the score values. The original score is considered
statistically significant at the 95% confidence level if it is not exceeded by more than 50
values of the corresponding scores obtained using the randomized data. The same method
is also applied to test the significance of MSS.
22
References
Barnston, A.G., A. Leetmaa, V.E. Kousky, R.E. Livezey, E. O'Lenic, H. van den Dool, A.J.
Wagner, and D.A. Unger, 1999: NCEP forecasts of the El Niño of 1997—98 and its U.S.
impacts. Bull. Amer. Meteor. Soc., 80, 1829–1852.
Bjerknes, J., 1966: A possible response of the atmospheric Hadley circulation to equatorial
anomalies of ocean temperature. Tellus, 18, 820-829.
Bjerknes, J., 1969: Atmospheric teleconnections from the equatorial Pacific. Mon. Wea. Rev.,
97, 163-172.
Bretherton, C.S., C. Smith, and J.M. Wallace, 1992: An intercomparison of methods for
finding coupled patterns in climate data. J. Climate, 5, 541-560.
Brier, G.W., 1950: Verification of forecasts expressed in terms of probabilities. Mon. Wea.
Rev., 78, 1-3.
Chen, W.Y. and H.M. van den Dool, 1997: Asymmetric impact of tropical SST anomalies on
atmospheric internal variability over the North Pacific. J. Atmos. Sci., 54, 725-740.
Déqué, M, J.F. Royer, and R. Stroe, 1994: Formulation of gaussian probability forecasts based
on model extended-range integrations. Tellus, 46A, 52-65.
Epstein, E.S., 1969: A scoring system for probability forecasts of ranked categories. J. Appl.
Meteor., 8, 985-987.
Horel, J.D. and J.W. Wallace, 1981: Planetary-scale atmospheric phenomena associated with
the interannual variability of sea surface temperature in the equatorial Pacific. Mon. Wea.
Rev., 109, 813-829.
Hoskins, B.J. and D.J. Karoly, 1981: The steady linear response of a spherical atmosphere to
thermal and orographic forcing. J. Atmos. Sci., 38, 1179-1196.
Kalnay, E., and Coauthors, 1996: The NCEP/NCAR 40-year reanalysis project. Bull. Amer.
Meteor. Soc., 77, 437-471.
23
24
Kim, K.-Y. and G.R. North, 1998: On EOF-based linear prediction algorithm: Theory. J.
Climate, 11, 3046-3056.
Kim, K.-Y. and G.R. North, 1999: EOF-based linear prediction algorithm: Examples. J.
Climate, 12, 2076-2092.
Kinter III, J.L., D.G. DeWitt, P.A. Dirmeyer, M.J. Fennessy, B.P. Kirtman, L. Marx, K.E.
Schneider, J. Shukla, and D.M. Straus, 1997: The COLA atmosphere-biosphere general
circulation model. Volume 1: formulation. COLA Tech. Report, 51, 46pp.
Krishnamurti, T.N., C.M. Kishtawal, T.E. LaRow, D.R. Bachiochi, Z. Zhang, C.E. Williford,
S. Gadgil, and S. Surendran, 1999: Improved weather and seasonal climate forecasts from
multimodel superensemble. Science, 285, 1548-1550.
Livezey, R.E., 1995: The evaluation of forecasts. Analysis of Climate Variability, H. von
Storch and A. Navarra, Eds., Springer, 177-196.
Mason, S.J., L. Goddard, N.E. Graham, E. Yulaeva, L. Sun, and P.A. Arkin, 1999: The IRI
seasonal climate prediction system and the 1997/98 El Niño event. Bull. Amer. Meteor.
Soc., 80, 1853–1874.
Mo, R., J. Fyfe, and J. Derome, 1998: Phase-locked and asymmetric correlations of the
wintertime atmospheric patterns with the ENSO. Atmosphere-Ocean, 36, 213-239.
Mo, R. and D.M. Straus, 1999: Probability forecasts of seasonal average anomalies based on
GCM ensemble means. Technical Report 74, Center for Ocean-Land-Atmosphere Studies,
25 pp.
Montgomery, D.C. and E.A. Peck, 1982: Introduction to Linear Regression Analysis. John
Wiley & Sons, New York, pp504.
Murphy, A.H., 1988: Skill scores based on the mean square error and their relationships to the
correlation coefficient. Mon. Wea. Rev., 116, 2417-2424.
Namias, J., 1969: Seasonal interactions between the North Pacific Ocean and the atmosphere
during the 1960s. Mon. Wea. Rev., 97, 173-192.
Palmer, T.N., C. Brankovic, and D.S. Richardson, 2000: A probability and decision-model
analysis of PROVOST seasonal multi-model ensemble integrations. Quart. J. Roy. Meteor.
Soc. (submitted).
Pan, J. and H. van den Dool, 1998: Extended-range probability forecasts based on dynamical
model output. Weather Forecasting, 13, 983-996.
Sardeshmukh, P.D. and B.J. Hoskins, 1988: The generation of global rotational flow by steady
idealized tropical divergence. J. Atmos. Sci., 45, 1228-1251.
Shukla, J., 1998: Predictability in the midst of chaos: A scientific basis for climate forecasting.
Science, 282, 728-731.
Shukla, J., J. Anderson, D. Baumhefner, C. Brankovic, Y. Chang, E. Kalnay, L. Marx, T.
Palmer, D. Paolino, J. Ploshay, S. Schubert, D.M. Straus, M. Suarez, and J. Tribbia, 2000a:
Dynamical seasonal prediction. Bull. Amer. Meteor. Soc. (submitted).
Shukla, J., D.A. Paolino, D.M. Straus, D. DeWitt, M. Fennessy, J.L. Kinter, L. Marx, and R.
Mo, 2000b: Dynamical seasonal predictions with the COLA atmospheric model. Quart. J.
Roy. Meteor. Soc. (accepted for publication).
Straus, D.M. and J. Shukla, 2000: Distinguishing between the SST-forced variability and
internal variability in mid-latitudes: Analysis of observations and GCM simulations. Quart.
J. Roy. Meteor. Soc. (accepted for publication).
Wallace, J.M. and D.S. Gutzler, 1981: Teleconnections in the geopotential height field during
the Northern Hemisphere winter. Mon. Wea. Rev., 109, 784-812.
Zhang, Y., J.M. Wallace, and N. Iwasaka, 1996: Is climate variability over the North Pacific a
linear response to ENSO? J. Climate, 9, 1468-1478.
25