Anthony DeAngelis
[http://www-pcmdi.llnl.gov/ipcc/about_ipcc.php][http://www.hydro.com.au/handson/links/images/rain.gif]
Outline
Introduction Data, Models, and Methodology Results
Spatial Comparisons over United States Analysis of Resolution Ranking of Model Performance
Conclusions Future Directions
Importance of Precipitation Agriculture, water resources, power, etc. Extreme Precipitation
Flooding takes 140 lives in the United States each year (USGS 2006).
Observational evidence of increases in the frequency and intensity of extreme precipitation throughout the world over 20th century (e.g., Groisman et al. 2005)
Model projections of future increases in heavy precipitation in response to increasing greenhouse gases (e.g., Pall et al. 2007)
Quantification of future changes in precipitation relies on model simulations
How well do models simulate precipitation? IPCC AR4 –Fairly realistic mean precipitation
by ensemble of PCMDI coupled models
Mean Precipitation 1980-1999
[IPCC AR4, Ch 8, Fig. 8.5]
CMAP Observations
Multi-model mean of
AOGCMs
How well do models simulate precipitation? IPCC AR4 –Fairly realistic mean precipitation
by ensemble of PCMDI coupled models Sun et al. 2007 –Overestimation of light
precipitation and underestimation of heavy precipitation by CMIP3 models
observations
Sun et al. (2007) Figure 1
model average
How well do models simulate precipitation? IPCC AR4 –Fairly realistic mean precipitation by
ensemble of PCMDI coupled models Sun et al. (2007) –Overestimation of light precipitation
and underestimation of heavy precipitation by CMIP3 models
Kiktev et al. 2003 – HadAM3 has little skill in simulating precipitation trends over 1950-1995
Higher resolution models perform better Iorio et al. 2004 – NCAR CCM3 – mean and extreme
precipitation Kimoto et al. 2005 – MIROC 3.0 – extreme precipitation
Models with embedded cloud resolving models or certain convective parameterizations perform better for extreme precipitation (Iorio et al. 2004, Emori et al. 2005)
What did I do?
Compared 20th century simulations from CMIP3 models with observations over the contiguous United States Looked at differences in spatial pattern of
precipitation characteristics for individual models
Used a longer and consistent time period for comparison (1961-1998) than previous studies
Compared two gridded observational datasets Assessed the role of resolution on model
performance for all models collectively
Observational Data and Climate Models Observations
Climate Prediction Center’s Daily United States Unified Precipitation (CPC) -0.25° x 0.25° lon-lat (1948-1998) [Higgins et al. 2007]
David Robinson’s daily gridded precipitation (DAVR) - 1.0° x 1.0° lon-lat (1900-2003) [Dyer and Mote 2006]
Climate Models 20th century simulations – forced with observed
atmospheric composition 18 CMIP3 models with daily precipitation from
1961-2000 and a standard (non 360 day) calendar One ensemble member for each model Meehl et al. (2007)
Model # Modeling Group Country Model ID Spatial Resolution(approximate - lon x lat)
1 Bjerknes Centre for Climate Research Norway BCCR BCM 2.0 2.81° x 2.81°
2 Canadian Centre for Climate Modelling & Analysis Canada CCCMA CGCM 3.1 T47 3.75° x 3.75°
3 Canadian Centre for Climate Modelling & Analysis Canada CCCMA CGCM 3.1 T63 2.81° x 2.81°
4 Centre National de Recherches Météorologiques France CNRM CM 3 2.81° x 2.81°
5 CSIRO Atmospheric Research Australia CSIRO MK 3.0 1.88° x 1.88°
6 CSIRO Atmospheric Research Australia CSIRO MK 3.5 1.88° x 1.88°
7 Geophysical Fluid Dynamics Laboratory USA GFDL CM 2.0 2.50° x 2.00°
8 Geophysical Fluid Dynamics Laboratory USA GFDL CM 2.1 2.50° x 2.00°
9 Goddard Institute for Space Studies USA GISS AOM 4.00° x 3.00°
10 Goddard Institute for Space Studies USA GISS E H 5.00° x 3.91°
11 Goddard Institute for Space Studies USA GISS E R 5.00° x 3.91°
12 Institute of Atmospheric Physics China IAP FGOALS 1.0 G 2.81° x 3.00°
13 Institute for Numerical Mathematics Russia INM CM 3.0 5.00° x 4.00°
14 Center for Climate System Research, National Institute for Environmental Studies, and Frontier Research Center for Global Change
Japan MIROC 3.2 MEDRES 2.81° x 2.81°
15 Max Planck Institute for Meteorology Germany MPI ECHAM 5 1.88° x 1.88°
16 Meteorological Research Institute Japan MRI CGCM 2.3.2 2.81° x 2.81°
17 National Center for Atmospheric Research USA NCAR CCSM 3.0 1.41° x 1.41°
18 National Center for Atmospheric Research USA NCAR PCM 1 2.81° x 2.81°
CMIP3 Models Used
More information: http://www-pcmdi.llnl.gov/ipcc/model_documentation/ipcc_model_documentation.php
Spatial Comparisons
Linear re-gridding to 2.5° x 2.5° lon-lat Typical model resolution that is fine enough
to resolve the coastlines Precipitation Quantities for 1961-1998
Mean Frequency of wet days (precip. ≥ 0.254
mm/day) Standard deviation for wet days divided by
mean for wet days – precipitation variability 99th percentile for all days Generalized extreme value normalized scale
parameter for yearly maximum daily precipitation distribution – extreme precipitation variability
Mean Precipitation 1961-1998 (mm/day)
Convective parameterizations?[Iorio et al. 2004]
Improper terrain representation?
Agreement with IPCC AR4
Frequency of Wet Days 1961-1998 (days/year)
Normalized Standard Deviation for Wet Days 1961-1998
(dimensionless)
Could be related to too many wet days
99th Percentile for All Days 1961-1998 (mm/day)
Convective parameterizations again?
Example Generalized Extreme Value (GEV) Distribution
Representative of New Jersey in Observations
location parameter- center of distribution(45 mm/day)
scale parameter- spread of distribution(9 mm/day)
I plot scale/locati
on(0.2 in this
case)
GEV Normalized Scale Parameter for Yearly Maximum 1961-1998 (dimensionless)
Not enough variability of precipitation extremes
GEV Normalized Scale Parameter for Yearly Maximum 1961-1998 (dimensionless)
Does Spatial Resolution Make a Difference? Linear re-gridding to 5.0° x 4.0° lon-lat
Error- root mean square of absolute difference between each model and observations average (Iorio et al. 2004)
Plot error against finite grid equivalent resolution (# of global grid cells)
Fit least squares linear regression to error vs. resolution plot
Error vs. Resolution Results Statistically significant improvement
in the frequency of wet days with higher resolution
= model average
Model # Modeling Group Country Model ID Spatial Resolution(approximate - lon x lat)
1 Bjerknes Centre for Climate Research Norway BCCR BCM 2.0 2.81° x 2.81°
2 Canadian Centre for Climate Modelling & Analysis Canada CCCMA CGCM 3.1 T47 3.75° x 3.75°
3 Canadian Centre for Climate Modelling & Analysis Canada CCCMA CGCM 3.1 T63 2.81° x 2.81°
4 Centre National de Recherches Météorologiques France CNRM CM 3 2.81° x 2.81°
5 CSIRO Atmospheric Research Australia CSIRO MK 3.0 1.88° x 1.88°
6 CSIRO Atmospheric Research Australia CSIRO MK 3.5 1.88° x 1.88°
7 Geophysical Fluid Dynamics Laboratory USA GFDL CM 2.0 2.50° x 2.00°
8 Geophysical Fluid Dynamics Laboratory USA GFDL CM 2.1 2.50° x 2.00°
9 Goddard Institute for Space Studies USA GISS AOM 4.00° x 3.00°
10 Goddard Institute for Space Studies USA GISS E H 5.00° x 3.91°
11 Goddard Institute for Space Studies USA GISS E R 5.00° x 3.91°
12 Institute of Atmospheric Physics China IAP FGOALS 1.0 G 2.81° x 3.00°
13 Institute for Numerical Mathematics Russia INM CM 3.0 5.00° x 4.00°
14 Center for Climate System Research, National Institute for Environmental Studies, and Frontier Research Center for Global Change
Japan MIROC 3.2 MEDRES 2.81° x 2.81°
15 Max Planck Institute for Meteorology Germany MPI ECHAM 5 1.88° x 1.88°
16 Meteorological Research Institute Japan MRI CGCM 2.3.2 2.81° x 2.81°
17 National Center for Atmospheric Research USA NCAR CCSM 3.0 1.41° x 1.41°
18 National Center for Atmospheric Research USA NCAR PCM 1 2.81° x 2.81°
CMIP3 Models Used
More information: http://www-pcmdi.llnl.gov/ipcc/model_documentation/ipcc_model_documentation.php
Error vs. Resolution Results Statistically significant improvement
in the frequency of wet days with higher resolution
All other quantities showed decreasing error with higher resolution, but the linear fit was not statistically significant
All quantities showed low percentage of model error variability explained by the linear fit (r2)
Other potential reasons for variability in model error Different vertical resolutions Different grid types (e.g., spectral
resolution vs. finite grid) Different cloud and convective
parameterizations Different microphysics schemes Different ocean components Different radiation schemes
Ranking of Model Performance Ratio: Root mean square error for
each model divided by the average root mean square error for all models for each precipitation quantity Eliminates biases from quantities with
different units (e.g., mean precipitation, frequency of wet days)
Take average of ratio over precipitation quantities for each model and rank them
CMIP3 Model Ranking
All Precipitation Quantities Mean, Frequency of Wet Days, and Normalized Standard Deviation for Wet Days
99th Percentile and GEV Normalized Scale
Model Rank Average Error Model Rank Average Error Model Rank Average Error
1. MPI ECHAM 5 0.6408 1. MPI ECHAM 5 0.5783 1. GFDL CM 2.1 0.65352. CSIRO MK 3.5 0.7559 2. CSIRO MK 3.5 0.6974 2. GFDL CM 2.0 0.68723. MRI CGCM 2.3.2 0.7666 3. MRI CGCM 2.3.2 0.7251 3. MPI ECHAM 5 0.73464. GFDL CM 2.1 0.7954 4. NCAR CCSM 3.0 0.7448 4. CCCMA CGCM 3.1 T63 0.74125. NCAR CCSM 3.0 0.7987 5. Model Average 0.8621 5. Model Average 0.74576. Model Average 0.8155 6. MIROC 3.2 MEDRES 0.8895 6. CNRM CM 3 0.77627. CCCMA CGCM 3.1 T63 0.8457 7. GFDL CM 2.1 0.8901 7. CCCMA CGCM 3.1 T47 0.79998. MIROC 3.2 MEDRES 0.8647 8. CCCMA CGCM 3.1 T63 0.9154 8. MIROC 3.2 MEDRES 0.82759. CCCMA CGCM 3.1 T47 0.8886 9. CSIRO MK 3.0 0.932 9. MRI CGCM 2.3.2 0.828810. GFDL CM 2.0 0.8982 10. CCCMA CGCM 3.1 T47 0.9477 10. CSIRO MK 3.5 0.843611. INM CM 3.0 0.9161 11. INM CM 3.0 0.9489 11. INM CM 3.0 0.867012. CSIRO MK 3.0 0.9222 12. BCCR BCM 2.0 1.0024 12. NCAR CCSM 3.0 0.879713. BCCR BCM 2.0 0.9712 13. GFDL CM 2.0 1.0388 13. CSIRO MK 3.0 0.907514. CNRM CM 3 1.0737 14. GISS E R 1.0968 14. NCAR PCM 1 0.9147
15. NCAR PCM 1 1.1117 15. IAP FGOALS 1.0 G 1.215 15. BCCR BCM 2.0 0.924316. IAP FGOALS 1.0 G 1.1613 16. NCAR PCM 1 1.243 16. IAP FGOALS 1.0 G 1.080717. GISS E H 1.3447 17. CNRM CM 3 1.272 17. GISS AOM 1.167118. GISS AOM 1.3743 18. GISS E H 1.3503 18. GISS E H 1.336419. GISS E R 1.8701 19. GISS AOM 1.5125 19. GISS E R 3.0301
More information: http://www-pcmdi.llnl.gov/ipcc/model_documentation/ipcc_model_documentation.php
CMIP3 Model Ranking
All Precipitation Quantities Mean, Frequency of Wet Days, and Normalized Standard Deviation for Wet Days
99th Percentile and GEV Normalized Scale
Model Rank Average Error Model Rank Average Error Model Rank Average Error
1. MPI ECHAM 5 0.6408 1. MPI ECHAM 5 0.5783 1. GFDL CM 2.1 0.65352. CSIRO MK 3.5 0.7559 2. CSIRO MK 3.5 0.6974 2. GFDL CM 2.0 0.68723. MRI CGCM 2.3.2 0.7666 3. MRI CGCM 2.3.2 0.7251 3. MPI ECHAM 5 0.73464. GFDL CM 2.1 0.7954 4. NCAR CCSM 3.0 0.7448 4. CCCMA CGCM 3.1 T63 0.74125. NCAR CCSM 3.0 0.7987 5. Model Average 0.8621 5. Model Average 0.74576. Model Average 0.8155 6. MIROC 3.2 MEDRES 0.8895 6. CNRM CM 3 0.77627. CCCMA CGCM 3.1 T63 0.8457 7. GFDL CM 2.1 0.8901 7. CCCMA CGCM 3.1 T47 0.79998. MIROC 3.2 MEDRES 0.8647 8. CCCMA CGCM 3.1 T63 0.9154 8. MIROC 3.2 MEDRES 0.82759. CCCMA CGCM 3.1 T47 0.8886 9. CSIRO MK 3.0 0.932 9. MRI CGCM 2.3.2 0.828810. GFDL CM 2.0 0.8982 10. CCCMA CGCM 3.1 T47 0.9477 10. CSIRO MK 3.5 0.843611. INM CM 3.0 0.9161 11. INM CM 3.0 0.9489 11. INM CM 3.0 0.867012. CSIRO MK 3.0 0.9222 12. BCCR BCM 2.0 1.0024 12. NCAR CCSM 3.0 0.879713. BCCR BCM 2.0 0.9712 13. GFDL CM 2.0 1.0388 13. CSIRO MK 3.0 0.907514. CNRM CM 3 1.0737 14. GISS E R 1.0968 14. NCAR PCM 1 0.9147
15. NCAR PCM 1 1.1117 15. IAP FGOALS 1.0 G 1.215 15. BCCR BCM 2.0 0.924316. IAP FGOALS 1.0 G 1.1613 16. NCAR PCM 1 1.243 16. IAP FGOALS 1.0 G 1.080717. GISS E H 1.3447 17. CNRM CM 3 1.272 17. GISS AOM 1.167118. GISS AOM 1.3743 18. GISS E H 1.3503 18. GISS E H 1.336419. GISS E R 1.8701 19. GISS AOM 1.5125 19. GISS E R 3.0301
More information: http://www-pcmdi.llnl.gov/ipcc/model_documentation/ipcc_model_documentation.php
CMIP3 Model Ranking
All Precipitation Quantities Mean, Frequency of Wet Days, and Normalized Standard Deviation for Wet Days
99th Percentile and GEV Normalized Scale
Model Rank Average Error Model Rank Average Error Model Rank Average Error
1. MPI ECHAM 5 0.6408 1. MPI ECHAM 5 0.5783 1. GFDL CM 2.1 0.65352. CSIRO MK 3.5 0.7559 2. CSIRO MK 3.5 0.6974 2. GFDL CM 2.0 0.68723. MRI CGCM 2.3.2 0.7666 3. MRI CGCM 2.3.2 0.7251 3. MPI ECHAM 5 0.73464. GFDL CM 2.1 0.7954 4. NCAR CCSM 3.0 0.7448 4. CCCMA CGCM 3.1 T63 0.74125. NCAR CCSM 3.0 0.7987 5. Model Average 0.8621 5. Model Average 0.74576. Model Average 0.8155 6. MIROC 3.2 MEDRES 0.8895 6. CNRM CM 3 0.77627. CCCMA CGCM 3.1 T63 0.8457 7. GFDL CM 2.1 0.8901 7. CCCMA CGCM 3.1 T47 0.79998. MIROC 3.2 MEDRES 0.8647 8. CCCMA CGCM 3.1 T63 0.9154 8. MIROC 3.2 MEDRES 0.82759. CCCMA CGCM 3.1 T47 0.8886 9. CSIRO MK 3.0 0.932 9. MRI CGCM 2.3.2 0.828810. GFDL CM 2.0 0.8982 10. CCCMA CGCM 3.1 T47 0.9477 10. CSIRO MK 3.5 0.843611. INM CM 3.0 0.9161 11. INM CM 3.0 0.9489 11. INM CM 3.0 0.867012. CSIRO MK 3.0 0.9222 12. BCCR BCM 2.0 1.0024 12. NCAR CCSM 3.0 0.879713. BCCR BCM 2.0 0.9712 13. GFDL CM 2.0 1.0388 13. CSIRO MK 3.0 0.907514. CNRM CM 3 1.0737 14. GISS E R 1.0968 14. NCAR PCM 1 0.9147
15. NCAR PCM 1 1.1117 15. IAP FGOALS 1.0 G 1.215 15. BCCR BCM 2.0 0.924316. IAP FGOALS 1.0 G 1.1613 16. NCAR PCM 1 1.243 16. IAP FGOALS 1.0 G 1.080717. GISS E H 1.3447 17. CNRM CM 3 1.272 17. GISS AOM 1.167118. GISS AOM 1.3743 18. GISS E H 1.3503 18. GISS E H 1.336419. GISS E R 1.8701 19. GISS AOM 1.5125 19. GISS E R 3.0301
More information: http://www-pcmdi.llnl.gov/ipcc/model_documentation/ipcc_model_documentation.php
CMIP3 Model Ranking
All Precipitation Quantities Mean, Frequency of Wet Days, and Normalized Standard Deviation for Wet Days
99th Percentile and GEV Normalized Scale
Model Rank Average Error Model Rank Average Error Model Rank Average Error
1. MPI ECHAM 5 0.6408 1. MPI ECHAM 5 0.5783 1. GFDL CM 2.1 0.65352. CSIRO MK 3.5 0.7559 2. CSIRO MK 3.5 0.6974 2. GFDL CM 2.0 0.68723. MRI CGCM 2.3.2 0.7666 3. MRI CGCM 2.3.2 0.7251 3. MPI ECHAM 5 0.73464. GFDL CM 2.1 0.7954 4. NCAR CCSM 3.0 0.7448 4. CCCMA CGCM 3.1 T63 0.74125. NCAR CCSM 3.0 0.7987 5. Model Average 0.8621 5. Model Average 0.74576. Model Average 0.8155 6. MIROC 3.2 MEDRES 0.8895 6. CNRM CM 3 0.77627. CCCMA CGCM 3.1 T63 0.8457 7. GFDL CM 2.1 0.8901 7. CCCMA CGCM 3.1 T47 0.79998. MIROC 3.2 MEDRES 0.8647 8. CCCMA CGCM 3.1 T63 0.9154 8. MIROC 3.2 MEDRES 0.82759. CCCMA CGCM 3.1 T47 0.8886 9. CSIRO MK 3.0 0.932 9. MRI CGCM 2.3.2 0.828810. GFDL CM 2.0 0.8982 10. CCCMA CGCM 3.1 T47 0.9477 10. CSIRO MK 3.5 0.843611. INM CM 3.0 0.9161 11. INM CM 3.0 0.9489 11. INM CM 3.0 0.867012. CSIRO MK 3.0 0.9222 12. BCCR BCM 2.0 1.0024 12. NCAR CCSM 3.0 0.879713. BCCR BCM 2.0 0.9712 13. GFDL CM 2.0 1.0388 13. CSIRO MK 3.0 0.907514. CNRM CM 3 1.0737 14. GISS E R 1.0968 14. NCAR PCM 1 0.9147
15. NCAR PCM 1 1.1117 15. IAP FGOALS 1.0 G 1.215 15. BCCR BCM 2.0 0.924316. IAP FGOALS 1.0 G 1.1613 16. NCAR PCM 1 1.243 16. IAP FGOALS 1.0 G 1.080717. GISS E H 1.3447 17. CNRM CM 3 1.272 17. GISS AOM 1.167118. GISS AOM 1.3743 18. GISS E H 1.3503 18. GISS E H 1.336419. GISS E R 1.8701 19. GISS AOM 1.5125 19. GISS E R 3.0301
More information: http://www-pcmdi.llnl.gov/ipcc/model_documentation/ipcc_model_documentation.php
Conclusions
CMIP3 models underestimate mean and extreme precipitation amounts near the Gulf Coast Convective parameterizations (Iorio et al. 2004)
CMIP3 models produce precipitation days too frequently, especially in the north and west Higher resolution models perform much better
CMIP3 models have too little variability in all precipitation and extreme precipitation in the northern interior west
The MPI ECHAM5 is the best, the model average is better than the majority of individual models, and the GISS models are the worst with 20th century precipitation characteristics over the US
Future Directions
Understand the reasons for differences in model performance What makes the MPI ECHAM5 so good?
Evaluate the ability of CMIP3 models to simulate precipitation changes Time period used here is too short for a
reliable analysis Expand the evaluation of CMIP3
precipitation to other regions
References
Dyer, J. L., and T. L. Mote, 2006: Spatial variability and patterns of snow depth over North America, Geophys. Res. Lett., 33, L16503, doi:10.1029/2006GL027258.
Emori, S., A. Hasegawa, T. Suzuki, and K. Dairaku, 2005: Validation, parameterization dependence and future projection of daily precipitation simulated with an atmospheric GCM, Geophys. Res. Lett., 32, L06708, doi:10.1029/2004GL022306.
Groisman, P. Y., R. W. Knight, D. R. Easterling, T. R. Karl, G. C. Hegerl, and V. N. Razuvaev, 2005: Trends in intense precipitation in the climate record, J. Clim., 18, 1326-1350.
Higgins, R. W., W. Shi, E. Yarosh, and R. Joyce, 2000: Improved United States precipitationquality control system and analysis. NCEP/Climate Prediction Center Atlas No. 7, published online at http://www.cpc.ncep.noaa.gov/research_papers/ncep_cpc_atlas/7/.
Iorio, J. P., P. B. Duffy, B. Govindasamy, S. L. Thompson, M. Khairoutdinov, and D. Randall, 2004: Effects of model resolution and subgrid scale physics on the simulation of precipitation in the continental United State, Clim. Dyn., 23, 243–258, doi:10.1007/s00382-004-0440-y.
Kiktev, D., D. M. H. Sexton, L. Alexander, and C. K. Folland, 2003: Comparison of modeled and observed trends in indices of daily climate extremes, J. Clim., 16, 3560–3571.
Kimoto, M., N. Yasutomi, C. Yokoyama, and S. Emori, 2005: Projected changes in precipitation characteristics near Japan under the global warming, Scientific Online Letters on the Atmosphere, 1, 85–88, doi:10.2151/sola.2005-023.
References Continued
Meehl, G. A., C. Covey, T. Delworth, M. Latif, B. McAvaney, J. F. B. Mitchell, R. J. Stouffer, and K. E. Taylor, 2007: The WCRP CMIP3 multi-model dataset: A new era in climate change research, Bull. Amer. Meteor. Soc., 88, 1383-1394.
Pall, P., M. R. Allen, and D. A. Stone, 2007: Testing the Clausius-Clapeyron constraint on changes in extreme precipitation under CO2 warming, Clim. Dyn., 28, 351-363.
Randall, D. A. and Coauthors, 2007: Climate Models and Their Evaluation. In: Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change [Solomon, S., D. Qin, M. Manning, Z. Chen, M. Marquis, K. B. Averyt, M. Tignor and H. L. Miller (eds.)]. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA.
Sun, Y., S. Solomon, A. Dai, and R. W. Portmann, 2007: How often will it rain? J. Clim., 20, 4801-4818.
United States Department of the Interior, United States Geological Survey, 2006: Fact Sheet: Flood Hazards- A National Threat. Available at http://pubs.usgs.gov/fs/2006/3026/.
For more plots, see http://envsci.rutgers.edu/~toine379/extremeprecip/home