1
Data Discovery and Access to The International Surface Pressure
Databank (ISPD)
Thomas Cram Gilbert P. Compo*Doug Schuster Chesley McColl*Steven Worley
National Center for Atmospheric Research, Boulder, CO*NOAA/CIRES, Boulder, CO
AGU 2012 Fall Meeting: IN44A-05
2
Research Data Archive (RDA) at NCARrda.ucar.edu
1. 600+ distinct datasets for climate and weather research
2. Collections: ocean & atmosphere observations, analyses, reanalyses, operational NWP outputs
3. Free and open accesshttp://rda.ucar.edu
AGU 2012 Fall Meeting: IN44A-05
3
ISPD Overview• World’s largest collection of surface & sea level pressure observations
Land station Marine observations Tropical cyclone best track
• Period (version 2): 1768 – 2010• Volume: 465 Gbyte• Available since Aug 2010
AGU 2012 Fall Meeting: IN44A-05
4
ISPD Overview• 60+ Contributors
o Atmospheric Circulation Reconstructions over the Earth (ACRE)o Australian Bureau of Meteorologyo British Antarctic Surveyo Cook Islands Meteorological Serviceo Danish Meteorological Instituteo Deutscher Wetterdienst (DWD; German Weather Service)o European and North Atlantic Daily to Multidecadal Climate Variability
(EMULATE)o ETH Zurich, Switzerlando GCOS/WCRP Working Group on Observational Data Sets for
Reanalysiso MANY MANY MORE….
• Assembled by NOAA/ESRL, CIRES (Univ. of Colorado), & NOAA/NCDC
AGU 2012 Fall Meeting: IN44A-05
5
20th Century Reanalysis• Global reanalysis of atmospheric
circulation• Period: 1869 – 2010• Assimilates ISPDv2 as input• Compo et al. (2011) QJRMS
AGU 2012 Fall Meeting: IN44A-05
6
20th Century Reanalysis:Oct 1950 mean 1000 hPa temperature
ISPD stations ISPD marine obs
AGU 2012 Fall Meeting: IN44A-05
7
ISPD stations ISPD marine obs
AGU 2012 Fall Meeting: IN44A-05
20th Century Reanalysis:Nov 1960 mean 1000 hPa temperature
8
• ISPD obs assimilated into 20CR• 20CR data quality control feedback
contained in ISPD Provides estimated uncertainty in obs Helps improve underlying
observational database
AGU 2012 Fall Meeting: IN44A-05
ISPD & the 20th Century Reanalysis
9
ISPD Sample Annual Station Distribution
1850 * Land stations only* No marine stations
AGU 2012 Fall Meeting: IN44A-05
10
18501900
* Land stations only* No marine stations
AGU 2012 Fall Meeting: IN44A-05
ISPD Sample Annual Station Distribution
11
18501900
1950* Land stations only* No marine stations
AGU 2012 Fall Meeting: IN44A-05
ISPD Sample Annual Station Distribution
12
18501900
19502000
* Land stations only* No marine stations
AGU 2012 Fall Meeting: IN44A-05
ISPD Sample Annual Station Distribution
13
ISPD Observations/Year
Figure courtesy Chesley McColl, NOAA/ESRL
2010: 53 Million
~ 1.5 Billion total observations
AGU 2012 Fall Meeting: IN44A-05
14
Data Access: Problem Background• Large computational/storage resources
needed– Store data– Extract desired data from large grids/files– Convert data to desirable format(s)
Scientific data centers have these resources
Individual researchers generally don’t
AGU 2012 Fall Meeting: IN44A-05
15
• Goals– Make data more accessible and easier to use for
individual researchers• Reasonable access volumes• Desired data formats• User defined parameters/grids
• Researchers stay focused on research
AGU 2012 Fall Meeting: IN44A-05
Data Access: Problem Background
16
ISPD Data Access Services
• Powerful computing resources @ NCAR
• Large disk storage (~ 0.5 PB)• Rich and detailed metadata• Direct file download via web• Customized data sub-setting• HDF-5 to ASCII software tools
AGU 2012 Fall Meeting: IN44A-05
17
ISPD Metadata Features• Both group- and file-level metadata• Drive interfaces for file grouping and sub-
setting tools• Support efficient back-end processing• Improve scalability• Provide “quick look” at data samples
AGU 2012 Fall Meeting: IN44A-05
18
ISPD Metadata Interface Example
AGU 2012 Fall Meeting: IN44A-05
19AGU 2012 Fall Meeting: IN44A-05
20AGU 2012 Fall Meeting: IN44A-05
21AGU 2012 Fall Meeting: IN44A-05
22
Data Access: ISPD Subset Interface
AGU 2012 Fall Meeting: IN44A-05
23
ISPD Data Access ServicesTemporal range sub-
setting (daily)
Spatial sub-setting Lat/Lon region Individual station ID
24
ISPD Data Access Services
Data sub-setting options (cont.)• Observation type
Land station Marine obs Radiosonde Dropsonde TC best track
25
ISPD Data Access Services
• Subsetting processed in delayed mode• E-mail notification• Download via server-provided scripts
(wget)
26
ISPD 2012 Subset metrics
Data accessed: ~ 6.5 TB
Data served: ~ 46 GB
27
Summary & Future Directions
• RDA – Supply “User Friendly” Data Parameter & spatial sub-setting Metadata discovery Format conversion Improved and additional services
• NWSC-Cheyenne opening – more computing power
AGU 2012 Fall Meeting: IN44A-05
28
• DOI assignment
• Geoscience Data Journal article
• ISPD v3 (1755-2010) Spring 2013
AGU 2012 Fall Meeting: IN44A-05
ISPD Forthcoming