1 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Data Support for Validation of
EOS Land Products
Data Support for Validation of
EOS Land Products
Larry Voorhees, Dick Olson, Bob Cook, ORNL* DAAC
Jeff Morisette, NASA GSFCJohn Dwyer, EDC DAAC
Terra Data Products ReviewNASA HQ
January 7-8, 2002
Larry Voorhees, Dick Olson, Bob Cook, ORNL* DAAC
Jeff Morisette, NASA GSFCJohn Dwyer, EDC DAAC
Terra Data Products ReviewNASA HQ
January 7-8, 2002 * Oak Ridge National Laboratory is managed by UT-Battelle, LLC, for the U.S. Dept. of Energy under contract DE-AC05-00OR22725.
2 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Contributors and ParticipantsContributors and Participants• Jeff Privette, Jeff Morisette, Nazmi El Saleous,
and Robert Wolfe, NASA GSFC
• John Dwyer, EDC DAAC
• Steve Running, U Montana
• Ranga Myneni, Boston U
• Alfredo Huete, U Arizona
• Dennis Baldocchi, UCLA
• Tom Boden, AmeriFlux ORNL
• Many other Science Team and EOS Validation Investigators
3 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
SummarySummary• Approach for supporting validation has been
designed, reviewed, and tested• Distribution systems in place and working• Data for validation are available and more data
are being added• MODIS Science Team, EOS Validation PIs, and
others are using the systems• Plans are in place to communicate validation
results via special journal issues and workshops• Need to extend correlative analysis to global
network of sites
4 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
OutlineOutline1. Validation Approach2. Data Available for Validation
– high resolution imagery– in situ data
3. Intercomparison Activities4. Data Use5. Communication6. Path Forward7. What works & what needs to be improved?
5 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
LAI/fPARNPP
In-situ ObservationsIn-situ Observations
??Remote SensingRemote Sensing
LAI/fPAR NPP
MODIS Land Validation: Level 2 to 4 data products primarily ecosystem variables
- Successful validation depends on determining measurement uncertainties for specific uses
- Validation of remote sensing land products at a global scale is an unprecedented scientific challenge
ValidationValidation
“Assessing by independent means the uncertainties of data products”
6 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
ApproachApproachTo compare MODIS products with parameters derived from higher-resolution airborne or satellite sensors and data from ground-based studies at validation ‘core sites’, flux towers, data from literature, etc.
7 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
ProductScience Team
EOS Validation Investigator
Additional Validation Data
Surface Reflectance Vermote Liang, Meyer AERONET
Land Temperature Wan Hook
Snow & Ice Hall Li, Nolin, Shi
BRDF / Albedo Strahler and Muller Liang, Privette (SAVE) AERONET, BSRN, SAFARI
Veg Index Huete Schowengerdt Chiba U / land cover helicopter
LAI / fPAR Myneni Gower, Meyer BigFoot, Krasnoyarsk
PSN / NPP Running Baldocchi, Olson BigFoot, FLUXNET, AmeriFlux
Fire / burn scars Justice Hao SAFARI 2000
Land Cover Strahler BigFoot, International Groups
VCC / VCF Townshend RESAC
MODIS Atmospheric King Multiple SAFARI 2000, others
MOPITT Drummond Multiple SAFARI 2000, others
MISR Diner Multiple SAFARI 2000, others
Land Products ValidationLand Products Validation
Based on MODIS Land Team Validation Update for Terra and Aqua, December 2000
Primary Products SupportedPrimary Products Supported
8 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
EOS Land Validation Core SitesEOS Land Validation Core Sites• Based on existing stations, resources, experts
• FLUXNET, AERONET, BigFoot, LTER, etc.
• Long-term monitoring
• Jointly nominated by Instrument and Validation PIs
• 26 sites stratified by six biomes
• Promote resource sharing and synergies in validating multiple sensors and products
• Imagery and in situ data available for many uses, not just validation
9 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
• Remote Sensing Data LandSat 5, ETM+, IKONOS, ASTER, airborne sensors, subsetted over the core sites
• in situ Site Data Collected by various groups at core sites
• Networks AERONET, FLUXNET, etc.
• Ancillary Site Data / GIS Layers - site variables - elevation - land cover - reference layers (e.g., political boundaries, airports, water bodies)
• Remote Sensing Data LandSat 5, ETM+, IKONOS, ASTER, airborne sensors, subsetted over the core sites
• in situ Site Data Collected by various groups at core sites
• Networks AERONET, FLUXNET, etc.
• Ancillary Site Data / GIS Layers - site variables - elevation - land cover - reference layers (e.g., political boundaries, airports, water bodies)
Data for EOS Land Validation Core SitesData for EOS Land
Validation Core Sites
10 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Remote Sensing Data for Core SitesRemote Sensing Data for Core Sites
EOS Core Site Imagery DataMODISSeaWiFSETM+ 2 1 7 3 1 1 7 1 1 5 5 2 1 8 1 2 9 5 1 10 2 1 1 2 5IKONOS 1 3 1 5 5 1 2 4 4 1 1 1 2 3 5 2 2 1 3 1 1 3 5ASTER 1 7 4 4 3 12GLCTSTM 90's eraMQUALSAVIRISCRESSGLCFEO-1MISR planned
11 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Validation Subsetting by MODAPSValidation Subsetting by MODAPS• EOS Core Sites
(200 x 200 km) to EDC
• EOS Core Sites and GT-NET sites (11 x 31 km) to UMt and ORNL
• LAI Inter-comp. Sites (200 x 200 km) to EDC
• SAFARI 2000 Sites (200 x 200 km) to UMd and South Africa Disaster Management Center Provided in hdf format,
integerized sinusoidal projection
Provided in hdf format,integerized sinusoidal projection
12 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Posting MODIS Productsat ORNL DAAC
Posting MODIS Productsat ORNL DAAC
• Subset to 7x7 km, re-projected to UTM to relate to field work – ORNL worked with EDC to refine
MODIS Reprojection Tool
• Reformat into ASCII• Provide a single expanding
table for each site-product– rows representing dates– columns values for the 49 pixels– includes QA flags
MODIS Products: – Veg Index– fPAR / LAI– PSN / NPP– BRDF / Albedo– Surface Reflectance– Surface Temperature– LC/LCC
MODIS Products: – Veg Index– fPAR / LAI– PSN / NPP– BRDF / Albedo– Surface Reflectance– Surface Temperature– LC/LCC
UTM
MODIS sinusoidalMODIS sinusoidal
13 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Data notavailable
MODIS 8-day LAI ProductMODIS 8-day LAI Product
All pixels in5x5 km cutoutGreen = rejectedBlue = accepted
Acceptedpixels basedon QA flags
Park Fallsforest site08NOV2000 – 16OCT2001
Park Fallsforest site08NOV2000 – 16OCT2001
The scatter of blue (3x3 km) and green (5x5 km) dots shows variability in LAI around the flux tower.
The scatter of blue (3x3 km) and green (5x5 km) dots shows variability in LAI around the flux tower.
LAI of center pixel
14 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
ORNL DAAC in situ Data ORNL DAAC in situ Data
• Data needs and schedules defined by working with MODLAND Validation scientists
• Investigators have responsibility to process, document, QA, and provide access to their data
• Provide Mercury system for search and retrieval, for both in situ and remote sensing data
• Eventually move Land Validation data to long-term archive (ORNL DAAC)
• Also provide Regional and Global data sets for modeling as part of ORNL DAAC mission
15 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
in situ Data for Core Sitesin situ Data for Core Sites
16 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Site Characteristics DataSite Characteristics Data Most core validation sites belong to several
networks that provide ancillary information
• ORNL is compiling and providing site characteristics data for the Core Sites– NPP, LAI, fPAR, phenology, micrometeorology, climatology,
soils, vegetation cover, etc.– Data can be used for validating other sensors or
for global change research
• Assemble metadata– documenting methods, processing, uncertainty, data source– examining relationships between parameters for QA checks
FLUXNET157 Towers
18 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
ORNL DAACValidation Data
ORNL DAACValidation Data
• Historical data from literature– Net Primary Productivity
(point and grid)– Leaf Area Index (LAI)
• Regional and Global data sets– ORNL DAAC archive
• Climate (7) • Vegetation (~100)• Soils (9)• Hydrology (3)
– 119 data sets registered in Mercury re veg, land use, soil, climate, hydrology, etc.
LAI – 1,000 estimatesLAI – 1,000 estimates
NPP – 7,000 0.5º grid cellsNPP – 7,000 0.5º grid cells
NPP – 1,000 site estimates
19 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
BigFoot: Linking in situ Measurements,
Remote Sensing, and Models
BigFoot: Linking in situ Measurements,
Remote Sensing, and Models
BigFootBigFoot
• Using ground measurements, remote sensing, and ecosystem models
• Measuring land cover, LAI, fPAR, and NPP
• Scaling from field measurements to 5 x 5 km grid
20 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
SAFARI 2000SAFARI 2000• Coordinated field campaign in southern Africa
with links to MODIS Land and MODIS Atmosphere validation• Intensive campaigns in 1999 - 2001 to examine
land-atmosphere interactions
• ORNL DAAC provides Mercury for access to:– in situ data collected at field sites
– airborne sensors (MAS, AirMISR)
– air chemistry measurements
21 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
GT-Net NPP ProjectGT-Net NPP Project• Coordinated by GTOS with
assistance from ILTER
• Use hierarchical modeling approach that combines satellite data and in situ observations
• Sites agreeing to measure NPP in exchange for MODIS products for their site – 25 sites in demo phase
22 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
LAI/FPAR Network LAI/FPAR Network • Joint activity of CEOS LPV and
MODLAND
• 31 sites participating worldwide
• MODLAND coordinating Landsat ETM+ acquisitions and MODIS LAI subsets– 12-18 month pathfinder
– leverage existing field campaigns
• Workshops:1998 to 2001– Proposed MODIS LAI to field LAI
inter-comparisonMajura ValleyMajura Valley KioloaKioloa
Moreton BayMoreton Bay
Lake GeorgeLake George
Batt ReefBatt ReefCooper CreekCooper Creek
MossmanMossmanDaintreeDaintree
Cape TribulationCape Tribulation
Townsville ReefTownsville Reef
Lake ArgyleLake Argyle
GoldsworthyGoldsworthy
Lake Eyre Lake Eyre
Mt FittonMt Fitton
UardryUardry
Tinga tinganaTinga tingana
Tanjil BrenTanjil Bren
AmburlaAmburla
Port HedlandPort Hedland
KakaduKakadu
PanoramaPanorama
Hyperion SitesHyperion Sites
Other CSIRO SitesOther CSIRO Sites
KeyKeyToolabinToolabin
KunothKunoth
X band sitesX band sites
ThangooThangoo
Lake FromeLake Frome
CIGSN SitesCIGSN Sites
Endevour ReefEndevour Reef
Ray MertonRay Merton
Earth Observation Earth Observation CentreCentre
http://www.eoc.csiro.au
23 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
MODIS - Model - Data Intercomparison
MODIS - Model - Data Intercomparison
• To validate Level 4 Land Products, daily photosynthesis and annual NPP
• Validating results of models embedded in NPP algorithm against other models
• Activities:– Real-time Validation– Ecosystem Model Data Intercomparison (EMDI)– LAI-Net– GT-NET activities
0
100
200
300
400
500
600
700
800
900
1000
1100
1200
1300
0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300
Measured NPP (g C m-2/yr)
Mo
de
l NP
P (
g C
m-2
/yr)
1 - Boreal Evergreen2 - Boreal Deciduous3 - Larch4 - Temperate forest5 - Dry Tropical Forest (ppt < 1500 mm/yr)6 - Semi-Wet Tropical Forest (ppt 1500-3000)7 - Wet Tropical Forest (> 3000)8 - C3 Grassland (Tave < 9 C)9 - C4 Grassland (Tave < 18)10 - C4 Grassland (Tave > 18)
Flux-Model-MODIS
Comparison
Flux-Model-MODIS
Comparison
ComparisonWorkshop?April 2002
CDIAC/AmeriFlux•Model initialization parameters•Weekly micromet data• Initial 0.5 hr flux data
DAAC/FLUXNET•MODIS products – 7x7 cutouts•FLUXNET gapfilled data•Links to field data via Mercury
MODAPS• 52 sites• 8 products• 11 km x 31 km• ISIN, .hdf
Modeling Groups
•5 groups, more coming
•Results posted on AmeriFlux
Individual Site Data
Additional validation data
17 sites
25 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
User Statistics for Validation Data
User Statistics for Validation Data
• Use of Mercury
• Referrals to registered data and documentation
• Data downloads of remote sensing images of core sites
26 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Validation Investigators Whose Data are Registered in MercuryValidation Investigators Whose Data are Registered in Mercury
Aber, JohnAhl, DouglasAugustine, John A.Baldocchi, DennisBruegge, Dr. Carol J.Burrows, Mr. Sean (5)Cohen, Warren B. (4)Conel, Dr. James E.Feldkirchner, Drew C.
(3)Gaitley, Barbara J.Gower, Dr. Stith T. (8)Gregory, Matthew J.
Honda, Dr. YoshiakiHuete, Dr. Alfredo R. (6)Jenkins, JulianKajiwara, Dr. KojiKirschbaum, Alan A. (3)Knjazihhin, YuriLiang, Dr. ShunlinMackay, Dr. David Scott Maiersperger, Thomas K.
(4)Meyer, Dave (4)Moran, SusanMorisette, Dr. Jeffrey T.
(11)Myers, Daryl R.
Myneni, Ranga B.Olson, Richard J. (2)Privette, Dr. Jeffrey L. (4)
Qin, WenhanRobbins, Cullen (4)Thome, Dr. Kurtis J.Townshend, Dr. John
R.G.Turner, David P.Walter-Shea, Elizabeth
A. Wylie, Bruce (3)al-Abbadi, Dr. Naif
(number of data sets indicated if >1)
27 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Validation Data through MercuryValidation Data through Mercury
• 57 validation data sets registered in Mercury– Land Validation – 49 data sets– SAFARI 2000 – 5 data sets from NRA selectees– ORNL DAAC Regional and Global Data – 3
• User statistics– July – December, 2001– 1,386 searches– 333 users
0
50
100
150
200
250
300
350
400
Jul Aug Sep Oct Nov Dec
UsersSearches
Mercury SearchesMercury
Searches
28 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Types of ReferralsTypes of Referrals
285
97
6Data Access
Documentation
Home Page
July 1 – November 30, 2001 (388 referrals, 49 unique users)
29 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Most Referred Data SetsMost Referred Data SetsJuly 1 - December 31, 2001
(388 referrals, 49 unique users)
Rank Data Set Title Referrals
1. Leaf Area Index (LAI) Data for BigFoot Sites, 1999-2001 47
2. Compiled Meteorological Data for BigFoot Sites 35
3. FPAR Data for BigFoot Sites, 2000-2001 33
4. Biomass Data for BigFoot Sites, 1999-2000 28
5. MQUALS Exotech Radiometer Data for Jornada LTER, New Mexico 22
6. Aboveground Net Primary Production (ANPP) for BigFoot Sites, 1999-2001
16
7. Aboveground Net Primary Production (ANPP) for BigFoot Sites, 1999-2000
13
8. HARV ANPP data for 2000 13
T9. Global Land Cover Test Sites (GLCTS) Data Products (AVHRR, MSS, TM, DEM, and Land Cover Maps) for EOS Land Validation Core Sites
11
T9. MODIS Land Data Products (MODLAND) for EOS Land Validation Core Sites
11
30 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Total Volume MODIS Files
Products downloaded from EDC by Core Site since Oct 99Products downloaded from EDC by Core Site since Oct 99
Source: MODIS Science Team Meeting, Dec 01Source: MODIS Science Team Meeting, Dec 01
ETM+ Scenes
30 GB30 GB
385385
9595
31 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
CommunicationsCommunications
• Guidance for data providers
• Publishing validation results
• ‘Popular articles’ for the general user community
32 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Guidance for Data ProvidersGuidance for Data Providers• Archiving Ecological Data and Information. Olson
and McCord. Chapter VII. In Michener and Brundt (ed.) 2000 “Ecological Data”. Blackwell Science.
• Managing Data from Multiple Disciplines, Scales, and Sites to Support Synthesis and Modeling. Olson et al. 1999. RSE.
• BigFoot Field Manual. Campbell et al.
1999. ORNL TM
• Guidelines for Submitting Tabular
Data. Cook et al. 2001. ESA Bulletin
• Mercury help and tutorial Web pages
Best Practices for Preparing Ecological
and Ground-Based Data Sets
to Share and Archive
Robert B. Cook, Richard J. Olson,
Paul Kanciruk, and Leslie A. Hook
Environmental Sciences Division
Oak Ridge National Laboratory
33 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Publishing Future Validation ResultsPublishing Future Validation Results
• Special section in IEEE TGARS• Spring/Summer 2002
• requirement that all related data be registered and
available via Mercury
• EMDI Workshop, April 2002
• SAFARI 2000 Meeting, October 2002
• Flux/Model/Data Intercomparison Workshop ?
• Other venues?
34 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Earth Observatory and NASA News articles feature:Earth Observatory and NASA News articles feature:
by Rachel Hauser, September 19, 2001http://earthobservatory.nasa.gov/Study/MODISCalibration
• MODLAND validation• FLUXNET database
August 2, 2001New NASA Field Campaign Sees the Forest for the Satellite New NASA Field Campaign Sees the Forest for the Satellite
by Rachel Hauser, September 24, 2001http://earthobservatory.nasa.gov/Study/Fluxnet/
35 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Path Forward
36 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Data resources for on-going validation (Terra, Aqua, NPP, etc.)
Data resources for on-going validation (Terra, Aqua, NPP, etc.)
• Make use of extant data• EOS Land Validation Core Sites should be
maintained, although EOS Investigations have either expired or will in early 2002
• Without formal “validation investigators” we need stronger connections with users who can provide feedback
• FLUXNET and BigFoot have continued funding• More collaboration with CEOS Land Product
Validation Subgroup
• Increase the quality and economy of global satellite product validation
– develop and promote international standards and protocols for field sampling, scaling, error budgeting, data exchange, and product evaluation
• Advocate mission-long validation programs for current and future earth observing satellites
Pilot projects• LAI• Fire / burn scars• Land cover
Pilot projects• LAI• Fire / burn scars• Land cover
Using EOS Land Validation Core Sites as an example to establish the ‘CEOS Land Validation Core sites’
Using EOS Land Validation Core Sites as an example to establish the ‘CEOS Land Validation Core sites’
38 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Validation . . . what worksValidation . . . what works• Growing buy-in to the ‘core sites’ concept
• Exploitation of in situ networks (e.g. flux towers, AERONET) and field investigations (e.g., LBA, SAFARI 2000)
• International partnerships and coordination
• Simplified data access with dedicated archives
• Frequent communication between Product PIs, Validation PIs, and data facilitators
39 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Validation . . . on-going concernsValidation . . . on-going concerns• 26 core and ~150 towers sites is a relatively small
number of sites, but it still takes a lot of effort
• Spatial differences / scaling - points vs. cells, and the need for a wide variety of ecosystem sites
• Temporal differences - satellite vs. field data
• Data inconsistencies - methods, variables, formats
• More data / better knowledge = greater cost– How much is enough?– Global representation – how many points in time, space,
and biome?
40 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Validation . . . on-going concerns (con’t)
Validation . . . on-going concerns (con’t)
• Control expectations– It takes a couple of years to validate a 1-yr Level 4 product– What’s the end point for validation? . . . will continually refine?
• Stronger interface to the user community and on-going collaboration with the international community
• Improve formal documentation and availability of results
• Consider establishing a ‘permanent’ NASA Office on Calibration and Validation
41 - ORNL DAAC | Terra Data Products Review, Jan 7-8, 2002
Comments . . . . Questions?Comments . . . . Questions?