Supporting the Validation of MODIS Land Products

1 - ORNL DAAC | EOS Data Products Review, Mar 10-11, 2003

Supporting the Validation of

MODIS Land Products

Supporting the Validation of

MODIS Land ProductsLarry Voorhees, Dick Olson, Bob Cook, ORNL*

DAACJeff Morisette, NASA GSFC

John Dwyer, LP DAAC

EOS Data Products ReviewNASA HQ

March 10-11, 2003

Larry Voorhees, Dick Olson, Bob Cook, ORNL* DAAC

Jeff Morisette, NASA GSFCJohn Dwyer, LP DAAC

EOS Data Products ReviewNASA HQ

March 10-11, 2003 * Oak Ridge National Laboratory is managed by UT-Battelle, LLC, for the U.S. Dept. of Energy under contract DE-AC05-00OR22725.

Much of the information in this image prepared by the NASA Goddard Space Flight Center came from MODIS.

This presentation should be viewed using ‘Slide Show’ with PowerPoint 2002


Contributors and ParticipantsContributors and Participants• Jeff Privette, Nazmi El Saleous, and

Robert Wolfe, NASA GSFC

• Tom Kalvelage, LP DAAC

• Steve Running, U Montana

• Ranga Myneni, Boston U

• Alfredo Huete, U Arizona

• Dennis Baldocchi, UC Berkeley

• Warren Cohen and the BigFoot Team

• Many other Science Teams and EOS Validation Investigators


OutlineOutline1. Validation Approach2. Data Available for Validation

– high resolution imagery– in situ data

3. User Statistics4. Communications and Future Activities5. What works, what’s ‘in the works’, and

what needs to be improved?


• Good science and resource management require understanding product accuracy / uncertainty

• Explicit statements of uncertainty foster an informed user community and improved use of data

• International environmental agreements imply products may be independently evaluated and possibly challenged

• As more, and similar, global products are produced by CEOS members, inter-use will require characterization of each product’s uncertainty

Why validate global land products?Why validate global land products?

the challenge . . .the challenge . . .the challenge . . .the challenge . . .

?Field and tower data are combined with high resolution imagery to produce high resolution products.

AggregateData Product

Correlate

Some graphics courtesy of BigFoot project

LP DAACLP DAAC

Field MeasurementsMODIS SceneORNL DAACORNL DAAC


validation in ‘stages’ . . .validation in ‘stages’ . . .validation in ‘stages’ . . .validation in ‘stages’ . . .

Stage 1 Product accuracy has been estimated using a small number of independent measurements obtained from selected locations and time periods and ground-truth/field program effort.

Stage 1 Product accuracy has been estimated using a small number of independent measurements obtained from selected locations and time periods and ground-truth/field program effort.

Stage 2 Product accuracy has been assessed over a widely distributed set of locations and time periods via several ground-truth and validation efforts.

Stage 2 Product accuracy has been assessed over a widely distributed set of locations and time periods via several ground-truth and validation efforts.

Stage 3 Product accuracy has been assessed and the uncertainties in the product well established via independent measurements in a systematic and statistically robust way representing global conditions.

Stage 3 Product accuracy has been assessed and the uncertainties in the product well established via independent measurements in a systematic and statistically robust way representing global conditions.


EOS Land Validation Core Sites• 26 sites stratified by six biomes• Based on existing stations, resources, experts

• FLUXNET, AERONET, BigFoot, LTER, etc. • Long-term monitoring• Jointly nominated by Instrument and Validation PIs

EOS Land Validation Core Sites• 26 sites stratified by six biomes• Based on existing stations, resources, experts

• FLUXNET, AERONET, BigFoot, LTER, etc. • Long-term monitoring• Jointly nominated by Instrument and Validation PIs

• Remote Sensing Data Landsat 5, ETM+, IKONOS, ASTER, airborne sensors, subsetted over the core sites

• in situ Site Data Collected by various groups at core sites, available via Mercury

• Networks AERONET, FLUXNET, etc.

• Ancillary Site Data/GIS Layers - site variables - elevation - land cover - reference layers (e.g., political boundaries, airports, water bodies)

EOS Validation Core Site Data

MODIS SubsetsSeaWiFS subsetsETM+ 1 5 13 5 1 4 15 1 1 5 9 3 1 11 1 8 11 2 1 12 4 1 2 4 5

IKONOS 1 1 1 4 1 4 7 1 2 4 4 1 1 1 2 3 5 2 2 3 3 1 1 4 1 5

ASTER 1 7 4 4 3 12

Global LC Test SitesGeoCover: 1990's era TMMQUALs AVIRIS Ancillary report (CRESS)Ancillary data (GLCF)AERONETEO-1 coverage

Remote Sensing Data for Core SitesRemote Sensing Data for Core SitesRemote Sensing Data for Core SitesRemote Sensing Data for Core Sites

in situin situ Data for Core Sites Data for Core Sitesin situin situ Data for Core Sites Data for Core Sites


0 40 80 120 160

armcartbarrow

bartonbebondvillboreasnboreass

cascadesharvard

howlandjiparanajornada

konzakrasnoyalaketaho

mandalgomaricopa

monguparkfall

sanpedrosevillet

skukuzatapajosuardry

usdaarsvcr

walkerbr

1999

2000

2001

2002

2003

Year represents the date of the ftp delivery, not necessarily the year of data acquisition

Number of files downloaded

Landsat downloads from LP DAAC by Core SiteLandsat downloads from LP DAAC by Core SiteSince October 1999Since October 1999

Landsat downloads from LP DAAC by Core SiteLandsat downloads from LP DAAC by Core SiteSince October 1999Since October 1999


Year represents the date of the ftp delivery, not necessarily the year of data acquisition

0 500 1000 1500 2000 2500 3000 3500

armcartbarrow

bartonbebondvillboreasnboreass

cascadesharvard

howlandjiparanajornada

konzakrasnoyalaketaho

mandalgomaricopa

monguparkfall

sanpedrosevillet

skukuzatapajosuardry

usdaarsvcr

walkerbr

Number of files downloaded

1999

2000

2001

2002

2003

MODIS downloads from LP DAAC by Core SiteMODIS downloads from LP DAAC by Core SiteSince October 1999Since October 1999

MODIS downloads from LP DAAC by Core SiteMODIS downloads from LP DAAC by Core SiteSince October 1999Since October 1999

LP DAAC also provides ftp access to MODIS and ETM+ over GOFC network, SAFARI 2000, and CEOS LAI-intercomparison sites, statistics not shown here

LP DAAC also provides ftp access to MODIS and ETM+ over GOFC network, SAFARI 2000, and CEOS LAI-intercomparison sites, statistics not shown here


FLUXNET (a collection of networks)• A global network of micrometeorological

tower sites that measure the exchanges of CO2, water vapor, and energy between terrestrial ecosystem and atmosphere

• Aggregated data available from ORNL

FLUXNET (a collection of networks)• A global network of micrometeorological

tower sites that measure the exchanges of CO2, water vapor, and energy between terrestrial ecosystem and atmosphere

• Aggregated data available from ORNL

> 200 towersand growing

Over 100 site-years of data

in-situin-situ data from ORNL for comparison data from ORNL for comparison with MODIS land products;with MODIS land products; NPP exampleNPP examplein-situin-situ data from ORNL for comparison data from ORNL for comparison with MODIS land products;with MODIS land products; NPP exampleNPP example

• Ecosystem site characteristics for core sites

• Ecosystem site characteristics for core sites

• Historical data from literature– Net Primary Productivity (NPP)

• point (~1,000) and grid (7,000 0.5 deg)

– Leaf Area Index (~1000)

• Regional and Global data sets– ORNL DAAC archive

• Climate (7) • Vegetation (~100)• Soils (9)• Hydrology (3)

– >180 data sets registered in Mercury (data held by others) regarding vegetation, land use, soil, climate, hydrology, etc.

www.daac.ornl.gov/FLUXNET/fluxnet.html

•MODAPS at GSFC (NASA) provides 11x 31 km subsets of selected products in HDF and SIN Projection for 274 sites in Collection 4

•ORNL DAAC subsets to 7 x 7 km and reformats from HDF images into ASCII files of pixel values and QA flags, expanding as data are collected

•ASCII files are posted on ORNL DAAC’s ftp site

MODIS ASCII Subsets ( 7 x 7 MODIS ASCII Subsets ( 7 x 7 km) km) MODIS ASCII Subsets ( 7 x 7 MODIS ASCII Subsets ( 7 x 7 km) km)

from Collection 3 November 2000 – July 2002

Red – Median of 5x5 Pixel Blue – Good QA Green – Failed QA

FPAR for Walker Branch

from Collection 3

“. . . the MODIS fluxtower cutouts are the best damn thing we ever did. People [ecologists] down here [Australia] are piling on that data, and it is producing way more use of MODIS data than otherwise would have happened!!” -- Steve Running E-mail, Feb 26, 2003 --

“. . . the MODIS fluxtower cutouts are the best damn thing we ever did. People [ecologists] down here [Australia] are piling on that data, and it is producing way more use of MODIS data than otherwise would have happened!!” -- Steve Running E-mail, Feb 26, 2003 --


Validation Support at ORNL: FY 2002Validation Support at ORNL: FY 2002Validation Support at ORNL: FY 2002Validation Support at ORNL: FY 2002• Developed process to produce MODIS ASCII subsets

• Tested and provided feedback on the MODIS Reprojection Tool

• Documented MODIS products (Collection 3) and provided QA flags for subsets, and identified improvements for the existing documentation (3 products for 25 sites)

• Participated in MODIS Vegetation and Albedo Workshops

• Described processing and availability of MODIS ASCII subsets

• Added 249 additional sites for validation (total 274)− More flux tower, field, and albedo sites

• Provided data for validation via Mercury


Name Product

MOD09A1 Surface Reflectance

MOD11A2 Surface Temperature

MOD12Q1 Land Cover

MOD13A2 NDVI / EVI

MOD15A2 LAI / fPAR

MOD17A2 GPP, PSN

MOD17A3 Annual NPP

MOD43 Albedo, NBAR

• Expanding number of sites and products for the MODIS subsets

• Posting MODIS Collection 4: new algorithms, SIN projection

• Supporting MODIS Validation Workshop – August 2003

• Expanding in-situ data collection – FLUXNET, Mercury

Validation Support at ORNL: FY 2003Validation Support at ORNL: FY 2003Validation Support at ORNL: FY 2003Validation Support at ORNL: FY 2003

MODIS ASCII Subsets:8 products, 8-day & annual

274 sites, 7x7 km

MODIS ASCII Subsets:8 products, 8-day & annual

274 sites, 7x7 km


User Statistics for Validation Data

User Statistics for Validation Data

• Use of Mercury

• Referrals to registered data and documentation

• Data downloads of in situ measurements (ORNL DAAC)

• Data downloads of remote sensing images of core sites (LP DAAC) – presented earlier


Validation Data through MercuryValidation Data through Mercury• 57 validation data sets registered in Mercury

– Land Validation and SAFARI 2000

• User statistics: July 2001 – January 2003

0

100

200

300

400

500

600

Jul Sep Nov Jan Mar May Jul Sep Nov Jan

Users

Searches

Mercury SearchesMercury Searches

2001 2002

- 1,409 users- 4,488 searches


Types of Referrals*Types of Referrals*

735

189

29Data Access

Documentation

Home Page

July 1, 2001 – January 30, 2003 (953 referrals)

*‘Referrals’ are links to where the data, documentation, and home pages reside.


ORNL DAACUser Statistics in CY 2002

ORNL DAACUser Statistics in CY 2002

Data Products Delivered in CY 2002(Total = 51,074)

Other1%

OTTER1%

NPP12%

FLUXNET21%

BOREAS4%

LBA19%

Vegetation Collections

1%

FIFE2%

RivDIS16%

SAFARI20009%

SoilCollections

2%

ClimateCollections

12%

Data Products Delivered in CY 2002(Total = 51,074)

Other1%

OTTER1%

NPP12%

FLUXNET21%

BOREAS4%

LBA19%


1%

FIFE2%

RivDIS16%

SAFARI20009%

SoilCollections

2%

ClimateCollections

12%

Users by Project in CY 2002(Total = 4,083)

LBA2%


9%

FIFE4%

BOREAS8%

FLUXNET6%

ClimateCollections

2%SoilCollections

7%

SAFARI20003%

RivDIS9%

OTTER2%

NPP41%

Other6%

Users by Project in CY 2002(Total = 4,083)

LBA2%


9%

FIFE4%

BOREAS8%

FLUXNET6%

ClimateCollections

2%SoilCollections

7%

SAFARI20003%

RivDIS9%

OTTER2%

NPP41%

Other6%

• NPP, FLUXNET, and Climate Collections data are among the most often requested products

• > 40% of all users request NPP data sets

Numerous user accolades . . .Numerous user accolades . . .“I believe this to be the most outstanding archive and method of distribution around. The government and labs have a lot to be proud of here. Thanks”

“I believe this to be the most outstanding archive and method of distribution around. The government and labs have a lot to be proud of here. Thanks”

“I have been receiving information from ORNL during the last year, and I am really very grateful . . . Thanks.”

“I have been receiving information from ORNL during the last year, and I am really very grateful . . . Thanks.” “excellent web site !”“excellent web site !”


CommunicationsCommunications• Publishing validation results

– Web sites, e.g.,

• http://modis.gsfc.nasa.gov/MODIS/LAND/VAL/• http://eosdatainfo.gsfc.nasa.gov/eosdata/ssinc/modland_dataprod.shtml

– Reports, e.g.,

• MODLAND “Special Issue”– Morisette, J. T., J.L. Privette, and C.O. Justice, A framework for the validation of MODIS land

products”, Remote Sensing of Environment, 83 (1-2) 77-96, 2002.

• Earth Observer report from the CEOS / WGCV Albedo workshop

– “Summary of the International Workshop on Albedo Product Validation” Nov./Dec. 2002, v.14, n.6, p.17-18 (available on-line at http://eospso.gsfc.nasa.gov/eos_observ/11_12_02/nov_dec_02.pdf

• EMDI Workshop, April 2002• SAFARI 2000 Meeting, October 2002• ‘Popular articles’ for the

general user community (Earth Observatory Web site)

• Collaborating with CEOS / WGCVJanuary 21, 2003January 21, 2003

September 24, 2001

September 19, 2001

• Increase the quality and economy of global satellite product validation

– develop and promote international standards and protocols for field sampling, scaling, error budgeting, data exchange, and product evaluation

• Advocate mission-long validation programs for current and future earth observing satellites

2002 activities– LAI Intercomparison “on-line”– Fire/Burned area regional activities– Albedo validation workshop– Web-posted “Case studies”

2002 activities– LAI Intercomparison “on-line”– Fire/Burned area regional activities– Albedo validation workshop– Web-posted “Case studies”

Using EOS Land Validation Core Sites as an example to establish the expanded ‘CEOS Land Validation Core sites’ through a joint CEOS WGISS / WGCV activity

Using EOS Land Validation Core Sites as an example to establish the expanded ‘CEOS Land Validation Core sites’ through a joint CEOS WGISS / WGCV activity


Validation . . . what worksValidation . . . what works• Growing buy-in to the ‘core sites’ concept

– where the field, airborne, satellite data suite is helping to address “scaling issues”

• Exploitation of in situ networks (e.g. flux towers, AERONET, LTER/ILTER) and field investigations (e.g., LBA, SAFARI 2000)

• International partnerships and coordination, mainly through CEOS/IGOS

• Simplified data access with dedicated archives and ‘easy to use’ data format (e.g., ASCII subsets)

• Frequent communication between Product PIs, Validation PIs, and data facilitators

• Best validation sites are those where investigators taking ground measurements are also interested in satellite products


Validation . . . what’s in the worksValidation . . . what’s in the works• Data for Stage 2 validation of MODLAND products

continue to be compiled and made available– the number of sites is growing through coordination

with science networks and CEOS members

• Improving formal documentation and making validation results more readily available to users

• Users can provide important feedback over and above formal ‘validation investigators’ (e.g., through DAAC surveys and user services, Fall 2003 Workshop)

• CEOS LPV topical meetings on Global Land Cover Validation (April 2003 and Fall 2003)

• FLUXNET Workshop, August 2003


Validation . . . on-going needs

Validation . . . on-going needs

• Promote stronger interface to the user community (e.g., through Validation Workshops)

• Develop stronger connection with the scientific community and NASA Roadmaps to establish how accurate the products need to be and how to relay products’ uncertainty so this information can be incorporated into how they are used

• Continue to promote collaboration between NASA’s Cal/Val program and the CEOS Working Group on Calibration and Validation


Comments . . . . Questions?Comments . . . . Questions?

Much of the information in this image prepared by the NASA Goddard Space Flight Center came from MODIS.

Date post:	04-Jan-2016
Category:	Documents
Upload:	anthony-stark
View:	34 times
Download:	7 times

Supporting the Validation of MODIS Land Products

Documents