1Computational Technologies
Project completion
Land Information Systems
PI: Paul Houser, GSFC (retired)CO-PI: Christa Peters-Lidard, GSFC
2Computational Technologies
Key Milestones• Design and install cluster system 8/02• Design policy for interoperability and community delivery 2/03• NOAH land surface model and Community Land Model
(CLM) operating at 5km resolution 3/03• Interoperability prototype tested with improved codes:
NOAH, CLM, and Variable Infiltration Capacity (VIC) land surface model 7/03
• Integrate database/visualization functions into Land Information System (LIS) framework 2/04
• Interoperability - Earth System Modeling Framework (ESMF) compliant version of CLM runs in the Global LIS 7/04
• Analyze predictive capability of the LIS. Users at several sites 8/04
Objective• Enable a major advance in the assimilation of
land data observations into land surface models to study and predict the regional and global water cycle
• Design and implement a Global Land Data Assimilation framework for ingesting Terabytes of observational data using cluster computing technology
Approach• Design a framework for integrating different land
surface models into an assimilation system• Construct a Land Data Assimilation cluster
computer capable of hosting terabytes of data• Optimize performance, throughput, and parallel
scaling to enable 1km resolution Co-I’sJames Geiger, Luther Lighty, Susan Olden, GSFC;Sujay Kumar, Yudong Tian, Goddard Earth Sciences
andTechnology (GEST) Center; Paul Dirmeyer, Centerfor Ocean-Land-Atmosphere Studies (COLA);Kenneth Mitchell, National Oceanic and
AtmosphericAdministration (NOAA); Eric Wood, Princeton
Land Information Systems
PI’s: Paul Houser & Christa Peters-Lidard,GSFC
http://esto.nasa.gov
NOAH (National Centers for Environmental Prediction, Oregon State University, United States Air Force, and Office of Hydrology)
Various data inputs: surface, snow, and precipitation
Results from regional land model of surface, root zone and entire profile of soil moisture
Effects from assimilation of soil moisture (top row) on the same three results
TRLin=4
3Computational Technologies
Accomplishments• The overarching objective of the LIS project was achieved; namely, to build a high-resolution, high-performance
land surface modeling and data assimilation system to support a wide range of land surface research activities and applications.
• The stated goals of LIS were accomplished:−LIS simulates the global land surface variables using various land surface models, driven by atmospheric “forcing data” (e.g., precipitation, radiation, wind speed, temperature, humidity) from various sources.−LIS performs high-performance, parallel computing for near real-time, high-resolution land surface modeling research and operations.−The high-resolution land surface simulation produces a huge data throughput, and LIS retrieves, stores, interpolates, sub-sets, and backs up the input and output data efficiently.−LIS provides intuitive web-based interfaces to users with varying levels of access to LIS data and system resources, and enforces user security policies.−LIS incorporates the ALMA (Assistance for Land surface Modeling Activities) and ESMF standards and the land surface modeling component was implemented on a custom-designed Linux cluster and an SGI Origin 3000.
Houser & Peters-Lidard, Land Information Systems
PI’s: Paul Houser & Christa Peters-Lidard,GSFC
Impact (see following chart) TRL=4in-6out
ESTO Monthly Review - October 31, 2005
Objective• Enable a major advance in the assimilation of
land data observations into land surface models to study and predict the regional and global water cycle
• Design and implement a Global Land Data Assimilation framework for ingesting Terabytes of observational data using cluster computing technology
At a September 6, 2005 ceremony, Goddard’s LIS received the prestigious recognition of NASA Software Invention of the Year. This award emphasizes software innovations that have a positive impact on NASA’s mission and other areas of science and technology.
4Computational Technologies
Houser & Peters-Lidard, Land Information Systems
TRL=4in-6out
Impact
•The LIS code was transferred to NOAA/National Centers for Environmental Prediction (NCEP) and the U.S. Air Force Weather Agency (AFWA), to improve the nation’s weather forecasting.
•In the mission support area, LIS is being used as a testbed for developing Level 4 products for the EOS-Aqua AMSR-E mission, as well as in an Observing System Simulation Experiment for the ESSP-4 proposal “Cold Land Processes Pathfinder”.
•In the Earth-Sun technology area, LIS is advancing modeling technologies in the ESTO/Advanced Information Systems Technology (AIST) project “Coupling High Resolution Earth System Models Using Advanced Computational Technologies” by coupling LIS with the Goddard Cumulus Ensemble (GCE) and Weather Research and Forecasting (WRF) atmospheric models via the ESMF framework.
•In the scientific research area, LIS is being used in several projects related to water and energy cycle modeling e.g., Goddard Modeling and Assimilation Office (GMAO), the Global Land Data Assimilation System (GLDAS); the World Meteorological Organization’s (WMO) Coordinated Enhanced Observing Period (CEOP), and the World Climate Research Programme’s (WCRP) Global Energy and Water Cycle Experiment (GEWEX) Global Soil Wetness Project-2.
•In the applications area, LIS is being used by projects in the areas of water resources, weather prediction, air quality, and homeland security with our partners at The Bureau of Reclamation, NOAA’s NCEP, the U.S. Environmental Protection Agency, the U.S. AFWA, and the U.S. Army Corps of Engineers/Engineering Research and Development Center.
•There are over 150 registered organizations from local and national government agencies, universities, commercial companies, and armed forces from 30 countries over the world.
6Computational Technologies
Accomplishments• The LIS cluster has 192 compute nodes and 8 IO
nodes. Supporting software, including system monitoring, system management, network traffic analysis, parallel computing environment, and performance testing suites, has been installed, configured and is running.
• Performance testing shows that the cluster is fully functional and ready for production.
The LIS Beowulf cluster, in Building 33 at GSFC, has 208 AMD XP processors 1.53 GHz and above, 112 GB of memory, 21 TB of disk space, 192 fast Ethernet and 10 gigabit Ethernet connections.
PI’s: Paul Houser & Christa Peters-Lidard,GSFCDescription
• Design and implement a Global Land Data Assimilation framework for ingesting Terabytes of observational data using cluster computing technology
Objective• Enable a major advance in the assimilation of
land data observations into land surface models to study and predict the regional and global water cycle
Key Milestones (12 milestones total)O1 - Design and install cluster system 8/02H - Design policy for interoperability and community delivery2/03F - NOAH and CLM operating at 5km resolution 3/03I - Interoperability prototype tested with improved codes
(NOAH, CLM, and VIC) 7/03G - Integrate database/viz functions into LIS framework 2/04J - Interoperability - ESMF compliant version of CLM runs
in the Global Land Information System 7/04K - Analyze predictive capability of the Land Information
System. Users at several sites 8/04
TRL=4in-4current
ESTO Monthly Review - November 7, 2002
Houser & Peters-Lidard, Land Information Systems
7Computational Technologies
Houser & Peters-Lidard, Land Information Systems
Key Milestones (12 milestones total)O1 - Design and install cluster system 8/02H - Design policy for interoperability and community delivery2/03F - NOAH and CLM operating at 5km resolution 3/03I - Interoperability prototype tested with improved codes
(NOAH, CLM, and VIC) 7/03G - Integrate database/viz functions into LIS framework 2/04J - Interoperability - ESMF compliant version of CLM runs
in the Global Land Information System 7/04K - Analyze predictive capability of the Land Information
System. Users at several sites 8/04
Description• Design and implement a Global Land Data
Assimilation framework for ingesting Terabytes of observational data using cluster computing technology
Objective• Enable a major advance in the assimilation of
land data observations into land surface models to study and predict the regional and global water cycle
Accomplishments• Developed documents for software design and
interoperability in cooperation with the LIS user community:- LIS Requirements Document- LIS Traceability Matrix- Software Design Document for the LIS- LIS Test Plan- User Interface Design Document for the LIS- Interface Design for Interoperability for the LIS- Software Design Document for the LIS: Data Management
PI’s: Paul Houser & Christa Peters-Lidard,GSFC
TRL=4in-4current
Interfaces for Interoperability in the Land Information System
ESTO Monthly Review - April 24, 2003
8Computational Technologies
Key Milestones (12 milestones total)O1 - Design and install cluster system 8/02H - Design policy for interoperability and community delivery2/03F - NOAH and CLM operating at 5km resolution 3/03I - Interoperability prototype tested with improved codes
(NOAH, CLM, and VIC) 7/03G - Integrate database/viz functions into LIS framework 2/04J - Interoperability - ESMF compliant version of CLM runs
in the Global Land Information System 7/04K - Analyze predictive capability of the Land Information
System. Users at several sites 8/04
Description• Design and implement a Global Land Data
Assimilation framework for ingesting Terabytes of observational data using cluster computing technology
Objective• Enable a major advance in the assimilation of
land data observations into land surface models to study and predict the regional and global water cycle
Accomplishments• Re-designed LDAS driver code to reduce size
of main data structures.• Parallelized the NOAH and CLM land surface
models.• Improved the NOAH and CLM codes within
the Global LDAS to operate at a 5km horizontal spatial resolution with a throughput of approximately 1 ms per grid cell per simulated day on the SGI O3K at ARC.
• Documented source code and scaling curves are on the LIS web site http://lis.gsfc.nasa.gov/
PI’s: Paul Houser & Christa Peters-Lidard,GSFC
TRL=4in-4current
Timing results on the SGI O3K 400MHz (lomax) at 5km
Houser & Peters-Lidard, Land Information Systems
ESTO Monthly Review - April 24, 2003
9Computational Technologies
Key Milestones (12 milestones total)O1 - Design and install cluster system 8/02H - Design policy for interoperability and community delivery2/03F - NOAH and CLM operating at 5km resolution 3/03I - Interoperability prototype tested with improved codes
(NOAH, CLM, and VIC) 7/03G - Integrate database/viz functions into LIS framework 2/04J - Interoperability - ESMF compliant version of CLM runs
in the Global Land Information System 7/04K - Analyze predictive capability of the Land Information
System. Users at several sites 8/04
Description• Design and implement a Global Land Data
Assimilation framework for ingesting Terabytes of observational data using cluster computing technology
Objective• Enable a major advance in the assimilation of
land data observations into land surface models to study and predict the regional and global water cycle
Accomplishments• Verified NOAH V2.5 and CLM V2.0 each run
on the LIS Cluster at 5 km resolution.• Verified VIC runs on LIS cluster and SGI 3000.• Verified ALMA mandatory outputs available
from CLM, NOAH, and VIC.• Developed/updated LIS documents: Software
Engineering Plan, Test Plan, User Interface Design Document, Interface Design for Interoperability, Data Management Document, Software Design Document, Requirements Document, Traceability Matrix, Test Report.
PI’s: Paul Houser & Christa Peters-Lidard,GSFC
TRL=4in-5current
Houser & Peters-Lidard, Land Information Systems
ESTO Monthly Review - October 30, 2003
Interfaces for Interoperability in the Land Information System
10Computational Technologies
Key Milestones (12 milestones total)O1 - Design and install cluster system 8/02H - Design policy for interoperability and community delivery2/03F - NOAH and CLM operating at 5km resolution 3/03I - Interoperability prototype tested with improved codes
(NOAH, CLM, and VIC) 7/03G - Integrate database/viz functions into LIS framework 2/04J - Interoperability - ESMF compliant version of CLM runs
in the Global Land Information System 7/04K - Analyze predictive capability of the Land Information
System. Users at several sites 8/04
Description• Design and implement a Global Land Data
Assimilation framework for ingesting Terabytes of observational data using cluster computing technology
Objective• Enable a major advance in the assimilation of
land data observations into land surface models to study and predict the regional and global water cycle
Accomplishments• Integrated the Land Information System (LIS)
with database and visualization functions. Made LIS more structured, flexible, efficient, interoperable, and portable.
• Improved NOAH, VIC and CLM codes within the Global LIS to operate at 1 km horizontal spatial resolution.
• Achieved throughput exceeding 0.1 ms per grid cell per day (greater than 4 times the Milestone-G goal) on the LIS cluster, better than one day/day.
• Provided code scaling curves and documented source code publicly available via the Web.
PI’s: Paul Houser & Christa Peters-Lidard,GSFC
TRL=4in-5current
Houser & Peters-Lidard, Land Information Systems
ESTO Monthly Review - October 6, 2004
Comparison of Land Surface Model (LSM) Performances
Tim
ing
(m
s/g
rid
/day) T
hro
ug
hp
ut (d
ays/d
ay)
Base Forcing
11Computational Technologies
Key Milestones (12 milestones total)O1 - Design and install cluster system 8/02H - Design policy for interoperability and community delivery2/03F - NOAH and CLM operating at 5km resolution 3/03I - Interoperability prototype tested with improved codes
(NOAH, CLM, and VIC) 7/03G - Integrate database/viz functions into LIS framework 2/04J - Interoperability - ESMF compliant version of CLM runs
in the Global Land Information System 7/04K - Analyze predictive capability of the Land Information
System. Users at several sites 8/04
Description• Design and implement a Global Land Data
Assimilation framework for ingesting Terabytes of observational data using cluster computing technology
Objective• Enable a major advance in the assimilation of
land data observations into land surface models to study and predict the regional and global water cycle
Accomplishments• Implemented LIS as a partially ESMF compliant
gridded component and evaluated the performance impact from adoption of ESMF code. Under worst-case test scenarios LIS code with ESMF 2.0 implementation ran slightly faster than the pre-ESMF 2.0 version.
• Provided or updated: - LIS Software Design Document - LIS Software Design Document - Data Management - LIS Interoperability Design Document - LIS User Interface Document - LIS Test Plan - LIS Verification Log - LIS-ESMF Prototype Public Release Home Page
PI’s: Paul Houser & Christa Peters-Lidard,GSFC
TRL=4in-6current
Houser & Peters-Lidard, Land Information Systems
ESTO Monthly Review - November 30, 2004
LIS software architecture and its components designed for the LIS cluster
12Computational Technologies
Key Milestones (12 milestones total)O1 - Design and install cluster system 8/02H - Design policy for interoperability and community delivery2/03F - NOAH and CLM operating at 5km resolution 3/03I - Interoperability prototype tested with improved codes
(NOAH, CLM, and VIC) 7/03G - Integrate database/viz functions into LIS framework 2/04J - Interoperability - ESMF compliant version of CLM runs
in the Global Land Information System 7/04K - Analyze predictive capability of the Land Information
System. Users at several sites 8/04
Description• Design and implement a Global Land Data
Assimilation framework for ingesting Terabytes of observational data using cluster computing technology
Objective• Enable a major advance in the assimilation of
land data observations into land surface models to study and predict the regional and global water cycle
Accomplishments• Demonstrated partially ESMF-compliant LIS from
milestone J – transferred to NOAA/NCEP as operational land data assimilation system for operational runs on their NCEP Central Computing Facility, an IBM-SP2.
• Analyzed predictive capability of the LIS using GrADS/DODS and web interfaces with users at NCEP, COLA, and Universities – rigorous tests run by LIS users
• Presented results to Review Board.• Documented source code made publicly available via
Web• Provided or updated: - Software Design Document - Interoperability Design Document - Software Design Document - Data Management - User Interface Document; Test Plan - Requirements Document; Maintenance Manual
TRL=4in-6current
Houser & Peters-Lidard, Land Information Systems
ESTO Monthly Review - October 31, 2005
Performance scaling tests on NCEP’s IBM platform. LIS has been ported to customer’s parallel computing platform and scales up to 16 processors.
PI’s: Paul Houser & Christa Peters-Lidard,GSFC