Date post: | 17-Dec-2015 |
Category: |
Documents |
Upload: | marcia-summers |
View: | 217 times |
Download: | 1 times |
Hydrologic Data and Modeling: Towards Hydrologic Information Science
David R. MaidmentCenter for Research in Water Resources
University of Texas at Austin
EPSCorR, VermontNovember 10, 2008
Hydrologic Data and Modeling
• New knowledge in hydrology
• Hydrologic data
• Hydrologic modeling
• Hydrologic information systems
Hydrologic Data and Modeling
• New knowledge in hydrology
• Hydrologic data
• Hydrologic modeling
• Hydrologic information systems
How is new knowledge discovered?
• By deduction from existing knowledge
• By experiment in a laboratory
• By observation of the natural environment
After completing the Handbook of Hydrology in 1993, I asked myself the question: how is new knowledge discovered in hydrology?
I concluded:
Deduction – Isaac Newton
• Deduction is the classical path of mathematical physics– Given a set of axioms– Then by a logical process– Derive a new principle or
equation
• In hydrology, the St Venant equations for open channel flow and Richard’s equation for unsaturated flow in soils were derived in this way.
(1687)Three laws of motion and law of gravitation
http://en.wikipedia.org/wiki/Isaac_Newton
Experiment – Louis Pasteur
• Experiment is the classical path of laboratory science – a simplified view of the natural world is replicated under controlled conditions
• In hydrology, Darcy’s law for flow in a porous medium was found this way.
Pasteur showed that microorganisms cause disease & discovered vaccinationFoundations of scientific medicine http://en.wikipedia.org/wiki/Louis_Pasteur
Observation – Charles Darwin
• Observation – direct viewing and characterization of patterns and phenomena in the natural environment
• In hydrology, Horton discovered stream scaling laws by interpretation of stream maps
Published Nov 24, 1859Most accessible book of great
scientific imagination ever written
Conclusion for Hydrology
• Deduction and experiment are important, but hydrology is primarily an observational science
• discharge, water quality, groundwater, measurement data collected to support this.
Great Eras of Synthesis
• Scientific progress occurs continuously, but there are great eras of synthesis – many developments happening at once that fuse into knowledge and fundamentally change the science
1900
1960
1940
1920
1980
2000
Physics (relativity, structure of the atom, quantum mechanics)
Geology (observations of seafloor magnetism lead to plate tectonics)
Hydrology (synthesis of water observations leads to knowledge synthesis)
2020
Hydrologic Science
Hydrologic conditions(Fluxes, flows, concentrations)
Hydrologic Process Science(Equations, simulation models, prediction)
Hydrologic Information Science(Observations, data models, visualization
Hydrologic environment(Physical earth)
Physical laws and principles(Mass, momentum, energy, chemistry)
It is as important to represent hydrologic environments precisely with
data as it is to represent hydrologic processes with equations
Hydrologic Data and Modeling
• New knowledge in hydrology
• Hydrologic data
• Hydrologic modeling
• Hydrologic information systems
HIS Team and Collaborators
• University of Texas at Austin – David Maidment, Tim Whiteaker, Ernest To, Bryan Enslein, Kate Marney
• San Diego Supercomputer Center – Ilya Zaslavsky, David Valentine, Tom Whitenack
• Utah State University – David Tarboton, Jeff Horsburgh, Kim Schreuders, Justin Berger
• Drexel University – Michael Piasecki, Yoori Choi• University of South Carolina – Jon Goodall, Tony
Castronova• CUAHSI Program Office – Rick Hooper, David
Kirschtel, Conrad Matiuk• National Science Foundation Grant EAR-0413265
HIS Goals
• Data Access – providing better access to a large volume of high quality hydrologic data;
• Hydrologic Observatories – storing and synthesizing hydrologic data for a region;
• Hydrologic Science – providing a stronger hydrologic information infrastructure;
• Hydrologic Education – bringing more hydrologic data into the classroom.
HIS Overview Report
• Summarizes the conceptual framework, methodology, and application tools for HIS version 1.1
• Shows how to develop and publish a CUAHSI Water Data Service
• Available at:
http://his.cuahsi.org/documents/HISOverview.pdf
HTML as a Web Language
Text and Picturesin Web Browser
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" /><title>Vermont EPSCoR</title><link rel="stylesheet" href="epscor.css" type="text/css" media="all" /><!-- <script type='text/javascript' language='javascript‘ src='Presets.inc.php'>--></head>
HyperText Markup Language
WaterML as a Web LanguageDischarge of the San Marcos River at Luling, TX June 28 - July 18, 2002
Streamflow data in WaterML language
Services-Oriented Architecture for Water Data
• Links geographically distributed information servers through internet
• Web Services Description Language (WSDL from W3C)
• We designed WaterML as a web services language for water data
• Functions for computer to computer interaction
HIS Servers in the WATERS Network
HIS Central at San Diego Supercomputer Center
Web Services
CUAHSI Point Observation Data Services
1. Data Loading– Put data into the CUAHSI Observations Data
Model
2. Data Publishing– Provide web services access to the data
3. Data Indexing– Summarize the data in a centralized
cataloging system
CUAHSI Point Observation Data Services
1. Data Loading– Put data into the CUAHSI Observations Data
Model
2. Data Publishing– Provide web services access to the data
3. Data Indexing– Summarize the data in a centralized
cataloging system
Data Values – indexed by “What-where-when”
Space, S
Time, T
Variables, V
s
t
Vi
vi (s,t)
“Where”
“What”
“When”A data value
Observations Data Model
Horsburgh, J. S., D. G. Tarboton, D. R. Maidment and I. Zaslavsky, (2008), "A Relational Model for Environmental and Water Resources Data," Water Resour. Res., 44: W05406, doi:10.1029/2007WR006392.
• 11 WATERS Network test bed projects• 16 ODM instances (some test beds have more than one ODM
instance)• Data from 1246 sites, of these, 167 sites are operated by WATERS
investigators
National Hydrologic Information Server
San Diego Supercomputer Center
HIS Implementation in WATERS Network Information System
CUAHSI Point Observation Data Services
1. Data Loading– Put data into the CUAHSI Observations Data
Model
2. Data Publishing– Provide web services access to the data
3. Data Indexing– Summarize the data in a centralized
cataloging system
Point Observations Information Model
Data Source
Network
Sites
Variables
Values
{Value, Time, Metadata}
Utah State Univ
Little Bear River
Little Bear River at Mendon Rd
Dissolved Oxygen
9.78 mg/L, 1 October 2007, 5PM
• A data source operates an observation network• A network is a set of observation sites• A site is a point location where one or more variables are measured• A variable is a property describing the flow or quality of water• A value is an observation of a variable at a particular time• A metadata quantity provides additional information about the value
GetSites
GetSiteInfo
GetVariableInfo
GetValues
Assemble Data From Different Sources
Ingest data using ODM Data Loader
Load Newly Formatted Data into ODM Tables in MS SQL/Server
Wrap ODM with WaterML Web Services for Online Publication
Utah State University
University of Florida
Texas A&MCorpusChristi
Publishing an ODM Water Data Service
USU ODM
UFL ODM
TAMUCC ODM
Observations Data Model (ODM)
WaterML
SnotelDataValues
SnotelMETADATA
ODM
WaterML
Metadata From:ODM Database in
San Diego, CA
Snotel Web Site in Portland, OR
SnotelWater Data
Service
Publishing a Hybrid Water Data Service Snotel Metadata are
Transferred to the ODM
Web Services can both Query the ODM for Metadata and use a Web Scraper for Data Values
Calling the WSDL Returns Metadata and Data Values as if from the same Database
Get Values from:
http://river.sdsc.edu/snotel/cuahsi_1_0.asmx?WSDL
Locations
Variable Codes
Date Ranges
WaterML and WaterOneFlow
GetSiteInfoGetVariableInfoGetValues
WaterOneFlowWeb Service
Client
Penn State
Utah StateNWIS
DataRepositories
Data
DataData
EXTRACTTRANSFORMLOAD
WaterML
WaterML is an XML language for communicating water dataWaterOneFlow is a set of web services based on WaterML
CUAHSI Point Observation Data Services
1. Data Loading– Put data into the CUAHSI Observations Data
Model
2. Data Publishing– Provide web services access to the data
3. Data Indexing– Summarize the data in a centralized
cataloging system
Data Series – Metadata description
Space
Variable, Vi
Site, Sj
End Date Time, t2
Begin Date Time, t1
Time
Variables
Count, C
There are C measurements of Variable Vi at Site Sj from time t1 to time t2
Series Catalog
Space
Variable, Vi
Site, Sj
End Date Time, t2
Begin Date Time, t1
Time
Variables
Count, C
Vi
Sj
t2
t1
C
Texas Hydrologic Information System
Sponsored by the Texas Water Development Board and using CUAHSI technology for state and local data sources (using state funding)
CUAHSI National Water Metadata CatalogIndexes:• 50 observation networks• 1.75 million sites• 8.38 million time series• 342 million data values
NWIS
STORET
TCEQ
• Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them
Data Searching
NWIS
NARR
NAWQANAM-12
request
request
request
request
request
requestrequest
request
request
return
return
return
return
return
returnreturn
return
return
Searching each data source separately
Michael PiaseckiDrexel University
Semantic Mediation Searching all data
sources collectively
NWIS
NAWQA
NARR
generic
request
GetValues
GetValues
GetValues
GetValues
GetValues
GetValuesGetValues
GetValues
GetValues HODM
Michael PiaseckiDrexel University
Hydroseekhttp://www.hydroseek.org
Supports search by location and type of data across multiple observation networks including NWIS and Storet
Bora Beran, Drexel
HydroTaggerOntology: A hierarchy of concepts
Each Variable in your data is connected to a corresponding Concept
NWISNWIS
ArcGISArcGIS
ExcelExcel
AcademicAcademic
UnidataUnidata
NASANASAStoretStoret
NCDCNCDC
SnotelSnotel
MatlabMatlab
JavaJava
Visual BasicVisual Basic
Operational services
CUAHSI Web ServicesCUAHSI Web Services
Data SourcesData Sources
ApplicationsApplications
Extract
Transform
Load
http://www.cuahsi.org/his/
Direct analysis from your favorite analysis environment. e.g. Matlab% create NWIS Class and an instance of the class
createClassFromWsdl('http://water.sdsc.edu/wateroneflow/NWIS/DailyValues.asmx?WSDL');WS = NWISDailyValues;% GetValues to get the datasiteid='NWIS:02087500';bdate='2002-09-30T00:00:00';edate='2006-10-16T00:00:00';variable='NWIS:00060';valuesxml=GetValues(WS,siteid,variable,bdate,edate,'');
1920 1930 1940 1950 1960 1970 1980 1990 2000 20100
0.5
1
1.5
2
2.5x 10
4
cfs
Daily Discharge NEUSE RIVER NEAR CLAYTON, NC
National Water Metadata Catalog
Synthesis and communication of the nation’s water data http://his.cuahsi.org
Hydroseek WaterML
Government Water Data
Academic Water Data
Hydrologic Data and Modeling
• New knowledge in hydrology
• Hydrologic data
• Hydrologic modeling
• Hydrologic information systems
• Project sponsored by the European Commission to promote integration of water models within the Water Framework Directive
• Software standards for model linking• Uses model core as an “engine”• http://www.openMI.org
OpenMI – Links Data and Simulation Models
CUAHSI Observations Data Model as an OpenMI component
Simple River Model
Trigger (identifies what value should be calculated)
Typical model architectureApplication
User interface + engineEngine
Simulates a process – flow in a channelAccepts inputProvides output
ModelAn engine set up to represent a particular location e.g. a reach of the Thames
Engine
Output data
Input data
Model application
Run
Write
Write
Read
User interface
Accepts Provides
Rainfall
(mm)
Runoff
(m3/s)
Temperature
(Deg C)
Evaporation
(mm)
Accepts Provides
Upstream Inflow
(m3/s)
Outflow
(m3/s)
Lateral inflow
(m3/s)
Abstractions
(m3/s)
Discharges
(m3/s)
River Model
Linking modelled quantities
Rainfall Runoff Model
Data transfer at run time
Rainfall runoff
Output data
Input data
User interface
River
Output data
Input data
User interface
GetValues(..)
Models for the processes
River(InfoWorks RS)
Rainfall(database)
Sewer(Mouse)
RR(Sobek-Rainfall
-Runoff)
Data exchange3 Rainfall.GetValues
River(InfoWorks-RS)
Rainfall(database)
Sewer(Mouse)
2 RR.GetValues
7 RR.GetValues
RR(Sobek-Rainfall
-Runoff)
1 Trigger.GetValues
6 Sewer.GetValues
call
data
4
5 8
9
Hydrologic Data and Modeling
• New knowledge in hydrology
• Hydrologic data
• Hydrologic modeling
• Hydrologic information systems
Continuous Space-Time Data Model -- NetCDF
Space, L
Time, T
Variables, V
D
Coordinate dimensions
{X}
Variable dimensions{Y}
Geostatistics
Time Series Analysis
Multivariate analysis
Hydrologic Statistics
How do we understand space-time correlation fields of many variables?