Date post: | 23-Dec-2015 |
Category: |
Documents |
Upload: | patience-price |
View: | 222 times |
Download: | 0 times |
1
Space-Time Datasets in Arc Hydro II
by Steve Grise (ESRI), David Maidment, Ernest To, Clark Siler
(CRWR)
2
CUAHSI Observations Data Model
Space-Time Datasets
Sensor and laboratory databases
From Robert Vertessy, CSIRO, Australia
3
Space-Time Dataset
• A set of records with – Time – Location– 1 or more variables
time
variables
x, y,
z
x, y,
z
x, y,
z
cba
cba
cba
x, y,
z
cba
x, y,
z
cba
x, y,
z
cba
4
Example: River Flow• For surface water resources, stream gages have a fixed location with
continuous measurements over time• Variables related to stream flow are the most common measurements• Data is typically measured regularly and continuously, but there are often
gaps due to device errors or routine maintenance• There are also cases of overflow or dry conditions where the values are
outside of the range of measurement for the device
time
variables
fixed
x, y,
z
cba
cba
cba
cba
cba
cba
cba
cba
cba
cba
cba
Data gap
cba
cba
An overflow condition could be recorded simply as > 500 cubic feet/second
stream flowriver heightmean velocity
5
Example: Water Quality• For water quality, sampling sites have a fixed location with intermittent measurements
over time– Four times per year is typical
• There is a sampling “event”, and a large number of chemical species are produced through laboratory analysis of water samples
• Data has metadata that specifies what laboratory procedure was used• Some data require a qualifier to be properly interpreted like “<“ to indicate a
measurement that is below a detection limit• Data are “Time stamped” with the time that the sampling event began. They are
considered “instantaneous data” observed at that time.
time
variables
fixed
x, y,
z
cba
cba
cba
cba turbidity
nitrateconductivityc
ba
t1 t2 t3 t4 t5
water quality sample
6
Display of data that vary in latitude, longitude, depth and time (Ernest To)
7
Data Structure for a single variable
These data are extracted from CUAHSIODM, and Offset = Depthin this instance
8
Example: Water Reservoir• For water reservoirs, data is recorded for the water level of the reservoir, along with
all inflows and outflows• A flow time series dataset describes the information required to do a water balance
on the reservoir contents• “Flow variables” apply over the entire time interval; “state variables” apply at instants
of time at beginning and end of interval;• Typically there are derived datasets
– Monthly data compiled from daily data– Annual data from monthly data
• Data are recorded regularly through time
time
variables
x, y,
z
cba
cba
cba
cba
cba
cba
cba
cba
x, y,
z
x, y,
z
x, y,
z
x, y,
z
x, y,
z
x, y,
z
x, y,
z
inflowoutflowstorage
Inflow
Outflow
Precip
Evap
Storage
9
Example: Water Rights Analysis• A water resources simulation model is run for monthly time steps for ~50 years and it
computes ~40 variables related to water supply reliability– Water rights diversion points, – Reservoirs, and – Other “control points” on the stream system
• Each model “run” generates millions of data values.• The “data cube” is completely filled in because it is all computed• Information products needed are graphs of variables at points, maps of feature
conditions at a single time point, and maps of averages through a defined time interval of feature conditions (i.e. dataset derived “on the fly”)
time
variables
x, y,
z
cba
cba
cba
cba
cba
cba
cba
cba
x, y,
z
x, y,
z
x, y,
z
x, y,
z
x, y,
z
x, y,
z
x, y,
z
% of time reliability% of volume reliabilityflow
Study area (watershed)
modeled point features
10
Maps and Charts
Plot a map for a time point Plot a graph for a space point
Space Time A set of variables ……
11
Example: Climate and Weather• Observations that come from weather balloons and other measuring
devices have dynamic location properties • For weather and climate forecast datasets, each data point represents an
area with consistent atmospheric characteristics• For weather observations, a large amount of data comes from fixed stations
so the datasets are similar to stream gage datasets
time
variables
x, y,
z
cba
cba
cba
cba
cba
cba
cba
cba
x, y,
z
x, y,
z
x, y,
z
x, y,
z
x, y,
z
x, y,
z
x, y,
z
temperatureair pressurerelative humidity
tttt
t
tt
t
balloon trajectory
forecast data
12
Example: Species Observations• In this type of dataset, observers are frequently moving
along a path such as a hiking trail or a boat cruise • Multiple species may be observed, and even the lack of
information is significant• Data is often recorded using offsets from the observer
location
time
variables
species group “a”species group “b”species group “c”
a a
c
a
c cb
aa
acc
c
b x, y,
z
x, y,
z
x, y,
z
x, y,
z
x, y,
z
x, y,
z
x, y,
z
x, y,
z
x, y,
z
x, y,
z
13
Other Datasets• There are many types of Time Series Datasets
– Observations– Samples– Model results– Remote sensing data/imaging
• Concepts are useful for many communities– Science– Business– Statistics– Planning– Health– Transportation
14
Space-Time Datasets:Implementation Concepts
• The general pattern can be described as– Time Series Values
• The data
– Time Series Descriptions • The metadata
• There are a number of ways to store and manage this information in a computer system
*
1
Time StepTime UnitIs RegularData TypeData OriginTime Reference SystemLocation Reference System
Time Series Description
1 *
VariableVariable UnitsVariable Unit Type
Variable Description
DateTimeLocationVariable[n]
Time Series Values
1
*
15
Example: Arc Hydro Version 1 Implementation
• Approach works well for an individual project with stream gage and other surface water data
• Constrained to 1 variable per time step• Limited in its ability to handle location
– Changes in x, y, z over time• i.e., Marine and species observation
datasets have an additional “cruise” or “observation” concepts linking multiple features
• FeatureID provided some flexibility, but did not directly support unique identity for features at different time steps
• In general, implementation patterns for the feature portion of the data model were not explored/explained
– Different spatial representations• Raster data• Multidimensional data
– GIS Layers and their properties were considered but not explained
– Inefficient approach with multiple variables
*
1
FeatureIDTSTypeIDTSDateTimeTSValue
Time Series TableTimeSeries
TSTypeIDVariableVarUnitsUnitTypeIsRegularTimeStepTimeUnitDataTypeOrigin
Time Series Type TableTSType
16
Arc Hydro Version 2Improvements
1. GIS Layer and representation focus
2. Use of Metadata
3. Improved Efficiency
4. More documented implementation patterns
5. General Time Series Dataset concepts applicable to many communities
17
Representations in GIS
• Time series data can be represented in different ways– Charts and graphs– Modeling simulations– Surfaces– Rasters– Vector feature classes
• GIS Layers provide a convenient set of representation types for different views into Time Series Datasets
18
Layers
• Layers represent data– Layer Properties
• Queries• Representation types• Display/styles• Variable(s)• Labels
• Layers deal with presentation of data, and they are closely linked to the data storage model
19
Vector Layers
Feature Series
A Feature Series is a collection of features indexed by time. Each feature in a feature series exists for only a period of time, making Feature Series an ideal structure for representing a series of flood inundation polygons. Feature Series can also be used to represent the movement of particles through the environment. In this case, the Feature Series would be a set of points, each valid for some instant in time.
FeatureIDFeatureGroupIDX, Y, ZTimeVariable
Multipatch Layer
FeatureIDFeatureGroupIDX, Y, ZTimeVariable
Polygon Layer
FeatureIDFeatureGroupIDX, Y, ZTimeVariable
Line Layer
FeatureIDGroupIDX, Y, ZTimeVariable
Point LayerTime LocationVariable[n]
Time Series Values
Derived*
20
Raster Layers
Raster Series
Stored* 1
Raster Series are collections of rasters indexed by time. Each raster is a "snapshot" of the environment at some instant in time. Grouping a series of rasters can describe how the environment changes over time. Raster Series are useful for describing the dynamics of spatially continuous phenomena, like ponded depth in the Everglades, or rainfall measured by NEXRAD.
Raster Catalog
RasterNameVariableTime
VariableXDimensionYDimensionOutputNameDimensionZvariableMVariable
Raster Layer
TimeLocationVariable[n]
Time Series Values
TimeLocationVariable[n]
Multi-Dimensional Dataset
Derived *
VariableTime
Raster Dataset
Stored*
1
21
Metadata• Each Time Series Dataset is a
complex structure, and there are many patterns
• Metadata is a tool that can be used to document datasets
– Facilitates search and discovery– Aids in sharing and re-use of data – Standards-based
metadata/cataloging methods are available
• In practice, once users understand the dataset, they tend to work with the Time Series Values and rarely re-visit the metadata in applications
• Shift in Arc Hydro II to use of FGDC/ISO metadata to document datasets and variables
– For the grey boxes in the diagram shown here
*
1
Time StepTime UnitIs RegularData TypeData OriginTime Reference SystemLocation Reference System
Time Series Description
1 *
VariableVariable UnitsVariable Unit Type
Variable Description
DateTimeLocationVariable[n]
Time Series Values
1
*
22
Improved Efficiency• In Arc Hydro 1, we tried to put all time series values into a single
table• This implied creating rows for each variable, or adding additional
columns/TSValues rows to datasets• Since it was table-based, it did not include feature and raster
representations, which required additional processing steps• By promoting multiple datasets with a flexible approach for
managing variables, data management activities will be improved, especially for larger datasets
*
1
FeatureIDTSTypeIDTSDateTimeTSValue
Time Series TableTimeSeries
TSTypeIDVariableVarUnitsUnitTypeIsRegularTimeStepTimeUnitDataTypeOrigin
Time Series Type TableTSType
PolygonPoint Line
RasterCatalog
RasterDataset
Table
Multi-Dimensional
Dataset
Multi-Patch
Single Time Series Table with 1 variable Time Series Datasets with multiple variables
23
Improved Efficiency
• For display, layers are built using Time Series Datasets
• Typically we “Select” or “Slice” 1 variable for presentation
• Layers can be built from source Values using InMemory layers, or built from Time Series Datasets
Time Series Layers with variable(s)
Time Series Datasets with variable(s)
VariableXDimensionYDimensionOutputNameDimensionZvariableMVariable
Raster Layer
FeatureIDFeatureGroupIDX, Y, ZTimeVariable
Multipatch Layer
FeatureIDFeatureGroupIDX, Y, ZTimeVariable
Polygon Layer
FeatureIDFeatureGroupIDX, Y, ZTimeVariable
Line Layer
FeatureIDGroupIDX, Y, ZTimeVariable
Point Layer
Time LocationVariable[n]
Time Series Datasets
*Time LocationVariable[n]
Time Series Values
Derived
24
Implementation Patterns
• Patterns will be explained for different types of implementations– Small/single project– Workgroup or multi-project environments– Very large datasets– Different spatial representation options– …– One key difference is that there will be multiple
datasets – basically one dataset per set of time series values
• Different dataset names and storage strategies• Documented with metadata
25
A General Spatial-Temporal Model
• A Space-Time Dataset is a set of records with – Time – Location– 1 or more variables
time
variables
x, y,
z
x, y,
z
x, y,
z
cba
cba
cba
x, y,
z
cba
x, y,
z
cba
x, y,
z
cba