Where does VIPER get its data from? Tom Pagano tom.pagano@por.usda.gov 503-414-3010.

transcript

Where does VIPER get its data from? Tom Pagano

tom.pagano@por.usda.gov 503-414-3010

What data do we need?

VIPER’s input and storage of data

Old and new databasesAWDBWYFORCDBSSNOTEL

Real-time data schedule

Local file import

What data do we need?

MetadataStation lists, lat/lon/elev

Historical time series dataReal-time time series data

Snowpack, Precipitation, etc

Forecast equations, configurations

VIPER Workbook

StationMetadata

HistoricalTime-Series

1 station POR

Real-timeData

All stations2 years

Equations/Configurations

VIPER Workbook

StationMetadata

1 station POR

Real-timeData

All stations2 years

Metadata

VIPER Workbook

StationMetadata

1 station POR

Real-timeData

All stations2 years

USGS webpage

Metadata

Old time series databases

Old databases and new databases

CDBS•“Final” data• Period of record ~1 year out of date•Daily, monthly•Snow, Precip, Temp Resv, Strm

SNOTEL•Provisional•Recent 2 years•Hourly (and daily)•All snotel elements•Archive kept (since ?)

WYFOR•Ephemeral•Recent 2 years•Monthly•Snow, Prec, Strm Resv, “Misc”•Archive kept (since ‘93)

AWDB•Provisional and final mixed•All years•All durations•All elements*

New time series database

*Planned

NOTE: AWDB load strategy being reworked so rest of slides may not apply in near future!

Example- where do historical monthly snow come from?

Back estimated snowcourse

Site automated (aug 1980)

Brumley Snotel

Example- where do historical monthly snow come from?

1x load of cdbs… updated if Dana notified of changes

1x load daily data in off season into monthly slots (snotel.. Or daily cdbs?)

SNOTEL

Historical streamflow data

Wyfor “under” CDBS

Historical streamflow data

1x load of old wyfor files since mid 1990’s

Then on top of that…

1x load of cdbs… updated if Dana notified of changes

The realtime data schedule and synchronization

AWDBdatabase

Wyfor SNOTEL

4x a day, complete load (1:20am, 5:20am, 11:20am, 10:20pm)

AWDBdatabase

Wyfor SNOTEL

Any DMP edits to snotel loadedevery 5 minutes to wyfor/awdb

DMPEdits

AWDBdatabase

Wyfor SNOTEL

Spreadsheet files loadedevery 10 minutes all day

to wyfor/awdb

CSV spreadsheet

files (non-snotel data)

CSVprocessor

AWDBdatabase

Wyfor SNOTEL

Direct edits of wyfor reach wyfor right awayBut reach AWDB 50 minutes after the hour

5-9am, 11am-4pm and 11pm. Worst method of entering data for viper.

Manual edits

If you have custom datasets, Viper can also access local CSV format files.

Same layout as individual station data sheets (e.g. data1, data2) Realtime data gets manually entered in “ExternalData” sheet.

OMG Viper is so slow!!!!

Viper is a network-hungry application.Within the USDA backbone, your connection should be very fast.Outside the backbone, speed varies from tolerable to disgruntling.

OMG Viper is so slow!!!!

Viper is a network-hungry application.Within the USDA backbone, your connection should be very fast.Outside the backbone, speed varies from tolerable to disgruntling.

The way viper connects to the database is the same way that users contact the new update report…. which can mean:

Where does VIPER get its data from? Tom Pagano tom.pagano@por.usda.gov 503-414-3010.

Documents