Data Management in Support of the Fish & Wildlife Program

Summary

Data Management in Support of the Fish & Wildlife Program

Summary

Presentation to the ISRP

July 16, 2002

Bruce Schmidt

Information is too valuable to be used only once!

The Basic Situation

• Information needs to be available from across multiple agencies and jurisdictions

• There are multiple sources of data, but little standardization of collection methods, data definitions, formatting or storage.

• There are multiple data needs• Relatively little has been invested in data

management

Database Support Is Vital to Regional Scale Management,

Planning, and Restoration• The region needs to get maximum value out

of the sampling work currently going on– Take existing data gathered for local use and

make them available to all regional entities for regional uses

– Make data from multiple independent sources consistent and compatible

Database Support Is Vital to Regional Scale Management,

Planning, and Restoration

• Add value to existing data– Georeferencing: provides ability to map data

and analyze on spatial basis, along with other related data

– Data analysis tools– Data access and query systems– Bring data together at convenient locations

Database Support Is Vital to Regional Scale Management,

Planning, and Restoration• Serve data to specific programs

– Regional Mark Information System– PIT Tag Information System– Fish Passage Center

• Serve specific kinds of data– PacFIN and RecFIN: Harvest data

• Serve multiple data types to many users– StreamNet – DART

Database Support Is Vital to Regional Scale Management,

Planning, and Restoration

• Database support will be needed for any new or revised data that are deemed high priority

Limitations and Challenges

• Current projects are limited to the data currently being collected

• Need clear regional guidance

• Many are unaware of available data related services

• Tendency to leave data management to last

• Tendency to react to new proposals rather than request specific help (react to funding proposals, seldom issue RFPs).

Review of Databases Funded through the

Columbia River Basin Fish and Wildlife Program

• Council Document ISRP 2000-3. Coutant, Goodman, Hanna, Huntly, Lettenmaier, McDonald, Riddell, Smoker, Whitney, Williams, and Urquhart

ISRP Report Conclusions:

• Suggested that the basic issue is the region not collecting the needed data

• Suggested that a distributed data system is best approach to data dissemination, which the data projects are working towards

• Recommend the Council provide strong directions to data sites (DART, parts of FPC and StreamNet) to ensure the continuation and improvement of analytical functions, maintenance of derived data, data validation, and service to data users.

Summary and Recommendations: Data Existence

• ISRP recommends formation of a joint working group of the existing data centers to develop recommendations for standardization of data collection protocols and to identify data deficiencies.

• ISRP recommends solicitation of proposals for a comprehensive monitoring program - draw heavily from existing efforts but go well beyond them...

Summary and Recommendations: Data Existence

• The basin states (Idaho, Montana, Oregon and Washington) and the tribes and federal agencies need to standardize their methods for collecting data.

Summary and Recommendations: Data Archiving

• ...function should be modularized so that there is one data archiving center responsible for each class of data.

• ...metadata be archived in a database structure that maintains association between primary data and their pertinent metadata.

• Adopt a policy requiring that reporting requirements for projects funded by the FWP include requirements for delivery of primary data, and their associated metadata

Summary and Recommendations: Data Access

• Data access function and calculation of derived data could be modularized...

• May be convenient but not absolutely necessary to archive derived data…

• User fees for some portion of funding

• Encourage competition

Summary and Recommendations:Overall

• No need to centralize the entire data storage and access system.

• Recommend continued support of the database programs - focus should move on to identifying and filling gaps in data and metadata.

• A distributed access system is most easily implemented given standardized methods and metadata - don’t have yet, biggest obstacle

• Database centers are doing reasonably well…remove some restrictions

Data Management in Support of the Fish & Wildlife Program Summary

• Outlined the services provided by existing data management projects

• Outlined data and data service needs

• Discussed a number of obstacles to regional information management that need to be addressed

• Projects under this program address some of these needs (as mentioned earlier), but more needs to be done.

The Regional Data Chain

• Key questions, data needs

• Data collection in the field

• Within-agency data management / use

• Regional data compilation

• Data dissemination regionally

• Regional use of data => key questions

Flow of Fish & Wildlife Agency Data From Field to Region

KeyQuestions,Data Needs

DataCollection

In Field

Within-AgencyData

Management

RegionalData

Compilation

DataDissemination

Regionally

Components: Priority questions

should drivesampling efforts

Agency missions /authority setdirection for datacollection

Components: Agency field crews Research project

crews Field monitoring

stations Remote telemetry Hatchery records Etc.

Components: Consolidation of field

data agency wide Written reports of data Some standardization of

data between offices Limited old mainframe

systems StreamNet assists

Components: Acquisition of data from

multiple agencies Standardization into

regional formats – assuredata are equivalent /comparable

Georeference data Example: StreamNet

Components: Internet delivery systems Make data queriable,

easy to locate Online mapping Custom products (CDs,

spreadsheets, maps, etc.) Data consistent region

wide

Impediments: Regional needs

differ from agencyneeds

No single agencyhas overallauthority

Difficult to get allagencies to agreeon priorityquestions / needs

Impediments: Is the priority data

being collected? Different purposes Different questions Different agency v.

regional priorities Data gaps Proper methods? Protocols? Consistency?

Impediments: Few state funded data

management systems inagencies (in some cases,StreamNet is all there is)

No data standards inagencies (in some cases)

Data not on Internet Time delay to report data

Impediments: Must wait for agency to

consolidate/finalize data Time to QC data Agencies generally do not

georeference fish data SN is a bridge from

management agencies toregional dissemination, butagency support of data isoften weak

Impediments: Spectrum of user

abilities Too easy to focus only

on new technology.That’s the easy part!

New technology mayassume agency data arealready on the web –false

Summary: Questions and dataneeds currently startat the agency, notthe regional level

Summary: DATA CONTENT isthe primary requisite ofthe data chain

Summary: Data management hasreceived low emphasis inthe fish and wildlifeagencies

Summary: StreamNet is the only datamanagement in some fishand wildlife agencies

Summary: This is the less expensiveand more glamorous endof the chain. Thetendency is to focus here,not on the real problems!

Two Important Components of Information Management

Data Delivery

Data Content

Most of the challenges listed in the Database Summary and in the ISRP

Report relate to data content

• To a large extent, content issues require resolution from beyond just the data management projects

• Data managers CAN provide valuable expertise in setting up sampling and delivery programs

• Data delivery will be a necessary part of any regional scope data effort

The database projects under this program work primarily toward

data delivery• Data delivery needs to be considered in any

regional information management system

• Data management expertise needs to be involved early in establishing sampling and information management programs

RPA 198

• RPA 198 calls for a common data management system for fish populations, water quality and habitat data

• Such an approach will serve more than just the needs identified in the Biological Opinion.

• Projects under this program should be integrated into a common system

• Projects under this program can provide expertise to help accomplish this RPA

These Projects Will Contribute to RPA 198

• Modular, distributed data projects should be incorporated as part of any common comprehensive system.

• Clear guidance is needed to assign roles and to avoid unnecessary duplication of effort

• Take advantage of existing expertise

RPA 198

• If construed as a computer database system to deliver data to the region, it will fail to address the DATA CONTENT issues raised in the Database Support Summary and the ISRP Report.

• If construed as a comprehensive approach toward managing data from collection to distribution, it can be a powerful force in support of resource management and restoration.

In Conclusion:

• Database support is critical to managing and distributing currently existing information for broader regional use

• Database support will be necessary for any new data and information needs

• A regional view is needed to establish information priorities

• Database expertise is critical for developing data collection and data delivery approaches

The data projects included under this program help to address many

of the issues and challenges:

• Organize data by type for convenient access• Provide similar data in one location• Acquire data from many sources, sometimes those

obscure to data users • Provide regional standardization to overcome disparate

formats, definitions and storage. (Data from non-standardized systems can not be combined or analyzed without being made consistent first.)

The data projects included under this program help to address many

of the issues and challenges:• Provide data services• Enhance utility of the information

– Georeference for mapping and spatial analysis– Tools– Consistent formats, regardless of data source