40
ASSESSING THE ACCURACY OF FISH-BASED BIOLOGICAL INDICATORS OF WETLAND CONDITION AT THE GREAT LAKES COASTAL MARGINS Jeff Buckley University of Windsor
ASSESSING THE ACCURACY OF FISH-BASED
BIOLOGICAL INDICATORS OF WETLAND
CONDITION AT THE GREAT LAKES COASTAL
MARGINS
Jeff Buckley
University of Windsor
GLEI – GREAT LAKES ENVIRONMENTAL
INDICATORS PROJECT
FISH-BASED BIOLOGICAL INDICATORS
Biological Indices
Wetland Fish Index (Seilheimer & Chow-
Fraser, 2007)
Fish Condition Index (Bhagat & Ciborowski
2005)
HOW SHOULD BIOLOGICAL
INDICATORS BE EVALUATED?
Degradation
Bhagat et al., 2007
Bio
log
ica
l In
de
x S
co
re
Sensitivity: 25%
Specificity: 100%
ROC ANALYSIS
Sensitivity
Ability of an
indicator to identify
degraded sites.
Specificity
Ability of a indicator
to identify non-
degraded sites.
Degradation
Indic
ato
r M
odel S
core
Sensitivity: 85%
Specificity: 55%
Sensitivity: 100%
Specificity: 25%
ROC ANALYSIS: AREA UNDER THE CURVE
Area Under the
Curve (AUC):
Measure of indicator
accuracy
Between 0 and 1
.500 – No better
than chance
1.00 - Optimal
accuracy
RECEIVER-OPERATING CHARACTERISTIC (ROC)
CURVE ANALYSIS
Test the classification accuracy of a model with a
continuous output.
Does not require choosing a threshold for
degradation.
Does not assume a linear relationship between
degradation and community response.
Can show the optimum threshold for classification.
WETLAND FISH INDEX
Original Data (Seilheimer & Chow-Fraser 2007)
Current Data
2011 - 2013
n = 98 n = 60
FISH CONDITION INDEX
Original Data (Bhagat & Ciborowski 2005)
Current Data
2011 - 2013
n = 73 n = 40
STRESSOR MEASURES
- Agricultural Stress
- Development Stress
- Cumulative Stress
- 3500 sites
- Watershed level
land use-based stress
CUMULATIVE STRESS
n = 60
AGRICULTURAL STRESS
n = 60
DEVELOPMENT STRESS
n = 60
AUC COMPARISON
Index Agriculture Development Cumulative
Stress
Wetland Fish
Index 0.824 0.663 0.836
RDC Index 0.507 0.697 0.733
Species
Richness 0.559 0.550 0.600
Shannon
Diversity 0.641 0.520 0.592
OPTIMAL INDICATOR THRESHOLD -
AGRICULTURE
n = 60
OPTIMAL INDICATOR THRESHOLD –
CUMULATIVE STRESS
n = 60
OPTIMAL INDICATOR THRESHOLD –
WATER QUALITY INDEX
n = 60
CONCLUSIONS
Classification accuracy is important when testing indices.
ROC Curve Analysis is a useful method for testing
classification accuracy.
The Wetland Fish Index is a valid classifier of biological
condition in Great Lakes coastal margins.
ACKNOWLEDGMENTS
Supervisor: Dr. Jan Ciborowski
Field Crews: Jasmine, Justin, Curtis, Pat, Carla, Janis
Lab members: Li, Michelle, Jesse, Steffi, Lyndon, Drew.
Collaborators: Joseph Gathman, Katya Kovalenko, Lucinda Johnson, Valerie Brady, Terry Brown, Euan Reavie, Bob Howe, George Host, Gerald Niemi, Nick Danz
"Funding for this work was provided by the Great Lakes National Program
Office under the United States Environmental Protection Agency, grant
number GL-00E00612-0. Although the research described in this work
has been partly funded by the US EPA, it has not been subjected to the
agency's required peer and policy review and therefore does not
necessarily reflect the views of the agency and no official endorsement
should be inferred."
FISH COMMUNITIES RESPONDING TO
THRESHOLDS
r2 = 0.266
p < 0.001
n = 60
FISH COMMUNITIES RESPONDING TO
THRESHOLDS
r2 = 0.266
p < 0.001
n = 60
MODELS AS AN INDICATOR OF PHOSPHORUS
Disturbed: Total Phosphorus = 100 ug/L n = 340, 96, 44
MODELS AS INDICATORS OF AGRICULTURAL
LAND COVER
Disturbed: Agricultural Land Cover = 10% n = 273, 66, 34
MODELS AS INDICATORS OF POPULATION
Disturbed: Population density = 100 people/km2
n = 273, 66, 34
SPECIES RICHNESS AND SHANNON
DIVERSITY METRICS
n = 340 Disturbed: Total Phosphorus 100 ug/L
INDICATOR MODEL VS. ORIGINAL STRESS
GRADIENT
Disturbed: WQI = 0, n = 340 Disturbed: Rank = 8, n = 17, 18
Anthropogenic Disturbance
Environmental response
(TP, DO, ChlA, etc.)
Biotic response
(Richness, tolerant species, etc.)
Environmental Condition
Biological
Indicator Model
Lab Analysis of
Chemical Data
AUC COMPARISON
Indicator
Model
Original
Stress
Measure
Phosphoru
s
Agricultur
al Stress
Populati
on
Stress
Average
AUC
Wetland
Fish
Index
.694 .797 .682 .717 .732
Scirpus
IBI .861 .522 .541 .585 .549
Typha
IBI .870 .562 .642 .589 .598
ROC ANALYSIS
Sensitivity
Ability of a test to
identify degraded
sites.
Specificity
Ability of a test to
identify undegraded
sites.
WFI 1 2 3 4 5
FISH BASED BIOLOGICAL INDICATOR MODELS
Evaluates sites based
on metrics such as:
Species Richness
% tolerant species
% native species
Scores sites:
0 - Heavily degraded
72 - Non-degraded
Evaluates sites based on
presence of key indicator
species such as:
Gizzard shad
Longear sunfish
Score sites:
1 - Heavily degraded
5 - Non-degraded
Reference-degraded
Condition Index Wetland Fish Index
Bhagat & Ciborowski, 2005 Seilheimer & Chow-Fraser, 2007
INDICATORS OF CONDITION
Measure:
Dissolved Oxygen
Chlorophyll-a
Phosphorus
Sources of pollution
Problems:
Expensive
Time consuming
Require precise water
collection & lab analysis
Adapted from Kurtz et al., 2001
BIOLOGICAL INDICATORS
Use the biota as
indicators.
Easy, cheap, quick,
requires relatively little
expertise.
Directly measures what
ecologists, managers
and stakeholders are
actually interested in.
Adapted from Kurtz et al., 2001
RECEIVER OPERATING
CHARACTERISTIC CURVE (ROC)
ANALYSIS
Degradation
Bhagat et al., 2007
Bio
log
ica
l In
de
x S
co
re
WETLANDS AND CONDITION
Port Colborne, Ontario
Lake Erie
Killarney, Ontario
Georgian Bay
EVALUATING BIOLOGICAL INDICES
Degradation
Bhagat et al., 2007
Bio
log
ica
l In
de
x S
co
re
AGRICULTURE TEST - METRICS
DEVELOPMENT TEST - METRICS
CUMULATIVE STRESS TEST - METRICS
GLEI PROJECT AND GLCWMC
Great Lakes Environmental Indicators Project
Great Lakes Coastal Wetland Monitoring Consortium
Goal: “To develop an integrated set of indicators that can be used to assess the condition of coastal margins of all five Great Lakes”
RESEARCH QUESTIONS
Do indices hold-up
to their original
disturbance
gradient?
Are the indices
robust against other
measures of stress
Use ROC to
optimize indices.
