ASSESSING THE ACCURACY OF FISH-BASED

BIOLOGICAL INDICATORS OF WETLAND

CONDITION AT THE GREAT LAKES COASTAL

MARGINS

Jeff Buckley

University of Windsor

GLEI – GREAT LAKES ENVIRONMENTAL

INDICATORS PROJECT

FISH-BASED BIOLOGICAL INDICATORS

Biological Indices

Wetland Fish Index (Seilheimer & Chow-

Fraser, 2007)

Fish Condition Index (Bhagat & Ciborowski

2005)

HOW SHOULD BIOLOGICAL

INDICATORS BE EVALUATED?

Degradation

Bhagat et al., 2007

Bio

log

ica

l In

de

x S

co

re

Sensitivity: 25%

Specificity: 100%

ROC ANALYSIS

Sensitivity

Ability of an

indicator to identify

degraded sites.

Specificity

Ability of a indicator

to identify non-

degraded sites.

Degradation

Indic

ato

r M

odel S

core

Sensitivity: 85%

Specificity: 55%

Sensitivity: 100%

Specificity: 25%

ROC ANALYSIS: AREA UNDER THE CURVE

Area Under the

Curve (AUC):

Measure of indicator

accuracy

Between 0 and 1

.500 – No better

than chance

1.00 - Optimal

accuracy

RECEIVER-OPERATING CHARACTERISTIC (ROC)

CURVE ANALYSIS

Test the classification accuracy of a model with a

continuous output.

Does not require choosing a threshold for

degradation.

Does not assume a linear relationship between

degradation and community response.

Can show the optimum threshold for classification.

WETLAND FISH INDEX

Original Data (Seilheimer & Chow-Fraser 2007)

Current Data

2011 - 2013

n = 98 n = 60

FISH CONDITION INDEX

Original Data (Bhagat & Ciborowski 2005)

Current Data

2011 - 2013

n = 73 n = 40

STRESSOR MEASURES

- Agricultural Stress

- Development Stress

- Cumulative Stress

- 3500 sites

- Watershed level

land use-based stress

CUMULATIVE STRESS

n = 60

AGRICULTURAL STRESS

n = 60

DEVELOPMENT STRESS

n = 60

AUC COMPARISON

Index Agriculture Development Cumulative

Stress

Wetland Fish

Index 0.824 0.663 0.836

RDC Index 0.507 0.697 0.733

Species

Richness 0.559 0.550 0.600

Shannon

Diversity 0.641 0.520 0.592

OPTIMAL INDICATOR THRESHOLD -

AGRICULTURE

n = 60

OPTIMAL INDICATOR THRESHOLD –

CUMULATIVE STRESS

n = 60

OPTIMAL INDICATOR THRESHOLD –

WATER QUALITY INDEX

n = 60

CONCLUSIONS

Classification accuracy is important when testing indices.

ROC Curve Analysis is a useful method for testing

classification accuracy.

The Wetland Fish Index is a valid classifier of biological

condition in Great Lakes coastal margins.

ACKNOWLEDGMENTS

Supervisor: Dr. Jan Ciborowski

Field Crews: Jasmine, Justin, Curtis, Pat, Carla, Janis

Lab members: Li, Michelle, Jesse, Steffi, Lyndon, Drew.

Collaborators: Joseph Gathman, Katya Kovalenko, Lucinda Johnson, Valerie Brady, Terry Brown, Euan Reavie, Bob Howe, George Host, Gerald Niemi, Nick Danz

"Funding for this work was provided by the Great Lakes National Program

Office under the United States Environmental Protection Agency, grant

number GL-00E00612-0. Although the research described in this work

has been partly funded by the US EPA, it has not been subjected to the

agency's required peer and policy review and therefore does not

necessarily reflect the views of the agency and no official endorsement

should be inferred."

FISH COMMUNITIES RESPONDING TO

THRESHOLDS

r2 = 0.266

p < 0.001

n = 60

FISH COMMUNITIES RESPONDING TO

THRESHOLDS

r2 = 0.266

p < 0.001

n = 60

MODELS AS AN INDICATOR OF PHOSPHORUS

Disturbed: Total Phosphorus = 100 ug/L n = 340, 96, 44

MODELS AS INDICATORS OF AGRICULTURAL

LAND COVER

Disturbed: Agricultural Land Cover = 10% n = 273, 66, 34

MODELS AS INDICATORS OF POPULATION

Disturbed: Population density = 100 people/km2

n = 273, 66, 34

SPECIES RICHNESS AND SHANNON

DIVERSITY METRICS

n = 340 Disturbed: Total Phosphorus 100 ug/L

INDICATOR MODEL VS. ORIGINAL STRESS

GRADIENT

Disturbed: WQI = 0, n = 340 Disturbed: Rank = 8, n = 17, 18

Anthropogenic Disturbance

Environmental response

(TP, DO, ChlA, etc.)

Biotic response

(Richness, tolerant species, etc.)

Environmental Condition

Biological

Indicator Model

Lab Analysis of

Chemical Data

AUC COMPARISON

Indicator

Model

Original

Stress

Measure

Phosphoru

s

Agricultur

al Stress

Populati

on

Stress

Average

AUC

Wetland

Fish

Index

.694 .797 .682 .717 .732

Scirpus

IBI .861 .522 .541 .585 .549

Typha

IBI .870 .562 .642 .589 .598

ROC ANALYSIS

Sensitivity

Ability of a test to

identify degraded

sites.

Specificity

Ability of a test to

identify undegraded

sites.

WFI 1 2 3 4 5

FISH BASED BIOLOGICAL INDICATOR MODELS

Evaluates sites based

on metrics such as:

Species Richness

% tolerant species

% native species

Scores sites:

0 - Heavily degraded

72 - Non-degraded

Evaluates sites based on

presence of key indicator

species such as:

Gizzard shad

Longear sunfish

Score sites:

1 - Heavily degraded

5 - Non-degraded

Reference-degraded

Condition Index Wetland Fish Index

Bhagat & Ciborowski, 2005 Seilheimer & Chow-Fraser, 2007

INDICATORS OF CONDITION

Measure:

Dissolved Oxygen

Chlorophyll-a

Phosphorus

Sources of pollution

Problems:

Expensive

Time consuming

Require precise water

collection & lab analysis

Adapted from Kurtz et al., 2001

BIOLOGICAL INDICATORS

Use the biota as

indicators.

Easy, cheap, quick,

requires relatively little

expertise.

Directly measures what

ecologists, managers

and stakeholders are

actually interested in.

Adapted from Kurtz et al., 2001

RECEIVER OPERATING

CHARACTERISTIC CURVE (ROC)

ANALYSIS

Degradation

Bhagat et al., 2007

Bio

log

ica

l In

de

x S

co

re

WETLANDS AND CONDITION

Port Colborne, Ontario

Lake Erie

Killarney, Ontario

Georgian Bay

EVALUATING BIOLOGICAL INDICES

Degradation

Bhagat et al., 2007

Bio

log

ica

l In

de

x S

co

re

AGRICULTURE TEST - METRICS

DEVELOPMENT TEST - METRICS

CUMULATIVE STRESS TEST - METRICS

GLEI PROJECT AND GLCWMC

Great Lakes Environmental Indicators Project

Great Lakes Coastal Wetland Monitoring Consortium

Goal: “To develop an integrated set of indicators that can be used to assess the condition of coastal margins of all five Great Lakes”

RESEARCH QUESTIONS

Do indices hold-up

to their original

disturbance

gradient?

Are the indices

robust against other

measures of stress

Use ROC to

optimize indices.