FCM ValidationFCM Validation –– DocumentDocumentFCM Validation – Document

Overview MeetingOverview MeetingOverview Meeting

May 10, 2006May 10, 2006May 10, 2006

FCM Validation (PSA)FCM Validation (PSA)FCM Validation (PSA)

Document OverviewDocument OverviewDocument Overview

10

Model Validation (FCM)Model Validation (FCM) –– ObjectivesObjectives

General ObjectiveGeneral Objective –– To apply the Phase 2To apply the Phase 2 Calibration model inCalibration model in nonnon--recalibratedrecalibrated formform toto independentindependent data sets:data sets: ●● Temporal extrapoTemporal extrapolation (1979lation (1979--2004)2004)

●● Spatial extrapolation (downstream model)Spatial extrapolation (downstream model)

●● Interspecies extrapolation (nonInterspecies extrapolation (non--FCM species)FCM species)

Assess model performance relative toAssess model performance relative to performance measuresperformance measures

Housatonic River ProjectHousatonic River Project

11

Phase 2 Calibration (PSA)Phase 2 Calibration (PSA) –– Adjustments to FCMAdjustments to FCM

New EFDC Linkages (1990New EFDC Linkages (1990 –– 2000)2000)

Thickness of bioavailable/bioturbationThickness of bioavailable/bioturbation layers adjustedlayers adjusted

Minor revision to respiration rate model forMinor revision to respiration rate model for

white sucker (activity multiplier increased)white sucker (activity multiplier increased)

Linked model application onlyLinked model application only

No additionalNo additional ““calibrationcalibration”” of modelof model

parameters established in Phase 1parameters established in Phase 1

Housatonic River ProjectHousatonic River Project

12Housatonic River ProjectHousatonic River Project

FCM Representative FishFCM Representative Fish

Bullhead

Bass

SunfishCyprinid Minnows

Upstream Downstream

Predators

Forage Fish

Bottom Fish

White Sucker

13Housatonic River ProjectHousatonic River Project

ValidationValidation –– FCMFCM

Representative SpeciesRepresentative Species

1

10

100

1000

1 10 100 1000

Validation Data Set, Measured PCB Concentrations (mg/kg ww) Expressed as Averages for each Combination of FCM Species, Age, Reach, and Date Collected

Sim

ulat

ed F

ish

Tiss

ue P

CB

Con

cent

ratio

ns (m

g/kg

ww

) Overprediction by 5-fold Overprediction by 2.5-fold Perfect fit line Underprediction by 2.5-fold Underprediction by 5-fold Unique Age, Species, Reach, Date 1< n < 7 n > 6

(Over-prediction)

(Under-prediction)

14Housatonic River ProjectHousatonic River Project

ValidationValidation –– By Sample SizeBy Sample Size

(Lipid(Lipid--normalized tPCB)normalized tPCB)

100

1000

10000

100 1000 10000

Validation Data Set, Measured PCB Concentrations (mg/kg lipid) Expressed as Averages for each Combination of FCM Species, Age, Reach, and Date Collected

Sim

ulat

ed F

ish

Tiss

ue P

CB

Con

cent

ratio

ns (m

g/kg

lipi

d) Overprediction by 5-fold

Overprediction by 2.5-fold Perfect fit line Underprediction by 2.5-fold Underprediction by 5-fold Unique Age, Species, Reach, Date 1< n < 7 n > 6

15Housatonic River ProjectHousatonic River Project

ValidationValidation –– By SpeciesBy Species

1

10

100

1000

1 10 100 1000

Validation Data Set, Measured PCB Concentrations (mg/kg ww) Expressed as Averages for each Combination of FCM Species, Age, Reach, and Date Collected

Sim

ulat

ed F

ish

Tiss

ue P

CB

Con

cent

ratio

ns (m

g/kg

ww

) Overprediction by 5-fold Overprediction by 2.5-fold Perfect fit line Underprediction by 2.5-fold Underprediction by 5-fold Bass Bullhead Sunfish White Sucker

16Housatonic River ProjectHousatonic River Project

ValidationValidation –– By River ReachBy River Reach Si

mul

ated

Fis

h Ti

ssue

PC

B C

once

ntra

tions

(mg/

kg w

w)

1000

100

10

1 1 10 100 1000

Validation Data Set, Measured PCB Concentrations (mg/kg ww) Expressed as Averages for each Combination of FCM Species, Age, Reach, and Date Collected

Overprediction by 5-fold Overprediction by 2.5-fold Perfect fit line Underprediction by 2.5-fold Underprediction by 5-fold Reach 5A Reach 5B Reach 5C Woods Pond

Reach 5

Reach 6

17Housatonic River ProjectHousatonic River Project

Bas

s Ti

ssue

tPC

B (m

g/kg

ww

)

100

10

ValidationValidation –– Adult Largemouth BassAdult Largemouth Bass

(By Time Period)(By Time Period)

(Mature Fish Aged 6+ Or Greater Only) 10001000

Jan-

79Ja

n-80

Jan-

81Ja

n-82

Jan-

83Ja

n-84

Jan-

85Ja

n-86

Jan-

87Ja

n-88

Jan-

89Ja

n-90

Jan-

91Ja

n-92

Jan-

93Ja

n-94

Jan-

95Ja

n-96

Jan-

97Ja

n-98

Jan-

99Ja

n-00

Jan-

01Ja

n-02

Jan-

03Ja

n-04

Jan-

05

(Young Fish Aged 1+ to 5+ Years Only)

Bas

s Ti

ssue

tPC

B (m

g/kg

ww

)

100

10

Jan-

79Ja

n-80

Jan-

81Ja

n-82

Jan-

83Ja

n-84

Jan-

85Ja

n-86

Jan-

87Ja

n-88

Jan-

89Ja

n-90

Jan-

91Ja

n-92

Jan-

93Ja

n-94

Jan-

95Ja

n-96

Jan-

97Ja

n-98

Jan-

99Ja

n-00

Jan-

01Ja

n-02

Jan-

03Ja

n-04

Jan-

05

Sampling/Simulation Date Sampling/Simulation Date Measured -R each 6 (Age 2+ to 5+) Simulated -R each 6 (Age 4+) Measured -R each 5B (Age 6+ to 9+) Measured -R each 6 (Age 6+ to 9+) Measured -R each 5C (Age 2+) Simulated -R each 5C (Age 2+) Simulated - Reach 5B (Age 8+) Simulated -R each 6 (Age 8+) Measured -R each 5B (Age 5+) Simulated -R each 5B (Age 5+) Simulated - Reach 5A (Age 8+) Simulated - Reach 5C (Age 8+) Simulated - Reach 5A (Age 3+) Simulated -R each 5D (Age 3+) Simulated - Reach 5D (Age 8+)

18Housatonic River ProjectHousatonic River Project

Interspecies ExtrapolationsInterspecies Extrapolations

(Predator Fish)(Predator Fish)

Chain pickerel

Black crappie

Yellow perch

Rainbow trout

Brown trout

Largemouth bass

19Housatonic River ProjectHousatonic River Project

ValidationValidation –– InterspeciesInterspecies

Extrapolation (lipidExtrapolation (lipid--normalized tPCB)normalized tPCB)

100

1000

10000

100 1000 10000

Validation Data Set, Measured PCB Concentrations (mg/kg lipid) Expressed as Averages for each Combination of FCM Species, Age, Reach, and Date Collected

Sim

ulat

ed F

ish

Tiss

ue tP

CB

Con

cent

ratio

ns (m

g/kg

lipi

d)

Overprediction by 5-fold Overprediction by 2.5-fold Perfect fit line Underprediction by 2.5-fold Underprediction by 5-fold Yellow Perch Other Predator Trout Carp Goldfish

20Housatonic River ProjectHousatonic River Project

ValidationValidation –– Model PerformanceModel Performance

MeasuresMeasures

Data Grouping Samples MAPE MB* Type of Measure Accuracy Bias

All Samples 442 38% 0.95

FCM Representative Species 247 42% 0.91

All Other Species1 237 33% 0.99

Reach 5A 35 36% 0.95

Reach 5B 195 38% 0.89

Reach 5C 8 60% 0.68

Reach 6 246 37% 1.01

Bass 153 38% 1.04

Bullhead 4 74% 0.67

Carp1,2 8 58% 1.71

Goldfish1,2 42 44% 0.71

Other Predator1 3 39% 1.31

Sunfish 89 48% 0.74

Trout1,2 6 74% 1.35

White Sucker 1 9% 0.91

Yellow Perch1 178 28% 1.03

All YOY Samples 244 33% 0.91

All Adult Fish 240 42% 0.99

MB* > 1.0 = Overprediction

MB* < 1.0 = Underprediction 1 Species extrapolations required

2 Lipid-normalized results used

MFD/QAPP validation performance measure:

0.40 < MB* < 2.50

Observed range:

0.68 < MB* < 1.66

barcodetext: SDMS DocID 495579barcode: *495579*