Date post: | 15-Aug-2015 |
Category: |
Documents |
Upload: | julie-fitzpatrick |
View: | 15 times |
Download: | 0 times |
Office of the Science AdvisorRisk Assessment Forum
EPA/100/R-14/004 July 2014 www.epa.gov/raf
U.S. Environmental Protection Agency Office of Research and Development Washington, DC 20460
Official Business Penalty for Private Use $300
PRESORTED STANDARDPOSTAGE & FEES PAID
EPAPERMIT NO. G-35
EPA/100/R‐14/004
RiskAssessmentForumWhitePaper:
ProbabilisticRiskAssessmentMethodsandCaseStudies
July25,2014
U.S.EnvironmentalProtectionAgencyOfficeoftheScienceAdvisorRiskAssessmentForum
ProbabilisticRiskAnalysisTechnicalPanelWashington,D.C.20460
ii
Disclaimer ThisdocumenthasbeenreviewedinaccordancewithU.S.EnvironmentalProtectionAgency(EPA)policyandapprovedforpublication.Mentionoftradenamesorcommercialproductsdoesnotconstituteendorsementorrecommendationforuse.
ThisdocumentwasproducedbyaTechnicalPaneloftheEPARiskAssessmentForum(RAF).Theauthorsdrewontheirexperienceindoingprobabilisticassessmentsandinterpretingthemtoimproveriskmanagementofenvironmentalandhealthhazards.Interviews,presentationsanddialogueswithriskmanagersconductedbytheTechnicalPanelhavecontributedtotheinsightsandrecommendationsinthiswhitepaperandtheassociateddocumenttitledProbabilisticRiskAssessmenttoInformDecisionMaking:FrequentlyAskedQuestions.
U.S.EnvironmentalProtectionAgency(USEPA).2014.RiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies.EPA/100/R‐09/001A.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.
iii
Foreword ThroughoutmanyoftheU.S.EnvironmentalProtectionAgency’s(EPA)programofficesandregions,variousformsofprobabilisticmethodshavebeenusedtoanswerquestionsaboutexposureandrisktohumans,otherorganismsandtheenvironment.Riskassessors,riskmanagersandothers,particularlywithinthescientificandresearchdivisions,haverecognizedthatmoresophisticatedstatisticalandmathematicalapproachescouldbeutilizedtoenhancethequalityandaccuracyofAgencyriskassessments.Variousstakeholders,insideandoutsideoftheAgency,havecalledforamorecomprehensivecharacterizationofrisks,includinguncertainties,toimprovetheprotectionofsensitiveorvulnerablepopulationsandlifestages.
TheEPAidentifiedtheneedtoexaminetheuseofprobabilisticapproachesinAgencyriskassessmentsanddecisions.TheRAFdevelopedthispaperandthecompaniondocument,ProbabilisticRiskAssessmenttoInformDecisionMaking:FrequentlyAskedQuestions,toprovideageneraloverviewofthevalueofprobabilisticanalysesandsimilarorrelatedmethods,aswellasprovideexamplesofcurrentapplicationsacrosstheAgency.Draftsofbothdocumentswerereleased,withslightlydifferenttitles,forpubliccommentandexternalpeerreviewinAugust2009.AnexternalpeerreviewwasheldinArlington,VirginiainMay2010.
Thegoalofthesepublicationsisnotonlytodescribepotentialandactualusesofthesetools,butalsotoencouragetheirfurtherimplementationinhuman,ecologicalandenvironmentalriskanalysisandrelateddecisionmaking.Theenhanceduseofprobabilisticanalysestocharacterizeuncertaintyinassessmentswillnotonlyberesponsivetoexternalscientificadvice(e.g.,recommendationsfromtheNationalResearchCouncil)onhowtofurtheradvanceriskassessmentscience,butalsowillhelptoaddressspecificchallengesfacedbymanagersandincreasetheconfidenceintheunderlyinganalysisusedtosupportAgencydecisions.
____________________________________________RobertKavlockInterimScienceAdvisorU.S.EnvironmentalProtectionAgency
iv
ProbabilisticRiskAnalysisTechnicalPanel
HalûkÖzkaynak,EPAOfficeofResearchandDevelopment(Co‐Chair)RobertHetes,EPAOfficeofResearchandDevelopment(Co‐Chair)KathrynGallagher,EPAOfficeofWater(Co‐Chair)ChrisFrey,NorthCarolinaStateUniversity(whileontemporaryassignmentatEPA)(WhitePaperCo‐Lead)JohnPaul,EPAOfficeofResearchandDevelopment(WhitePaperCo‐Lead)PaskyPascual,EPAOfficeofResearchandDevelopment(WhitePaperCo‐Lead)MarianOlsen,EPARegion2(CaseStudyLead)MikeClipper,EPAOfficeofSolidWasteandEmergencyResponseMichaelMessner,EPAOfficeofWaterKeeveNachman,currentlyatTheJohnsHopkinsBloombergSchoolofPublicHealthZacharyPekar,EPAOfficeofAirandRadiationRitaSchoeny,EPAOfficeofResearchandDevelopmentCynthiaStahl,EPARegion3DavidHrdy,EPAOfficeofPesticideProgramsJohnLangstaff,EPAOfficeofAirandRadiationElizabethMargosches,EPAOfficeofChemicalSafetyandPollutionPrevention(retired)AudreyGalizia,EPAOfficeofResearchandDevelopmentNancyRios‐Jafolla,EPARegion3DonnaRandall,EPAOfficeofChemicalSafetyandPollutionPreventionKhoanDinh,EPAOfficeofChemicalSafetyandPollutionPrevention(retired)HarveyRichmond,EPAOfficeofAirandRadiation(retired)
OtherContributingAuthors
JonathanChen,EPAOfficeofChemicalSafetyandPollutionPreventionLisaConner,EPAOfficeofAirandRadiationJanetBurke,EPAOfficeofResearchandDevelopmentAllisonHess,EPARegion2KellySherman,EPAOfficeofChemicalSafetyandPollutionPreventionWoodrowSetzer,EPAOfficeofResearchandDevelopmentValerieZartarian,EPAOfficeofResearchandDevelopment
EPARiskAssessmentForumScienceCoordinators
JulieFitzpatrick,EPAOfficeoftheScienceAdvisorGaryBangs,EPAOfficeoftheScienceAdvisor(retired)
ExternalPeerReviewersScottFerson(Chair),AppliedBiomathematicsAnnetteGuiseppi‐Elie,DuPontEngineeringDaleHattis,ClarkUniversityIgorLinkov,U.S.ArmyEngineerResearchandDevelopmentCenterJohnToll,WindwardEnvironmentalLLC
v
Table of Contents Foreword.................................................................................................................................................................iii
ListofFiguresandTables...............................................................................................................................viii
ListofAcronymsandAbbreviations..............................................................................................................ix
EXECUTIVESUMMARY.........................................................................................................................................1
1. INTRODUCTION:RELEVANCEOFUNCERTAINTYTODECISIONMAKING:HOWPROBABILISTICAPPROACHESCANHELP..............................................................................................4
1.1. EPADecisionMaking......................................................................................................................................4 1.2. TheRoleofProbabilisticRiskAnalysisinCharacterizingUncertaintyand
Variability............................................................................................................................................................4 1.3. GoalsandIntendedAudience......................................................................................................................5 1.4. OverviewofThisDocument.........................................................................................................................5 1.5. WhatAreCommonChallengesFacingEPARiskDecisionMakers?............................................5 1.6. WhatAreKeyUncertaintyandVariabilityQuestionsOftenAskedbyDecision
Makers?.................................................................................................................................................................6 1.7. WhyIstheImplementationofProbabilisticRiskAnalysisImportant?.....................................9 1.8. HowDoesEPATypicallyAddressScientificUncertaintyandVariability?............................10 1.9. WhatAretheLimitationsofRelyingonDefault‐BasedDeterministicApproaches?........11 1.10. WhatIsEPA’sExperiencewiththeUseofProbabilisticRiskAnalysis?.................................12
2. PROBABILISTICRISKANALYSIS..............................................................................................................15
2.1. WhatAreVariabilityandUncertainty,andHowAreTheyRelevanttoDecisionMaking?..............................................................................................................................................................15
2.1.1. Variability..................................................................................................................................15 2.1.2. Uncertainty...............................................................................................................................15
2.2. WhenIsProbabilisticRiskAnalysisApplicableorUseful?..........................................................17 2.3. HowCanProbabilisticRiskAnalysisBeIncorporatedIntoAssessments?...........................17 2.4. WhatAretheScientificCommunity’sViewsonProbabilisticRiskAnalysis,and
WhatIstheInstitutionalSupportforItsUseinPerformingAssessments?..........................18 2.5. AdditionalAdvantagesofUsingProbabilisticRiskAnalysisandHowItCan
ProvideMoreComprehensive,RigorousScientificInformationinSupportofRegulatoryDecisions...................................................................................................................................19
2.6. WhatAretheChallengestoImplementationofProbabilisticAnalyses?...............................19 2.7. HowCanProbabilisticRiskAnalysisSupportSpecificRegulatoryDecision
Making?..............................................................................................................................................................20 2.8. DoesProbabilisticRiskAnalysisRequireMoreResourcesThanDefault‐Based
DeterministicApproaches?.......................................................................................................................21 2.9. DoesProbabilisticRiskAnalysisRequireMoreDataThanConventional
Approaches?....................................................................................................................................................21 2.10. CanProbabilisticRiskAnalysisBeUsedtoScreenRisksorOnlyinComplexor
RefinedAssessments?..................................................................................................................................22 2.11. DoesProbabilisticRiskAnalysisPresentUniqueChallengestoModelEvaluation?.........23 2.12. HowDoYouCommunicatetheResultsofProbabilisticRiskAnalysis?.................................23 2.13. AretheResultsofProbabilisticRiskAnalysisDifficulttoCommunicatetoDecision
MakersandStakeholders?.........................................................................................................................24
vi
3. ANOVERVIEWOFSOMEOFTHETECHNIQUESUSEDINPROBABILISTICRISKANALYSIS.........................................................................................................................................................26
3.1. WhatIstheGeneralConceptualApproachinProbabilisticRiskAnalysis?..........................26 3.2. WhatLevelsandTypesofProbabilisticRiskAnalysesAreThereandHowAre
TheyUsed?.......................................................................................................................................................26 3.3. WhatAreSomeSpecificAspectsofandIssuesRelatedtoMethodologyfor
ProbabilisticRiskAnalysis?......................................................................................................................29 3.3.1. DevelopingaProbabilisticRiskAnalysisModel.......................................................29 3.3.2. DealingWithDependenciesAmongProbabilisticInputs......................................29 3.3.3. ConductingtheProbabilisticAnalysis...........................................................................30
4. SUMMARYANDRECOMMENDATIONS...................................................................................................33
4.1. ProbabilisticRiskAnalysisandRelatedAnalysesCanImprovetheDecision‐MakingProcessatEPA................................................................................................................................33
4.2. MajorChallengestoUsingProbabilisticRiskAnalysistoSupportDecisions......................34 4.3. RecommendationsforEnhancedUtilizationofProbabilisticRiskAnalysisatEPA..........34
GLOSSARY...............................................................................................................................................................37
REFERENCES..........................................................................................................................................................42
BIBLIOGRAPHY.....................................................................................................................................................48
ProbabilisticRiskAnalysisMethodologyGeneral....................................................................................48 ProbabilisticRiskAnalysisandDecisionMaking.........................................................................................49 ProbabilisticRiskAnalysisMethodologySpecificAspects...................................................................49 SensitivityAnalysis...................................................................................................................................................50 CaseStudyExamplesofProbabilisticRiskAnalysisEPA(SeeAlsothePRACase
StudiesAppendix).........................................................................................................................................51 CaseStudyExamplesofProbabilisticRiskAnalysisOther..................................................................52
APPENDIX:CASESTUDYEXAMPLESOFTHEAPPLICATIONOFPROBABILISTICRISKANALYSISINU.S.ENVIRONMENTALPROTECTIONAGENCYREGULATORYDECISIONMAKING............................................................................................................................................................54
A. OVERVIEW.......................................................................................................................................................55
B. INTRODUCTION.............................................................................................................................................55
C. OVERALLAPPROACHTOPROBABILISTICRISKANALYSISATTHEU.S.ENVIRONMENTALPROTECTIONAGENCY............................................................................................56
C.1. U.S.EnvironmentalProtectionAgencyGuidanceandPoliciesonProbabilisticRiskAnalysis............................................................................................................................................................56
C.2. CategorizingCaseStudies..........................................................................................................................56
Group1CaseStudies..........................................................................................................................57 Group2CaseStudies..........................................................................................................................57 Group3CaseStudies..........................................................................................................................58
vii
D. CASESTUDYSUMMARIES..........................................................................................................................63
D.1. Group1CaseStudies....................................................................................................................63
CaseStudy1:SensitivityAnalysisofKeyVariablesinProbabilisticAssessmentofChildren’sExposuretoArsenicinChromatedCopperArsenatePressure‐TreatedWood........................................................................................................................................63 CaseStudy2:AssessmentoftheRelativeContributionofAtmosphericDepositiontoWatershedContamination...................................................................................64
D.2. Group2CaseStudies....................................................................................................................65
CaseStudy3:ProbabilisticAssessmentofAnglingDurationUsedintheAssessmentofExposuretoHudsonRiverSedimentsviaConsumptionofContaminatedFish...............................................................................................................................65 CaseStudy4:ProbabilisticAnalysisofDietaryExposuretoPesticidesforUseinSettingToleranceLevels..............................................................................................................65 CaseStudy5:One‐DimensionalProbabilisticRiskAnalysisofExposuretoPolychlorinatedBiphenylsviaConsumptionofFishFromaContaminatedSedimentSite.........................................................................................................................................66 CaseStudy6:ProbabilisticSensitivityAnalysisofExpertElicitationofConcentration‐ResponseRelationshipBetweenFineParticulateMatterExposureandMortality.....................................................................................................................69 CaseStudy7:EnvironmentalMonitoringandAssessmentProgram:UsingProbabilisticSamplingtoEvaluatetheConditionoftheNation’sAquaticResources................................................................................................................................................70
D.3. Group3CaseStudies....................................................................................................................71
CaseStudy8:Two‐DimensionalProbabilisticRiskAnalysisofCryptosporidiuminPublicWaterSupplies,WithBayesianApproachestoUncertaintyAnalysis...........................................................................................................................71 CaseStudy9:Two‐DimensionalProbabilisticModelofChildren’sExposuretoArsenicinChromatedCopperArsenatePressure‐TreatedWood...................................72 CaseStudy10:Two‐DimensionalProbabilisticExposureAssessmentofOzone......74 CaseStudy11:AnalysisofMicroenvironmentalExposurestoFineParticulateMatterforaPopulationLivinginPhiladelphia,Pennsylvania..........................................76 CaseStudy12:ProbabilisticAnalysisinCumulativeRiskAssessmentofOrganophosphorusPesticides........................................................................................................77 CaseStudy13:ProbabilisticEcologicalEffectsRiskAssessmentModelsforEvaluatingPesticideUse...................................................................................................................78 CaseStudy14:ExpertElicitationofConcentration‐ResponseRelationshipBetweenFineParticulateMatterExposureandMortality.................................................80 CaseStudy15:ExpertElicitationofSea‐LevelRiseResultingFromGlobalClimateChange......................................................................................................................................81 CaseStudy16:KnowledgeElicitationforaBayesianBeliefNetworkModelofStreamEcology......................................................................................................................................82
CASESTUDYREFERENCES................................................................................................................................85
viii
List of Figures and Tables Figures
Figure1.GeneralPhasesoftheRiskAssessmentProcess.....................................................................................8Figure2.TieredApproachforRiskAssessment.......................................................................................................10Figure3.GraphicalDescriptionoftheLikelihood(Probability)ofRisk........................................................25Figure4.DiagrammaticComparisonofThreeAlternativeProbabilisticApproachesforthe
SameExposureAssessment...........................................................................................................................28
AppendixFigures
FigureA‐1.MonteCarloCancerSummaryBasedonaOne‐DimensionalProbabilisticRiskAnalysisofExposuretoPolychlorinatedBiphenyls........................................................................72
FigureA‐2.MonteCarloNoncancerHazardIndexSummaryBasedonaOne‐DimensionalProbabilisticRiskAnalysisofExposuretoPolychlorinatedBiphenyls...................................73
FigureA‐3.UncertaintyDistributionModelResults...............................................................................................80
Tables
Table1.SelectedExamplesofEPAApplicationsofProbabilisticRiskAssessmentTechniques.........14
AppendixTable
TableA‐1.CaseStudyExamplesofEPAApplicationsofDeterministicandProbabilisticRiskAssessmentTechniques..................................................................................................................................59
ix
List of Acronyms and Abbreviations 1‐DMCA One‐DimensionalMonteCarloAnalysis2‐DMCA Two‐DimensionalMonteCarloAnalysisAPEX AirPollutantsExposureModelBBN BayesianBeliefNetworkCAA CleanAirActCASAC CleanAirScientificAdvisoryCommitteeCCA ChromatedCopperArsenateCPSC ConsumerProductSafetyCommissionCRA CumulativeRiskAssessmentCSFII ContinuingSurveyofFoodIntakebyIndividualsCWA CleanWaterActDEEM DietaryExposureEvaluationModelDRA DeterministicRiskAssessmentDRES DietaryRiskEvaluationSystemERA EcologicalRiskAssessmentEMAP EnvironmentalMonitoringandAssessmentProgramEPA U.S.EnvironmentalProtectionAgencyFACA FederalAdvisoryCommitteeActFAQ FrequentlyAskedQuestionsFDA U.S.FoodandDrugAdministrationFFDCA FederalFood,Drug,andCosmeticActFIFRA FederalInsecticide,Fungicide,andRodenticideActFQPA FoodQualityProtectionActGAO GovernmentAccountabilityOfficeHHRA HumanHealthRiskAssessmentHI HazardIndexIPCC IntergovernmentalPanelonClimateChangeIRIS IntegratedRiskInformationSystemLHS LatinHypercubeSamplingLOAEL Lowest‐Observed‐Adverse‐EffectLevelLT Long‐TermLT2 Long‐Term2EnhancedSurfaceWaterTreatmentRuleMCA MonteCarloAnalysisMCS MonteCarloSimulationMEE MicroexposureEventAnalysisMOEs MarginsofExposureNAAQS NationalAmbientAirQualityStandardsNAS NationalAcademyofSciencesNERL NationalExposureResearchLaboratoryNOAEL No‐Observed‐Adverse‐EffectLevelNRC NationalResearchCouncilOAQPS OfficeofAirQualityPlanningandStandardsOAR OfficeofAirandRadiationOCSPP OfficeofChemicalSafetyandPollutionPreventionOERR OfficeofEmergencyandRemedialResponseOGWDW OfficeofGroundwaterandDrinkingWaterOMB OfficeofManagementandBudgetOP OrganophosphorusPesticide
x
OPP OfficeofPesticideProgramsORD OfficeofResearchandDevelopmentOSA OfficeoftheScienceAdvisorOSTP OfficeofScienceandTechnologyPolicyOSWER OfficeofSolidWasteandEmergencyResponseOW OfficeofWaterPAH PolycyclicAromaticHydrocarbonPCB PolychlorinatedBiphenylPCCRARM Presidential/CongressionalCommissiononRiskAssessmentandRiskManagementPDP PesticideDataProgramPM ParticulateMatterPRA ProbabilisticRiskAssessmentRAF RiskAssessmentForumRfC ReferenceConcentration(Inhalation)RfD ReferenceDose(Oral)RIA RegulatoryImpactAnalysisRME ReasonableMaximumExposureSAB ScienceAdvisoryBoardSAP ScientificAdvisoryPanelSHEDS StochasticHumanExposureandDoseSimulationModelSHEDS‐PM StochasticHumanExposureandDoseSimulationModelforParticulateMatterSTPC ScienceandTechnologyPolicyCouncilTRIM.Expo TotalRiskIntegratedMethodology/ExposureModelUI UncertaintyIntervalUSDA U.S.DepartmentofAgricultureUSGS U.S.GeologicalSurveyWHO WorldHealthOrganization
1
EXECUTIVE SUMMARY Probabilisticriskassessment(PRA),initssimplestform,isagroupoftechniquesthatincorporateuncertaintyandvariabilityintoriskassessments.Variabilityreferstotheinherentnaturalvariation,diversityandheterogeneityacrosstime,spaceorindividualswithinapopulationorlifestage,whileuncertaintyreferstoimperfectknowledgeoralackofpreciseknowledgeofthephysicalworld,eitherforspecificvaluesofinterestorinthedescriptionofthesystem(USEPA2011c).Variabilityanduncertaintyhavethepotentialtoresultinoverestimatesorunderestimatesofthepredictedrisk.
PRAprovidesestimatesoftherangeandlikelihoodofahazard,exposureorrisk,ratherthanasinglepointestimate.StakeholdersinsideandoutsideoftheAgencyhaverecommendedamorecompletecharacterizationofrisks,includinguncertaintiesandvariability,inprotectingmoresensitiveorvulnerablepopulationsandlifestages.PRAcanbeusedtosupportriskmanagementbyassessmentofimpactsofuncertaintiesoneachofthepotentialdecisionalternatives.
Numerousadvisorybodies,suchastheScienceAdvisoryBoard(SAB)andtheNationalResearchCouncil(NRC)oftheNationalAcademyofSciences(NAS),haverecommendedthatEPAincorporateprobabilisticanalysesintotheAgency’sdecision‐makingprocess.EPA’sRiskAssessmentForum(RAF)formedaTechnicalPanel,consistingofrepresentativesfromtheAgency’sprogramandregionaloffices,todevelopthiswhitepaperanditscompaniondocument,titledProbabilisticRiskAssessmenttoInformDecisionMaking:FrequentlyAskedQuestions(FAQ).TheRAFisrecommendingthedevelopmentofAgencyresources,suchasaclearinghouseofPRAcasestudies,bestpractices,resourcesandseminars,toraisegeneralknowledgeabouthowtheseprobabilistictoolscanbeused.
TheintendedgoalofthiswhitepaperistoexplainhowEPAcanuseprobabilisticmethodstoaddressdata,modelandscenariouncertaintyandvariabilitybycapitalizingonthewidearrayoftoolsandmethodsthatcomprisePRA.ThiswhitepaperdescribeswherePRAcanfacilitatemoreinformedriskmanagementdecisionmakingthroughbetterunderstandingofuncertaintyandvariabilityrelatedtoAgencydecisions.Theinformationcontainedinthisdocumentisintendedforbothriskanalystsandmanagersfacedwithdeterminingwhenandhowtoapplythesetoolstoinformtheirdecisions.Thisdocumentdoesnotprescribeaspecificapproachbut,rather,describesthevariousstagesandaspectsofanassessmentordecisionprocessinwhichprobabilisticassessmenttoolsmayaddvalue.
ProbabilisticRiskAssessment
PRAisananalyticalmethodologyusedtoincorporateinformationregardinguncertaintyand/orvariabilityintoanalysestoprovideinsightregardingthedegreeofcertaintyofariskestimateandhowtheriskestimatevariesamongdifferentmembersofanexposedpopulation,includingsensitivepopulationsorlifestages.Traditionalapproaches,suchasdeterministicanalyses,oftenreportrisksas“centraltendency,”“highend”(e.g.,90thpercentileorabove)or“maximumanticipatedexposure,”butPRAcanbeusedtodescribemorecompletelytheuncertaintysurroundingsuchestimatesandidentifythekeycontributorstovariabilityoruncertaintyinpredictedexposuresorriskestimates.Thisinformationthencanbeusedbydecisionmakerstoachieveascience‐basedlevelofsafety,tocomparetherisksrelatedtodifferentmanagementoptions,ortoinvestinresearchwiththegreatestimpactonriskestimateuncertainty.
Tosupportregulatorydecisionmaking,PRAcanprovideinformationtodecisionmakersonspecificquestionsrelatedtouncertaintyandvariability.Forexample,inthecontextofadecisionanalysisthathasbeenconducted,PRAcan:identify“tippingpoints”wherethedecisionwouldbedifferentif
2
theriskestimatesweredifferent;estimatethedegreeofconfidenceinaparticulardecision;andhelptoestimatetrade‐offsrelatedtodifferentrisksormanagementoptions.PRAcanprovideuseful(evencritical)informationabouttheuncertaintiesandvariabilityinthedata,models,scenario,expertjudgmentsandvaluesincorporatedinriskassessmentstosupportdecisionmakingacrosstheAgency.
PRAisapplicabletobothhumanhealthriskassessment(HHRA)andecologicalriskassessment(ERA);however,therearedifferencesbetweenhowPRAisusedforthetwo.BothHHRAandERAhaveasimilarstructureandusethesameriskassessmentsteps,butHHRAfocusesonindividuals,asinglespecies,morbidityandmortality,butERAismoreconcernedwithmultiplepopulationsoforganisms(e.g.,individualspeciesoffishinariver)orecologicalintegrity(e.g.,willthetypesofspecieslivingintheriverchangeovertime).InERA,therealsoisarelianceonindicatorsofimpacts(e.g.,sentinelspeciesandothermetrics).
RiskAssessmentatEPA
PRAbeganplayinganincreasinglyimportantroleinAgencyriskassessmentsfollowingthe1997releaseofEPA’sPolicyforUseofProbabilisticAnalysisinRiskAssessmentattheU.S.EnvironmentalProtectionAgency(USEPA1997a)andpublicationoftheGuidingPrinciplesforMonte‐CarloAnalysis(USEPA1997b).PRAwasamajorfocusinanassociatedreviewofEPAriskassessmentpracticesbytheSAB(USEPA2007b).TheNRCrecommendedthatEPAadopta“tiered”approachforselectingthelevelofdetailusedinuncertaintyandvariabilityassessment(NRC2009).Furthermore,theNRCrecommendedthatadiscussionaboutthelevelofdetailusedforuncertaintyanalysisandvariabilityassessmentshouldbeanexplicitpartoftheplanning,scopingandproblemformulationstepintheriskassessmentprocess.BoththiswhitepaperandthecompanionFAQdocumenttakeintoaccountrecommendationsonriskassessmentprocessesdescribedintheNRC’sreportScienceandDecisions:AdvancingRiskAssessment(NRC2009)andEnvironmentalDecisionsintheFaceofUncertainty(IOM2013).
EPA’srecentriskassessmentpublications,includingthedocumenttitledFrameworkforHumanHealthRiskAssessmenttoInformDecisionMaking(UAEPA2014b)aswellasthiswhitepaper,emphasizetheimportanceofcommunicatingtheresultsofaPRAbecauseitprovidestherangeandlikelihoodestimatesforoneormoreaspectsofhazard,exposureorrisk,ratherthanasinglepointestimate.Riskassessorsareresponsibleforsharinginformationonprobabilisticresultssothatdecisionmakershaveaclearunderstandingofquantitativeassessmentsofuncertaintyandvariability,andhowthisinformationwillaffectthedecision.EffectivecommunicationbetweentheriskassessoranddecisionmakeriskeytopromoteunderstandinganduseoftheresultsfromthePRA.
PRAgenerallyrequiresmoreresourcesthanstandardAgencydefault‐baseddeterministicapproaches.Appropriatelytrainedstaffandtheavailabilityofadequatetools,methodsandguidanceareessentialfortheapplicationofPRA.Properapplicationofprobabilisticmethodsrequiresnotonlysoftwareanddata,butalsoguidanceandtrainingforanalystsusingthetools,andformanagersanddecisionmakerstaskedwithinterpretingandcommunicatingtheresults.Inmostcircumstances,probabilisticassessmentsmaytakemoretimeandefforttoconductthanconventionalapproaches,primarilybecauseofthecomprehensiveinclusionofavailableinformationonmodelinputs.Thepotentiallyhigherresourcecostsmaybeoffset,however,byamoreinformeddecisionthanwouldbeprovidedbyacomparabledeterministicanalysis.
ContentoftheWhitePaperandFrequentlyAskedQuestionsCompanionDocuments
ThesetwodocumentsdescribehowPRAcanbeappliedtoenhancethescientificfoundationofEPA’sdecisionmakingacrosstheAgency.ThiswhitepaperdescribesthechallengesfacedbyEPA
3
decisionmakers,definesandexplainsthebasicprinciplesofprobabilisticanalysis,brieflyhighlightsinstanceswherethesetechniqueshavebeenimplementedinEPAdecisions,anddescribescriteriathatmaybeusefulindeterminingwhetherandhowtheapplicationofprobabilisticmethodsmaybeusefuland/orapplicabletodecisionmaking.Thiswhitepaperalsodescribescommonlyemployedmethodstoaddressuncertaintyandvariability,includingthoseusedintheconsiderationofuncertaintyinscenariosanduncertaintyinmodels.Additionally,itaddressesuncertaintyandvariabilityintheinputsandoutputsofmodelsandtheimpactoftheseuncertaintiesoneachofthepotentialmanagementoptions.Ageneraldescriptionoftherangeofmethodsfromsimpletocomplex,rapidtomoretimeconsumingandleasttomostresourceintensiveisprovided,aswellasusesofthesemethods.
Bothdocumentsaddressissuessuchasuncertaintyandvariability,theirrelevancetodecisionmakingandthePRAgoaltoprovidequantitativecharacterizationoftheuncertaintyandvariabilityinestimatesofhazard,exposure,orrisk.ThedifferencebetweenthewhitepaperandtheFAQsdocumentisthelevelofdetailprovidedaboutPRAconceptsandpractices,andtheintendedaudience(e.g.,riskassessorsvs.decisionmakers).DetailedexamplesofapplicationsofthesemethodsareprovidedintheAppendixofthiswhitepaper,whichistitled“CaseStudyExamplesoftheApplicationofProbabilisticRiskAnalysisinU.S.EnvironmentalProtectionAgencyDecisionMaking.”ThewhitepaperAppendixincludes16casestudies—11HHRAand5ERAexamples—thatillustratehowEPA’sprogramandregionalofficeshaveusedprobabilistictechniquesinriskassessment.Toaidindescribinghowthesetoolswereapplied,the16casestudiesaresubdividedamong3categoriesforpurposesofthisdocument.Group1includes2casestudiesdemonstratingpointestimate,includingsensitivityanalysis;Group2iscomprisedof5casestudiesdemonstratingprobabilisticriskanalysis,includingone‐dimensionalMonteCarloanalysisandprobabilisticsensitivityanalysis;andGroup3includes9casestudiesdemonstratingadvancedprobabilisticriskanalysis,includingtwo‐dimensionalMonteCarloanalysiswithmicroexposure(microenvironments)modeling,Bayesianstatistics,geostatisticsandexpertelicitation.
TheFAQdocumentprovidesanswerstocommonquestionsregardingPRA,includingkeyconceptssuchasscientificandinstitutionalmotivationsfortheuseofPRA,andchallengesintheapplicationofprobabilistictechniques.TheprincipalreasonforincludingPRAasanoptionintheriskassessor’stoolboxisitsabilitytosupporttherefinementandimprovementoftheinformationleadingtodecisionmakingbyincorporatingknownuncertainties.
4
1. INTRODUCTION: RELEVANCE OF UNCERTAINTY TO DECISION MAKING: HOW PROBABILISTIC APPROACHES CAN HELP
1.1. EPA Decision Making TodiscusswhereprobabilisticapproachescanaidEPA’sdecisionmaking,itisimportanttogenerallydescribetheAgency’scurrentdecision‐makingprocessesandhowbetterunderstandingandimprovingelementswithintheseprocessescanclarifywhereprobabilisticapproachesmightprovidebenefits.TheenhanceduseofPRAandcharacterizationofuncertaintywouldallowEPAdecisionmakersopportunitiestouseamorerobustandtransparentprocess,whichmayallowgreaterresponsivenesstooutsidecommentsandrecommendations.SuchanapproachwouldsupporthigherqualityEPAassessmentsandimproveconfidenceinAgencydecisions.
Therearetwomajorareasinthedecision‐makingprocessthatmightbeimprovedwithPRA.Scientistscurrentlyaregenerallyfocusedonthefirstarea—theunderstandingofdata,modelandscenariouncertaintiesandvariability.Thesecondareaisonethathasnot,untilrecentlyandonlyinalimitedfashion,beenusedbyEPAdecisionmakers.Thisareaisformaldecisionanalysis.Withdecisionanalytictechniques,decisionmakerscanweightherelativeimportanceofriskinformationcomparedtootherinformationinmakingthedecision,understandhowuncertaintyaffectstherelativeattractivenessofpotentialdecisionalternatives,andassessoverallconfidenceinadecision.Inadditiontodata,modelandscenariouncertainty,thereisaseparatecategoryofuncertaintiesspecificallyassociatedwithhowthedecisioncriteriarelatetothedecisionalternatives.Althoughitisquiterelevanttoriskmanagementdecisions,thetopicanddecisionanalysisingeneralareoutsideofthescopeofthisreport.Thiswhitepaperfocusesontechnicalinformationthatwouldallowbetterunderstandingoftherelationshipsamongalternativedecisionsinassessingrisks.
1.2. The Role of Probabilistic Risk Analysis in Characterizing Uncertainty and Variability
Probabilisticanalysesincludetechniquesthatcanbeappliedformallytoaddressbothuncertaintyandvariability,typicallyarisingfromlimitationsofdata,modelsoradequatelyformulatingthescenariosusedinassessingrisks.Probabilityisusedinscience,business,economicsandotherfieldstoexamineexistingdataandestimatethechanceofanevent,fromhealtheffectstoraintomentalfatigue.Onecanuseprobability(chance)toquantifythefrequencyofoccurrenceorthedegreeofbeliefininformation.Forvariability,probabilitydistributionsareinterpretedasrepresentingtherelativefrequencyofagivenstateofthesystem(e.g.,thatthedataaredistributedinacertainway);foruncertainty,theyrepresentthedegreeofbelieforconfidencethatagivenstateofthesystemexists(e.g.,thatwehavetheappropriatedata;CullenandFrey1999).PRAoftenisdefinednarrowlytoindicateastatisticalorthoughtprocessusedtoanalyzeandevaluatethevariabilityofavailabledataortolookatuncertaintyacrossdatasets.
Forthepurposesofthisdocument,PRAisatermusedtodescribeaprocessthatemploysprobabilitytoincorporatevariabilityindatasetsand/ortheuncertaintyininformation(suchasdataormodels)intoanalysesthatsupportenvironmentalrisk‐baseddecisionmaking.PRAisusedherebroadlytoincludebothquantitativeandqualitativemethodsfordealingwithscenario,modelandinputuncertainty.Probabilistictechniquescanbeusedwithothertypesofanalysis,suchasbenefit‐costanalysis,regulatoryimpactanalysisandengineeringperformancestandards;thus,theycanbeusedforavarietyofapplicationsandbyexpertsinmanydisciplines.
5
1.3. Goals and Intended Audience TheprimarygoalsofthiswhitepaperaretointroducePRA,describehowitcanbeusedtobetterinformandimprovethedecision‐makingprocess,andprovidecasestudieswhereithasbeenusedinhumanhealthandecologicalanalysesatEPA(seetheAppendixforthecasestudies).AsecondarygoalofthispaperistobridgecommunicationgapsregardingPRAamonganalystsofvariousdisciplines,betweentheseanalystsandAgencydecisionmakers,andamongaffectedstakeholders.Thiswhitepaperalsoisintendedtoserveasacommunicationtooltointroducekeyconceptsandbackgroundinformationonapproachestoriskanalysisthatincorporateuncertaintyandprovideamorecomprehensivetreatmentofvariability.Riskanalysts,decisionmakersandaffectedstakeholderscanbenefitfromunderstandingthepotentialusesofPRA.PRAandrelatedapproachescanbeusedtoidentifyadditionalresearchthatmayreduceuncertaintyandmorethoroughlycharacterizevariabilityinariskassessment.ThiswhitepaperexplainshowPRAcanenhancethedecision‐makingprocessesfacedbymanagersatEPAbybettercharacterizingdata,model,scenarioanddecisionuncertainties.
1.4. Overview of This Document ThiswhitepaperprovidesanoverviewofEPA’sinterestandexperienceinaddressinguncertaintyandvariabilityusingprobabilisticmethodsinriskassessment;identifieskeyquestionsaskedorfacedbyAgencydecisionmakers;demonstrateshowconventionaldeterministicapproachestoriskanalysismaynotanswerthesequestionsfully;providesexamplesofapplications;andshowshowandwhy“probabilisticriskanalysis”(broadlydefined)couldprovideaddedvalue,comparedtotraditionalmethods,withregardtoregulatorydecisionmakingbymorefullycharacterizingriskestimatesandexploringdecisionuncertainties.Forthepurposesofthiswhitepaper,PRAandrelatedtoolsforbothhumanhealthandecologicalassessmentsincludearangeofapproaches,fromstatisticaltools,suchassensitivityanalysis,tomulti‐dimensionalMonteCarlomodels,geospatialapproachesandexpertelicitation.KeypointsaddressedbythisdocumentincludedefinitionsandkeyconceptspertainingtoPRA,benefitsandchallengesofPRA,ageneralconceptualframeworkforPRA,conclusionsregardingproductsandinsightsobtainedfromPRA,andexampleswhereEPAhasusedPRAinhumanhealthandecologicalanalyses.AGlossaryandaBibliographyalsoareprovided.
1.5. What Are Common Challenges Facing EPA Risk Decision Makers?
EPAoperatesunderstatutoryandregulatoryconstraintsthatoftenlimitthetypesofcriteriathatcanbeconsidered(includingwhethertheuseofPRAisappropriate)andimposestricttimeframesinwhichdecisionsmustbemade.Typically,thedecisionbeginswithunderstanding(1)whoorwhatwillbeprotected;(2)therelationshipbetweenthedataanddecisionalternatives;and(3)theimpactofdata,modelanddecisionuncertaintiesrelatedtoeachdecisionalternative.Theseareamongtheconsiderationsoftheplanningandscopingandproblemformulationphasesofriskassessment(USEPA2014).EPAdecisionmakersneedtoconsidermultipledecisioncriteria,whichareinformedbyvaryingdegreesofconfidenceintheunderlyinginformation.Decisionmakersneedtobalancetheregulatory/statutoryrequirementsandtimeframes,resources(i.e.,expertise,costsoftheanalysis,reviewtimes,etc.)toconducttheassessment,managementoptions,andstakeholderswhileatthesametimekeepingriskassessmentanddecisionmakingseparate.
Uncertaintycanbeintroducedintoanyassessmentatanystepintheprocess,evenwhenusinghighlyaccuratedatawiththemostsophisticatedmodels. Uncertaintycanbereducedorbettercharacterizedthroughknowledge.Variabilityornaturalheterogeneityisinherentinnaturalsystemsandthereforecannotbereduced,butcanbeexaminedanddescribed.Uncertaintyindecisionsisunavoidablebecausereal‐worldsituationscannotbeperfectlymeasured,modeledor
6
predicted.Asaresult,EPAdecisionmakersfacescientificallycomplexproblemsthatarecompoundedbyvaryinglevelsofuncertaintyandvariability.Ifuncertaintyandvariabilityhavenotbeenwellcharacterizedoracknowledged,potentialcomplicationsariseintheprocessofdecisionmaking.Increaseduncertaintycanmakeitmoredifficulttodetermine,withreasonableconfidence,thebalancepointbetweenthecostsofregulationandtheimplicationsforavoidingdamagesandproducingbenefits.Characterizationfacilitatedbyprobabilisticanalysescanprovideinsightintoweighingtherelativecostsandbenefitsofvaryinglevelsofregulationandalsocanassistinriskcommunicationactivities.
Decisionmakersoftenwanttoknowwhoisatriskandbyhowmuch,thetradeoffsbetweenalternativeactionsandthelikelyorpossibleconsequencesofdecisions.Tothisend,itisparticularlyusefulfordecisionmakerstounderstandthedistributionofriskacrosspotentiallyimpactedpopulationsandecologicalsystems.Itcanbeimportanttoknowthenumberofindividualsexperiencingdifferentmagnitudesofrisk,thedifferencesinriskmagnitudeexperiencedbyindividualsindifferentlifestagesorpopulationsortheprobabilityofaneventthatmayleadtounacceptablelevelsofrisk.Giventhelimitationsofdata,traditionalmethodsofriskanalysesarenotwellsuitedtoproducesuchestimates.Probabilisticanalyticalmethodsarecapableofaddressingtheseshortcomingsandcancontributetoamorethoroughrecognitionoftheimpactofdatagapsontheprojectedriskestimates.AlthoughPRAcanbeusedtocharacterizetheuncertaintyandvariabilityinsituationswithlimiteddata,currentlythereisnotextensiveexperienceusingPRAtocharacterizetherangeofeffectsordose‐responserelationshipsforpopulations,includingsensitivepopulationsandlifestages.
OtherchallengesfacingEPAdecisionmakersincludetheneedtoconsidermultipledecisioncriteria,whichareinformedbyvaryingdegreesofconfidenceintheunderlyinginformation,understandingtherelationshipbetweenandamongthosedecisioncriteria(includingmulti‐pollutantandmulti‐mediaeffects)andthedecisionalternatives,andthetimelinessofthedecisionmaking.Furthermore,evenwhenPRAisused,EPAdecisionmakersmustbemindfulofpotentialmisusesandobfuscationswhenconductingorpresentingPRAresults.DecisionmakersalsoneedtoconsidertheevolvingsciencebehindPRA.AstheuseofPRAincreasesdecisionmakerswillbecomemorefamiliarwiththetechniquesandtheirapplication.
Ariskassessmentprocessneedstoconsideruncertainties,variabilityandtherationaleorfactorsinfluencinghowtheymaybeaddressedbyadecisionmaker.Decisionmakersneedafoundationforestimatingthevalueofcollectingadditionalinformationtoallowforbetterinformeddecisions.Therearecostsassociatedwithignoringuncertainty(McConnell1997andToll1999),andafocusbydecisionmakersontheinformationprovidedbyuncertaintyanalysiscanstrengthentheirchoices.
1.6. What Are Key Uncertainty and Variability Questions Often Asked by Decision Makers?
Asdescribedabove,determiningthedecision‐makingcontextandspecificconcernsisacriticalfirststeptowarddevelopingausefulandresponsiveriskassessmentthatwillsupportthedecision.Forexample,theappropriatefocusandlevelofdetailoftheanalysisshouldbecommensuratewiththeneedsofthedecisionmakerandstakeholders,aswellastheappropriateuseofscience.Analysesoftenareconductedatalevelofdetaildictatedbytheissuebeingaddressed,thebreadthandqualityoftheavailableinformationuponwhichtobaseananalysis,andthesignificancesurroundingadecision.Theanalyticalprocesstendstobeiterative.Althoughaguidingsetofquestionsmayframetheinitialanalyses,additionalquestionscanarisethatfurtherdirectorevenreframetheanalyses.
7
BasedonaseriesofdiscussionswithAgencydecisionmakersandriskassessors,sometypicalquestionsaboutuncertaintyandvariabilityrelevanttoriskanalysesincluding:
Factorsinfluencingdecisionuncertainty:
Wouldmydecisionbedifferentifthedataweredifferent,improvedorexpanded?Wouldadditionaldatacollectionandresearchlikelyleadtoadifferentdecision?Howlongwillittaketocollecttheinformation,howmuchwoulditcost,andwouldtheresultingdecisionbesignificantlyaltered?
Whataretheliabilitiesandconsequencesofmakingadecisionunderthecurrentlevelofknowledgeanduncertainty?
Howdothealternativesandtheirassociateduncertaintyandvariabilityaffectthetargetpopulationorlifestage?
Considerationsforevaluatingdataormethoduncertainty:
Howrepresentativeorconservativeistheestimateduetodataormethoduncertainty(alsoincorporatingvariability)?
Whatarethemajorgapsinknowledge,andwhatarethemajorassumptionsusedintheassessment?Howreasonablearetheassumptions?
Issuesarisingwhenaddressingvariability:
Canaprobabilisticapproach(e.g.,tobettercharacterizeuncertaintiesandvariability)beaccomplishedinatimelymanner?
Whatisthedesiredpercentileofthepopulationtobeprotected?Bychoosingthispercentile,whomaynotbeprotected?
Thequestionsthatariseconcerninguncertaintyandvariabilitychangedependingonthestageandnatureofthedecision‐makingprocessandanalysis.GeneralphasesoftheriskassessmentprocessareillustratedinFigure1.Forfurtherinformationontheprocessofdecisionmaking,wesuggestreferringtothedescriptionprovidedbyEPARegion3ontheMulti‐CriteriaIntegratedResource
8
Figure 1. General Phases of the Risk Assessment Process. Risk assessment is an iterative process comprised of planning, scoping and problem formulation; analysis (e.g., hazard identification, dose‐response assessment and exposure assessment); interpretation and risk characterization; and risk communication. The highlighted boxes explain how PRA fits into the overall process.
AssessmentInternetpageathttp://www.epa.gov/reg3esd1/data/mira.htm.TheutilityofvariouslevelsofanalysisandsophisticationinansweringthesequestionsisillustratedinthecasestudiesdescribedinSection1.10andpresentedintheAppendixofthiswhitepaper.ReferencestoexamplesbeyondtheseEPAcasestudiescanbefoundintheBibliography.Additionally,Lesteretal.(2007)identifiedmorethan20PRAapplicationcasestudies(includingEPAexamples)performedsince2000;thesecasestudyexamplesarecategorizedassite‐specificapplicationsandregionalriskassessments.
9
1.7. Why Is the Implementation of Probabilistic Risk Analysis Important?
TheprincipalreasonfortheinclusionofPRAasanoptionintheriskassessor’stoolboxisPRA’sabilitytosupportrefinementandimprovementoftheinformationleadingtodecisionmakingbyincorporatingknownuncertainties.Beginningasearlyasthe1980s,expertscientificadvisorygroups,suchastheNationalResearchCouncil(NRC),recommendedthatriskanalysesincludeacleardiscussionoftheuncertaintiesinriskestimation(NRC1983).TheNRCstatedtheneedtodescribeuncertaintyandtocapturevariabilityinriskestimates(NRC1994).ThePresidential/CongressionalCommissiononRiskAssessmentandRiskManagement(PCCRARM)recommendedagainstarequirementorneedfora“brightline”orsingle‐numberlevelofrisk(PCCRARM1997).SeeSection2.4formoreinformationregardingthescientificcommunity’sopinionontheuseofPRA.
Regulatoryscienceoftenrequiresselectionofalimitforacontaminant,yetthatlimitalwayscontainsuncertaintyastohowprotectiveitis.PRAandrelatedtoolsquantitativelydescribetheveryrealvariationsinnaturalsystemsandlivingorganisms,howtheyrespondtostressors,andtheuncertaintyinestimatingthoseresponses.
RiskcharacterizationbecameEPApolicyin1995(USEPA1995b),andtheprinciplesoftransparency,clarity,consistencyandreasonablenessareexplicatedinthe2000RiskCharacterizationHandbook(USEPA2000a).Transparency,clarity,consistencyandreasonablenesscriteriarequiredecisionmakerstodescribeandexplaintheuncertainties,variabilityandknowndatagapsintheriskanalysisandhowtheyaffecttheresultingdecision‐makingprocesses(USEPA1992,1995a,2000a).
TheuseofprobabilisticmethodsalsohasreceivedsupportfromsomedecisionmakerswithintheAgency,andthesemethodshavebeenincorporatedintoanumberofEPAdecisionstodate.Programoffices,suchastheOfficeofPesticidePrograms(OPP),OfficeofSolidWasteandEmergencyResponse(OSWER),OfficeofAirandRadiation(OAR),andOfficeofWater(OW),aswellastheOfficeofResearchandDevelopment(ORD),haveutilizedprobabilisticapproachesindifferentwaysandtovaryingextents,forbothhumanexposureandecologicalriskanalyses.Inaddition,OSWERhasprovidedexplicitguidanceontheuseofprobabilisticapproachesforexposureanalysis(USEPA2001).SomeprogramofficeshaveheldtrainingsessionsonMonteCarlosimulation(MCS)softwarethatisusedfrequentlyinprobabilisticanalyses.
TheNRCrecommendedthatEPAshouldadoptatieredapproachforselectingthelevelofdetailusedinuncertaintyandvariabilityassessment(NRC2009).Furthermore,NRCrecommendedthatadiscussionaboutthelevelofdetailusedforuncertaintyanalysisandvariabilityassessmentshouldbeanexplicitpartoftheplanning,scopingandproblemformulationstepintheriskassessmentprocess.ThewaythatPRAfitsintoagraduatedhierarchical(tiered)approachismorefullydescribedinSection2.10andillustratedinFigure2.
Whenitisbeneficialtorefineriskestimates,theuseofPRAcanhelpinthecharacterizationandcommunicationofuncertainty,variabilityandtheimpactofdatagapsinriskanalysesforassessors,decisionmakersandstakeholders(includingthetargetpopulationorlifestage).
10
Figure 2. Tiered Approach for Risk Assessment. The applicability of a probabilistic approach depends on the needs of decision makers and stakeholders. Assessments that are high in complexity and regulatory significance benefit from the application of probabilistic techniques. Source: Adapted from USEPA 2004a and WHO 2008.
1.8. How Does EPA Typically Address Scientific Uncertainty and Variability?
Environmentalassessmentscanbecomplex,suchascoveringexposuretomultiplechemicalsinmultiplemediaforawide‐rangingpopulation.TheAgencyhasdevelopedsimplifiedapproachestocharacterizerisksassociatedwithsuchcomplexassessmentsthroughtheuseofpointestimatesformodelvariablesorparameters.Suchanapproachtypicallyproducespointestimatesofrisks(e.g.,10‐5oralifetimeprobabilityofcancerriskofoneindividualin100,000).Theseoftenarecalled“deterministic”assessments.Asaresultoftheuseofpointestimatesforvariablesinmodelalgorithms,deterministicriskresultsusuallyarereportedaswhatareassumedtobeeitheraverageorworst‐caseestimates.Theydonotcontainanyquantitativeestimateoftheuncertaintyinthatestimate,norreportwhatpercentileoftheexposedpopulationtheestimateapplies.ThemethodstypicallyusedinEPAriskassessmentsrelyonacombinationofpointvalueswithpotentiallyvaryinglevelsofconservatismandcertainty,yieldingapointestimateofexposureatsomepointintherangeofpossiblerisks.
Becauseuncertaintyisinherentinallriskassessments,itisimportantthattheriskassessmentprocessenablehandlinguncertaintiesinalogicalwaythatistransparentandscientificallydefensible,consistentwiththeAgency’sstatutorymissionandresponsivetotheneedsofdecisionmakers(NRC1994).Uncertaintyisafactorinbothecologicalandhumanhealthriskassessments.
11
Forhumanhealthriskassessments,uncertaintiesariseforbothnoncancerandcancerendpoints.Thus,whendataaremissing,EPAoftenusesseveraloptionstoprovideboundariesonuncertaintyandvariabilityinanattempttoavoidriskunderestimation;attemptingtogiveasinglequantificationofhowmuchconfidencethereisintheriskestimatemaynotbeinformativeorfeasible.
Inexposureassessment,forexample,thepracticeatEPAistocollectnewdatawheretheyareneededandwheretimeandresourcesallow.Alternativeapproachestoaddressuncertaintyincludenarrowingthescopeoftheassessment;usingscreening‐leveldefaultassumptionsthatincludeupper‐endvaluesand/orcentraltendencyvaluesthataregenerallycombinedtogenerateriskestimatesthatfallwithinthehigherendofthepopulationriskrange(USEPA2004b);applyingmodelstoestimatemissingvalues;usingsurrogatedata(e.g.,dataonaparameterthatcomefromadifferentregionofthecountrythantheregionbeingassessed);orapplyingprofessionaljudgment.Theuseofindividualassumptionscanrangefromqualitative(e.g.,assumingthatoneissecuredtotheresidencelocationanddoesnotmovethroughtimeorspace)tomorequantitative(e.g.,usingthe95thpercentileofasampledistributionforaningestionrate).Thisapproachalsocanbeappliedtothepracticeofhazardidentificationanddose‐responseassessmentwhendataaremissing.Identifyingthesensitivityofexposureorriskestimatestokeyinputscanhelpfocuseffortstoreduceuncertaintybycollectingadditionaldata.
CurrentEPApracticestoaddressuncertaintyandvariabilityarefocusedontheevaluationofdata,model,andscenariouncertaintyandvariability.Inaddition,decisionmakersarefacedwithcombiningmanydifferentdecisioncriteriathatmaybeinformedbyscienceandPRAaswellasbyexpertjudgmentortheweightingofvaluestochooseadecisionalternative.Data,model,andscenariouncertaintiesandvariability(includingtheirprobabilitydistributions),aswellasexpertjudgment,canbeimportantconsiderationsintheselectionofonealternativeoveranother(Costanzaetal.1997;Morganetal.2009;StahlandCimorelli2005;Wrightetal.2002).
1.9. What Are the Limitations of Relying on Default-Based Deterministic Approaches?
Default‐baseddeterministicapproachesareappliedtodata,modelandscenariouncertainties.Deterministicriskassessment(DRA)oftenisconsideredatraditionalapproachtoriskanalysisbecauseoftheexistenceofestablishedguidanceandproceduresregardingitsuse,theeasewithwhichitcanbeperformed,anditslimiteddataandresourceneeds.TheuseofdefaultssupportingDRAprovidesaproceduralconsistencythatallowsforriskassessmentstobefeasibleandtractable.DecisionmakersandmembersofthepublictendtoberelativelyfamiliarwithDRA,andtheuseofsuchanapproachaddressesassessment‐relateduncertaintiesprimarilythroughtheincorporationofpredetermineddefaultvaluesandconservativeassumptions.Itaddressesvariabilitybycombininginputparametersintendedtoberepresentativeoftypicalorhigherendexposure(i.e.,consideredtobeconservativeassumptions).Theintentionoftenistoimplicitlyprovideamarginofsafety(i.e.,morelikelytooverestimateriskthanunderestimaterisk)orconstructascreening‐levelestimateofhigh‐endexposureandrisk(i.e.,anestimaterepresentativeofmorehighlyexposedandsusceptibleindividuals).
DRAprovidesanestimationoftheexposuresandresultingrisksthataddressesuncertaintiesandvariabilitiesinaqualitativemanner.ThemethodstypicallyusedinEPADRArelyonacombinationofpointvaluessomeconservativeandsometypicalyieldingapointestimateofexposurethatisatsomeunknownpointintherangeofpossiblerisks.AlthoughthisconservativebiasalignswiththepublichealthmissionofEPA(USEPA2004b),thedegreeofconservatismintheseriskestimates(andinanyconcomitantdecision)cannotbeestimatedwellorcommunicated(HattisandBurmaster1994).Typically,thisresultsinunquantifieduncertaintyinriskstatements.
12
Quantitativeinformationregardingtheprecisionorpotentialsystematicerrorandthedistributionofexposures,effectsandresultingrisksacrossdifferentmembersofanexposedpopulationareusuallynotprovidedwithestimatesgeneratedusingdefaultapproaches.AlthoughDRAmaypresentqualitativeinformationregardingtherobustnessoftheestimates,theimpactofdataandmodellimitationsonthequalityoftheresultscannotbequantified.Relianceondeterministicallyderivedestimationsofriskcanresultindecisionmakingbasedsolelyonpointestimateswithanunknowndegreeofconservatism,whichcancomplicatethecomparisonofrisksormanagementoptions.
Inriskassessmentsofnoncancerendpoints,metricssuchasanoralreferencedose(RfD)andaninhalationreferenceconcentration(RfC)aretypicallyused.Theuseofconservativedefaultslonghasbeenthetargetofcriticism(Finkel1989)andhasledtothepresumptionbycriticsthatEPAassessmentsareoverlyconservativeandunrealistic.TheuseofPRAwouldbeadvantageousineliminatingasinglevalueandmightbelesslikelytoimplyundueprecisionandlessentheneedforconservativeassumptions,therebyreducingbiasintheestimate.Intheprobabilisticframework,aprobabilitydistributionwouldbeusedtoexpressthebeliefthatanyparticularvaluerepresentsthedoseorexposureconcentrationthatwouldposenoappreciableriskofadverseeffects(NRC2009).EPAisinvestigatingtheuseofPRAtoderiveriskvaluesforRfDandRfCinEPA’sIntegratedRiskInformationSystem(IRIS)Database(www.epa.gov/IRIS/).
EPAcommissionedawhitepaper(HattisandLynch2010)presentedattheHazardousAirPollutantWorkshop,2009,illustratingtheimplementationofprobabilisticmethodsindefiningRfDsandassessingthebenefitsforreducingexposuretotoxicantsthatactinpartthroughtraditionalindividualthresholdprocesses.TheuseofPRA,amongotherthings,makesprovisionforinteractionswithbackgroundpathologicalprocesses,asrecommendedbytheNRC(2009),andshowshowthesystemcaninformassessmentsfor“data‐poor”toxicants.
PRAmaybemoresuitablethanDRAforcomplexassessments,includingthoseofaggregateandcumulativeexposuresandtime‐dependentindividualexposure,doseandeffectsanalyses.Identificationandprioritizationofcontributorysourcesofuncertaintycanbedifficultandtimeconsumingwhenusingdeterministicmethods,leadingtodifficultiesinmodelevaluationandthesubsequentappraisalofriskestimates(CullenandFrey1999).Quantitativeanalysesofmodelsensitivitiesareessentialfortheprioritizationofkeyuncertaintiesacriticalprocessinidentifyingstepsfordatacollectionorresearchtoimproveexposureorriskestimates.
1.10. What Is EPA’s Experience with the Use of Probabilistic Risk Analysis?
EPA’sexperiencewithPRAhas,todate,primarilybeenlimitedtotheevaluationofdata,modelandscenariouncertainties.Toassistwiththegrowingnumberofprobabilisticanalysesofexposuredataintheseuncertaintyareas,EPAissuedGuidingPrinciplesforMonteCarloAnalysis(USEPA1997b).Givenadequatesupportingdataandcredibleassumptions,probabilisticanalysistechniques,suchasMonteCarloanalysis,canbeviablestatisticaltoolsforanalyzinguncertaintyandvariabilityinriskassessments.EPA’spolicyfortheuseofprobabilisticanalysisinriskassessment,releasedin1997,isinclusiveofhumanexposureandecologicalriskassessmentsanddoesnotruleoutprobabilistichealtheffectsanalyses(USEPA1997a).Subsequently,EPA’sSABandScientificAdvisoryPanel(SAP)havereviewedPRAapproachestorisksusedbyEPAofficessuchasOAR,OPPandothers.SeveralprogramshavedevelopedspecificguidanceontheuseofPRA,includingOPPandOSWER(USEPA1998a,2001).
ToillustratethepracticalapplicationofPRAtoproblemsrelevanttotheAgency,severalexamplecasestudiesarebrieflydescribedhere.TheAppendixtitledCaseStudyExamplesofApplicationof
13
ProbabilisticRiskAnalysisinU.S.EnvironmentalProtectionAgencyRegulatoryDecisionMaking,discussestheseandothercasestudiesingreaterdetail,includingtheproceduresandoutcomes.TheAppendixincludes16casestudies—11HHRAand5ERAexamples—thatareintendedtoillustratehowsomeofEPA’sprogramsandofficescurrentlyutilizePRA.Toaidindescribinghowprobabilisticanalyseswereused,the16casestudiesaresubdividedamong3categoriesofPRAtools:Group1—pointestimate,includingsensitivityanalysis;Group2—probabilisticriskanalysis,includingone‐dimensionalMonteCarloanalysis(1‐DMCA)andprobabilisticsensitivityanalysis;andGroup3—advancedprobabilisticriskanalysis,includingtwo‐dimensionalMonteCarloanalysis(2‐DMCA)withmicroexposure(microenvironments)modeling,Bayesianstatistics,geostatisticsandexpertelicitation.
ItisusefultonotethattheNRC(2009)recommendedatieredapproachtoriskassessmentusingbothqualitativeandquantitative(deterministicandprobabilistic)tools,withthecomplexityoftheanalysisincreasingasprogressismadethroughthetiers.TheuseofPRAtoolstoaddressissuesofuncertaintyandvariabilityinatieredapproachisdescribedmorecompletelyinSection2.10andwasillustratedinFigure2.ThethreetiersillustratedinthatfigureapproximatelycorrespondtothethreegroupsofEPAcasestudiesdescribedintheAppendixthatprovideexamplesoftheuseofvariousPRAtools.
TableA‐1intheAppendixoffersasummaryofthe16casestudiesbasedonthetypeofriskassessment,thePRAtoolsusedintheassessment,andtheEPAprogramorregionalofficeresponsiblefortheassessment.Someoftheapproachesthatareprofiledinthesecasestudiescanbeusedintheplanningandscopingphasesofriskassessmentsandriskmanagement.Other,morecomplexPRAapproachesareusedtoanswermorespecificquestionsandprovidearicherdescriptionoftherisks.MoststudiesshowthatPRAcanimproveorexpandoninformationgeneratedbydeterministicmethods.Insomeofthecasestudies,theuseofmultiplePRAtoolsisillustrated.Forexample,CaseStudy1describestheuseofapointestimatesensitivityanalysistoidentifyexposurevariablescriticaltotheanalysissummarizedinCaseStudy9.Bothofthesecasestudiesfocusonchildren’sexposuretochromatedcopperarsenate(CCA)‐treatedwood.InCaseStudy9,anMCAwasusedasanexampleofatwo‐dimensional(i.e.,addressingbothvariabilityanduncertainty)probabilisticexposureassessment.
Overall,thecasestudiesillustratethattheAgencyalreadyhasappliedthescienceofPRAtoecologicalriskandhumanexposureestimationandhasbegunusingPRAtodescribehealtheffects.Someoftheapplicationshaveusedexisting“off‐the‐shelf”software,whereasothershaverequiredsignificanteffortandresources.Oncedeveloped,however,someofthemorecomplexmodelshavebeenusedmanytimesfordifferentassessments.Alloftheassessmentshavebeenvalidatedbyinternalandexternalpeerreview.Table1givessomehighlightsthecasestudiesfromdeterministictomorecomplexassessments,whicharedescribedinmoredetailintheAppendix.
14
Table 1. Selected Examples of EPA Applications of Probabilistic Risk Assessment Techniques
Case Study No. Description Group Type of Risk
Assessment Office/Region
2
Atmospheric Deposition to Watershed Contamination: The Office of Research and Development (ORD) developed an analysis of nitrogen, mercury and polycyclic aromatic hydrocarbons (PAHs) depositions toward watershed contamination in the Casco Bay Estuary in southwestern Maine.
Group 1: Point Estimate
Ecological ORD
5
Hudson River Polychlorinated Biphenyl (PCB)-Contaminated Sediment Site: Region 2 evaluated the variability in risks to anglers who consume recreationally caught fish contaminated with PCBs from sediment contamination in the Hudson River.
Group 2: 1-D Monte Carlo Analysis
Human Health Superfund/ Region 2
(New York)
7
Environmental Monitoring and Assessment Program (EMAP): ORD developed and the Office of Water (OW) applied probabilistic sampling techniques to evaluate the Nation’s aquatic resources under the Clean Water Act (CWA) Section 305(b).
Group 2: Probabilistic Sensitivity Analysis
Ecological ORD/OW
9
Chromated Copper Arsenate (CCA) Risk Assessment: ORD and the Office of Pesticide Programs (OPP) conducted a probabilistic assessment of children’s exposure (addressing both variability and uncertainty) to arsenic and chromium from contact with CCA-treated wood play sets and decks.
Group 3: 2-D Monte Carlo Analysis
Human Health ORD/OPP
13
Evaluating Ecological Effects of Pesticide Uses: OPP developed a probabilistic model, which evaluates acute mortality levels in generic and specific ecological species for user-defined pesticide uses and exposures.
Group 3: Probabilistic Analysis
Ecological OPP
14
Fine Particulate Matter Health Impacts: ORD and the Office of Air and Radiation (OAR) used expert elicitation to more completely characterize, both qualitatively and quantitatively, the uncertainties associated with the relationship between reduction in fine particulate matter (PM2.5) and benefits of reduced PM2.5-related mortality.
Group 3: Expert Elicitation
Human Health ORD/OAR
15
2. PROBABILISTIC RISK ANALYSIS 2.1. What Are Uncertainty and Variability, and How Are They Relevant
to Decision Making? Theconceptsofuncertaintyandvariabilityareintroducedhere,andtherelevanceoftheseconceptstodecisionmakingisdiscussed.
2.1.1. Variability Variabilityreferstorealdifferencesovertime,spaceormembersofapopulationandisapropertyofthesystembeingstudied(e.g.,drinkingwaterconsumptionratesforeachofthemanyindividualadultresidentslivinginaspecificlocationordifferencesinbodylengthsorweightsforhumansorecologicalspecies)(CullenandFrey1999;USEPA2011c).Variabilitycanarisefrominherentlyrandomprocesses,suchasvariationsinwindspeedovertimeatagivenlocationorfromtruevariationacrossmembersofapopulationthat,inprinciple,couldbeexplained,butwhich,inpractice,maynotbeexplainableusingcurrentlyavailablemodelsordata(e.g.,therangeofleadlevelsinthebloodofchildren6yearsoldoryoungerfollowingaspecificdegreeofleadexposure).OfparticularinterestinbothHHRAandERAisinter‐individualvariability,whichtypicallyreferstodifferencesbetweenmembersofthesamepopulationineitherbehaviorrelatedtoexposure(e.g.,dietaryconsumptionratesforspecificfooditems),orbiokineticsrelatedtochemicaluptake(e.g.,gastrointestinaluptakeratesforleadfollowingintake)ortoxicresponse(e.g.,differencesamongindividualsorspeciesintheinternaldoseneededtoproduceaspecificamountofneurologicalimpairment).
Inter‐individualvariabilityisillustratedinCaseStudy5intheAppendix,whichassessesaPCB‐contaminatedsedimentsiteintheHudsonRiver.Inthiscasestudy,thequantificationofvariabilityisillustratedthroughtheuseofaPRAtool—1‐DMCA—todescribethevariabilityofexposureasafunctionofindividualexposurefactors(i.e.,youngchildren’sfishingestion).
2.1.2. Uncertainty Uncertaintyisthelackofknowledgeofthetruevalueofaquantityorrelationshipsamongquantities(USEPA2011c).Forexample,theremaybealackofinformationregardingthetruedistributionofvariabilitybetweenindividualsforconsumptionofcertainfooditems.Thereareanumberoftypesofuncertaintiesforbothriskanalysis.Thefollowingdescriptionsofthetypesofuncertainty(adaptedfromCullenandFrey1999)addressesuncertaintiesthatariseduringriskanalyses.Theseuncertaintiescanbeseparatedbroadlyintothreecategories:(1)scenariouncertainty;(2)modeluncertainty;and(3)inputorparameteruncertainty.Eachoftheseisexplainedintheparagraphsthatfollow.
Scenariouncertaintyreferstoerrors,typicallyofomission,resultingfromincorrectorincompletespecificationoftheriskscenariotobeevaluated.Theriskscenarioreferstoasetofassumptionsregardingthesituationtobeevaluated,suchas:(1)thespecificsourcesofchemicalemissionsorexposuretobeevaluated(e.g.,oneindustrialfacilityoraclusterofvariedfacilitiesimpactingthesamestudyarea);(2)thespecificreceptorpopulationsandassociatedexposurepathwaystobemodeled(e.g.,indoorinhalationexposure,track‐industorconsumptionofhome‐produceddietaryitems);and(3)activitiesbydifferentlifestagestobeconsidered(e.g.,exposureonlyathome,orconsiderationofworkplaceorcommutingexposure).Mis‐specificationoftheriskscenariocanresultinunderestimation,overestimationorothermischaracterizationofrisks.Underestimationmayoccurbecauseoftheexclusionofrelevantsituationsortheinclusionofirrelevantsituationswithrespecttoaparticularanalysis.Overestimationmayoccurbecauseoftheinclusionof
16
unrealisticorirrelevantsituations(e.g.,assumingcontinuousexposuretoanintermittentairbornecontaminantsourceratherthanaccountingformobilitythroughouttheday).
Modeluncertaintyreferstolimitationsinthemathematicalmodelsortechniquesthataredevelopedtorepresentthesystemofinterestandoftenstemsfrom:(1)simplifyingassumptions;(2)exclusionofrelevantprocesses;(3)mis‐specificationofmodelboundaryconditions(e.g.,therangeofinputparameters);or(4)misapplicationofamodeldevelopedforotherpurposes.Modeluncertaintytypicallyariseswhentheriskmodelreliesonmissingorimproperlyformulatedprocesses,structuresorequations.SourcesofmodeluncertaintyaredefinedintheGlossary.
Inputorparameteruncertaintytypicallyreferstoerrorsincharacterizingtheempiricalvaluesusedasinputstothemodel(e.g.,engineering,physical,chemical,biologicalorbehavioralvariables).Inputuncertaintycanoriginatefromrandomorsystematicerrorsinvolvedinmeasuringaspecificphenomenon(e.g.,biomarkermeasurements,suchastheconcentrationofmercuryinhumanhair);statisticalsamplingerrorsassociatedwithsmallsamplesizes(e.g.,ifthedataarebasedonsamplesselectedwitharandom,representativesamplingdesign);theuseofsurrogatedatainsteadofdirectlymeasureddata;theabsenceofanempiricalbasisforcharacterizinganinput(e.g.,theabsenceofmeasurementsforfugitiveemissionsfromanindustrialfacility);ortheuseofsummarymeasuresofcentraltendencyratherthanindividualobservations.Nonlinearrandomprocessescanexhibitabehaviorthat,forsmallchangesininputvalues,producesalargevariationinresults.
InputorparameteruncertaintyisillustratedinCaseStudy3intheAppendixtitled“ProbabilisticAssessmentofAnglingDurationUsedintheAssessmentofExposuretoHudsonRiverSedimentsviaConsumptionofContaminatedFish.”Inthiscasestudy,aprobabilisticanalysisofoneparameterinanexposureassessment—thetimeanindividualspendsfishinginalargeriversystem—wasassessedusingsensitivityanalysis.Thisanalysiswasconductedbecausetherewasuncertaintythattheindividualexposuredurationbasedonresidencedurationmayunderestimatethetimespentfishing(i.e.,anglingduration).Thefulldistributionofthecalculatedvalueswasusedinconductingthe1‐DMCAforthefishconsumptionpathway,whichispresentedinCaseStudy5.
Decisionuncertaintyreferstoadecisionanalysisthatwouldincludenotonlytheimpactofscenario,modelandinputuncertaintiesontherelativeattractivenessofpotentialdecisionalternatives,butalsowouldincludethedegreetowhichspecificchoices(suchasselectinginputdata,models,andscenarios,andevenhowtheproblemordecisionanalysisisframed)impacttherelativeattractivenessofpotentialdecisionalternatives.Indecisionmaking,analystsusedatatorepresentdecisioncriteriathatdecisionmakersandotherstakeholdersbelievewillhelpthemtoanswertheirdecisionquestion(s).ThesequestionsmightincludewhichpolicyalternativebestmeetsAgencygoals(thatmustbearticulated)orwhichriskassessmentscenariobestdescribestheobservedeffects.Data,modelandscenariouncertaintieswillinfluencetheriskassessmentresultsandthose,inturn,willinfluencetheriskmanagementoptions.Decisionmakerswhounderstandtheuncertaintyassociatedwiththeirspecificchoicescanbemoreconfidentthatthedecisionwillproducetheresultsthattheyseek.Inaddition,thesedecisionmakerswillbeabletodefendtheirdecisionsbetterandexplainhowthedecisionmeetsAgencyandstakeholdergoals.
Whilethisisbeyondthescopeofthisdocument,StahlandCimorelli(2005and2012)illustratehowuncertaintythroughoutthedecisionmakingprocesscanbeassessed.Thesecasestudiesexploredtheassessmentofozonemonitoringnetworksandairqualitymanagementpoliciesthatseektominimizetheadverseimpactsfromozone,fineparticulatematterandairtoxicssimultaneously.Thesecasestudiesdemonstratetheimportanceandfeasibilityofbetterunderstandingtheuncertaintyintroducedbyspecificchoices(e.g.,selectinginputdata,models,andscenarios)whenmakingpublicpolicydecisions.
17
2.2. When Is Probabilistic Risk Analysis Applicable or Useful? PRAmaybeparticularlyuseful,forexample,inthefollowing(Cooke1991;CullenandFrey1999;NRC2009;USEPA2001):
Whenascreening‐levelDRAindicatesthatrisksarepossiblyhigherthanalevelofconcernandamorerefinedassessmentisneeded.
Whentheconsequencesofusingpointestimatesofriskareunacceptablyhigh.
Whensignificantequityorenvironmentaljusticeissuesareraisedbyinter‐individualvariability.
Toestimatethevalueofcollectingadditionalinformationtoreduceuncertainty.
Toidentifypromisingcriticalcontrolpointsandlevelswhenevaluatingmanagementoptions.
Torankexposurepathways,sites,contaminantsandsoonforthepurposesofprioritizingmodeldevelopmentorfurtherresearch.
Whencombiningexpertjudgmentsonthesignificanceofthedata.
Whenexploringtheimpactoftheprobabilitydistributionsofstakeholderanddecision‐makervaluesontheattractivenessofpotentialdecisionalternatives(Fischhoff1995;Illing1999;KunreutherandSlovic1996;USEPA2000b).
Whenexploringtheimpactoftheprobabilitydistributionsofthedata,modelandscenariouncertainties,andvariabilitytogethertocomparepotentialdecisionalternatives.
PRAmayaddminimalvaluetotheassessmentinthefollowingtypesofsituations(CullenandFrey1999;USEPA1997a):
Whenascreening‐leveldeterministicriskassessmentindicatesthatrisksarenegligible,presumingthattheassessmentisknowntobeconservativeenoughtoproduceoverestimatesofrisk.
Whenthecostofavertingtheexposureandriskissmallerthanthecostofaprobabilisticanalysis.
Whenthereislittleuncertaintyorvariabilityintheanalysis(thisisararesituation).
2.3. How Can Probabilistic Risk Analysis Be Incorporated Into Assessments?
AsillustratedintheaccompanyingcasestudiesintheAppendix,probabilisticapproachescanbeincorporatedintoanystageofariskassessment,fromproblemformulationorplanningandscopingtotheanalysisofalternativedecisions.Insomesituations,PRAcanbeusedselectivelyforcertaincomponentsofanassessment.Itiscommoninassessmentsthatsomemodelinputsareknownwithhighconfidence(i.e.,basedonsite‐specificmeasurements),whereasvaluesforotherinputsarelesscertain(i.e.,basedonsurrogatedatacollectedforadifferentpurpose).Forexample,anexposuremodelermaydeterminethatrelevantairqualitymonitoringdataexists,butthereisalackofdetailedinformationonhumanactivitypatternsindifferentmicroenvironments.Thus,anassessmentofthevariabilityinexposuretoairbornepollutantsmightbebasedondirectuseofthemonitoringdata,whereasassessmentofuncertaintyandvariabilityintheinhalationexposurecomponentmightbebasedonstatisticalanalysisofsurrogatedataoruseofexpertjudgment.Theuncertaintiesarelikelytobelargerforthelatterthantheformercomponentoftheassessment;
18
effortstocharacterizeuncertaintiesassociatedwithpollutantexposureswouldfocusonthelatter.PRAalsodealswithdependencyissues;adescriptionoftheseissuesisavailableinSection3.3.2.
2.4. What Are the Scientific Community’s Views on Probabilistic Risk Analysis, and What Is the Institutional Support for Its Use in Performing Assessments?
TheNRCandIOMrecentlyemphasizedtheirlong‐standingadvocacyforPRA(NRC2007aandb;IOM2013).Datingfromits1983RiskAssessmentintheFederalGovernment:ManagingtheProcess(NRC1983)—whichfirstformalizedtheriskassessmentparadigm—throughreportsreleasedfromthelate1980sthroughtheearly2000s,variousNRCpanelshavemaintainedconsistentlythatbecauseriskanalysisinvolvessubstantialuncertainties,theseuncertaintiesshouldbeevaluatedwithinariskassessment.Thesepanelsnotedthat:
1. Whenevaluatingthetotalpopulationrisk,EPAshouldconsiderthedistributionofexposureandsensitivityofresponseinthepopulation(NRC1989).
2. Whenassessinghumanexposuretoairpollutants,EPAshouldpresentmodelresultsalongwithestimateduncertainties(NRC1991).
3. WhenconductingERA,EPAshoulddiscussthoroughlyuncertaintyandvariabilitywithintheassessment(NRC1993).
4. “Uncertaintyanalysisistheonlywaytocombatthe‘falsesenseofcertainty,’whichiscausedbyarefusaltoacknowledgeand[attemptto]quantifytheuncertaintyinriskpredictions,”asstatedintheNRCreport,ScienceandJudgmentinRiskAssessment(NRC1994).
5. EPA’sestimationofhealthbenefitswasnotwhollycrediblebecauseEPAfailedtodealformallywithuncertaintiesinitsanalyses(NRC2002).
6. EPAshouldadopta“tiered”approachforselectingthelevelofdetailusedinuncertaintyandvariabilityassessment.Furthermore,theNRCrecommendedthatadiscussionofthelevelofdetailusedforuncertaintyanalysisandvariabilityassessmentshouldbeanexplicitpartoftheplanning,scopingandproblemformulationphaseoftheriskassessmentprocess(NRC2009).
7. EPAshoulddevelopmethodstosystematicallydescribeandaccountforuncertaintiesindecision‐relevantfactorsinadditiontoestimatesofhealthriskinitsdecision‐makingprocess(IOM2013).
AskedtorecommendimprovementstotheAgency’sHHRApractices,EPA’sSABechoedtheNRC’ssentimentsandurgedtheAgencytocharacterizeuncertaintyandvariabilitymorefullyandsystematicallyandtoreplacesingle‐pointuncertaintyfactorswithasetofdistributionsusingprobabilisticmethods(ParkinandMorgan2007).ThekeyprinciplesofriskassessmentcitedbytheOfficeofScienceandTechnologyPolicy(OSTP)andtheOfficeofManagementandBudget(OMB)include“explicit”characterizationoftheuncertaintiesinriskjudgments;theyproceedtocitetheNationalAcademyofScience’s(NAS)2007recommendationtoaddressthe“variabilityofeffectsacrosspotentiallyaffectedpopulations”(OSTP/OMB2007).
19
2.5. Additional Advantages of Using Probabilistic Risk Analysis and How It Can Provide More Comprehensive, Rigorous Scientific Information in Support of Regulatory Decisions.
ExternalstakeholderspreviouslyhaveusedtheAdministrativeProcedureActandtheDataQualityActtochallengetheAgencyforalackoftransparencyandconsistencyorfornotfullyanalyzingandcharacterizingtheuncertaintiesinriskassessmentsordecisions(Fisheretal.2006).ThemorecompleteimplementationofPRAandrelatedapproachestodealwithuncertaintiesindecisionmakingwouldaddressstakeholderconcernsinregardtocharacterizinguncertainties.
Theresultsofanyassessment,includingPRA,aredependentontheunderlyingmethodsandassumptions.Accompaniedbytheappropriatedocumentation,PRAmaycommunicateamorerobustrepresentationofrisksandcorrespondinguncertainties.Thischaracterizationmaybeintheformofarangeofpossibleestimatesasopposedtothemoretraditionallypresentedsingle‐pointvalues.Dependingontheuseoftheassessment,rangescanbederivedforvariabilityanduncertainty(oracombinationofthetwo)inbothmodelinputsandresultingestimationsofrisk.
PRAquantifieshowexposures,effectsandrisksdifferamonghumanpopulationsorlifestagesortargetecologicalorganisms.PRAalsoprovidesanestimationofthedegreeofconfidencewithwhichtheseestimatesmaybemade,giventhecurrentuncertaintyinscientificknowledgeandavailabledata.A2007NRCpanelstatedthattheobjectiveofPRAsisnottodecide“howmuchevidenceissufficient”toadoptanalternativebut,rather,todescribethescientificbasesofproposedalternativessothatscientificandpolicyconsiderationsmaybemorefullyevaluated(NRC2007a).EPA’sSABsimilarlynotedthatPRAsprovidemore“valueofinformation”throughaquantitativeassessmentofuncertaintyandclarifythescienceunderlyingAgencydecisions(USEPA2007b).
TheSABarticulatedanumberofadvantagesforEPAdecisionmakersfromtheutilizationofprobabilisticmethods(ParkinandMorgan2007):
AprobabilisticreferencedosecouldhelpreducethepotentiallyinaccurateimplicationofzeroriskbelowtheRfD.
Byunderstandingandexplicitlyaccountingforuncertaintiesunderlyingadecision,EPAcanestimateformallythevalueofgatheringmoreinformation.Bydoingso,theAgencycanbetterprioritizeitsinformationneedsbyinvestinginareasthatyieldthegreatestinformationvalue.
StrategicuseofPRAwouldallowEPAtosendtheappropriatesignaltotheintellectualmarketplace,therebyencouraginganalyststogatherdataanddevelopmethodologiesnecessaryforassessinguncertainties.
2.6. What Are the Challenges to Implementation of Probabilistic Analyses?
Currently,EPAisusingPRAinavarietyofprogramstosupportdecisions,butchallengesremainregardingtheexpandeduseofthesetoolswithintheAgency.Thechallengesinclude:
AlackofunderstandingofthevalueofPRAfordecisionmaking.PRAhelpstoimprovetherigorofthedecision‐makingprocessbyallowingdecisionmakerstoexploretheimpactsofuncertaintyandvariabilityonthedecisionchoices.
Aclearinstitutionalunderstandingofhowtoincorporatetheresultsofprobabilisticanalysesintodecisionmakingislacking.
20
PRAtypicallyrequiresadifferentskillsetthanusedincurrentevaluations,andlimitedresources(staff,time,trainingormethods)toconductPRAareavailable.
Communicatingprobabilisticanalysisresultsandtheimpactofthoseresultsonthedecision/policyoptionscanbecomplex.
Communicationwithstakeholdersisoftendifficultandresultsintheappearanceofregulatorydelaysduethenecessityofanalyzingnumerousscenariosusingvariousmodels.
PRAcomplicatesdecisionmakingandriskcommunicationininstanceswhereamorecomprehensivecharacterizationoftheuncertaintiesleadstoadecreaseinclarityregardinghowtoestimateriskforthescenariounderconsideration.ThesechallengesarediscussedinmoredetailinSections2.7through2.13.
2.7. How Can Probabilistic Risk Analysis Support Specific Regulatory Decision Making?
Decisionmakerssometimesperceivethatthebinarynatureofregulatorydecisions(e.g.,Doesanexposureexceedareferencedoseornot?DoemissionscomplywithAgencystandardsornot?)precludestheuseofariskrangedevelopedthroughPRA.Generally,itisnecessarytoexplaintherationaleunderlyingaparticulardecision.PRA’sprimarypurposeistoprovideinformationtoenhancetheabilitytomaketransparentdecisionsbasedonthebestavailablescience.Byconductingasensitivityanalysisoftheinfluenceoftheuncertaintyonthedecision‐makingprocess,itcanbedeterminedhoworifPRAcanhelptoimprovetheprocess.
PRAcanprovideinformationtodecisionmakersonspecificquestionsrelatedtouncertaintyandvariability.Forquestionsofuncertaintyandtominimizethelikelihoodofunintendedconsequences,PRAcanhelptoprovidethefollowingtypesofinformation:
Characterizationoftheuncertaintyinestimates(i.e.,Whatisthedegreeofconfidenceintheestimate?).Couldthepredictionbeoffbyafactorof2,afactorof10orafactorof1,000?
Criticalparametersandassumptionsthatmostaffectorinfluenceadecisionandtheriskassessment.
“Tippingpoints”wherethedecisionwouldbealterediftheriskestimatesweredifferent,orifadifferentassumptionwasvalid.
Estimatethelikelihoodthatvaluesforcriticalparameterswilloccurortestthevalidityofassumptions.
Estimatethedegreeofconfidenceinaparticulardecisionand/orthelikelihoodofspecificdecisionerrors.
Thepossibilityofalternativeoutcomeswithadditionalinformation,orestimatetradeoffsrelatedtodifferentrisksorrisk‐managementdecisions.
Theimpactofadditionalinformationondecisionmaking,consideringthecostandtimetoobtaintheinformationandtheresultingchangeindecision(i.e.,thevalueoftheinformation).
Fortheconsiderationofvariability,PRAcanhelptoprovidethefollowingtypesofinformationforexposures:
Explicitlydefinedexposuresforvariouspopulationsorlifestages(i.e.,Whoarewetryingtoprotect?).Thatis,willtheregulatoryactionkeep50percent,90percent,99.9percentorsomeotherfractionofthepopulationbelowaspecifiedexposure,doseorrisktarget?
21
Variabilityintheexposures,amongvariouspopulationsorlifestages,andinformationonthepercentileofthepopulationthatisbeingevaluatedintheriskassessment(e.g.,variationsinthenumberoflitersofwaterperkilogram[kg]bodyweightperdayconsumedbythepopulation).Thisinformationishelpfulinaddressingcomments:
OntheconservatismofEPA’sriskassessments;
Concernsaboutwhethertheirparticularexposureswereevaluatedintheriskassessment;
Whomorwhatisbeingprotectedbyimplementingadecision;and
Whetherandwhatadditionalresearchmaybeneededtoreduceuncertainty.
PRAhelpstoinformdecisionsbycharacterizingthealternativesavailabletothedecisionmakerandtheuncertaintyheorshefaces,andbyprovidingevaluationmeasuresofoutcomes.Uncertaintiesoftenarerepresentedasprobabilitiesorprobabilitydistributionsnumericallyoringraphs.Aspartofadecisionanalysis,stakeholderscanmorefullyexaminehowuncertaintiesinfluencethepreferenceamongalternatives.
2.8. Does Probabilistic Risk Analysis Require More Resources Than Default-Based Deterministic Approaches?
PRAgenerallycanbeexpectedtorequiremoreresourcesthanstandardAgencydefault‐baseddeterministicapproaches.ThereisextensiveexperiencewithinEPAinconductingandreviewingDRA.Theseassessmentstendtofollowstandardizedmethodsthatminimizetheeffortrequiredtoconductthemandtocommunicatetheresults.Probabilisticassessmentsoftenentailamoredetailedanalysis,andasaresult,theseassessmentsrequiresubstantiallymoreresources,includingtimeandeffort,thandodeterministicapproaches.
Appropriatelytrainedstaffandtheavailabilityofadequatetools,methodsandguidanceareessentialfortheapplicationofPRA.Properapplicationofprobabilisticmethodsrequiresnotonlysoftwareanddata,butalsoguidanceandtrainingforanalystsusingthetoolsandformanagersanddecisionmakerstaskedwithinterpretingandcommunicatingtheresults.
Anupfrontincreaseinresourcesneededtoconductaprobabilisticassessmentcanbeexpected,butdevelopmentofstandardizedapproachesand/ormethodscanleadtotheroutineincorporationofPRAinAgencyapproaches(e.g.,OPP’suseoftheDietaryExposureEvaluationModel[DEEM;http://www.epa.gov/pesticides/science/deem/],aprobabilisticdietaryexposuremodel).Theinitialand,insomecases,ongoingresourcecost(e.g.,fordevelopmentofsite‐specificmodelsforsiteassessments)maybeoffsetbyamoreinformeddecisionthanacomparabledeterministicanalysis.Probabilisticmethodsareusefulforidentifyingeffectivemanagementoptionsandprioritizingadditionaldatacollectionorresearchaimedatimprovingriskestimation,ultimatelyresultingindecisionsthatenableimprovedenvironmentalprotectionwhilesimultaneouslyconservingmoreresources.
2.9. Does Probabilistic Risk Analysis Require More Data Than Conventional Approaches?
TherearedifferencesofopinionwithinthetechnicalcommunityastowhetherPRArequiresmoredatathanothertypesofanalyses.AlthoughsomeemphaticallybelievethatPRArequiresmoredata,othersarguethatprobabilisticassessmentsmakebetteruseofalloftheavailabledataandinformation.StahlandCimorelli(2005)discusswhenandhowmuchdataarenecessaryforadecision.PRAcanbenefitfrommoredatathanmightbeusedinaDRA.Forexample,whereDRA
22
mightemployselectedpointestimates(e.g.,themeanor95thpercentilevalues)fromavailabledatasetsforuseinmodelinputs,PRAfacilitatestheuseoffrequency‐weighteddatadistributions,allowingforamorecomprehensiveconsiderationoftheavailabledata.Inmanycases,thedatathatwereusedtodevelopthepresumptive95thpercentilecanbeemployedinthedevelopmentofprobabilisticdistributions.
RestrictionofPRAtoprincipallydata‐richsituationsmaypreventitsbroaderapplicationwhereitismostuseful.BecausePRAincorporatesinformationondataquality,variabilityanduncertaintyintoriskmodels,theinfluenceofthesefactorsonthecharacterizationofriskcanbecomeagreaterfocusofdiscussionanddebate.
AkeybenefitofusingPRAisitsabilitytorevealthelimitationsaswellasthestrengthsofdatathatoftenaremaskedbyadeterministicapproach.Indoingso,PRAcanhelptoinformresearchagendas,aswellassupportregulatorydecisionmaking,basedonthestateofthebestavailablescience.Insummary,PRAtypicallyrequiresmoretimefordevelopinginputassumptionsthanaDRA,butwhenincorporatedintotherelevantstepsoftheriskassessmentprocess,PRAcandemonstrateaddedbenefits.Insomecases,PRAcanprovideadditionalinterpretationsthatcompensatefortheextraeffortrequiredtoconductaPRA.
2.10. Can Probabilistic Risk Analysis Be Used to Screen Risks or Only in Complex or Refined Assessments?
Probabilisticmethodstypicallyarenotnecessarywheretraditionaldefault‐baseddeterministicmethodsareadequateforscreeningrisks.Suchmethodsarerelativelylowcost,intendedtoproduceconservativelybiasedestimates,andusefulforidentifyingsituationsinwhichrisksaresolowthatnofurtheractionisneeded.Theapplicationofprobabilisticmethodscanbetargetedtosituationsinwhichascreeningapproachindicatesthatariskmaybeofconcernorwhenthecostofmanagingtheriskishigh,creatinganeedforinformationtohelpinformdecisionmaking.PRAfitsdirectlyintoagraduatedhierarchicalapproachtoriskanalysis.Thistieredapproach,depictedinFigure2,isaprocessforasystematicinformedprogressiontoincreasinglymorecomplexriskassessmentmethods,dependingonthedecision‐makingcontextandneed.Highertiersreflectincreasingcomplexityandoftenwillrequiremoretimeandresources.Ananalysismighttypicallystartatalowertierandonlyprogresstoahighertierifthereisaneedforamoresophisticatedassessmentcommensuratewiththeimportanceoftheproblem.Highertiersalsoreflectincreasingcharacterizationofvariabilityand/oruncertaintyintheriskestimate,whichmaybeimportantforrisk‐managementdecisions.ThecasestudiesdescribedintheAppendixarepresentedinthreegroupsthatgenerallycorrespondtothetiersidentifiedinFigure2.Group1casestudiesarepointestimate(sensitivityanalysis)examples(Tier1);Group2casestudiesincludemostmoderate‐complexityPRAexamples(Tier2);andGroup3casestudiesareadvanced(highcomplexity)PRAexamples(Tier3).
ThetieredapproachinFigure2depictsacontinuumfromscreeninglevelpointestimatethatisdonewithlittledataandconservativeassumptionstoPRAthatrequiresanextensivedatasetandmorerealistic(lessconservative)assumptions.Inbetween,therecanbeawidevarietyoftiersofincreasingcomplexity,ortheremaybeonlyafewreasonablechoicesbetweenscreeningmethodsandhighlyrefinedanalyses(USEPA2004a).Asimilarfour‐tieredapproachforcharacterizingthevariabilityand/oruncertaintyintheestimatedexposureorriskanalysis(WHO2008)hasbeenadaptedbyEPAintheriskandexposureassessmentsconductedfortheNationalAmbientAirQualityStandards(NAAQS).
PRAalsocouldbeusedtoexaminemorefullytheexistingdefault‐basedmethodsbasedonthecurrentstateofinformationandknowledgetodetermineifsuchmethodsaretrulyconservative
23
andadequateforscreening(e.g.,indose‐responseanalysesdealingwithhazardcharacterization)(Swartoutetal.1998;Hattisetal.2002).
Theuseofaspectrumofdatashouldbeemployedbothindeterminingscreeningrisksandinmorecomplexassessments.ForHHRA,datafromhuman,animal,mechanisticandotherstudiesshouldbeusedtodevelopaprobabilisticcharacterizationofcancerandnoncancerrisksandtoidentifyuncertainties.TheNRCrecommendedthatEPAfacilitatethisapproachbyredefiningRfDandRfCwithintheprobabilisticframeworktotakeintoaccounttheprobabilityofharm(NRC2009).ItislikelythatbothDRAandPRAwillbepartofthisframework.
2.11. Does Probabilistic Risk Analysis Present Unique Challenges to Model Evaluation?
Theconceptof“validation”ofmodelsusedforregulatorydecisionmakinghasbeenatopicofintensediscussion.Inarecentreportontheuseofmodelsinenvironmentalregulatorydecisionmaking,theNRCrecommendedusingthenotionofmodel“evaluation”ratherthan“validation,”suggestingthatuseofaprocessthatencompassestheentirelifecycleofthemodelandincorporatesthespectrumofinterestedpartiesintheapplicationofthemodeloftenextendsbeyondthemodelbuilderanddecisionmaker.Suchaprocesscanbedesignedtoensurethatjudgmentofthemodelapplicationisbasednotonlyonitspredictivevaluedeterminedfromcomparisonwithhistoricaldata,butalsoonitscomprehensiveness,rigorindevelopment,transparencyandinterpretability(NRC2007b).
Modelevaluationisimportantinallriskassessments.InthecaseofPRA,thereisanadditionalquestionastothevalidityoftheassumptionsregardingprobabilityandfrequencydistributionsformodelinputsandtheirdependencies.Probabilisticinformationcanbeaccountedforduringevaluationanalysesbyconsideringtherangeofuncertaintyinthemodelpredictionandwhethersucharangeoverlapswiththe“true”valuebasedonindependentdata.Thus,probabilisticinformationcanaidincharacterizingtheprecisionofthemodelpredictionsandwhetherapredictionissignificantlydifferentfromabenchmarkofinterest.Forexample,comparisonsofprobabilisticmodelresultsandmonitoringdatawereperformedformultiplemodelsindevelopingthecumulativepesticideexposuremodel.ConcurrentPRAmodelevaluationsusingaBayesiananalysisalsohavebeenpublished(Clyde2000).
Whenriskassessorsdevelopmodelsofrisk,theyrelyontwopredominantstatisticalmethods.Bothmethodsarisefromaxiomsofprobability,buteachappliestheseaxiomsdifferently.Underthefrequentistapproach,onedevelopsandevaluatesamodelbytestingwhetherthemodel—asappliedtotheobservations—conformstoidealizeddistributions.UndertheBayesianapproach,onedevelopsandevaluatesamodelbytestingwhich—amongalternativemodels—bestyieldstheunderlyingdistributiondescribingthedata.Thepracticaldifferencesbetweenthesetwoapproachescanperhapsbestbeappreciatedwhenconsideringthestructuraluncertaintyinmodels(Section3.3.3).BecauseBayesiansestimatemodelparameterswiththeexpectationthattheseparameters—orevenmodelstructures—willbeupdatedasnewdatabecomeavailable,theyhavedevelopedformaltechniquestoprovideuncertaintyboundsaroundtheseparameterestimates,selectmodelsthatbestexplainthegivendata,orcombinetheresultsofalternativemodels.
2.12. How Do You Communicate the Results of Probabilistic Risk Analysis?
Effectivecommunicationmakesiteasierforregulatorsandstakeholderstounderstandthedecisioncriteriadrivingthedecision‐makingprocess.Inotherwords,communicationofPRAresultswithinthedecision‐makingcontextfacilitatesunderstanding.Thespecificapproachesforreportingresults
24
fromPRAvarydependingontheassessmentobjectiveandtheintendedaudience.Beyondthebasic1997principlesandthepolicyfromthesameyear(USEPA1997aandb),theRiskAssessmentGuidanceforSuperfund:VolumeIII—PartA,ProcessforConductingProbabilisticRiskAssessmentalsoprovidessomeguidanceonthequalityandcriteriaforacceptanceaswellascommunicationbasics(USEPA2001).TherehavebeenlimitedstudiesofhowinformationfromPRAregardinguncertaintyandvariabilitycanorshouldbecommunicatedtokeyaudiences,suchasdecisionmakersandstakeholders(e.g.,MorganandHenrion1990;Bloometal.1993;Krupnicketal.2006).Amongtheanalystcommunity,thereoftenisaninterestinvisualizationofthestructureofascenarioandmodelusinginfluencediagramsanddepictionoftheuncertaintyandvariabilityinmodelinputsandoutputsusingprobabilitydistributionsintheformofcumulativedensityfunctionsorprobabilitydistributionfunctions(Figure3).Sensitivityofthemodeloutputtouncertaintyandvariabilityinmodelinputscanbedepictedusinggraphicaltools.
Insomecases,thesegraphicalmethodscanbeusefulforthoselessfamiliarwithPRA,butinmanycasesthereisaneedtotranslatethequantitativeresultsintoamessagethatextractsthekeyinsightswithoutburdeningthedecisionmakerwithobscuretechnicaldetails.Inthisregard,theuseofrangesofvaluesforaparticularmetricofdecision‐makingrelevance(e.g.,therangeofuncertaintyassociatedwithaparticularestimateofrisk)maybeadequate.ThepresentationofPRAresultstoadecisionmakermaybeconductedbestasaninteractivediscussion,inwhichaprincipalmessageisconveyed,followedbyexplorationofissuessuchasthesource,qualityanddegreeofconfidenceassociatedwiththeinformation.ThereisaneedforthedevelopmentofrecommendationsandacommunicationplanregardinghowtocommunicatetheresultsofPRAtodecisionmakersandstakeholders,buildingontheexperienceofvariousprogramsandregions.
2.13. Are the Results of Probabilistic Risk Analysis Difficult to Communicate to Decision Makers and Stakeholders?
Researchhasshownthattheabilityofdecisionmakerstodealwithconceptsofprobabilityanduncertaintyvaries.Bloometal.(1993)surveyedagroupofseniormanagersatEPAandfoundthatmanycouldinterpretinformationaboutuncertaintyifitwascommunicatedinamannerresponsivetodecision‐makerinterests,capabilitiesandneeds.Inamorerecentsurveyofex‐EPAofficials,Krupnicketal.(2006)concludedthatmosthaddifficultyunderstandinginformationonuncertaintywithconventionalscientificpresentationapproaches.ThefindingsofthesestudieshighlighttheneedforpracticalstrategiesforthecommunicationofresultsofPRAanduncertaintyinformationbetweenriskanalystsanddecisionmakers,aswellasbetweendecisionmakersandotherstakeholders.TheOfficeofEmergencyandRemedialResponse(OERR)hascompiledguidancetoassistanalystsandmanagersinunderstandingandcommunicatingtheresultsofPRA(USEPA2001).
Riskanalystsneedtofocusonhowtouseuncertaintyanalysistocharacterizehowconfidentdecisionmakersshouldbeintheirchoices.AsWilson(2000)explained,“…uncertaintyisthebaneofanydecisionmaker’sexistence.Thus,anyonewhowantstoinformdecisionsusingscientificinformationneedstoassurethattheiranalysestransformuncertaintyintoconfidenceinconclusions.”Hence,althoughenvironmentalriskassessmentsarecomplicatedanditiseasytogetlostinthedetails,presentinganddiscussingtheseresultswithinthecontextofthedecisionfacilitatesunderstanding.Thetranslationofuncertaintyintoconfidencestatementsforcesa“top‐down”perspectivethatpromotesaccountingforwhetherandhowuncertaintiesaffectchoices(Tolletal.1997).
25
Figure 3. Graphical Description of the Likelihood (Probability) of Risk. Hypothetical fitted data distribution with upper and lower confidence intervals are depicted for the output of a 2‐D MCA model.
Example of 2-D MCA Output Graphical
Example 2-D MCA Output
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.0E-06 1.0E-05 1.0E-04 1.0E-03
Risk
Cum
ulat
ive P
roba
bility
Lower confidence bound
Best Estimate
Upper confidence bound
26
3. AN OVERVIEW OF SOME OF THE TECHNIQUES USED IN PROBABILISTIC RISK ANALYSIS
3.1. What Is the General Conceptual Approach in Probabilistic Risk Analysis?
PRAincludesseveralmajorsteps,whichparalleltheacceptedenvironmentalhealthriskassessmentprocess.Theseinclude:(1)problemand/ordecisioncriteriaidentification;(2)gatheringinformation;(3)interpretingtheinformation;(4)selectingandapplyingmodelsandmethodsforquantifyingvariabilityand/oruncertainty;(5)quantifyinginter‐individualorpopulationuncertaintyandvariabilityinmetricsrelevanttodecisionmaking;(6)sensitivityanalysistoidentifykeysourcesofvariabilityanduncertainty;and(7)interpretingandreportingresults.
Problemformulationentailsidentifyingtheassessmentendpointsorissuesthatarerelevanttothedecision‐makingprocessandstakeholders,andthatcanbeaddressedinascientificassessmentprocess.Followingproblemformulation,informationisneededfromstakeholdersandexpertsregardingthescenariostoevaluate.Basedonthescenariosandassessmentendpoints,theanalystsselectordevelopmodels,whichinturnleadstoidentificationofmodelinputdatarequirementsandacquisitionofdataorotherinformation(e.g.,expertjudgmentencodedastheresultofaformalelicitationprocess)thatcanbeusedtoquantifyinputstothemodels.Thedataorotherinformationformodelinputsisinterpretedintheprocessofdevelopingprobabilitydistributionstorepresentvariability,uncertaintyorbothforaparticularinput.Thus,steps(1)through(4)listedabovearehighlyinteractiveanditerativeinthatthedatainputrequirementsandhowinformationistobeinterpreteddependonthemodelformulation,whichdependsonthescenarioandthatinturndependsontheassessmentobjective.Theassessmentobjectivemayhavetoberefineddependingontheavailabilityofinformation.
Onceascenario,modelandinputsarespecified,themodeloutputisestimated.AcommonapproachistouseMonteCarloAnalysis(MCA)orotherprobabilisticmethodstogeneratesamplesfromtheprobabilitydistributionsofeachmodelinput,runthemodelbasedononerandomvaluefromeachprobabilisticinput,andproduceonecorrespondingestimateofthemodeloutputs.Thisprocessisrepeatedtypicallyhundredsorthousandsoftimestocreateasyntheticstatisticalsampleofmodeloutputs.Theseoutputdataareinterpretedasaprobabilitydistributionoftheoutputofinterest.Sensitivityanalysiscanbeperformedtodeterminewhichmodelinputdistributionsaremosthighlyassociatedwiththerangeofvariationinthemodeloutputs.Theresultsmaybereportedinawidevarietyofformsdependingontheintendedaudience,rangingfromqualitativesummariestotables,graphsanddiagrams.
DetailedintroductionstoPRAmethodologyareavailableelsewhere,suchasAngandTang(1984),CullenandFrey(1999),EPA(2001),andMorganandHenrion(1990).AfewkeyaspectsofPRAmethodologyarebrieflymentionedhere.ReaderswhoseekmoredetailshouldconsultthesereferencesandseetheBibliographyforadditionalreferences.
3.2. What Levels and Types of Probabilistic Risk Analyses Are There and How Are They Used?
Therearemultiplelevelsandtypesofanalysisusedtoconductriskassessments(illustratedinFigure2andTable1,respectively).Graduatedapproachestoanalysisarewidelyrecognized(e.g.,USEPA1997a,2001;WHO2008).Theideaofagraduatedapproachistochoosealevelofdetailandrefinementforananalysisthatisappropriatetotheassessmentobjective,dataquality,informationavailableandimportanceofthedecision(e.g.,resourceimplications).
27
Asdiscussedinsection1.8,thereisavarietyofapproachestoriskassessmentthatdifferintheircomplexityandthemannerinwhichtheyaddressuncertaintyandvariability.InDRAonedoesnotformallycharacterizeuncertaintyorvariabilitybutrathertypicallyreliesonusingdefault‐basedassumptionsandfactorstogenerateasingleestimateofrisk.InPRAthereisavarietyofapproachestoexplicitlyaddressorcharacterizeuncertaintyorvariabilityinriskestimatesandthesedifferintermsofhowtheyaccomplishthis,thedataused,andtheoverallcomplexity.Someexamplesare:
Sensitivityanalysis
MonteCarloanalysisofvariabilityinexposuredata
Humanhealthorecologicaleffectsdata
MonteCarloanalysisofuncertainty
“Cumulative”PRAmulti‐pathwayormulti‐chemical
Two‐dimensionalPRAofuncertaintyandvariability
Decisionuncertaintyanalysis
Geospatialanalysis
Expertelicitation
TheDRAapproachesdescribedinSection1.8areexamplesoflowerlevelsinagraduatedapproachtoanalysis.Riskatthelowerlevelsofanalysisisassessedbyconservative,boundingassumptions.Iftheriskestimateisfoundtobeverylowdespitetheuseofconservativeassumptions,thenthereexistsagreatdealofcertaintythattheactualriskstothepopulationofinterestforthegivenscenarioarebelowthelevelofconcernandnofurtherinterventionisrequired,assumingthatthescenarioandmodelspecificationsarecorrect.WhenaconservativeDRAindicatesthatariskmaybehigh,itispossiblethattheriskestimateisbiasedandtheactualriskmaybelower.Insuchasituation,dependingontheresourceimplicationsofthedecision,itmaybeappropriatetoproceedwithamorerefinedorhigherlevelofanalysis.Therelativecostsofinterventionversusfurtheranalysisshouldbeconsideredwhendecidingwhethertoproceedwithadecisionbasedonalowerlevelanalysisortoescalatetoahigherlevelofanalysis.Insomedeterministicassessments(e.g.,ecologicalrisks),theassumptionsarenotwellassuredofconservatism,andtheestimatedrisksmightbebiasedtoappearlowerthantheunseenactualrisk.
AmorerefinedanalysiscouldinvolvetheapplicationofDRAmethods,butwithalternativesetsofassumptionsintendedtocharacterizecentraltendencyandreasonableupperboundsofexposure,effectsandriskestimates,suchthattheestimatescouldbeforanactualindividualinthepopulationofinterestratherthanahypotheticalmaximallyexposedindividual.Suchanalysesarenotlikelytoprovidequantificationregardingtheproportionofthepopulationatorbelowaparticularexposureorrisklevelofconcern,uncertaintiesforanygivenpercentileoftheexposedpopulation,orprioritiesamonginputassumptionswithrespecttotheircontributionstouncertaintyandvariabilityintheestimates.
Tomorefullyanswerthequestionsoftenaskedbydecisionmakers,theanalysiscanbefurtherrefinedbyincorporatingquantitativecomparisonsofalternativemodelingstrategies(torepresentstructuraluncertaintiesassociatedwithscenariosormodels),quantifyingrangesofuncertaintyandvariabilityinmodeloutputs,andprovidingthecorrespondingrangesformodeloutputsofinterest.Whenperformingprobabilisticanalyses,choicesaremaderegardingwhethertofocusonthequantificationofvariabilityonly,uncertaintyonly,bothvariabilityanduncertaintytogether(representingarandomlyselectedindividual),orvariabilityanduncertaintyindependently(e.g.,in
28
atwo‐dimensionaldepictionofprobabilitybandsforestimatesofinter‐individualvariability;seeFigure4).Thesimultaneousbutdistinctpropagationofuncertaintyandvariabilityinatwo‐dimensionalframeworkenablesquantificationofuncertaintyintheriskforanypercentileofthepopulation.Forexample,onecouldestimatetherangeofuncertaintyintheriskfacedbythemedianmemberofthepopulationorthe95thpercentilememberofthepopulation.Suchinformationcanbeusedbyadecisionmakertogaugetheconfidencethatshouldbeplacedinanyparticularestimateofrisk,aswellastodeterminewhetheradditionaldatacollectionorinformationmightbeusefultoreducetheuncertaintyintheestimates.TheOPPassessmentofChromatedCopperArsenate‐treatedwoodusedsuchanapproach.(SeeCaseStudy9intheAppendix.)
Figure 4. Diagrammatic Comparison of Three Alternative Probabilistic Approaches for the Same Exposure Assessment. In Option 1 (one dimensional Monte Carlo analysis), only variability is quantified. In Option 2 (one dimensional Monte Carlo analysis), both uncertainty and variability are combined. In Option 3 (two dimensional Monte Carlo analysis), variability and uncertainty are analyzed separately. Source: WHO 2008.
Whenconductingananalysisforthefirsttime,itmaynotbeknownorclear,priortoanalysis,whichcomponentsofthemodelorwhichmodelinputscontributethemosttotheestimatedriskoritsuncertaintyandvariability.Asaresultofcompletingananalysis,however,theanalystoftengainsinsightintothestrengthsandweaknessesofthemodelsandinputinformation.Probabilisticanalysisandsensitivityanalysiscanbeusedtogethertoidentifythekeysourcesofquantifieduncertaintyinthemodeloutputstoinformdecisionsregardingprioritiesforadditionaldatacollection.Ideally,timeshouldbeallowedforcollectingsuchinformationandrefiningtheanalysistoarriveatamorerepresentativeandrobustestimateofuncertaintyandvariabilityinrisk.Thus,thenotionofiterationindevelopingandimprovingananalysisiswidelyrecommended.
0%
20%
40%
60%
80%
100%
% o
f spe
cies
100 1000 104 105
95% Conf. IntervalMedianEC50
2D
MLE
1DPerc
entil
e co
nsum
er
Exposure (g/day)
1 10 100 1000
Option 1. Only variability
Option 3. Separating variability & uncertainty (2D MC)
Option 2. Combining variability & uncertainty (1D MC)
0%
20%
40%
60%
80%
100%
% o
f spe
cies
100 1000 104 105
95% Conf. IntervalMedianEC50
2D
MLE
1DPerc
entil
e co
nsum
er
Exposure (g/day)
1 10 100 1000
Option 1. Only variability
Option 3. Separating variability & uncertainty (2D MC)
Option 2. Combining variability & uncertainty (1D MC)
29
Thenotionofiterationcanbeappliedbroadlytotheriskassessmentframework.Forexample,afirstefforttoperformananalysismayleadtoinsightthattheassessmentquestionsmightbeimpossibletoaddress,orthatthereareadditionalassessmentquestionsthatmaybeequallyormoreimportant.Thus,iterationcanincludereconsiderationoftheinitialassessmentquestionsandthecorrespondingimplicationsfordefinitionofscenarios,selectionofmodelsandprioritiesforobtainingdataformodelinputs.Alternatively,inatime‐limiteddecisionenvironment,probabilisticandsensitivityanalysesmayofferinsightintotheeffectofmanagementoptionsonriskestimates.
3.3. What Are Some Specific Aspects of and Issues Related to Methodology for Probabilistic Risk Analysis?
ThissectionbrieflydescribesafewkeyaspectsofPRA,modeldevelopmentandassociateduncertainties.DetailedintroductionstoPRAmethodologyareavailableelsewhere,suchasAngandTang(1984),MorganandHenrion(1990),CullenandFrey(1999)andEPA(2001).Formoredetailedinformation,consultthesereferencesandseetheBibliographyforadditionalsources.
3.3.1. Developing a Probabilistic Risk Analysis Model TherearetwokeyissuesthatshouldbeconsideredindevelopingaPRAmodel;asdiscussedbelow.
StructuralUncertaintyinScenarios
Apotentiallykeysourceofuncertaintyinananalysisisthescenario,whichincludesspecificationofpollutantsources,transportpathways,exposureroutes,timingandlocations,geographicextentandrelatedissues.Thereisnoformalizedmethodologyfordealingquantitativelywithuncertaintyandvariabilityinscenarios.Decisionsregardingwhattoincludeorexcludefromascenariocouldberecastashypothesesregardingwhichagents,pathways,microenvironments,etc.,contributesignificantlytotheoverallexposureandriskofinterest.Inpractice,however,theuseofqualitativemethodstoframeanassessmenttendstobemorecommon,giventheabsenceofaformalquantitativemethodology.
CoupledModels
Forsource‐to‐outcomeriskassessments,itoftenisnecessarytoworkwithmultiplemodels,eachofwhichrepresentsadifferentcomponentofascenario.Forexample,theremaybeseparatemodelsforemissions,airquality,exposure,doseandeffects.Suchmodelsmayhavedifferentspatialandtemporalscales.Whenconductinganintegratedassessment,theremaybesignificantchallengesandbarrierstocouplingsuchmodelsintoonecoherentframework.Sometimes,thecouplingisdonedynamicallyinasoftwareenvironment.Inothercases,theoutputofonemodelmightbeprocessedmanuallytopreparetheinformationforinputtothenextmodel.Furthermore,theremaybefeedbackbetweencomponentsofthescenario(e.g.,poorairqualitymightaffecthumanactivity,which,inturn,couldaffectbothemissionsandexposures)thatareincompletelycapturedornotincluded.Thus,thecouplingofmultiplemodelscanbeapotentiallysignificantsourceofstructuraluncertainty(Özkaynak2009).
3.3.2. Dealing With Dependencies Among Probabilistic Inputs Whenrepresentingtwoormoreinputstoamodelasprobabilitydistributions,thequestionarisesastowhetheritisreasonabletoassumethatthedistributionsarestatisticallyindependent.Ifthereisadependence,itcouldbeassimpleasalinearcorrelationbetweentwoinputs,oritcouldbemorecomplicated,suchasnonlinearornonmonotonicrelationships.Dependenciestypicallyarenotimportantiftheriskestimateorothermodeloutputissensitivetooneornoneoftheprobabilisticinputsthatmighthaveinterdependence.Furthermore,dependenciestypicallyarenotofpracticalimportanceiftheyareweak.Whendependenciesexistandmightsignificantlyinfluence
30
theriskestimate,theycanbetakenintoaccountusingavarietyofstatisticalsimulationmethodsor,perhapsmoreappropriately,bymodelingthedependenceanalyticallywherepossible.DetailsonmethodsforassessingtheimportanceofpossibledependenciesandofquantifyingthemwhenneededaredescribedinFersonetal.(2004and2009).
Forsometypesofmodels,suchasairqualitymodels,itisnotpossibletointroduceaprobabilitydistributiontooneinput(e.g.,ambienttemperatureataparticularlocation)withoutaffectingvariablesatotherlocationsortimes(e.g.,temperaturesinotherlocationsatthesametimesortemporaltrendsintemperature).Insuchcases,itisbettertoproducean“ensemble”ofalternativetemperaturefields,eachofwhichisinternallyconsistent.Individualmembersofanensembleusuallyarenotinterpretedasrepresentingaprobabilitysample;however,comparisonofmultipleensemblesofmeteorologicalconditions,forexample,canprovideinsightintonaturalsourcesofvariabilityinambientconcentrations.
3.3.3. Conducting the Probabilistic Analysis
QuantifyingUncertaintyandVariabilityinModelInputsandParameters
Afterthemodelsareselectedordevelopedtosimulateascenarioofinterest,attentiontypicallyturnstothedevelopmentofinputdataforthemodel.Thereisasubstantialamountofliteratureregardingtheapplicationofstatisticalmethodsforquantifyinguncertaintyandvariabilityinmodelinputsandparametersbasedonempiricaldata(e.g.,AngandTang1984;CullenandFrey1999;MorganandHenrion1990;USEPA2001).Forexample,acommonlyusedmethodforquantifyingvariabilityinamodelinputistoobtainasampleofdata,selectatypeofparametricprobabilitydistributionmodeltofittothedata(e.g.,normal,lognormalorotherform),estimatetheparametersofthedistributionbasedonthedata,critiquethegoodness‐of‐fitusinggraphical(e.g.,probabilityplot)andstatistical(e.g.,Anderson‐Darling,Chi‐SquareorKolmogorov‐Smirnovtests)methodsandchooseapreferredfitteddistribution.Thismethodologycanbeadjustedtoaccommodatevarioustypesofdata,suchasdatathataresamplesfrommixturesofdistributionsorthatcontainnon‐detected(censored)values.Uncertaintiescanbeestimatedbasedonconfidenceintervalsforstatisticsofinterest,suchasmeanvalues,ortheparametersoffrequencydistributionsforvariability.Varioustextsandguidancedocuments,bothAgencyandprogrammatic,describetheseapproaches,includingtheGuidingPrinciplesforMonteCarloAnalysis(USEPA1997b).
Themostcommonmethodforestimatingaprobabilitydistributionintheoutputofamodel,basedonprobabilitydistributionsspecifiedformodelinputs,isMCS(CullenandFrey1999;MorganandHenrion1990).MCSispopularbecauseitisveryflexible.MCScanbeusedwithawidevarietyofprobabilitydistributionsaswellasdifferenttypesofmodels.ThemainchallengeforMCSisthatitrequiresrepetitivemodelcalculationstoconstructasetofpseudo‐randomnumbersformodelinputsandthecorrespondingestimatesformodeloutputsofinterest.TherearealternativestoMCSthataresimilarbutmorecomputationallyefficient,suchasLatinHypercubeSampling(LHS).TechniquesareavailableforsimulatingcorrelationsbetweeninputsinbothMCSandLHS.Formodelswithverysimplefunctionalforms,itmaybepossibletouseexactorapproximateanalyticalcalculations,butsuchsituationsareencounteredinfrequentlyinpractice.Theremaybesituationsinwhichthedatadonotconformtoawell‐definedprobabilitydistribution.Insuchcases,algorithms(suchasMarkovChainMonteCarlo)canestimateaprobabilitydistributionbycalculatingamathematicalformdescribingthepatternofobserveddata.Thisform,calledthelikelihoodfunction,isakeycomponentofBayesianinferenceand,therefore,servesasthebasisforsomeoftheanalyticalapproachestouncertaintyandvariabilitydescribedbelow.
Theuseofempiricaldatapresumesthatthedatacomprisearepresentative,randomsample.Ifknownbiasesorotherdataqualityproblemsexist,orifthereisascarcityorabsenceofrelevantdata,thennaïverelianceonavailableempiricaldataislikelytoresultinmisleadinginferencesin
31
theanalysis.Alternatively,estimatesofuncertaintyandvariabilitycanbeencoded,usingformalprotocols,basedonelicitationofexpertjudgment(e.g.,MorganandHenrion1990,USEPA2011a).Elicitationofexpertjudgmentforsubjectiveprobabilitydistributionsisusedinsituationswherethereareinsufficientdatatosupportastatisticalanalysisofuncertainty,butinwhichthereissufficientknowledgeonthepartofexpertstomakeaninferenceregardinguncertainty.Forexample,EPAconductedanexpertelicitationstudyontheconcentration‐responserelationshipbetweentheannualaverageambientlessthan2.5micrometer(µm)diameterparticulatematter(PM2.5)exposureandannualmortality(IEC2006;seealsoCaseStudies6and14intheAppendix).Subjectiveprobabilitydistributionsthatarebasedonexpertjudgmentcanbe“updated”withnewdataastheybecomeavailableusingBayesianstatisticalmethods.
StructuralUncertaintyinModels
Theremaybesituationsinwhichitprovesusefultoevaluatenotjusttheuncertaintiesininputsandparametervalues,butalsouncertaintiesregardingwhetheramodeladequatelycaptures—inahypothesized,mathematical,structuredform—therelationshipunderinvestigation.Aqualitativeapproachtoevaluatingthestructuraluncertaintyinamodelincludesdescribingthecriticalassumptionswithinamodel,thedocumentationofamodelorthemodelquality,andhowthemodelfitsthepurposeoftheassessment.Quantitativeapproachestoevaluatingstructuraluncertaintyinmodelsaremanifold.Theseincludeparameterizationofageneralmodelthatcanbereducedtoalternativefunctionalforms(e.g.,MorganandHenrion1990),enumerationofalternativemodelsinaprobabilitytree(e.g.,Evansetal.1994),comparingalternativemodelsbyevaluatinglikelihoodfunctions(e.g.,Royall1997;BurnhamandAnderson2002),poolingresultsofmodelalternativesusingBayesianmodelaveraging(e.g.,Hoetingetal.1999)ortestingthecausalrelationshipswithinalternativemodelsusingBayesianNetworks(Pearl2009).
SensitivityAnalysis:IdentifyingtheMostImportantModelInputs
Probabilisticmethodstypicallyfocusonhowuncertaintyorvariabilityinamodelinputaffect[orresultin]withrespecttouncertaintyorvariabilityinamodeloutput.Afteraprobabilisticanalysisiscompleted,sensitivityanalysistypicallytakestheperspectiveoflookingbacktoevaluatehowmuchofthevariationinthemodeloutputisattributabletoindividualmodelinputs(e.g.,FreyandPatil2002;Mokhtarietal.2006;Saltellietal.2004).
Therearemanytypesofsensitivityanalysismethods,includingsimpletechniquesthatinvolvechangingthevalueofoneinputatatimeandassessingtheeffectonanoutput,andstatisticalmethodsthatevaluatewhichofmanysimultaneouslyvaryinginputscontributethemosttothevarianceofthemodeloutput.Sensitivityanalysiscananswerthefollowingkeyquestions:
Whatistheimpactofchangesininputvaluesonmodeloutput?
Howcanvariationinoutputvaluesbeapportionedamongmodelinputs?
Whataretherangesofinputsassociatedwithbestorworstoutcomes?
Whatarethekeycontrollablesourcesofvariability?
Whatarethecriticallimits(e.g.,theemissionreductiontarget)?
Whatarethekeycontributorstotheoutputuncertainty?
Thus,sensitivityanalysiscanbeusedtoinformdecisionmaking.
Iteration
Therearetwomajortypesofiterationinriskassessmentmodeling.Oneisiterativerefinementofthetypeofanalysis,perhapsstartingwitharelativelysimpleDRAasascreeningstepinaninitial
32
levelofanalysisandproceedingtomorerefinedtypesofassessmentsasneededinsubsequentlevelsofanalysis.ExamplesofmorerefinedlevelsofassessmentincludeapplicationofsensitivityanalysistoDRA;theuseofprobabilisticmethodstoquantifyvariabilityonly,uncertaintyonly,orcombinedvariabilityanduncertainty(torepresentarandomlyselectedindividual);ortheuseoftwo‐dimensionalprobabilisticmethodsfordistinguishingandsimultaneouslycharacterizingbothuncertaintyandvariability.
Theothertypeofiterationoccurswithinaparticularlevelandincludesiterativeeffortstoformulateamodel,obtaindataandevaluatethemodeltoprioritizedataneeds.Forexample,amodelmayrequirealargenumberofinputassumptions.Toprioritizeeffortsofspecifyingdistributionsforuncertaintyandvariabilityformodelinputs,itisusefultodeterminewhichmodelinputsarethemostinfluentialwithrespecttotheassessmentendpoint.Therefore,sensitivitycanbeusedbasedonpreliminaryassessmentsofrangesordistributionsforeachmodelinputtodeterminewhichinputsarethemostimportanttotheassessment.Refinedeffortstocharacterizedistributionsthencanbeprioritizedtothemostimportantinputs.
33
4. SUMMARY AND RECOMMENDATIONS 4.1. Probabilistic Risk Analysis and Related Analyses Can Improve
the Decision-Making Process at EPA PRAcanprovideuseful(evencritical)informationabouttheuncertaintiesandvariabilityinthedata,models,scenario,expertjudgmentsandvaluesincorporatedinriskassessmentstosupportdecisionmakingacrosstheAgency.AsdiscussedinthispaperPRAisananalyticalmethodologycapableofincorporatinginformationregardinguncertaintyand/orvariabilityinriskanalysestoprovideinsightonthedegreeofcertaintyofariskestimateandhowtheriskestimatevarieswithintheexposedpopulation.TraditionalapproachessuchasDRA,oftenreportrisksusingdescriptorssuchas“centraltendency,”“highend”(e.g.,90thpercentileorabove)or“maximumanticipatedexposure”.BycontrastPRAcanbeusedtodescribemorecompletelytheuncertaintysurroundingsuchestimates,aswellastoidentifythekeycontributorstouncertaintyandvariabilityinpredictedexposuresorriskestimates.Thisinformationthencanbeusedbydecisionmakerstoweighalternatives,ortomakedecisionsonwhethertocollectadditionaldata,ortoconductadditionalresearchinordertoreducetheuncertaintyandfurthercharacterizevariabilitywithintheexposedpopulation.Informationonuncertaintiesandvariabilityinexposureandresponsecanultimatelyimprovetheriskestimates.
PRAcanbeusedtoobtaininsightonwhetheronemanagementalternativeismorelikelytoreduceriskscomparedtoanother.Inaddition,PRAcanfacilitatethedevelopmentofmodelingscenariosandthesimultaneousconsiderationofmultiplemodelalternatives.Probabilisticmethodsofferanumberoftoolsdesignedtoincreaseconfidenceindecisionmakingthroughtheincorporationofinputuncertaintyandvariabilitycharacterizationandprioritizationinriskanalyses.Forexample,onePRAtool,sensitivityanalysescanbeusedtoidentifyinfluentialknowledgegapsintheestimationofrisk;thisimprovestransparencyinthepresentationoftheseuncertaintiesandimprovestheabilitytocommunicatethemostrelevantinformationmoreclearlytodecisionmakersandstakeholders.PRAallowsonetoinvestigatepotentialchangesindecisionsthatcouldresultfromthecollectionofadditionalinformation.However,theadditionalresources(e.g.,time,costs,orexpertise)toundertakeneedtobeweighedagainstthepotentialimprovementsinthedecisionmakingprocess.Ultimately,PRAmayenhancethescientificfoundationoftheEPA’sapproachtodecisionmaking.
Thevarioustoolsandmethodsdiscussedinthiswhitepapercanbeutilizedatallstagesofriskanalysisandalsocanaidthedecision‐makingprocessby,forexample,characterizinginter‐individualvariabilityanduncertainties.
PRAandrelatedmethodsareemployedinvaryingdegreesacrosstheAgency.BasicguidanceexistsatEPAontheuseandacceptabilityofPRAforriskestimation,butimplementationvariesgreatlywithinprograms,officesandregions.TheuseofMonteCarloorotherprobability‐basedtechniquestoderivearangeofpossibleoutputsfromuncertaininputsisafairlywell‐developedapproachwithinEPA.Althoughhighlysophisticatedhumanexposureassessmentandecologicalriskapplicationshavebeendeveloped,theuseofPRAmodelstoassesshumanhealtheffectsanddose‐responserelationshipshasbeenmorelimitedattheAgency.
TheevaluationoftheapplicationofPRAtechniquesunderspecificlawsandregulationsvariesbyprogram,officeandregion.Movingforward,itisimportanttobroadendiscussionsbetweenriskassessorsandriskmanagersregardinghowPRAtoolscanbeusedtosupportspecificdecisionsandhowtheycanbeusedwithintheregulatoryframeworkusedbyprograms,offices,andregionstomakedecisions.Thiscanbeaccomplishedbyexpandingthedialoguebetweenassessorsand
34
managesatalllevelsregardinghowthePRAtoolshavebeenusedandhowtheyhaveenhanceddecisionmaking.
IncreaseduseofPRAandconsistentapplicationofPRAtoolsinsupportofEPAdecisionmakingrequiresenhancedinternalcapacityforconductingtheseassessments,aswellasimprovedinterpretationandcommunicationofsuchinformationinthecontextofdecisions.ImprovementsofAgencycapacitycouldbeaccomplishedthroughsharingofexperiences,knowledgeandtrainingandincreasedavailabilityoftoolsandmethods.
4.2. Major Challenges to Using Probabilistic Risk Analysis to Support Decisions
ThechallengesforEPAaretwo‐fold.AsanAgencyresponsibleforprotectinghumanhealthandtheenvironment,EPAmakesregulatoryandpolicydecisions,eveninthepresenceofconflictingstakeholderpositionsandtheinevitableuncertaintiesinthescience.ThefirstchallengeforEPAistodeterminehowtoconductitsdecision‐makingresponsibilities,weighingdeterminationsofwhatconstitutestoomuchuncertaintytomakeadecision,againstpotentialadverseconsequencesofpostponingdecisions.
Thesecondchallenge,isthatalthoughcurrentPRAtechniquesareavailablethatwouldhelptoinformEPAdecision‐makingprocesses,researchandguidanceareneededtoimprovethesemethodsforamorecompleteimplementationofPRAinHHRAandERA.Inparticular,additionalguidanceisneededtohelpanalystsanddecisionmakersbetterunderstandhowtoincorporatePRAapproachesintothedecision‐makingprocess.Thisincludes,guidanceonwhichstatisticaltoolstouseandwhentousethem,andhowprobabilisticinformationcanhelptoinformthescientificbasisofdecisions.BothDRAandPRAaswellasappropriatestatisticalmethodsmaybeusefulatanystageoftheriskanalysisanddecision‐makingprocess,fromplanningandscopingtocharacterizingandcommunicatinguncertainty.
AsnotedinSection3.3,therearesignificantchallengesinproperlyaccountingforuncertaintyandvariabilitywhenmultiplemodelsarecoupledtogethertorepresentthesource‐to‐outcomecontinuum.Moreover,thecouplingofmultiplemodelsmightneedtoinvolveinputsandcorrespondinguncertaintiesthatareincorporatedintomorethanonemodel,potentiallyresultingincomplexdependencies.Integrativeresearchoncoupledmodeluncertaintieswillbequitevaluable.
Theremaybemismatchesinthetemporalandspatialresolutionofeachmodelthatconfoundtheabilitytopropagateuncertaintyandvariabilityfromonemodeltoanother.Forsomemodels,thekeyuncertaintiesmaybeassociatedwithinputs,whereasforothermodels,thekeyuncertaintiesmaybeassociatedwithstructureorparameterizationalternatives.Modelintegrationandharmonizationactivitieswillbeimportanttoaddressingthesetechnicalissues.
4.3. Recommendations for Enhanced Utilization of Probabilistic Risk Analysis at EPA
Someexamplesofareaswhereneworupdatedguidancewouldbehelpfularethese:
IdentificationofdifferenttypesofinformationrequiredforthevariousAgencydecision‐makingprocesses,suchasdataanalysis,tools,models,anduseofexperts.
Useofprobabilisticapproachestoevaluatehealtheffectsdata.
UseofprobabilisticapproachesforERA.
35
Integratingprobabilisticexposureandriskestimatesandcommunicatinguncertaintyandvariability.
Inordertosupportthedevelopmentofguidanceontheseorrelatedtopics,followingstudiesorresearcharerecommended:
TheuseofPRAmodelstoevaluatetoxicitydatahasbeenverylimited.Scientific,technicalandpolicy‐baseddiscussionsareneededinthisarea.
Additionalresearchonformalmethodsfortreatingmodeluncertaintieswillbevaluable.
Somestepstoimproveimplementationincludethese:
InformingdecisionmakersabouttheadvantagesanddisadvantagesofusingPRAtechniquesintheirdecision‐makingprocessesthroughlectures,webinarsandcommunicationsregardingthetechniquesandtheiruseinEPA.
IncorporatingadiscussionofPRAtoolsduringPlanningandScopingforHHRAsandERAs.
ContinuingthedialoguebetweenassessorsandmanagersonhowtousePRAwithintheregulatorydecisionmakingprocess.
ConductingmeetingsanddiscussionsofPRAtechniquesandtheirapplicationwithbothmanagersandassessorstoaidinprovidinggreaterconsistencyandtransparencyinEPA’sriskassessmentandriskmanagementprocessandindevelopingEPA’sinternalcapacity.
Developinga“CommunityofPractice”forfurtherdiscussionregardingtheapplicationofPRAtechniquesandtheuseofthesetoolsindecisionmaking.
Riskassessorsandriskmanagersneedinformationandtrainingsothattheycanbetterutilizethesetools.Educationandexperiencewillgeneratefamiliaritywiththesetools,whichwillhelpanalystsanddecisionmakersbetterunderstandandconsidermorefullyutilizingthesetechniqueswithintheirregulatoryprograms.Increasedtrainingisneededtofacilitateunderstandingonalllevelsandmayincludethefollowing:
ProvidingintroductoryaswellasadvancedtrainingtoallEPAoffices.
TrainingriskassessorsandriskmanagersinthePRAtechniquessothattheycanlearnaboutthevarioustoolsavailable,theirapplications,softwareandreviewconsiderations,andresourcesforadditionalinformation(e.g.,expertsandsupportserviceswithintheAgency).
Providingeasilyavailable,flexible,modulartrainingforalllevelsofexperiencetofamiliarizeEPAemployeeswiththemenuoftoolsandtheircapacities.
Providingliveandrecordedseminarsandwebinarsforintroductoryandsupplementaleducation,aswellasperiodic,centralizedhands‐ontrainingsessionsdemonstratinghowtoutilizesoftwareprograms.
TrainingiscriticalbothforanimprovedunderstandingbutalsotobuildincreasedcapacityintheAgencyandexplicitstepscouldincludethese:
Demonstrating,throughinformationalopportunitiesandresourcelibraries,thevarioustoolsandmethodsthatcanbeusedatallstagesofriskanalysistoaidthedecision‐makingprocessbycharacterizinginter‐individualvariabilityanduncertainties.
Promotingthesharingofexperience,knowledge,modelsandbestpracticesviameetingsofriskassessorsandmanagers;electronicexchanges,suchastheEPAPortalEnvironmental
36
ScienceConnector(https://ssoprod.epa.gov/sso/jsp/obloginESCNew.jsp);andmoredetaileddiscussionsofthecasestudies.
AsEPAworkstowardthemoreintegratedevaluationofenvironmentalproblems,thiswillincludenotjusttheimprovedunderstandingofsinglepollutants/singlemedia,butmulti‐pollutant,multi‐mediaandmulti‐receptoranalysiswithinadecisionanalyticframework.EPAisbeginningtobuildsuchintegratedcapabilityintoanalyticaltoolslikePRA(BabendreierandCastleton2005;Stahletal.2011).
TheRAFwillbetakingaleadershiprolethroughtheUncertaintyandVariabilityWorkgrouptomorefullyevaluatetheapplicationanduseofPRAtoolsandbroadeningthedialoguebetweenassessorsandmanagers.UpdatesontheprogressofthisTechnicalPanelwillbeprovidedontheRAFwebpageat:www.epa.gov/raf.
37
GLOSSARY Analysis.Examinationofanythingcomplextounderstanditsnatureortodetermineitsessentialfeatures(WHO2004).
Assessment.Adeterminationorappraisalofpossibleconsequencesresultingfromananalysisofdata(2011b).
Assessmentendpoint.Anexplicitexpressionoftheenvironmentalvaluethatistobeprotected,operationallydefinedbyanecologicalentityanditsattributes.Forexample,salmonarevaluedecologicalentities;reproductionandageclassstructurearesomeoftheirimportantattributes.Together,salmon“reproductionandageclassstructure”formanassessmentendpoint(USEPA1998b).
Bayesianprobability.Anapproachtoprobability,representingapersonaldegreeofbeliefthatavalueofrandomvariablewillbeobserved.Alternatively,theuseofprobabilitymeasurestocharacterizethedegreeofuncertainty(Gelmanetal.2004).
BayesianAnalysis.Bayesiananalysisisamethodofstatisticalinferenceinwhichtheknowledgeofprioreventsisusedtopredictfutureevents(USEPA2011b).
Correlation.Anestimateofthedegreetowhichtwosetsofvariablesvarytogether,withnodistinctionbetweendependentandindependentvariables.Correlationreferstoabroadclassofstatisticalrelationshipsinvolvingdependence(USEPA2012).
Criticalcontrolpoint.Acontrollablevariablethatcanbeadjustedtoreduceexposureandrisk.Forexample,acriticalcontrolpointmightbetheemissionratefromaparticularemissionsource.Theconceptofcriticalcontrolpointisfromthehazardassessmentandcriticalcontrolpointconceptforriskmanagementthatisusedinspaceandfoodsafetyapplications,amongothers(USEPA2006c).
Criticallimit.Anumericalvalueofacriticalcontrolpointatorbelowwhichriskisconsideredtobeacceptable.Acriterionthatseparatesacceptabilityfromunacceptability(USEPA2006c).
Deterministic.Amethodologyrelyingonpoint(i.e.,exact)valuesasinputstoestimaterisk;thisobviatesquantitativeestimatesofuncertaintyandvariability.Resultsalsoarepresentedaspointvalues.Uncertaintyandvariabilitymaybediscussedqualitativelyorsemi‐quantitativelybymultipledeterministicriskestimates(USEPA2006b).
Deterministicriskassessment(DRA).Riskevaluationinvolvingthecalculationandexpressionofriskasasinglenumericalvalueor“singlepoint”estimateofrisk,withuncertaintyandvariabilitydiscussedqualitatively(USEPA2012).
Ecologicalriskassessment.Theprocessthatevaluatesthelikelihoodthatadverseecologicaleffectsmayoccurorareoccurringasaresultofexposuretooneormorestressors(USEPA1998b).
Ecosystem.Thebioticcommunityandabioticenvironmentwithinaspecifiedlocationinspaceandtime(USEPA1998b).
Ensemble.Amethodforpredictivemodelingbasedonmultiplemeasuresofthesameeventovertime(e.g.,theamountofcarbondioxidepresentintheatmosphereatselectedtimepoints).Thecollectionofdatainputisknownasanensembleandcanbeusedtodevelopaquantificationofpredictionvariabilitywithinthemodel.Ensemblemodelingisusedmostcommonlyinatmosphericpredictioninforecasting,althoughensemblemodelinghasbeenappliedtobiologicalsystemstobetterquantifyrisksofeventsorperturbationswithinbiologicalsystems(FuentesandFoley2012).
Environment.Thesumofallexternalconditionsaffectingthelife,developmentandsurvivalofanorganism(USEPA2010a).
38
Expertelicitation.Asystematicprocessofformalizingandquantifying,typicallyinprobabilisticterms,expertjudgmentsaboutuncertainquantities(USEPA2011a).
Frequentist(orfrequency)probability.Aviewofprobabilitythatconcernsitselfwiththefrequencywith which an event occurs givenalongsequenceofidenticalandindependenttrials(USEPA1997b).
Hazardidentification.Theriskassessmentprocessofdeterminingwhetherexposuretoastressorcancauseanincreaseintheincidenceorseverityofaparticularadverseeffect,andwhetheranadverseeffectislikelytooccur(USEPA2012).
Humanhealthriskassessment.1.Theprocesstoestimatethenatureandprobabilityofadversehealtheffectsinhumanswhomaybeexposedtochemicalsincontaminatedenvironmentalmedia,noworinthefuture(USEPA2010b).2.Theevaluationofscientificinformationonthehazardouspropertiesofenvironmentalagents(hazardcharacterization),thedose‐responserelationship(dose‐responseassessment),andtheextentofhumanexposuretothoseagents(exposureassessment).Theproductoftheriskassessmentisastatementregardingtheprobabilitythatpopulationsorindividualssoexposedwillbeharmedandtowhatdegree(riskcharacterization)(USEPA2006a).
Inputs.Quantitiesthatareappliedtoamodel(WHO2008).
LikelihoodFunction.Anapproachtomodelingexposureinwhichlong‐termexposureofanindividualissimulatedasthesumofseparateshort‐termexposureevents(USEPA2001).
Microenvironment.Well‐definedsurroundingssuchasthehome,office,automobile,kitchen,store,etc.,thatcanbetreatedashomogenous(orwellcharacterized)intheconcentrationsofachemicalorotheragent(USEPA1992).
Microexposureevent(MEE)analysis.Anapproachtomodelingexposureinwhichlong‐termexposureofanindividualissimulatedasthesumofseparateshort‐termexposureevents(USEPA2001).
Model.Amathematicalrepresentationofanaturalsystemintendedtomimicthebehavioroftherealsystem,allowingdescriptionofempiricaldata,andpredictionsaboutuntestedstatesofthesystem(USEPA2006b).
Modelboundaries.1.Decisionsregardingthetime,space,numberofchemicals,etc.,usedinguidingmodelingofthesystem.Riskscanbeunderstatedoroverstatedifthemodelboundaryismis‐specified.Forexample,ifastudyareaisdefinedtobetoolargeandincludesasignificantnumberoflow‐exposureareas,thenapopulation‐levelriskdistributioncanbedilutedbyincludinglessexposedindividuals,whichcan,inturn,resultinarisk‐baseddecisionthatdoesnotprotectsufficientlythemostexposedindividualsinthestudyarea.2.Designatedareasofcompetenceofthemodel,includingtime,space,pathogens,pathways,exposedpopulations,andacceptablerangesofvaluesforeachinputandjointlyamongallinputsforwhichthemodelmeetsdataqualityobjectives(WHO2008).
Modeling.Developmentofamathematicalorphysicalrepresentationofasystemortheorythataccountsforallorsomeofitsknownproperties.Modelsoftenareusedtotesttheeffectofchangesofcomponentsontheoverallperformanceofthesystem(USEPA2010a).
Modeluncertainty(sourcesof):
Modelstructure.Asetofassumptionsandinferenceoptionsuponwhichamodelisbased,includingunderlyingtheoryaswellasspecificfunctionalrelationships(WHO2008).
39
Modeldetail.Levelofsimplicityordetailassociatedwiththefunctionalrelationshipsassumedinthemodelcomparedtotheactualbutunknownrelationshipsinthesystembeingmodeled(WHO2008).
Extrapolation.Useofmodelsoutsideoftheparameterspaceusedintheirderivationmayresultinerroneouspredictions.Forexample,athresholdforhealtheffectsmayexistatexposurelevelsbelowthosecoveredbyaparticularepidemiologicalstudy.Ifthatstudyisusedinmodelinghealtheffectsatthoselowerlevels(anditisassumedthatthelevelofresponseseeninthestudyholdsforlowerlevelsofexposure),thendiseaseincidencemaybeoverestimated(USEPA2007a).
MonteCarloanalysis(MCA)orsimulation(MCS).Arepeatedrandomsamplingfromthedistributionofvaluesforeachoftheparametersinagenericexposureorriskequationtoderiveanestimateofthedistributionofexposuresorrisksinthepopulation(USEPA2006b).
One‐dimensionalMonteCarloanalysis(1‐DMCA).Anumericalmethodofsimulatingadistributionforanendpointofconcernasafunctionofprobabilitydistributionsthatcharacterizevariabilityoruncertainty.Distributionsusedtocharacterizevariabilityaredistinguishedfromdistributionsusedtocharacterizeuncertainty(WHO2008).
Parameter.Aquantityusedtocalibrateorspecifyamodel,suchas‘parameters’ofaprobabilitymodel(e.g.,meanandstandarddeviationforanormaldistribution).Parametervaluesoftenareselectedbyfittingamodeltoacalibrationdataset(WHO2008).
Probability.Afrequentistapproachconsidersthefrequencywithwhichsamplesareobtainedwithinaspecifiedrangeorforaspecifiedcategory(e.g.,theprobabilitythatanaverageindividualwithaparticularmeandosewilldevelopanillness)(WHO2008).
Probabilisticriskanalysis(PRA).Calculationandexpressionofhealthrisksusingmultipleriskdescriptorstoprovidethelikelihoodofvariousrisklevels.Probabilisticriskresultsapproximateafullrangeofpossibleoutcomesandthelikelihoodofeach,whichoftenispresentedasafrequencydistributiongraph,thusallowinguncertaintyorvariabilitytobeexpressedquantitatively(USEPA2012).
Problemformulation.Theinitialstageofariskassessmentwherethepurposeoftheassessmentisarticulated,exposureandriskscenariosareconsidered,aconceptualmodelisdeveloped,andaplanforanalyzingandcharacterizingriskisdetermined(USEPA2004a).
Referenceconcentration(RfC).Anestimate(withuncertaintyspanningapproximatelyanorderofmagnitude)ofacontinuousinhalationexposuretothehumanpopulation(includingsensitivesubgroups)thatislikelytobewithoutanappreciableriskofdeleteriouseffectsduringalifetime.ItcanbederivedfromaNo‐Observed‐Adverse‐EffectLevel(NOAEL),Lowest‐Observed‐Adverse‐EffectLevel(LOAEL),orbenchmarkconcentration,withuncertaintyfactorsgenerallyappliedtoreflectlimitationsofthedataused.ItisgenerallyusedinEPA’snoncancerhealthassessments(USEPA2007a).
Referencedose(RfD).Anestimate(withuncertaintyspanningapproximatelyanorderofmagnitude)ofadailyoralexposuretothehumanpopulation(includingsensitivesubgroups)thatislikelytobewithoutanappreciableriskofdeleteriouseffectsduringalifetime.ItcanbederivedfromaNOAEL,LOAELorbenchmarkdose,withuncertaintyfactorsgenerallyappliedtoreflectlimitationsofthedataused.ItistypicallyusedinEPA’snoncancerhealthassessments(USEPA2011c).
Risk.1.Riskincludesconsiderationofexposuretothepossibilityofanadverseoutcome,thefrequencywithwhichoneormoretypesofadverseoutcomesmayoccur,andtheseverityor
40
consequencesoftheadverseoutcomesifsuchoccur.2.Thepotentialforrealizationofunwanted,adverseconsequencestohumanlife,health,propertyortheenvironment.3.Theprobabilityofadverseeffectsresultingfromexposuretoanenvironmentalagentormixtureofagents.4.Thecombinedanswersto:Whatcangowrong?Howlikelyisit?Whataretheconsequences?(USEPA2011c).
Riskanalysis.Aprocessforidentifying,characterizing,controllingandcommunicatingrisksinsituationswhereanorganism,system,subpopulationorpopulationcouldbeexposedtoahazard.Riskanalysisisaprocessthatincludesriskassessment,riskmanagementandriskcommunication(WHO2008).
Riskassessment.1.Aprocessintendedtocalculateorestimatetherisktoagiventargetorganism,system,subpopulationorpopulation,includingtheidentificationofattendantuncertaintiesfollowingexposuretoaparticularagent,takingintoaccounttheinherentcharacteristicsoftheagentofconcern,aswellasthecharacteristicsofthespecifictargetsystem(WHO2008).2.Theevaluationofscientificinformationonthehazardouspropertiesofenvironmentalagents(hazardcharacterization),thedose‐responserelationship(dose‐responseassessment),andtheextentofhumanexposuretothoseagents(exposureassessment)(NRC1983).Theproductoftheriskassessmentisastatementregardingtheprobabilitythatpopulationsorindividualssoexposedwillbeharmedandtowhatdegree(riskcharacterization;USEPA2000a).3.Qualitativeandquantitativeevaluationoftheriskposedtohumanhealthortheenvironmentbytheactualorpotentialpresenceoruseofspecificpollutants(USEPA2012).
Risk‐baseddecisionmaking.Aprocessthroughwhichdecisionsaremadeaccordingtotheriskeachposedtohumanhealthandtheenvironment(USEPA2012).
Riskmanagement.Adecision‐makingprocessthattakesintoaccountenvironmentallaws;regulations;andpolitical,social,economic,engineeringandscientificinformation,includingariskassessment,toweighpolicyalternativesassociatedwithahazard(USEPA2011c).
Scenario.Asetoffacts,assumptionsandinferencesabouthowexposuretakesplacethataidstheexposureassessorinevaluating,estimatingorquantifyingexposures(USEPA1992).Scenariosmightincludeidentificationofpollutants,pathways,exposureroutesandmodesofaction,amongothers.
Sensitivityanalysis.Theprocessofchangingonevariablewhileleavingtheothersconstanttodetermineitseffectontheoutput.Thisprocedurefixeseachuncertainquantityatitscrediblelowerandupperbounds(holdingallothersattheirnominalvalues,suchasmedians)andcomputestheresultsofeachcombinationofvalues.Theresultshelptoidentifythevariablesthathavethegreatesteffectonexposureestimatesandhelpfocusfurtherinformation‐gatheringefforts(USEPA2011b).
Tieredapproach.Referstovarioushierarchicaltiers(levels)ofcomplexityandrefinementfordifferenttypesofmodelingapproachesthatcanbeusedinriskassessment.Adeterministicriskassessmentwithconservativeassumptionsisanexampleofalowerleveltypeofanalysis(Tier0)thatcanbeusedtodeterminewhetherexposuresandrisksarebelowlevelsofconcern.Examplesofprogressivelyhigherlevelsincludetheuseofdeterministicriskassessmentcoupledwithsensitivityanalysis(Tier1),theuseofprobabilistictechniquestocharacterizeeithervariabilityoruncertaintyonly(Tier2),andtheuseoftwo‐dimensionalprobabilistictechniquestodistinguishbetweenbutsimultaneouslycharacterizebothvariabilityanduncertainty(Tier3)(USEPA2004aandWHO2008).
Two‐dimensionalMonteCarloanalysis(2‐DMCA).Anadvancednumericalmodelingtechniquethatusestwostagesofrandomsampling,alsocallednestedloops,todistinguishbetween
41
variabilityanduncertaintyinexposureandtoxicityvariables.Thefirststage,oftencalledtheinnerloop,involvesacomplete1‐DMCAsimulationofvariabilityinrisk.Inthesecondstage,oftencalledtheouterloop,parametersoftheprobabilitydistributionsareredefinedtoreflectuncertainty.Theseloopsarerepeatedmanytimesresultinginmultipleriskdistributions,fromwhichconfidenceintervalsarecalculatedtorepresentuncertaintyinthepopulationdistributionofrisk(WHO2008).
Uncertainty.Uncertaintyoccursbecauseofalackofknowledge.Itisnotthesameasvariability.Forexample,ariskassessormaybeverycertainthatdifferentpeopledrinkdifferentamountsofwaterbutmaybeuncertainabouthowmuchvariabilitythereisinwaterintakeswithinthepopulation.Uncertaintyoftencanbereducedbycollectingmoreandbetterdata,whereasvariabilityisaninherentpropertyofthepopulationbeingevaluated.Variabilitycanbebettercharacterizedwithmoredatabutitcannotbereducedoreliminated.Effortstoclearlydistinguishbetweenvariabilityanduncertaintyareimportantforbothriskassessmentandriskcharacterization,althoughtheybothmaybeincorporatedintoanassessment(USEPA2011c).
Uncertaintyanalysis.Adetailedexaminationofthesystematicandrandomerrorsofameasurementorestimate;ananalyticalprocesstoprovideinformationregardinguncertainty(USEPA2006b).
Valueofinformation.Ananalysisthatinvolvesestimatingthevaluethatnewinformationcanhavetoariskmanagerbeforetheinformationisactuallyobtained.Itisameasureoftheimportanceofuncertaintyintermsoftheexpectedimprovementinariskmanagementdecisionthatmightcomefrombetterinformation(USEPA2001).
Variability.Referstotrueheterogeneityordiversity,asexemplifiedinnaturalvariation.Forexample,amongapopulationthatdrinkswaterfromthesamesourceandwiththesamecontaminantconcentration,therisksfromconsumingthewatermayvary.Thismayresultfromdifferencesinexposure(e.g.,differentpeopledrinkingdifferentamountsofwaterandhavingdifferentbodyweights,exposurefrequenciesandexposuredurations),aswellasdifferencesinresponse(e.g.,geneticdifferencesinresistancetoachemicaldose).Thoseinherentdifferencesarereferredtoasvariability.Differencesamongindividualsinapopulationarereferredtoasinter‐individualvariability,anddifferencesforoneindividualovertimearereferredtoasintra‐individualvariability(USEPA2011c).
42
REFERENCES Ang,A.H‐S.,andW.H.Tang.1984.ProbabilityConceptsinEngineeringPlanningandDesign,Volume2:Decision,Risk,andReliability.NewYork:JohnWiley&Sons.
Babendreier,J.E.,andK.J.Castleton.2005.“InvestigatingUncertaintyandSensitivityinIntegrated,MultimediaEnvironmentalModels:ToolsforFRAMES‐3MRA.”EnvironmentalModeling&Software20:1043–1055.
Bloom,D.L.,D.M.Byrne,andJ.M.Andreson.1993.CommunicatingRisktoSeniorEPAPolicy‐Makers:AFocusGroupStudy.PreparedbyBloomResearchandtheOfficeofAirQualityPlanningandStandards.ResearchTrianglePark,NC:USEPA.
Burnham,K.P.,andD.R.Anderson.2002.ModelSelectionandInference:APracticalInformation‐TheoreticApproach.2nded.NewYork:Springer‐Verlag.
Clyde,M.,P.Guttorp,andE.Sullivan.2000.EffectsofAmbientFineandCoarseParticlesonMortalityinPhoenix,Arizona.NationalResearchCenterforStatisticsandtheEnvironmentTechnicalReportSeries,No.40.
Cooke,R.M.1991.ExpertsinUncertainty:OpinionandSubjectiveProbabilityinScience.NewYork:OxfordUniversityPress.
Costanza,R.,S.O.Funtowicz,andJ.R.Ravetz.1997.“AssessingandCommunicatingDataQualityinPolicy‐RelevantResearch.”InFrontiersinEcologicalEconomics:TransdisciplinaryEssays,byRobertCostanza,283–308.Cheltenham,UK:E.Elgar.
Cullen,A.C.,andH.C.Frey.1999.ProbabilisticExposureAssessment:AHandbookforDealingwithVariabilityandUncertaintyinModelsandInputs.NewYork:PlenumPress.
Evans,J.S.,J.D.Graham,G.M.Gray,andR.L.Sielken.1994.“ADistributionalApproachtoCharacterizingLow‐DoseCancerRisk.”RiskAnalysis14(1):25–34.
Ferson,S.,R.Nelsen,J.Hajagos,D.Berleant,J.Zhang,W.T.Tucker,L.Ginzburg,andW.L.Oberkampf.2004.DependenceinProbabilisticModeling,Dempster‐ShaferTheory,andProbabilityBoundsAnalysis.SAND20043072.Albuquerque,NM:SandiaNationalLaboratories.www.ramas.com/depend.pdf.
Ferson,S.,W.L.Oberkampt,andL.Ginzburg.2009.“ValidationofImpreciseProbabilityModels.”InternationalJournalofReliabilityandSafety3(1–3):3–22.
Finkel,A.M.1989.“IsRiskAssessmentReallyTooConservative?:RevisingtheRevisionists.”JournalofEnvironmentalLaw14:427–67.
Fisher,L.,P.Pascual,andW.Wagner.2006.MappingtheRoleofModelsinU.S.andE.U.RiskRegulation:ALegalBackgroundPaper.Washington,D.C.:TheAssociationofUnionConstructorsWorkshop.
Fischhoff,B.1995.“RankingRisks.”Risk:Health,Safety&Environment6:191–202.
Frey,H.C.,andS.R.Patil.2002.“IdentificationandReviewofSensitivityAnalysisMethods.”RiskAnalysis22(3):553–78.
Fuentes,M.andK.Foley.2012.“EnsembleModels.”InAbdel,H.andWalterW.El‐Shaarawai(eds.)EncyclopediaofEnvironmetrics.Indianapolis:JohnWileyandSons.
Gelman,A.,J.B.Carlin,H.S.Stern,andD.B.Rubin.2004.BayesianDataAnalysis.2nded.BocaRaton,FL:ChapmanandHall/CRC.
43
Hattis,D.,S.Baird,andR.Goble.2002.“AStrawManProposalforaQuantitativeDefinitionoftheRfD.”DrugandChemicalToxicology25(4):403–36.
Hattis,D.B.,andD.E.Burmaster.1994.“AssessmentofVariabilityandUncertaintyDistributionsforPracticalRiskAnalyses.”RiskAnalysis14(5):713–30.
Hattis,D.,andM.K.Lynch.2010.AlternativestoPollutant‐by‐PollutantDose‐ResponseEstimationforAirToxics(FinalReport).EPA‐W‐05‐022,WA3‐80.Washington,D.C.:USEPA.
Hoeting,J.A.,D.Madigan,A.E.Raftery,andC.T.Volinsky.1999.“BayesianModelAveraging:ATutorial.”StatisticalScience14(4):382–417.
IEC(IndustrialEconomics,Inc.).2006.ExpandedExpertJudgmentAssessmentoftheConcentration‐ResponseRelationshipBetweenPM2.5ExposureandMortality(FinalReport).ResearchTrianglePark,NC:USEPA.http://www.epa.gov/ttn/ecas/regdata/Uncertainty/pm_ee_report.pdf.
InstituteofMedicine.2013.EnvironmentalDecisionsintheFaceofUncertainty.Washington,D.C.:TheNationalAcademiesPress.
Illing,H.P.A.1999.“AreSocietalJudgmentsBeingIncorporatedIntotheUncertaintyFactorsUsedinToxicologicalRiskAssessment?”RegulatoryToxicologyandPharmacology29(3):300–308.
Krupnick,A.,R.Morgenstern,M.Batz,P.Nelson,D.Burtraw,J.Shih,andM.McWilliams.2006.NotaSureThing:MakingRegulatoryChoicesUnderUncertainty.Washington,D.C.:ResourcesfortheFuture.http://www.rff.org/rff/Documents/RFF‐Rpt‐RegulatoryChoices.pdf.
Kunreuther,H.C.,andP.Slovic.1996.“Science,ValuesandRisk.”AnnalsoftheAmericanAcademyofPoliticalandSocialScience545:116–25.
Lester,R.R.,L.C.Green,andI.Linkov.2007.“Site‐SpecificApplicationsofProbabilisticHealthRiskAssessment:ReviewoftheLiteratureSince2000.”RiskAnalysis27:635–58.
McConnell,K.E.1997.“UsingCost‐BenefitAnalysisintheManagementofContaminatedSediments(AppendixE).”InContaminatedSedimentsinPortsandWaterways:CleanupStrategiesandTechnologies,bytheCommissiononEngineeringandTechnicalSystemsoftheNationalResearchCouncil,239–56.Washington,D.C.:TheNationalAcademiesPress.
Mokhtari,A.,H.C.Frey,andJ.Zheng.2006.“EvaluationandRecommendationofSensitivityAnalysisMethodsforApplicationtoStochasticHumanExposureandDoseSimulation(SHEDS)Models.”JournalofExposureScienceandEnvironmentalEpidemiology16(6):491–506.
Morgan,G.,H.Dowlatabadi,M.Henrion,D.Keith,R.Lempert,S.McBrid,M.Small,andT.Wilbanks,eds.2009.BestPracticeApproachesforCharacterizing,Communicating,andIncorporatingScientificUncertaintyinDecisionMaking.FinalReport.Washington,D.C.:U.S.ClimateChangeScienceProgram,NationalOceanicandAtmosphericAdministration.http://library.globalchange.gov/sap‐5‐2‐best‐practice‐approaches‐for‐characterizing‐communicating‐and‐incorporating‐scientific‐uncertainty‐in‐decisionmaking.
Morgan,M.G.,andM.Henrion.1990.Uncertainty:AGuidetoDealingwithUncertaintyinQuantitativeRiskandPolicyAnalysis.NewYork:CambridgeUniversityPress.
NRC(NationalResearchCouncil).1983.RiskAssessmentintheFederalGovernment:ManagingtheProcess.CommitteeontheInstitutionalMeansforAssessmentofRiskstoPublicHealth.Washington,D.C.:TheNationalAcademiesPress.
NRC.1989.RiskAssessmentintheFederalGovernment:ManagingtheProcess.Washington,D.C.:TheNationalAcademiesPress.
44
NRC.1991.RethinkingtheOzoneProbleminUrbanandRegionalAirPollution.Washington,D.C.:TheNationalAcademiesPress.
NRC.1993.IssuesinRiskAssessment.Washington,D.C.:TheNationalAcademiesPress.
NRC.1994.ScienceandJudgmentinRiskAssessment.Washington,D.C.:TheNationalAcademiesPress.
NRC.2002.EstimatingthePublicHealthBenefitsofProposedAirPollutionRegulations.Washington,D.C.:TheNationalAcademiesPress.
NRC.2007a.ScientificReviewoftheProposedRiskAssessmentBulletinFromtheOfficeofManagementandBudget.Washington,D.C.:TheNationalAcademiesPress.
NRC.2007b.ModelsinEnvironmentalRegulatoryDecisionMaking.Washington,D.C.:TheNationalAcademiesPress.
NRC.2009.ScienceandDecisions:AdvancingRiskAssessment.Washington,D.C.:TheNationalAcademiesPress.
OSTP(OfficeofScienceandTechnologyPolicy)/OMB(OfficeofManagementandBudget).2007.Memorandum(M‐07‐24)fortheHeadsofExecutiveDepartmentsandAgencies;fromSusanE.Dudley,Administrator,OfficeofInformationandRegulatoryAffairs,OfficeofManagementandBudget,toSharonL.Hays,AssociateDirectorandDeputyDirectorforScience,OfficeofScienceandTechnologyPolicy:UpdatedPrinciplesforRiskAnalysis.September19,2007.
ÖzkaynakH.,H.C.Frey,J.Burke,andR.W.Pinder.2009.“AnalysisofCoupledModelUncertaintiesinSource‐to‐DoseModelingofHumanExposurestoAmbientAirPollution:APM2.5Case‐Study.”AtmosphericEnvironment43:1641–49.
Parkin,R.T.,andM.G.Morgan.2007.ConsultationonEnhancingRiskAssessmentPracticesandUpdatingEPA’sExposureGuidelines.EPA/SAB‐07/003.Washington,D.C.:USEPA.USEPA.2007.LetterFromM.G.MorganandR.T.Parkin,ScienceAdvisoryBoard,toS.Johnson,U.S.EnvironmentalProtectionAgency.February28,2007.EPA/SAB‐07/003.http://yosemite.epa.gov/sab/sabproduct.nsf/55E1B2C78C6085EB8525729C00573A3E/$File/sab‐07‐003.pdf.
PCCRARM(Presidential/CongressionalCommissiononRiskAssessmentandRiskManagement).1997.FrameworkforEnvironmentalHealthRiskManagementFinalReport,Volume1.Washington,D.C.:PCCRARM.http://www.riskworld.com/nreports/1997/risk‐rpt/pdf/EPAJAN.PDF.
Pearl,J.2009.Causality:Models,Reasoning,andInference.2nded.NewYork:CambridgeUniversityPress.
Royall,R.M.1997.StatisticalEvidence:ALikelihoodParadigm.London:Chapman&Hall.
Saltelli,A.,S.Tarantola,F.Campolongo,andM.Rattoet.2004.SensitivityAnalysisinPractice.NewYork:JohnWiley&Sons.
Stahl,C.H.,andA.J.Cimorelli.2005.“HowMuchUncertaintyIsTooMuchandHowDoWeKnow?ACaseExampleoftheAssessmentofOzoneMonitorNetworkOptions.”RiskAnalysis25(5):1109–20.
Stahl,C.H.,andA.J.Cimorelli.2012.“TheDemonstrationoftheNecessityandFeasibilityofUsingaClumsyDecisionAnalyticApproachinWickedEnvironmentalProblems.”IntegratedEnvironmentalAssessmentandManagement9(1):17.
45
Stahl,C.H.,A.J.Cimorelli,C.Mazzarella,andB.Jenkins.2011.“TowardSustainability:ACaseStudyDemonstratingTransdisciplinaryLearning.”IntegratedEnvironmentalAssessmentandManagement7(3):483–98.
Swartout,J.C.,P.S.Price,M.L.Dourson,H.L.Carlson‐Lynch,andR.E.Keenan.1998.“ProbabilisticFrameworkfortheReferenceDose(ProbabilisticRfD).”RiskAnalysis18(3):271–82.
Toll,J.,S.Pavlou,D.Lee,L.Zaragosa,andP.Shelley.1997.“UsingDecisionAnalysisintheManagementofContaminatedSediments(AppendixE).”InContaminatedSedimentsinPortsandWaterways:CleanupStrategiesandTechnologies,bytheCommissiononEngineeringandTechnicalSystemsoftheNationalResearchCouncil,257–84.Washington,D.C.:TheNationalAcademiesPress.
Toll,J.E.1999.“ElementsofEnvironmentalProblem‐Solving.”HumanandEcologicalRiskAssessment5(2):275–80.
USEPA(U.S.EnvironmentalProtectionAgency).1992.GuidelinesforExposureAssessment.EPA/600/Z‐92/001.Washington,D.C.:RiskAssessmentForum,USEPA.http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=15263.
USEPA.1995a.GuidanceforRiskCharacterization.Washington,D.C.:SciencePolicyCouncil,USEPA.http://www.epa.gov/spc/pdfs/rcguide.pdf.
USEPA.1995b.PolicyforRiskCharacterization.Washington,D.C.:USEPA.http://www.epa.gov/oswer/riskassessment/pdf/1995_0521_risk_characterization_program.pdf.
USEPA.1997a.PolicyforUseofProbabilisticAnalysisinRiskAssessmentattheU.S.EnvironmentalProtectionAgency.Washington,D.C.:USEPA.http://www.epa.gov/spc/pdfs/probpol.pdf.
USEPA.1997b.GuidingPrinciplesforMonteCarloAnalysis.EPA/630/R‐97/001.Washington,D.C.:USEPA.http://www.epa.gov/raf/publications/pdfs/montecar.pdf.
USEPA.1998a.GuidanceforSubmissionofProbabilisticHumanHealthExposureAssessmentstotheOfficeofPesticidePrograms(Draft).Washington,D.C.:USEPA.http://www.epa.gov/fedrgstr/EPA‐PEST/1998/November/Day‐05/6021.pdf.
USEPA.1998b.GuidelinesforEcologicalRiskAssessment.EPA/630/R‐95/002F.Washington,D.C.:RiskAssessmentForum,USEPA.http://www.epa.gov/raf/publications/pdfs/ECOTXTBX.PDF.
USEPA.2000a.EPASciencePolicyCouncilHandbook:RiskCharacterization.EPA/100/B‐00/002.Washington,D.C.:USEPA.http://www.epa.gov/spc/pdfs/rchandbk.pdf.
USEPA.2000b.TowardIntegratingEnvironmentalDecision‐Making.EPA/SAB/EC‐00/011.Washington,D.C.:ScienceAdvisoryBoard,USEPA.
USEPA.2001.RiskAssessmentGuidanceforSuperfund:VolumeIII—PartA,ProcessforConductingProbabilisticRiskAssessment.EPA/540/R‐02/002.Washington,D.C.:USEPA.http://www.epa.gov/oswer/riskassessment/rags3adt/pdf/rags3adt_complete.pdf.
USEPA.2004a.AirToxicsRiskAssessmentReferenceLibrary.VolumeI:TechnicalResourceManual.EPA/453/K‐04/001A.ResearchTrianglePark,NC:USEPA.http://www.epa.gov/ttn/fera/data/risk/vol_1/title_page_volume_1.pdf.
USEPA.2004b.AnExaminationofEPARiskAssessmentPrinciplesandPractices.EPA/100/B‐04/001.Washington,D.C.:USEPA.http://www.epa.gov/osa/pdfs/ratf‐final.pdf.
USEPA.2006a.AFrameworkforAssessingHealthRisksofEnvironmentalExposurestoChildren.EPA/600/R‐05/093F.Washington,D.C.:USEPA.http://ofmpub.epa.gov/eims/eimscomm.getfile?p_download_id=459047.
46
USEPA.2006b.AirToxicsRiskAssessmentReferenceLibrary,Volume3:Community‐ScaleAssessment.EPA/452/K‐06/001C.Washington,D.C.:USEPA.http://www.epa.gov/ttn/fera/risk_atra_vol3.html.
USEPA.2006c.HazardAnalysisCriticalControlPointStrategiesforDistributionSystemMonitoring,HazardAssessmentandControl.Washington,D.C.:USEPA.http://www.epa.gov/ogwdw/disinfection/tcr/pdfs/issuepaper_tcr_haccp‐strategies.pdf.
USEPA.2007a.Concepts,MethodsandDataSourcesforCumulativeHealthRiskAssessmentofMultipleChemicals,ExposuresandEffects:AResourceDocument.EPA/600/R‐06/013F.Washington,D.C.:USEPA.http://cfpub.epa.gov/ncea/risk/recordisplay.cfm?deid=190187.
USEPA.2007b.LetterFromM.G.MorganandR.T.Parkin,ScienceAdvisoryBoard,toS.Johnson,U.S.EnvironmentalProtectionAgency.February28,2007.EPA/SAB‐07/003.http://yosemite.epa.gov/sab/sabproduct.nsf/55E1B2C78C6085EB8525729C00573A3E/$File/sab‐07‐003.pdf.
USEPA.2010a.EnvironmentalMonitoringandAssessmentProgramMasterGlossary.LastmodifiedNovember8.http://www.epa.gov/emap
2/html/pubs/docs/resdocs/mglossary.html#ee.
USEPA.2010b.HumanHealthRiskAssessmentOnondagaLake,LakeBottomSubsite:SedimentConsolidationArea,Camillus,NY.Washington,D.C.:USEPA.http://www.epa.gov/region2/superfund/npl/onondagalake/healthriskassess.pdf.
USEPA.2011a.ExpertElicitationTaskForceWhitePaper.Washington,D.C.:USEPA.http://www.epa.gov/stpc/pdfs/ee‐white‐paper‐final.pdf.
USEPA.2011b.ExposureFactorsHandbook:2011Edition.EPA/600/R‐09/052F.Washington,D.C.:NationalCenterforEnvironmentalAssessment,OfficeofResearchandDevelopment,USEPA.http://cfpub.epa.gov/ncea/risk/recordisplay.cfm?deid=236252.
USEPA.2011c.IntegratedRiskInformationSystemGlossary.LastmodifiedAugust21.http://ofmpub.epa.gov/sor_internet/registry/termreg/searchandretrieve/glossariesandkeywordlists/search.do?details=&glossaryName=IRIS%20Glossary.
USEPA.2012.WasteandCleanupRiskAssessmentGlossary.OfficeofSolidWasteandEmergencyResponse,USEPA.LastmodifiedDecember24.http://www.epa.gov/oswer/riskassessment/glossary.htm.
USEPA.2014a.ProbabilisticRiskAssessmenttoInformDecisionMaking:FrequentlyAskedQuestions.EPA/100/R‐09/001B.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.
USEPA.2014b.FrameworkforHumanHealthRiskAssessmenttoInformDecisionMaking.EPA/100/R‐14/001.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.
Wilson,J.D.2000.“RiskManagementCannotAbideUncertainty.”InJ.H.ExnerandM.L.Trehy(eds.)SymposiapaperspresentedbeforetheDivisionofEnvironmentalChemistry,AmericanChemicalSociety,August20–24,2000.PreprintsofExtendedAbstracts40(2):268–69.Washington,D.C.:AmericanChemicalSociety.
WHO(WorldHealthOrganization).2004.InternationalProgrammeonChemicalSafetyRiskAssessmentTerminology.Geneva:WHO.http://www.who.int/ipcs/methods/harmonization/areas/ipcsterminologyparts1and2.pdf.
47
WHO.2008.UncertaintyandDataQualityinExposureAssessment.Geneva:WHO.http://www.who.int/ipcs/publications/methods/harmonization/exposure_assessment.pdf.
Wright,G.,F.Bolger,andG.Rowe.2002.“AnEmpiricalTestoftheRelativeValidityofExpertandLayJudgmentsofRisk.”RiskAnalysis22(6):1107–22.
48
BIBLIOGRAPHY
Probabilistic Risk Analysis MethodologyGeneral Cullen,A.C.,andH.C.Frey.1999.TheUseofProbabilisticTechniquesinExposureAssessment:AHandbookforDealingwithVariabilityandUncertaintyinModelsandInputs.NewYork:Plenum.
Dakins,M.E.,J.E.Toll,M.J.Small,andK.P.Brand.1996.“Risk‐BasedEnvironmentalRemediation:BayesianMonteCarloAnalysisandtheExpectedValueofSampleInformation.”RiskAnalysis16(1):67–79.
Ferson,S.,andV.Kreinovich.2006.ModelingCorrelationandDependenceAmongIntervals.UniversityofTexasatElPasoDepartmentalTechnicalReports(CS),Paper131.http://digitalcommons.utep.edu/cs_techrep/131.
Finkel,A.M.1990.ConfrontingUncertaintyinRiskManagement:AGuideforDecision‐Makers.Washington,D.C.:ResourcesfortheFuture.
Frey,H.C.,D.Crawford‐Brown,J.Zheng,andD.Loughlin.2003.HierarchyofMethodstoCharacterizeUncertainty:StateofScienceofMethodsforDescribingandQuantifyingUncertainty(Draft).ResearchTrianglePark,NC:USEPA.
Frey,C.,J.Penman,L.Hanle,S.Monni,andS.Ogle.2006.Chapter3,“Uncertainties.”InS.Eggleston,L.Buendia,K.Miwa,T.Ngara,andK.Tanabe(eds.)2006IPCCGuidelinesforNationalGreenhouseGasInventories,VolumeI:GeneralGuidanceandReporting.Kanagawa,Japan:NationalGreenhouseGasInventoriesProgramme,Inter‐GovernmentalPanelonClimateChange.
Hattis,D.1996.“HumanInterindividualVariabilityinSusceptibilitytoToxicEffects—FromAnnoyingDetailtoaCentralDeterminantofRisk.”Toxicology111:5–14.
Hattis,D.,andE.Anderson.1999.“WhatShouldBetheImplicationsofUncertainty,Variability,andInherent‘Biases’/‘Conservatism’forRiskManagementDecisionMaking?”RiskAnalysis19:95–107.
Hattis,D.,andM.K.Lynch.2007.“EmpiricallyObservedDistributionsofPharmacokineticandPharmacodynamicVariabilityinHumans—ImplicationsfortheDerivationofSinglePointComponentUncertaintyFactorsProvidingEquivalentProtectionasExistingRfDs.”InJ.C.LipscombandE.V.Ohanian(eds.)ToxicokineticsinRiskAssessment,69–93.NewYork:InformaHealthcareUSA,Inc.
Hattis,D.2009.“FailuretoCommunicate—Comment:Regulator/RiskPerspectiveonBurzalaandMazzuchi’sPaper.”InR.M.Cooke(ed.).UncertaintyModelinginDoseResponse:BenchTestingEnvironmentalToxicity,153–59.NewYork:JohnWiley&Sons,Inc.
Loughlin,D.,H.C.Frey,K.Hanisak,andA.Eyth.2003.ImplementationRequirementsfortheDevelopmentofaSensitivity/UncertaintyAnalysisToolforMIMS(Draft).ResearchTrianglePark,NC:USEPA.
Morgan,M.G.,andM.Henrion.1990.Uncertainty:AGuidetoDealingwithUncertaintyinQuantitativeRiskandPolicyAnalysis.NewYork:CambridgeUniversityPress.
NARSTO(NorthAmericanResearchStrategyforTroposphericOzone).2005.ImprovingEmissionInventoriesforEffectiveAirQualityManagementAcrossNorthAmerica,ANARSTOAssessment.NARSTO‐05‐001.Pasco,WA:NARSTO.
NRC(NationalResearchCouncil).1994.ScienceandJudgmentinRiskAssessment.Washington,D.C.:TheNationalAcademiesPress.
49
USEPA(U.S.EnvironmentalProtectionAgency).1997.GuidingPrinciplesforMonteCarloAnalysis.EPA/630/R‐97/001.Washington,D.C.:USEPA.http://www.epa.gov/raf/publications/pdfs/montecar.pdf.
Probabilistic Risk Analysis and Decision Making Bernardini,A.,andF.Tonon.2010.BoundingUncertaintyinCivilEngineering:TheoreticalBackground.NewYork:Springer.
Bloom,D.L.,D.M.Byrne,andJ.M.Andreson.1993.CommunicatingRisktoSeniorEPAPolicy‐Makers:AFocusGroupStudy.ResearchTrianglePark,NC:USEPA.
Krupnick,A.,R.Morgenstern,M.Batz,P.Nelson,D.Burtraw,J.‐S.Shih,andM.McWilliams.2006.NotaSureThing:MakingRegulatoryChoicesUnderUncertainty.Washington,D.C.:ResourcesfortheFuture.
McGarity,T.O.,andW.E.Wagner.2003.“LegalAspectsoftheRegulatoryUseofEnvironmentalModeling.”EnvironmentalLawReporter33:10751–74.
NRC.1996.UnderstandingRisk:InformingDecisionsinaDemocraticSociety.Washington,D.C.:TheNationalAcademiesPress.
NRC.2009.ScienceandDecisions:AdvancingRiskAssessment.Washington,D.C.:TheNationalAcademiesPress.
Pascual,P.,N.Steiber,andE.Sunderland.2003.DraftGuidanceontheDevelopment,Evaluation,andApplicationofRegulatoryEnvironmentalModels.Washington,D.C.:USEPA.
Thompson,K.M.,andJ.D.Graham.1996.“GoingBeyondtheSingleNumber:UsingProbabilisticRiskAssessmenttoImproveRiskManagement.”HumanandEcologicalRiskAssessment2(4):1008–34.
USEPA.1997.MemofromFredHansen.UseofProbabilisticTechniques(includingMonteCarloAnalysis)inRiskAssessment,datedMay15,1997.Washington,D.C.:USEPA.
Vick,S.G.2002.DegreesofBelief:SubjectiveProbabilityandEngineeringJudgment.Reston,VA:ASCEPress.
Probabilistic Risk Analysis MethodologySpecific Aspects Abdel‐Aziz,A.,andH.C.Frey.2003.“DevelopmentofHourlyProbabilisticUtilityNOxEmissionInventoriesUsingTimeSeriesTechniques:PartI—UnivariateApproach.”AtmosphericEnvironment37:5379–89.
Abdel‐Aziz,A.,andH.C.Frey.2003.“DevelopmentofHourlyProbabilisticUtilityNOxEmissionInventoriesUsingTimeSeriesTechniques:PartII—MultivariateApproach.”AtmosphericEnvironment37:5391–401.
Abdel‐Aziz,A.,andH.C.Frey.2004.“PropagationofUncertaintyinHourlyUtilityNOxEmissionsThroughaPhotochemicalGridAirQualityModel:ACaseStudyfortheCharlotte,NCModelingDomain.”EnvironmentalScience&Technology38(7):2153–60.
Casman,E.A.,M.G.Morgan,andH.Dowlatabadi.1999.“MixedLevelsofUncertaintyinComplexPolicyModels.”RiskAnalysis19(1):33–42.
50
Crawford‐Brown,D.J.,andK.G.Brown.1987.“AFrameworkforAssessingtheRationalityofJudgmentsinCarcinogenicityHazardIdentification.”Risk:Health,SafetyandEnvironment8:307–37.
Draper,D.1995.“AssessmentandPropagationofModelUncertainty.”JournaloftheRoyalStatisticalSociety57:45–97.
Evans,J.S.,J.D.Graham,G.M.Gray,andR.L.Sielken.1994.“ADistributionalApproachtoCharacterizingLow‐DoseCancerRisk.”RiskAnalysis14(1):25–34.
Frey,H.C.,andD.E.Burmaster.1999.“MethodsforCharacterizingVariabilityandUncertainty:ComparisonofBootstrapSimulationandLikelihood‐BasedApproaches.”RiskAnalysis19(1):109–30.
Frey,H.C.,andD.S.Rhodes.1996.“Characterizing,Simulating,andAnalyzingVariabilityandUncertainty:AnIllustrationofMethodsUsinganAirToxicsEmissionsExample.”HumanandEcologicalRiskAssessment:AnInternationalJournal2(4):762–97.
Frey,H.C.,andY.Zhao.2004.“QuantificationofVariabilityandUncertaintyforAirToxicEmissionInventoriesWithCensoredEmissionFactorData.”EnvironmentalScience&Technology38(22):6094–100.
Landis,W.G.2005.RegionalScaleEcologicalRiskAssessmentUsingtheRelativeRiskModel.BocaRaton,FL:CRCPress.
Maginnis,C.M.2006.AScreeningLevelIntegratedEcologicalandHumanHealthRiskAssessmentforLakeWhatcom,WhatcomCounty,Washington.Master’sThesis,WesternWashingtonUniversity.
Shlyakhter,A.I.1994.“AnImprovedFrameworkforUncertaintyAnalysis:AccountingforUnsuspectedErrors.”RiskAnalysis14:441–47.
Zheng,J.,andH.C.Frey.2004.“QuantificationofVariabilityandUncertaintyUsingMixtureDistributions:EvaluationofSampleSize,MixingWeightsandSeparationbetweenComponents.”RiskAnalysis24(3):553–71.
Zheng,J.,andH.C.Frey.2005.“QuantitativeAnalysisofVariabilityandUncertaintywithKnownMeasurementError:MethodologyandCaseStudy.”RiskAnalysis25(3):663–76.
Sensitivity Analysis Agro,K.E.,C.A.Bradley,andN.Mittmann.1997.“SensitivityAnalysisinHealthEconomicandPharmacoeconomicStudies—anAppraisaloftheLiterature.”Pharmacoeconomics11(1):75–88.
Cukier,R.I.,H.B.Levine,andK.E.Shuler.1978.“NonlinearSensitivityAnalysisofMulti‐parameterModelSystems.”JournalofChemicalPhysics26(1):1–42.
Frey,H.C.,andR.Patil.2002.“IdentificationandReviewofSensitivityAnalysisMethods.”RiskAnalysis22(3):553–77.
Frey,H.C.,A.Mokhtari,andJ.Zheng.2004.RecommendedPracticeRegardingSelection,Application,andInterpretationofSensitivityAnalysisMethodsAppliedtoFoodSafetyProcessRiskModels.Washington,D.C.:U.S.DepartmentofAgriculture.http://www.ce.ncsu.edu/risk/Phase3Final.pdf
Helton,J.C.,andF.J.Davis.2002.“IllustrationofSampling‐BasedMethodsforSensitivityAnalysis.”ReliabilityEngineering&SystemSafety81(1):23–69.
51
Mokhtari,A.,andH.C.Frey.2005.“RecommendedPracticeRegardingSelectionofSensitivityAnalysisMethodsAppliedtoMicrobialFoodSafetyProcessRiskModels.”HumanandEcologicalRiskAssessment11(3):591–605.
Mokhtari,A.,H.C.Frey,andJ.Zheng.2006.“EvaluationandRecommendationofSensitivityAnalysisMethodsforApplicationtoEPA‐StochasticHumanExposureandDoseSimulation(SHEDS)Models.”JournalofExposureScienceandEnvironmentalEpidemiology16(6):491–506.
Saltelli,A.,K.Chan,andM.Scott.2000.SensitivityAnalysis,ProbabilityandStatisticsSeries.NewYork:JohnWiley&Sons.
Sobol,I.M.1993.“SensitivityEstimatesforNonlinearMathematicalModels.”MathematicalModelingandComputation1(4):407–14.
Xue,J.,V.G.Zartarian,H.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.“AProbabilisticArsenicExposureAssessmentforChildrenWhoContactChromatedCopperArsenate(CCA)‐TreatedPlaysetsandDecks,Part2:SensitivityandUncertaintyAnalyses.”RiskAnalysis26(2):533–41.
Case Study Examples of Probabilistic Risk AnalysisEPA (See Also the PRA Case Studies Appendix) Babendreier,J.E.,andK.L.Castleton.2005.“InvestigatingUncertaintyandSensitivityinIntegratedMultimediaEnvironmentalModels:ToolsforFRAMES‐3MRA.”EnvironmentalModeling&Software20:1043–55.
Blancato,J.N.,F.W.Power,R.N.Brown,andC.C.Dary.2006.ExposureRelatedDoseEstimatingModel(ERDEM):APhysiologically‐BasedPharmacokineticandPharmacodynamic(PBPK/PD)ModelforAssessingHumanExposureandRisk.EPA/600/R‐06/061.ResearchTrianglePark,NC:USEPA.http://www.epa.gov/heasd/products/erdem/237edrb05‐Report.pdf
Burke,J.2005.EPAStochasticHumanExposureandDoseSimulationforParticulateMatter(SHEDS‐PM)User’sGuide.ResearchTrianglePark,NC:USEPA.
Frey,H.C.2003.EvaluationofanApproximateAnalyticalProcedureforCalculatingUncertaintyintheGreenhouseGasVersionoftheMulti‐ScaleMotorVehicleandEquipmentEmissionsSystem.AnnArbor,MI:USEPA.
Frey,H.C.,andY.Zhao.2003.DevelopmentofProbabilisticEmissionInventoriesofBenzene,FormaldehydeandChromiumfortheHoustonDomain.ResearchTrianglePark,NC:USEPA.
Frey,H.C.,R.Bharvirkar,andJ.Zheng.1999.QuantitativeAnalysisofVariabilityandUncertaintyinEmissionsEstimation.ResearchTrianglePark,NC:USEPA.
Frey,H.C.,J.Zheng,Y.Zhao,S.Li,andY.Zhu.2002.TechnicalDocumentationoftheAuvToolSoftwareforAnalysisofVariabilityandUncertainty.ResearchTrianglePark,NC:USEPA.
ÖzkaynakH.,H.C.Frey,J.Burke,andR.W.Pinder.2009.“AnalysisofCoupledModelUncertaintiesinSource‐to‐DoseModelingofHumanExposurestoAmbientAirPollution:APM2.5Case‐Study.”AtmosphericEnvironment43:1641–49.
Zartarian,V.G.,J.Xue,H.A.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2005.AProbabilisticExposureAssessmentforChildrenWhoContactCCA‐TreatedPlaysetsandDecksUsingtheStochasticHumanExposureandDoseSimulationModelfortheWoodPreservativeScenario(SHEDS‐WOOD),FinalReport.EPA/600/X‐05/009.Washington,D.C.:USEPA.http://www.epa.gov/heasd/sheds/CCA_all.pdf
52
Zartarian,V.G.,J.Xue,H.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.“AProbabilisticArsenicExposureAssessmentforChildrenWhoContactCCA‐TreatedPlaysetsandDecks,Part1:ModelMethodology,VariabilityResults,andModelEvaluation.”RiskAnalysis26(2):515–32.
Zheng,J.,andH.C.Frey.2002.AuvToolUser’sGuide.ResearchTrianglePark,NC:USEPA.
Case Study Examples of Probabilistic Risk AnalysisOther
Bloyd,C.,J.Camp,G.Conzelmann,J.Formento,J.Molburg,J.Shannon,M.Henrion,etal.1996.TrackingandAnalysisFramework(TAF)ModelDocumentationandUser’sGuide.ANL/DIS/TM‐36.Washington,D.C.:ArgonneNationalLaboratory.
Frey,H.C.,andS.Li.2003.“QuantificationofVariabilityandUncertaintyinAP‐42EmissionFactors:CaseStudiesforNaturalGas‐FueledEngines.”JournaloftheAir&WasteManagementAssociation53(12):1436–47.
Hanna,S.R.,Z.Lu,H.C.Frey,N.Wheeler,J.Vukovich,S.Arunachalam,M.Fernau,andD.A.Hansen.2001.“UncertaintiesinPredictedOzoneConcentrationsDuetoInputUncertaintiesfortheUAM‐VPhotochemicalGridModelAppliedtotheJuly1995OTAGDomain.”AtmosphericEnvironment35(5):891–903.
Munns,W.R.,Jr.,H.A.Walker,andJ.F.Paul.1989.“AnEcologicalRiskAssessmentFrameworkforExaminingtheImpactsofOceanicDisposalStatusandTrendsProgram.”InOceans‘89,Volume2:OceanPollution,bytheMarineTechnologySocietyandtheOceanicEngineeringSociety(U.S.),664–69.IEEEPublicationNo.89,CH2780‐5.NewYork:IEEEPress.
Munns,W.R.,Jr.,H.A.Walker,J.F.Paul,andJ.H.Gentile.1996.“AProspectiveAssessmentofEcologicalRiskstoUpperWaterColumnPopulationsfromOceanDisposalatthe106‐MileDumpsite.”JournalofMarineEnvironmentalEngineering3:279–97.
Nocito,J.A.,H.A.Walker,J.F.Paul,andC.Menzie.1988.“ApplicationofaRiskAssessmentforMarineDisposalofSewageSludgeatMidshelfandOffshelfSites.”InProceedingsofthe7thInternationalOceanDisposalSymposium,644–63.Ottawa:EnvironmentCanada.
Nocito,J.A.,H.A.Walker,J.F.Paul,andC.A.Menzie.1989.“ApplicationofaRiskAssessmentFrameworkforMarineDisposalofSewageSludgeatMidshelfandOffshelfSites.”InG.W.SuterIIandM.A.Lewis(eds.).AquaticToxicologyandEnvironmentalFate:EleventhVolume,101–20.ASTMSTP1007.Philadelphia,PA:AmericanSocietyforTestingandMaterials.
Paul,J.F.,H.A.Walker,andV.J.Bierman,Jr.1983.“ProbabilisticApproachfortheDeterminationofthePotentialAreaofInfluenceforWasteDisposedatthe106‐MileOceanDisposalSite.”InJ.B.Pearce,D.C.Miller,andC.Berman(eds.).106‐MileWasteDisposalSiteCharacterizationUpdateReport,A–1toA–8.NOAATechnicalMemorandumNMFS‐F/NEC‐26.WoodsHole,MA:NationalMarineFisheriesService.
Paul,J.F.,V.J.Bierman,Jr.,W.R.Davis,G.L.Hoffman,W.R.Munns,C.E.Pesch,P.F.Rogerson,andS.C.Schimmel.1988.“TheApplicationofaHazardAssessmentResearchStrategytotheOceanDisposalofaDredgedMaterial:ExposureAssessmentComponent.”InD.A.WolfeandT.P.O’Connor(eds.).OceanicProcessesinMarinePollution,Volume5,123–35.Melbourne,FL:KreigerPublishingCo.
Prager,J.C.,V.J.Bierman,Jr.,J.F.Paul,andJ.S.Bonner.1986.“SamplingtheOceansforPollution:ARiskAssessmentApproachtoEvaluatingLow‐LevelRadioactiveWasteDisposalatSea.”DangerousPropertiesofIndustrialMaterialsReport6(3):2–26.
53
Walker,H.A.,J.F.Paul,J.A.Nocito,andJ.H.Gentile.1988.“EcologicalandHumanHealthRisksforSewageSludgeDisposalatthe106‐MileSite.”InProceedingsofthe7thInternationalOceanDisposalSymposium,625–43.Ottawa:EnvironmentCanada.
54
APPENDIX: CASE STUDY EXAMPLES OF THE APPLICATION OF PROBABILISTIC RISK ANALYSIS IN U.S.
ENVIRONMENTAL PROTECTION AGENCY REGULATORY DECISION MAKING
55
A. OVERVIEW ThisAppendixfocusesonexamplesofhowprobabilisticriskanalysis(PRA)approacheshavebeenusedatEPAtoinformregulatorydecisions.TheAppendixwaspreparedbyrepresentativesfromvariousEPAprogramofficesandregionscurrentlyinvolvedinthedevelopmentandapplicationofPRAtechniques.TheTechnicalPanelselectedthecasestudyexamplesbasedonthemembers’knowledgeofthespecificPRAprocedures,thetypesoftechniquesdemonstrated,theavailabilitytothereaderthroughtheInternetandtheconditionofhavingbeenpeerreviewed;theyalsowereselectedtobeillustrativeofaspectrumofPRAusedatEPA.ThecasestudiesarenotdesignedtoprovideanexhaustivediscussionofthewidevarietyofapplicationsofPRAusedwithintheAgency,buttohighlightspecificexamplesreflectingtherangeofapproachescurrentlyappliedwithinEPA.
ThisAppendixisintendedtoserveasaresourceformanagersfacedwithdecisionsregardingwhentoapplyPRAtechniquestoinformenvironmentaldecisions,andforexposureandriskassessorswhomaynotbefamiliarwiththewidevarietyofavailablePRAapproaches.ThedocumentoutlinescategoriesofPRAsclassifiedbythecomplexityofanalysistoaidthedecision‐makingprocess.ThisapproachidentifiesvariousPRAtools,whichincludetechniquesrangingfromasimplesensitivityanalysis(e.g.,identificationofkeyexposureparametersordatavisualization)requiringlimitedtime,resourcesandexpertisetodevelop(Group1);toprobabilisticapproaches,includingMonteCarloanalysis,thatprovidetoolsforevaluatingvariabilityanduncertaintyseparatelyandthatrequiremoreresourcesandspecializedexpertise(Group2);tosophisticatedtechniquesofexpertelicitationthatgenerallyrequiresignificantinvestmentofemployeetime,additionalexpertiseandexternalpeerreview(Group3).
ThecasestudiesinthisAppendixusedPRAtechniqueswithinthisrankedframeworktoprovideadditionalinformationformanagers.ThecasestudysummariesareprovidedinaformatdesignedtohighlighthowtheresultsofthePRAswereconsideredindecisionmaking.Thesesummariesincludespecificinformationontheconductoftheanalysesasanaidindeterminingwhattoolsmightbeappropriatetodevelopspecificexposureorriskassessmentsforothersites.
Thecasestudiesrangefromexamplesoflessresource‐intensiveanalyses,whichmightassistinidentifyingkeyexposureparametersortheneedformoredata,tomoredetailedandresource‐intensiveapproaches.Examplesofapplicationsinhumanhealthandecologicalriskassessmentincludetheexposureofchildrentochromatedcopperarsenate(CCA)‐treatedwood,therelationbetweenparticulatesinairandhealth,dietaryexposurestopesticides,modelingsealevelchange,samplingwatersheds,andmodelingbirdandanimalexposures.
B. INTRODUCTION Historically,EPAhasuseddeterministicriskassessments,orpointestimatesofrisk,toevaluatecancerrisksandnoncancerhealthhazardstohigh‐endexposedindividuals(90thpercentileorhigher)andtheaverageexposedindividual(50thpercentile)and,whereappropriate,risksandhazardstopopulations,asrequiredbyspecificenvironmentallaws(USEPA1992a).Theuseofdefaultvaluesforexposureparametersinriskassessmentsprovidesaproceduralconsistencythatallowsriskassessmentstobefeasibleandtractable(USEPA2004).ThemethodstypicallyusedinEPAdeterministicriskassessments(DRA)relyonacombinationofpointvalues―someconservativeandsometypical―yieldingapointestimateofexposurethatisatsomeunknownpointintherangeofpossiblerisks(USEPA2004).
ThisAppendixpresentscasestudiesofPRAconductedbyEPAoverthepast10to15years.TableA‐1summarizesthecasestudiesbytitle,techniquedemonstrated,classificationasahumanhealthriskassessment(HHRA)orecologicalriskassessment(ERA),andtheprogramorregional
56
officeresponsiblefordevelopingthecasestudies.ThisAppendix,providesa“snapshot”oftheutilizationofPRAacrossvariousprogramsinEPA.
C. OVERALL APPROACH TO PROBABILISTIC RISK ANALYSIS AT THE U.S. ENVIRONMENTAL PROTECTION AGENCY
C.1. U.S. Environmental Protection Agency Guidance and Policies on Probabilistic Risk Analysis
ThecasestudiespresentedherebuildontheprinciplesofPRAoutlinedinEPA’s1997PolicyforUseofProbabilisticAnalysisinRiskAssessmentattheU.S.EnvironmentalProtectionAgency(USEPA1997a)andGuidingPrinciplesforMonteCarloAnalysis(USEPA1997b),aswellassubsequentguidancedocumentsondevelopingandusingPRA.GuidancehasbeendevelopedfortheAgencyandindividualprograms.SpecificdocumentsthatrefertotheuseofPRAincludetheRiskAssessmentGuidanceforSuperfund:VolumeIII(USEPA2001);RiskAssessmentForum(RAF)FrameworkforEcologicalRiskAssessment(USEPA1992b);GuidelinesforEcologicalRiskAssessment(USEPA1998);GuidanceforRiskCharacterization(USEPA1995a);PolicyonEvaluatingHealthRiskstoChildren(USEPA1995b);PolicyforUseofProbabilisticAnalysisinRiskAssessment(USEPA1997a);GuidanceonCumulativeRiskAssessment,Part1:PlanningandScoping(USEPA1997c);andRiskCharacterizationHandbook(USEPA2000a);andFrameworkforHumanHealthRiskAssessmenttoInformDecisionMaking(USEPA2014).
Asshownintheindividualcasestudies,therangeandscopeofthePRAwilldependontheoverallobjectivesofthedecisionthattheanalysiswillinform.TheGuidingPrinciplesforMonteCarloAnalysis(USEPA1997b)layoutthegeneralapproachthatshouldbetakeninallcases,beginningwithdefiningtheproblemandscopeoftheassessmenttoselectingthebesttoolsandapproach.TheGuidingPrinciplesalsodescribetheprocessofestimatingandcharacterizingvariabilityanduncertaintyaroundriskestimates.StahlandCimorelli(2005)andtheRiskAssessmentGuidanceforSuperfund:VolumeIII(USEPA2001)highlighttheimportanceofcommunicationbetweentheriskassessorandmanager.StahlandCimorelli(2005)andJamieson(1996)indicatethatitisimportanttodeterminewhetheraparticularlevelofuncertaintyisacceptableornot.Theauthorsalsosuggestthatthisdecisiondependsoncontext,valuesandregulatorypolicy.TheRiskAssessmentGuidanceforSuperfund:VolumeIII(Chapter2andAppendixFinUSEPA2001)describesaprocessfordeterminingtheappropriatelevelofPRAusingarankedapproachfromthelessresource‐andtime‐intensiveapproachestomoresophisticatedanalyses.Furthermore,theRiskAssessmentGuidanceforSuperfund:VolumeIIIoutlinesaprocessfordevelopingaPRAworkplanandachecklistforPRAreviewers(Chapter2andAppendixFinUSEPA2001).ThisguidancealsoprovidesinformationregardinghowtocommunicatePRAresultstodecisionmakersandstakeholders(Chapter6inUSEPA2001).
C.2. Categorizing Case Studies
Therankedapproachusedforcategorizationisaprocessforasystematic,informedprogressiontoincreasinglycomplexriskassessmentmethodsofPRA,whichisoutlinedintheRiskAssessmentGuidanceforSuperfund(USEPA2001).TheuseofcategoriesprovidesaframeworkforevaluatingthevarioustechniquesofPRA.Highercategoriesreflectincreasingcomplexityandoftenwillrequiremoretimeandresources.Highercategoriesalsoreflectincreasingcharacterizationofvariabilityanduncertaintyintheriskestimate,whichmaybeimportantformakingspecificriskmanagementdecisions.Centraltotheapproachisasystematic,informedprogressionusingan
57
iterativeprocessofevaluation,deliberation,datacollection,planningandscoping,development,andupdatestotheworkplanandcommunication.Allofthesestepsfocusondeciding:
1. Whetherornottheriskassessment,initscurrentstate(e.g.,DRA)issufficienttosupportdecisions(i.e.,aclearpathtoexitingtheprocessisavailableateachstep).
2. Iftheassessmentisdeterminedtobeinsufficient,whetherornotprogressiontoahigherlevelofcomplexity(orrefinementofthecurrentanalyses)wouldprovideasufficientbenefittowarranttheadditionaleffortofperformingaPRA.
ThisAppendixgroupscasestudiesaccordingtolevelofeffortandcomplexityoftheanalysisandtheincreasingsophisticationofthemethodsused(TableA‐1).Althougheachgroupgenerallyrepresentsincreasingeffortandcost,thismaynotalwaysbetrue.Thegroupsalsoareintendedtoreflecttheprogressionfromsimpletocomplexanalysisthatisdeterminedbytheinteractiveplanningandscopingeffortsoftheriskassessorsandmanagers.Theuseofparticulartermstodescribethegroups,including“tiers,”wasavoidedduetospecificprogrammaticandregulatoryconnotations.
Group 1 Case Studies Assessmentswithinthisgrouptypicallyinvolveasensitivityanalysisandserveasaninitialscreeningstepintheriskassessment.Sensitivityanalysesidentifyimportantparametersintheassessmentwhereadditionalinvestigationmaybehelpful(KurowickaandCooke2006).Sensitivityanalysiscanbesimpleorinvolvemorecomplexmathematicalandstatisticaltechniques,suchascorrelationandregressionanalysis,todeterminewhichfactorsinariskmodelcontributemosttothevarianceintheriskestimate.
Withinthesensitivityanalyses,arangeoftechniquesisavailable:simple,“back‐of‐the‐envelope”calculations,wheretheriskparametersareevaluatedusingarangeofexposureparameterstodeterminetheparameterthatcontributesmostsignificantlytotherisk(CaseStudy1);analysestoranktherelativecontributionsofvariablestotheoverallrisk(CaseStudy2);anddatavisualizationusinggraphicaltechniquestoarraythedataorMonteCarlosimulations(e.g.,scatterplots).
Moresophisticatedanalysesmayincludesensitivityratios(e.g.,elasticity);sensitivityscores(e.g.,weightedsensitivityratios);correlationcoefficientorcoefficientofdetermination;r2(e.g.,Pearsonproductmoment,Spearmanrank);normalizedmultipleregressioncoefficients;andgoodness‐of‐fittestsforsubsetsoftheriskdistribution(USEPA2001).
Thesensitivityanalysestypicallyrequireminimalresourcesandtime.ResultsofthesensitivityanalysesareusefulinidentifyingkeyparameterswhereadditionalGroup2orGroup3analysesmaybeappropriate.Sensitivityanalysesalsoarehelpfulinidentifyingkeyparameterswhereadditionalresearchwillhavethehighestimpactontheriskassessment.
Group 2 Case Studies Casestudieswithinthisgroupincludeamoresophisticatedapplicationofprobabilistictools,includingPRAofspecificexposureparameters(CaseStudies3and4),one‐dimensionalanalyses(CaseStudy5)andprobabilisticsensitivityanalysis(CaseStudies6and7).
TheGroup2casestudiesrequirelargertimecommitmentsfordevelopment,specializedexpertiseandadditionalanalysisofexposureparameterdatasources.Dependingonthenatureoftheanalysis,peerinvolvementorpeerreviewmaybeappropriatetoevaluatetheproductsoftheanalysis.
58
Group 3 Case Studies Assessmentswithinthisgrouparethemostresource‐andtime‐intensiveanalysesofthethreecategories.Riskanalysesincludetwo‐dimensionalMonteCarloanalysis(2‐DMCA)thatevaluatesmodelvariabilityanduncertainty(CaseStudies8,9and10);microexposureeventanalysis(MEE),inwhichlong‐termexposureofanindividualissimulatedasthesumofseparateshort‐termexposureevents(CaseStudy11);andprobabilisticanalysis(CaseStudies12and13).
Othertypesofanalyseswithinthisgroupincludetheexpertelicitationmethodthatisasystematicprocessofformalizingandquantifying,intermsofprobabilities,experts’judgmentsaboutuncertainquantities(CaseStudies14and15);Bayesianstatistics,whichisaspecializedbranchofstatisticsthatviewstheprobabilityofaneventoccurringasthedegreeofbelieforconfidenceinthatoccurrence(CaseStudy16);andgeostatisticalanalysis,whichisanotherspecializedbranchofstatisticsthatexplicitlytakesintoaccountthegeo‐referencedcontextofthedataandtheinformation(e.g.,attributes)attachedtothedata.
TheGroup3analysesrequireadditionaltimeandexpertiseintheplanningandanalysisoftheassessment.Withinthisgroup,thelevelofexpertiseandresourcecommitmentsmayvarywiththetechniques.Expertelicitation,forexample,requiressignificantlymoretimeforplanning,identificationofexpertsandmeetings,whencomparedwiththeothertechniques.
59
Table A‐1. Case Study Examples of EPA Applications of Deterministic and Probabilistic Risk Assessment Techniques
Case Study Number
Title and Case Study Description Type of Risk Assessment
Office/Region
Group 1: Point Estimate—Sensitivity Analysis
1
Sensitivity Analysis of Key Variables in Probabilistic Assessment of Children’s Exposure to Arsenic in Chromated Copper Arsenate (CCA) Pressure-Treated Wood. This case study demonstrates use of a point estimate sensitivity analysis to identify exposure variables critical to the analysis summarized in Case Study 9. The sensitivity analysis identified critical areas for future research and data collection and better characterized the amount of dislodgeable residue that exists on the wood surface.
Human Health
Office of Research and Development (ORD) and Office of Pesticide Programs (OPP)
2
Assessment of the Relative Contribution of Atmospheric Deposition to Watershed Contamination. An example of a workbook that demonstrates how “back-of-the-envelope” analysis of potential exposure rates can be used to target resources to identify other inputs before further analysis of air inputs in watershed contamination. Identification of key variables aided in identifying uncertainties and data gaps to target resource expenditures for further analysis. A case study example of the application of this technique also is identified.
Ecological ORD
Group 2: Probabilistic Risk Analysis, One-Dimensional Monte Carlo Analysis (1-D MCA) and Probabilistic Sensitivity Analysis
Group 2: Probabilistic Risk Analysis
3
Probabilistic Assessment of Angling Duration Used in the Assessment of Exposure to Hudson River Sediments via Consumption of Contaminated Fish. A probabilistic analysis of one parameter in an exposure assessment―the time an individual fishes in a large river system. Development of site-specific information regarding exposure, with an existing data set for this geographic area, was needed to represent this exposed population. This information was used in the one-dimensional PRA described in Case Study 5.
Human Health
Superfund/ Region 2 (New York)
4
Probabilistic Analysis of Dietary Exposure to Pesticides for Use in Setting Tolerance Levels. The probabilistic Dietary Exposure Evaluation Model (DEEM) provides more accurate information on the range and probability of possible exposures.
Human Health OPP
60
Table A‐1. Case Study Examples of EPA Applications of Deterministic and Probabilistic Risk Assessment Techniques
Case Study Number
Title and Case Study Description Type of Risk Assessment
Office/Region
Group 2: One-Dimensional Monte Carlo Analysis (1-D MCA)
5
One-Dimensional Probabilistic Risk Analysis of Exposures to Polychlorinated Biphenyls (PCBs) via Consumption of Fish From a Contaminated Sediment Site. An example of a one-dimensional PRA (1-D MCA) of the variability of exposure as a function of the variability of individual exposure factors to evaluate the risks to anglers who consume recreationally caught fish from a PCB-contaminated river.
Human Health
Superfund/ Region 2
(New York)
Group 2: Probabilistic Sensitivity Analysis
6
Probabilistic Sensitivity Analysis of Knowledge Elicitation of the Concentration-Response Relationship Between PM2.5 Exposure and Mortality. An example of how the probabilistic analysis tools can be used to conduct a probabilistic sensitivity analysis following an expert elicitation (Group 3) presented in Case Study 14.
Human Health Office of Air
and Radiation (OAR)
7
Environmental Monitoring and Assessment Program (EMAP): Using Probabilistic Sampling Techniques to Evaluate the Nation’s Ecological Resources. A probability-based sampling program designed to provide unbiased estimates of the condition of an aquatic resource over a large geographic area based on a small number of samples.
Ecological ORD
Group 3: Advanced Probabilistic Risk Analysis―Two-Dimensional Monte Carlo Analysis (2-D MCA) Including Microexposure Modeling, Bayesian Statistics, Geostatistics and Expert Elicitation
Group 3: Two-Dimensional Probabilistic Risk Analysis
8
Two-Dimensional Probabilistic Risk Analysis of Cryptosporidium in Public Water Supplies, With Bayesian Approaches to Uncertainty Analysis. An analysis of the variability in the occurrence of Cryptosporidium in raw water supplies and in the treatment efficiency, as well as the uncertainty in these inputs. This case study includes an analysis of the dose-response relationship for Cryptosporidium infection.
Human Health Office of
Water (OW)
9
Two-Dimensional Probabilistic Model of Children’s Exposure to Arsenic in Chromated Copper Arsenate (CCA) Pressure-Treated Wood. A two-dimensional model that addresses both variability and uncertainty in the exposures of children to CCA pressure-treated wood. The analysis was built on the sensitivity analysis described in Case Study 2.
Human Health OPP/ORD
61
Table A‐1. Case Study Examples of EPA Applications of Deterministic and Probabilistic Risk Assessment Techniques
Case Study Number
Title and Case Study Description Type of Risk Assessment
Office/Region
10
Two-Dimensional Probabilistic Exposure Assessment of Ozone. A probabilistic exposure assessment that addresses short-term exposures to ozone. Population exposure to ambient ozone levels was evaluated using EPA’s Air Pollutants Exposure (APEX) model, also referred to as the Total Risk Integrated Methodology/Exposure (TRIM.Expo) model.
Human Health OAR
Group 3: Microexposure Event Analysis
11
Analysis of Microenvironmental Exposures to Fine Particulate Matter (PM2.5) for a Population Living in Philadelphia, Pennsylvania. A microexposure event analysis to simulate individual exposures to PM2.5 in specific microenvironments, including the outdoors, indoor residences, offices, schools, stores and a vehicle.
Human Health Region 3
(Philadelphia) and ORD
Group 3: Probabilistic Analysis
12
Probabilistic Analysis in Cumulative Risk Assessment of Organophosphorus Pesticides. A probabilistic computer software program used to integrate various pathways, while simultaneously incorporating the time dimensions of the input data to calculate margins of exposure.
Human Health OPP
13
Probabilistic Ecological Effects Risk Assessment Models for Evaluating Pesticide Uses. A multimedia exposure/ effects model that evaluates acute mortality levels in generic or specific avian species over a user-defined exposure window.
Ecological OPP
Group 3: Expert Elicitation and Bayesian Belief Network
14
Expert Elicitation of Concentration-Response Relationship Between Fine Particulate Matter (PM2.5) Exposure and Mortality. A knowledge elicitation used to derive probabilistic estimates of the uncertainty in one element of a cost-benefit analysis used to support the PM2.5 regulations.
Human Health ORD/ OAR
15
Expert Elicitation of Sea-Level Change Resulting From Global Climate Change. An example of a PRA that describes the probability of sea level rise and parameters that predict sea level change.
Ecological
Office of Policy,
Planning, and Evaluation
62
Table A‐1. Case Study Examples of EPA Applications of Deterministic and Probabilistic Risk Assessment Techniques
Case Study Number
Title and Case Study Description Type of Risk Assessment
Office/Region
16
Knowledge Elicitation for Bayesian Belief Network Model of Stream Ecology. An example of a Bayesian belief network model of the effect of increased fine-sediment load in a stream on macroinvertebrate populations.
Ecological ORD
63
D. CASE STUDY SUMMARIES D.1. Group 1 Case Studies
Case Study 1: Sensitivity Analysis of Key Variables in Probabilistic Assessment of Children’s Exposure to Arsenic in Chromated Copper Arsenate Pressure-Treated Wood ThiscasestudyprovidesanexampleoftheapplicationofsensitivityanalysistoidentifyimportantvariablesforpopulationexposurevariabilityforaGroup2assessment(CaseStudy9)andtoindicateareasforfurtherresearch.Specifically,EPA’sOfficeofResearchandDevelopment(ORD),incollaborationwiththeOfficeofPesticidePrograms(OPP),usedsensitivityanalysestoidentifythekeyvariablesinchildren’sexposuretoCCA‐treatedwood.
Approach.Thesensitivityanalysesusedtwoapproaches.Thefirstapproachestimatedbaselineexposurebyrunningtheexposuremodelwitheachinputvariablesettoitsmedian(50thpercentile)value.Next,alternativeexposureestimatesweremadebysettingeachinputtoits25thor75thpercentilevaluewhileholdingallotherinputsattheirmedianvalues.Theratiooftheexposureestimatecalculatedwhenaninputwasestimatedatits25thor75thpercentiletotheexposureestimatecalculatedwhentheinputwasatitsmedianvalueprovidedameasureofthatinput’simportancetotheoverallexposureassessment.Thesecondapproachappliedamultiplestepwiseregressionanalysistothedatapointsgeneratedfromthefirstapproach.Thecorrelationbetweentheinputvariablesandtheexposureestimatesprovidedanalternativemeasureoftheinputvariable’srelativeimportanceintheexposureassessment.Thesetwoapproacheswereusedintandemtoidentifythecriticalinputstotheexposureassessmentmodel.
ResultsofAnalysis.Thetwosensitivityanalysestogetheridentifiedsixcriticalinputvariablesthatmostinfluencedtheexposureassessment.Thecriticalinputvariableswere:woodsurfaceresidue‐to‐skintransferefficiency,woodsurfaceresiduelevels,fractionofhandsurfaceareamouthedpermouthingevent,averagefractionofnonresidentialoutdoortimespentplayingonaCCA‐treatedplayset,frequencyofhandwashingandfrequencyofhand‐to‐mouthactivity.
ManagementConsiderations.Theresultsofthesensitivityanalyseswereusedtoidentifythemostimportantinputparametersinthetreatedwoodriskassessments.Theprocessalsoidentifiedcriticalareasforfutureresearch.Inparticular,theassessmentpointedtoaneedtocollectdataontheamountofdislodgeableresiduethatistransferredfromthewoodsurfacetoachild’shanduponcontact,andtobettercharacterizetheamountofdislodgeableresiduethatexistsonthewoodsurface.
SelectedReferences.ThefinalreportontheprobabilisticexposureassessmentofCCA‐treatedwood:
Zartarian,V.G.,J.Xue,H.A.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.AProbabilisticExposureAssessmentforChildrenWhoContactCCA‐TreatedPlaysetsandDecksUsingtheStochasticHumanExposureandDoseSimulationModelfortheWoodPreservativeScenario(SHEDS‐WOOD),FinalReport.EPA/600/X‐05/009.Washington,D.C.:USEPA.
Seealso:Xue,J.,V.G.Zartarian,H.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.“AProbabilisticExposureAssessmentforChildrenWhoContactChromatedCopperArsenate(CCA)‐TreatedPlaysetsandDecks,Part2:SensitivityandUncertaintyAnalyses.”RiskAnalysis26:533–41.
64
Case Study 2: Assessment of the Relative Contribution of Atmospheric Deposition to Watershed Contamination Watershedcontaminationcanresultfromseveraldifferentsources,includingthedirectreleaseofpollutionintoawaterbody,inputfromupstreamwaterbodiesanddepositionfromairbornesources.Effortstocontrolwaterbodycontaminationbeginwithananalysisoftheenvironmentalsourcestoidentifytheparametersthatprovidethegreatestcontributionanddeterminewheremitigationand/oranalysisresourcesshouldbedirected.
Approach.Thiscasestudyprovidesanexampleofa“back‐of‐the‐envelope”deterministicanalysisofthecontributionofairdepositiontooverallwatershednitrogen?Nutrient?contaminationtoidentifyuncertaintiesand/ordatagaps,aswellastotargetresourceexpenditures.NitrogeninputshavebeenstudiedinseveraleastandGulfCoastestuariesduetoconcernsabouteutrophication.Nitrogenfromatmosphericdepositionisestimatedtobeashighas10to40percentofthetotalinputofnitrogentomanyoftheseestuaries,andperhapshigherinafewcases.Forawatershedthathasnotbeenstudiedyet,aback‐of‐the‐envelopecalculationcouldbepreparedbasedoninformationaboutthenitrogendepositionratesmeasuredinasimilararea.Toestimatethedepositionloaddirectlytothewaterbody,onewouldmultiplythenitrogendepositionratebytheareaofthewaterbody.Theanalystthencouldestimatethenitrogenloadfromothersources(e.g.,pointsourcedischargesandrunoff)toestimateatotalnitrogenloadforthewaterbody.Theestimateofloadingduetoatmosphericdepositionthencouldbedividedbythetotalnitrogenloadforthewaterbodytoestimatethepercentofcontributiondirectlytothewaterbodyfromatmosphericdeposition.
TheMay2003reportbytheCascoBayAirDepositionStudyTeamtitledEstimatingPollutantLoadingFromAtmosphericDepositionUsingCascoBay,MaineasaCaseStudyisananalysisusingthemethodologydescribedabove.TheCascoBayEstuary,locatedinsouthwesternMaine,isusedasacasestudy.ThepaperalsoincludestheresultsofafieldairdepositionmonitoringprogramconductedinCascoBayfrom1998to2000andfavorablycomparestheestimatesdevelopedfortherateofdepositionofnitrogen,mercuryandpolycyclicaromatichydrocarbons(PAHs)tothefieldmonitoringresults.Theestimationapproachisausefulstartingpointforunderstandingthesourcesofpollutantsenteringwaterbodiesthatcannotbeaccountedforthroughrunofforpointsourcedischarges.
ResultsofAnalysis.TheapproachoutlinedabovewasappliedtotheCascoBayEstuaryinMaine.Resources,toolsandstrategiesforpollutionabatementcanbeeffectivelytargetedatprioritysourcesifestuariesaretobeprotected.Understandingthesourcesandannualloadingofcontaminantstoanestuaryfacilitatesgoodwaterqualitymanagementbydefiningtherangeofcontrolsofbothairandwaterpollutionneededtoachieveadesiredresult.Thecostofconductingmonitoringtodetermineatmosphericloadingtoawaterbodycanbeprohibitivelyhigh.Also,collectionofmonitoringdataisalong‐termundertakingbecauseaminimumof3yearsofdataisadvisableto“smoothout”inter‐annualvariability.Theestimationtechniquesdescribedinthispapercanserveasausefulandinexpensive“first‐cut”atunderstandingtheimportanceoftheatmosphereasapollutionsourceandcanhelptoidentifyareaswherefieldmeasurementsareneededtoguidefuturemanagementdecisions.
ManagementConsiderations.Ifareviewofinformationonairdepositionavailablefortheanalysisindicatesawiderangeofpotentialdepositionrates,furtherstudyofthisinputwouldleadtobettercharacterizationoftheaircontributiontooverallcontamination.Iftheback‐of‐theenvelopeanalysissuggeststhatairdepositionisverysmallrelativetootherinputs,thenresourcesshouldbetargetedatstudyingorreducingotherinputsbeforeproceedingwithfurtheranalysisoftheairinputs.
65
SelectedReferences.Theback‐of‐the‐envelopecalculationisoutlinedinFrequentlyAskedQuestionsaboutAtmosphericDeposition:AHandbookforWatershedManagers.http://www.epa.gov/air/oaqps/gr8water/handbook/airdep_sept.pdf.
FurtheranalysisisavailableinDepositionofAirPollutantstotheGreatWaters—ThirdReporttoCongress.http://www.epa.gov/air/oaqps/gr8water/3rdrpt/index.html.
TheCascoBayEstuaryexampleisavailableathttp://epa.gov/owow/airdeposition/cascobay.pdf.
D.2. Group 2 Case Studies
Case Study 3: Probabilistic Assessment of Angling Duration Used in the Assessment of Exposure to Hudson River Sediments via Consumption of Contaminated Fish InassessingthehealthimpactofcontaminatedSuperfundsites,exposuredurationtypicallyisassumedtobethesameasthelengthoftimethatanindividuallivesinaspecificarea(i.e.,residenceduration).InconductingtheHHRAfortheHudsonRiverPolychlorinatedBiphenyl(PCB)SuperfundSite,however,therewasconcernthatexposuredurationbasedonresidencedurationmayunderestimatethetimespentfishing(i.e.,anglingduration).
RiskAnalysis.Anindividualmaymovefromoneresidencetoanotherandcontinuetofishinthesamelocation,oranindividualmaychoosetostopfishingirrespectiveofthelocationofhisorherhome.EPARegion2developedasite‐specificdistributionofanglingdurationusingthefishingpatternsreportedinaNewYorkState‐wideanglingsurvey(Connellyetal.1990)andmigrationdataforthefivecountiessurroundingmorethan40milesoftheUpperHudsonRivercollectedaspartoftheU.S.Census.
ResultsofAnalysis.The50thand95thpercentilevaluesfromthedistributionofanglingdurationswerehigherthanthedefaultvaluesbasedonresidencedurationusingstandarddefaultexposureassumptionsforresidentialscenarios.Thesevalueswereusedasabaseforthecentraltendencyandreasonablemaximumexposurepointestimates,respectively,inthedeterministicassessment.
ManagementConsiderations.Theinformationprovidedinthisanalysiswasusedinthepointestimateanalysis.ThefulldistributionwasusedinconductingaGroup2PRAforthefishconsumptionpathway,whichispresentedasCaseStudy5.
SelectedReferences.ThefinalriskassessmentwasreleasedinNovember2000andisavailableathttp://www.epa.gov/hudson/reports.htm.
Furtherinformation,includingEPA’sJanuary2002responsetocommentsontheriskassessment,isavailableathttp://www.epa.gov/hudson/ResponsivenessSummary.pdf.
Case Study 4: Probabilistic Analysis of Dietary Exposure to Pesticides for Use in Setting Tolerance Levels UndertheFederalFood,Drug,andCosmeticAct(FFDCA),EPAmayauthorizeatoleranceorexemptionfromtherequirementofatolerancetoallowapesticideresidueinfood,onlyiftheAgencydeterminesthatsuchresidueswouldbe“safe.”Thisdeterminationismadebyestimatingexposuretothepesticideandcomparingtheestimatedexposuretoatoxicologicalbenchmarkdose.Until1998,theOPPusedasoftwareprogramcalledtheDietaryRiskEvaluationSystem(DRES)toconductitsacutedietaryriskassessmentsforpesticideresiduesinfoods.AcuteassessmentsconductedwithDRESassumedthat100percentofagivencropwithregistereduseofapesticide
66
wastreatedwiththatpesticideandallsuchtreatedcropitemscontainedpesticideresiduesatthemaximumlegal(tolerance)level,matchingthistoareasonablyhighconsumptionvalue(aroundthe95thpercentile).TheresultingDRESacuteriskestimateswereconsidered“high‐end”or“bounding”estimates.Itwasnotpossible,however,toknowwherethepesticideexposureestimatesfromtheDRESsoftwarefitintheoveralldistributionofexposuresduetothelimitsofthetoolsbeingused.
Toaddressthesedeficiencies,OPPdevelopedanacuteprobabilisticdietaryexposureguidancetouseamodelthatwouldestimatetheexposuretopesticidesinthefoodsupply.Ratherthanthecrude“high‐end,”single‐pointestimatesprovidedbydeterministicassessments,theprobabilisticDietaryExposureEvaluationModel(DEEM)providesspecificinformationabouttherangeandprobabilityofpossibleexposures.Dependingonthecharacterizationoftheinput,thiscouldincludethe95thpercentileregulation—generallyforlowertiersthatdonotincludethepercentofcroptreated—tothe99.9thpercentileforthemorerefinedassessments,whichwouldincludethepercentofcroptreatedinformation.
ProbabilisticAnalysis.Thiscasestudyprovidesanexampleofaone‐dimensionalPRAofdietaryexposuretopesticides(Group2).TheDEEMgeneratesacute,probabilisticdietaryexposureassessmentsusingdataon:(1)thedistributionofdailyconsumptionofspecificcommodities(e.g.,wheat,cornandapples)byspecificindividuals;and(2)thedistributionofconcentrationsofaspecificpesticideinthosefoodcommodities.DataoncommodityconsumptionarecollectedbytheU.S.DepartmentofAgriculture(USDA)initsContinuingSurveyofFoodIntakebyIndividuals(CSFII).Pesticideresidueconcentrationsonfoodcommoditiesaregenerallyobtainedfromcropfieldtrials,USDA’sPesticideDataProgram(PDP),U.S.FoodandDrugAdministration(FDA)monitoringdata,ormarketbasketsurveys.Usingthesedata,DEEMisabletocalculateanestimateoftherisktothegeneralU.S.population,inadditionto26populationsubgroups,including5subgroupsforinfantsandchildren(infantslessthan1,children1to2,children3to5,youths6to12andteens13to19yearsofage).
ResultsofAnalysis.DEEMhasbeenusedinriskassessmentstosupporttolerancelevelsforseveralpesticides(e.g.,phosalone)andaspartofcumulativeriskassessmentsfororganophosphoruscompounds(seeCaseStudy12)andotherpesticides.
ManagementConsiderations.UsingtheDRES,decisionswerebeingmadewithoutacompleterepresentationofthedistributionofriskamongthepopulationandwithoutfullknowledgeofwhereinthedistributionofrisktheDRESriskestimatelay.Thiswasofconcernnotonlyforregulatorsinterestedinpublichealthprotection,butalsoforthepesticideregistrantswhocouldarguethattheAgencywasarbitrarilyselectingthelevelatwhichtoregulate.FormostcasesreviewedbyOPPtodate,estimatedexposureatthe99.9thpercentilecalculatedbyDEEMprobabilistictechniquesissignificantlylowerthanexposurecalculatedusingDRES‐typedeterministicassumptionsattheunknownpercentile.
SelectedReferences.AlinktotheDEEMmodelisavailableathttp://www.epa.gov/pesticides/science/deem/index.html.
Case Study 5: One-Dimensional Probabilistic Risk Analysis of Exposure to Polychlorinated Biphenyls via Consumption of Fish From a Contaminated Sediment Site EPARegion2conductedapreliminarydeterministicHHRAattheHudsonRiverPCBsSuperfundsite.TheDRAdemonstratedthatconsumptionofrecreationallycaughtfishprovidedthehighest
67
exposureamongrelevantexposurepathways,whichresultedincancerrisksandnoncancerhealthhazardsthatexceededregulatorybenchmarks.
ProbabilisticAnalysis.Becauseofthesize,complexityandhighlevelofpublicinterestinthissite,EPARegion2implementedaGroup2probabilisticassessmenttocharacterizethevariabilityinrisksassociatedwiththefishconsumptionexposurepathway.Theanalysiswasaone‐dimensionalMonteCarloanalysis(1‐DMCA)ofthevariabilityofexposureasafunctionofthevariabilityofindividualexposurefactors.Uncertaintywasassessedusingsensitivityanalysisoftheinputvariables.Datatocharacterizethedistributionsofexposureparametersweredrawnfromthepublishedliterature(e.g.,fishconsumptionrate)orfromexistingdatabases,suchastheU.S.Censusdata(e.g.,anglingduration,seeCaseStudy3).Mathematicalmodelsoftheenvironmentalfate,transportandbioaccumulationofPCBsintheHudsonRiverpreviouslydevelopedwereusedtoforecastchangesinPCBconcentrationovertime.
ResultsofAnalysis.TheresultsofthePRAwereconsistentwiththedeterministicresults.Forthecentraltendencyindividual,pointestimateswerenearthemedian(50thpercentile).Forthereasonablemaximumexposure(RME)individual,pointestimatevalueswereatorabovethe95thpercentileoftheprobabilisticanalysis.TheDRAandPRAwerethesubjectofaformalpeerreviewbyapanelofindependentexperts.
TheMonteCarlo‐basedcasescenarioistheonefromwhichpointestimateexposurefactorsforfishingestionweredrawn;thus,thepointestimateRMEsandtheMonteCarlo‐basedcaseestimatescanbecompared.Similarly,thepointestimatecentraltendency(average)andtheMonteCarlo‐basedcasemidpoint(50thpercentile)arecomparable.Forcancerrisk,thepointestimateRMEforfishingestion(1x10‐3)fallsapproximatelyatthe95thpercentilefromtheMonteCarlo‐basedcaseanalysis.Thepointestimatecentraltendencyvalue(3x10‐5)andtheMonteCarlo‐basedcase50thpercentilevalue(6x10‐5)aresimilar.Fornoncancerhealthhazards,thepointestimateRMEforfishingestion(104forayoungchild1to6yearsofage)fallsbetweenthe95thand99thpercentilesoftheMonteCarlo‐basedcase.Thepointestimatecentraltendencyhazardindex(HI;12forayoungchild)isapproximatelyequaltothe50thpercentileoftheMonteCarlo‐basedcaseHIof11.
FiguresA‐1andA‐2provideacomparisonofresultsfromtheprobabilisticanalysiswiththatoftheDRAforcancerrisksandnoncancerhealthhazards.FiguresA‐1andA‐2plotpercentilesfor72combinationsofexposurevariables(e.g.,distributionsfromcreelanglersurveys’residenceduration,fishinglocationsandcookinglosses)ofthenoncancerHIvaluesandthecancerrisks,respectively.Ineachofthesefigures,thevariabilityofcancerriskornoncancerHIsforanglerswithintheexposedpopulationisplottedonthey‐axisforparticularpercentileswithinthepopulation.Thisvariabilityisafunctionofthevariationsinfishconsumptionrates,fishingduration,differencesinfishspeciesingestedandsoforth.TheuncertaintyintheestimatesisindicatedbytherangeofeithercancerriskornoncancerHIvaluesplottedonthex‐axis.Thisuncertaintyisafunctionofthe72combinationsoftheexposurefactorinputsexaminedinthesensitivityanalysis.Thisanalysisprovidesasemi‐quantitativeconfidenceintervalforthecancerrisksandHIvaluesatanyparticulatepercentile.Asthesefiguresshow,theintervalsspansomewhatlessthantwoordersofmagnitude(e.g.,<100‐fold).Theverticallinesindicatethedeterministicendpoints.
ManagementConsiderations.Earlyandcontinuedinvolvementofthecommunityimprovedpublicacceptanceoftheresults.Inaddition,carefulconsiderationofthemethodsusedtopresenttheprobabilisticresultstothepublicleadtogreaterunderstandingofthefindings.
SelectedReferences.ThefinalriskassessmentwasreleasedinNovember2000andisavailableathttp://www.epa.gov/hudson/reports.htm.
68
Furtherinformation,includingEPA’sJanuary2002responsetocommentsontheriskassessment,isavailableathttp://www.epa.gov/hudson/ResponsivenessSummary.pdf.
Figure A‐1. Monte Carlo Cancer Summary Based on a One‐Dimensional Probabilistic Risk Analysis of Exposure to Polychlorinated Biphenyls. The estimated cancer rate was calculated based on the consumption of fish from a contaminated sediment site. Source: USEPA 2000b.
Figure A‐2. Monte Carlo Noncancer Hazard Index Summary Based on a One‐Dimensional Probabilistic Risk Analysis of Exposure to Polychlorinated Biphenyls. The incremental individual hazard index (HI) was calculated based on the consumption of fish from a contaminated sediment site. Source: USEPA 2000b.
Fraction of Anglers
With HI ≤
Indicated Value
Fraction of Anglers
With Risk ≤
Indicated Value
69
Case Study 6: Probabilistic Sensitivity Analysis of Expert Elicitation of Concentration-Response Relationship Between Fine Particulate Matter Exposure and Mortality In2002,theNationalResearchCouncil(NRC)recommendedthatEPAimproveitscharacterizationofuncertaintyinthebenefitsassessmentforproposedregulationsofairpollutants.NRCrecommendedthatprobabilitydistributionsforkeysourcesofuncertaintybedevelopedusingavailableempiricaldataorthroughformalelicitationofexpertjudgments.Inresponsetothisrecommendation,EPAconductedanexpertelicitationevaluationoftheconcentration‐responserelationshipbetweenfineparticulatematter(PM2.5)exposureandmortality,akeycomponentofthebenefitsassessmentofthePM2.5regulation.FurtherinformationontheexpertelicitationprocedureandresultsisprovidedinCaseStudy14.Toevaluatethedegreetowhichtheresultsoftheassessmentdependedonthejudgmentsofindividualexpertsoronthemethodsofexpertelicitation,aprobabilisticsensitivityanalysiswasperformedontheresults.
ProbabilisticAnalysis.Theexpertelicitationprocedureusedcarefullyconstructedinterviewstoelicitfromeachof12expertsanestimateoftheprobabilisticdistributionfortheaverageexpecteddecreaseinU.S.annual,adult,all‐causemortalityassociatedwitha1microgrampercubicmeter(μg/m3)decreaseinannualaveragePM2.5levels.Thiscasestudyprovidesanexampleoftheuseofprobabilisticsensitivityanalysis(Group2)asoneelementoftheoverallassessment.Forthesensitivityanalysis,asimplifiedhealthbenefitsanalysiswasconductedtoassessthesensitivityoftheresultstotheresponsesofindividualexpertsandtothreefactorsinthestudydesign:(1)theuseofparametricornonparametricapproachesbyexpertstocharacterizetheiruncertaintyinthePM2.5mortalitycoefficient;(2)participationinthePre‐ElicitationWorkshop;and(3)allowingexpertstochangetheirjudgmentsafterthePost‐ElicitationWorkshop.Theindividualquantitativeexpertjudgmentswereusedtoestimateadistributionofbenefits,intheformofthenumberofdeathsavoided,associatedwithareductioninambient,annualaveragePM2.5concentrationsfrom12to11μg/m3.The12individualdistributionsofestimatedavoideddeathswerepooledusingequalweightstocreateasingleoveralldistributionreflectinginputfromeachexpert.Thisdistributionservedasthebaselineforthesensitivityanalysis,whichcomparedthemeansandstandarddeviationsofthebaselinedistributionwithseveralvariants.
ResultsofAnalysis.Thefirstanalysisexaminedthesensitivityofthemeanandstandarddeviationoftheoverallmortalitydistributiontotheremovalofindividualexperts’distributions.Ingeneral,theresultssuggestedafairlyequaldivisionbetweenthoseexpertswhoseremovalshiftedthedistributionmeanupandthosewhoshifteditdown.Therewererelativelymodestimpactsofindividualexperts.Thestandarddeviationofthecombineddistributionalsowasnotaffectedstronglybytheremovalofindividualexperts.Thesecondanalysisevaluatedwhethertheuseofparametricornonparametricapproachesaffectedtheoverallresults.Theresultssuggestedthattheuseofparametricdistributionsledtodistributionswithsimilarorslightlyincreaseduncertaintycomparedwithdistributionsprovidedbyexpertswhoofferedpercentilesofanonparametricdistribution.ThelastanalysisevaluatedwhetherparticipationinthePre‐orPost‐ElicitationWorkshopsaffectedtheresults.Participationineitherworkshopdidnotappeartohaveasignificanteffectonexperts’judgmentsbasedonmeasuresofchangeinthebaselinedistribution.Overall,thesensitivityanalysesdemonstratedthattheassessmentwasrobust,withlittledependenceonindividualexperts’judgmentsoronthespecificelicitationmethodsevaluated.
ManagementConsiderations.ThesensitivityanalysisdemonstratedtherobustnessofthePM2.5expertelicitation‐basedassessmentbyshowingthatthepanelofexpertswasgenerallywellbalancedandthatalternativeelicitationmethodswouldnothavemarkedlyalteredtheoverallresults.
70
SelectedReferences.ThedetailsofthisanalysisareprovidedintheIndustrialEconomics,Inc.,documenttitled:ExpandedExpertJudgmentAssessmentoftheConcentration‐ResponseRelationshipBetweenPM2.5ExposureandMortality,FinalReport,September21,2006.Thisdocumentisavailableathttp://www.epa.gov/ttn/ecas/regdata/Uncertainty/pm_ee_report.pdf.
Theexpertelicitationassessment,alongwiththeRegulatoryImpactAnalysis(RIA)ofthePM2.5standard,isavailableathttp://www.epa.gov/ttn/ecas/ria.html.
Case Study 7: Environmental Monitoring and Assessment Program: Using Probabilistic Sampling to Evaluate the Condition of the Nation’s Aquatic Resources Monitoringisakeytoolusedtoidentifythelocationswheretheenvironmentisinahealthybiologicalconditionandrequiresprotection,andwhereenvironmentalproblemsareoccurringandneedremediation.Mostmonitoring,however,isnotperformedinawaythatallowsforstatisticallyvalidassessmentsofwaterqualityconditionsinunmonitoredwaters(USGAO2000).StatesthuscannotadequatelymeasurethestatusandtrendsinwaterqualityasrequiredbytheCleanWaterAct(CWA)Section305(b).
TheEnvironmentalMonitoringandAssessmentProgram’s(EMAP)focushasbeentodevelopunbiasedstatisticalsurveydesignframeworksandsensitiveindicatorsthatallowtheconditionofaquaticecosystemstobeassessedatstate,regionalandnationalscales.AcornerstoneofEMAPhasbeentheuseofprobabilisticsamplingtoallowrepresentative,unbiased,cost‐effectiveconditionassessmentsforaquaticresourcescoveringlargeareas.EMAP’sstatisticalsurveymethodsareveryefficient,requiringrelativelyfewsamplelocationstomakevalidscientificstatementsabouttheconditionofaquaticresourcesoverlargeareas(e.g.,theconditionofallofthewadeablestreamsinthewesternUnitedStates).
ProbabilisticAnalysis.Thisresearchprogramprovidesmultiplecasestudiesusingprobabilisticsamplingdesignsfordifferentaquaticresources(estuaries,streamsandrivers).AnEMAPprobability‐basedsamplingprogramdeliversanunbiasedestimateoftheconditionofanaquaticresourceoveralargegeographicareafromasmallnumberofsamples.Theprincipalcharacteristicsofaprobabilisticsamplingdesignare:thepopulationbeingsampledisunambiguouslydescribed;everyelementinthepopulationhastheopportunitytobesampledwithaknownprobability;andsampleselectionisconductedbyarandomprocess.Thisapproachallowsstatisticalconfidencelevelstobeplacedontheestimatesandprovidesthepotentialtodetectstatisticallysignificantchangesandtrendsinconditionwithrepeatedsampling.Inaddition,thisapproachpermitstheaggregationofdatacollectedfromsmallerareastopredicttheconditionofalargegeographicarea.
TheEMAPdesignframeworkallowstheselectionofunbiased,representativesamplingsitesandspecifiestheinformationtobecollectedatthesesites.Thevalidityoftheoverallinferencerestsonthedesignandsubsequentanalysistoproduceregionallyrepresentativeinformation.TheEMAPusestheapproachoutlinedintheEPA’sGeneralizedRandomTessellationStratifiedSpatially‐BalancedSurveyDesignsforAquaticResources(Olsen2012).Thespatiallybalancedaspectspreadsoutthesamplinglocationsgeographically,butstillensuresthateachelementhasanequalchanceofbeingselected.
ResultsofAnalysis.DatacollectedusingtheEMAPapproachhasallowedtheAgencytomakescientificallydefensibleassessmentsoftheecologicalconditionoflargegeographicareasforreportingtoCongressunderCWA305(b).TheEMAPapproachhasbeenusedtoprovidethefirstreportsontheconditionofthenation’sestuaries,streams,riversandlakes,anditisscheduledtobe
71
usedforwetlands.EMAPfindingshavebeenincludedinEPA’sReportontheEnvironmentandtheHeinzCenter’sTheStateoftheNation’sEcosystems.DatacollectedthroughanEMAPapproachimprovetheabilitytoassessecologicalprogressinenvironmentalprotectionandrestoration,andprovidevaluableinformationfordecisionmakersandthepublic.TheuseofprobabilisticanalysismethodsallowsmeaningfulassessmentandregionalcomparisonsofaquaticecosystemconditionsacrosstheUnitedStates.Finally,theprobabilisticapproachprovidesscientificcredibilityforthemonitoringnetworkandaidsinidentifyingdatagaps.
ManagementConsiderations.UseofanEMAPapproachaddressescriticismsfromtheGovernmentAccountabilityOffice(GAO),theNationalAcademyofSciences(NAS),theHeinzCenter(anonprofitenvironmentalpolicyinstitution),andothersthatnotedthenationlackedthedatatomakescientificallyvalidcharacterizationsofwaterqualityregionallyandacrosstheUnitedStates.Theprogramprovidescost‐effective,scientificallydefensibleandrepresentativedata,topermittheevaluationofquantifiabletrendsinecosystemcondition,tosupportperformance‐basedmanagementandfacilitatebetterpublicdecisionsregardingecosystemmanagement.EMAP’sapproachnowhasbeenadoptedbyEPA’sOfficeofWater(OW)tocollectdataontheconditionofallthenation’saquaticresources.OW,OfficeofAirandRadiation(OAR)andOfficeofChemicalSafetyandPollutionPrevention(OCSPP;formerlytheOfficeofPrevention,Pesticides,andToxicSubstances)nowhaveenvironmentalaccountabilityendpointsusingEMAPapproachesintheirAgencyperformancegoals.
SelectedReferences.GeneralinformationconcerningEMAPisavailableathttp://www.epa.gov/emap/index.html.
InformationonEMAPmonitoringdesignsisavailableathttp://www.epa.gov/nheerl/arm/designpages/monitdesign/monitoring_design_info.htm.
EPA’sGeneralizedRandomTessellationStratifiedSpatially‐BalancedSurveyDesignsforAquaticResourcesdocumentisavailableathttp://www.epa.gov/nheerl/arm/documents/presents/grts_ss.pdf.
USGAO(U.S.GovernmentAccountabilityOffice).2000.WaterQuality:KeyEPAandStateDecisionsLimitedbyInconsistentandIncompleteData.GAO/RCED‐00‐54.Washington,D.C.:USGAO.http://www.environmental‐auditing.org/Portals/0/AuditFiles/useng00ar_ft_key_epa.pdf.
USEPA(U.S.EnvironmentalProtectionAgency).2002.EMAPResearchStrategy.ResearchTrianglePark,NC:EnvironmentalMonitoringandAssessmentProgram,NationalHealthandEnvironmentalEffectsResearchLaboratory(NHEERL),USEPA.http://www.epa.gov/nheerl/emap/files/emap_research_strategy.pdf.
D.3. Group 3 Case Studies
Case Study 8: Two-Dimensional Probabilistic Risk Analysis of Cryptosporidium in Public Water Supplies, With Bayesian Approaches to Uncertainty Analysis ProbabilisticassessmentoftheoccurrenceandhealtheffectsassociatedwithCryptosporidiumbacteriainpublicdrinkingwatersupplieswasusedtosupporttheeconomicanalysisofthefinalLong‐Term2EnhancedSurfaceWaterTreatmentRule(LT2).EPA’sOfficeofGroundWaterandDrinkingWater(OGWDW)conductedthisanalysisandestablishedabaselinediseaseburdenattributabletoCryptosporidiuminpublicwatersuppliesthatusesurfacewatersources.Next,itmodeledsourcewatermonitoringandfinishedwaterimprovementsthatwillberealizedasaresult
72
oftheLT2.Post‐RuleriskisestimatedandtheLT2’shealthbenefitistheresultofsubtractingthisfromthebaselinediseaseburden.
ProbabilisticAnalysis.ProbabilisticassessmentwasusedforthisanalysisasameansofaddressingthevariabilityintheoccurrenceofCryptosporidiuminrawwatersupplies,thevariabilityinthetreatmentefficiency,andtheuncertaintyintheseinputsandinthedose‐responserelationshipforCryptosporidiuminfection.ThiscasestudyprovidesanexampleofaPRAthatevaluatesbothvariabilityanduncertaintyatthesametimeandisreferredtoasatwo‐dimensionalPRA.Theanalysisalsoincludedprobabilistictreatmentsofuncertaindose‐responseandoccurrenceparameters.MarkovChainMonteCarlosamplesofparametersetsfilledthisfunction.ThisBayesianapproach(treatingtheunknownparametersasrandomvariables)differsfromclassicaltreatments,whichwouldregardtheparametersasunknown,butfixed(Group3:AdvancedPRA).Theriskanalysisusedexistingdatasets(e.g.,theoccurrenceofCryptosporidiumandtreatmentefficacy)toinformthevariabilityoftheseinputs.UncertaintydistributionswerecharacterizedbasedonprofessionaljudgmentorbyanalyzingdatausingBayesianstatisticaltechniques.
ResultsofAnalysis.TheriskanalysisidentifiedtheCryptosporidiumdose‐responserelationshipasthemostcriticalmodelparametersintheassessment,followedbytheoccurrenceofthepathogenandtreatmentefficiency.BysimulatingimplementationoftheRuleusingimprecise,biasedmeasurementmethods,theassessmentprovidedestimatesofthenumberofpublicwatersupplysystemsthatwouldrequirecorrectiveactionandthenatureoftheactionslikelytobeimplemented.Thisinformationaffordedarealisticmeasureofthebenefits(inreduceddiseaseburden)expectedwiththeLT2.InresponsetoScienceAdvisoryBoard(SAB)comments,additionalCryptosporidiumdose‐responsemodelswereaddedtomorefullyreflectuncertaintyinthiselementoftheassessment.
ManagementConsiderations.TheLT2underwentexternalpeerreview,reviewbyEPA’sSABandintensereviewbytheOfficeofManagementandBudget(OMB).Occurrenceanddose‐responsecomponentsoftheriskanalysismodelwerecommunicatedtostakeholdersduringtheRule’sFederalAdvisoryCommitteeAct(FACA)process.DuetotherigoroftheanalysisandthesignedFACA“AgreementinPrinciple,”theOMBreviewwasstraightforward.
SelectedReferences.ThefinalassessmentofoccurrenceandexposuretoCryptosporidiumwasreleasedinDecember2005andisavailableathttp://www.epa.gov/safewater/disinfection/lt2/regulations.html.
Case Study 9: Two-Dimensional Probabilistic Model of Children’s Exposure to Arsenic in Chromated Copper Arsenate Pressure-Treated Wood ProbabilisticmodelsweredevelopedinresponsetoEPA’sOctober2001FederalInsecticide,Fungicide,andRodenticideAct(FIFRA)ScientificAdvisoryPanel(SAP)recommendationstouseprobabilisticmodelingtoestimatechildren’sexposurestoarsenicinCCA‐treatedplaysetsandhomedecks.
ProbabilisticAnalysis.EPA’sORD,incollaborationwiththeOfficeofPesticidePrograms(OPP),developedandappliedaprobabilisticexposureassessmentofchildren’sexposuretoarsenicandchromiumfromcontactwithCCA‐treatedwoodplaysetsanddecks.Thiscasestudyprovidesanexampleoftheuseoftwo‐dimensional(i.e.,addressingbothvariabilityanduncertainty)probabilisticexposureassessment(Group3:AdvancedPRA).Thetwo‐dimensionalassessmentemployedamodificationofORD’sStochasticHumanExposureandDoseSimulation(SHEDS)model
73
tosimulatechildren’sexposuretoarsenicandchromiumfromCCA‐treatedwoodplaysetsanddecks,aswellasthesurroundingsoil.StafffrombothORDandOPPcollaboratedinthedevelopmentoftheSHEDS‐Woodmodel.
ResultsofAnalysis.AdraftoftheprobabilisticexposureassessmentreceivedSAPreviewinDecember2003;thefinalreportwasreleasedin2005.Theresultsoftheprobabilisticexposureassessmentwereconsistentwithorintherangeoftheresultsofdeterministicexposureassessmentsconductedbyseveralotherorganizations.Themodelresultswereusedtocompareexposuresunderavarietyofscenarios,includingcoldversuswarmweatheractivitypatterns,useofwoodsealantstoreducetheavailabilityofcontaminantsonthesurfaceofthewood,anddifferenthand‐washingfrequencies.Themodelingofalternativemitigationscenariosindicatedthattheuseofsealantscouldresultinthegreatestexposurereduction,whileincreasedfrequencyofhandwashingalsocouldreduceexposure.
OPPusedtheSHEDS‐Woodmodelexposureresultsinitsprobabilisticchildren’sriskassessmentforCCA(USEPA2008).Thisincludedrecommendationsforriskreduction(useofsealantsandcarefulattentiontochildren’shandwashing)tohomeownerswithexistingCCAwoodstructures.Inaddition,theexposureassessmentwasusedtoidentifyareasforfurtherresearch,including:theefficacyofwoodsealantsinreducingdislodgeablecontaminantresidues,thefrequencywithwhichchildrenplayonoraroundCCAwood,andtransferefficiencyandresidueconcentrations.Tobettercharacterizetheefficacyofsealantsinreducingexposure,EPAandtheConsumerProductSafetyCommission(CPSC)conducteda2‐yearstudyofhowdislodgeablecontaminantresiduelevelschangedwiththeuseofvarioustypesofcommerciallyavailablewoodsealants.
ManagementConsiderations.TheOPPusedSHEDSresultsdirectlyinitsfinalriskassessmentforchildrenplayingonCCA‐treatedplaygroundequipmentanddecks.Themodelenhancedriskassessmentandmanagementdecisionsandprioritizeddataneedsrelatedtowoodpreservatives.Themodelingproductwaspivotalintheriskmanagementandre‐registrationeligibilitydecisionsforCCA,andinadvisingthepublichowtominimizehealthrisksfromexistingtreatedwoodstructures.IndustryalsoisusingSHEDStoestimateexposurestoCCAandotherwoodpreservatives.SomestatesareusingtheriskassessmentasguidanceinsettingtheirregulationsforCCA‐relatedplaygroundequipment.
SelectedReferences.ThefinalprobabilisticriskassessmentbasedontheSHEDS‐Woodexposureassessmentisavailableathttp://www.epa.gov/oppad001/reregistration/cca/final_cca_factsheet.htm.
ThemodelresultswereincludedinthefinalreportontheprobabilisticexposureassessmentofCCA‐treatedwoodsurfaces:Zartarian,V.G.,J.Xue,H.A.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.AProbabilisticExposureAssessmentforChildrenWhoContactCCA‐TreatedPlaysetsandDecksUsingtheStochasticHumanExposureandDoseSimulationModelfortheWoodPreservativeScenario(SHEDS‐Wood),FinalReport.EPA/600/X‐05/009.Washington,D.C.:USEPA.
ResultsofthesealantstudieswerereleasedinJanuary2007andareavailableathttp://www.epa.gov/oppad001/reregistration/cca/index.htm#reviews.
Theresultsoftheanalysiswerepublishedas:Zartarian,V.G.,J.Xue,H.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.“AProbabilisticArsenicExposureAssessmentforChildrenwhoContactCAA‐TreatedPlaysetsandDecks,Part1:ModelMethodology,VariabilityResults,andModelEvaluation.”RiskAnalysis26:515–31.
Moreinformationontheanalysiscanbefoundbyconsultingthefollowingresource:
74
USEPA(U.S.EnvironmentalProtectionAgency).2008.CaseStudyExamplesoftheApplicationofProbabilisticRiskAnalysisinU.S.EnvironmentalProtectionAgencyRegulatoryDecision‐Making(InReview).Washington,D.C.:RiskAssessmentForum,USEPA
Case Study 10: Two-Dimensional Probabilistic Exposure Assessment of Ozone AspartofEPA’srecentreviewoftheozoneNationalAmbientAirQualityStandards(NAAQS),theOfficeofAirQualityPlanningandStandards(OAQPS)conducteddetailedprobabilisticexposureandriskassessmentstoevaluatepotentialalternativestandardsforozone.Atdifferentstagesofthisreview,theCleanAirScientificAdvisoryCommittee(CASAC)OzonePanel(anindependentscientificreviewcommitteeofEPA’sSAB)andthepublicreviewedandprovidedcommentsonanalysesanddocumentspreparedbyEPA.Ascopeandmethodsplanfortheexposureandriskassessmentswasdevelopedin2005(USEPA2005).ThisplanwasintendedtofacilitateconsultationwiththeCASAC,aswellaspublicreview,andtoobtainadviceontheoverallscope,approachesandkeyissuesinadvanceofthecompletionoftheanalyses.Thiscasestudydescribestheprobabilisticexposureassessment,whichaddressesshort‐termexposurestoozone.TheexposureestimateswereusedasaninputtotheHHRAforlungfunctiondecrementsinallchildrenandasthmaticschool‐agedchildrenbasedonexposure‐responserelationshipsderivedfromcontrolledhumanexposurestudies.
ProbabilisticAnalysis.PopulationexposuretoambientozonelevelswasevaluatedusingEPA’sAirPollutantsExposure(APEX)model,alsoreferredtoastheTotalRiskIntegratedMethodology/Exposure(TRIM.Expo)model.Exposureestimatesweredevelopedforrecentozonelevels,basedon2002to2004airqualitydata,andforozonelevelssimulatedtojustmeettheexisting0.08ppm,8‐hourozoneNAAQSandseveralalternativeozonestandards,basedonadjustingthe2002to2004airqualitydata.Exposureestimatesweremodeledfor12urbanareaslocatedthroughouttheUnitedStatesforthegeneralpopulation,allschool‐agechildrenandasthmaticschool‐agechildren.Thisexposureassessmentisdescribedinatechnicalreport(USEPA2007b).TheexposuremodelAPEXisdocumentedinauser’sguideandtechnicaldocument(USEPA2006).AMonteCarloapproachwasusedtoproducequantitativeestimatesoftheuncertaintyintheAPEXmodelresults,basedonestimatesoftheuncertaintiesforthemostimportantmodelinputs.Thequantificationofmodelinputuncertainties,adiscussionofmodelstructureuncertainties,andtheresultsoftheuncertaintyanalysisaredocumentedinLangstaff(2007).
ResultsofAnalysis.UncertaintyintheAPEXmodelpredictionsresultsfromuncertaintiesinthespatialinterpolationofmeasuredconcentrations,themicroenvironmentmodelsandparameters,people’sactivitypatterns,andtoalesserextent,modelstructure.Thepredominantsourcesofuncertaintyappeartobethehumanactivitypatterninformationandthespatialinterpolationofambientconcentrationsfrommonitoringsitestootherlocations.Theprimarypolicy‐relevantfindingwasthattheuncertaintyintheexposureassessmentissmallenoughtolendconfidencetotheuseofthemodelresultsforthepurposeofinformingtheAdministrator’sdecisionontheozonestandard.
FigureA‐3illustratestheuncertaintydistributionforonemodelresult,thepercentofchildrenwithexposuresabove0.08ppm,8‐hourwhileatmoderateexertion.Thepointestimateof20percentistheresultfromtheAPEXsimulationusingthebestestimatesofthemodelinputs.ThecorrespondingresultfromtheMonteCarlosimulationsrangesfrom17to26percent,witha95percentuncertaintyinterval(UI)of19to24percent.NotethattheUIsarenotsymmetricbecausethedistributionsareskewed.
75
ManagementConsiderations.Theexposureanalysisalsoprovidedinformationonthefrequencywithwhichpopulationexposuresexceededseveralpotentialhealtheffectbenchmarklevelsthatwereidentifiedbasedontheevaluationofhealtheffectsinclinicalstudies.
TheexposureandriskassessmentsaresummarizedinChapters4and5,respectively,oftheOzoneStaffPaper(USEPA2007a).TheexposureestimatesoverthesepotentialhealtheffectbenchmarkswerepartofthebasisfortheAdministrator’sMarch27,2008,decisiontorevisetheozoneNAAQSfrom0.08to0.075ppm,8‐houraverage(seethefinalrulefortheozoneNAAQS1).
Figure A‐3. Uncertainty Distribution Model Results. The estimated percentage of children with 8‐hour exposures above 0.08 ppm at moderate exertion (the point estimate is 20%).
SelectedReferences.Moreinformationontheanalysiscanbefoundbyconsultingthefollowingresources:
Langstaff,J.E.2007.AnalysisofUncertaintyinOzonePopulationExposureModeling.OfficeofAirQualityPlanningandStandardsStaffMemorandumtoOzoneNAAQSReviewDocket.OAR‐2005‐0172.http://www.epa.gov/ttn/naaqs/standards/ozone/s_ozone_cr_td.html
USEPA(U.S.EnvironmentalProtectionAgency).2005.OzoneHealthAssessmentPlan:ScopeandMethodsforExposureAnalysisandRiskAssessment.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/naaqs/standards/ozone/s_o3_cr_pd.html
USEPA.2006.TotalRiskIntegratedMethodology(TRIM)—AirPollutantsExposureModelDocumentation(TRIM.Expo/APEX,Version4)VolumeI:User’sGuide;VolumeII:TechnicalSupportDocument.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.June2006.http://www.epa.gov/ttn/fera/human_apex.html
USEPA.2007a.ReviewofNationalAmbientAirQualityStandardsforOzone:PolicyAssessmentofScientificandTechnicalInformation—OAQPSStaffPaper.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/naaqs/standards/ozone/s_ozone_cr_sp.html
1NationalAmbientAirQualityStandardsforOzone,FinalRule.73Fed.Reg.16436(Mar.27,2008).
0
50
100
150
200
250
300
350
400
450
17% 18% 19% 20% 21% 22% 23% 24% 25% 26%
Percent of children
Num
ber o
f Mon
te C
arlo
sim
ulat
ions
76
USEPA.2007b.OzonePopulationExposureAnalysisforSelectedUrbanAreas.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/naaqs/standards/ozone/s_ozone_cr_td.html
Case Study 11: Analysis of Microenvironmental Exposures to Fine Particulate Matter for a Population Living in Philadelphia, Pennsylvania ThiscasestudyusedtheStochasticHumanExposureandDoseSimulationmodelforParticulateMatter(SHEDS‐PM)developedbyEPA’sNationalExposureResearchLaboratory(NERL)toprepareaprobabilisticassessmentofpopulationexposuretoPM2.5inPhiladelphia,Pennsylvania.Thiscasestudysimulationwaspreparedtoaccomplishthreegoals:(1)estimatethecontributionofPM2.5ofambient(outdoor)origintototalPM2.5exposure;(2)determinethemajorfactorsthatinfluencepersonalexposuretoPM2.5;and(3)identifyfactorsthatcontributethegreatestuncertaintytomodelpredictions.
ProbabilisticAnalysis.Thetwo‐dimensionalprobabilisticassessmentusedamicroexposureeventtechniquetosimulateindividualexposurestoPM2.5inspecificmicroenvironments(outdoors,indoorresidence,office,school,store,restaurantorbar,andinavehicle).Thepopulationforthesimulationwasgeneratedusingdemographicdataatthecensus‐tractlevelfromtheU.S.Census.Characteristicsofthesimulatedindividualswereselectedrandomlytomatchthedemographicproportionswithinthecensustractforgender,age,employmentstatusandhousingtype.TheassessmentusedspatiallyandtemporallyinterpolatedambientPM2.5measurementsfrom1992to1993and1990U.S.CensusdataforeachcensustractinPhiladelphia.Thiscasestudyprovidesanexampleofbothtwo‐dimensional(variabilityanduncertainty)probabilisticassessmentandmicroexposureeventassessment(Group3:AdvancedPRA).
ResultsofAnalysis.ResultsoftheanalysisshowedthathumanactivitypatternsdidnothaveasstronganinfluenceonambientPM2.5exposuresaswasobservedforexposuretoindoorPM2.5
sources.ExposuretoPM2.5ofambientorigincontributedasignificantpercentofthedailytotalPM2.5
exposures,especiallyforthesegmentofthepopulationwithoutexposuretoenvironmentaltobaccosmokeintheresidence.DevelopmentoftheSHEDS‐PMmodelusingthePhiladelphiaPM2.5casestudyalsoprovidedusefulinsightsintodataneedsforimprovinginputsintotheSHEDS‐PMmodel,reducinguncertaintyandfurtherrefinementofthemodelstructure.Someoftheimportantdataneedsidentifiedfromtheapplicationofthemodelinclude:dailyPM2.5measurementsovermultipleseasonsandacrossmultiplesiteswithinanurbanarea,improvedcapabilityofdispersionmodelstopredictambientPM2.5concentrationsatgreaterspatialresolutionandovera1‐yeartimeperiod,measurementstudiestobettercharacterizethephysicalfactorsgoverninginfiltrationofambientPM2.5intoresidentialmicroenvironments,furtherinformationonparticle‐generatingsourceswithintheresidence,anddatafortheotherindoormicroenvironmentsnotspecifiedinthemodel.
ManagementConsiderations.TheapplicationoftheSHEDS‐PMmodeltothePhiladelphiapopulationgaveinsightsintodataneedsandareasformodelrefinement.ThecontinueddevelopmentandevaluationoftheSHEDS‐PMpopulationexposuremodelarebeingconductedaspartofORD’sefforttodevelopasource‐to‐dosemodelingsystemforPMandairtoxics.Thistypeofexposureanddosemodelingsystemisconsideredtobeimportantforthescientificandpolicyevaluationofthecriticalpathways,aswellastheexposurefactorsandsourcetypesinfluencinghumanexposurestoavarietyofenvironmentalpollutants,includingPM.
77
SelectedReferences.Theresultsoftheanalysiswerepublishedin:
Burke,J.,M.Zufall,andH.Özkaynak.2001.“APopulationExposureModelforParticulateMatter:CaseStudyResultsforPM2.5inPhiladelphia,PA.”JournalofExposureAnalysisandEnvironmentalEpidemiology11(6):470–89.
Georgepoulos,P.G.,S.W.Wang,V.M.Vyas,Q.Sun,J.Burke,R.Vedantham,T.McCurdy,andH.Özkaynak.2005.“ASource‐to‐DoseAssessmentofPopulationExposuretoFinePMandOzoneinPhiladelphia,PA,DuringaSummer1999Episode.”JournalofExposureAnalysisandEnvironmentalEpidemiology15(5):439–57.
Case Study 12: Probabilistic Analysis in Cumulative Risk Assessment of Organophosphorus Pesticides In1996,CongressenactedtheFoodQualityProtectionAct(FQPA),whichrequiresEPAtoconsider“availableevidenceconcerningthecumulativeeffectsoninfantsandchildrenofsuchresiduesandothersubstancesthathaveacommonmechanismoftoxicity”whensettingpesticidetolerances(i.e.,themaximumamountofpesticideresiduelegallyallowedtoremainonfoodproducts).FQPAalsomandatedthatEPAcompletelyreassessthesafetyofallexistingpesticidetolerances(thoseineffectsinceAugust1996)toensurethattheyaresupportedbycurrentscientificdataandmeetcurrentsafetystandards.Becauseorganophosphoruspesticides(OPs)wereassignedpriorityfortolerancereassessment,thesepesticideswerethefirst“commonmechanism”groupidentifiedbyEPA’sOPP.Theultimategoalassociatedwiththiscumulativeriskassessment(CRA)wastoprovideabasisforthedecisionmakertoestablishsafetolerancelevelsforthisgroupofpesticides,whilemeetingtheFQPAstandardforprotectinginfantsandchildren.
ProbabilisticAnalysis.Thiscasestudyprovidesanexampleofanadvancedprobabilisticassessment(Group3).In2006,EPAanalyzedexposuresto30OPsthroughfoodconsumption,drinkingwaterintake,andexposureviapesticideapplication.Distributionsofhumanexposurefactors,suchasbreathingrates,bodyweightandsurfaceareasusedintheassessment,camefromtheAgency’sExposureFactorsHandbook(USEPA1997d).EPAusedCalendex,aprobabilisticcomputersoftwareprogram(availableathttp://www.epa.gov/pesticides/science/deem/)tointegratevariouspathways,whilesimultaneouslyincorporatingthetimedimensionsoftheinputdata.Basedontheresultsoftheexposureassessment,EPAcalculatedmarginsofexposure(MOEs)forthetotalcumulativeriskfromallpathwaysforeachagegroup(infantlessthan1;children1–2,3–5,6–12;youth13–19;andadults20–49and50+yearsofage).
ThefoodcomponentoftheOPsCRAwashighlyrefined,asitwasbasedonresiduemonitoringdatafromtheUSDA’sPDPandsupplementedwithinformationfromtheFDA’sSurveillanceMonitoringProgramsandTotalDietStudy.TheresiduedatawerecombinedwithactualconsumptiondatafromUSDA’sContinuingSurveyofFoodIntakesbyIndividuals(CSFII)usingprobabilistictechniques.TheCRAevaluateddrinkingwaterexposuresonaregionalbasis.TheassessmentfocusedonareaswherecombinedOPexposureislikelytobehighestwithineachregion.Primarily,theanalysisusedprobabilisticmodelingtoestimatetheco‐occurrenceofOPresiduesinwater.Monitoringdatawerenotavailablewithenoughconsistencytobethesolebasisfortheassessment;however,theywereusedtocorroboratethemodelingresults.DatasourcesforthewatercomponentoftheassessmentincludedUSDAAgriculturalUsageReportsforFieldCrops,FruitsandVegetables;USDATypicalPlantingandHarvestingDatesforFieldCropsandFreshMarketandProcessingVegetables;localsourcesforrefinements;andmonitoringstudiesfromtheU.S.GeologicalSurvey(USGS)andothersources.Finally,exposureviatheoral,dermalandinhalationroutesresultingfromapplicationsofOPsinandaroundhomes,schools,officesandotherpublicareaswereassessedprobabilisticallyforeachofthesevenregions.Thedatasourcesforthispartof
78
theassessmentincludedinformationfromsurveysandtaskforces,specialstudiesandreportsfrompublishedscientificliterature,EPA’sExposureFactorsHandbook(USEPA1997d),andothersources.
ResultsofAnalysis.TheOPsCRApresentedpotentialriskfromsingle‐day(acute)exposuresacross1yearandfromaseriesof21‐dayrollingaveragesacrosstheyear.MOEsatthe99.9thpercentilewereapproximately100orgreaterforallpopulationsforthe21‐dayaverageresults.Theonlyexceptionisabriefperiod(roughly2weeks)inwhichdrinkingwaterexposures(identifiedfromtheExposureFactorsHandbook,USEPA1997d)attributedtophorateuseonsugarcaneresultedinMOEsnear80forchildrenages1to2years.Generally,exposuresthroughthefoodpathwaydominatedtotalMOEs,andexposuresthroughdrinkingwaterweresubstantiallylowerthroughoutmostoftheyear.Residentialexposuresweresubstantiallysmallerthanexposuresthroughbothfoodanddrinkingwater.
TheOPsCRAwasveryresourceintensive.Workbeganin1997withthepreparationofguidancedocumentsandthedevelopmentofaCRAmethodology.Over2to3years,morethan25peoplespent50to100percentoftheirtimeworkingontheassessment,withupto50peopleworkingontheCRAatcriticalperiods.EPAhasspentapproximately$1milliononthisassessment(e.g.,forcomputers,modelsandcontractorsupport).
ManagementConsiderations.TheOPsCRAwasalandmarkdemonstrationofthefeasibilityofaregulatory‐levelassessmentoftheriskfrommultiplechemicals.Uponcompletion,EPApresentedtheCRAatnumerouspublictechnicalbriefingsandFIFRASAPmeetings,andmadeallofthedatainputsavailabletothepublic.TheOPP’ssubstantialefforttocommunicatemethodologies,approachesandresultstothestakeholdersaidedinthesuccessoftheOPsCRA.ThestakeholdersexpressedappreciationforthetransparentnatureoftheOPsCRAandrecognizedtheinnovationandhardworkthatwentintodevelopingtheassessments.
SelectedReferences.The2006assessmentandrelateddocumentsareavailableathttp://www.epa.gov/pesticides/cumulative/common_mech_groups.htm#op.
USEPA(U.S.EnvironmentalProtectionAgency).1997d.ExposureFactorsHandbook.Washington,D.C.:NationalCenterforEnvironmentalAssessment,USEPA.http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=12464.
Case Study 13: Probabilistic Ecological Effects Risk Assessment Models for Evaluating Pesticide Use AspartoftheprocessofdevelopingandimplementingaprobabilisticapproachforERA,anillustrativecasewascompletedin1996.ThiscaseinvolvedbothDRAandPRAfortheeffectsofahypotheticalchemicalX(ChemX)onbirdsandaquaticspecies.FollowingtherecommendationsoftheSAPandinresponsetoissuesraisedbyOPPriskmanagers,theAgencybegananinitiativetorefinetheERAprocessforevaluatingtheeffectsofpesticidestoterrestrialandaquaticspecieswithinthecontextofFIFRA,themainstatutoryauthorityforregulatingpesticidesatthefederallevel.ThekeygoalsandobjectivesofEPA’sinitiativewereto:
Incorporateprobabilistictoolsandmethodstoprovideanestimateonthemagnitudeandprobabilityofeffects.
Buildonexistingdatarequirementsforregistration.
Utilize,whereverpossible,existingdatabasesandcreatenewonesfromexistingdatasourcestominimizetheneedtogenerateadditionaldata.
Focusadditionaldatarequirementsonreducinguncertaintyinkeyareas.
79
Afterproposingafour‐levelriskassessmentscheme,withhigherlevelsreflectingmorerefinedriskassessmenttechniques,theAgencydevelopedpilotmodelsforbothterrestrialandaquaticspecies.Refinedriskassessmentmodels(LevelII)thenweredevelopedandusedinagenericchemicalcasestudythatwaspresentedtotheSAPin2001.
ProbabilisticAnalysis.Thiscasestudydescribesanadvancedprobabilisticmodelfortheecologicaleffectsofpesticides(Group3).TheterrestrialLevelIImodel(version2.0)isamultimediaexposure/effectsmodelthatevaluatesacutemortalitylevelsingenericorspecificavianspeciesoverauser‐definedexposurewindow.Thespatialscaleisatthefieldlevel,whichincludesthefieldandsurroundingarea.Bothareasareassumedtomeetthehabitatrequirementsforeachspecies,andcontaminationofedgeoradjacenthabitatfromdriftisassumedtobezero.ForeachindividualbirdconsideredinarunoftheLevelIImodel,arandomselectionofvaluesismadeforthemajorexposureinputparameterstoestimateanexternaloraldoseequivalentforthatindividual.Theestimateddoseequivalentiscomparedtoarandomlyassignedtolerancefortheindividualpreselectedfromthedose‐responsedistribution.Ifthedoseisgreaterthanthetolerance,theindividualisscored“dead,”ifnot,theindividualisscored“notdead.”Aftermultipleiterationsofindividuals,aprobabilitydensityfunctionofpercentmortalityisgenerated.
FromMay29to31,1996,theAgencypresentedtwoERAcasestudiestotheSAPforreviewandcomment.AlthoughrecognizingandgenerallyreaffirmingtheutilityofEPA’scurrentdeterministicassessmentprocess,theSAPofferedanumberofsuggestionsforimprovement.Foremostamongtheirsuggestionswasarecommendationtomovebeyondtheexistingdeterministicassessmentapproachbydevelopingthetoolsandmethodologiesnecessarytoconductaprobabilisticassessmentofeffects.Suchanassessmentwouldestimatethemagnitudeandprobabilityoftheexpectedimpactanddefinethelevelofcertaintyandvariationinvolvedintheestimate;riskmanagerswithinEPA’sOPPalsohadrequestedthisinformationinthepast.
TheaquaticLevelIImodelisatwo‐dimensionalMonteCarloriskmodelconsistingofthreemaincomponents:(1)exposure,(2)effectsand(3)risk.TheexposurescenariosusedatLevelIIareintendedtoprovideestimatesforvulnerableaquatichabitatsacrossawiderangeofgeographicalconditionsunderwhichapesticideproductisused.TheLevelIIriskevaluationprocessyieldsestimatesoflikelihoodandmagnitudeofeffectsforacuteexposures.Fortheestimateofacuterisks,adistributionofestimatedexposureandadistributionoflethaleffectsarecombinedthrougha2‐DMCAtoobtainadistributionofjointprobabilityfunctions.Fortheestimateofchronicrisks,adistributionofexposureconcentrationsiscomparedtoachronicmeasurementendpoint.Theriskanalysisforchroniceffectsprovidesinformationonlyontheprobabilitythatthechronicendpointassessedwillbeexceeded,notonthemagnitudeofthechroniceffectexpected.
ResultsofAnalysis.AspartoftheprocessofdevelopingandimplementingaprobabilisticapproachforERA,acasestudywascompleted.ThecasestudyinvolvedbothDRAsandPRAsforeffectsofChemXonbirdsandaquaticspecies.ThedeterministicscreenconductedonChemXconcludedqualitativelythatitcouldposeahighrisktobothfreshwaterfishandinvertebratesandshowedthatPRAwaswarranted.Basedontheprobabilisticanalysis,itwasconcludedthattheuseofChemXwasexpectedtoinfrequentlyresultinsignificantfreshwaterfishmortalitiesbutroutinelyresultinreducedgrowthandotherchroniceffectsinexposedfish.Substantialmortalitiesandchroniceffectstosensitiveaquaticinvertebrateswerepredictedtooccurroutinelyafterpeakexposures.
ManagementConsiderations.Initsreviewofthecasestudy,theFIFRASAPcongratulatedtheAgencyontheeffortmadetoconductPRAofpesticideeffectsinecosystems.Thepanelcommentedthattheapproachhadprogressedgreatlyfromearlierefforts,andthattheintricacyofthemodelswassurprisinglygood,giventhetimeintervalinwhichtheAgencyhadtocompletethetask.
80
Followingthecasestudy,EPArefinedthemodelsbasedontheSAPcomments.Inaddition,theterrestrialLevelIImodelwasrefinedtoincludedermalandinhalationexposure.
SelectedReferences.Anoverviewofthemodelsisavailableathttp://www.epa.gov/oppefed1/ecorisk/fifrasap/rra_exec_sum.htm#Terrestrial.
Case Study 14: Expert Elicitation of Concentration-Response Relationship Between Fine Particulate Matter Exposure and Mortality In2002,theNRCrecommendedthatEPAimproveitscharacterizationofuncertaintyinthebenefitsassessmentforproposedregulationsofairpollutants.NRCrecommendedthatprobabilitydistributionsforkeysourcesofuncertaintybedevelopedusingavailableempiricaldataorthroughformalelicitationofexpertjudgments.AkeycomponentofEPA’sapproachforassessingpotentialhealthbenefitsassociatedwithairqualityregulationstargetingemissionsofPM2.5anditsprecursorsistheeffectofchangesinambientPM2.5levelsonmortality.Avoidedprematuredeathsconstitute,onamonetarybasis,between85and95percentofthemonetizedbenefitsreportedinEPA’sretrospectiveandprospectiveSection812Abenefit‐costanalysesoftheCleanAirAct(CAA;USEPA1997eand1999)andinRegulatoryImpactAnalysis(RIA)forrulessuchastheHeavyDutyDieselEngine/FuelRule(USEPA2000c)andtheNon‐RoadDieselEngineRule(USEPA2004).InresponsetotheNRCrecommendation,EPAconductedanexpertelicitationevaluationoftheconcentration‐responserelationshipbetweenPM2.5exposureandmortality.
ProbabilisticAnalysis.Thiscasestudyprovidesanexampleoftheuseofexpertelicitation(Group3)toderiveprobabilisticestimatesoftheuncertaintyinoneelementofacost‐benefitanalysis.Expertelicitationusescarefullystructuredinterviewstoelicitfromeachexpertabestestimateofthetruevalueforanoutcomeorvariableofinterest,aswellastheiruncertaintyaboutthetruevalue.Thisuncertaintyisexpressedasasubjectiveprobabilisticdistributionofvaluesandreflectseachexpert’sinterpretationoftheoryandempiricalevidencefromrelevantdisciplines,aswellastheirbeliefsaboutwhatisknownandnotknownaboutthesubjectofthestudy.ForthePM2.5expertelicitation,theprocessconsistedofdevelopmentofanelicitationprotocol,selectionofexperts,developmentofabriefingbook,conductofelicitationinterviews,theuseofexpertinputpriortoandfollowingindividualelicitationofjudgmentsandtheexpertjudgmentsthemselves.Theelicitationinvolvedpersonalinterviewswith12healthexpertswhohadconductedresearchontherelationshipbetweenPM2.5exposuresandmortality.
ThemainquantitativequestionaskedeachexperttoprovideaprobabilisticdistributionfortheaverageexpecteddecreaseinU.S.annual,adultandall‐causemortalityassociatedwitha1μg/m3decreaseinannualaveragePM2.5levels.Whenansweringthemainquantitativequestion,eachexpertwasinstructedtoconsiderthatthetotalmortalityeffectofa1μg/m3decreaseinambientannualaveragePM2.5mayreflectreductionsinbothshort‐termpeakandlong‐termaverageexposurestoPM2.5.Eachexpertwasaskedtoaggregatetheeffectsofbothtypesofchangesintheiranswers.Theexpertsweregiventheoptiontointegratetheirjudgmentsaboutthelikelihoodofacausalrelationshiporthresholdintheconcentration‐responsefunctionintotheirowndistributionsortoprovideadistribution“conditionalon”oneorbothofthesefactors.
ResultsofAnalysis.TheprojectteamdevelopedtheinterviewprotocolbetweenOctober2004andJanuary2006.DevelopmentoftheprotocolwasinformedbyanApril2005symposiumheldbytheprojectteam,wherenumeroushealthscientistsandanalystsprovidedfeedback;detailedpretestingwithindependentEPAscientistsinNovember2005;anddiscussionwiththeparticipatingexpertsatapre‐elicitationworkshopinJanuary2006.TheelicitationinterviewswereconductedbetweenJanuaryandApril2006.Followingtheinterviews,theexpertsreconvenedforapost‐elicitation
81
workshopinJune2006,inwhichtheprojectteamanonymouslysharedtheresultsofallexpertswiththegroup.
ThemedianestimatesforthePM2.5mortalityrelationshipweregenerallysimilartoestimatesderivedfromthetwoepidemiologicalstudiesmostoftenusedinbenefitsassessment.However,inalmostallcases,thespreadoftheuncertaintydistributionselicitedfromtheexpertsexceededthestatisticaluncertaintyboundsreportedbythemostinfluentialepidemiologicstudies,suggestingthattheexpertelicitationprocesswassuccessfulindevelopingmorecomprehensiveestimatesofuncertaintyforthePM2.5mortalityrelationship.TheuncertaintydistributionsforPM2.5
concentration‐responseresultingfromtheexpertelicitationprocesswereusedintheRIAfortherevisedNAAQSstandardforPM2.5(promulgatedonSeptember21,2006).BecausetheNAAQSareexclusivelyhealth‐basedstandards,thisRIAplayednopartinEPA’sdeterminationtorevisethePM2.5NAAQS.BenefitsestimatesintheRIAwerepresentedasrangesandincludedadditionalinformationonthequantifieduncertaintydistributionssurroundingthepointsontheranges,derivedfrombothepidemiologicalstudiesandtheexpertelicitationresults.OMB’sreviewoftheRIAwascompletedinMarch2007.
ManagementConsiderations.TheNAAQSareexclusivelyhealth‐basedstandards,sotheseanalyseswerenotusedinanymannerbyEPAindeterminingwhethertorevisetheNAAQSforPM2.5.Moreover,therequesttoengageintheexpertelicitationdidnotcomefromtheCASAC,theofficialpeerreviewbodyfortheNAAQS;adecisiontoconducttheanalysesdoesnotreflectCASACadvicethatsuchanalysesinformNAAQSdeterminations.TheanalysesaddressedcommentsfromtheNRCthatrecommendedthatprobabilitydistributionsforkeysourcesofuncertaintybeaddressed.TheanalyseswereusedinEPA’sretrospectiveandprospectiveSection812Abenefit‐costanalysesoftheCAA(USEPA1997eand1999)andinRIAsforrulessuchastheHeavyDutyDieselEngine/FuelRule(USEPA2000c)andtheNon‐RoadDieselEngineRule(USEPA2004).InresponsetotheNRCrecommendation,EPAconductedanexpertelicitationevaluationoftheconcentration‐responserelationshipbetweenPM2.5exposureandmortality.
SelectedReference.Theassessmentisavailableathttp://www.epa.gov/ttn/ecas/ria.html.
Case Study 15: Expert Elicitation of Sea-Level Rise Resulting From Global Climate Change TheUnitedNationsFrameworkConventiononClimateChangerequiresnationstoimplementmeasuresforadaptingtorisingsealevelandothereffectsofchangingclimate.Todecideonanappropriateresponse,coastalplannersandengineersweighthecostofthesemeasuresagainstthelikelycostoffailingtoprepare,whichdependsontheprobabilityofthesearisingaparticularamount.TheU.S.NationalAcademyofEngineeringrecommendedthatassessmentsofsealevelriseshouldprovideprobabilityestimates.Coastalengineersregularlyemployprobabilityinformationwhendesigningstructuresforfloods,andcourtsuseprobabilitiestodeterminethevalueoflandexpropriatedbyregulations.This1995casestudydescribesthedevelopmentofaprobabilitydistributionforsealevelrise,usingmodelsemployedbypreviousassessments,aswellastheexpertopinionsof20climateandglaciologyreviewersabouttheprobabilitydistributionsforparticularmodelcoefficients.
ProbabilisticAnalysis.Thiscasestudyprovidesanexamplebothofananalysisdescribingtheprobabilityofsealevelrise,aswellasanexpertelicitationofthelikelihoodofparticularmodelsandprobabilitydistributionsofthecoefficientsusedbythosemodelstopredictfuturesealevelrise(Group3).Theassessmentoftheprobabilityofsealevelriseusedexistingmodelsdescribingthechangeinfivecomponentsofsealevelriseassociatedwithgreenhousegas‐relatedclimatechange(thermalexpansion,smallglaciers,polarprecipitation,meltingandicedischargefromGreenland
82
andicedischargefromAntarctica).Toprovideastartingpointfortheexpertelicitation,initialprobabilitydistributionswereassignedtoeachmodelcoefficientbasedonthepublishedliterature.
Aftertheinitialprobabilisticassessmentwascompleted,thedraftreportwascirculatedtoexpertreviewersconsideredmostqualifiedtorenderjudgmentsonparticularprocessesforrevisedestimatesofthelikelihoodofparticularmodelsandthemodelcoefficients’probabilitydistributions.Expertsrepresentingbothextremesofclimatechangescience(thosewhopredictedtrivialconsequencesandthosewhopredictedcatastrophiceffects;individualswhosethoughtshadbeenexcludedfrompreviousassessments)wereincluded.Theexpertswereaskedtoprovidesubjectiveassessmentsoftheprobabilitiesofvariousmodelsandmodelcoefficients.Thesesubjectiveprobabilityestimateswereusedinplaceoftheinitialprobabilitiesinthefinalmodelsimulations.Differentrevieweropinionswerenotcombinedtoproduceasingleprobabilitydistributionforeachparameter;instead,eachreviewer’sopinionswereusedinindependentiterationsofthesimulation.Thegroupofsimulationsresultedintheprobabilitydistributionofsealevelrise.
ResultsofAnalysis.Theanalysis,completedwithabudgetof$100,000,providedaprobabilisticestimateofsealevelriseforusebycoastalengineersandregulators.Theresultssuggestedthatthereisa65percentchancethatthesealevelwillrise1millimeter(mm)peryearmorerapidlyinthenext30yearsthanithasbeenrisinginthelastcentury.Undertheassumptionthatnonclimaticfactorsdonotchange,theprojectionssuggestedthatthereisa50percentchancethattheglobalsealevelwillrise45centimeters(cm),anda1percentchanceofa112cmrisebytheyear2100.Themedianpredictionofsealevelrisewassimilartothemidpointestimateof48cmpublishedbytheIntergovernmentalPanelonClimateChange(IPCC)shortlythereafter(IPCC1996).Thereportalsofounda1percentchanceofa4to5meterriseoverthenext2centuries.
ManagementConsiderations.Therearetworeports(USEPA1995c;TitusandNarayanan1996)thatdiscussseveralusesoftheresultsofthisstudy.Byprovidingaprobabilisticrepresentationofsealevelrise,theassessmentallowscoastalresidentstomakedecisionswithrecognitionoftheuncertaintyassociatedwithpredictedchange.Rollingeasementsthatvestwhenthesearisestoaparticularlevelcanbeproperlyvaluedinboth“arms‐length”transactionsalesorwhencalculatingtheallowabledeductionforacharitablecontributionoftheeasementtoaconservancy.Cost‐benefitassessmentsofalternativeinfrastructuredesigns—whichalreadyconsiderfloodprobabilities—alsocanconsidersealevelriseuncertainty.Assessmentsofthebenefitsofpreventinganaccelerationofsealevelrisecanincludemorereadilylow‐probabilityoutcomes,whichcanprovideabetterassessmentofthetrueriskwhenthedamagefunctionisnonlinear,whichoftenisthecase.
SelectedReferences.
USEPA(U.S.EnvironmentalProtectionAgency).1995c.TheProbabilityofSeaLevelRise.EPA/230/R‐95/008.Washington,D.C.:ClimateChangeDivision,USEPA.http://nepis.epa.gov/Exe/ZyPURL.cgi?Dockey=20011G1O.txt
IPCC(IntergovernmentalPanelonClimateChange).1996.ClimateChange1995:TheScienceofClimateChange.ContributionofWorkingGroupItotheSecondAssessmentoftheIntergovernmentalPanelonClimateChange.Cambridge:CambridgeUniversityPress.
Titus,J.G.,andV.Narayanan.1996.“TheRiskofSeaLevelRise.”ClimaticChange33(2):151–212.
Case Study 16: Knowledge Elicitation for a Bayesian Belief Network Model of Stream Ecology Theidentificationofthecausalpathwaysleadingtostreamimpairmentisacentralchallengetounderstandingecologicalrelationships.Bayesianbeliefnetworks(BBNs)areapromisingtoolfor
83
modelingpresumedcausalrelationships,providingamodelingstructurewithinwhichdifferentfactorsdescribingtheecosystemcanbecausallylinkedandcalculatinguncertaintiesexpressedforeachlinkage.
BBNscanbeusedtomodelcomplexsystemsthatinvolveseveralinterdependentorinterrelatedvariables.Ingeneral,aBBNisamodelthatevaluatessituationswheresomeinformationalreadyisknown,andincomingdataareuncertainorpartiallyunavailable.Theinformationisdepictedwithinfluencediagramsthatpresentasimplevisualrepresentationofadecisionproblem,forwhichquantitativeestimatesofeffectprobabilitiesareassigned.Assuch,BBNshavethepotentialforrepresentingecologicalknowledgeanduncertaintyinaformatthatisusefulforpredictingoutcomesfrommanagementactionsorfordiagnosingthecausesofobservedconditions.Generally,specificationofaBBNcanbeperformedusingavailableexperimentaldata,literaturereviewinformation(secondarydata)andexpertelicitation.AttemptstospecifyaBBNforthelinkagebetweenfinesedimentloadandmacroinvertebratepopulationsusingdatafromliteraturereviewshavefailedbecauseoftheabsenceofconsistentconceptualmodelsandthelackofquantitativedataorsummarystatisticsneededforthemodel.Inlightofthesedeficiencies,aneffortwasmadetouseexpertelicitationtospecifyaBBNfortherelationshipbetweenfinesedimentloadresultingfromhumanactivityandpopulationsofmacroinvertebrates.ThegoalsofthiseffortweretoexaminewhetherBBNsmightbeusefulformodelingstreamimpairmentandtoassesswhetherexpertopinioncouldbeelicitedtomaketheBBNapproachusefulasamanagementtool.
ProbabilisticAnalysis.ThiscasestudyprovidesanexampleofexpertelicitationinthedevelopmentofaBBNmodeloftheeffectofincreasedfinesedimentloadinastreamonmacroinvertebratepopulations(Group3).Forthepurposeofthisstudy,astreamsetting(aMidwestern,low‐gradientstream)andascenarioofimpairment(introductionofexcessfinesediment)wereused.Fivestreamecologistswithexperienceinthespecifiedgeographicsettingwereinvitedtoparticipateinanelicitationworkshop.Aninitialmodelwasdepictedusinginfluencediagrams,withthegoalsofstructuringandspecifyingthemodelusingexpertelicitation.Theecologistswereguidedthroughaknowledgeelicitationsessioninwhichtheydefinedfactorsthatdescribedrelevantchemical,physicalandbiologicalaspectsoftheecosystem.Theecologiststhendescribedhowthesefactorswereconnectedandwereaskedtoprovidesubjective,quantitativeestimatesofhowdifferentattributesofthemacroinvertebrateassemblagewouldchangeinresponsetoincreasedlevelsoffinesediment.Elicitedinputwasusedtorestructurethemodeldiagramandtodevelopprobabilisticestimatesoftherelationshipsamongthevariables.
ResultsofAnalysis.Theelicitedinputwascompiledandpresentedintermsofthemodelasstructuredbythestreamecologistsandtheirmodelspecifications.TheresultswerepresentedbothasrevisedinfluencediagramsandwithBayesianprobabilistictermsrepresentingtheelicitedinput.Thestudyyieldedseveralimportantlessons.Amongthesewerethattheelicitationprocesstakestime(includinganinitialsessionbyteleconferenceaswellasa3‐dayworkshop),definingascenariowithanappropriatedegreeofdetailiscriticalandelicitationofcomplexecologicalrelationshipsisfeasible.
ManagementConsiderations.Thestudywasconsideredsuccessfulforseveralreasons.First,thefeasibilityoftheelicitationapproachtobuildingstreamecosystemmodelswasdemonstrated.Thestudyalsoresultedinthedevelopmentofnewgraphicaltechniquestoperformtheelicitation.TheelicitedinputwasinterpretedintermsofpredictivedistributionstosupportfittingacompleteBayesianmodel.Furthermore,thestudywassuccessfulinbringingtogetheragroupofexpertsinaparticularsubjectareaforthepurposeofsharinginformationandlearningaboutexpertelicitationinsupportofmodelbuilding.Theexerciseprovidedinsightsintohowbesttoadaptknowledgeelicitationmethodstoecologicalquestionsandinformedtheassembledstreamecologistsontheelicitationprocessandonthepotentialbenefitsofthismodelingapproach.Theexplicit
84
quantificationofuncertaintyinthemodelnotonlyenhancestheutilityofthemodelpredictions,butalsocanhelpguidefutureresearch.
SelectedReferences.
Black,P.,T.Stockton,L.Yuan,D.Allan,W.Dodds,L.Johnson,M.Palmer,B.Wallace,andA.Stewart.2005.“UsingKnowledgeElicitationtoInformaBayesianBeliefNetworkModelofaStreamEcosystem.”Eos,Transactions,AmericanGeophysicalUnion86(18),JointAssemblySupplement,Abstract#NB41E‐05.
Yuan,L.2005.“TI:ABayesianApproachforCombiningDataSetstoImproveEstimatesofTaxonOptima.”Eos,Transactions,AmericanGeophysicalUnion86(18),JointAssemblySupplement,Abstract#NB41E‐04.
85
CASE STUDY REFERENCES Black,P.,T.Stockton,L.Yuan,D.Allan,W.Dodds,L.Johnson,M.Palmer,B.Wallace,andA.Stewart.2005.“UsingKnowledgeElicitationtoInformaBayesianBeliefNetworkModelofaStreamEcosystem.”Eos,Transactions,AmericanGeophysicalUnion86(18),JointAssemblySupplement,Abstract#NB41E‐05.
Burke,J.,M.Zufall,andH.Özkaynak.2001.“APopulationExposureModelforParticulateMatter:CaseStudyResultsforPM2.5inPhiladelphia,PA.”JournalofExposureAnalysisandEnvironmentalEpidemiology11(6):470–89.
Connelly,N.A.,T.L.Brown,andB.A.Knuth.1990.NewYorkStatewideAnglerSurvey1988.Albany,NY:BureauofFisheries,NewYorkDepartmentofEnvironmentalConservation.
Georgepoulos,P.G.,S.W.Wang,V.M.Vyas,Q.Sun,J.Burke,R.Vedantham,T.McCurdy,andH.Özkaynak.2005.“ASource‐to‐DoseAssessmentofPopulationExposuretoFinePMandOzoneinPhiladelphia,PA,DuringaSummer1999Episode.”JournalofExposureAnalysisandEnvironmentalEpidemiology15(5):439–57.
IPCC(IntergovernmentalPanelonClimateChange).1996.ClimateChange1995:TheScienceofClimateChange.ContributionofWorkingGroupItotheSecondAssessmentoftheIPPC.Cambridge:CambridgeUniversityPress.
Jamieson,D.1996.“ScientificUncertaintyandthePoliticalProcess.”AnnalsoftheAmericanAcademyofPoliticalandSocialScience545:35–43.
Kurowicka,D.,andR.Cooke.2006.UncertaintyAnalysisWithHighDimensionalDependentModeling.WileySeriesinProbabilityandStatistics.NewYork:JohnWiley&Sons.
Langstaff,J.E.2007.AnalysisofUncertaintyinOzonePopulationExposureModeling.OfficeofAirQualityPlanningandStandardsStaffMemorandumtoOzoneNAAQSReviewDocket.OAR‐2005‐0172.Washington,D.C.:USEPA.http://www.epa.gov/ttn/naaqs/standards/ozone/s_ozone_cr_td.html.
OlsenAR,KincaidTM,PaytonQ.2012.Spatiallybalancedsurveydesignsfornaturalresources.In:GitzenRA,MillspaughJJ,CooperAB,LichtDS(eds)DesignandAnalysisofLong‐TermEcologicalMonitoringStudies.CambridgeUniversityPress,Cambridge,UK,pp126‐150
Stahl,C.H.,andA.J.Cimorelli.2005.“HowMuchUncertaintyIsTooMuchandHowDoWeKnow?ACaseExampleoftheAssessmentofOzoneMonitorNetworkOptions.”RiskAnalysis25(5):1109–20.
Titus,J.G.andV.Narayanan.1996.“TheRiskofSeaLevelRise:ADelphicMonteCarloAnalysisinWhichTwentyResearchersSpecifySubjectiveProbabilityDistributionsforModelCoefficientsWithinTheirRespectiveAreasofExpertise.”ClimaticChange33(2):151–212.
USEPA(U.S.EnvironmentalProtectionAgency).1992a.GuidelinesforExposureAssessment.EPA/600/Z‐92/001.Washington,D.C.:RiskAssessmentForum,USEPA.http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=15263.
USEPA.1992b.FrameworkforEcologicalRiskAssessment.EPA/630/R‐92/001.Washington,D.C.:RiskAssessmentForum,USEPA.http://www.epa.gov/raf/publications/framework‐eco‐risk‐assessment.htm.
USEPA.1995a.GuidanceforRiskCharacterization.Washington,D.C.:SciencePolicyCouncil,USEPA.http://www.epa.gov/spc/pdfs/rcguide.pdf.
86
USEPA.1995b.PolicyonEvaluatingHealthRiskstoChildren.Washington,D.C.:SciencePolicyCouncil,USEPA.http://www.epa.gov/spc/pdfs/memohlth.pdf.
USEPA.1995c.TheProbabilityofSeaLevelRise.EPA/230/R‐95/008.Washington,D.C.:OfficeofPolicy,Planning,andEvaluation,USEPA.http://nepis.epa.gov/Exe/ZyPURL.cgi?Dockey=20011G1O.txt.
USEPA.1997a.PolicyforUseofProbabilisticAnalysisinRiskAssessmentattheU.S.EnvironmentalProtectionAgency.Washington,D.C.:SciencePolicyCouncil,USEPA.http://www.epa.gov/spc/pdfs/probpol.pdf.
USEPA.1997b.GuidingPrinciplesforMonteCarloAnalysis.EPA/630/R‐97/001.Washington,D.C.:RiskAssessmentForum,USEPA.http://www.epa.gov/raf/publications/pdfs/montecar.pdf.
USEPA.1997c.GuidanceonCumulativeRiskAssessment.Part1:PlanningandScoping.Washington,D.C.:SciencePolicyCouncil,USEPA.http://www.epa.gov/spc/pdfs/cumrisk2.pdf.
USEPA.1997d.ExposureFactorsHandbook.Washington,D.C.:NationalCenterforEnvironmentalAssessment,USEPA.http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=12464.
USEPA.1997e.FinalReporttoCongressonBenefitsandCostsoftheCleanAirAct,1970to1990.EPA/410/R‐97/002.Cincinnati,OH:OfficeofAirandRadiation,USEPA.
USEPA.1998.GuidelinesforEcologicalRiskAssessment.EPA/630/R‐95/002F.Washington,D.C.:RiskAssessmentForum,USEPA.http://www.epa.gov/raf/publications/pdfs/ECOTXTBX.PDF.
USEPA.1999.TheBenefitsandCostsoftheCleanAirAct1990to2010:EPAReporttoCongress.EPA/410/R‐99/001.Washington,D.C.:OfficeofAirandRadiation,USEPA.http://www.epa.gov/oar/sect812/1990‐2010/chap1130.pdf.
USEPA.2000a.EPASciencePolicyCouncilHandbook:RiskCharacterization.EPA/100/B‐00/002.Washington,D.C.:SciencePolicyCouncil,USEPA.http://www.epa.gov/spc/pdfs/rchandbk.pdf.
USEPA.2000b.Phase2Report:FurtherSiteCharacterizationandAnalysis.Volume2F—RevisedHumanHealthRiskAssessment,HudsonRiverPCBsReassessmentRI/FS.NewYork:Region2,USEPA.http://www.epa.gov/hudson/revisedhhra‐text.pdf.
USEPA.2000c.FinalHeavyDutyEngine/DieselFuelRule:AirQualityEstimation,SelectedHealthandWelfareBenefitsMethods,andBenefitAnalysisResults.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttnecas1/regdata/Benefits/tsdhddv8.pdf.
USEPA.2001.RiskAssessmentGuidanceforSuperfund:VolumeIII—PartA,ProcessforConductingProbabilisticRiskAssessment.EPA/540/R‐02/002.Washington,D.C.:OfficeofEmergencyandRemedialResponse,USEPA.http://www.epa.gov/oswer/riskassessment/rags3adt/pdf/rags3adt_complete.pdf.
USEPA.2002.EMAPResearchStrategy.EPA620/R‐02/002.ResearchTrianglePark,NC:EnvironmentalMonitoringandAssessmentProgram,NHEERL,USEPA.http://www.epa.gov/nheerl/emap/files/emap_research_strategy.pdf.
USEPA.2004.AnExaminationofEPARiskAssessmentPrinciplesandPractices.EPA/100/B‐04/001.Washington,D.C.:OfficeoftheScienceAdvisor,USEPA.http://www.epa.gov/osa/pdfs/ratf‐final.pdf.
USEPA.2005.OzoneHealthAssessmentPlan:ScopeandMethodsforExposureAnalysisandRiskAssessment.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/naaqs/standards/ozone/data/o3_health_assessment_plan_april05.pdf.
87
USEPA.2006.TotalRiskIntegratedMethodology(TRIM)—AirPollutantsExposureModelDocumentation(TRIM.Expo/APEX,Version4)VolumeI:User’sGuide;VolumeII:TechnicalSupportDocument.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/fera/human_apex.html.
USEPA.2007a.ReviewofNationalAmbientAirQualityStandardsforOzone:PolicyAssessmentofScientificandTechnicalInformation—OAQPSStaffPaper.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/naaqs/standards/ozone/data/2007_07_ozone_staff_paper.pdf.
USEPA.2007b.OzonePopulationExposureAnalysisforSelectedUrbanAreas.EPA/452/R‐07‐010.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/naaqs/standards/ozone/data/2007_07_o3_exposure_tsd.pdf.
USEPA.2014.FrameworkforHumanHealthRiskAssessmenttoInformDecisionMaking.EPA/100/R‐14/001.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.http://www.epa.gov/raf/frameworkhhra.htm
USGAO(U.S.GovernmentAccountabilityOffice).2000.WaterQuality:KeyEPAandStateDecisionsLimitedbyInconsistentandIncompleteData.GAO/RCED‐00‐54.Washington,D.C.:USGAO.http://www.environmental‐auditing.org/Portals/0/AuditFiles/useng00ar_ft_key_epa.pdf.
Xue,J.,V.G.Zartarian,H.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.“AProbabilisticArsenicExposureAssessmentforChildrenWhoContactChromatedCopperArsenate(CCA)‐TreatedPlaysetsandDecks,Part2:SensitivityandUncertaintyAnalyses.”RiskAnalysis26(2):533–41.
Yuan,L.2005.“TI:ABayesianApproachforCombiningDataSetstoImproveEstimatesofTaxonOptima.”Eos,Transactions,AmericanGeophysicalUnion86(18),JointAssemblySupplement,Abstract#NB41E‐04.
Zartarian,V.G.,J.Xue,H.A.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.AProbabilisticExposureAssessmentforChildrenWhoContactCCA‐TreatedPlaysetsandDecksUsingtheStochasticHumanExposureandDoseSimulationModelfortheWoodPreservativeScenario(SHEDS‐Wood),FinalReport.EPA/600/X‐05/009.Washington,D.C.:USEPA.
Zartarian,V.G.,J.Xue,H.A.Özkaynak,W.Dang,G.Glen,L.Smith,andC.Stallings.2006.“AProbabilisticArsenicExposureAssessmentforChildrenWhoContactChromatedCopperArsenate(CAA)‐TreatedPlaysetsandDecks,Part1:ModelMethodology,VariabilityResults,andModelEvaluation.”RiskAnalysis26(2):515–31.