+ All Categories
Home > Documents > Washington State IV-E Waiver Demonstration Project · Washington State IV-E Waiver ... Washington...

Washington State IV-E Waiver Demonstration Project · Washington State IV-E Waiver ... Washington...

Date post: 18-Aug-2018
Category:
Upload: lamkiet
View: 215 times
Download: 0 times
Share this document with a friend
29
Washington State IV-E Waiver Demonstration Project Family Assessment Response Interim Evaluation Report Submitted: December 2016 4450 Arapahoe Avenue, Suite 100, Boulder, CO 80303
Transcript

Washington State IV-E Waiver Demonstration Project

Family Assessment Response Interim Evaluation Report

Submitted:

December 2016

4450 Arapahoe Avenue, Suite 100, Boulder, CO 80303

Family Assessment Response: Interim Evaluation Report 1

Introduction and Background Thisreportdescribesthefirsttwoyears(January2014throughDecember2015)ofWashingtonState’simplementationoftheFamilyAssessmentResponse(FAR)asaTitleIV-EWaiverdemonstrationproject.Inaddition,preliminaryoutcomeandcostdataarepresentedanddiscussed.WashingtonState’sTitleIV-EWaiverDemonstrationProjectfocusesontheimplementationofFamilyAssessmentResponse(FAR),adifferentialresponsepathwayforscreened-inallegationsofabuseandneglectasanalternativetotraditionalChildProtectiveServices(CPS)investigations.TheoriginalFARframeworkoutlinedspecificstepstobetakenbytheDepartmentofSocialandHealthServices(DSHS)tofocuschildwelfareresourcesonfourareastoimproveoutcomesforsafety,permanency,andwell-being:

1. Increasedconnectionswithextendedfamily,naturalsupports,andcommunitytoenhancechildsafetybyengagingfamiliesoutsideofthetraditionalinvestigativeprocess.Byofferingservicesandsupportwithoutaformal“finding”regardingchildabuseorneglect,thestatehopesfamilieswillbemoreopentoacceptingservices.

2. Provisionofconcretegoodsandservicestosupportfamilies,safelypreventplacementinout-of-homecare,safelyreunifychildrenwiththeirfamilies,andimprovechildandfamilywell-being.

3. Expandeduseofevidence-basedpracticestoprovidetargetedinterventionsthateffectivelyaddresstheneedsofchildrenandtheirfamilies,improvechildsafetyinthehome,preventout-of-homeplacement,andincreasechildandfamilywell-being.

4. ExpansionofWashingtonState’spracticemodels,specifically,SolutionBasedCasework1andtheSafetyFramework.

TargetPopulation:FARfocusesonchildrenandtheirfamilieswhoarereported(screenedin)toCPSforneglectandlow-to-moderatephysicalabusewithanon-emergent,72-hourresponsetime.TheFARimplementationandevaluationhasbenefitedfromthedevelopmentandimplementationoftwodistinctStructuredDecisionMaking(SDM)tools:anIntakeToolandaRiskAssessmentTool.Intake&RiskAssessmentTool:TheWashingtonStateChildren’sAdministration(CA)workedwiththeChildren’sResearchCenter(CRC)todevelopanSDMIntakeTooltodeterminewhichfamiliesareeligibleforFAR.Thistoolguidesintakeworkersthroughaseriesofquestionsaimedtodeterminewhetherthereisanallegationofchildabuseorneglectasdefinedinstatestatute.OnceacasescreensinforaCPSresponse,theSDMIntakeToolhelpsintakestaffdetermine

1Children’sAdministrationmadechangestopracticemodelsduringtheFARimplementation.Thisisdiscussedintheimplementationsectionofthisreport.

Family Assessment Response: Interim Evaluation Report 2

whetheraninvestigativeorFARresponseisappropriateforthefamily.AnexistingSDMOverallRiskAssessmentToolhasalsobeenutilizedintheFARandinvestigativepathwaystohelpdeterminefamilyriskfactorsandneedsforservices.InOctoberof2013,theCAtrainedintakestaffintheimplementationoftheFARpathway.TheSDMIntakeToolwasfullyimplementedacrossthestateatthattime.ThismeansthatFAReligibilitywasdeterminedforallscreened-inintakesregardlessofwhetheranofficehadbegunFARimplementation.2ThisstatewideintakecreatedtheopportunitytocarefullymatchcomparisongroupsfortheFARevaluation.TheFARpathwayisoptional.Familieschoosetoparticipate,and,unlikemanyotherstatesimplementinganalternativeresponse,participantsmustsignanagreementofparticipation(thisagreementisalsosignedbythecaseworker).TheFARagreementispartoftheenablinglegislationfortheprogram’simplementation.FamilieswhodeclinetoparticipateinFARaretypicallytransferredtotheinvestigativepathway.3Implementationofalternativeresponse(AR)modelsinotherstatesinformedthedevelopmentoftheWashingtonFARmodel.ToprovidecontextforevaluationfindingsregardingtheimplementationandpreliminaryoutcomesofFAR,weattheTriWestGroup(TriWest)reviewedevaluationsofdifferentialresponseeffortsinsixotherstates:Colorado,Illinois,Minnesota,Missouri,Nevada,andNewYork.Thesestateswerechosenfortheirrespectiveprograms’similaritiestotheWashingtonFARmodelandfortheavailabilityofsimilarprocessandoutcomemeasures.WeusedfindingsfromtheseprogramstoinformourevaluationworkandtodiscussfindingswithWashingtonFARstakeholders.Ourreviewrelieddirectlyonformalevaluationsofalternativeresponse(AR)demonstrations(sometimesalsocalled“differentialresponse”).Whilemanystateshaveimplemented—orareintheprocessofimplementing—ARdemonstrationsforchildabuseandneglectcases,evaluationresultswerenotavailableforallstates,eitherbecausesomestateshavenotcompletedformalevaluationscontainingdetailedoutcomeanalysisorbecausewewereunabletoobtainpublishedevaluationresults.Thus,thereviewwasnotintendedtobeacompleteinventoryofoutcomeresultsfromallARdemonstrationsintheUnitedStates.Additionally,whileotherorganizations(suchasCaseyFamilyProgramsandtheQualityImprovementCenter

2ThephasedrolloutofFARinofficesacrossthestateisdiscussedlaterinthisreport.3Insomecases,familiesparticipateintheassessmentprocessundertheFARpathwaybutfailtosigntheFARagreement.Ifthecaseworkerbelievesnofurtherservicesoractionsarenecessary,thecasemaybeclosedwithoutbeingtransferredtotheinvestigativepathway.

Family Assessment Response: Interim Evaluation Report 3

onDifferentialResponse)offerabbreviatedoutcomesummariesofselectedARprograms,wechosetorelyontheoriginalevaluationdocumentsforthepurposesofthisreport.Researchfocusedonaspectsofprogramstructure(includingscope,jurisdiction,intakes,programeligibility,andthestructureoftheintervention),theevaluation(includingsamplingmethodologyandevaluationdesign),anddemonstrationoutcomes(includingre-referralrates,removalrates,caseloadandcaselengthdata,serviceprovision,andcostsofthedemonstration.)Thisreportmostlyomitsqualitativefindingssuchassurveyandinterviewresultsfromfamily,caseworker,administrator,andcommunitymembers,thoughtheresourcescitedinthereportoftencontainadditionaldataconcerningtopicssuchaschangeincaseworkerattitudes,familyengagement,andfamilysatisfactionwithAR.StateswithoutcomespresentedinthisreportincludeColorado,Illinois,Minnesota,Missouri,Nevada,NewYork,NorthCarolina,andOhio.AdditionaleffortsweremadetofindprimarysourcesforprogramsinArizona,Connecticut,Florida,Hawaii,Iowa,Kentucky,Louisiana,Maine,Maryland,Oklahoma,Tennessee,Texas,Vermont,andWyoming.Evaluationsorotherlessformalprimarysourcesofprogramdataforthislattergroupofstatesoftendidnotcontainenoughdetaileddataonprogramoutcomes(e.g.,removalandre-referralrates)towarrantinclusionhere.Additionally,evaluationsforsomeofthesestatesarecurrentlystillinprogress.OverallfindingsfromtheseevaluationswereconsistentwiththeexperiencesofWashingtonState.Findingsrelatedtoparticularoutcomequestionsarecitedineachrelevantsection.StagedRollOutofFARinWashingtonState

TheimplementationofFARinWashingtonStatewasplannedtooccurinmultiplephases.This“phased”approachbecameacentralfeatureoftheFARevaluation.BecauseonlysomeofficesimplementedFARatspecifictimes,familiesreceivingCPSservicesinnon-FARofficesservedasasourceforacomparisongroup.Additionally,thephasedimplementationallowedtheCAtoassesssuccessesandchallengestoimplementationinofficesfromearlyphasesandtomakemid-coursecorrectionstoensurebetterimplementationinlaterphases.Initially,FARwasimplementedinthree“pilot”sites(pleaseseemaponthefollowingpage)inJanuaryof2014.Thesethreesiteswereselectedbasedontheirgeographicallocationsandtheirreadinesstoimplementthenewpathway.ThemaponthefollowingpageshowsthelocationofofficesinwhichFARwasimplemented(markedwithastar)andindicatesthedegreetowhichFARwasavailableinthecounty.CountieswithfullFARavailabilityareindicatedindarkgreen,whilecountieswithsomeFARimplementation(butwheretheentirecountywasnotcovered)areshowninlightgreen.GrayshadingindicatesthatFARwasnotavailableatthetimeofthatspecificrolloutphase.

Family Assessment Response: Interim Evaluation Report 4

Followingthesix-monthpilotsiteimplementation,theCAbeganaddingFARintonewofficeseachquarter.TheofficesidentifiedinthemapbelowbeganimplementingtheFARpathwayinJulyof2014.

Family Assessment Response: Interim Evaluation Report 5

InOctoberof2014,anadditionalfiveofficeswereaddedacrossthestate(pleaseseemapbelow).

Thefinalmap(below)showstheextentofstatewideFARimplementationattheendofthesecondprojectyear(2015).AllremainingofficeswillbeimplementingFARbymid-2017.

EvaluationMethods

ThecomprehensiveevaluationoftheTitleIV-EWaiverProjectincludesanexaminationofprojectprocesses,outcomes,andcostsintheimplementationoftheFARmodel.Themodelisbeingimplementedonarollingbasis,allowingformatchingbetweenlocalofficesimplementingthewaivertonon-FARofficesscheduledtorolloutinlaterquarters.Inadditiontomatchesat

Family Assessment Response: Interim Evaluation Report 6

thelocalofficelevel,wealsomatchedindividualsparticipatinginFARtothosewhowereservedviatraditionalservicesinnon-FARoffices.Specificresearchquestionsaddressedbytheprocessandoutcomeevaluation,aswellthecostanalysis,aredetailedintheappropriatesectionsbelow.Theevaluationisdesignedtoanswerthefollowingquestions:

• HowwastheFARmodelimplemented(descriptive)?• WasthestateabletousethewaivertoimplementFARwithfidelity?• Whatwerethebiggestchallengestoimplementation?• HowdidimplementationchangechildwelfarepracticeinthestateofWashington?• DidtheFARimplementationresultingreaterorlesserdisproportionalityinservices

offeredtofamilies?• DidtheFARimplementationreducechildmaltreatmentinparticipatingfamilies?• DidtheFARimplementationreduceout-of-homeplacement?• DidtheFARimplementationresultinimprovedchildandfamilyfunctioning?• WastheimplementationofFARunderthewaivercost-neutral?

Thetablebelowoutlinesthedatasourcesutilizedforthisevaluation.DataCollectionTool Population ProgramPurpose

FAMLINK

Washington’sSACWISsystem. Alladministrativedata,includingintakesintoFARorInvestigations.

SDMIntakeTool(administeredbyintake)

AllreferralstotheChildren’sAdministration

DetermineeligibilityforFARpathway.

SDMRiskAssessmentTool(afterintakeandFAReligibilitydetermination)(administeredbyallCPScaseworkers)

FARpathwayfamiliesInvestigativepathwayfamilies

Assessfamilyriskfactorsandneedforservices.

FamilySurvey(administeredbyParentAllies)

FARpathwayfamilies

Assessfamilyperspectivearoundkeyprocessandoutcomevariables.

Sitevisitsandkeyinformantinterviews

Caseworkers(FARandinvestigative),supervisorsandadministratorsinallFARimplementingoffices

Collectdataregardingprogramimplementationandfidelity.

Washington’sStateAutomatedChildWelfareInformationSystem(SCWIS)isFAMLINK.TheFAMLINKdatasystemextractsprovideinformationonallreferralstoCPSinthestate.The

Family Assessment Response: Interim Evaluation Report 7

systemwasusedtoidentifyunduplicated4familieswithanintakeduringthestudyperiod(n=91,433).IntakedatainFAMLINKarethenusedtoseparatefamiliesintostudycohorts(treatment,control,excluded)basedonwhether1)theintakeisscreened-inandnota“risk-only”case,5and2)whethertheintakeisFAR-eligible.Thediagramonpage10ofthisreportshowstheflowofthoseintakesintospecifictreatmentandcontrolgroups.InadditiontoadministrativedatafromFAMLINK,TriWestcollectedFARimplementationdatathroughsitevisitsandKeyInformantInterviews,withallcaseworkers(bothFARandInvestigativeworkers),supervisors,andadministrators.Thevisitsandinterviewswereconducted3–4monthsfollowingtheimplementationofFARintherespectiveoffice.Duringthefirsttwoyearsofimplementation,TriWestconducted29sitevisitsand399KeyInformantInterviews.Dataarealsocollectedfromparents/guardianswhoparticipateinFARthroughaFamilySurvey.Atcaseclosure,parents/guardiansaresentacaseclosureletterthatremindsthemthatanevaluationteammembermaycontactthemtocompleteatelephonesurvey.Theletteralsoprovidesinformationforcompletingaweb-basedorautomatedtelephonesurveyifthatmethodispreferred.Eachmonth,CAcompilesalistofallclosuresandsendsTriWestrecentphonenumbersofFARparticipantswhoindicated,intheFARagreement,thattheywerewillingtobecontactedregardingthesurvey.Parentalliescalltheseparents/guardianstoconductthefulltelephonesurveys.FARparentsorguardianswhoparticipateinthefulllivetelephonesurveyareoffereda$10Walmartgiftcardasatokenofappreciation.Thosecompletingtheshorterweb-basedortelephonesurveysareoffereda$5giftcard.Atotalof240surveyswerecompletedduringthefirsttwoyearsofFARimplementation.AfulldescriptionofsurveyresponseratescanbefoundintheDecember2016FamilySurveySummaryreport.FurtherinformationregardingevaluationdatacollectionisprovidedintheFAREvaluationPlanandinthetechnicalappendixtothisdocument(AppendixA).Theevaluationutilizesanintenttotreat(eligibility)design,whichmeansthat,inofficesthathaveimplementedFAR,allfamiliesthatareassignedtotheFARpathwaybytheSDMIntake

4Thestudyidentifiedfamiliesbyfirstintakewithinaspecificstudyperiod(cohort).Whilethecountofintakesisunduplicatedforeachcohort,onefamilymaybecountedagaininasubsequentcohort.5Risk-onlycasesarethosecasesinwhichachildisatimminentriskofharmbutthereisnotChildAbuseorNeglect(CA/N)tobeinvestigated.ThesecaseswouldnotbeassignedtoaCPSInvestigationand,therefore,arenoteligibleforthealternativeFARresponse.Seehttps://www.dshs.wa.gov/ca/2000-child-protective-services/2200-intake-process-and-responseforafulllistofdefinitions.

Family Assessment Response: Interim Evaluation Report 8

Tool,excludingsupervisoroverrides,areincludedintheFARtreatmentgroup.Iffamiliesdeclinetoparticipateorarelatertransferredtotheinvestigativepathwayduetosafetyconcerns,theyarestillincludedinthetreatmentgroup.Becauseofthephasedimplementationandthestatewideuseoftheintaketool,apoolofFAR-eligiblefamiliesbeingservedinofficesthatarenotyetimplementingFARareavailableforinclusioninamatchedcomparisongroup.Utilizingpropensityscorematching,TriWestcreatedacomparisongroupoffamiliesmatchedtoFARfamilieson26demographic,CPS,andriskassessmentvariables.

Thediagrambelowshowstheevaluationdesign.

Offices&with&Family&Assessment&

Response&(FAR)&

Offices&with&Inves9ga9ve&Pathway&Only&

Intent&to&Treat&(Eligible&for&FAR)&

Not&Eligible&for&FAR&

Assessment&Intent&to&Treat&B&Received'FAR'

Intent&to&Treat&B&Opted&Out&of&FAR&

Inves9ga9ve&BMatched&

Comparison&

Inves9ga9ve&B&&&&&Not&Eligible&

Eligibility&Design&

Assessment&Would&have&been&

FAR&eligible&

Would&not&have&been&FAR&eligible&

Family Assessment Response: Interim Evaluation Report 9

FAR(treatment)familiesaregroupedintosix-monthstudycohortsbasedonthedateoftheirfirstFAR-eligibleintakeduringtheperiod.6EachcohortincludesfamiliesservedinalloftheofficesimplementingFARduringtheperiod.Forexample,thefirstcohortincludesallfamiliesservedinthefirstsixmonthsoftheproject(January1,2014throughJune30,2014),whichonlyincludesthefirstthreepilotsites.However,thenextevaluationcohortincludesthefirstthreepilotsites,aswellasthenexttwophasesofoffices(rolledoutJuly2014throughDecember2014).

StudyCohortNumberof

FamilieswithaFARIntake

NumberofSampled7FARGroupFamilies

NumberofMatchedComparisonGroup

Families

Cohort1(Jan–Jun2014)Phase1Offices(pilot)

664 664 664

Cohort2(Jul–Dec2014)Phase1-3Offices

2,630 2,630 2,630

Cohort3(Jan–Jun2015)Phase1-5Offices

5,593 2,000 2,000

Cohort4(Jul–Dec2015)Phase1-5Offices8

5,432 1,000 1,000

Thediagramonthefollowingpageshowstheflowfromintaketoinclusionintoeachofthestudygroups.

6Familieswereonlyincluded/countedoncepercohort,throughaspecificfamilycouldbeincludedinmultiplecohortsduetonewintakes.7BeginningwithCohort3,arandomsampleofFARfamilieswasusedforcomparativeanalysis.AsmoreofficesimplementedFAR,thecomparisonpooloffamiliesinnon-FARofficesbecametoosmalltodrawacomparisongroupthesamesizeasthefullFARgroup.8DuetoadelayinFARimplementation,noadditionalofficesbeganFARimplementationduringtheCohort5studyperiod.

Family Assessment Response: Interim Evaluation Report 10

Far Case Disposition(of 8,897)

Cohort 1 Cohort 2 Cohort 3 Cohort 4 Total

0=Missing 0 0 8 80 881=Remained FAR 597 2,350 4,974 4,786 12,7072=Declined FAR (no investigation)

39 (5.9%)

171 (6.5%)

317 (5.7%)

288 (5.3%)

815

3=Transferred(including investigation)

27 (4.1%)

81 (3.1%)

126 (2.3%)

125 (2.3%)

359

Disposition 15(excluded from analysis)

1 28 168 153 350

Cases Screened Out42,103

(Intake type=0)Cohort 1: 12,470Cohort 2: 10,572Cohort 3: 10,413Cohort 4: 8,648

FAR Cases14,319

(Intake type=1)Cohort 1: 664Cohort 2: 2,630Cohort 3: 5,593Cohort 4: 5,432

Investigative Cases30,102

(Intake type=2)Cohort 1: 12,336Cohort 2: 7,700Cohort 3: 5,673Cohort 4: 4,393

Risk-Only Cases4,197

(Intake type=3)Cohort 1: 1,111Cohort 2: 1,031Cohort 3: 950Cohort 4: 1,105

Missing Values712

(Intake type=NA)Cohort 1: 6Cohort 2: 83Cohort 3: 295Cohort 4: 328

Total Intakes91,433

Cohort 1: 26,587Cohort 2: 22,016Cohort 3: 22,924Cohort 4: 19,906

Cohort Sample PeriodsCohort 1: Jan–Jun, 2014 Cohort 3: Jan–Jun, 2015Cohort 2: Jul–Dec, 2014 Cohort 4: Jul–Dec, 2015

Cases that Would Have Been Eligible for FAR

18,655(Potential Control Observations)Cohort 1: 8,515Cohort 2: 4,953Cohort 3: 3,192Cohort 4: 1,995

Cases Not Eligible for FAR Even If Available

9,065Cohort 1: 2,663Cohort 2: 2,002Cohort 3: 2,163Cohort 4: 2,237

Other Investigative Cases2,382

Cohort 1: 1,158Cohort 2: 745Cohort 3: 318Cohort 4: 161

Family Assessment Response: Interim Evaluation Report 11

Evaluation of Family Assessment Response (FAR) Implementation in Washington State

Asmentionedpreviously,thisreportaddressesFARimplementationandpreliminaryoutcomesforthefirsttwoprogramyears(January2014throughDecember2015).Duringthosetwoyears,TriWestvisitedeachofficeseveralmonthsafterFARimplementationtodiscusssuccesses,challenges,andstaffperceptionsofchangescausedbytheadditionofthenewCPSpathway.Keyinformantinterviews(KIIs)wereconductedwithcaseworkersfrombothFARandinvestigativepathways,9supervisors,administrators,andcommunityserviceproviders.Basedonfindingsfromthesesitevisits,aswellascaseconsultationsandmoreinformaldiscussionswithcaseworkersinthefield,theChildren’sAdministration(CA)madeseveralimportantprogramchangestotheFARimplementation.Thesechangesarediscussedattheendofthissection.CaseworkerReportsofPreparednessforFARImplementation(KII)

OnerecurringthemeininterviewswithbothFARandinvestigativecaseworkersisthatFARseemstobeabetterfitforsomecaseworkersthanothers.TheCAallowedvoluntarytransfersfrominvestigativecaseworktoFARcasework,thus,mostcaseworkersprovidingservicestofamiliesintheFARpathwayhadchosentobeincludedinthatprogram.Thisvoluntaryassignmentlikelybenefittedimplementationascaseworker“buy-in”totheFARmodelwasanimportantfeatureofsuccess.Overallratingsofpreparednessforimplementationwerefairlyhigh,fallingbetween“somewhatprepared”and“mostlyprepared”(or2.7onafour-pointscale).ThesescoreswerethesameforYearOneandYearTwoandwerevirtuallyidenticalforFARcaseworkersandinvestigativeworkers.

9Interviewswithinvestigativecaseworkerswereaddedaftersitevisitstoeachofthethreepilotsites.

2.7 2.72.5 2.6

1.0

2.0

3.0

4.0

FARYr1AverageRa4ng FARYr2AverageRa4ng InvYr1AverageRa4ng InvYr2AverageRa4ng

Ra#ngScale:4-VeryPrepared3-MostlyPrepared2-SomewhatPrepared1-NotPrepared

PreparednessforFARatImplementa#on

Family Assessment Response: Interim Evaluation Report 12

ImplementationSuccessesandChallenges

OfficestaffingpatternsatthetimeoftheFARrolloutseemedtomoststronglyinfluenceimplementation,withfullystaffedofficesreportingsmootherimplementation.Staffvacancies(duetovacations,leave,andordinaryturnover)thatoccurredatthetimeofimplementationcreatedachallengeforstaff.Additionally,trainingwasoriginallyratedsomewhatpoorlybycaseworkers.However,significantchangestothetrainingcurriculumandlanguageusedtodescribeFARweremadeoverthecourseofthefirstprojectyear.Caseworkers’perspectivesofFARtrainingimprovedinYearTwoofimplementation.TwofeaturesoftheFAR-enablinglegislationwerecitedasbarrierstoimplementingFARsuccessfully:therequirementthatfamiliessigntheFARagreementandthe45-daytimelimitformostFARcases.CaseworkersobservedthatsomefamiliesseemedparticularlyreluctanttosigntheFARagreement,eitherbecausetheydidnottrust“thestate”andwereworriedthatwereadmittingtowrongdoing,becauseofadviceofcounsel,orbecauseofanactivechildcustodycaseinwhichaformalfindingwasdesired.WhileitispossibleunderFARtoextendthetimeperiodupto90days,mostcaseworkerstriedtoworkwithintheinitial45-daytimelimit.Someseemedunawareofthepossibilityofextendingthecaseto90days.Caseworkersconsistentlyreportedthatthe45-daytimeperiodwastooshortformostservicesneededbyfamiliesand,inparticular,thatitlimitedtheirabilitytouseevidence-basedpractices(EBPs)becausebythetimeafamilywasreferredandbeganservices,therewasnotenoughtimetocompletetheservice.Asaresult,caseworkersreportedusingfewEBPswithfamilies.Somespecificprovidersdidattempttomodifyprogramstoaccommodateashortenedtimeframe,butthisdidnotsignificantlyresolvetheissue.Overall,caseworkersinYearOnereportedthatthebarriersdescribedabovecauseda“noticeablebarrier”toFARimplementation.However,astrainingforandcommunicationaboutFARimproved,thoseratingsimprovedsomewhatforFARworkers.Investigativeworkerstendedtoratebarriersaslower(“somewhat”comparedto“noticeable”).Theirperspectivesdidnotchangeacrossthetwoyears.

Family Assessment Response: Interim Evaluation Report 13

Despiteimplementationchallengesduringthefirsttwoprogramyears,mostrespondentsacrossofficesfeltthatFARhadledtoarelativelyhighdegreeofpositivechange.ThesechangesweretypicallyrelatedtotheexperiencesofFARfamiliesandFARcaseworkers’abilitytoprovidecommunityservicestomeetfamilies’needs.FARfamiliesweremuchmoreengagedwithsocialworkersoncetheyunderstoodthattheseworkerswerenotseekingafinding.FamiliesalsoappreciatedtheincreasedtransparencyandhonestyinherentintheFARmodel.FamilieswhohadpreviousexperienceswithCPSlikedtheFARpathwaymoreandfelttheyhadbetterexperiencesthroughFAR.RespondentsalsoreportedmorecommunitysupportandcommentedthatcommunitiesarebeginningtoseeCPSmorepositively.Caseworkers,onaverage,aremorefamiliarwithcommunityservicesandarebetterabletoworkwithfamiliestohelpthemmeettheirneedsafterFARimplementation.Asshowninthefigurebelow,bothFARandinvestigativeworkersreported,onaverage,“noticeable”positivechangesintheofficeasaresultofFARimplementation.ThesepositiveratingswerealittlelowerforinvestigativeworkersinYearTwo.

3.02.7

2.32.4

1.0

2.0

3.0

4.0

FARYr1AverageRa2ng FARYr2AverageRa2ng InvYr1AverageRa2ng InvYr2AverageRa2ng

Ra#ngScale:4-VeryMuchaBarrier3-No2ceableBarrier2-SomewhataBarrier1-NotaBarrier

ImpactofBarrierstoImplementa#on

2.8 2.8 2.92.6

1.0

2.0

3.0

4.0

FARYr1AverageRa4ng FARYr2AverageRa4ng InvYr1AverageRa4ng InvYr2AverageRa4ng

Ra#ngScale:4-ExtensiveChanges3-No4ceableChanges2-SomeChanges1-NoChanges

ExtentofPosi#veChanges

Family Assessment Response: Interim Evaluation Report 14

OneparticularreasonforthelowerratingsofpositivechangeinYearTwoisthatsomeinvestigatorsexpressedfrustrationwithnotbeingincludedasmuchastheycouldhavebeenintheFARofficerollout.FARimplementationhadadivisiveeffectwithinsomeoffices.ThishappenedforseveralreasonsbutwasmorepronouncedwheninvestigatorsfeltthatFARwasbeingapproachedasthenewest“great”thingandthattheirinvestigativeworkwaslessvalued.Additionally,shiftingcaseloadsandstaffvacanciesoftencreatedinitialhighcaseloadsthatoftenledtoconflictbetweenthetwogroupswithinsomeoffices.Overall,theresponsetoFARfrominvestigativeteamstendedtobemixed.SometeamsfeltthatsupportandcommunicationtoinvestigatorswasnotapriorityduringFARimplementation.Ascanbeseeninthechartbelow,FARcaseworkersinbothimplementationyearsreportedthatcaseworkerengagementhad“noticeable”change,whileinvestigativeworkersreported,onaverage,lessthan“some”change.

MostrespondentsreportedthatFAROfficeLeadswereabletomakesignificantprogresswithinthecommunityintermsoffindingresourcesandeducatingvariousstakeholdergroupsaboutCPSandtheFARmodel.Insomeoffices,theFAROfficeLeaddepartedafterthefirstseveralmonthsofimplementation.Caseworkersreportedtheseearlydeparturesashavingadetrimentalimpactontheirworkandtheofficerelationshipwiththecommunity.However,otherofficesreportedthatstrategiesputinplacebysupervisorsandworkershelpedthemcontinuetobuildrelationshipswithinthecommunityandtoidentifyresources.Overall,thefirsttwoyearsofimplementationdemonstratedthatFARofficesneedtoplanfortakingovercommunityoutreachresponsibilitiesoncetheFAROfficeLeadpositionexpires.

2.8 2.8

1.71.4

1.0

2.0

3.0

4.0

FARYr1AverageRa3ng FARYr2AverageRa3ng InvYr1AverageRa3ng InvYr2AverageRa3ng

Ra#ngScale:4-ChangedALot3-No3ceableChange2-SomeChange1-NoChange

HowMuchCaseworkerEngagementChanged

Family Assessment Response: Interim Evaluation Report 15

FamilyPerspectivesofFARImplementation

InadditiontoconductingkeyinformantinterviewsinFARofficestoexamineimplementationchallengesandsuccesses,TriWestalsoworkedwithparentallies(parentswithpreviousCPSinvolvementwhonowworkasfamilyadvocates)tosurveyFARfamiliesregardingtheirviewsofFARprocessesandoutcomes.ThissectionofthereportdiscusseskeyfeaturesoftheFARmodelandfamilies’perceptionsofhowwellthosewereimplemented.Itisimportanttonotethatkeylimitations(e.g.,theoptionalinclusioninthesurvey,problemswithdisconnectedphonenumbers,etc.)existinsurveyingfamilies.10OneimportantfacetofFARistousealessformalapproach(andtheabsenceofaformal“finding”)toincreasetrustandoverallengagementinthecaseprocess.Ascanbeseeninthegraphbelow,mostrespondents(88%)reportedbeingactivelyengagedinthecaseprocess“always,oralmostalways.”

“Iwasactivelyengagedwiththecaseprocess.”(N=231) Twootherimportantratingsaroundfamilyengagementaddressedtheextenttowhichthefamilyfeltthattheiropinionswerebeingconsideredwhendevelopingacaseplanorlinkingthefamilytoservices.Ascanbeshowninthetwochartsonthefollowingpage,morethanhalfoftherespondentsfeltthattheircaseworkerhadhelpedthemtoidentifythingstheirfamilyneeded.Morethantwothirdsreportedthattheircaseworker“alwaysoralmostalways”listenedtotheiropinionsaboutwhetherthefamilyneededservices.

10Surveymethodology,responseratesaswellasmorerecentlysurveydataarereportedintheDecember2016FamilySurveySummaryReport.ThisreportwillbeincludedwiththeWashingtonStateIV-EWaiverJanuary2017Semi-AnnualProgressReport.

AlwaysorAlmostAlways88%

SomeoftheTime7%

NotO7en2%Never3%

Family Assessment Response: Interim Evaluation Report 16

ChangestoFARDuringYearsOneandTwo

Asmentionedpreviously,severalchangestoFARoccurredduringYearsOneandTwoincludingasignificantsetofchangestargetingFARtrainingandcommunicationasbasedoninformationprovidedtotheCAfromourevaluationwork,caseconsultationswithoffices,andmoreinformalcommunicationswiththefield.Therewasclarificationoverthe“place”ofFARinchildwelfare,witharecommendationforstrongermessagingthatthisisstillaCPSresponseandthatchildsafetyneededtocontinuetobethesingularguidingpriorityinallcases.Additionally,trainingwasimprovedtoincludemoreinformationonthecontinuedfocusofchildsafety,clarificationaroundthevoluntarynatureofFAR,andimprovedprocessesforexplainingtheintakeprocessanddecisionmakingaroundtheassignmentofintakestoeithertheFARortheInvestigativepathway.Additionally,thelanguageintheFARagreementwaschanged(andtheagreementitselfshortened)toaddressconcernsthatitwasleadingfamiliestodeclineparticipationinFAR.Earlyindicationsshowthatthesechangeshaveimprovedimplementationinoffices,andweanticipateseeingcontinuedimprovementsintheassessmentofFARimplementationintothefuture.WecontinuetoworkcloselywithCAtodeveloparatingsystemtoassessfidelityofFARimplementationwithinofficesandtodeterminethedegreetowhichimplementationaffectsoutcomes.Thiswillbedetailedinfuturesemi-annualprogressreportsandwillbereportedintheFinalEvaluationReport.

Yes,VeryMuch45%

Yes,ALi3le25%

NotMuch12%

NotAtAll18%

AlwaysorAlmostAlways67%

SomeoftheTime22%

NotO8en5%

Never8%

Mycaseworkerhelpedidentifythingsthatcausedmyfamilyproblems.(N=228)

Mycaseworkerlistenedtowhetherornotmyfamilyneededservices.(N=225)

Family Assessment Response: Interim Evaluation Report 17

Also,CAoriginallyplannedtousetheChildandAdolescentNeedsandStrengths(CANS)tooltohelpdeterminefamilyserviceneeds.However,fewcaseworkersreportedusingthetoolthewayitwasintendedandfurtherreportedthatthetooladdedtoanalreadylengthydatacollectionprocesswithfamilies.Therefore,CAhasrecentlymadethedecisiontodiscontinueuseoftheCANS.OnechangetotheFARmodelthatdoesaffectthenumbersoffamiliesserved(whichwillbereportedinthenextsectionofthisreport)wasthedecisiontomovefamilies(regardlessofrisk)outofFAReligibilityiftheintakeinvolvedaphysicalabuseallegationofachildagedthreeyearsoryounger.ThisadjustmentdecreasedthenumberofFAR-eligiblefamiliesandthusloweredtheactualnumbersserved.AnalysisofMinorityDisproportionalitywithinFAR

Theissueofminoritydisproportionalitywithinthechildwelfaresystem,generally,isimportanttoCA.Thus,thisevaluationexaminedthedegreetowhichdecision-makingregardingFARdifferedacrossracialandethnicgroups.Onceanintake“screens-in”toCPS(whichistosay,theintakeworkerdeterminesthereissufficientinformationforaCPSresponse),theSDMIntakeToolhelpstheintakeworkertodeterminewhetherthecasemeetstheeligibilitycriteriatobereferredtotheFARpathwayor,noteligible,totheinvestigativepathway.Thetablebelowshowsthepercentageofallscreened-inallegationsthatareassignedtotheFARpathwaybyrace/ethnicity.NotethatthesepercentagesareassignmentstoFARacrossthestateeventhoughFARwasnotimplementedinalloffices.Afterfullimplementation,allpercentagesareexpectedtoincrease.

Race/EthnicityPercentof

IntakesAssignedtoFARPathway

TotalIntakes 30%

NativeAmerican 22%

AsianAmerican 28%

Black 29%

White 31%

Hispanic 27%

Multi-racial(Native) 30%

Multi-racial(Black) 33%

Multi-racial(White) 29%

Family Assessment Response: Interim Evaluation Report 18

ThepercentageofintakesthatareassignedtoFAR(asopposedtobeingscreenedtoinvestigationorassignedas“risk-only”)isconsistentacrosstheracial/ethniccategories,exceptforNativeAmericans,whohavealowerproportionofcasesscreeningtoFAR.Whilethedatadonotprovideanexplanationforthisdifference,thereisapossibilitythatthesecasesmaybetransferredovertotribalentitiesthatdonothaveaFARoptionormightbemorelikelytooccurinofficeswhereFARhasnotrolledout.ThisspeculationissupportedbythepercentageoffamiliesassignedtotheFARpathwaywhenchildrenaremembersofaWashingtonStatetribalentity,whichisevenlowerthantheoverallNativeAmericanrateofFARassignment(whichincludesWashingtonStatetribalmembers).TheCAcontinuestomonitorthisdifferenceasimplementationcontinues.DisproportionalityinRemainingintheFARPathway

OnceacaseisassignedtotheFARpathway,thevastmajorityoffamilies(91%)agreetoparticipateandcompletetheircaseunderFAR.However,insomecases,afamilymayeitherrefusetoparticipateormayhaveacasetransferredtoinvestigationsbyaworkerwhobelievesFARisnotanappropriatepathwayduetoaconcernforchildsafety.Thefollowingtableshowsdifferences,byrace/ethnicity,infamilies’pathwaydispositionaftertheirinitialpathwayassignmenttoFAR.

DisproportionalityofFARDisposition–Cohorts1-4(Years1&2)

Race/Ethnicity RemainFAR DeclinedFARTransferfor

Safety

Total 91% 6% 3%

NativeAmerican 86% 10% 4%

AsianAmerican 93% 6% 2%

Black 92% 5% 3%

White 91% 6% 2%

Hispanic 91% 5% 4%

Multi-racial(Native) 90% 6% 4%

Multi-racial(Black) 93% 4% 4%

Multi-racial(White) 92% 5% 2%

Asshowninthetableabove,theproportionofcasestransferredtoinvestigationsarevirtuallythesameforallfamilies.However,NativeAmericanfamiliesaresignificantlymorelikelytodeclinetoparticipateinFAR.IndiscussingthisphenomenonwithFARcaseworkers,TriWestlearnedthattheFARagreement,inparticular,seemedtobeasignificantbarrierforNative

Family Assessment Response: Interim Evaluation Report 19

Americanfamilies.CAisworkingtoamendthelegislativerequirementfortheFARagreementtotrytoalleviatesomeofthedisparityinNativeAmericanfamiliesdecliningtoparticipateinFAR.Preliminary Program Outcomes

ToassesstheimpactofFARonthegoalsofimprovingsafety,permanency,andwell-beingoutcomes,TriWesthasanalyzeddataonnewintakesintoCPSfollowingtheirinitialintakes,childremovalsfromthehome,andfamilyreportsofsuccessfuloutcomes.Dataarereportedforthefirstfourevaluationcohorts(seethetableonpage9forcohortdescriptions).Duetothetimingofintakes,Cohort4hasonlysixmonthsoffollow-updataatthistime.NewAcceptedIntakes

ThetablebelowshowstheproportionofFARandComparisongroupfamilieswithanewacceptedCPSintakewithinthreemonthsfollowingtheirinitialFAR(orinvestigative)case.TheComparisongrouphadasmall(butstatisticallysignificant)lowerproportionofnewintakeswhenconsideringallnewacceptedintakes.FARfamilieshadmorere-referralsingeneral,butmanycontinuedtobeFAR-eligiblereferrals,indicatingthatrisklevelshadbeenstayingthesameforthesefamilies.ComparisongroupfamilieswereeligibleforFARintheirfirstintakebutgenerallyhadfewersubsequentFAR-eligiblereferralsand,insomecases,hadsignificantlymorenon-eligiblereferrals,anindicatorthatthesefamilieswerefacinggreaterchallengeswhentheyreturned(asindicatedbyriskatintake).

FAROutcomes:FamilieswithNewCPSIntakesWithin3MonthsAfterInitialIntake,Cohorts1-4

FARMatched

ComparisonGroup

PercentoffamilieswithanynewacceptedCPSintake 12.9% 11.1%*

PercentoffamilieswithanewFAR-eligibleintake 9.8% 6.9%*

Percentoffamilieswithanewnon-FAR-eligibleintake 4.2% 5.1%*

Percentoffamilieswithanew“risk-only”intake 0.7% 0.7%

*Differencesaresignificantatthep<.05level.

Thesesamepatternsholdfornewintakesat6monthsand12months,asshowninthefollowingtables.Again,theComparisongrouphadalowerproportionoffamilieswithanynewintakes,butthisdifferencewasbeingdrivenentirelybyhavingfewerFAR-eligibleintakes.Comparisongroupfamiliescontinuedtohaveslightlylowerratesofnewnon-FAR-eligibleintakes.

Family Assessment Response: Interim Evaluation Report 20

FAROutcomes:FamilieswithNewCPSIntakesWithin6MonthsAfterInitialIntake,Cohorts1–4

FARMatched

ComparisonGroup

PercentoffamilieswithanynewacceptedCPSintake 19.8% 16.6%*

PercentoffamilieswithanewFAR-eligibleintake 14.7% 10.3%*

Percentoffamilieswithanewnon-FAR-eligibleintake 7.3% 8.2%

Percentoffamilieswithanew“risk-only”intake 1.4% 1.4%

*Differencesaresignificantatthep<.05level.

FAROutcomes:FamilieswithNewCPSIntakes12MonthsAfterInitialIntake,Cohorts1–3

FARMatched

ComparisonGroup

PercentoffamilieswithanynewacceptedCPSintake 28.4% 22.9%*

PercentoffamilieswithanewFAR-eligibleintake 21.5% 14.4%*

Percentoffamilieswithanewnon-FAR-eligibleintake 11.3% 12.1%

Percentoffamilieswithanew“risk-only”intake 2.4% 2.6%

*Differencesaresignificantatthep<.05level.

WhenanalyzingtheseparateeffectsofFARoneachcohort,wefoundthateachcohorthadahigheraveragenumberofacceptedintakesforFARfamilies.Thisincreasewasstatisticallysignificantforonlysomeofthetimeperiods(e.g.,3,6,and12months)andcohorts,anditdidnotpresentanobvioustrend.SeethetechnicalappendixforadetailedanalysisoftheeffectofFARbycohort.Findingsregardingnewintakesvariedthroughouttheotherstatesincludedintheliteraturereview.SomestatesdidfindsignificantimprovementsinnewintakesforFARfamilies,whileothersfoundnochangeorevenincreasednewintakeratesforARfamilies.Severalevaluationsalsoconcludedthatthebestpredictorofre-referralswaswhetherafamilyhadpreviousreferralswithCPS.Accordingtotheseevaluations,whenpredictingthelikelihoodofnewintakes,priorexperiencewithCPSdwarfedtheeffectsofpathway(ARvs.IR).Thisdistinctionisconsistentwithourevaluationfindings.WhenexaminingnewintakesbasedonpriorCPSinvolvement,therewerenosignificantdifferencesbasedonFARorComparisongroup

Family Assessment Response: Interim Evaluation Report 21

assignmentforfamilieswhohadnopriorintakes.FamilieswithpriorCPSinvolvementhadasignificantlygreaterlikelihoodofhavinganewintake.Seethetechnicalappendixofthisdocumentfordataregardingnewintakesbasedonpriorinvolvement.IndiscussingthesepreliminaryfindingswithFARfieldstaffandleadershipatCA,wefoundthattherewasaperceptionthatFARfamiliesmaycontinuetoreceivenewFAR-eligibleintakesatagreaterrateduetounmetservicesneeds.Thesefamiliestendtohavecomplicatedneedpatterns,whichoftencannotbeaddressedinthelimitedwindowof45days.ItisworthnotingthatstatesthathavefoundthatARhashadanimpactonreducingsubsequentintakesdonothavesuchstrictlimitsonthelengthoftimeacasecanbeopen.Theiroverallcaselengthaveragesarenotparticularlyhigh,buttheseotherstatesdohavetheflexibilitytokeepcasesopenlongerifnecessarytoprovideservices.CAdidaninternalreviewofFARcasesandfoundthat10percentwouldhavebenefittedfromservicesthatcouldhavebeenprovidedifthecasewereleftopenforalongerperiodratherthanclosedduetothe90-daytimelimit.ThisfindingsuggeststhatcreatingaprovisiontoallowanadditionaltimeextensiontoaFARcasewouldaffectarelativelysmallnumberofcasesbutinthosecases,couldprovidemoreneededservicestofamilies.FARleadershipisworkingonarequesttoamendtheprogramlegislationtoallowformoreflexibilitytokeepcasesopenlongerifthereisaneedforafamilytoreceivelonger-termservices.

Family Assessment Response: Interim Evaluation Report 22

RemovalRates

Asshowninthetablebelow,at3months,theComparisongrouphadaslightlyhigher,butstatisticallysignificant,rateofremovalsthandidFARfamilies.However,thispatternofasignificantdifferencedidnotpersistoverlongeroutcometimeframes(6monthsand12months).Itshouldbenoted,though,thatonlythefirstthreecohortshaddataavailableonremovalsforthefull12-monthwindowaftertheFARintake.

Removalsat3,6,and12MonthsAfterIntake

FAR

MatchedComparisonGroup

PercentofFamilieswithaRemovalwithin3monthsofintake

3.5% 4.0%*

PercentofFamilieswithaRemovalwithin6monthsofintake

5.1% 5.5%

PercentofFamilieswithaRemovalwithin12monthsofintake

7.0% 7.4%

(Cohorts1-2)FamiliesservedJanuary1–December31,2014(Cohort3)FamiliesservedJanuary1–June30,2015(Cohort4)FamiliesservedJuly1,2015–December31,2015(3&6monthoutcomesonly)

WhentheeffectofFARonremovalsisanalyzedseparatelybycohort,therearenostatisticallysignificantdifferencesinremovalratesbetweenFARandComparisongroupfamiliesforanyindividualcohortduringanyofthethreetimeperiods.WhiletheintenttotreatdesignnecessitatesthatallfamiliesinitiallyassignedtoFARareincludedinouranalysis,wedidexaminedifferencesinremovalratesbasedonwhetherafamilyactuallycompletedtheFARintervention.Asexpected,familieswhocompletedFARhadlowerratesofremovalsthatdidfamilieswhoeitherdeclinedparticipationorwhoweretransferredduetoconcernsregardingchildsafety.

Family Assessment Response: Interim Evaluation Report 23

FARRemovals,byCaseDisposition(3-and6-MonthRemovalRates)

3Months 6Months 12MonthsComparisonGroup3/6/12months

OverallRemovalRate 3.4% 5.30% 7.3% 4.5%/6.1%/7.7%

RemovalrateforfamilieswhoremaininFAR(89%ofallFARintakes)

2.6% 4.2% 6.5%

Removalrateforfamilieswhodeclinedorweretransferred(11%ofallFARintakes)

4.3% 7.1% 14.2%

(Cohorts1-2)FamiliesservedJanuary1–December31,2014

Family Satisfaction with FAR and Self-Reported Outcomes

Findingadifferentpathwaytoengagefamilies,toestablishtrust,andtoencouragefamiliestoacceptsupportandparticipateinservices,theFARmodelstressesworkingtogetherwithfamiliesandestablishingarelationshipthatislessadversarialthantraditionalCPSinvestigations.ToassessthedegreetowhichFARisabletoachievethisobjectiveandtoconsiderfamilies’perspectivesoftheirownimprovement,weaskedFARfamiliestoreportthedegreetowhichtheyweresatisfiedwiththeservicestheyreceivedfromFARandtheperceptionsofchangesintheirfamily’swell-being.TelephoneinterviewswereconductedwiththosefamilieswhoagreedtobecontactedbyresearcherswhentheyassignedtheinitialFARagreement.Thelargestchallengewithconductingtheseinterviewshasbeenreachingparents/caregiversbyphone.Inmanycasesphonenumberschangebetweencaseclosuresandourattemptstoconductsurveys.Inothercases,wemaydialanumbermultipletimeswithoutreceivingaresponse.11

11Itisimportantwhenconsideringtheseresultstonotethattherespondentsdorepresentthosefamilieswhowecanreachandwhoarewillingtotalktous.Inotherwords,therespondentsarenotnecessarilyfullyrepresentativeoftheentirepopulation.

Family Assessment Response: Interim Evaluation Report 24

ThemajorityofrespondentsreportedbothapositiveexperiencewithFARandpositiveoutcomesfollowingtheirparticipation.Asshownbelow,90%ofrespondentswereeitherverysatisfied(65%)or“mostlysatisfied”(25%)withtheywaythattheyandtheirfamilywastreatedbytheirFARcaseworker.Additionally,morethanhalfofrespondentsreportedthattheirfamilywasdoingeither“muchbetter”(38%)or“somewhatbetter”(23%)becauseoftheirFARparticipation.

Overall,howisyourfamilydoingbecauseofFAR?(N=228)

Verysa(sfied65%

Mostlysa(sfied25%

NA3%

Mostlydissa(sfied3%

Verydissa(sfied4%

Verysa(sfied65%

Mostlysa(sfied25%

NA3%

Mostlydissa(sfied3%

Verydissa(sfied4%

Howsatisfiedareyouwithhowyouweretreated?(N=228)

Family Assessment Response: Interim Evaluation Report 25

Morethree-quarters(79%)ofrespondentsreportedthattheywereeither“verysatisfied”(51%)or“mostlysatisfied”(28%)withtheservicestheyreceivedorwereofferedthroughtheirparticipationinFAR.

Moreover,63%ofrespondentswhohadhadapreviouschildwelfareexperiencereportedthatthisexperiencewithCPSwas“muchbetter”thantheirpreviouschildwelfareexperiences.ThisresponseindicatesthatFARisimprovingfamilyexperienceswithCPSovertime.

Overall,howsatisfiedareyouwiththeservicesyoureceived(orwereoffered)?

(N=225)Verysa(sfied

51%

Mostlysa(sfied28%

NA12%

MostlyDissa(sfied

4%

VeryDissa(sfied5%

Muchbe(er61%

Somewhatbe(er9%

Nochange18%

SomewhatWorse6%

MuchWorse6%

Overall,howwasthisexperiencebasedonyourpreviouschildwelfareexperiences?(N=88)

Family Assessment Response: Interim Evaluation Report 26

Cost Analysis FARhastwodistinctandoppositeeffectsonthecostofservices.ThefirsteffectisthatFARincreases,foralltimeintervals,theprobabilitythatfamilieswilluseaservicethatrequiresCAfunding.Thesecondeffectisthatforthosefamilies(FARandComparison)whodouseCA-fundedservices,FARfamilieshavereducedaveragecosts.Inotherwords,FARfamiliesaremorelikelytouseCAservices,butthoseservicestendtocostlessthancostsforComparisongroupfamilieswhouseCAservices.OnecomplicationwithanalyzingFARcostdataisthatservicecostsvarybycase.Formostfamilies(FARandComparison),thetotalservicecostsarezero;however,forsomefamilies,costscanbelarge.ThedistributionofthesedataisskewedsuchthatthemediancostofservicesprovidedbyCAforallfamiliesiszero.However,themean(average)costissubstantiallyabovezero.Themeanisthereforenot“typical”orrepresentative.Thevariancebetweenmedianandmeancanbeproblematicforanalysis.Manysimplestatisticaltests,suchasaT-testforthedifferenceinmeans,arepotentiallyinvalidwithdatathataremostlyzerosandhighlyskewed.Onecommontechniqueforanalysisofdataofthistypeisa“hurdle”model.Applyingthismodel,wehaveestablishedthatthefirsthurdlepredictstheprobabilitythatafamilywillrequireanycosts.Thesecondhurdlepredictsthemagnitudeofthecostsforanyfamilywithpositivecosts.Thetablebelowpresentstheoveralltwo-stephurdlemodelresultsforFARandComparisongroupfamilies.Dataforallofthecostanalysiscanbefoundinthetechnicalappendix(AppendixA)inthisdocument.

ServiceCostsAnalysis(WithoutSeparateCohortTreatment)

TimeInterval

HurdleExpectedValue

MagnitudeofEffect

FAR Comparison Hurdle1:DoesFARaffect

whetherfamilieshaveanypaidservices?

Hurdle2:DoesFARaffect

highercosts?

Combined

3Months $345 $228 Yes(morelikely) Yes $117*

6Months $645 $655 Yes(morelikely) No -$10*

12Months $1,258 $1,724 Yes(morelikely) No -$465**P-value=0.00

Howtoreadthecostdatatable:Thetableabovepresentskeyresults.The“HurdleExpectedValue”sectionisdividedintothetwogroups,FARandComparison.TheFARcolumnpresents

Family Assessment Response: Interim Evaluation Report 27

theexpectedcostsifeveryeligiblefamilywasservedundertheFARpathway.TheComparisoncolumnpresentstheexpectedcostsifeveryfamilyinsteadreceivedtheInvestigativeapproach.ThedifferencebetweenthetwocolumnsistheestimatedeffectofFAR.The“MagnitudeofEffect”sectionofthetablepresents,intheright-mostcolumn(“Combined”),thisdifference.Theothertwocolumnsin“MagnitudeofEffect”respondtokeycostquestions.Hurdle1designateswhetherFARhadapositiveornegativeeffectontheprobabilitythatafamilywillhaveanypaidservices.Foreachtimeinterval,FARfamiliesaremorelikelythanComparisonfamiliestouseservicespaidforbyCA.12Thiseffectisstatisticallysignificant(p<.00).Hurdle2considerstheaveragecostsofCApaidservicesfortherespectivegroups,FARandComparison.AnegativeeffectindicateslowercostsforFARfamilieswhohavecoststhanforComparisongroupfamilieswhohavecosts.Thisfindingisalsostatisticallysignificant(p<.00).The“Combined”columnpresentstheactualaveragevarianceincostspertimeintervalgiventhecombinedeffectofbothhurdles.Accordingtothesefindings,FARfamiliesaremorelikelytohaveapaidserviceofanykind.AndFARservicescost,onaverage,$117moreperfamilyovera3-monthperiod.However,over6monthsand12months,servicesforFARfamiliescost,onaverage,$10and$465lessthanComparisonfamilycostsperfamilyserved.TheseresultsareconsistentwiththeFARmodel:servicesareprovidedtofamiliestoresolveproblemsandpreventfutureinvestigationsandremovals. Summary and Conclusions

Atthetwo-yearmark,theFARprogramoffersseveralnotablefindings.Ononehand,bothcaseworkersandfamiliesservedbytheFARprogramreportoverallhighlevelsofsatisfactionwiththeimplementationoftheFARpathway.Ontheother,outcomesforfamilies,asmeasuredbyreductionsinnewintakesandremovals,havenotshownsignificantbenefitsforFARfamilies.However,thesenon-dynamicmeasuresmaynottellthefullstory.Forexample,therelativelystagnantmeasureofbenefitsincludesmeasuresfrompartialimplementationandearlyimplementation.Assuch,implementationadjustmentsbasedonearlyfindingsandincreasedfamiliaritywiththeFARmodelforcaseworkers,supervisors,administrators,andothersmayleadtomorepositiveoutcomesatfutureintervals.

12ThisincludesonlythosecostspaidforbyCA(notcommunityservicesnotpaidbyCA),butdoesincludeallcosts,includingthoserelatingtoplacement.

Family Assessment Response: Interim Evaluation Report 28

Asmuchasweremainoptimisticaboutthewaysthatgreaterfamiliarity,experience,andmodificationwillbenefitongoingFARimplementation,wedoofferrecommendationsforthisbenchmark.SomeofthespecificrequirementsputforthinFARenablinglegislationsmayhaveunintentionallylimitedtheeffectivenessoftheprogram.Basedonourfindingsininterviews,fieldresearch,literaturereview,andotherdata,weinclude,aspartoftheinterimevaluation,tworecommendationsthatarelikelytoaddresssomeofthelimitsrevealedinthisreport.Theserecommendationsarelistedasfollows:

EliminatetheFARAgreement.CaseworkersreportthattheFARAgreementcanbeasignificantbarrierforsomefamilies.OneparticularconcernisthatNativeAmericanfamilies,comparedwithothergroups,aredisproportionatelydecliningtoparticipateinFAR.Whiletheadministrativedatadonotallowustoconclusivelydeterminethattheagreementisthereasonforthisdisproportionality,interviewswithcaseworkersindicatethatNativeAmericanfamiliesareoftenreluctanttosignanofficialgovernmentdocumentinordertoparticipate.WhilewecannotguaranteethattheeliminationoftheFARAgreementwilleliminatetheobserveddisproportionalityforNativeAmericanfamiliesaltogether,evidencesuggeststhatthisadjustmentwillleadtoareductionindisproportionality.Allowforanadditionalcaseextensionforcasesinwhichtheadditionaltimeisneededtoprovideaspecificservice.Thecurrenttimelimitsignificantlyrestrictswhatservicescanbeprovidedtofamilies,particularlyEvidence-BasedPracticesthathaveservicedurationslongerthan90days.Evenforthoseservicesthathavea60–90-daytimeframe,thetimeneededtocompleteacomprehensivefamilyassessment,selectaservice,andmakeareferraltoaprovidercansignificantlytruncatetheavailabletime.

Finally,inadditiontotheabove,werecommendthatCAcontinueitsongoingeffortstomonitortrainingqualityandprovidefollow-upresourcesintheformsofcaseconsultations.TheseCAefforts,togetherwithimplementationoftheabove-listedpolicyrecommendations,willlikelyaidCAinitseffortsandassisttheFARprograminitseffectivenessandservicetothefamiliesofWashingtonState.


Recommended