Pilot implementation of an electronicpatient-reported outcome measure forplanning and monitoring participation-focused care in early interventionE. C. Albrecht1†, V. C. Kaelin2†, B. L. Rigau3, J. K. Dooling-Litfin4, E. A. Scully4, N. J. Murphy5, B. M. McManus5‡,M. A. Khetani2,3,6*‡ and on behalf of the High Value Early Intervention Research Group4
Abstract
Background: Family-centered care is a valued approach to improving child and family outcomes in earlyintervention (EI), yet there is need to implement interventions that support information exchange for shareddecision-making when planning and monitoring EI care. This study aims at estimating the feasibility, acceptability,and value of implementing the Young Children’s Participation and Environment Measure (YC-PEM), a validelectronic patient-reported outcome (e-PRO) that is designed to support family engagement when planning careand monitoring outcomes of care.
Methods: Data were gathered from caregivers (N = 139) that were enrolled in a Phase 1 trial of the YC-PEM e-PROas implemented within 1 month of their child’s next EI evaluation of progress. YC-PEM e-PRO feasibility wasestimated according to enrollment and completion rates, and mean completion time. Chi-square tests were usedto examine parent perceptions of YC-PEM e-PRO acceptability by caregiver education and family income. Caregiverfeedback via open-ended responses were content coded to inform intervention and protocol optimizations. YC-PEM e-PRO value was estimated via composite and item-level scores to capture the extent of participation difficultyin home and community activities, and common areas of need regarding caregivers desired change in their child’sparticipation.
* Correspondence: [email protected]†Albrecht, E. and Kaelin, V.C. are co-first authors.‡McManus, B. and Khetani, M.A. are co-senior authors.2Rehabilitation Sciences, College of Applied Health Sciences, University ofIllinois at Chicago, Chicago, IL 60612, USA3Department of Occupational Therapy, College of Applied Health Sciences,University of Illinois at Chicago, 1919 West Taylor Street, Room 316A,Chicago, IL 60612-7250, USAFull list of author information is available at the end of the article
Albrecht et al. BMC Medical Informatics and Decision Making (2020) 20:199 https://doi.org/10.1186/s12911-020-01189-9
Results: Feasibility of implementing the YC-PEM e-PRO in routine EI care was mixed, as evidenced by lowenrollment rates (21.0–29.2%), a high completion rate (85.3%), and limited missing data (80.6% of completed casescontained no missing data). More than half of the participants reported that the completion of the YC-PEM e-PROwas at least somewhat helpful, regardless of family income or caregiver education, providing support for itsacceptability. As for its value, the YC-PEM e-PRO results were viewed by 64% of caregivers, whose desire for changemost often pertained to the child’s participation in non-discretionary activities at home and structured activities inthe community.
Conclusions: Results may support the implementation of YC-PEM e-PRO as a feasible, acceptable, and valuedoption for engaging families in planning the child’s EI care. Results also inform select intervention and protocoloptimizations prior to undertaking a multi-site pragmatic trial of its effectiveness on family engagement and shareddecision-making within an EI clinical workflow.
Trial registration: Trial number: NCT03904797. Trial registered at Clinicaltrials.gov. Registered 22 March 2019.Retrospectively registered.
Keywords: Family-centered care, Young children, Participation, Online, Care planning, Collaborative goal-setting,Early intervention
BackgroundEarly intervention (EI) is a federally funded, state adminis-tered service system through Part C of the Individualswith Disabilities Education Act. EI provides access to re-habilitation and developmental services for approximately2–3% of infants and young children with developmentalneeds nationally [1]. Children are deemed eligible for EIservices if they have a diagnosed disability (e.g., cerebralpalsy), developmental delay based on standardized testscores (e.g., fine motor delay), or are at risk for develop-mental delay (e.g., very low birth weight). EI services aredelivered within the child’s natural learning environment(e.g., home, community) [2–4], to improve 1) family care-giving capacity, including their ability to help their child toparticipate in activities that foster the child’s skill develop-ment; and 2) the child’s cognitive, social, and adaptivefunctioning. Family-centered care (FCC) is a federallymandated approach to delivering EI care and involves pro-viders partnering with families to design and monitor carethat is responsive to family priorities for their child’s par-ticipation in valued activities [3].Despite its importance, EI providers struggle to imple-
ment family-centered care in their clinical workflows[5–7]. Prior studies suggest that approximately one-thirdof enrolled families nationally report dissatisfaction withthe family-centeredness of their child’s EI care [8]. Fam-ily disengagement is linked to less robust implementa-tion of intervention strategies and poorer childoutcomes [9].One key opportunity to implement FCC within an EI
clinical workflow is when planning care. However, de-signing family-centered and participation-focused careoften relies on a standard face-to-face approach andtakes considerable provider and family resources tocomplete [10]. While caregivers have expertise on the
activities that are problematic for their child [6], currentliterature shows that they may lack self-efficacy or suffi-cient resources to identify and communicate their con-cerns to members of the child’s EI team [11, 12].As EI programs transition to electronic data capture
for purposes of accountability and quality improvement[13, 14], the implementation of electronic assessmentoptions may be a scalable strategy that EI programs canuse to give families options for providing input abouttheir child’s EI care [15–17]. The Young Children’s Par-ticipation and Environment Measure (YC-PEM) is anelectronic patient-reported outcome (e-PRO) measurethat is designed to support FCC by giving caregivers avalid and reliable way to communicate about theirchild’s current participation and areas of participationneed, while also allowing EI programs to aggregate thesedata to examine trends in this patient-important out-come over time [17–21].To assess its clinical utility prior to full scale deploy-
ment within an EI system of care, the YC-PEM e-PROwas implemented with EI programs that have electronicdata capture. It was first implemented with a smalluniversity-affiliated EI program (annual enrollment: 80families) to examine its feasibility, value, and acceptabil-ity. Feasibility was mixed when the YC-PEM e-PRO wasintroduced by EI providers during EI service visits.While less than half of eligible families (i.e., 44%) optedto complete the YC-PEM e-PRO over a 2.5 monthperiod, there were no missing e-PRO data among thosewho completed it [22]. The YC-PEM e-PRO was foundto be valuable in detecting the extent to which caregiverswanted their child’s participation to change, and couldbe used to show significant associations between EI ser-vice intensity and the child’s level of participation in val-ued home activities [22]. In providing feedback on its
Albrecht et al. BMC Medical Informatics and Decision Making (2020) 20:199 Page 2 of 11
acceptability, EI providers suggested that the YC-PEM e-PRO be tested when it is implemented as part of thechild’s annual evaluation of progress, rather than the EIservice visit, because EI service coordinators routinelyassess for participation need when assessing the child’sprogress on an annual basis [22].Since university-affiliated EI programs have stronger
provider capacity for research engagement, a logical nextstep for assessing the scalability of implementing theYC-PEM e-PRO assessment across diverse EI programsis to establish its feasibility, value, and acceptability whendeployed in larger, non-university affiliated EI programs.In non-university affiliated EI programs, EI providers’lack of routine involvement in research may contributeto disengagement with testing YC-PEM e-PRO imple-mentation, thereby making it a key setting to understandimplementation [16]. There is need to establish YC-PEMfeasibility when implemented into routine annual evalua-tions of the child’s progress, YC-PEM value for under-standing common areas of family need to address in anindividualized family service plan, and YC-PEM accept-ability from the parent perspective [23]. This will help to1) determine whether the implementation of the YC-PEM e-PRO is practical [24], 2) identify further assess-ment and protocol modifications needed prior to futureimplementation efforts, and 3) assure individual site pre-paredness for implementation [24] and further testing ofthe intervention [16].The purpose of this study is to assess the feasibility
(aim 1; i.e., enrollment rates, completion rate, comple-tion time), acceptability (aim 2; i.e., caregiver perspectiveon usefulness of YC-PEM), and value (aim 3; i.e., percentof caregivers accessing online report, percent of care-givers reporting desire for change in their child’s partici-pation) of implementing the YC-PEM e-PRO as anevidence-based option to customize family assessment inan EI workflow. Study results will inform YC-PEM e-PRO optimizations and protocol modifications prior toundertaking a multi-site pragmatic trial.
MethodsParticipants and settingThis is a single-arm, non-randomized pilot implementa-tion trial [25]. Multi-institutional ethics approval was ob-tained prior to recruitment and data collection (March2017–August 2018) (University of Illinois at Chicago,#2016–0139). Participants were family caregivers (n =149), the end users of the YC-PEM e-PRO, and were re-cruited from a large, urban, and non-university affiliatedEI program. This EI program was reported to servenearly 1000 families of children 0–3 years of age annu-ally and has reported less provider familiarity with re-search engagement [16]. During the 18-month datacollection period, a total of 776 caregivers were eligible
to be invited into this study. These families were there-fore approached by EI staff 1 month prior to the child’sannual evaluation of progress. Each caregiver confirmedhis or her eligibility online by verifying that they were atleast 18 years old; could read, write, and speak English orSpanish; had internet access; and had a child between 0and 3 years old who had received EI for at least 3months. For this study, we analyzed a subset of data onfamilies who completed the English version of the YC-PEM e-PRO (n = 139), because the English but not theSpanish version has been previously validated in termsof its psychometric properties [17–21].
YC-PEM e-PRO interventionThe YC-PEM e-PRO is an electronic health tool thatcaregivers can use to evaluate their young child’s partici-pation in valued activities within the home (e.g., meal-time), daycare/pre-school (e.g., classroom learning), andcommunity settings (e.g., community events). Comple-tion time for the entire YC-PEM e-PRO is 30–40 min[26]. When completing the YC-PEM e-PRO, caregiversanswer questions about how often their child partici-pates in activities (7-point scale, from never to once ormore each day), their child’s level of involvement inthose activities (5-point scale, from not very involved tovery involved), and their desire for their child’s participa-tion to change (yes, no). After evaluating their child’sparticipation, caregivers are asked about the impact ofvarious environmental features (e.g., physical layout ofthe home) and resources (e.g., information and supplies)on their child’s participation in activities within a setting.Details of the YC-PEM e-PRO have been described else-where [17–21]. For this study, caregivers were instructedto complete two of three sections of the YC-PEM e-PRO(i.e., home and community) within a month, prior to thechild’s annual evaluation of progress. The decision to ad-minister the home and community sections was made inconsultation with the EI program, and based on low andvariable daycare/preschool enrollment rates amongtheir enrolled families. This approach to decision-making about trial design is congruent with acommunity-engaged research approach [16].
MeasuresMeasure selection was informed by the Family ofParticipation-Related Constructs (fPRC) [19, 27, 28] anda systematic review of salient child and family status andprocess factors that are associated with children’s par-ticipation [29]. The fPRC is a contemporary frameworkthat defines participation as the child’s attendance andlevel of involvement in activities, as influenced by intrin-sic and extrinsic factors such as a child’s environment[27].
Albrecht et al. BMC Medical Informatics and Decision Making (2020) 20:199 Page 3 of 11
Child and family characteristics and early interventionservice useCaregivers who expressed interest, and confirmed theireligibility online to participate in the study were directedto an online demographic questionnaire (n = 163) to re-port on their level of education, age, marital status, race,relationship to child, and family income. Additionally,caregivers reported on their child’s social and clinicalcharacteristics, including the child’s age, race, ethnicity,sex, and developmental condition type (diagnosis versusdevelopmental delay).EI service use was captured via record abstraction and
reported out according to amount (hours), duration(months), intensity (hours per month), and type. EI ser-vice amount was estimated as the total number of hoursof EI services, as well as the total number of hours pereach of the following four core EI services: physical ther-apy (PT), occupational therapy (OT), speech therapy(ST), and developmental intervention (DI). Service dur-ation was calculated by subtracting the date of EI entry(i.e., the child’s date of EI eligibility evaluation) from thedate of study enrollment, as reported in months. Serviceintensity (hours per month) was then derived by dividingtotal service amount (hours) by service duration(months), to yield an estimate of total hours per monthof EI services. Receipt of each of the four core EI ser-vices were also included as number of EI services re-ceived (one, two, three or more).
YC-PEM e-PRO feasibilityThe feasibility of implementing the YC-PEM e-PROwithin an EI workflow was assessed via web analytic dataon study enrollment rates (number of families who wereapproached by EI staff, expressed interest, and con-firmed their eligibility online), completion rate (numberof families who completed the YC-PEM e-PRO online),and mean completion time (minutes).
YC-PEM e-PRO acceptabilityFollowing completion of the YC-PEM e-PRO online,caregivers were invited to share their perceptions of howuseful YC-PEM e-PRO was for planning care by their re-sponses to the question, “Did you find this survey help-ful for understanding your child’s participation in thehome and community? Why or why not?”
YC-PEM e-PRO valueFor this study, we estimated value in two ways. First, weused web analytic data to estimate the percentage offamilies who accessed their child’s online summary re-port following e-PRO completion. Second, when com-pleting the YC-PEM e-PRO, caregivers identifiedwhether or not they desired a change (yes, no) in theirchild’s current participation within specific activities at
home (13 activities) and/or the community (11 activ-ities). These data highlight areas of parent engagementwith EI care as well as caregiver dissatisfaction with thechild’s participation. Therefore, these data may providekey insight into areas of family need. Since the purposeof family assessment is to solicit information about fam-ily need to inform intervention priorities, these YC-PEMe-PRO data are presented to show their value for EIfamily assessment.For this study, a percent desire change score was calcu-
lated for each item and as a mean score. To calculatethe percent desire change score per item, the number of‘yes, change desired’ responses were summed up, dividedby the total number of responses, and multiplying by100. These item scores afford for detailed description ofcommon areas of participation need across activitieswithin each setting. The mean percent desire changecomposite score was derived for each setting (home,community), by taking the average across individual set-ting scores, which were calculated by summing the num-ber of ‘yes’ responses across items, dividing by thenumber of items, and multiplying by 100. The YC-PEMe-PRO internal consistency reliability estimates wereconsidered good for data obtained in this study (ɑ = .80for home desire change, ɑ = .84 for community desirechange) [30].
Data collectionTwenty-two EI service coordinators participated in a 90-min video-conference session to learn about project pur-pose, provide feedback on recruitment processes andmaterials, and learn how to enroll participants via theproject website. Service coordinators who completed thissession used a script and flyer to then recruit familiesduring the scheduling of their child’s annual review of EIprogress. The recruitment protocol was modified in re-sponse to low enrollment [16], such that a designated EIstaff member was paired with research staff to recruitparticipants. Further details about protocol modifica-tions are published elsewhere [16].Eligible and interested caregivers visited the project
website to create an account, confirmed study eligibility,provided informed consent and HIPAA authorizationfor abstracting select EI service use data, and completeda demographic questionnaire and the YC-PEM e-PRO.Caregivers received immediate access to an online reportsummarizing their e-PRO responses to share with theirchild’s EI team and were mailed $10.00 gift cards after e-PRO completion.
Data analysisSPSS 24.0 was used to perform all analyses for thisstudy. Descriptive statistics were used to summarizesample characteristics and EI service utilization
Albrecht et al. BMC Medical Informatics and Decision Making (2020) 20:199 Page 4 of 11
according to service dosage (amount, duration, intensity)and number of core EI services. For continuous vari-ables, sample means, medians, and standard deviationswere calculated. Due to skewed distribution, inter-quartile ranges were used to describe EI serviceutilization. Sample proportions were calculated for cat-egorical variables.For YC-PEM e-PRO feasibility (aim 1), enrollment and
completion rates, as well as mean completion time wereestimated for the 139 families who completed the Eng-lish YC-PEM e-PRO. The success of feasibility was de-termined as e-PRO enrollment and completion rates of50% or higher based on family assessment completionrate within usual care (personal conversation with E.A.Scully, March 1, 2019). Therefore, the criterion for deter-mining whether the YC-PEM e-PRO is feasible is n = 388participants (i.e., 50% of the 776 eligible participants), ashas been done in previous studies [22].For YC-PEM e-PRO acceptability (aim 2), caregiver
feedback via open-ended responses were coded usingtwo approaches. In the first coding approach, responseswere sorted into a one of three categories (helpful;somewhat helpful; not helpful) to create a new variablethat captured the extent to which caregivers perceivedthe YC-PEM e-PRO to be useful for planning EI care.Given the previously reported positive association be-tween participation, and family education and income[31], chi-square tests were used to examine perceptionsof YC-PEM e-PRO acceptability by caregiver education(high school or some college; college degree; graduatetraining) and family income ($0–$50,000; $50,001–$100,000; > $100,001) [32]. The criterion we used to evaluatewhether acceptability was associated with caregiver’slevel of education or family income was p < .05.In the second coding approach, responses underwent
inductive content analysis [33], whereby data weregrouped into categories to identify supportive and prob-lematic aspects of the user experience. Data weresummed for each category, to help interpret the salienceof each aspect described. Supportive aspects were ex-pected to inform optimizations to the recruitment proto-col, whereas aspects of concern were intended to informoptimizations to the YC-PEM e-PRO prior to furthertesting. Study staff independently coded each open-ended response and met to resolve coding discrepanciesthrough discussion, resulting in the development of anew category to best fit the data.For YC-PEM e-PRO value (aim 3), the percentage
of participants who viewed a summary of their e-PROresponses via an online report was estimated. Inaddition, mean percent desire change scores were cal-culated for each setting (home, community) to iden-tify the extent of participation difficulty in eachcontext, and item scores were rank ordered to
identify common areas of home and community par-ticipation need.
ResultsSample characteristicsAs shown in Table 1, more than half of the children in-cluded in this study were between 24 and 35months old(54.0%), male (51.1%), and had a developmental delay(no diagnosis) (72.7%). On average, children had re-ceived EI services for 13.87 months. Most of the childrenreceived multiple EI services, with the most commonservices being ST (69.1%) and PT (52.5%).
YC-PEM e-PRO feasibility (aim 1)In total, 163 of 776 (21%) families that were eligible tobe invited into this study expressed interest, confirmedtheir eligibility online, and enrolled over 18 months, withthe highest enrollment rate (29.2%) occurring followingthe transition to a standard research protocol in the final10 months of the data collection period. Fourteen fam-ilies were either lost to follow-up or declined to partici-pate due to lack of time, lack of interest, or poor timing(i.e. child’s annual review occurred prior to recruitment),resulting in 149 of the 163 enrolled families who com-pleted the YC-PEM e-PRO in English or Spanish. Tenfamilies were excluded based on their completion of theSpanish YC-PEM e-PRO, resulting in 139 families andan 85.3% completion rate (see Fig. 1). Mean completiontime was 21.3 min (range = 12.3–29.9). Of those 139caregivers completing the English YC-PEM e-PRO, ap-proximately four out of every five (80.6%) had no miss-ing data. In cases with missing data, 98.6% of cases hadless than 20% of missing data.
YC-PEM e-PRO acceptability (aim 2)More than half of the 139 families who completed theYC-PEM e-PRO (58.4%) of the included caregivers per-ceived the implementation of YC-PEM e-PRO to be atleast somewhat helpful. Specifically, 38.0% (n = 54) ofcaregivers perceived the YC-PEM e-PRO as helpful,20.4% (n = 29) as somewhat helpful, and 16.2% (n = 23)did not find it helpful. Chi-square results indicated thatYC-PEM e-PRO acceptability was not significantly asso-ciated with family income categories (χ2=.48, p = .79) orcaregiver education categories (χ2= 2.26, p = .69).A subset of caregivers identified supportive and prob-
lematic aspects related to the relevance (n = 65), wording(n = 15), and structure (n = 14) of the YC-PEM e-PRO(see Table 2). These caregivers reported that the imple-mentation of YC-PEM e-PRO helped them to reflect ontheir child’s strengths, challenges and priorities; strat-egies to promote the child’s participation in current ac-tivities; and new activities to try. These caregiversexpressed concerns around the relevancy of items based
Albrecht et al. BMC Medical Informatics and Decision Making (2020) 20:199 Page 5 of 11
on child age and/or disability; and complex itemwording.
YC-PEM e-PRO value (aim 3)Approximately 64% (n = 89) of the 139 families whocompleted the YC-PEM e-PRO proceeded to view theirYC-PEM e-PRO report online. On average, caregiversdesired for their child’s participation to change in 27% ofhome activities and 26% of community activities. Asshown in Fig. 2, item scores revealed that the most com-mon areas of desired participation change pertained tonon-discretionary and structured activities. In the home,46% of participants desired their child’s participation tochange for cleaning up activities, and 44% reported thatthey desired for their child’s participation to change with
respect to personal care management. In the commu-nity, 42% of participants reported that they desired fortheir child’s participation to change in classes and les-sons, and organized physical activities.
DiscussionThe YC-PEM e-PRO is a recommended approach toevaluating a young child’s participation in valued activ-ities [19, 20], yet there is critical need to identify sup-ports and barriers to its uptake within a clinicalworkflow to support its adoption within an EI system ofcare [23]. This pilot trial contributes additional evidenceto the feasibility, acceptability to caregivers, and valuewhen implementing the YC-PEM e-PRO option to sup-port family engagement to plan EI care [22].
Table 1 Child and family social characteristics and early intervention service use at study enrollment
Insurance Type, Public Insurancea 39 (28.1) Type of EI Services Receiveda
Had a Diagnosis 38 (27.3) PT 73 (52.5)
Child Age (months) OT 44 (31.7)
12 to 24 64 (46.0) ST 96 (69.1)
over 24 75 (54.0) DI 51 (36.7)
Child Raceab Number of EI Services Receiveda
White 104 (74.8) 1 37 (26.6)
Black 9 (6.5) 2 57 (41.0)
Asian 2 (1.4) 3 or more 36 (25.9)
American Indian/Alaska Native 2 (1.4) Median [IQR]
Native Hawaiian/ Other Pacific Islander 1 (.7) Total Per Child EI Hoursa
Other 4 (2.9) All Services 69.00 [56.50,107.50]
Multiple Races 13 (9.4)
Child Ethnicitya PT 7.00 [.00, 57.00]
Hispanic or Latino 27 (19.4) OT .00 [.00, 11.00]
Not Hispanic or Latino 106 (76.3) ST 28.00 [.00, 56.00]
Unknown 4 (2.9) DIEI Intensitya (hours per month)
.00 [.00, 15.75]5.82 [4.67, 8.18]
Respondent Type (mother or female guardian) 132 (95.0)
Caregiver Education Level
High school or some college 32 (23.0)
College degree 40 (28.8)
Graduate training 67 (48.2)
Family Incomea
$0–50,000 29 (20.9)
$50,001-100,000 32 (23.1)
$100,001+ 73 (52.5)
Abbreviations: EI Early intervention, PT Physical therapy, OT Occupational therapy, ST Speech therapy, DI Developmental interventionaMissing databRespondents could select multiple responses
Albrecht et al. BMC Medical Informatics and Decision Making (2020) 20:199 Page 6 of 11
FeasibilityEnrollment rates were similar to prior work, withless than half of the eligible caregivers opting foran electronic option as part of their child’s annualreview of progress; however, despite low enrollment,nearly four out of every five caregivers who com-pleted the YC-PEM e-PRO did so in its entirety, an80% completion rate [22]. Therefore, evidence forthe feasibility of implementing the YC-PEM e-PROin routine EI care was mixed. Feasibility estimateson enrollment in this study were lower than the50% criterion for feasibility, primarily due to diffi-culty with provider engagement in recruitment,which resulted in the need to modify the recruit-ment protocol multiple times during the 18-monthdata collection period. These problems with
recruitment informed a key protocol modification,which is the co-creation of new infrastructure tostrengthen EI provider capacity for research engage-ment [16]. This new infrastructure will support acommunity engaged research approach to designingand executing a study protocol (inclusive of studysubject recruitment) and interpreting and dissemin-ating study results. To our knowledge, there is noprecedence for such infrastructure in EI. However,this may be a key driver of successful implementa-tion in future testing, as most EI programs are notuniversity-affiliated and may vary in their researchcapacity.Completion time is also similar to prior phases of
YC-PEM e-PRO testing [17] and shorter than otherfamily assessment options that are part of usual care(a semi-structured interview that typically takes be-tween 45 min and 2 h to complete). These findingsshould be interpreted with caution, however, as lon-ger sessions may be needed in some cases to be re-sponsive to family needs and their communicationpreferences. The YC-PEM e-PRO is not meant to re-place usual care but rather to reinforce family-centered care by giving families another option tocontribute to an assessment of their child’s needswhen planning their child’s EI care [15]. From thisperspective, this finding indicates that the online op-tion provides an efficient choice for families to com-municate areas of difficulty for their child. Efficiencymay be among several factors (e.g., flexibility, priv-acy) that caregivers consider when choosing toutilize the YC-PEM e-PRO. Thus, our findings sug-gest that the YC-PEM provides individualized familyassessment options that meet the needs of familiespreferring both short and longer spaces to communi-cate their family’s needs and priorities for EI care.
AcceptabilityMore than half of caregivers perceived the use of YC-PEM e-PRO as at least somewhat helpful. Caregiversoften reported that it helped them to reflect on theirchild’s current participation and their own strategies forsupporting it. This was the intended use of YC-PEM e-PRO as shared with EI staff during their initial training,which is to give parents an online way to gather theirthoughts about their child, when and where it is mostconvenient for them to do so. While these results war-rant replication with a larger and more diverse sample,these preliminary findings seem to suggest that, forthose caregivers who provided such feedback, anticipa-tory planning may decrease pressure and power differen-tials [34] that hinder shared decision-making whenplanning EI care [15].
Fig. 1 Enrollment diagram
Albrecht et al. BMC Medical Informatics and Decision Making (2020) 20:199 Page 7 of 11
Some caregivers expressed concern about the relevanceof the YC-PEM e-PRO, either because they perceivedtheir child to be too young to participate in activities, orbecause they perceived their child’s disability to hindertheir capability to participate in activities. One potentialreason for this finding may relate to parental expectationsthat their child needs to build developmental skills (e.g.,use a pincer grasp) and independently perform functionaltasks (e.g., feed self with finger foods) before they can par-ticipate in valued activities (e.g., engage in mealtime athome or dine out for meals at a local restaurant). Thispoint of feedback may be best addressed by improvingthe assessment instructions where the concept of partici-pation is first introduced to parents. This assessmentmodification should include how participation is distinctfrom developmental skills and functional capabilities andhow it is shaped by child and environmental
characteristics [28, 35–39]. Another caregiver concern re-lated to items being difficult to understand, which sug-gests that wording in the assessment may needmodification prior to further testing.
ValueThe intended value of the YC-PEM e-PRO is to help in-dividual families to communicate their needs and prior-ities, and for the EI program to be able to aggregatethese data to learn about common areas of need amongthe families served and how those needs might changeover time as a function of EI service use. The majority ofcaregivers accessed their online e-PRO report suggestingits value in bolstering family engagement in EI care.However, we acknowledge the lack of data on rates ofsharing the YC-PEM e-PRO report with the child’s EIteam. Prior studies, though, have indicated caregiver
Table 2 Caregiver feedback on acceptability of the implementation of YC-PEM e-PRO
Areas of Support Areas of Concern
Relevance · Helped me to reflect on my child’s participation, including strengths,challenges, and priorities (n = 16)· Helped to think about our helpful strategies and possible newstrategies (n = 6)· Gave us ideas for activities to participate in (n = 5)· It helped by breaking down the day to day activities that aresometimes forgotten about (n = 4)· Helped me to think about the impact of the environment (n = 3)
· Felt like some questions did not fit our child’s disability and/oryoung age (n = 15)· It did not fit our family situation (e.g., foster parent) and/orfocus (e.g., we did not desire any change) (n = 10)· Hard to learn from it or relate it to daily life (n = 6)
· Wording of some questions was difficult to understand (n = 13)
Structure · Good choice of questions (n = 2)· Specificity level of the activities was helpful (n = 1)· Questions were thorough (n = 1)
· Some questions and/or their response options wereoverwhelming or difficult to answer (n = 10)
Abbreviation: YC-PEM e-PRO Young Children’s Participation and Environment Measure electronic patient-reported outcome
Fig. 2 Percent of participation need in home activities (a) and community activities (b)
Albrecht et al. BMC Medical Informatics and Decision Making (2020) 20:199 Page 8 of 11
interest in sharing a YC-PEM e-PRO report with serviceproviders [40]. Work is underway to employ a know-ledge translation framework for engaging providers andparents as stakeholders to optimize the content and lay-out of the online report, which may further increase itsuptake in an early childhood programmatic context.Percent desire change estimates were significantly
lower in this study as compared to prior phases of study,which could be a function of EI service duration.Whereas children in the prior phase had been enrolledin EI for 7 months on average, families in this study hadreceived services for close to twice that length of time(13.87months on average). While changes in participa-tion may be gradual, the YC-PEM e-PRO has the poten-tial to detect participation change as a function of EIservice use. In an era of accountability, EI programs arepressed to demonstrate high value care [3]. Measuresthat can detect clinically meaningful change can supportthese efforts.
Limitations and strengthsStudy results should be interpreted in light of severallimitations. The study sample is larger and more diversethan prior phases of testing [41], yet is skewed towardshigher education and income levels when compared tothe overall enrollment at this EI program [32]. Futurestudies with more diverse samples according to socio-economic status may detect greater variability in YC-PEM e-PRO acceptability. These studies may benefitfrom the use of culturally adapted versions of the e-PROto maximize reach [21, 42, 43]. Open-ended items tocapture caregiver feedback on YC-PEM e-PRO accept-ability did not afford for opportunities to clarify and/orfurther probe about solutions to concerns reported (e.g.,allowing parents to provide suggestions for alternativewording) to inform optimizations prior to further test-ing. Finally, we did not discern between those who werescreened ineligible versus those who declined and theirreason for declining, resulting in a more conservative en-rollment rate. In future phases, EI staff should report onthis when recruiting families; similarly, the team shouldcollect demographic or service use data on participantswho declined to examine if there are differences betweenthose who enrolled and those who declined.Despite these limitations, there are a number of key
strengths to this study that extend evidence about YC-PEM e-PRO implementation across more diverse EIcontexts. This includes evidence of its feasibility whenintroduced into an EI service coordinator’s routineworkflow within a large, non-university affiliated EI pro-gram, evidence of its acceptability from the parent per-spective, and evidence of its value for providing familieswith relevant information to view and share with mem-bers of their child’s EI team.
ConclusionElectronic patient-reported outcomes data may offer afeasible, acceptable and valued alternative for obtainingfamily input on children’s participation in valued activ-ities to support care planning and outcomes monitoringin EI. Results provide evidence that the implementationof YC-PEM e-PRO may be feasible across EI programsthat vary by size and enrollment. Results also extend theevidence of acceptability via caregiver feedback, as wellas value by showing the extent to which the e-PRO re-port was accessed and how the data can be used to de-tect areas of participation need for individual caregiversand the total EI enrollment. Together, results suggestthat the implementation of the YC-PEM e-PRO withinan EI clinical workflow merits further study to improveparent activation for shared decision-making.
Abbreviationse-PRO: Electronic patient-reported outcome; EI: Early intervention; YC-PEM: Young Children’s Participation and Environment Measure; FCC: Family-centered care; fPRC: Family of participation-related constructs; PT: Physicaltherapy; OT: Occupational therapy; ST: Speech therapy; DI: Developmentalintervention
AcknowledgementsThis work was supported by funding from the National Institutes of Health(1R03HD084909-01A1, P2CHD065702, K12 HD05593, and L40HD085277). Theuse of Research Electronic Data Capture (REDCap) is supported by theNational Center for Advancing Translational Sciences, National Institutes ofHealth (UL1TR002003). The content is solely the responsibility of the authorsand does not necessarily represent the official views of the funding agencies.We thank our colleague Ann Howell at Rocky Mountain Human Services forrecruitment support. We also thank Vivian Villegas, Weronika Zuczek, SaraBrumm, Andrea Gurga, and Jessica Jarvis from the Children’s Participation inEnvironment Research Lab for assisting with data collection or manuscriptpreparation. Collaborating authors of the High Value Early InterventionResearch Group who contributed to this submission are: Jamie Bane, HaleyCarle, Amy Jatsko, Amanda Pedrow, and Laura Sciarcon.
Authors’ contributionsMK took the lead in conceptualizing the study and drafting all sections ofthe manuscript. BM provided feedback on study design, and assisted withdrafting introduction and discussion sections of the manuscript. EA and VKanalyzed the data and, together with BR, drafted the results section. VK alsoassisted with drafting the discussion and conclusion sections. BR assistedwith analysis of data and preparation of the manuscript. BR and ES providedoversight for data collection and BR, ES, JDL, and the High Value EarlyIntervention Research Group interpreted key findings and provided criticalfeedback during manuscript preparation. NM drafted portions of themethods section of the manuscript and provided edits on the fullmanuscript. All authors provided editing of the manuscripts and read andapproved the final manuscript.
FundingThis work was supported by funding from the National Institutes of Health(1R03HD084909-01A1, P2CHD065702, K12 HD05593, and L40HD085277). Theuse of Research Electronic Data Capture (REDCap) is supported by theNational Center for Advancing Translational Sciences, National Institutes ofHealth (UL1TR002003).
Availability of data and materialsNIH funds through the Center for Large Data Research (CLDR) (P2CHD065702;PI: Ottenbacher) were secured to partner with the Inter-university Consortiumfor Political and Social Research (ICPSR) in curating the data source for publicuse, effective June 2019 (https://doi.org/10.3886/ICPSR37320.v2).
Albrecht et al. BMC Medical Informatics and Decision Making (2020) 20:199 Page 9 of 11
Ethics approval and consent to participateMulti-institutional ethics approval (University of Illinois at ChicagoInstitutional Review Board (protocol number: 2016–0139), University ofColorado Institutional Review Board) was obtained prior to recruitment anddata collection (March 2017–August 2018). Eligible and interested caregiversvisited the project website to create an account, confirmed study eligibility,provided informed consent and HIPAA authorization for abstracting select EIservice use data.
Consent for publicationNot Applicable.
Competing interestsThe YC-PEM e-PRO was used in this study and is licensed for distributionthrough CanChild Centre for Childhood Disability Research. M. Khetani shares inrevenue from YC-PEM sales for research and development activities in her lab.
Author details1Invest in Kids, 1775 Sherman Street, Suite 1445, Denver, CO 80203, USA.2Rehabilitation Sciences, College of Applied Health Sciences, University ofIllinois at Chicago, Chicago, IL 60612, USA. 3Department of OccupationalTherapy, College of Applied Health Sciences, University of Illinois at Chicago,1919 West Taylor Street, Room 316A, Chicago, IL 60612-7250, USA. 4RockyMountain Human Services, 9900 E Iliff Ave, Denver, CO 8023, USA. 5HealthSystems, Management, and Policy, Colorado School of Public Health, 13001E. 17th Place, Mail Stop B119, Aurora, CO 80045, USA. 6CanChild Centre forChildhood Disability Research, McMaster University, 1280 Main Street West,Hamilton, Ontario L8S 4L8, Canada.
Received: 11 July 2019 Accepted: 16 July 2020
References1. IDEA Infant and Toddlers Coordinators Association. Percentage of all
children (including at risk) under three receiving services. 2008–2012 childcount data. 2018. http://ideainfanttoddler.org/association-reports.php.Accessed 28 Mar 2019.
2. Bruder MB, Dunst CJ. Factors related to the scope of early interventionservice coordinator practices. Infants Young Child. 2008;21:176–85.
3. Bruder MB. Early childhood intervention: a promise to children and familiesfor their future. Except Child. 2010. https://doi.org/10.1177/001440291007600306.
4. Whipple W. Key principles of early intervention and effective practices: acrosswalk with statements from discipline specific literature. 2014. https://dese.mo.gov/sites/default/files/se-fs-first-steps-key-principles-crossalk-disciplines.pdf. Accessed 28 Mar 2019.
5. Bamm EL, Rosenbaum P. Family-centered theory: origins, development,barriers, and supports to implementation in rehabilitation medicine. ArchPhys Med Rehabil. 2008. https://doi.org/10.1016/j.apmr.2007.12.034.
6. Hanna K, Rodger S. Towards family-centred practice in paediatricoccupational therapy: a review of the literature on parent-therapistcollaboration. Aust Occup Ther J. 2002. https://doi.org/10.1046/j.0045-0766.2001.00273.x.
7. King G, Kertoy M, King S, Law M, Rosenbaum P, Hurley P. A measure ofparents’ and service providers’ beliefs about participation in family-centeredservices. Child Heal Care. 2003. https://doi.org/10.1207/S15326888CHC3203_2.
8. Ross SM, Smit E, Twardzik E, Logan SW, McManus BM. Patient-centeredmedical home and receipt of part C early intervention among young CSHCN and developmental disabilities versus delays: NS-CSHCN 2009–2010.Matern Child Health J. 2018. https://doi.org/10.1007/s10995-018-2540-z.
9. The National Center for Parent Family and Community Engagement. Familyengagement and school readiness. 2019. https://eclkc.ohs.acf.hhs.gov/sites/default/files/pdf/rtp-school-readiness.pdf. Accessed 28 Mar 2019.
10. McWilliam RA. Protocol for the routine-based interview. 2009. https://www.iidc.indiana.edu/styles/iidc/defiles/ECC/RBI Protocol.pdf. Accessed 28 Mar2019.
11. An M, Palisano RJ. Family–professional collaboration in pediatricrehabilitation: a practice model. Disabil Rehabil. 2014. https://doi.org/10.3109/09638288.2013.797510.
12. Brewer K, Pollock N, Wright FV. Addressing the challenges of collaborativegoal setting with children and their families. Phys Occup Ther Pediatr. 2014.https://doi.org/10.3109/01942638.2013.794187.
13. The DaSy Center. DaSy Data System Framework. Menlo Park: SRIInternational; 2014.
14. Colorado Office of Early Childhood Division of Community and FamilySupport. Early Intervention Data System User Guide. 2018. https://dcfs.my.salesforce.com/sfc/p/#410000012srR/a/41000000Cg0G/q6k5NPuu2c3XjSiGwAjY6Cr_50Hi1WuzgPwuENUVa0k. Accessed 4 Sept 2018.
15. MacKean GL, Thurston WE, Scott CM. Bridging the divide between familiesand health professionals’ perspectives on family-centred care. Health Expect.2005. https://doi.org/10.1111/j.1369-7625.2005.00319.x.
16. Rigau B, Scully B, Liftin J, Murphy N, McManus B, Khetani M. Communityengagement to pilot electronic patient-reported outcomes (e-PROs) in earlyintervention: lessons learned. J Clin Transl Sci. 2018. https://doi.org/10.1017/cts.2018.15.
17. Khetani MA, Graham JE, Davies PL, Law MC, Simeonsson RJ. Psychometricproperties of the young children’s participation and environment measure.Arch Phys Med Rehabil. 2015. https://doi.org/10.1016/j.apmr.2014.09.031.
18. Khetani MA. Validation of environmental content in the young children’sparticipation and environment measure. Arch Phys Med Rehabil. 2015.https://doi.org/10.1016/j.apmr.2014.11.016.
19. Adair B, Ullenhag A, Rosenbaum P, Mats G, Keen D, Imms C. Measures usedto quantify participation in childhood disability and their alignment withthe family of participation-related constructs: a systematic review. Dev MedChild Neurol. 2018. https://doi.org/10.1111/dmcn.13959.
20. Schiariti V, Fowler E, Brandenburg JE, Levey E, Mcintyre S, Sukal-Moulton T,Ramey SL, Rose J, Sienko S, Stashinko E, Vogtle L, Feldman RS, Koenig JI. Acommon data language for clinical research studies: the National Instituteof Neurological Disorders and Stroke and American Academy for CerebralPalsy and Developmental Medicine Cerebral Palsy Common Data ElementsVersion 1.0 recommendations. Dev Med Child Neurol. 2018. https://doi.org/10.1111/dmcn.13723.
21. Lim CY, Law M, Khetani M, Rosenbaum P, Pollock N. Psychometricevaluation of the young Children’s participation and environment measure(YC-PEM) for use in Singapore. Phys Occup Ther Pediatr. 2018. https://doi.org/10.1080/01942638.2017.1347911.
22. Khetani MA, McManus BM, Arestad K, Richardson Z, Charlifue-Smith R,Rosenberg C, et al. Technology-based functional assessment in earlychildhood intervention: A pilot study. Pilot Feasibility Stud. 2018. https://doi.org/10.1186/s40814-018-0260-1.
23. Jones G. Raising the profile of pilot and feasibility studies in relation to thedevelopment, evaluation and implementation of patient-reported outcomemeasures. Pilot Feasibility Stud. 2018. https://doi.org/10.1186/s40814-017-0151-x.
24. Perry CK, Damschroder LJ, Hemler JR, Woodson TT, Ono SS, Cohen DJ.Specifying and comparing implementation strategies across seven largeimplementation interventions: a practical application of theory. ImplementSci. 2019. https://doi.org/10.1186/s13012-019-0876-4.
25. National Academies of Sciences Engineering and Medicine. Implementationscience: method and approaches. In: Beachy SH, Olson S, Addie S, editors.Applying an implementation science approach to genomic medicine:workshop summary. Washington, DC: The National Academies Press; 2016.p. 7–18.
26. Khetani M, Bosak D, Jarvis J, Teplicky R. Young Children’s Participation andEnvironment User Guide ( Version 1. 2); 2019. p. 1–38. https://www.canchild.ca/en/shop/23-yc-pem-young-children-s-participation-and-environment-measure. Accessed 18 Dec 2019.
27. Imms C, Granlund M, Wilson PH, Steenbergen B, Rosenbaum PL, GordonAM. Participation, both a means and an end: a conceptual analysis ofprocesses and outcomes in childhood disability. Dev Med Child Neurol.2017. https://doi.org/10.1111/dmcn.13237.
28. Imms C, Adair B, Keen D, Ullenhag A, Rosenbaum P, Granlund M.“Participation”: a systematic review of language, definitions, and constructsused in intervention research with children with disabilities. Dev Med ChildNeurol. 2016. https://doi.org/10.1111/dmcn.12932.
29. Arakelyan S, Maciver D, Rush R, O’hare A, Forsyth K. Family factorsassociated with participation of children with disabilities: a systematicreview. Dev Med Child Neurol. 2019. https://doi.org/10.1111/dmcn.14133.
30. Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normedand standardized assessment instruments in psychology. Psychol Assess.1994;6:284–90.
Albrecht et al. BMC Medical Informatics and Decision Making (2020) 20:199 Page 10 of 11
31. Khetani MA, Graham JE, Alvord C. Community participation patterns amongpreschool-aged children who have received part C early interventionservices. Child Care Hlth Dev. 2013;39(4):490–9.
32. Hebbeler K, Spiker D, Bailey D, et al. Early Intervention for Infants & Toddlerswith Disabilities and Their Families: Participants, Services, and Outcomes.Final Report of the National Early Intervention Longitudinal Study (NEILS).;2007. https://www.sri.com/sites/default/files/publications/neils_finalreport_200702.pdf. Accessed 29 Nov 2019.
33. Mayring P. Qualitative content analysis: theoretical foundation, basicprocedures and software solution. 2014. http://nbn-resolving.de/urn:nbn:de:0168-ssoar-395173. Accessed 28 Mar 2019.
34. Whalley Hammell KR. Client-centred occupational therapy: the importanceof critical perspectives. Scand J Occup Ther. 2015. https://doi.org/10.3109/11038128.2015.1004103.
35. Anaby D, Law M, Coster W, Bedell G, Khetani M, Avery L, et al. Themediating role of the environment in explaining participation of childrenand youth with and without disabilities across home, school, andcommunity. Arch Phys Med Rehabil. 2014. https://doi.org/10.1016/j.apmr.2014.01.005.
36. Khetani M, Albrecht E, Jarvis J, Pogorzelski D, Cheng E, Choong K.Determinants of change in home participation among critically ill children.Dev Med Child Neurol. 2018. https://doi.org/10.1111/dmcn.13731.
37. Dunst CJ, Hamby D, Trivette CM, Raab M, Bruder MB. Young children’sparticipation in everyday family and community activity. Psychol Rep. 2002;91:875–97.
38. Albrecht EC, Khetani MA. Environmental impact on young children’sparticipation in home-based activities. Dev Med Child Neurol. 2017. https://doi.org/10.1111/dmcn.13360.
39. Guralnick MJ. Early intervention for children with intellectual disabilities: Anupdate. J Appl Res Intellect Disabil. 2017. https://doi.org/10.1111/jar.12233.
40. Khetani MA, Lim HK, Corden ME. Caregiver input to optimize the design ofa pediatric care planning guide for rehabilitation: descriptive study. JMIRRehabil Assist Technol. 2017;4(2):e10.
41. McManus B, Khetani M. Early Intervention Colorado (EI-CO) ParticipantCharacteristics, Service Use, and Outcomes, Colorado, 2014–2016. AnnArbor: Inter-university Consortium for Political and Social Research[distributor]; 2018. https://doi.org/10.3886/ICPSR37131.v1.
42. Di Marino E, Tremblay S, Khetani M, Anaby D. The effect of child, family andenvironmental factors on the participation of young children withdisabilities. Disabil Health J. 2018. https://doi.org/10.1016/j.dhjo.2017.05.005.
43. Åström FM, Khetani M, Axelsson AK. Young children’s participation andenvironment measure: Swedish cultural adaptation. Phys Occup TherPediatr. 2018. https://doi.org/10.1080/01942638.2017.1318430.
Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims inpublished maps and institutional affiliations.
Albrecht et al. BMC Medical Informatics and Decision Making (2020) 20:199 Page 11 of 11
