41
Healthy Start National Evaluation: Data Sharing/Use Agreement Process Webinar January 24, 2017
Healthy Start National Evaluation:
Data Sharing/Use Agreement Process Webinar
January 24, 2017
Presentation Overview
2
• Welcome and Introduction of Webinar Speakers • Overview of the Healthy Start (HS) Program and Evaluation
• Purpose of Data Sharing/Use Agreement • Overview of Data Sharing/Use Agreement Signature and Submission Process • Resources Available to HS Grantees, Vital Records Offices, and PRAMS
Programs
• Tasks for HS Grantees, Vital Records Offices, and PRAMS Programs • Timeline of Activities • Q&A Session
• Established in 1991 as a presidential initiative to: • Reduce racial and ethnic disparities in infant mortality; • Improve local health care systems; and • Increase consumer and community voice in health care
decisions.
• Targets communities where infant mortality is highest.
• Enlists full community support.
• Encourages innovation and creativity.
The National Healthy Start Program
3 3
Five (5) Approaches of Transformed Healthy Start Program
4
1) Improve women’s health;
2) Promote quality services;
3) Strengthen family resilience;
4) Achieve collective impact; and
5) Increase accountability through quality improvement, performance monitoring, and evaluation.
4
5
Three Levels of Funding and Accountability
• Level 1: Community-based Healthy Start: • Up to $750,000 annually; 500 clients; implements the essential HS
program activities.
• Level 2: Enhanced Services Healthy Start: • Up to $1.2 million per year; 800 clients; level 1 services plus other
services such as FIMR, PPOR, MMMR; drives collective impact and supports community level change.
• Level 3: Leadership and Mentoring Healthy Start:
• Up to $2 million per year; 1,000 clients; level 2 services and supports the development of a place-based initiative and serves as the backbone organization for achieving collective impact; mentoring and state/national level leadership; participate in HS COIIN.
5
6
National Evaluation Goal
• Goal: To determine the effect of the transformed HS program on changes in participant-level characteristics (e.g., behaviors, HS services utilization, and health outcomes)
• National Evaluation Components • Implementation • Utilization • Outcome
7
Evaluation Aims Implementation Evaluation: • To document the implementation of the transformed HS program components (e.g.,
activities, type of services, intervention models) and their alignment with the five HS approaches.
• To examine factors that help explain effective implementation of the transformed HS program.
Utilization Evaluation: • To assess how many women and infants participated in the transformed HS program.
• To assess to what extent services were delivered to the highest risk target populations (women and infants), as intended.
• To examine factors (personal, program, organizational) that help explain the volume of services used (e.g., high service delivery versus low service delivery programs).
Outcome Evaluation: • To assess the transformed HS program’s impact on HS participants compared to non-HS
controls.
• To examine factors (program/organizational) of the transformed HS program that are associated with improved participant behaviors, utilization, and health outcomes.
8
Outcome Evaluation Questions
The outcome evaluation is designed to answer the following questions: • What impact did the transformed HS program have on
HS participants when compared to non-HS controls? • What factors (program/organizational) of the
transformed HS program are associated with improved participant behaviors, utilization, and health outcomes?
9
Outcome Evaluation Data Sources
Client Level Data (For all HS Grantees)
• All HS participants will complete client-level screening tools at enrollment and follow-up visits
• Data will be used for quality improvement (internal pre-post comparisons), crude benchmarking compared with national databases, and to assess dose effects of HS participation when linked to vital records and PRAMS
Vital Records (For all HS Grantees)
• All HS participants will be linked to Vital Records
• Data will be used to compare HS participants and non-participants with strong generalizability and power (100% of grantees) but less robust internal validity due to more limited information on control and outcome variables
PRAMS (For 15 HS Grantees)
• A stratified, random sample of HS grantees (15) will be selected for PRAMS oversampling
• Data will be used to compare HS participants and non-participants with strong internal validity (many control and outcome variables) but less external validity (15% of grantees)
10
Outcome Evaluation
Data Sources: Client-level Data, VR, PRAMS Analysis Plan: • Will estimate the effect of program participation by comparing outcomes of HS
participants and non-participants using multivariate techniques.
• Individual-level propensity score matching will ensure that outcome comparisons between participants and non-participants are balanced with respect to observed characteristics.
• Multiple comparison groups, including internal references among program participants, will be used to test the sensitivity of results and promote causal inference (e.g., postpartum versus prenatal enrollees, dose-response effects).
• Analyses will also examine variation in effects by program and organizational characteristics to identify critical practices that can be spread and scaled to maximize impact across grantees.
11
Linking Data Sources
HS Participant Individual Identifiers
•Grantees provide individual identifiers to state Vital Records Offices
•Individual Identifiers include: •Mother’s: name (first, last, maiden); DOB; address at time
of delivery (street, city, zip code); SSN, gravidity, parity •Infant’s: DOB (or expected month/date of delivery if
unknown); birth hospital; sex; name (first, last) •Date of enrollment •Unique client-id #
Vital Records
•Link HS participants to vital records (e.g., birth certificate) for all grantees using a unique client-id; linkage performed one-time but may be annual.
•Identify HS participants for PRAMS oversampling for 15 randomly selected grantees; linkage to birth certificate performed monthly.
PRAMS •Using linked birth certificate, sample
all HS participants in 15 randomly selected grantee locations to take PRAMS survey.
Client-level Data
Using client-ID, client-level data will be linked to the 2017 vital records (100 grantees) and PRAMS (15 grantees) to assess services received and impact on HS participants.
The HS client-ID and enrollment date will be maintained throughout the linkage process.
12
Collecting Participants’ Individual Identifiers
13
For participants that agree to be in the evaluation and have a known or expected delivery in CY 2017, grantees should collect the following Individual Identifiers: • Mother’s name • Mother’s date of birth (or age in years) • Mother’s date of enrollment • Mother’s Unique Healthy Start Client ID # • Mother’s address at time of delivery (street, city, zip code, county) • Mother’s social security number (while this identifier is not required, it is strongly recommended to
collect this identifier to increase successful linkage to the vital records) • Mother’s race • Mother’s ethnicity • Mother’s Medicaid status (yes/no) • Mother’s gravidity (# previous pregnancies) • Mother’s parity (# previous live births) • Infant date of birth (or expected month or date of delivery if unknown) • Infant birth hospital • Infant sex • Infant name (first, last) • Infant birth weight
*Healthy Start grantees should collect as many individual identifiers as possible, but please note the bolded identifiers are required for successful linkage to vital records.
Collecting Participants’ Individual Identifiers (cont’d)
14
• Individual identifiers will be used to link the HS participant to her infant’s vital records.
• The more identifiers collected and submitted to the VRO, the more likely the linkage will be accurate.
• Grantees will submit participant individual identifiers to their state/jurisdiction VRO.
• Monthly – Monthly submission is ONLY for 15 sites participating in the PRAMS oversample; monthly submissions will begin in March 2017 and end February 2018.
• One-Time Submission – The one-time submission is for ALL HS grantees and it will occur in April 2018 (may be repeated annually).
• All HS grantees will provide individual identifiers to VROs for participants with a known or expected delivery in CY 2017 thru March 2018 – we want to ensure we capture any participants that may deliver prematurely.
Vital Records and PRAMS Linkage
15
1. State/jurisdiction VROs receive HS participant’s individual identifiers.
2. VROs link HS participants to their infants birth certificate (conducted monthly for PRAMS oversample sites and one-time for remaining HS sites).
3. VROs will flag all matched records by appending the HS client-id # and date of enrollment to the record.
4. For PRAMS oversample sites, VROs will transfer the flagged HS records to the state PRAMS programs monthly. • PRAMS programs will sample and survey the flagged HS participants in monthly
batches. • PRAMS programs will monitor survey response rates for HS participants vs. non-
participants.
5. VROs will calculate linkage rates for HS participants with a known delivery
date in 2017 (aim of ≥95%) and those with an expected delivery date in 2017 (aim of ≥80%).
Vital Records and PRAMS Linkage (cont’d)
16
6. After completing linkages, VROs will transfer birth data to MCHB/HRSA for HS participants and non-participant controls in the same county. • A year later, VROs will update linkages to include death certificates and transfer records to
MCHB/HRSA
7. CDC PRAMS will transfer the full PRAMS file for both HS and non-HS PRAMS respondents in the selected states to MCHB/HRSA.
8. Using the HS client-id # and date of enrollment, MCHB/HRSA will link participant-level data (e.g., Screening Tool data) to participants’ vital records and PRAMS data.
9. HRSA evaluation team and contractor (Abt Associates) will conduct evaluation analyses.
17
Data Sharing Agreement Purpose & Process
Purpose & Overview of the Data Sharing/Use Agreement
18
• Data Sharing/Use Agreement is between HS grantees, Vital Records Offices (VRO), PRAMS programs (where applicable), and MCHB/HRSA.
• Purpose of the Data Sharing/Use Agreement • To obtain agreement that HS grantees will provide linkage variables (i.e.,
participant individual identifiers) to state/jurisdiction VROs, and that VROs and PRAMS programs will provide data to MCHB/HRSA.
• The agreement outlines the following: • Description of the requested data; • Scope of the agreement; • Permitted uses and limitations of data use; and • Data transmission schedule and funds provided.
• Agreement is a model/exemplar document and can be modified as needed.
Process for Signing Data Sharing/Use Agreement
19
1) VROs review and sign Data Sharing Agreement (DSA) and submit it to MCHB/HRSA.
2) MCHB/HRSA provides a legal review of any suggested revisions by VROs and sends DSA to HS grantees and PRAMS programs (where applicable).
3) HS grantees and PRAMS programs review and sign the DSA and submit it to MCHB/HRSA.
4) MCHB/HRSA provides a legal review of any suggested revisions, resolve any differences between VROs and changes requested by HS grantee and/or PRAMS program.
5) MCHB/HRSA signs the DSA. *********************************************
• Reminders will be sent to check the status of signatures and determine expected submission dates.
• Abt Associates, in partnership with NAPHSIS, will provide monitoring and technical assistance to VROs, HS grantees, and PRAMS programs throughout the process.
20
Data Sharing Agreement Details
NAPHSIS Ron Hyman
Goals
• Review the proposed Data Use Agreement • Highlight sections that may require jurisdiction-specific
information/modification • Discuss signatures required by various parties to the agreement. • Review timelines • Discuss flow of the agreement documents between agencies • Questions
Review – 1 of 3
• Introduction & Purpose • Description of Data • Scope of the Agreement • Representation of Authority – Insert your jurisdiction's authority • Right to Use Data/Ownership • Permitted Uses of Data • Limitations on Data Use
Review – 2 of 3
• Data Transmission Schedule and Fees • Transmission formats/mechanisms
• Authorization of Users • Healthy Start grantee’s staff
• Data Retention • Confidentiality and Data Security • Unauthorized or Inadvertent Use or Disclosure
• Who contacts who? When? How? • Term and Termination of Agreement
Review – 3 of 3
• Disputes • Notices
• Designated Representatives
• Publications • Information Disclaimer • Binding Nature and Assignment • Governing Law • Force Majeure and Entire Understanding and Modification
Signatures
• Format • Who is authorized to sign? • Order of agency signatures • Number of originals?
Timelines/Flow
• Deadline Date – March 15, 2017 • Getting agreement on final language by all parties • Reviews
• By whom? Legal, Upper management, Contracts, etc.
• Communication between partners • Preventing things from getting stuck/falling through the cracks • Don’t assume -> Keep communications flowing both ways
Questions
• How can we help? • Obstacles?
Resources for HS Grantees and Vital Records Offices
MCHB HS Evaluation Contract will provide: • Outreach to HS grantees and VROs to facilitate customization and signing of
the model data sharing and transfer agreements developed by NAPHSIS. • Promote and monitor the timeliness of data transfer from HS grantees to VROs. • Provide technical assistance (TA) to HS grantees regarding collection and
transfer of individual identifiers, as needed. • Provide TA to VROs regarding data linkage and transfer (e.g.,
software/hardware requirements, linkage protocols, transfer mechanisms and formats), as needed.
• Develop and implement a process to monitor birth and death certificate linkage rates overall and by available data (i.e., known versus estimated date of delivery).
• Work with VROs and HS grantees to improve linkage rates, where necessary and possible.
• Provide guidance on outreach activities to HS grantees to promote and improve HS participants’ response rates to PRAMS.
CDC Resources for PRAMS Programs
• Project Coordinator • Hire new coordinator to provide ongoing project
management/coordination.
• PRAMS Statistician • Statistical support to develop and modify sampling plans for
participating PRAMS sites, and provide ongoing statistical support related to PRAMS variables/files and analysis of linkage results.
• Enhanced Technical Assistance • PRAMS Program Managers will provide enhanced TA to selected
PRAMS programs oversampling HS participants.
Funding for Annual Linkage
• Vital Records Offices (n=39) • Upon signing the data sharing/use agreement(s), VROs will
receive a one-time disbursement through the VSCP contract with NCHS.
• Funding amount depends on the number of projected HS participants per state/jurisdiction.
• <500 participants $2,000 • 500 - 1,749 participants $2,500 • ≥1,750 participants $3,000
• These funds will cover staff time to conduct the linkage between
vital records (once each for birth and death certificates).
Funding for PRAMS Oversampling • Healthy Start Grantees (n=15)
• Shall receive a one-time disbursement of $1,500. • The funds may be used for outreach, materials, and training to promote HS
participants’ response rates to PRAMS.
• Vital Records Offices (n=13) • Shall receive a one-time disbursement of $6,000 per HS grantee selected
(for example, if state has 2 HS locations included, the VR Office will receive $12,000).
• The funds will cover staff time for monthly birth certificate linkages and the alteration and testing of the sampling program/algorithm to accommodate the PRAMS oversample.
• PRAMS Programs (n=13)
• Shall receive funds in the amount of $100 per birth/HS respondent (e.g., PRAMS program would receive $25,000 for a Level 1 HS grantee expected to have 250 births/year).
• The funds will cover staff time for the additional interviews, survey printing, incentives, supplies, mailings, and data entry required by the oversampling.
Tasks for Healthy Start Grantees
33
1) Sign data sharing/use agreement.
2) Consent participants to provide services and participate in the HS evaluation.
3) Collect as many individual identifiers as possible from the participants that agree to participate in the evaluation AND have a known or expected delivery in CY 2017.
4) Complete a one-time transfer of all requested individual identifiers to VRO in April 2018. • May be repeated annually; transfer mechanism and format to be
determined with the data sharing/use agreement. • Grantees participating in PRAMS oversampling will be transferring
participant data monthly beginning in March 2017.
Tasks for Vital Records Offices
34
1) Sign data sharing/use agreement.
2) Link HS participants to their infant’s birth certificate in May 2018.
3) Complete linkage to include any subsequent infant death certificates (May 2019).
4) Transfer data to HRSA (data can also be transferred to grantee if requested.
Tasks for PRAMS Programs
35
1) Sign data sharing/use agreement.
2) Assign project lead and develop plan to implement oversample and maintain response rates.
3) Adjust sampling plan to include HS participants, and include these participants in monthly batch samples.
4) Monitor response rates for HS participants versus non-participants.
5) Provide MCHB/HRSA with the full PRAMS file of all PRAMS participants in the selected states. • Maintain HS client ID and enrollment date to identify HS participants.
Timeline
2017
• January - March: • Circulate model data sharing/use agreement between HS grantees,
VROs, PRAMS programs (where applicable), and MCHB/HRSA.
• January: • HS grantees begin collecting individual identifiers (informed consent to
be provided).
• March - May: • Data sharing/use agreement between Vital Records Office (VROs),
Healthy Start grantees, PRAMS programs (where applicable), and MCHB/HRSA in place (MCHB/HRSA and Abt Associates will facilitate this process).
Timeline (PRAMS only) 2017
• March:
• HS grantees selected for PRAMS oversampling begin monthly transfer of participant individual identifiers to VROs (data template and technical assistance will be provided).
• Subsequent monthly transfers through March 2018 will add updated information for previously submitted participants.
• VROs begin monthly linkage for PRAMS batch sampling (this will continue until February 2018).
• April: • PRAMS programs begin including HS participants delivering in CY 2017 in
monthly PRAMS batches (April 1 sample mostly reflects January births).
Timeline 2018
• April:
• All HS grantees provide individual participant identifiers to VROs for those with a known or expected delivery in CY 2017 through March 2018 (to capture any participants that may deliver prematurely).
• May: • VROs complete linkage and transfer 2017 linked HS participant birth
certificate data + non-participant data in same counties served by HS grantee to MCHB/HRSA (birth certificate elements, format, and transfer mechanism TBD).
• September: • CDC PRAMS provides MCHB/HRSA with the full PRAMS file of all PRAMS
participants in the selected states (both HS participants and non-participants).
• VROs participating in PRAMS oversampling send any additional requested birth certificate items for PRAMS sample to MCHB/HRSA.
Timeline 2019
• May:
• VROs update linkage of HS participants and non-participant controls to include any subsequent infant death certificates and send data to MCHB/HRSA.
• VROs participating in PRAMS oversampling update linkage of any infant
deaths to PRAMS sample and send data to MCHB/HRSA.
Q & A Session
Thank You!
Contact Information:
Jamelle Banks, MCHB/HRSA Phone: 301-443-1726; Email: [email protected]
Sarika Rane Parasuraman, MCHB/HRSA
Phone: 301-945-3116; Email: [email protected]
