19
Data Quality in TC LHIN: Challenges, Opportunities, & Why it Matters Caroline Bennett-AbuAyyash Human Rights & Health Equity Office
Data Quality in TC LHIN:
Challenges, Opportunities,
& Why it Matters
Caroline Bennett-AbuAyyash Human Rights & Health Equity Office
2 years of data collection:
Demographic data from ~130,000 patients
What language would you feel most comfortable speaking in with your health-care provider?
Spoken Language
Were you born in Canada? If NO, what year did you arrive in Canada? Born in Canada
Which of the following best describes your racial or ethnic group? Race/Ethnicity
Do you have any of the following? [disability list] Disability
What is your gender? Gender
What is your sexual orientation? Sexual Orientation
What is your total family income before taxes last year? Income
How many people does this income support (including yourself)?
#PPL income supports
The First Next Step: Data Quality
• Data is generally considered high quality if: "they
are fit for their intended purpose(s) in a given
context
Know
who we
serve
Develop
interventions
Deliver
patient-
tailored
care
Identify
inequities
BAD DATA
Data NOT used
Data IS used
Rely on unclear & inaccurate information
Create distorted pictures
Missed inequities
Unreliable baseline
Poor legitimacy Bad
decisions
Inaccurate reporting
INEQUITIES PERSIST
Cost of ‘data acquisition’
Time for training data collectors, collecting data, data entry, meetings
Cost of materials, system upgrades, allocated hours for related activities
“There are multiple reasons why data quality problems are not addressed.
These range from low awareness of the cost of data quality, tolerance for
errors, to skepticism over the ability to improve things and see returns”*
*Source: Diamond, C. C., Mostashari, F., & Shirky, C. (2009). Collecting and sharing data for population health: A new paradigm. Health Affairs, 28, 454-466.
POOR DATA QUALITY = Added cost of quality improvement efforts
Where to start Quick wins for assessing data quality
How to plan Strategies and practices for improving data quality
Achieving High Quality Data
Participation Rates: A Quick Win
Participation
Rates
TC LHIN Hospital Participation Rates
>80% participation 7 hospitals
50%-80% participation 2 hospitals
20%-50% participation 3 hospitals
<20% participation 1 hospital
Missing Data Rates: A Quick Win
1.0% 7.0%
2.1%
3.1%
29.4%
11.7%
3.4%
9.2%
0.2%
0.9%
7.4%
5.1%
5.9%
8.4%
34.7%
21.5%
0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% 45.0%
Language
Born in Canada
Racial/Ethnic group
Disability
Gender
Sexual Orientation
Income
# Supported by Income
Missing
PNA/DNK
TC LHIN Hospital Missing Data Rates
Feedback
(qualitative)
• Data collectors
• Other staff
• Patients
Getting Feedback: A Quick Win
TC LHIN
hospitals’
feedback
Improving Data Quality
Pre-data collection
•Training on collection & entry
•Clear workflow
•Organizational readiness
Ongoing Data Collection
•Data verification
•Patient engagement
Post-data collection
•Quantitative: Data reports
•Qualitative: Feedback
•Audits: Data entry checks
Pre-data collection: TC LHIN
• UHN Communication
Plan
• Over 90% reported
that all staff received
training on data
collection
• Formalize data collection workflow
• Introduce clear data entry guidelines
• Ensure systems differentiate between ‘never asked’
and ‘declined to participate’
• Support respectful and accessible environments
Successes Next Priorities
Improving Data Quality
Pre-data collection
•Training on collection & entry
•Clear workflow
•Organizational readiness
Ongoing Data Collection
•Data verification
•Patient engagement
Post-data collection
•Quantitative: Data reports
•Qualitative: Feedback
•Audits: Data entry checks
• Over 85% of hospitals
include one-on-one
interaction with
patients during data
collection
• Refresher training (e.g. one hour e-learning)
• Address staff anxiety with responding to patient
questions
• Support respectful and accessible environments
Successes Next Priorities
Ongoing Data Collection: TC LHIN
Improving Data Quality
Pre-data collection
•Training on collection & entry
•Clear workflow
•Organizational readiness
During data collection
•Data verification
•Patient engagement
Post-data collection
•Data audit reports
• Feedback
•Accountability
Post-data Collection: TC LHIN
• All hospitals have a
demographic data
summary dashboard
• Address IT issues with pulling data reports
• Incorporate staff feedback into data collection
processes
• Support respectful and accessible environments
Successes Next Priorities
The Thorn in our Side: IT Systems
Common issues emerging:
- IT build that doesn’t differentiate between ‘sex’ and ‘gender’
- Sample sizes that differ between questions
- Data summaries significantly smaller than expected
- Difficulty differentiating between ‘missing’, ‘declined’, and ‘was not
asked’
Assigned at birth, refers to biology
(organs, hormones) Person’s sense of self and
can be male, female, trans,
two-spirit, gender queer, …
BEFORE ADDRESSING IT ISSUES- ASK:
• Do you have any existing issues with data collection?
• Are you sure it’s an IT problem and not data collection issue?
Moving Forward: Measuring Health Equity in TC LHIN
Assess Performance
SCORE CARD
Completion Rate: 75% of all patients (TC LHIN target)
Participation Rate: >80% of patients agreed to
participate
Missing Data Rate: <10% missing data
Pre-data collection
•Training on collection & entry
•Clear workflow
•Organizational readiness
Ongoing Data Collection
•Data verification
•Patient engagement
Post-data collection
•Data reports
• Feedback
•Accountability
Focus on Best Practices
17% IT/technology problems
14% organizational management issues such as
disinterest or apathy regarding data, lack of
accountability
12% poor planning or lack of planning
11% lack of training
10% data entry issues
9% lack of controls/responsibility
7% collaboration problems
4% lack of resources
3% lack of expertise in dealing with data
2% “everything”
What’s the
major
cause of
government
data
problems?*
*Responses from 74 officials in 46 states (More info here)
Thank You!
