Data for Decision-Making Processes: Linking Data to

Quality Improvement Initiatives

Bisma Sayed, M.S.W.University of MiamiDepartment of Sociology

John Dow, M.S.W.South Florida Behavioral Health Network

Understand the value of utilizing data for decision-making

Determine what should be measured and what data elements should be used

Data analysis and interpretation

Recognize limitations

Validate findings using other data sources

Objectives

The recent recession coupled with health care reform has had cascading consequences on behavioral health care service delivery in Florida.

Current funding limitations and budget cuts have increased the urgency for cost-effective and efficient delivery of behavioral health care services.

How can behavioral health care organizations lower cost, raise quality, and still offer accessible services to increasing numbers of consumers?◦ Meet standards◦ Coordinate◦ Demonstrate outcomes◦ Eliminate duplication◦ Produce reportable, effective, and sustainable

results

Behavioral Health Care Service Delivery

Quality Improvement vs. Quality Assessment

Quality Improvement

Quality Improvement Processes allow organizations to analyze current practices, identify strengths and weaknesses, set goals, and monitor progress

Quality in the behavioral health care setting may be defined as the ‘extent to which a health care service or product produces a desired outcome’

Behavioral Health Care: Quality Improvement Initiatives

Quality of care measures◦ Effective◦ Appropriate◦ Safe◦ Efficient◦ Responsive◦ Accessible◦ Continuous◦ Capable◦ Sustainable

Quality Improvement Initiatives

Examine current organizational functioning

Identify target problems

Identify quality of care measures

Identify goals (short term or long term)

Measure baseline performance on quality measures

Quality Improvement Processes

Develop and conduct interventions designed to affect the targeted measures

Repeat measurement of performance based on quality indicator

Document and disseminate results.

Quality Improvement Processes

“If you do not measure it (or cannot measure it), it didn’t happen.”

How can we measure it?

Quality Assessment

Data provides the foundation for quality improvement initiatives◦ Timely◦ Transparent◦ Presented with humility◦ Based on past lessons learned◦ Accountability◦ Presented with compassion and understanding

DATA

The shift to evidence based care coupled with increased technological and statistical advances have resulted in an explosion of data. . .

. . . However, this knowledge remains to be harnessed

The influx of data has led organizations to report data, rather than analyze data.

Data Reporting Data Analysis

Behavioral Health Care: Data and Quality Improvement Initiatives

Data

Information

Knowledge

Decision

Action

Quality Improvement and Data: A continuous relationship

Quality Improvement◦ What is happening?◦ What factors affect delivery◦ How can we influence them

Reactive and Proactive

We need data to guide this.

“Data helps to push improvement (by identifying problems) and pull improvement (by identifying opportunities)”

The link between QI and Data

Facts and statistics collected together for reference or analysis

What is data?

•Surveys• Literature

Reviews•Key

informants•Surveillance

data• Focus Groups

Develop overall goal for improvement

Identify objectives using quality of care measures

Identify target populations

Identify data to be collected

Why use Data for Decision Making?

Determine data sources and/or collection method.

Determine data storage, management, and analysis techniques.

Analyze and Interpret Data

Utilize data for decision-making

How to use Data for Decision Making?

Data Collection and Management

Plan◦ Consider scope and purpose◦ Target Audience

Learn ( Do not reinvent the wheel)◦ Literature Reviews◦ Other sources of data

Test◦ Pilot-test on a smaller scale to identify challenges

Team work◦ Involve and Integrate

Data Collection and Management

Internal Data External Data Administrative or Clinical

Regardless of source of data or type of data, it must be reliable and valid◦ What is reliability and validity?

Types of Data

Process mapping: (Who? How long? Steps? Costs?)

Brainstorm

Quantitative or Qualitative◦ Nominal◦ Ordinal◦ Interval◦ Ratio

Data collection techniques and tools

Surveys and questionnaires

◦ Ethical Standards

◦ Confidentiality and Anonymity

◦ Response Rates

◦ Existing Surveys

◦ Guidance

◦ Pilot test

Data Collection

What is your target population? ◦ Consumers? Their families? Providers?

Community?

Data Collection

Clear and Understandable◦ Specific◦ Not loaded or leading◦ No double barreled question◦ No jargon or acronyms

Allow choice of only one option

Provide reasonable ranges of variation in the response options

Data Collection: Survey Questions

Social Desirability Bias

Target towards population◦ Appropriate for age, culture and literacy

Include adequate demographic information

Data Collection: Survey Questions

Why do we sample?

Sampling must be representative of your population

Selection bias

Sampling

Important step that can cause significant error if not done properly

Identify inconsistencies◦ For example, the mean age of adolescents

sampled across the nation is 23.5. The range is 13-56.

◦ Why do we have a 56 year old adolescent?

Data entry, checking, and cleaning

Spreadsheet programs◦ Reporting, not analysis

Database programs◦ Database changes – Store data with reports◦ Reporting, not analysis

Statistical Programs◦ Analysis

Storing and Managing Data

Data Analysis

Understand the variables◦ Categorical and numerical variables

Frequency Distribution Median and Percentile Counts and Sums Measures of central tendency Measures of variability

Analyzing Quantitative Data

Measures of Central Tendency◦ Mean◦ Median◦ Mode

Data Analysis for Numerical Variables

Range

Standard Deviation

What does this tell you about your population?

Measures of Variability

What is the goal of data analysis in QI?

Descriptive Analyses and Measures of Variation are useful, but. . .

Inferential statistics can add to the power of your conclusions.◦ Examine Relationship/Estimate size of difference◦ Confidence Intervals◦ Tests of statistical significance

Statistical Analyses

Correlation Analysis◦ Correlation Coefficient: Pearson Product Moment

Correlation Coefficient (r)

Scatter plots◦ Linear Relationships◦ Non-Linear Relationships

Correlation does not equal causation

Statistical Analyses

Nominal Level Data: Non-Parametric Tests◦ Chi Square◦ Cramer’s V/ Contingency Coefficient/Others

Numerical Data: Parametric Tests◦ T-tests (independent or dependent)◦ ANOVA◦ Regressions

Confidence Intervals◦ What are they? ◦ How can they be used?◦ Sample size matters

Statistical Analyses

When you combine your sample value with the margin of error, you obtain a confidence interval.

The confidence interval is the level of confidence that the sample value represents the true value as seen in the overall population.

Statistical Analyses

For example, the waiting time for appointments for clients referred to your clinic might be expressed as a mean of 13.5 weeks with a 95% confidence interval of 11.6 to 15.3 weeks (95% CI 11.6-15.3).

This means that you expect your population on average would wait between 11.6 and 15.3 weeks for an appointment.

The p value is the probability that the difference you have observed in your study samples could be due to chance.

Smaller p value = lowered probability that results are due to chance

Statistical Significance

Data Analysis: Statistical Significance

The size of the p value depends on the size of the sample, so be aware of possible mistakes that can occur in interpreting these values.

Statistical significance does not mean clinical significance.

Data Analysis: Statistical Significance

Keep it simple Consistent units Decimal Points Include raw numbers and percentages Always include n Identify missing data Group data appropriately

Presenting Data: Tabulating Data

Keep it simple Avoid complexity Clear headings Scale Carefully Raw numbers and percentages Always include n Group data appropriately

Presenting Data: Graphs

Basic population characteristics: Pie chart; bar graph

Measures of magnitude including comparisons: Bar chart or box plot

Presenting Data: Graphs

Frequency: Pie chart; bar chart

Trends over time: Line graph

Distribution of Data: Histogram; Scatter plot

Relationship between two things: scatter diagram

Presenting Data: Graphs

Data Analysis and Interpretation

Whether you are collecting your own data or relying on external sources, there is a difference between compiling/reporting data and analyzing data◦ Data : petabytes◦ Reports : terabytes◦ Excel : gigabytes◦ PowerPoint : megabytes◦ Insights : bytes◦ One business decision based on actual data:

Priceless1

Data Analysis and Interpretation

What is the problem?

What can you improve?

How can you improve?

Have you achieved improvement?

Have we sustained improvement?

Data Analysis and Interpretation

State and national datasets provide important information about key health indicators and can serve as basis for comparison.

However, we must be careful in interpreting and analyzing this data.◦ Understand limitations

Understand how data is presented◦ Mean, Median, Mode◦ Raw sums or percentages

Data Analysis and Interpretation

Level of variables◦ Individual◦ Community◦ State

State level data can help guide decisions, but you must examine individual data in your community to determine if the problem exists at a local level.

Data Analysis and Interpretation

Data Analysis and Interpretation: Ecological Fallacy

What does data drive?◦ Assessment◦ Priority setting◦ Allocation of resources◦ Directives to staff and community◦ Evaluation of clinical outcomes ◦ Basis of QI for providers◦ Feedback◦ Sets the groundwork for comprehensive planning

Using Data for Decision-Making

Assess performance and identify gaps

Understand the needs and opinion of stakeholders

Prioritize problems and improvement projects

Establish overall aims and targets for improvement

Using Data for Decision-Making

Establish a clear case for the need for improvement.

Data assists in sustained improvement: feedback to reinforce change and demonstrate benefits.

Using Data for Decision-Making

Thank you