Part I – Data Collection and Measurement

Ruth S. Gubernick, MPHQuality Improvement Advisor

Lori Morawski, MPH CHESManager, Quality Improvement Programs

Florida Pediatric Medical Home Demonstration Project Learning Session I

September 23-24, 2011

Disclosure

We have no relevant financial relationships with the manufacturers of any commercial products and/or provider of commercial services discussed in this CME activity. We do not intend to discuss an unapproved/investigative use of a commercial product/device in their presentation.

Measurement, Data Collection & Evidence of Change

Importance of measurement – Why? How will we know that a change is an improvement?

Clarify and be directly linked to goals Seek usefulness over perfection Be integrated into daily work whenever possible Be graphically and visibly displayed For PDSA cycles, be simple and feasible enough to accomplish in close time proximity to tests of change

How will we know that a change is an improvement?

Plan

DoStudy

Act

What are we trying to accomplish?

What changes can we make that will result in improvement?

The Improvement ModelThe Improvement Model

The Improvement GuideAssociates in Process Improvement

Measurement for Quality Improvement

You can’t improve what you can’t (or don’t) measure

A good aim statement provides clear direction. Measures tell a team if the changes they make

are making a difference Measurement tells you where you are and

where you are going

Measures

Need to define Target population Numerator Denominator

Florida Pediatric Medical Home Demonstration Project: Example

Target Population: All Medicaid/KidCare infants/children seen by clinicians

in participating practice

Numerator: Total number of patients whose individual and family

concerns are elicited at this visit

Denominator: All Medicaid/KidCare infants/children seen in

participating practice whose charts are reviewed during the month of interest.

Effective Measurement

Seek usefulness, not perfection

Keep measurement simple, think big, but start small

Effective Measurement: Outcomes

Outcome measures: represents the voice of the customer or patient

Hospitalizations or ED visits due to asthma Hospital readmissions w/in 30 days due to

asthma Patient satisfaction with time it takes to

schedule an appointment

Effective Measurement: Processes

Process measures: represents the workings of the system Percent of patients with all expressed

concerns addressed or with plans made to address them

Percent of patients who have a medical summary or comprehensive care plan created or updated/maintained at this visit

Effective Measurement

Build measurement into daily work routine Data should be easy to obtain and timely Small samples over time

Use quantitative and qualitative data Quantitative data is highly informative Qualitative data is easy to obtain

Why Plot Data Over Time

You develop a process for patients/families to have a current copy of their medical summary or comprehensive care plan reviewed and offered to them at their visit. The 6 months before implementing the process the average % of patients/families having a current copy of their medical summary or comprehensive care plan reviewed and offered to them is 10%. Six months after the process is implemented, the average % is 90%.

How will you answer the question: was this change an improvement?

0102030405060708090

100

Change

Run Charts

Aug. 2

011

Septe

mbe

rO

ctob

erNov

embe

rDec

embe

rJa

n. 2

012

Febr

uary

Mar

ch

ChangeAug

. 201

1Sep

tem

ber

Oct

ober

Novem

ber

Decem

ber

Jan.

201

2Fe

brua

ryM

arch

Data Collection Web Site www.aap.org/qualityimprovement/quiin/workspaces/MedHome/DataCollection.html

Requirements of Teams

From October 15, 2011 through March 30, 2012 you will be asked to submit a total of 10 patient chart reviews using the EQIPP.

Data cycles will be open on the 15th of each month. Please do not submit data until the 15th of every month. Data will be due by the 30th of each month. Your practice will have the ability to close your own data cycle once you

have submitted 10 total chart reviews each month. If you do not have 10 charts to review in a particular month, please

contact project staff and they will manually close your data cycle for you.

You will need to complete the on-line chart review survey for each patient chart you review. You have the option of first completing a “hardcopy” survey of each chart reviewed, using the PDF Data Collection Tool, but you must then submit that data using EQIPP.

Data Collection Tool

Chart Review Log Sheet

Date of visit Log Number(“Patient Code”)

Patient NameMedical Record or other ID #

Medicaid, Medicaid health plan orKidCare

ID’d as CYSHCN

001

002

003

004

005

006

007

008

009

010

EQIPP Data Input

EQIPP Data User View

Each individual user can analyze their own results real time!

Data Reporting

QI Advisor and AAP staff will run monthly reports to share with participating practices: Project measure reports Measure Reports (all practices) Practice Reports (all measures)

Data Explorer and Reporting

Reports

Reports include comprehensive data for all practices – provide more information than EQIPP

Reports will be shared monthly prior to monthly calls Reports will be posted on project workspace Monthly calls:

3rd Wednesday of the month 2nd Tuesday of the month

Monthly Progress

Provides information about Tests of change completed and tools used each month Assessment of team progress Other qualitative measures

Instructions Insert your practice’s Aim statement. Indicate the change package items you have tested. Describe specific

changes (by domain) and tools you have tested. Rate your team’s progress using the scale. Report your team’s learning. Complete the Systems Index. Submit your Monthly Progress Report by the 30th of the month for which

you are submitting, using the Survey Monkey link (URL) that will be sent each month via the project listserv

Example Chart from Monthly Progress Report Index

Practice-based Systems Index by Month

012345

6789

10

Apr-0

9M

ayJu

ne July

AugSep

tOct Nov

To

tal

Nu

mb

er o

f B

oxe

s C

hec

ked

Aug.

2011 Se

pt

Oct

.

Nov

.

Dec Ja

n.20

12 Feb.

Mar

ch

Project Workspace

All data and run charts will be available for review on the Project Workspace Web site

http://www.aap.org/qualityimprovement/quiin/workspaces/

MedHome/Home.html Compare your team’s data to other teams and to the

aggregate data!

Data Collection Cycle Table

Data Cycle Table

Data Cycle Label Month of Data Cycle

Data Cycle Opens Data Cycle Closes Data to be submitted

Baseline August August 15, 2011 August 30, 2011 1st 20 chart of Medicaid, Medicaid health plan or KidCare patients seen in August

Follow-up 1 October October 15, 2011 October 30, 2011 1st 10 charts of Medicaid, Medicaid health plan or KidCare patients seen in October

Follow-up 2 November November 15, 2011 November 30, 2011 1st 10 charts of Medicaid, Medicaid health plan or KidCare patients seen in November

Follow-up 3 December December 15, 2011 December 30, 2011 1st 10 charts of Medicaid, Medicaid health plan or KidCare patients seen in December

Follow-up 4 January January 15, 2012 January 30, 2012 1st 10 charts of Medicaid, Medicaid health plan or KidCare patients seen in January

Follow-up 5 February February 15, 2012 February 28, 2012 1st 10 charts of Medicaid, Medicaid health plan or KidCare patients seen in February

Follow-up 6 March March 15, 2012 March 30, 2012 1st 10 charts of Medicaid, Medicaid health plan or KidCare patients seen in March

Teams should not submit data until the Data Cycle Opens (15th of month)Note: teams can close their own data cycle after submitting 10 charts. If they do not have 10 charts in a particular month, staff will close the data cycle

for them.

Questions