Testing and Implementing Change

Learning Session 2November 14, 2002

Vicki Grant&

Ron Moen

What are we trying toaccomplish?

How will we know that achange is an improvement?

What change can we make thatwill result in improvement?

Model for Improvement

Act Plan

Study Do

What are we trying to accomplish?

Maximize enrollment and retention of adults and children in Medicaid, SCHIP and Food Stamps (where applicable) by improving county and state eligibility processes

·       

M1=Caseload

Cases beginning of month + approvals in month - closures in the month

M2=% Approvals

100 x approvals in the month/total application decisions made in the month

M3=% Denials

100 x denials in the month/total application decisions made in the month

M3.1=Denials by reason

100 x denials by reason /total denials

M4=% Closures

100 x closures in the month/cases at beginning of month + approvals

M4.1 Closures by reason

100 x closures by reason /total closures

100 x renewal closures in month / cases up for renewal in month

M4.2% Closed Renewals

How will we know that a change is an improvement?

Improve Customer service Improve Policy/Procedures Improve Work FlowChange Work Environment Improve Intra-system CommunicationsError ProofingEliminate WasteFocus on Variation

What changes can we make that will result in an improvement?

Aim: Improve the notification system

Redesigning the Medicaid notice will improve communication

Improved communicatio

n

A PS D

APS D

A PS DD S

P A

DATAD SP A

Cycle 1:Test on employees

Cycle 2:Field test on customers in 1 office in 1 week

Cycle 3:Field test at a different site for 1 week

Cycle 4: Field test at pilot site for 1 month

Cycle 5: Implement and spread to county/state

Project: Cycle #: 2 Date: Sept. 6, 2002

Objective: Determine if the redesign of the Medicaid notice will improve communication to our customers

Plan

Questions Predictions1. Did we simplify the sentences? (Yes, because we “unpacked” them)2. Did we simplify the vocabulary?(Yes, used a language they can read)3. Is the new format easier to understand? (Yes, better organization

makes the text less busy)

What data will be collected during this time? (Forms to be used) Preference scale (Which is better: current or new)

Who: Next customer after 8:00, 9:00, 10:00, etc Administered and observation by Bob from our team

What: Preference test with the new Medicaid notice

When: Tues-Thursday collect data, Team will analyze data on FridayWhere: Field test at the Glendale site

Use the PDSA Cycle for :

Testing or adapting a change idea Implementing a change Holding the gains Spreading the changes to the rest

of your system

3 Principles for Testing a Change

Test on a small scaleCollect data over timeBuild knowledge sequentially and

include a wide range of conditions in the sequence of tests

Testing on a Small Scale

• Have others that have some knowledge about the change review and comment on its feasibility

• Test the new product or the new process on the members of the team that developed the change before introducing it to others

• Incorporate redundancy in the test by making the change side-by-side with the existing process or product

Testing on a Small Scale (Cont.)

• Conduct the test in one facility or office in the organization, or with one customer

• Conduct the test over a short time period

• Test the change on a small group of volunteers

• Develop a plan to simulate the change in some way

Decrease the Time Frame for a PDSA Test Cycle

YearsQuartersMonthsWeeksDaysHoursMinutes

Drop down next “two levels” to plan Test Cycle!

Conserve Measurement Resources

• Integrate into daily work• Use sampling

Length of Stay for Main ED Discharged Patientsn=1 per week

50

100

150

200

250

300

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29Week

LO

S (M

in.)

Avg=180, SD=50

Avg=135, SD=35

Median Length of Stay for Main ED Discharged Patientsn=14 per week

100

120

140

160

180

200

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Week

LO

S (M

in.)

Median Length of Stay for Main ED Discharged Patientsn=28 per week

100

120

140

160

180

200

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Week

LO

S (M

in.)

Median Length of Stay for Main ED Discharged Patientsn=300 per week

100

120

140

160

180

200

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Week

LO

S (M

in.)

Team examples of testing

Use teams reports and story boards to fill in here

Minnesota: Use of credit card payments

Delaware: Test verification check-list

Nassau County: Telephone survey of mail recertification process

Testing – Trying and adapting existing knowledge on small scale. Learning what works in your system.

Implementation – Making this change a part of the day-to-day operation of the system (Not after just one test!)

Testing vs. Implementation

Cycles for Implementation

The change is permanent - need to develop all support processes to maintain change.

High expectation to see improvement (no failures).

Increased scope will lead to increased resistance.

Generally takes more time than tests.

Approaches to ImplementationParallel

Changes run parallel as current processes are running

When old processes “turn off”, the new ones “turn on”

Sequential Incorporates changes over time (completeness)

or by location (coverage)“Just Do It”

One cycle to study the effect of the change Used for simple changes only

Holding the Gains Communicate

As soon as aim is chosen Continuing as project progresses through

storyboards, newsletters, etc. Successes and learning

StandardizationDocumentationMeasurementTraining

Spread

Spread means making improvements beyond the scope of the Collaborative Team

Readiness Success of collaborative team Intention Organizational priority Leadership responsibility

Target Population for Spread

Population of Focus for your BTS Aim

Spread

- Other offices - Other departments

- Other counties - Other key systems