Building & Improving a Performance

Management System

Public Health Foundation

Utah Department of Health

Amanda McCarty, MS, MBA

…PHF Mission:

We improve the public’s health by strengthening the quality and performance of public health practice

Innovative Solutions.

Measurable Results.

www.phf.org

2

Workshop Objectives

– Learn to use a PM System model based on Quality

Improvement (QI) principles, the Essential Services of

Public Health, and the domains of PHAB to structure a

PM system for your public health agency

– Practice using parts of the model so you can help agency

program managers develop goals, objectives, and

performance measures in ways that support improving

health outcomes

– Target engagement of key stakeholders to help make PM

system development successful

3

Agenda

1. Turning Point-compatible PM System Model

based on Quality Improvement (QI) Cycles

2. Parts of a QI-based PM System: – Aligning goals and objectives for a selected program

– Drafting performance measures for a selected program

3. PM System Development as QI: Design,

Deployment, Assessment, & Improvement – Analyzing stakeholders to engage in system development

4. Action Planning

4

Overview: Context

The Turning Point Framework

5

• Performance Standards: Organizational or system expectations

to improve public health practices based on internal or external

goals or benchmarks

• Performance Measures: Clearly defined indicators for collecting

data to assess achievement of standards

• Reporting of Progress: Documenting and analyzing results vs.

expectations and communicating such information as feedback to

guide future performance improvement decisions

• Quality Improvement: A process to manage change and improve

performance in public health policies, programs, or infrastructure

based on standards, data, and reports

The Four Major Parts of Turning Point

6

The Turning Point Framework

– A good description of what you need for performance

management but does not provide guidance on developing an

organization-wide Performance Management System

– Questions this model leaves unanswered are:

– How do we select standards & measure against them?

– What process do we use to determine what programs or

practices to improve?

– How do we make this a “system” to manage our

organization, not just more things to do?

– How do we do it in the PH context, e.g., incorporating

SHA/CHA, SHIP/CHIP, MAPP, Strategic Planning, and

Accreditation?

7

Performance Management &

Performance Management Systems

Performance Management:

Using performance information to help make better

decisions.

Performance Management System:

Using performance information on a regular basis as

part of a continually repeated cycle of performance

monitoring, analysis, and improvement, in which

measured results are fed back into decision making to

improve future performance.

8

QI “PDCA” Cycle Meets the PM System Definition

9

Check

Act

Do

Plan

Sometimes called “PDSA” for “Plan-Do-Study-Act”

Develop improvement plans & targets

Implement improvements & collect data

Monitor, analyze, & interpret data

Decide on further improvements

As applied to specific processes & practices

Organization-wide PM Systems

– Applies a PM cycle to management and governance systems

throughout the organization, usually involving:

• Planning: strategic, policy, operational, & financial (e.g., budget)

• Operations: program & policy implementation

• Monitoring & analysis of performance at all levels: from strategic

to operational (sometimes to employee level)

• Decision making to sharpen plans & strategies and improve

performance at all levels

– Can incorporate PH context such as SHA/CHA, SHIP/CHIP,

the essential services of PH or domains of PHAB

– Can be viewed as a large-scale “systemic” QI PDCA Cycle

10

QI Framework for a Public Health Organization-wide PM System

11

Plan SHIP/CHIP (e.g., MAPP)

Strategic Plans

Operating Plans

Financial Plan (Budget)

Do Program & Policy Implementation

Operational QI

Check Performance Monitoring

& Reporting for the Population & Organization (incl. SHA/CHA, MAPP)

Analysis of Results

Act Performance-based

Decision Making

Decisions on Changes

to Improve Results P

D

C

A

Expectations

Data Information

Direction

Engagement of:

• Leaders & managers

• Employees

• Partners

• Other stakeholders

Info Technology Backbone

QI Framework for a PH PM

System Includes All Four

Turning Point Quadrants

12

Plan SHIP/CHIP (e.g., MAPP)

Strategic Plan

Operating Plans

Financial Plan (Budget)

Do Program & Policy Implementation

Operational QI

Check Performance Monitoring

& Reporting for the Population & Organization (incl. SHA/CHA, MAPP)

Analysis of Results

Act Performance-based

Decision Making

Decisions on Changes

to Improve Results P

D

C

A

Expectations

Data Information

Direction

TP: Performance Standards

& Performance Measures

TP: Quality

Improvement

TP: Reporting of Progress

Plan SHIP/CHIP (e.g., MAPP)

Strategic Plan

Operating Plans

Financial Plan (Budget)

At least 4 types of plans should be aligned:

they should mutually support each other

SHIP/CHIP

STRATEGIC PLAN

OPERATING PLANS

(“Business Plan,” “Service Plan,”

or “Performance Plan”)

BUDGET

13

Focuses on strategic change &

efforts to support SHIP/CHIP

Covers all programs or

organizational units

14

Views from Different Parts of the System

30,000 ft.

10,000 ft.

Ground

20,000 ft.

Vision & Mission

SHIP/CHIP with Health

Outcome Priorities

Strategic Plan with Priority Change Goals

Performance Budget with Negotiated Targets

SHA/CHA & Public Health Policy

Priorities

Altitude

Operating Plan with Objectives, Performance Measures,

Improvement Plans, & Initial Targets

Programs, Services, Projects & Initiatives, Performance Monitoring & Improvement

Alignment of Plans Across the Organization

– The PHAB domains (including the Essential

Services of PH) provide a way to align all

operating plans of a PH agency

– In addition to alignment of the SHIP/CHIP and

strategic plan

– The operating plans of all programs, projects, or

organizational units must be:

• Aligned with the public health agency’s mission

and goals

• Consistent (or not inconsistent) with the

strategic plan 15

Goal Alignment is NOT the Same as Goal Profusion

Top Level Strategy Map

17

Zoom in: Maps for programs or

organizational units

Zoom in again: Measures, targets, timeframes for

program objectives

Structures the performance mgt. system

Cascading The Strategy Map by Program Or Unit

Questions to ask your organization

Who are the stakeholders in Performance Management?

What important things happened over the past 12-18 months and

how did the team perform?

What important issues does the team face?

Are there things that the organization/work group/program is doing

that it should not be doing or that it could modify?

Are there things that the team is neglecting to do that it should do?

What things could the team do that would help you perform

better?

Measurement Problem

– Top strategic goals & measures may be outcome-

focused:

• Reduce number of smokers in our service area by 20,000/year

– But as deployment goes down to programs and

organizational units the measures often become more

output or process focused, e.g.,

• Number of stop smoking clinics held

• Number of pamphlets handed out at a Stop Smoking Fair

– A disconnect occurs:

• Is it credible that doing well on these program measures will

achieve the desired outcome?

19

Solving the Measurement Problem

– Help program staff understand that they need to: • Use evidence to show that improving results on their output or

process measures really does drive outcomes, or …

• Develop new measures and targets that have an evidence-base,

or …

• Use the PDCA cycle to develop their own evidence.

– Ultimately, program measures • Can be of any type (e.g., output, process, timeliness, customer

satisfaction, intermediate or “participant” outcomes)

• So long as improvement in the program measures logically

contribute to improving outcomes, i.e.:

• Improving program measures drives better outcomes

20

Or is a “Measurement Problem” Really a

“Strategy Design Problem”?

– Program measures may be inadequate to drive

outcomes because the program cannot adequately

address an issue on its own

– Many public health issues can only be adequately

addressed by multiple programs and partners

– Efforts of all partners must be strategically-aligned

and measurement of outcomes and drivers must

reflect efforts and accomplishments of all partners

21

Highest Level Outcomes

22

Population

Outcomes

Place

Outcomes

23

Lagging & Leading Indicators

Population & place outcomes

tend to “lag” …

So you need short term or

intermediate actions that lead to

enable mid-course corrections to your

strategy.

Some intermediate actions are

“participant outcomes” or other

“intermediate outcomes”

24

Performance Action to Outcomes

Chains of Success in a Performance Management System

Program Action Plans

Long term Population or Place Outcomes

Program activities, including short term process outcomes & intermediate outcomes

Environmental Health Enforcement & Investigation Example

25

Goal: Minimize EH Risks & Disparities in Risk Objective: Enforce Environmental Health Codes

Performance measure: Number establishments in top safety tier for its type

Performance measure: % inspections on schedule (including re-inspections) Outcome measure: Total outstanding EH violations Outcome measure: Number EH-related illnesses & injuries per 1,000 residents

Outcome measure: Number food borne illnesses per 1,000 residents Objective: Develop Policies that Incentivize Compliance Performance measure: % fee & fine schedules updated & approved by BOH Objective: Engage the Community to Reduce Need for Enforcement Performance measure: Number of targeted establishments that participate in training

Tools for Implementing a PM System

Establishing Agency, Division, Office or Program Goals & Objectives

Goal 1 Goal 2 Goal 3

Objective 1 for Goal 1 Objective 1 for Goal 2 Objective 1 for Goal 3

Objective 2 for Goal 1 Objective 2 for Goal 2 Objective 2 for Goal 3

Objective 3 for Goal 1 Objective 3 for Goal 2 Objective 3 for Goal 3

Objective Alignment Exercise

For your group’s program:

1. For each box at the top, enter an organizational or “health status”

goal that the group’s program is responsible for reach, or aligns

with the program.

2. For each organizational or “health status” goal, identify one or

more objectives, especially outcomes, and enter them in the

spreadsheet under that goal.

– Objectives are

• Specific

• Measurable

• Achievable

• Results-Oriented

• Time-bound

27

Example

Establishing Measures for Objectives/Activities

% of eating estblishments

inspected at least 1x every 12

months

% of eating establishments

that pass inspection

# of eating establishments the

fail re-inspection after first

failure

Associated Measures Associated Measures Associated Measures

Objective 1 for Goal 1 Objective 1 for Goal 2 Objective 1 for Goal 3

Enforce EH codes and laws 0 0

Goal 1 Goal 2 Goal 3

Improve EH outcomes and

eliminate disparities

0 0

Key Attributes of a Performance Measure

Validity... a valid measure is one that captures the essence of what it

professes to measure.

Reliability... a reliable measure has a high likelihood of yielding the same

results in repeated trials, so there are low levels of random error in

measurement.

Responsiveness ... a responsive measure should be able to detect change.

Functionality... a functional measure is directly related to objectives.

Credibility... a credible measure is supported by stakeholders.

Understandability... an understandable measure is easily understood by all,

with minimal explanation.

Availability... an available measure is readily available through the means on

hand.

Measure Alignment Exercise

Associated Measures Associated Measures Associated Measures

Objective 2 for Goal 1 Objective 2 for Goal 2 Objective 2 for Goal 3

Investigate and contain EH

hazards0 0

# of confirmed new food

borne illnesses per quarter

# of qualified homes given a

home lead testing kits per

quarter

PHAB Domains Alignment Exercise

Domain

1 Monitoring Health Status to Identify and Solve Community Health Problems

2 Diangosing and Investigating Health Problems and Health Hazards in the Community

3

4

5

6

7

8

9

10 Research for New Insights and Innovative Solutions to Health Problems

Essential Service Description

Informing, Educating and Empowering People About Health Issues

Mobilizing Community Partnerships and Action to Identify and Solve Health Problems

Developing Policies and Plan that Support Individual and Community Health Efforts

Enforcement of Laws and Regulations that Protect Health and Ensure Safety

Linking People to Needed Personal Health Services and Assuring the Provision of Health Care

When Otherwise Unavailable

Assuring a Competent Public and Personal Health Care Workforce

Evaluating the Effectiveness, Accessibility and Quality of Personal and Population-Based

Health Service

PHAB Domains Alignment Exercise

Domain Domain Domain

Objective 1 for Goal 1 Objective 1 for Goal 2 Objective 1 for Goal 3

Enforce EH codes and laws 0 0

Associated Measures Associated Measures Associated Measures

% of eating estblishments

inspected at least 1x every 12

months

0 0

6

% of eating establishments that

pass inspection0 0

# of eating establishments the

fail re-inspection after first

failure

0 0

1

2

Data Alignment Exercise

For each measure, consider the following:

0 0 0

Data Considerations

Objective 1 for Goal 1 Objective 1 for Goal 2 Objective 1 for Goal 3

Where are we going to get the data?

How are we going to get the data?

What is the data source?

What is the data frequency?

Data NotesAssociated Measures Associated Measures

0 0 0

0 0 0

Data Notes Associated Measures

Data Frequency

For your group’s program:

Identify any performance measures for which you think

you should get data more often than quarterly for

operations management. Enter the frequency (e.g.,

monthly, weekly, daily) under “Data Frequency”

34

Final Spreadsheet

Operating Plan

Goals,

Objectives, &

Measures for:

Accreditation

Goal 1: Objective or Activity: Enforce EH codes and laws

PHAB

Domain

Previous

Period

Current

PeriodTarget

& No.

PeriodsQI Plan Notes

Improve EH

outcomes and

Objective or Activity: Investigate and contain EH hazards

3 173.00 100.00 80.00

2 2.00 3.00 3.00

State Health Department

% of eating estblishments inspected at least 1x every 12 months

% of eating establishments that pass inspection

Measures:

# of eating establishments the fail re-inspection after first failure

Objectives & Performance Measures

Measures:

95% 95% 6 quarterly

# of confirmed new food borne illnesses per quarter

# of qualified homes given a home lead testing kits per quarter

6

1

2

Trend Direction Support & Documentation

0.00 0.00 3.00 6 quarterly

Frequency

72% 68% 100% 6 quarterly

96%

1

3

2

2

1

1

Trend Line of a Measure vs. Target

99% 98% 98% 84% 80% 88% 99%

Percent inspections on schedule (including re-inspections)

36

Target = 95%

Actual

95% 99%

PLAN

– Performance Management System functionality:

• Align operations with strategy (in SHIP/CHIP & strategic plan)

• Guide day-to-day operations & encourage regular, frequent QI

• Help identify improvements needed at all levels

• Support decisions to improve performance & strategy

– PM system development usually led by a design or implementation

team:

• Staffed with “owner” (member of leadership), “cross section” of users

(Program & Division Staff), IT staff, consistent with PHAB 9.2.1 A: • A current, functioning PM committee or team.

• Seeks broad engagement from stakeholders, as in PHAB 9.1.1 A: • Engage staff at all organizational levels in establishing … a PM system.

• Defines purpose of the PM System & budget or resources to run it

• Decides whether to use the approach and spreadsheets provided here,

revise them, or use a different model

37

Performance Management Team

– Design/Implementation team transitions into an ongoing “PM

SYSTEM TEAM” to:

• Decide on software, hardware, user operating guidance

• Does this system meet your needs? Does it need changed?

• Guide initial deployment & system improvement over time

• Ensure the PM system has adequate ongoing resources to run

effectively: so it meets user needs on time, all the time.

• Meets PHAB Measure 9.1.2 A on an ongoing basis:

• A current, functioning PM committee or team.

– “System Ownership” group that:

• Has full accountability for the system

• Makes sure the system meets the needs of its users to continually

improve performance

• Entertains & decides on user requests for changes

• Spend significant time focused on the system long-term

38

DO: Deploy the System

P

D

C

A

D

E

P

L

O

Y

M

E

N

T

F

E

E

D

B

A

C

K

39

DO: Deploy the System

– For successful PM system deployment, assess

stakeholders and stakeholder groups for:

• Expected level of their support for system development and

deployments (e.g., what barriers, if any, are they associated

with?)

• Their level of influence: How badly you need them for the PM

system to succeed

– Involve stakeholders differently based on that

assessment.

40

Concluding Discussion, Assignment, Follow-up

Discussion:

• Wrap Up

• Questions?

• Reflections on the day?

Next Steps:

• Engage staff at all organizational levels

• Develop agency-wide Goals, Objectives & Measures

• For all measures, determine & enter an “Annual Target.” As

possible, Break down into quarterly targets & enter in those fields.

Follow-up Schedule

• Post-Workshop Webinar

• Discuss Implementation & Maintenance

41