Managing Measures Training

8/11/2019 Managing Measures Training

1/68

Managing with Measuresfor Performance Improvement

Office of Quality Management

Office of Research ServicesNational Institutes of Health

October 2005


2/68

2

Session Outline :

Background Overview Developing Measures Effectively

Managing With Measures Service Group Examples

Questions Session Evaluation


3/68

3

Background

The President's Management Agenda, announced inthe summer of 2001, is an aggressive strategy forimproving the management of the Federalgovernment. It focuses on five areas of managementweakness across the government where

improvements and the most progress can be made.

Factual Example -The Federal government spends over $25 billion annually on ITsystems and services. Do these services improve efficiency orreduce costs? It is likely they do, but without measuring andcommunicating the results, how will anyone know?

More information at www.whitehouse.gov/omb/budintegration/pma_index.html


4/68

4

OVERVIEW


5/68

5

Overview

Organizations succeed when resources within aservice group work together to achieve a commongoal.

This is also true for performance measurement whichincludes establishing operating strategies, definingprocesses that contribute to the strategies, andevaluating, using and communicating the results toimprove performance.


6/68

6

Overview (continued)

Refine asneeded

Iterations of Performance Measurement

Organization value proposition &objectives

Customer needs& requirements

DetermineNew objective added

Prioritizedrequirements

Select

Performancemeasures

Develop/refine

Measured results

Collect &evaluate

Results not used

Requirement successfully met-shift focus

New mandated requirement

Target met- newstretch target set


7/687


Some of the leading methods that are helpful inimplementing and managing for performanceimprovement include:

The Balanced Scorecard ( Performance ManagementPlan )

Productivity Continuous Improvement

Process Improvement

These methods compliment each other and should beincorporated for performance improvement.


8/688


Balanced Scorecard ( Performance Management Plan ):

A new management concept (1992) helps managers atall levels monitor results in their key areas:

financial performance, customer knowledge, internal business processes, learning and growth.

This allows the monitoring of present performance, butalso tries to capture information about how well the

organization is positioned to perform well in the future.


9/689


Productivity:

Productive business tend to succeed over lessproductive competitors because their costs are lower forthe same level of output. How can management

act to increase productivity? Ensure that the overall direction of the organization is correct

and clear. Ensure that communication is clear, thorough and open. If

everyone is aligned, less time and energy is wasted. Detect problems early, and nip them in the bud. Ensure the group is not reacting to imaginary problems. Use

process behavior charts to track key processes, to be abletell the difference between real changes in your environmentand processes, and normal variation.


10/6810


Productivity (continued ):

Ensure the group is not reacting to imaginary problems. Useprocess behavior charts to track key processes, to be abletell the difference between real changes in your environmentand processes, and normal variation.

Talk with staff for input. They usually know what isconstraining them from doing better work. Ask them, listen,and involve them in the solution.

Systematically remove barriers to progress. Get rid of scrap.Get rid of rework. Reduce inspection as much as possible.

Understand your real costs. What does a machine reallycost? Can a process be done better inside theorganization?


11/6811


Continuous Improvement (Kaizen):

In contrast to the usual emphasis on revolutionary,innovative change on an occasional basis, continuousimprovement looks for uninterrupted, ongoingincremental change. In other words, there is alwaysroom for improvement and continuously trying tobecome better.

The principles of implementation are : human resources are the most important organizational

assets processes should evolve by gradual improvement rather

than radical changes,

improvement should be based on statistical/quantitativeevaluation of process performance.


12/6812


Every activity, or work done, in an organization is partof a process ; many processes make up discreteservices.

Activities

Output

InputIncludes resources such asstaff, equipment,information, forms, money

Includes all steps taken toget a specific output

Final goods or servicesfrom the process, forexample assessment of

policies, spot inspections,software training


13/6813


Entersalesorder

Approved

for credit?

Reject

order

Inventoryavailable?

Back-order

Fill order Stop

Start

No

Yes

Yes

No

Example: Order Process


14/6814


Processes should add value, i.e. accomplishsomething.

Focusing and measuring areas within processesresults in more rapid, substantial improvements.


15/6815


Many processes are not optimal there is generallysome wasted effort, lost time, scrap,miscommunication, or re-work.

These problems all have costs - some small, andsome great.

Good News - Management practitioners have, overthe years, developed tools which can be used to

measure and systematically improve processes andoverall operations.


16/6816

The best tools stand the test of time and give a lot ofleverage over common problems.

The most widely used management methods and toolsinclude:

Quality Control Charts Pie charts Bar charts Run Charts Radar Charts

Scatter Plots Histograms Pareto Charts Normal Test Plots Process Capability Calculations

Control Charts (Process Behavior)

Scattergram

Classic Tools

Other Quality Management Tools Relations Diagram Affinity Diagrams Brainstorms Building Consensus Cause and Effect Diagrams Flowcharts Force Field Diagrams Tree Diagrams


17/6817

A Few Tools Highlighted

Flowchart/process map

Scattergram


18/6818

Survey

Scattergram

A Few Tools Highlighted (continued)


19/6819

Cause and Effect Diagram

Scattergram



20/6820

Pareto Diagram

Scattergram



21/68

21

Process Behavior (or Control) Chart

Scattergram



22/68

22

DEVELOPING MEASURES EFFECTIVELY


23/68

23

Measures

What is a Measure? A measure is a means of assessing performance in a

particular area.

Why Measure? Measures are at the heart of a good, customer-

focused process management system, and anyactivity directed at continuous improvement.


24/68

24

Measures (continued)

Measures help drive performance improvement.

Measures help provide a basis for assessingefficiency and effectiveness of services.


25/68

25


The Performance Management Plan (PMP) has been,

and is a useful tool to plan and take the pulse of anorganization.

The PMP assists organizations with :

Effective organizational performancemeasurement

Implementation of strategy


26/68

26


The PMP focuses on documenting a groups valueproposition, main operating strategy, and objectives in

four perspectives:

Financial How do we add valuefor customers whilecontrolling costs ?

Value Proposition

Customer Who do we define as our

customer? How do wecreate value for our

customer? Internal Process To satisfy customers

while meeting budgetaryconstraints, at what

business processes must

we excel?

How do we enableourselves to grow and

change, meeting ongoingcustomer needs

Employee Learningand Growth

Strategy


27/68

27

Steps to Develop and Manage with Measures


28/68

28

Steps to Develop and Manage with Measures

1. Link Processes to Objectives

Using the PMP framework, establish a clearunderstanding of your service groups purpose, orvalue, and your objectives. Update as necessary.

Identify linkages between your processes (work thatis done) and defined objectives to increase thelikelihood that improved process results willcontribute to total service group accomplishments.


29/68

29

2. Identify and Select Performance Measures

Within the objectives, identify what process is beingmeasured, where applicable, and the goals of eachprocess being measured.

Decide how requirements will be met. Know the purpose of the results. Understand why the results matter. Measure that which is most important.

Steps to Develop and Manage with Measures (continued)

NOTE:Effective performance measuresreflect a strong customer focus.


30/68

30

3. Establish Baseline to Compare Future

Performance

Baselines are an essential element of performancemeasurement.

If no baselines exist, select measures that willestablish the basis for comparing futureperformance, collect data about current processes,work output , and service group outcomes .



31/68

31

4. Select Processes with the Greatest Impact

Select those processes, tied to the objectives, withineach PMP perspective with the greatest operationalimpact.

Remember to highlight areas that will allowmeasurement of key elements of your primarystrategy.



32/68

32

5. Collect Data

Address data collection during development ofmeasures.

Decide the what, when, where, how and by whomof collecting the data and document it.

Validate data. Review performance measures to verify they represent

what they intend to gauge Are we measuring the right

thing?



33/68

33

Validate Data (example)


Internal Business ProcessRealize efficiencies in key areas Goals Measures

Improve HR processes andautomate where cost effective

Processes have beenautomated

Decrease in processing time Increase in customer satisfaction

Objective


34/68

34

5. Collect Data (continued)

Document measures using any one of thesuggested forms on the next three slides:



35/68

35

Simple Measurement Plan (simple tracking for each measure, can be done in MS Word or Excel)

Description : What is the measure capturing

The measure is: List the actual measure, e.g., number of, percentage of, etc.

The target is: May be baseline, or projections computed against baseline Data Source: Where to get the data

ResponsiblePerson/Component: Who, or which group will collect the data

Frequency: How often will the data be collected, and reported on

Process:

Date Developed:

Measure Type (input, output, efficiency, quality or outcome):


36/68

36

Data Collection Plan Template

NOTE : actual template is blank; the content in each column describes what should be filled in.

Methodology POC Source of Data Frequency TargetMethodology is asummary of how youplan to gather andanalyze the data.What steps need to

to be taken to ensurethat the data isaccessible (e.g.,forms developed fordata collection,survey developedand approved foruse, databasedeveloped to analyzethe data)

The personwith primaryresponsibilityfor datacollection and

analysis

Where the data willbe obtained (e.g.,ordering systems,budget systems,rate studies, hard

copy forms, e-mails, phone calls,observations)

How oftenthe measurewill becollected,analyzed,

and reported(e.g., daily,weekly,monthly,quarterly,annually)

The measure outcomeyou hope to attain.These should be setby service groupleaders in conjunction

with theirdivision/office leadersbased on analysis ofdata. If the measureis being collected forthe first time,"baseline" is mostlikely the target asdata is not yetavailable for analysis.

Performance Objective: The Performance Objective from the PMP

The performance measurefrom the PMP

Performance Measure

Available at http://oqm.ors.od.nih.gov/performance_mgmt.htm#templates


37/68

37

Measure Roadmap Template

Measurement Inten t: What is the intent or purpose of the measure

Measure : The Measure (from the PMP) ; How is the objective going to be measured?

Strategic Objective : The objective (from the PMP) stated in terms of the desiredoutcome

Measurement Definition/Formula :

Frequency of Data Update :Monthly, Quarterly, Annually (How frequently will the data be updated?)

Units of Measure :

Notes/Assumptions : What should be known about the measure? e.g., Doesthe baseline data need to be gathered?Is the data retrieval dependent on outside s ources (vendors, ot herorganizations, surveys). Do new questionsMeasurement/Information : Data Elements and Sources : Where is the data located (e.g., a report?)

Source For and Approach to Setting Targets : How has the target been established? Who will ultimately decide if the targets are right? Havetargets been met?

Target Setting Responsibility :optional

Accountability for Meeting Ta rget :optional

Tracking/Reporting Responsibility : thesame person reponsible for the m easurecan track this

Measure Availability :

Target Availability :

THE M EASURE ROADMAP

FY06 FY07Target Baseline Year What year was thebaselineestablished? What isthe baseline value?

FY05 Projected

NOTE : actual template is blank; the content in each box describes what should be filled in.

Available at http://oqm.ors.od.nih.gov/performance_mgmt.htm#templates


38/68

38

6. Analyze Documented Results

Examine results in context of the objectives,environment, and external factors.

After analyzing the performance data and

information, interpret the information in order toascertain whether the objectives have been met,and if not, why not.

In cases where the objectives are not being met,

develop an action plan to correct this.

E.g., If feedback indicates that fewer customers are utilizinga new service provided by your service group, note this andmake the necessary modifications to improve or tailor the

service.



39/68

39

7. Communicate Results

Graphically display data, using process behavior,pie, and bar charts, or other form to summarizeperformance rather than narrative alone.

Develop brief updates to share monthly among staff

showing monthly progress.


Objective:

Customer fiscal service

Shows cost per customer



40/68

40

Monthly error rate for sorting incoming mail, October 2003 to May 2005

Shows process is predictable where the expectation is between 0% and .36% error ratefor any month (between 0 and 36 errors per 10,000 pieces of incoming mail).

Note: Data gathered by random sampling .


7. Communicate Results (continued) : second example


41/68

41


Objective:

7. Communicate Results (continued) : third example

B ildi X C f t i D il l f 4/2/2004 t 7/1/2004


42/68

42

Building X Cafeteria: Daily gross sales from 4/2/2004 to 7/1/2004

4/2/2004

7/1/2004

Average Gross sales:$1170.The lower limit: $857The upper limit: $1484.

The process during this period was predictable and the group can expect theamount of sales to vary from between $857 to $1484, given no substantial changes,for example sudden menu change or sudden population increase or decrease.

l d h


43/68

43

8. Integrate Measurement with ManagementProcesses

The benefit of performance measurement comesfrom using data to take the appropriate actions toimprove performance.

To remaining viable, demonstrate service groupvalue through results.

Integrate results (input, activity, output) into planning,budgeting, and operating processes in the business.


S f S D l d M i h M


44/68

44

1. Link Processes to Objectives

2. Identify and Select Performance Measures3. Establish Baseline to Compare Future Performance4. Select Processes with the Greatest Operational

Impact5. Collect Data6. Analyze Results7. Communicate Results8. Integrate Measurement with Management

Processes

Summary of Steps to Develop and Manage with Measures:


45/68

MANAGING WITH MEASURES:

ADDITONAL EXAMPLES FOCUSING ONSTEPS 6 & 7


46/68

CUSTOMER PERSPECTIVE


47/68

47

Objective and Measures Service Group: Procure and Deliver Animal Product

C1b and C1d: These measures are used to drive performance towardsreaching an objective. By focusing on each of these measures, theworkforce will be taking actions that will satisfy their customers and raisethe customer satisfaction scores.

Objective Measure

C1b: Percentage ofprocurement ordersprocessed within 24 hours

C1d: Number of complaints

related to responsiveness ofprocurement services

C1f: ORS CustomerScorecard rating for animalprocurement responsiveness

C1: Increaseresponsiveness tocustomers' varying

needs

D t Sh t


48/68

48

Data Sheet

Monthand Year

Number of OrdersProcessed Within24 Hours

TotalNumber ofOrdersProcessed

Percentage ofOrders ProcessedWithin 24 Hours

Oct FY03 714 1020 70%Nov FY03 763 1060 72%Dec FY03 686 915 75%

Jan FY03 859 1074 80%Feb FY03 795 1169 68%Mar FY03 986 1429 69% Apr FY03 1098 1547 71%May FY03 825 1178 70%Jun FY03 1153 1406 82%Jul FY03 1068 1256 85%

Aug FY03 1051 1330 79%Sep FY03 868 1058 82%Oct FY04 863 1015 85%Nov FY04 920 1045 88%

Dec FY04 838 1010 83%

AutomaticallyCalculated


49/68

Output Measure:

% of Procurements Processed within 24 hours (C1b)

Run Chart (line graph) displays process performance over time


50/68

50

Run Chart (line graph) displays process performance over time

50%

60%

70%

80%

90%

100%

O c t F Y 0 3

N o v F

Y 0 3 D e

c F Y 0 3

J a n F

Y 0 3 F e

b F Y 0 3

M a r F

Y 0 3 A p

r F Y 0 3

M a y F

Y 0 3

J u n e

F Y 0 3

J u l y

F Y 0 3

A u g F

Y 0 3

S e p t F Y 0 3

O c t F Y 0 4

N o v F

Y 0 4 D e

c F Y 0 4

Month, Fiscal Year

% P

r o c e s s e

d

% procurement orders processed wi thin 24 hours

Average = 77.27

Percentage of procurement orders processed within 24 hours (C1b)

WHAT DOES THIS SHOW? After a drop in February, there was a steady increase in the % of ordersprocessed in 24 hours. Investigating this phenomenon could unearth animprovement or process change that was instituted. Also, the average lineadded clarifies movement of the data away from the average.

For more robust monitoring of the process, and better information about whenthe process is showing variation beyond what is expected, try using a processbehavior chart.

Range 88 68 = 20

88

68

Process Behavior Chart displays process performance over time


51/68

51

AVE=77.3%

LCL=65.7%

UCL= 88.9%

% of orders processed within 24 hours

Process Behavior Chart displays process performance over timeand makes it easy see both special and common cause variation ina process

WHAT DOES THIS SHOW?Look at variation over time and show whether animmediate remedy is needed to modify the processor if the entire process should be redesigned.The process is in control, or predictable, but stepscan be taken to improve the process, so that evencommon cause variation is reduced for example,explore ways to increase order process efficiency.

Analysis


52/68

52

Analysis

The Process Behavior Chart (PBC) indicates theaverage percentage of procurement orders processedwithin 24 hours is 77.3%.

The PBC indicates that you can expect between65.7 % and 88.9% of orders will be processed within24 hours each month unless there is a change to the

process .

Is our performance good enough? Are we meetingcustomer expectations? Refer to baselines andcalculated targets, as well as customer agreementsand satisfaction ratings.


53/68

Bar Chart like pie charts are useful for comparing


54/68

54

Bar Chart like pie charts, are useful for comparingclasses or groups of data.

7.64

7.5

7.23

7.07

7.71

7.31

6.92

7.08

6.17

5.57

4.86

6.71

4.86

5.71

5.71

5.57

6.14

5.4

1 2 3 4 5 6 7 8 9 10

Handling of Problems

Competence

Convenience

Responsiveness

Availability

Reliability

Timeliness

Quality

Cost

FY03 FY02

N = 7

N = 14

Note: ORS Customer Scorecard graphs are provided in reports generated by theOffice of Quality Management (OQM)

7.64

7.5

7.23

7.07

7.71

7.31

6.92

7.08

6.17

5.57

4.86

6.71

4.86

5.71

5.71

5.57

6.14

5.4

1 2 3 4 5 6 7 8 9 10

Handling of Problems

Competence

Convenience

Responsiveness

Availability

Reliability

Timeliness

Quality

Cost FY03 N = 16 88% response rate

FY02 N=14 78% response rate


55/68

LEARNING AND GROWTH


56/68

56

Objectives and MeasuresService Group: Provide NIH Events Management Services


57/68

Input Measure:Measure: Employee Satisfaction Survey Results (LG3e)

i Ch d h l f d i i


58/68

58

90%

10%

87%

13%

Management Questions : 87% of DEMS staff understand how their jobcontributes to the mission, vision and values of DEMS

Customer Orientation Questions : 90% of DEMS staff say DEMS services goals areaimed at meeting customer expectations

Teamwork and Innovation Questions : 93% of DEMS staff feel they can serve customers in

a timely fashion

Pie Chart used to show classes or groups of data in proportion tothe whole data set.

Employee/Staff Survey Results

93%

7%

l


59/68

59

Analysis

A high percentage of staff feel they understand theorganizations mission, the goals match customer expectations,and they can serve customers in a timely manner.

Lower percentages on the charts should be addressed.

Practical corrective actions should be employed to maintainand/or improve performance.

The information serves as input to the overall service andprocesses provided by the group; improvement in learning andgrowth areas drives behavior to achieve objectives.


60/68

FINANCIAL PERSPECTIVE

Financial Objective and Measures


61/68

61

Service Group: Provide Transportation and Parking Services

F1: Minim ize the unit cost forproviding parking services oncampus

F1. # vehicles stacked andoverall vehicles managed

F2: Minimize the unit cost for satelliteparking facilities off campus

F2. # parking spaces (usageversus space availability)

F3: Minimize the unit cost forproviding shuttle services

F3. # shuttle riders

F4: Minimize the unit cost ofadministering and coordinatingalternative transportation

F4. # transhare/carpool/vanpoolparticipants

F5: Minimize the unit cost for issuingand tracking parking permits

F5. # of permits

F i n a n c

i a l

Objective Measure


62/68

Input Measure:Minimize the unit cost for providing shuttle services (F3)

Calculation Example


63/68

63

Calculation ExampleMinimize the unit cost for providing shuttle services (F3)

Fiscal Year

Operating BudgetShuttle Riders

Budgeted Unit Cost% Change

Fiscal Year

Operating BudgetEmployee Shuttle Riders


Fiscal Year

Operating BudgetPatient Shuttle Riders


$1,018,760 $1,101,300 $293,21731,836 40,246 9,882

FY02 FY03 1st QTR FY04

$3,395,868 $3,671,000 $977,389


26.9% -13.5%

830,423 707,628 217,876$4.09 $5.19 $4.49

$2,377,108 $2,569,700 $684,172798,587 667,382 207,994$2.98 $3.85 $3.29


29.4% -14.6%

$32.00 $27.36 $29.67-14.5% 8.4%

Note: Estimate that budget for patient shuttle is 30% of total budget for the discrete service; budgetto employee shuttles is 70% of budget for the discrete service.

A l i E l


64/68

64

Analysis Example

Determined the basic unit of service delivery number ofriders

Counted the number of riders on shuttle this becomes thedenominator for the unit cost formula

Determined the dollar resources expended in providing

shuttle service this becomes the numerator of the unit costformula The ratio of numerator ($) divided by denominator (riders)

yields cost per rider on the shuttles Use this and other data to reveal methods to maintain (or if

possible lower) the unit cost example is to encouragemore riders - lowers unit cost ( reason: more riders for sameresources)


65/68

Integrating Data into Business Management


66/68

66

Questions to integrate performance data intoBusiness Management:

How can the performance data be used to improvebusiness decisions?

Examine data for trends over time and across functions.What do the results mean? Do the results contribute togoals and objectives?

Have the right measures been included in the PMP? Doesthe PMP reflect our priorities?

Are the performance results helping the manager makebetter decisions? If not, notify the team or individuals thatare responsible about the data that is needed.

If the performance results exceed the targets, how can theservice group take advantage of additional benefits toimprove service and reduce operating costs? Whatopportunities do the additional benefits make possible?


67/68

67

Questions to integrate performance data intoManagement (continued):

What is the level of customer satisfaction? If the results fall short of the targets, what inhibited

performance? Were the inhibitors technical ororganizational in nature?

Source: Performance Based Management, P. Plunkett


68/68

For more information on Performance Management in the Office of Research services:http://oqm.ors.od.nih.gov/performance_mgmt.htm

AcknowledgmentsThis training was developed by the Office of Quality Management.

Date post:	02-Jun-2018
Category:	Documents
Upload:	pakeya77420
View:	218 times
Download:	0 times

Managing Measures Training

Documents