1

Performance Management for Justice Information

Sharing

David J. RobertsGlobal Justice Consulting

2006 Symposium on Justice & Public Safety Information SharingMarch 12, 2006 Washington, DC

2

“Performance evaluation is more than an academic exercise, a matter of methodologies and numbers. How performance is measured affects not only what the public knows about the police, but also the character of police operations and the management climate. Because performance evaluations establish priorities, incentives, and requirements, they are much too important to be left to technicians. Performance measurement should be viewed as an integral, ongoing part of the management of policing.”

Measuring Performance

Source: David H. Bayley, “Measuring Overall Effectiveness,” in Lawrence T. Hoover (ed.), Quantifying Quality in Policing

(Washington, DC: Police Executive Research Forum, 1996), pp. 37-54.

3

Process vs. Impact Evaluations• Process evaluations focus on how the initiative was

executed; the activities, efforts, and workflow associated with the response. Process evaluations ask whether the response occurred as planned, and whether all components worked as intended. Fundamentally, a process evaluation posits the question, “Are we doing the thing right?”

• Impact evaluations focus on the outcome (the what) of the initiative; the output (products and services) and outcome (results, accomplishment, impact). Did the problem decline or cease? And if so, was the response the proximate cause of the decline? Fundamentally, the impact evaluation posits the question, “Are we doing the right thing(s)?”

4

Managing Performance“Performance measurement, in simplest terms, is the

comparison of actual levels of performance to pre-established target levels of performance. To be effective, performance must be linked to the organizational strategic plan. Performance-based management essentially uses performance measurement information to manage and improve performance and to demonstrate what has been accomplished. In other words, performance measurement is a critical component of performance-based management.”

Source: Will Artley, D.J. Ellison and Bill Kennedy, The Performance-Based Management Handbook, Volume 1:

Establishing and Maintaining a Performance-Based Management Program (Washington, DC: U.S. Department of Energy,

2001), p. 4.

5

Establishing a Performance Management Program

The Six Steps to Establishing a Performance-Based Management Program

Source: Will Artley, DJ Ellison and Bill Kennedy, The Performance-Based Management

Handbook, Volume 1: Establishing and Maintaining a Performance-Based Management

Program (Washington, DC: U.S. Department of Energy, 2001)

6

Step 1: Define Mission and Strategic Performance Objectives1. Mission statements identify the overall purpose for which

the organization is organized. 2. Vision statements describe the future business

environment and the role of the organization within it. 3. Value statements reflect fundamental beliefs and values

guiding the agency, the nature of their responsibilities, and the philosophy underlying their approach.

4. Assumptions are also frequently discussed in strategic planning efforts, describing business environmental conditions that are expected in the future.

5. Business strategies identify how objectives are to be accomplished, e.g., community-oriented policing

7

Step 2: Establishing an Integrated Performance Management FrameworkMajor Elements in Creating a Performance Management

Framework1. Define the Relationship of Performance Measurement

to the Strategic Planning Process2. Build the Performance Management Team3. Address Stakeholder/Customer Needs4. Understand Performance Measurement Terminology5. Manage Performance Measurement6. Accept Accountability for Measures 7. Communicate8. Know How to Check/Test Your Measures 9. Learn From Others 10. What Do You Measure Yourself Against

8

Performance Planning Template• Performance Plan Template Defined/Action taken • Mission/Vision/Values/Assumptions Relate project to Agency Mission/Vision, etc.• Strategic Performance Objective 1 Defined• Operation/Activity Title• Purpose Describe purpose(s) of initiative• Execution Define how you’re going to do it• Performance Target(s) Identify target(s) (e.g., reduce violent crime by 10%)• Performance Measures How you’re going to measure it• Owner Person responsible/accountable• Resources Resources needed for this initiative• Budget Funding dedicated to this initiative• FTE Staffing dedicated to this initiative

• Strategic Performance Objective 2 Defined• Operation/Activity Title• Purpose Describe purpose(s) of initiative• Execution Define how you’re going to do it• Performance Target(s) Target(s) (e.g., reduce violent crime by 10%)• Performance Measures How you’re going to measure it• Owner Person responsible/accountable• Resources Resources needed for this initiative• Budget Funding dedicated to this initiative• FTE Staffing dedicated to this initiative

9

Step 3: Establish Accountability for Performance

• Building Accountability:– Authority refers to the power to influence or

command thought, opinion, or behavior. – Responsibility means that one is liable to be called to

account as the primary cause, motive, or agent.– Accountability, on the other hand, is an obligation or

willingness to accept responsibility and to account for one's actions.

• Internal vs. External Accountability– Internal Organizational Accountability – Internal

organizational accountability refers to the establishment of the upward and downward flow of accountabilities between management and individuals and teams within the organization.

– External Organizational Accountability – In external organizational accountability, the organization answers to/reports to its stakeholders on both its organizational performance and organizational behavior.

10

Step 4: Establish a Process/System for Collecting Data to Assess Performance Develop a Plan:1. Information Requirements2. Information Sources3. Data Collection Processes4. Data Collection and Reporting Frequencies5. Data Collection Costs 6. Data Protection7. Data Quality8. Trial Run

11

Step 5: Establish a Process/System to Analyze, Review, and Report Data Data Analysis Strategies1. Assess the Quality of Data2. Employ Analytic Methods3. Calibrate Baseline Measures4. Test Hypotheses 5. Know What to Measure6. Data Presentation—How Will the Data be Used and

Reported?7. Create an Executive Dashboard

12

Sample Executive Dashboard

13

Step 6: Establish a Process/System to Use Information to Drive Improvement 1. Shape Organizational Culture2. Make Information Broadly Available3. Reengineer Business Processes4. Build Performance Management into Everyday

Operations a. Executive Support and Organizational Commitment

is Requiredb. Comprehensive System of Accountability and

Responsibility is Requiredc. Flattens the Organizationd. Identify Problems Earlye. Ensure Progress and Keep Projects on Target f. Demonstrate Value

14

Discussion of Performance Measures for Information Sharing1. Describe mission, vision and objectives of

information sharing initiatives.

2. Define measures and targets presently used in jurisdictions.

3. Are there universal performance measures that are appropriate in evaluating the level of information sharing within a jurisdiction?

4. Identify data sources, measures and reports.

5. Discussion…

15

Resources• Will Artley, DJ Ellison and Bill Kennedy, The Performance-Based Management Handbook,

Volume 1: Establishing and Maintaining a Performance-Based Management Program (Washington, DC: U.S. Department of Energy, 2001) at http://www.orau.gov/pbm/pbmhandbook/pbmhandbook.html

• John E. Eck, Assessing Responses to Problems: An Introductory Guide for Police Problem-Solvers (Washington, DC: Center for Problem-Oriented Policing, no date), at http://www.popcenter.org/Tools/tool-assessing.htm

• Michael Geerken, The Art of Performance Measurement for Criminal Justice Information System Projects, (Washington, DC: U.S. Department of Justice, Bureau of Justice Assistance, 2006 [forthcoming])

• Robert H. Langworthy (ed.), Measuring What Matters: Proceedings from the Policing Research Institute Meetings, (Washington, DC: NIJ/COPS, July 1999, NCJ 170610), pp. 37-53.

• David J. Roberts, Law Enforcement Tech Guide: Creating Performance Measures that Work! A Guide for Law Enforcement Executives and Managers to Assess and Measure Performance, (Washington, DC: U.S. Department of Justice, Office of Community Oriented Policing Services, 2006 [forthcoming from SEARCH and COPS])