Benchmarking in Economic Development Programs

Eric L. RinehartEconomic Development Solutions, Inc

Presentation Objectives

Context

Performance Tracking Competitive Benchmarking Report Cards/Scorekeeping

Conclusion Appendices

Presentation Output

To inform and educate ....

Presentation Outcome

To finish my presentation

Before you finish listening

What is Benchmarking?

Current Efforts

Absolute

Relative

Standards

Future Goal

World Class

Past Efforts

How Benchmarking is Practiced Performance Tracking/Monitoring: measuring

the furtherance of organizational objectives

Best Program/Practice Benchmarking: learning best practices of leading performers

Report Cards/Score-Keeping: compilation of indicators tracking community movement to the future

Why Benchmark? - Pressure for Accountability

Public Expectation

improvement in lives return on investment

Organizations best use of effort service to customers

Manager's Operations

efficiency of approach quality of services quantity of services

Oversight Authorities

GPRA: process SEA: elements

Private Sector vs.Public Sector

Needs of Stakeholders: Private: Similarly minded owners/operators Public: Distinct stakeholder perspectives

(good job, right things, actual benefit)

Nature of Results: Private: Direct measurable results in dollars Public: Indirect results; jobs and taxes due to private

investment, sales, innovation, efficiency

Planning Steps Production Function Techniques context Global 'Objective' Summative Eval Formative Evalmission/vision Outcome/Benefit Client Benefit

Public Benefitgoals/scope Output/Results Srvc. Delivered

Plan vs Perform. methodology Process/Activity Quality Mgmt.

Best Practices resources Inputs/Resources Acctng Standard

Performance Tracking Techniques - driven by plans and production

Performance Tracking: Planning

1993 Gov't Performance & Results Act Strategic Planning Performance Plan Program Performance Indicators Rpt

Service Efforts and Accomplishments Specify Measurable Objectives Detail the Strategies Employed Publish Activity Indicators Publish Outcome Indicators

Performance Tracking: Functions

If Inputs: Activities and Resources

And Process: Methods and Procedures

Then Outputs: Direct Results

Then Impact: Expected Outcome

Then Goals: Global Objectives & Strategies

Inputs Level of Resources Accounting Standards

Process Competitive Benchmark Quality Management

Outputs Level of Services Plan vs Performance

Impacts Customer Benefit Public Benefit Cost - Benefit

Performance Tracking: Techniques

Strategic Plan - Agency Goals (& Objectives)

Leadership: Increase the capabilities of individuals involved

Infrastructure: Physical and social infrastructure improvements

Workforce: Develop a skilled and adaptive workforce

Capital: Provide access to capital for business growth and expansion

Technology: Provide access to modern business practices

Market Development: Linkages between customers/ markets

Industry: Retention, expansion and attraction

Performance Plan and Report

Dept. Outcomes/Outputs,for each agency objective

Target Measures,for each outcome area

Plan vs. Actual Report,a baseline amount or percentage and quarterly reports

Tourism Promotion Examples Strategies Domestic and International Advertising

Support Local Marketing Organizations Develop Destination Attractions/Sites

Inputs Advertising Media Purchases Process Tourist/Traveler Inquiry Fulfillment

Visitor Volume at Welcome Centers Outputs Percent of callers traveling in Illinois Outcomes Leisure Traveler Satisfaction

Leisure Travel ExpendituresJobs Created/Retained

Industry Recruitment Examples

Strategies Recruitment/Visit Team Advertising and PromotionState/local Location Incentives

Inputs Accuracy, Currency of DataResponse Time to Inquiries

Process Number of Contacts (Presentations) MadeNumber of Leads (Suspects) Identified

Outputs Number of Businesses AttractedValue of Plant and Equipment Invested

Outcomes Jobs Created/RetainedTaxes Generated

Competitive Benchmarking Process (Best Practices)

Key Results: select critical areas of strategic importanceto the organization

Indicators: select reliable indicators of performance Comparison: compare one's performance to others (or

to standards) using these indicators

Learning: determine where the leaders are and what can be learned by them

Changes: make changes in operations based on what you have learned

Competitive Benchmarking Private Sector Examples

World Class Practices Mgmt, Engineering, Operation

customer responsiveness continuous improvement quality management cycle time reduction cross functional workforce self directed work teams

Performance Measures Productivity, Quality, Efficiency

first pass yields scrap & rework costs customer rejects cycle time customer order lead time on-time deliveries

Benchmarking within Economic Development

Best Practices World Class Practice: Management, Engineering

Operations Practices Performance Measures: Productivity, Quality, &

Efficiency of Performance Best Programs

Formative Analysis: Compare 'architecture' to aid in the design of a program in its early stages.

Summative Evaluation: Compare program results to judge overall worth in relation to alternatives .

Context:- Global Economy- Industry Cluster Focus

Vision:- High Quality Job Creation- Stable Tax Base

Goals:- Business Competitiveness- Community Excellence

Adapted from "Emerging Principles in Economic Development: A Benchmarking Tool", June '95.

Scope:- Public Facilities Capacity- Community Betterment- Business Vitality- Marketplace Capability

Organization: - Regional Collaboration - Partnerships

Process: - Comprehensive Strategy - Strategic, Mission Driven - Customer Orientation - Measurement and Evaluation - Learning

Benchmarking "Principles" of Economic Development

Report Cards - Stated Purposes

the compilation and tracking of 'indicators' to track community progress towards a desired future.

the use of marks or 'indicators' to compare and contrast states or regions.

Report Cards & Score Keeping - Examples

Compendium of Economic Performance Peoria Scope

University of Ill Flash Index Area Report Cards

Business Climate Surveys CfED Report Card of the States

State Progress ReportsOregon BenchmarksMinnesota Milestones

Relevance: (purpose) Measures context only, and not follow-up functions, so they won't help you do your job.

Reliability: (areas for marks) Poor selection of the phenomena to be studied, they are rarely a cogent reflection of society and the economy which is more complex.

Validity: (marks themselves) The use of existing indicators and measures of historical phenomena cause concern that outside or unknown events precipitated the effect noted.

Materiality: (the comparisons) The trend indicators typically used are descriptive and distributive, rather than diagnostic, and therefore are not necessarily informative or enlightening.

Report Cards - Not Ready for Prime Time

Phenomena: State final outcomes expected and therefore to be considered for measurement.

Rationale: Describe the cause and effect model to lay out a logical process to get to the outcomes.

Key Results: Select critical areas of strategic importance. Indicators: Select reliable indicators of performance. Comparison: Compare one's performance against relative

or absolute measures using the indicators. Analysis: Suggest methods for improvement.

Report Cards - Alternative Methods Are Needed

CONCLUSIONS

Embrace 'Benchmarking' Positive Attitudes Key Concept: Relationships

Benchmarking Efforts:Economic Development Application

Best Program Benchmarking: Doing the right things: use formative & summative evaluations.

Performance Monitoring: Doing things right: document procedures & measure their completion.

Report Cards: Right the things done: compile realistic indicators of progress towards the future.

Organizational Environment for Measurement Personal Leadership Openness and Trust

Agreement on Basic Assumptions

Commitment to Evaluation Resources for Measurement Applications of Results

Planning Steps Production Function Techniques market analysis Context Summative Eval Formative Evalmission/vision Outcome/Benefit Client Benefit

Public Benefitgoals/scope Output/Results Srvc. Delivered

Plan vs Perform. methodology Process/Activity Quality Mgmt.

Best Practices resources Inputs Acctng Standard

The Key Concept to Remember - Relationships

Planning Definitions

Strategic Planning Implementation Planning Performance Tracking

Strategic Planning. A systematic process by which an organization identifies and analyzes situations or conditions, formulates conclusions and statements to address the situation, devises alternative strategies, & applies resources to change conditions.

Vision: A value statement in which a preferred state exists or a negative state does not exist.

Mission: Is that which an agency has a calling to do. Approach or Scope: General focus or thrust, the broad limits,

within which planning and programming take place. Goal: The end to which a design tends; its aim. Strategies: Statements of different feasible courses of action

to resolve a situation.

Strategic Planning Definitions

Implementation Planning. The process of determination of a role and definition of objectives; a selection of the programs required to fulfill the role; and the development of time-phased plans for the systematic achievement of those objectives (Harleigh B. Treker, New Understandings of Administration).

Program Activity: A project area as defined by the agency. Objectives: An objective is a specific, measurable event with

standards of achievement that are to be realized. A job or task requiring both time and resources.

Tactics: A job or task requiring both time and resources. Tactics (or activities or services) state 'how', usually in chronological order, a strategy is to be accomplished.

Implementation Plng Definitions

Performance Objective: Target level of performance expressed as a tangible measurable objective.

Output Measure: A defined level of performance of program efforts, specified in quantifiable, measurable terms.

Performance Indicator: Particular value or characteristic used to measure outputs or outcomes.

Baseline of Performance: The initial or starting level of activity or effort, expressed in qualitative or quantitative form.

Plan vs. Performance Measures: A periodic accounting of appropriate program related measurements so as to compare actual program results with established program targets.

Performance Tracking Definitions

Evaluation Techniques

The Organizing Framework Input Evaluation Techniques Process Evaluation Techniques- Numerical Process Evaluation Techniques- Qualitative Output Evaluation Techniques Impact Evaluation Techniques Global Objectives Evaluation Techniques

Inputs: The activities and resources put into an effort (if inputs are adequate, then processes can be performed).

Process: The methods and procedures used to operate the program, (if processes are managed efficiently and effectively, then outputs will be produced).

Outputs: Direct results, often called deliverables or outputs, themselves are not valuable in their own right (if outputs are produced then the purpose will be achieved, assuming the correct outputs have been identified).

Purpose: The expected impact, or outcome, what positive change we expect to occur (if the purpose is achieved, then this will achieve the goal).

The Production Function - The Organizing Framework

INPUT EVALUATION – Inputs are the activities and resources put into an effort (if inputs are available, then some work process can be undertaken). Input measures, such as staffing, work hours dedicated and spending, are routine elements of most budgets.

Level of Resources: Number of staff, amount of resources, number of projects, number of partners.

Accounting Standards: Application of generally accepted accounting principles in the management and control of money.

Employee Involvement: Application and development of the workforce as a critical agency asset.

Input Evaluation Techniques:

PROCESS EVALUATION -- Process measurements consider the methods and procedures used to operate the program, (correctly managed process will produce outputs expected). This type of evaluation helps determine how well the program is operating and identifies opportunities for program improvement.

Performance Goals: Both indicators and a base line to provide immediate information on success or failure. Output indicators define the amount of service units provided, efficiency indicators are used to compute the costs, either in terms of dollars or work hours, per unit of output

Plan versus Performance Measures: Periodic tabulation and comparison of indicators to determine if the program is delivering services at the level expected.

Process Evaluation Techniques - Numerical:

Project Monitoring Tool: A standard operating procedures manual, what business call a business plan, listing legal requirements; administrative (and organizational) qualifications; the customers targeted, the reporting forms to be used; list of services to be performed and so on.

Customer Satisfaction: Survey of how customers perceive the services and their levels of satisfaction in terms of content, format and timeliness.

Competitive Benchmark: Identification of the best practices of each operations (to improve standard operating procedures and achieve economies of scale or scope).

Internal Quality Management: Continuous improvement or standardization of the service or product being delivered.

Process Evaluation Techniques- Qualitative:

OUTPUT EVALUATION -- Outputs or direct results, sometimes called deliverables, are valuable as an intermediate measure (assuming we have identified all the outputs necessary to achieve the purpose)

Project Deliverables: Identify direct products of an operation, i.e., nature of activities; categories of services;

Plan versus Performance: A routine quarterly or annual report on whether planned results are achieved.

Customer Demographics, to identify the types of customer being served and whether the project is serving the customers it should. Demographic profile information such as company size, type of industry, etc. is needed.

Output Evaluation Techniques:

Impact Evaluation Techniques:

IMPACT EVALUATION -- (Purpose) The expected impact, outcome, or results showing what positive change are expected to occur (if the purpose is achieved, then this will contribute to the achievement of the global objectives.)

Customer Benefit: Consider the nature of change or progress within the firms or individuals assisted.

Public Benefit (Impact): How has the program changed lives such as jobs created and retained or new taxes generated. What side effects have occurred?

Cost Benefit: Finally, using REMI or other regional econometric modeling technique, calculate the costs of the program effort in relation to the benefit

Global Objectives Evaluation Techniques:

Formative Analysis: Evaluation which deals with program procedure or quality issues, to determine how to design a program's operations. It is used to fine-tune and improve program strategy and enhance delivery. It requires ongoing input regarding program operation from experts, colleagues and customers.

Summative Evaluation: Looks at whether the program actually made a difference, summing up the overall program usefulness. This type of evaluation requires descriptive questions (what is happening); explanatory questions (why is it happening that way); and evaluative questions (how well is it going).