Using Metrics to Drive Research Administration Performance

NCURA Annual MeetingNovember 2012

Marcia Smith, Associate Vice Chancellor, Research Administration, UCLAPatti Manheim, OCGA Director, Research Administration, UCLAElizabeth H. Adams, Executive Director, Office for Sponsored Research, Northwestern University, Evanston CampusNate Haines, Senior Director, Higher Education Consulting, Huron Consulting Group

Agenda

Introduction

Benefits of Metrics

Developing Metrics

Determine Metrics to Track

Collecting Metrics

Discussion (All)

2

Introductions

3

Benefit of Metrics

4

Benefit of Metrics

5

Metrics

Change Behavior

Drive Performance

Support Investments

Benefits of Metrics

6

Drive Performance1) Motivate teams to achieve desired outcomes

• Increasing transparency can trigger positive culture changes and improve outcomes.

• Using metrics to monitor business processes improves accountability so high performers can be recognized and bottlenecks addressed.

2) Define business processes and responsibilities• Implementing metrics requires an organization to identify its

desired outputs resulting in defined business processes.• Use of metrics helps identify operational bottlenecks which can

be a result of personnel having varied understandings of processes, roles and responsibilities.

3) Monitor the impact of new processes• As new processes are implemented, metrics provide confirmation

that the change is working.• Easily identify bottlenecks• Help motivate team

Benefits of Metrics

7

Change Behavior1) Manage stakeholder expectations

• Metrics enable clear communication of process goals and current status to stakeholders.

• Concrete information enables stakeholders to determine whether their needs are being met.

• If an institution sets reasonable and clear goals which are communicated then customer perception of performance can improve simply through a better understanding.

2) Evaluate staff performance• Metrics allow leadership to objectively track staff contributions

both individually and collectively against operational goals.• With metrics, personnel know their assessments are objective,

why they are receiving their assessments, and how to improve them.

Benefits of Metrics

8

Support Investments in Research Administration Infrastructure:1) Improve decision making and prioritization

• Metrics provide leadership with insights as to where attention and resources are needed.

• Concrete information improves decision-making and allows managers to better understand and identify opportunities to improve compliance, financial management, and operational performance.

Developing Metrics

9

Identify Greatest Opportunity

10

Operational Performance

• Turnaround times

• Workload Distribution

Compliance Management

• Cost transfers

• Effort Reporting

• Expiring Protocols

Financial/Cost Management

• Timeliness of financial reporting

• Bottlenecks• Outstanding

collections• Write-offs

Customer Perception

• Turnaround Times

• Workload Distribution

Taking a focused approach will allow you to more quickly implement successful, lasting, and measureable improvements

UCLA Case Study

11

Pre-Award Metrics

Post-Award Metrics

UCLA Case Study

12

6 9 7 7 11 9 9 16 16 12 11 6 18 1264 51

84 74107 132

100131

182214

147120

66 59

113

52

126 134

169 143

102

101

125

144

119131

11291

87

54

98 79

91 86

91

115

105

92

8890

68

64

126

85

139

120

150124

116

143

167

178

193204

114

104

2

1

2

3

58

3

10

10

15

13 10

5

0

7

8

12

11

711

9

15

14

25

4030

9

1

0

100

200

300

400

500

600

700

800

Nov 09Data

Dec 09 Data

Jan 10 Data

Feb 10 Data

Mar 10 Data

Apr 10 Data

May 10 Data

Jun 10 Data

Jul 10 Data

Aug 10 Data

Sep 10 Data

Oct 10 Data

Nov 10 Data

Dec 10 Data

OCGA Award Acceptance Details

Supplement

Renewal

New

No Cost Extension

Mod / Amendment

Continuation

Admin Change

Determine Metrics to Track?

13

Pre-Award Metrics

14

Area Metric

Proposal Submission Average days complete submission received before deadline, missing/incomplete proposals, detailed metrics

Proposals/Awards Submission

Number and $$ of proposals/awards, detailed metrics by categories

Award Set-Up Timeline for central office(s) to process set-up*

Award Set-up Timeline for receipt of award document to financial account activation

Award/Outgoing Subcontract Execution

Timeline with central office

Award/Outgoing Subcontract Execution

Timeline with internal Departments, Principal Investigator, Sponsor, Subrecipient

Advance Accounts Number and how long they have been open

15

Proposal Submission

0-1 Day in Advance of Deadline

2-3 Days in Advance of Deadline

4-5 Days in Advance of Deadline

5+ Days in Advance of Deadline

261

65.2543.5

56.55

Proposal Submission to OCGA

Number of Proposals Submitted on a Monthly Basis

16

Award Intake Process (Pilot)

Turnaround time for Expedited Awards has improved by over 80% during the award setup pilot

Award Set-Up

17

Award Intake Process (Current)

Full implementation January 2012 Award setup has slowed for expedited awards, but is still 65% faster

than previous processing timeline

Award Set-Up

Award Setup (Current)

18

New process has identified hold-ups • Shaping policy and procedure decisions• Awards processed 6 days faster when all internal documents are

present

Post-Award Metrics

19

Area Metric

Letter of Credit • Number and $ of draws• $ unbilled• $ in LOC Clearing Account(s)

Invoicing • Monthly unbilled number and $• Monthly billed number and $

Accounts Receivable • Number and $ in aging buckets 30/60/90/120+ days

Overdrafts • Number and $ in aging buckets for overspent accounts

Closeout • Awards open 120+ days past end date

Cost Transfers • Number submitted/approved less than 90 or 90+days

Post-Award Metrics

20

Area Metric

Financial Reporting • Monthly number submitted• Monthly percentage submitted on time• Monthly number past due (i.e. backlog)

Cash Application • Number and $ of payments sitting in holding accounts

Bad Debt • Number and $ of write-offs

Effort Reporting • Number and % certified on time• Number and % outstanding• Number and % certified by PI or individual• Number of recertifications

On-Time Submission

21

On-time submission increased by 35% for Invoices On-time submission increased by 48% for Reports, from a

low of 14% at the start of FY10

Backlog – Invoices and Reports

22

Backlogs have decreased by 64% since the start of FY10

Collecting Metrics

23

Collecting Metrics

24

Identify Opportunities

Determine Metrics to track

Identify systems to deliver outputs

Analyze output to ensure integrityThings to consider:

• Audience for metrics Effectiveness of metrics is greatly affected by the selection of

recipients who will be reviewing the metrics. Wrong metrics or wrong audience diminishes the value of the

metrics. Data integrity

Is the data that you are using to populate the metrics accurate? Is the correct logic being used? Establish standardized processes for entering data

Interpretation of data Are assumptions being made? Know what each data field means; develop a data dictionary

Discussion

25