School Bus Operator Contract Performance …...School Bus Operator Contract Performance Management...

School Bus Operator Contract Performance Management Program

Resource Package

June, 2011

Council of Ontario Senior Business Officials

Table of Contents

1.0 Fundamentals of Contract Management .................................................................................................................. 1

1.1 Contract Management Essentials ........................................................................................................................ 1

1.1.1 The Role of the Contract Manager ............................................................................................................... 2

1.1.2 Contract Performance Management in Context ........................................................................................... 2

2.0 Contract Performance Management Principles & Approach .................................................................................... 3

2.1 Program Approach ............................................................................................................................................... 3

2.1.1 Compliance Monitoring & Performance Measurement Described ................................................................ 3

2.1.2 Performance Management Scalability & Applicability................................................................................... 4

2.2 Performance Measurement Principles ................................................................................................................. 5

Principle 1 – Focus Your Performance Measures on Desired Outcomes ............................................................. 6

Principle 2 – Ensure That the Factor Chosen is Actually Measurable ................................................................... 6

Principle 3 – Actual Performance Must be Controllable by the Operator .............................................................. 7

3.0 Contract Performance Management Program Description ...................................................................................... 8

3.1 Contract Performance Management Program Responsibilities ......................................................................... 10

3.1.1 Program Assignments ................................................................................................................................ 10

3.1.2 Program Communications Protocol ............................................................................................................ 11

3.2 Contract Management Calendar ........................................................................................................................ 13

3.3 Contract Compliance Monitoring Plan ............................................................................................................... 14

3.3.1 Annual Contract Compliance Audit ............................................................................................................. 14

3.3.2 Periodic Bus Route Audits .......................................................................................................................... 14

3.3.3 Facility Audits ............................................................................................................................................. 15

3.4 Contract Performance Measurement Plan......................................................................................................... 16

3.5 Annual Contract Performance Assessment ....................................................................................................... 18

Appendix A – Contract Performance Management Program Reference Matrix ........................................................... 19

Appendix B – Contract Compliance Reference Tools, Samples & Templates ............................................................. 29

B.1 Sample Annual Compliance Audit Procedures & Standards ............................................................................. 30

B.1.1 Annual Audit Procedures ........................................................................................................................... 30

B.1.2 Annual Audit Standards ............................................................................................................................. 30

B.2 Annual Contract Compliance Audit Checklist ................................................................................................ 32

B.3 Annual Contract Compliance Audit Report ................................................................................................... 38

B.4 Sample Route Audit Form ............................................................................................................................. 40

Appendix C – Performance Measurement Reference Tools, Samples & Templates ................................................... 42

C.1 Performance Measure Matrix ............................................................................................................................ 43

C.2 Performance Measure Glossary ....................................................................................................................... 45

C.3 Sample Performance Measure Tracking Summary .......................................................................................... 47

C.4 Sample Operator Performance Report ............................................................................................................. 47

Page | 1

1.0 Fundamentals of Contract Management

The award of a contract should not be viewed as the final outcome of the competitive procurement process. Rather, it

should be viewed as the start of a new phase within an ongoing cycle of continuous improvement. The bus operators

holding your contracts should be subjected to a rigorous and consistent program of compliance monitoring and

performance measurement relative to the requirements of the contract and the consortium‟s operating policies and

practices. This performance management program should be conducted within the larger context of contract

management as a functional responsibility.

Figure 1 – The Contract Management Continuous Improvement Cycle

At its most basic level, contract management processes help to ensure that the operators selected through a

competitive solicitation meet their obligations over the entire term of the contract. A more comprehensive view, as

illustrated in Figure 1, establishes a continuous improvement framework for contract management. The purpose for

contract management processes should be to enhance the operators’ capabilities, improve consortium

operations, and advance the quality of subsequent competitive procurement cycles.

A vigorous contract management program must therefore be considered an essential responsibility of the consortium

and an extension of the competitive procurement initiative. The RFP and vendor selection process result in a

contractual obligation that establishes a baseline set of performance expectations for the bus operators and the

consortium. In the same way that unenforced policies will lose their meaning and applicability, unmanaged contracts

will degrade the meaning, applicability, and value gained through the competitive procurement initiative.

1.1 Contract Management Essentials

A comprehensive contract management approach encompasses four key elements, each of which must be included

within a structured, consistent, and fair program:

1. Resources – The consortium must ensure that there are people in place with the appropriate skills and

available time to carry out contract management responsibilities. These assignments should be completed

and clearly articulated in time to provide for a seamless transition from the competitive solicitation to

operational management of the resulting agreements. Contract management must be viewed as an ongoing

component of consortium operations and requires the assignment of ongoing management resources.

2. Oversight – The core of consortium‟s contract management program must incorporate systematic,

consistent, and fair mechanisms for overseeing the performance of bus operators relative to the

requirements of the contract and the consortium‟s established policies and practices. In a practical sense

this takes the form of compliance monitoring relative to the specific contractual obligations of the bus

Page | 2

operators, and performance measurement relative to the service delivery goals and objectives of the

consortium.

3. Development – In addition to oversight, a concurrent objective of contract management should be to

increase the capabilities the bus operator over the term of the agreement. Encouraging continuous

improvement among the bus operators and throughout the consortium organization enhances overall

performance of the consortium.

4. Strategy – The consortium should model contract management processes in such a way that actively

encourages a healthy professional relationship with its bus operators. A strategic outlook beyond the

specific requirements of the current contract should consider the goals of improving the size and quality of

the vendor base and encouraging healthy ongoing competition.

1.1.1 The Role of the Contract Manager

The contract management framework described above illustrates the importance of having a named

Contract Manager to oversee each individual contractual relationship as an essential starting point.

Depending on the size and scope of the consortium, there can be one or several assigned contract

managers. The functional responsibilities can be a full-time occupation or collateral to the contract

manager‟s other responsibilities. Proper preparation will ensure a successful program, regardless of size.

The assigned contract manager should have detailed knowledge of the contract and requisite knowledge of

the student transportation sector in Ontario. The position should be formally assigned, with appropriate

documented job descriptions. Specific objectives and goals for the contract manager should be established

and the reporting relationship within the consortium clearly established.

1.1.2 Contract Performance Management in Context

The contract management framework described above also describes the critical role of contract

compliance monitoring and performance measurement activities. As illustrated in Figure 2, these

components of contract management are integral to the continuous improvement cycle that should be the

overall objective for your contract management program.

Figure 2 – Contract Performance Management

Compliance monitoring activities focus on risk management while performance measurement,

reporting, feedback, and corrective action improve operator capacity. Taken together, these two

contract management activities form the Contract Performance Management Program that is the

focus of this resource package.

The prior discussion is included as a precursor to ensure that the performance monitoring and measurement

program framework detailed in the subsequent sections is placed in an appropriate context. Consortium

managers should utilize the information and details on the following pages with knowledge of how to

incorporate these into a wider program of overall contract management, and with an understanding of how

contract management as a function fits into a cycle of continuous improvement.

Page | 3

2.0 Contract Performance Management Principles & Approach

2.1 Program Approach

Before describing a specific performance management program, it is important for the consortium manager to

understand a series of important principles behind the approach. As introduced earlier, the performance

management portion of the contract management function is focused on two core responsibilities:

1. Risk management and mitigation through active contract compliance monitoring; and

2. Improving the capacity and capabilities of the operators to provide effective and efficient services through

performance measurement, reporting, and feedback.

These are the core contract management responsibilities that, executed properly, ensure ongoing strong

performance by your bus operators and encourage the development and sustainability of a healthy base of suppliers.

To begin, it is vital that your consortium‟s contract managers understand the difference between compliance

monitoring and performance measurement.

2.1.1 Compliance Monitoring & Performance Measurement Described

Each of these two techniques holds a valuable place in the contract performance management program, but they are

very different in application.

Compliance monitoring:

creates a level playing field for all operators; and

ensures that the terms and conditions of the contract are met by all operators.

Performance measurement:

facilitates continuous improvement relative to performance-related contractual obligations;

provides a means for the operator to set itself apart from its competitors; and

encourages healthy ongoing competition within the supplier base.

There are specific requirements within the bus operators‟ contract that must be met. Each stipulation of the

contractual relationship is established in writing, and many require a specific action on the part of the operator.

Compliance monitoring ensures that these requirements are met and

that the operator remains in good standing relative to its responsibilities.

The majority of contract requirements are binary in nature where the

operator is either in compliance or not. These elements are not subject to

interpretation. The monitoring efforts of the contract manager should be

focused on first making this compliance determination for each provision

of the contract, and then on taking the remedial action required should

the operator be found out of compliance.

Another aspect of contract compliance must also be acknowledged and considered. The consortium itself, as

counterparty to the contract also has responsibilities and requirements that must be met. The compliance

monitoring aspect of the program therefore also touches on the importance of ensuring the consortium‟s own

compliance with the contract requirements, and how this relates to maintaining a positive professional relationship

with the bus operators.

Figure 3 – Compliance Monitoring

Page | 4

Performance measurement is substantially different from contract compliance. Performance measurement leverages

and expands upon the specific contractual requirements in order to improve the effectiveness and efficiency of the

consortium‟s services. Some of the specific contract requirements lend themselves to measurement rather than

compliance monitoring. Other aspects of the operators‟

responsibilities affect the overall performance of the consortium, but

are not specifically referenced in the contract. An example of the

former includes an operator‟s responsibility to comply with route

schedules supplied by staff. This is not a binary “yes/no” compliance

item, but rather can be actively measured and tracked over time as

a measure of performance. An example of the latter might be the

relative rate of complaints received by consortium staff concerning

the operators‟ performance. This is not a contractual stipulation, but

is clearly related to the operators‟ overall performance and in turn

influences the performance of the consortium as a whole.

Given this description, it becomes clear that performance measurement in the contract management context

should be an adjunct to, but not a surrogate for, the consortium’s overall use of key performance indicators.

The measures used in the contract performance management program should be specific to the performance of the

operators relative to their contractual obligations while key indicators for the consortium should reflect the overall

performance of the transportation system as a whole.

The measures and techniques for compliance monitoring and performance measurement that are utilized within the

Contract Performance Management Program should also be mindful of two other key factors, as described further

below.

2.1.2 Performance Management Scalability & Applicability

A valuable adage applies to the use of performance measurement as a tool. This is that you can only effectively

manage what you can measure. The inverse of this saying applies when considering the use of performance

measurement techniques within a program of contract management: you should only measure what you will actually

manage. Performance measurement will require resources, planning, and effort on the part of consortium staff and

the bus operators. It is the consortium manager‟s responsibility to ensure that value is to be gained before investing

in these resources.

The contract compliance monitoring and performance measurement plans

described in subsequent sections of this package are designed to be flexible

and scalable in their application. The consortium must determine its own

internal capacity and level of development before selecting those aspects that

it can put to effective use. Implementation in phases would allow staff to

learn by doing, and for the use of the plans to increase as experience is

gained. This will avoid burdening staff and bus operators with data

collection and reporting requirements that are not to be actively utilized for

actual contract management activities.

The plans are also designed to promote consistency in contract

performance management throughout the sector. The ultimate goal is for

the Contract Performance Management Program to be an integral part of

Figure 4 – Performance Measurement

Page | 5

a continuous improvement cycle that works to elevate the effectiveness and efficiency of the services provided by

every transportation consortium. Each manager must begin with those aspects of the program that best advance this

objective while gradually expanding the depth and breadth of the program over time.

Regardless of the starting point, consistency in application is of utmost importance. Should, for example, the

consortium determine that compliance monitoring is the only aspect of the program of immediate relevance, then it

must ensure that compliance audits are completed consistently, on time, and in the same manner for all of it bus

operators. It is not necessary for every consortium to be utilizing every aspect of the program. The overall

effectiveness of the program will only be diminished when those aspects chosen are inconsistently or unfairly

applied.

Continuous improvement is the objective. Feedback is the conduit that will enable achievement of this goal. The final

baseline aspect of the program that is critical for the consortium manager to understand is that consistent compliance

monitoring and measurement of performance, absent any feedback to the operators or corrective actions taken,

provides no particular value. There is a need to use the information and data gathered to first analyze and then to

modify performance based on the conclusions reached. The consortium must utilize the information gained internally,

but also put the information in the hands of the bus operators being monitored and measured. Regular reporting and

feedback mechanisms, together with collaborative problem solving are prerequisites if there is to be real value added

by the Contract Performance Management Program.

2.2 Performance Measurement Principles

Before moving on to the program specifics, a baseline understanding of the principles behind the selection of specific

performance measures is required. There are three core principles that should be followed when identifying specific

factors to be monitored or measured. Understanding these principles is critical to comprehending the underlying logic

behind the specific performance measures and processes contained throughout the remainder of this resource

package. The three principles relate directly to performance measurement, but also encourage further clarity and

draw a further distinction between measures of performance and contract compliance monitoring. Figure 5

summarizes these principles in a conceptual framework. A more detailed description of each principle follows below

this diagram.

Figure 5 – Performance Measurement Conceptual Framework

Page | 6

Principle 1 – Focus Your Performance Measures on Desired Outcomes

The natural tendency for any contract manager is to measure inputs instead of outcomes. Inputs are generally easier

to identify and track but fail to provide valuable data for understanding actual performance relative to contractual

requirements or the quality of the service being delivered.

Example: student behavior management

Ensuring that a bus operator conducts regular training on managing student behavior may be a valuable

factor to include in contract compliance monitoring activities. But this is not an outcome-driven measurement

of performance. The desired outcome in this example should be the minimization of discipline-related

incidents on the bus, not to determine whether the related training has been conducted. The training

requirement may indeed be identified as a contract compliance requirement and be subject to monitoring,

but as described earlier this only establishes a starting baseline to ensure that all operators are meeting

their contractual obligations. True enhancement of actual performance over time requires a more subtle and

nuanced approach and a focus on identifying and measuring desired outcomes, which leads to the second

core principle.

Principle 2 – Ensure That the Factor Chosen is Actually Measurable

Elements included in the Contract Performance Management Program that are strictly compliance-based are

inherently measurable because there is a binary “yes/no” determination. For true performance related elements, the

data must be accessible and a definition or performance standard must be established.

Example: Insurance certificates

This is a simple example of typical compliance item. There is a straight forward determination against a

contractual requirement as to whether the required insurance certificates have been submitted, on time, and

with evidence of the contractually required insurance coverage. This is not a measure of performance.

Example: student behavior management

To continue the earlier example, if the desired outcome is to determine whether student discipline problems

are being minimized by the operator, there must be a consistent way to gather data on discipline incidents,

including a definition of what constitutes an incident and a data collection mechanism such as a report form

or online reporting tool. Then, there must be a standard of performance established against which to

measure performance. This does not need to be a hard standard or one that is applicable throughout the

sector. Rather, it can be defined as an internal consortium goal, a comparison of one operator against all

others, or simply be tracked as a trend of performance over time. The key is to understand that the factor

chosen must be measurable and comparable. Only then does the information gathered, analyzed, and

reported become useful for management decision making and influencing future bus operator performance.

This is the essence of the value to be garnered from performance measurement.

Example: On-time arrival of buses

This is in oft-cited example of a likely measurement of bus operator performance. However, for it to be

useful there must be data available. If it is not readily available, it must be developed. Technology such as

GPS is rapidly improving and increasing the availability of extensive and detailed data on bus movements.

Unlike the student behavior management example, however, the quantity of data and the ability to

consistently measure on-time arrival is far more difficult. The second requirement also presents difficulty.

Against what standard would the operators be measured? The consortium must consider all of the inputs

Page | 7

that effect on-time performance before utilizing this complex factor in a performance measurement plan.

This leads to the third and final core principle.

Principle 3 – Actual Performance Must be Controllable by the Operator

For compliance items, this is established by contract and is not disputable. For true performance elements the factors

chosen should eliminate, where practicable, the possibility of external influences.

Example: On-time arrival of buses

To continue the earlier example, on-time arrival at school is a commonly thought of measure of bus operator

performance. But to utilize this as a valid measure of operator performance, the consortium must ensure

that the route is accurate as designed and that there are no other external factors, such as traffic, that can

influence the results. This is a difficult standard to meet, and largely negates the utility of this as a measure

of operator performance, although it may find significant utility as a key indicator of overall consortium

performance. There is a much higher standard to be met for performance measurement than for simple

contract compliance monitoring. Proper and effective use of performance measurement requires great care

and forethought in its application and use as a technique.

The balance of this resource package provides a description for a comprehensive Contract Performance

Management Program. It is designed as a template for use by all transportation consortia. This template is

necessarily generic, and is meant to be customizable to suit the unique conditions and circumstances of your

consortium. It is also not designed to be definitive in content, but rather to provide a consistent framework from which

future expansion and customization is possible without undermining the validity, applicability, or consistency of the

baseline program.

The Contract Performance Management Program that follows incorporates each of the basic fundamentals,

approach, and principles introduced in the preceding sections of this resource package. It begins with an overall

description of the program essentials. Following this are two appendices that provide sample forms, templates, and

tools to support each of the two major sub-parts to the program: the Contract Compliance Management Plan; and the

Performance Measurement Plan.

Page | 8

3.0 Contract Performance Management Program Description

The remainder of this resource package outlines a generic Contract Performance management program that can be

customized for use by each individual consortium. The program has five component parts. These are described in

Table 1, which includes a cross-reference to the page numbers where the user can find a detailed description of the

component and any tools, samples, and templates supporting the implementation of the program component.

Implementation of the program must be customized for each consortium, and the template forms, samples, and tools

provided within this package are each designed to provide a starting point for you to design a contract performance

management program that meets the unique operating conditions of your consortium. The generic steps to follow in

designing your customized program are illustrated in Figure 6.

Figure 6 – Contract Performance Management Program Implementation Steps

Page | 9

Table 1 – Contract Performance Management Program Cross-Reference

Program Component Component Description Description

Cross-

Reference

Supporting Tools Cross-

Reference

1. Contract

Performance

Management

Program

Responsibilities

outlines specific responsibilities and general instructions for overseeing the program

Page 10 Appendix A: Contract Performance Management Program Reference Matrix (Page 19)

2. Contract

Management

Calendar

provides a generic annual calendar to identify a regular schedule of events and milestones for each contract year

Page 13 None

3. Contract Compliance

Monitoring Plan

describes the particular elements of the program related to contract compliance monitoring

Page 14 Appendix B:

Sample Compliance Audit Procedures & Standards (Page 30)

Sample Compliance Audit Checklist (Page 32)

Sample Compliance Audit Report (Page 38)

Sample Route Audit Form (Page 40)

4. Contract

Performance

Measurement Plan

describes the particular elements of the program related to operator performance measurement and reporting

Page 16 Appendix C:

Performance Measure Matrix (Page 43)

Performance Measure Glossary (Page 45)

Performance Measure Tracking Summary Spreadsheet (Page 47)

Sample Operator Performance Report (Page 47)

5. Annual Contract

Performance

Assessment

provides a template for a formal annual assessment and review of bus operator performance

Page 18 None

Page | 10

3.1 Contract Performance Management Program Responsibilities

3.1.1 Program Assignments

Responsibility for determining a contract management strategy, assigning staff resources, providing program

oversight, and managing bus operator relations must ultimately reside with the senior consortium manager. The bus

operator contract template facilitates the distribution of contract management assignments throughout the consortium

organization1. This provision of the contract states that “except where otherwise expressly provided above, for the

purposes of administering the provisions of this Agreement, “the Consortium” shall be deemed to include the

Consortium‟s duly authorized officers and employees.”

Assignment of specific responsibilities and tasks for a successful program must be dependent on the size and needs

of the consortium. In most cases, more than one responsibility will be vested with an individual manager or staff

member. In all cases the following requirements should be specifically addressed in the program staffing plan:

Program administrator responsibilities – The program administrator, typically the consortium manager,

should retain overall accountability for the program. The responsibilities of the administrator include the

determination of program strategy, the specific elements of compliance monitoring and performance

measurement to be pursued, management of the program calendar, and design and implementation of the

bus operator assessment portion of the program.

Contract manager responsibilities – The contract manager should be vested with the responsibility for

executing all aspects of the program as it relates to the assigned contract. This is the primary point of

contact for the bus operator, and the primary point of responsibility for management of the consortium

relationship with the bus operator. This includes ensuring that the consortium meets its contractual

responsibilities relative to the bus operator.

The responsibilities for all contract managers must be clearly defined and fairly applied across all contracts.

The specific responsibilities of each contract manager should include:

Providing for the consistent application of established contract management and performance

management processes;

Managing contract paperwork and timetables to ensure the integrity and validity of the physical

contract;

Actively managing the ongoing professional relationship between the bus operator and the

consortium;

Working with the bus operator community and other contract managers to monitor the health of the

supplier base and to ensure the continued viability of the competitive procurement process;

Managing and mitigating risk through rigid contract compliance monitoring activities; and

Improving the capabilities and capacity of the bus operator through performance measurement and

feedback mechanisms.

Consortium staff responsibilities – Specific program tasks can be distributed throughout the organization.

Depending on the size and scope of the program, various data collection, auditing, monitoring, and

analytical tasks will be required. Many of these, such as the use of various technologies for data collection

and analysis, will be most efficiently pursued through staff specialization. With clear delineation of program

1 Paragraph 15.2 “Administration Provisions for the Consortium”

Page | 11

administration and contract management responsibilities beforehand, the consortium can best organize

these activities for maximum effectiveness.

Bus operator responsibilities – A successful program will be one in which the bus operators are active and

engaged participants. The contract template establishes their service requirements, including the

designation of a named “Transportation Officer2” to serve as the operator‟s primary information conduit to

the consortium. This structure does not, however, guarantee a successful relationship. The bus operators

must be engaged from the outset of the contractual relationship to fully understand the purpose and intent of

the program, and to fully understand their obligations under the contract.

School board responsibilities – The client school boards of the consortium also have a role to play in a

successful program. Much of the interaction between the consortium‟s end users (the students being

transported and their guardians) and the consortium‟s providers (the bus operators) occurs without direct

involvement by consortium staff. The school boards‟ representatives should be briefed and aware of

program intent, purpose, and requirements. They should be engaged by the program administrator and

contract managers to help support the continuous improvement cycle through information collection and

transfer activities. This in turn supports the effective implementation of the program‟s compliance monitoring

and performance measurement activities, and supports the cycle of continuous improvement.

3.1.2 Program Communications Protocol

Regular communications, both structured and unstructured, will be a critical success factor in a successful contract

performance management program. The competitive procurement process fundamentally alters the nature of the

relationship between the provider (the bus operator) and the buyer of the service (the consortium). The formality

introduced requires a concurrent elevation of the communications protocol. The structured and unstructured

elements of the communications plan should include the following elements:

Contract Manager & Transportation Officer ongoing communications – The most important element in the

communications plan will be the regular, informal communications that occur between the bus operator and

the consortium. The contract establishes a framework for this relationship, but cannot hope to document the

required behavior of the counterparties for every possible situation and iteration that might occur over the

contract term. A clearly established relationship between the two key individuals within each organization,

and open ongoing communications between them, is an absolute requirement. It must be made clear to all

parties that this is a formal relationship, and that all matters pertaining to the contract and contract

performance must be passed via this channel of communication.

Annual and ongoing submission requirements – The contract requires that various pieces of information be

passed between the parties at the outset of the contractual relationship and periodically throughout the

contract term. These requirements are summarized in the Contract Management Calendar and the Contract

Compliance Monitoring Plan that follow. This forced communication provides a mechanism to enhance the

level of cooperation that occurs throughout the annual service delivery cycle.

Periodic route audits – Another key conduit for communications between the parties will be periodic route

audits. This mechanism will be utilized to collect data to support both the Contract Compliance Monitoring

Plan and the Contract Performance Measurement Plan, but also serves as an opportunity for consortium

and bus operator staff to interact, improve the relationship, and solve problems on a tactical level.

2 Paragraph 11.1 “Operator‟s Transportation Officer”

Page | 12

Periodic performance assessment reporting – Ongoing constructive feedback regarding operator

performance is a critical aspect of the continuous improvement cycle. The output of the Contract

Performance Measurement Plan should be summarized within a periodic report that is submitted to, and

discussed with, the bus operator. A generic sample report is included within the Contract Performance

Measurement Plan that follows. This is a tool that provides a mechanism to help guide the ongoing

communications necessary for a successful contract management program.

Annual contract compliance audit – A complement to the route audits, this mechanism will also be utilized to

collect data to support the Contract Performance Management Program. It will also facilitate ongoing

communication between consortium and bus operator staff. A template contract audit procedure and

checklist is included with the Contract Compliance Monitoring Plan that follows.

Annual formal performance assessment meeting – This should be treated as the capstone to annual service

delivery cycle. The meeting should bring together not only the Contract Manager and Transportation Officer,

but also the Program Administrator and the senior administrators from the bus operator. The meeting should

be structured, have a positive tone, and be utilized to leverage improvements to overall performance in the

contract year that follows. A sample agenda and performance assessment template are included in the

Annual Contract Performance Assessment description that follows.

Formal appeal procedures – A formal appeal process should be established to serve as an outlet for the bus

operator in the unlikely situation where the relationship between the Contract Manager and the

Transportation Officer is ineffective. This should be formally documented and provided to the bus operator

at the outset of the contractual relationship. The bus operator contract template contains a dispute

resolution clause that can serve as the basis for these procedures.

Page | 13

3.2 Contract Management Calendar

There is a natural life cycle to the contractual relationship established between the consortium and the bus operator

that coincides with the term of the contract. Embedded within this overall life cycle is an annual service delivery cycle.

The specific elements of the contract performance management program work within these two cycles. The

suggested timing for the various elements of the program are summarized in the following generic contract

management calendar.

Contract Month Annual Service Cycle Requirement

July Complete contract initiation tasks (new contracts)

Establish contract management program (new contracts)

Performance assessments (current contracts)

Operator compliance analysis (current contracts)

August Begin service (new contracts)

Begin performance measurement (new contracts)

Performance assessments (current contracts)

Staff review of Consortium contractual obligations (all contracts)

September Monthly performance data collection

October Monthly performance data collection

November Monthly performance data collection

Route audits

December Monthly performance data collection

Route audits

CVOR data collection

January Monthly performance data collection

Route audits

Driver turnover analysis

February Monthly performance data collection

Route audits

Customer satisfaction survey

March Monthly performance data collection

Route audits

April Monthly performance data collection

Route efficiency analysis

Compliance audits

May Monthly performance data collection

Compliance audits

Safety violations analysis

June Monthly performance data collection

Compliance audits

Page | 14

3.3 Contract Compliance Monitoring Plan

3.3.1 Annual Contract Compliance Audit

The primary mechanism supporting this portion of the Contract Performance Management program is the annual

contract compliance audit. The purpose for this audit is to ensure that every bus operator meets the minimum

requirements established by the contract. The result of the audit provides input to the Contract Performance

Measurement Plan and should also become an integral part of the Annual Contract Performance Assessment.

Each compliance element in the annual contract compliance audit should be separately determined on a pass-fail

basis (i.e., either “in compliance” or “not in compliance”). The overall assessment for the bus operator however, must

necessarily be more subjective. There is an extensive list of compliance items to be verified, and a satisfactory level

of compliance may not require a 100 percent rating. Rather, the Program Administrator must develop a standard

against which all bus operators are fairly rated and where the results support the overall assessment of the operator‟s

performance.

There are three primary tools required to conduct an annual contract compliance audit. These are described as

follows. Samples and templates for each of these are provided within Appendix A of this resource package.

Compliance Audit Checklist – A compliance audit checklist is the key tool to be utilized in gathering the data

required to determine compliance. There is likely to be an extensive list of contract compliance items, some

of which are simple to verify and others that require a significant amount of staff effort. Depending on the

resources available, the consortium‟s relative level of development, and the status of the competitive

procurement process the consortium can choose to start with a program that limits the number of

compliance items verified annually and gradually expands to incorporate all compliance items. The Annual

Contract Compliance Audit Checklist included in Appendix A provides a starting point for the development of

a customized audit checklist that suits the needs of each individual consortium. This checklist is based on

the contract template issued by the Ministry of Education.

Compliance Audit Procedures & Standards – The checklist must be accompanied by a set of procedures

that describe how it is to be applied and the standards against which compliance is measured. For example,

the procedures must define when and how the audit will be conducted, whether compliance is to be

measured only at the time of the audit or whether the operator is provided with an opportunity to correct any

discrepancies, and the consequences for non-compliance. The Annual Compliance Audit Procedures &

Standards included within Appendix A provide a starting point for the development of a customized

approach that suits the needs of each individual consortium.

Compliance Audit Report – The results of the audit will only be useful for improving performance if they are

clearly articulated and discussed. A final companion piece to the audit checklist should be a compliance

audit report that summarizes and describes the results of the audit, the implications for the bus operator,

and the recommendations for improvement. This report should become an integral component of the

operator‟s annual performance assessment. The Annual Contract Compliance Audit Report included within

Appendix A provide a starting point for the development of a customized approach that suits the needs of

each individual consortium.

3.3.2 Periodic Bus Route Audits

A second critical mechanism within the Contract Compliance Monitoring Plan is the conduct of periodic bus route

audits. These audits serve a dual purpose. First, they represent compliance monitoring requirement that cannot be

satisfied through the annual compliance audit. The requirement for bus operators to comply with the bus routes and

Page | 15

schedules provided by the consortium, and to operate all routes in compliance with the consortium‟s policies and

procedures is fundamental to the contractual relationship. It is also the fundamental part of the services provided by

the bus operators. As such, compliance must be demonstrated in an ongoing fashion. A regular schedule of route

audits provides the mechanism to ensure this compliance. In addition, the route audits provide a means to collect

data that will also support the Contract Performance Measurement Plan.

A route audit checklist is a key tool to be utilized in gathering the performance measurement data and other

information required to determine compliance. Methods for conducting route audits are likely to vary considerably

with the technology and staffing resources available to the individual consortia and the consortia‟s operating

geography. A sample Route Audit Form is included within Appendix A. This form supports the data collection

requirements of the suggested Contract Performance Measurement Plan, and provides a basic and generic starting

point for the development of a customized approach that suits the needs of each individual consortium.

3.3.3 Facility Audits

Each bus operator is a passenger carrying motor carrier that also falls under the regulations of the Ontario Ministry of

Transportation. There are a significant number of requirements imposed, and a robust monitoring and measurement

program put in place by MTO for the purpose of ensuring compliance. The Commercial Vehicle Operator's

Registration (CVOR) system and the Carrier Safety Rating (CSR) program were developed by the MTO to promote

the safe operation of trucks and buses. The CVOR system is part of the Carrier Safety Rating (CSR) program. The

Ministry of Transportation monitors carriers and assigns each a Safety Rating based on several factors: collisions,

inspections, and convictions, as well as the results of facility audits. The CVOR Level 1 operator rating is a matter of

public record, and is incorporated into the operator contract template. The Level 2 operator rating provides summary

data for two years and detailed event data for five years. It is not incorporated into the contract template. This MTO

program does not need to be repeated, and has been incorporated into the suggested Contract Performance

Management Program within the Contract Performance Measurement Plan that follows.

Page | 16

3.4 Contract Performance Measurement Plan

The primary tool supporting this portion of the program is a package of interrelated performance measures that follow

the principles described in the introductory sections of this resource package. This provides for an integrated

performance measurement and reporting program that encourages a comprehensive understanding of operator

performance, how different measures interact with each other to describe overall performance, and that provides for

feedback to operators that can be acted upon to improve performance. This is not designed to be evaluative in the

sense that operators will be judged one against the other. Rather, it is designed to provide consortium managers with

a structured and consistent tool to first understand, and then take action to improve operator and consortium

performance. The compliance monitoring program ensures that operators meet their contractual obligations. The

performance measurement program provides an opportunity for these same operators to demonstrate their

commitment and wherewithal to provide service above and beyond the minimum requirements established by the

contract.

The structure for the package of measures is loosely based on the Effectiveness and Efficiency framework, and more

specifically based on four major categories of outcome-driven measures:

Safety – Measures that assist the consortium in determining and improving the overall safety of

transportation operations

Cost - Measures that assist the consortium to improve the overall efficiency of its operations.

Reliability – Measures that assist the consortium in determining and improving the reliability of the bus

operators‟ operations and fleet

Responsiveness – Measures that assist the consortium in determining and improving the responsiveness of

the bus operators to the needs of the consortium and its customers

The specific measures recommended for use are provided in matrix form within Appendix B. The Performance

Measure Matrix describes whether the measure is a primary, secondary, or tertiary measure, which can be utilized by

the consortium for prioritization and program phase-in planning. Figure 7 illustrates this approach. Tier 1 measures

should be considered for implementation first, and Tier 3 measures last. Tier 1 measures provide the foundation from

which a more robust and finely tuned performance measurement plan is constructed over time.

Figure 7 – Performance Measurement Phasing

The Performance Measure Matrix further describes which of the four categories each measure belongs to, and

whether the measure relates to a contractual compliance item or is a separate performance measurement item. The

package of suggested measures is not intended to be definitive. Rather, it is designed to provide a consistent

Page | 17

beginning framework for the sector as a whole, with the expectation that the measures utilized will expand and evolve

over time as the relative level of contract management sophistication improves.

There is a companion Performance Measure Glossary provided within Appendix B. This is associated with the

Performance Measure Matrix and describes, for each measure:

The standard of performance – These are not presented as benchmarks, but rather describe the type of

standard (e.g., manufacturer standard, peer comparison, internal consortium standard, contractual standard,

etc.) that should be utilized for the specific measure.

What should be measured – This describes whether the measure is best suited to instance sampling, an

average over time, or some other method.

How the measure should be calculated – This describes the actual calculation of the measure to provide

consistency across all users.

The likely sources of required data – This describes where the data required for calculating the measure can

be collected or developed.

When and how frequently the factor should be measured – This provides recommendations for the timing

and periodicity of measurement.

How the results should be reported and tracked – This provides a recommendation for how to record and

summarize the results of the measurement.

These two matrices combined provide the core reference tools for the Contract Performance Measurement Plan.

Unlike the Contract Compliance Monitoring Plan where the elements are clearly articulated and ascribed in a series

of checklists and forms, the Contract Performance Measurement Plan should be considered as ongoing. Data

collection, measurement, analysis, and reporting of operator performance should be a continuous effort. To this end,

a Sample Performance Measure Tracking Summary and Sample Operator Performance Report are provided in

Appendix B and in spreadsheet format as templates for beginning construction of a customized tracking and

reporting program that meets the needs of the individual consortium.

Page | 18

3.5 Annual Contract Performance Assessment

The capstone for the contract performance management program is the Annual Contract Performance Assessment

that should be completed for each bus operator. This is the primary formal method of feedback to be provided to the

operator. The Annual Contract Compliance Audit Report (see Appendix A) and the Sample Operator Performance

Report (see Appendix B), or the local consortium developed alternatives, should provide the primary source

information for completing the assessment.

With a high quality Contract Performance Measurement Program in place, this formal annual assessment should be

productive and collaborative. The Contract Manager and Transportation Officer will have been in constant

communication throughout the service year. Information will have been flowing back and forth between the operator

and the consortium to support specific elements of the compliance monitoring and performance measurement

programs, and informally between the staff of the two organizations to manage day-to-day requirements and solve

problems. With the expectations of the program clearly established and rigidly enforced, the Annual Contract

Performance Assessment therefore becomes an opportunity to celebrate accomplishments and work collaboratively

to improve performance. The assessment should consist of the following key elements:

Consortium Assessment of Operator Performance – Either a memorandum or formal letter, with the Annual

Contract Compliance Audit Report and the Operator Performance Report as attachments, should be

provided to the operator in advance of the assessment meeting summarizing the Program Administrator‟s

assessment of operator performance for the service year, and making recommendations for improvement. A

key element of this report should be a statement of compliance with the contractual requirements. If the

operator is found to be out of compliance and remedial steps have not been taken, contract termination

provisions might be invoked. This initial assessment document serves as a discussion piece and guide for

the balance of the assessment process.

Assessment Meeting – The Program Administrator, Contract Manager, Transportation Officer, and a senior

management representative from the operator should meet to discuss the consortium report and determine

action items for the following service year. The form and content for each bus operator assessment meeting

will depend on the circumstances surrounding each operator‟s performance. For the sake of fairness,

however, the agenda should be roughly consistent for all meetings. As a sample, the agenda might include

the following elements:

Introductions, review of agenda, and opening statement by the Program Administrator

Review of operator performance for the contract year by the Contract Administrator

Operator representative comments, questions, or rebuttal

Review of operator improvement recommendations for the following contract year by the Contract

Administrator

Operator representative comments, questions, or explanation

Wrap-up and next steps by the Program Administrator

Operator Response – The bus operator should be provided with the opportunity and encouraged to submit a

formal written response to the consortium report and assessment meeting. This response can lay out the

operator‟s response to the improvement recommendations, and plans for implementation. This response

and the actions taken can then be utilized as an additional input to the following annual performance

assessment.

Page | 19

Appendix A – Contract Performance Management Program Reference Matrix

This following table provides a master cross-reference matrix for the entire program relative to requirements of the operator contract template. The matrix references where

the contract requires consortium action, whether the contract requirement should be audited for compliance, whether the contract requirement should be considered for

inclusion in the performance measurement plan, what mechanism should be utilized for monitoring within the overall program, and where the required data or information

can be accessed. This table serves as a primary reference tool for understanding the consortium‟s requirements under the contract, and for developing the consortium‟s

overall Contract Performance Management Program. It should be noted again that this matrix is based on the contract template; any changes or modifications made to that

document by the consortium must be considered separately from this matrix.

Contract Reference

Contract Section Title

Contract Compliance Requirement Consortium Action

Required

Compliance Audit Item

Performance Measurement

Item

Program Monitoring Mechanism

Possible Data

Sources

1. Preamble Contract dates and signatories to be entered X Contract initiation

1.1 Agreement Precedence

RFP and RFP response dates to be entered X Contract initiation

1.2 Article Headings None

1.3 Definitions Contract rates and service dates to be entered X Contract initiation

2. Contract Term and Early Termination

None

2.1 Term and Optional Extensions

Contract term to be entered X Contract initiation

2.2 Option to Renew for Two Additional School Years

Contract extension terms to be entered X Contract initiation

2.3

Termination at any Time by the Consortium

Determination of operator compliance with contract terms generally

Determination of operator compliance with Provincial statutes and regulations

X

Annual Operator

Assessment

2.4 Cancellation or Alteration of Routes by Consortium with Notice

Notification requirements to be entered

Notification provided in accordance with requirements

Rates paid for modified routes in accordance with requirements

X Contract initiation

Consortium operations

Page | 20

Contract Reference



Required



Item


Possible Data

Sources

2.5 Cancellation of Services by the Consortium During Inclement Weather





2.6 Cancellation of Services by the Consortium as a Result of an Operator Labour Dispute

Notification provided in accordance with requirements X Annual Contract Audit

Checklist

Consortium records

2.7 Cancellation of Services by the Consortium as a Result of Board Labour Disputes


X Consortium operations

3. Services None

3.1

Consortium Shall Determine and Communicate Routes and Schedules


Schedule C (routes) to be attached to contract


Service provided in accordance with Schedule C

X X Contract initiation



Program

Route Audit

4. Safety Program Submission of a complete copy of the Operator‟s current safety program

X Annual Contract Audit

Checklist

Operator Submission

5. Operator None

5.1 Inadmissibility of Charter Runs

Charter runs or other fares are not permitted to prejudice the operation of any regular route


Checklist

Onsite Records Review

Page | 21

Contract Reference



Required



Item


Possible Data

Sources

5.2 Drivers to be Aware of and Follow the Conditions of the Agreement

It is the responsibility of the Operator to ensure that its drivers are aware of and follow the relevant and appropriate conditions of the Agreement


Checklist

Route Audit

5.3 Independent Contractor

None

5.4 Operator to be Deemed in the Business of Carrying Passengers for Compensation

None

5.5 Operator Representations and Warranties

None

5.6 No Assignment of Subcontracting Without Consent

Prior consent is required in writing X Annual Operator Site

Audit Checklist


5.7 Indemnification by the Operator

None

5.8 Compliance with Acts, Regulations, and Policies

A list of Acts and Regulation is to be provided

Policies are to be provided, including changes

Operators are to comply with Acts, Regulations, & Policies




Program

Route Audit


Consortium Records

5.9 Evidence of Financial Ability

Provision of evidence of financial ability to provide services.


Checklist

Operator Submission

Page | 22

Contract Reference



Required



Item


Possible Data

Sources

5.10 Driver and Vehicle Qualifications and Consent to Disclosure

An up to date Driver‟s License Abstract and current Commercial Vehicle Operator‟s Registration Search is required prior to the start date of the contract and as requested by the Consortium

At the start of each year of the Agreement, a Consent to Disclosure for the Operator and drivers is required


Annual Contract Audit

Checklist

Operator Submission

6. Default None

7. Insurance None

7.1 Evidence of Insurance Evidence of insurance required fifteen (15) day before the Contract Start Date


Operator Submission

7.2 Insurance Coverage Confirm required minimum coverage is in place


Checklist

Operator Submission

7.3 No cancellation Without Notice to the Consortium

Proof of insurance must be provided each year within the term of the Contract


Checklist

Operator Submission

7.4 Workplace Safety and Insurance Board Insurance

Proof of coverage and a clearance certificate indicating that the Operator is in good standing


Checklist

Operator Submission

8. Routing None

8.1 Base Rate on a Per Route, Per Day Basis

None

8.2 Days in Each School Year

Page | 23

Contract Reference



Required



Item


Possible Data

Sources

8.3 Operator Compliance with the Routes and Schedules

The operator shall strictly comply with route schedule supplied by the Consortium with no changes made without written authorization

Drivers are not to change any pupil from one vehicle to another without written authorization unless under emergency conditions, in which cases the Operator must advise the Consortium as soon as possible

An up to date route schedule is to be kept in the bus at all times; Uptodate copies must be made available and issued to replacement drivers by the Operator.

X X Consortium operations


Program

Route changes to operators

Route Audit

8.4 Requests for Transportation Changes

No student may be transported unless previously authorized by the Consortium

X Performance Measurement

Program

Route Audit

8.5 Consortium May Require Vehicles to Service More Than One Route

None

8.6 One Driver – One Route and Route Doubling

Unless specifically authorized in writing by the Consortium, each route will be served by the same driver each day; For permanent changes or changes with a duration likely to extend greater than two weeks, written notice must be provided

the Operator shall not, without prior notification to the Consortium and without specific prior acceptance of such notice, use Route Doubling

X X Annual Contract Audit

Checklist


Program


Consortium records

8.7 Pickup and Discharge Locations and Timing

Operator‟s vehicles shall stop only at those locations designated for the picking up and discharging of students pursuant to the attached schedule

Drop-off no earlier than 15 minutes before school bell; Pickup not later than 15 minutes following dismissal at school (Consortium must ensure route design compliance)

X X Consortium operations


Program

Route Audit

Page | 24

Contract Reference



Required



Item


Possible Data

Sources

8.8 Operator Covenants with Respect to Student Transportation

Date of required submission to be entered

Operators shall provide a bus route data sheet including the make, model number, and year of manufacture of the serving any route


Annual Contract Audit

Checklist

Operator Submission

8.9 Obligation to Perform Trial Runs Prior to the Start of the School Year

Consortium shall determine the requirement for trial runs

Operator shall ensure that all drivers perform trial runs of their route(s)

X X Annual Contract Audit

Checklist


9. Vehicles None

9.1 Licensing and Equipment

All vehicles must comply with current federal, provincial, and municipal laws and regulations


Checklist


9.2 Passenger Loading Specifications

At no time will the number of students in any vehicle exceed the manufacturer‟s passenger loading specifications


Program

Route Audit

Operator Reported

Route Errors

9.3 Vehicle Age Vehicles used in the provision of transportation shall confirm to maximum and average age specifications as of August 1st in each year


Checklist

Operator Submission

9.4 Licensing in Accordance with Acts, Regulations, and Standards

The Operator shall use a motor vehicle properly licensed as required


Checklist


9.5 Vehicle Characteristics

For mini vans and cars, passenger doors must be equipped with child-proof locks

Children 12 years of age or younger will not ride in the front seat in any vehicle equipped with front seat airbag safety devices


Checklist

Onsite Fleet Review

9.6 Vehicle Communications

Operator shall ensure that each vehicle is equipped with a twoway radio or equally reliable means of communication

Contact with the Operator‟s main dispatcher will be maintained at all times during the scheduled bus routes


Checklist

Onsite Fleet Review

Route Audit

Page | 25

Contract Reference



Required



Item


Possible Data

Sources

9.7 Ministry of Transport Annual Inspection Certificate

Ministry of Transport Annual Inspection Certificate must be provided


Checklist


9.8 Maintaining Log Books and Consortium Right to Inspect

Daily driver log books are to be available for inspection by the Consortium on request


Checklist


9.9 Consortium Rights to Require Mechanical Fitness Reports

None

9.10 Failure to Comply with Articles 9.7, 9.8, and 9.9

None

9.11 Compliance with the Definition of a School Bus

The Operator shall ensure that all school buses comply with the definition of a “school bus”


Checklist

Onsite Fleet Review

9.12 Additional School Bus Equipment in Case of Breakdowns or Delays

The Operator will provide for a specified number of spare vehicles


Checklist


9.13 Display of Route Numbers

All vehicles shall display vehicle signs indicating route numbers


Checklist

Route Audit

9.14 Vehicle Cleanliness The interior and exterior of the vehicles must be maintained to provide a clean appearance


Checklist

Route Audit

10. Drivers

10.1 Driver Licensing Requirements

Each driver shall have and maintain in good standing at all times a Class „B‟ or „E‟ license and any additional license required


Checklist


10.2 Copy of the Driver Training Manual

A current driver training manual is to be provided to the Consortium


Checklist

Operator Submission

10.3 Hiring of Bilingual Drivers

The Operator shall make every possible effort to hire bilingual drivers for routes that service French or English language schools


Program

Page | 26

Contract Reference



Required



Item


Possible Data

Sources

10.4 Driver Characteristics Drivers shall be appropriately dressed

Drivers must carry an Identity Card

Drivers must be experienced

Drivers shall not eat, drink, smoke, or use cell phones


Checklist

Route Audit

10.5 Driver Actions List of driver actions required while providing services X Annual Contract Audit

Checklist

Route Audit

10.6 Consortium Rights in Event of Driver‟s Breach of Contract Conditions

None

11. Administration None

11.1 Operator‟s Transportation Officer

The Operator will name one key person in its organization as “Transportation Officer”; The person in question shall act as a liaison between the Operator and the Consortium


Program

11.2 Notification of Delays The school and Consortium is to be notified in any instance where students may arrive later than fifteen (15) later than scheduled to either the school or home.


Program

Operator reported data

11.3 Communications with Parents and Consortium

None

11.4 Service and Safety Surveys and Programs

The Operator‟s current safety program will be made available to the Consortium


Checklist

Operator Submission

12. Consideration and Accounting

None

12.1 Basis for Payment Monthly invoices are to be submitted by each Operator, with required details and itemization


Checklist

Consortium records

12.2 Fuel Computation and Fuel Adjustments

Monthly invoices are to be submitted by each Operator, with required details and itemization


Checklist

Consortium records

12.3 Base Costs None

Page | 27

Contract Reference



Required



Item


Possible Data

Sources

12.4 Other Adjustments None

12.5 Payment Calculation None

12.6 Payment Terms None

13. Accidents None

13.1 Operator and Driver Actions in Event of Accidents

Operator to inform the appropriate school principal or vice-principal and transportation office at the Consortium forthwith and submit detailed written accident report within forty-eight (48) hours of the incident


Checklist

Consortium records


14. Notices None

14.1 Timing and Delivery of Notices

None

15. General None

15.1 Amendments to the Agreement

None

15.2 Administration Provisions for the Consortium

None – Enables contract management by Consortium staff

15.3 Successors and Permitted Assigns

None

15.4 Confidentiality of All Information

None

15.5 Operator‟s Right to Provide Services to Other Boards

Operator to provide copies of other authorized agreements


Checklist

Operator Submission


15.6, 15.7 Dispute Resolution None

15.8 Right to Audit None – Enables audit provisions of contract management program

Page | 28

Contract Reference



Required



Item


Possible Data

Sources

16. Standards of Performance

None – Enables the performance measurement provisions of the contract management program

16.1 Conformance to Standards of Performance

None currently specified X Performance Measurement

Program

16.2 Performance Failure None currently specified – enables future addition of remedies provisions

17. Surety, Bonds and/or Letters of Credit

A performance surety will be provided by the Operator X Annual Contract Audit

Checklist

Operator Submission


18. Survival None

19. Non-Waiver None

20. Rights Cumulative None

21. Time of Essence None

Page | 29

Appendix B – Contract Compliance Reference Tools, Samples & Templates

This appendix includes a series of sample documents and forms for use as templates and to be customized for use

in developing the consortium‟s contract compliance plan. They include:

B.1 – Sample Annual Compliance Audit Procedures & Standards

B.2 – Sample Contract Compliance Audit Checklist

B.3 – Sample Contract Compliance Audit Report

B.4 – Sample Route Audit Form

Page | 30

B.1 Sample Annual Compliance Audit Procedures & Standards

The annual operator compliance audit should be conducted during the same month of each contract year for each

bus operator. Completion depends on the accumulation of operational data and, to maximize relevance, should be

conducted toward the end of each school year between the months of April and June. Completion of the audit will

likely require information from five sources3:

Operator Submission Requirements – These are contractual obligations that require the submission of

documents and information by the bus operator for each contract year.

Review of Operator Records – Compliance with these elements is determined through a review of operator

records during an onsite audit at the operator‟s facility.

Onsite Fleet Review – These are contract compliance elements that are best verified through a physical

inspection of the operator‟s fleet.

Route Audit Records – These are compliance items that can be determined through an examination of route

audit records.

Consortium Records – Compliance with these elements is best determined through an examination of the

consortium‟s own records.

B.1.1 Annual Audit Procedures

The suggested procedures to be followed in conducting each compliance audit should include the following elements:

1. Notify the bus operator at least one month in advance of the dates selected for the audit. Include in this

notification the final date on which all operator submission requirements must be satisfied and the date on

which the onsite records review and fleet assessment will be conducted.

2. Gather internal consortium information required for audit completion.

3. Review the results of past compliance audits to ensure completion of any required corrective actions.

4. Complete the operator submission, route audit, and consortium records portions of the audit checklist,

noting any deficiencies for discussion with the bus operator during the onsite component of the audit.

5. Complete the onsite records review and fleet review, noting any deficiencies for discussion with the bus

operator.

6. Complete the preliminary Annual Compliance Audit Report and submit to the bus operator for action and

response.

7. Receive the bus operator response and complete the final Compliance Audit Report; submit the final report

to the bus operator and retain for internal review and reference purposes.

B.1.2 Annual Audit Standards

The following suggested standards should be applied consistently to all bus operator compliance audits:

1. The auditor must examine all relevant records and/or conduct a physical examination sufficient to satisfy

any doubts regarding compliance with each stated contractual requirement. The rationale for any

3 These correspond to the sub-sections of the companion Sample Contract Compliance audit Checklist.

Page | 31

requirement deemed to be out of compliance must be fully documented and explained in the preliminary

audit assessment.

2. The preliminary report must contain a determination of whether the bus operator is “fully compliant”,

“substantially compliant” or “not in compliance” with the contract, and this determination must be fairly

determined and assigned to each operator.

3. The compliance determination is to be completed on the date of the audit; submission of information to

challenge or change the compliance determination for any contractual requirement after the date of the audit

shall be considered as a corrective action.

4. The bus operator must submit its response to the overall compliance determination, and describe the

corrective actions taken or to be taken for each specific contract requirement determined to be out of

compliance. Failure to submit a response within a predetermined number of days following receipt of the

preliminary report, with the exception of those operators deemed to be “fully compliant”, shall be cause for

contract termination.

5. The consortium shall evaluate the operator response and determine whether the response and actions

taken or to be taken will result in the operator becoming “fully compliant”. If so, the final report shall indicate

this determination. If not, the final report shall specify the steps to be taken by the operator to become fully

compliant and the dates required for completion.

6. The auditor should calculate and record the bus operator‟s compliance rating in accordance with the

Performance Measurement Plan.

Page | 32

B.2 Annual Contract Compliance Audit Checklist4

Bus Operator Name:

Contract Year:

Ref Contract Title Requirement Compliance Corrective Action and Notes

Yes No

Operator Submission Requirements

4. Safety Program Submission of a complete copy of the Operator‟s current safety program

5.9 Evidence of Financial Ability

Provision of evidence of financial ability to provide services.

5.10 Driver and Vehicle Qualifications and Consent to Disclosure

An up to date Driver‟s License Abstract and current Commercial Vehicle Operator‟s Registration Search is required prior to the start date of the contract and as requested by the Consortium

At the start of each year of the Agreement, a Consent to Disclosure for the Operator and drivers is required

7.1 Evidence of Insurance

Evidence of insurance required fifteen (15) day before the Contract Start Date

7.2 Insurance Coverage

Confirm required minimum coverage is in place

4 The Annual Contract Compliance Checklist is also provided separately in electronic format.

Page | 33


Yes No

7.3 No cancellation Without Notice to the Consortium

Proof of insurance must be provided each year within the term of the Contract

7.4 Workplace Safety and Insurance Board Insurance

Proof of coverage and a clearance certificate indicating that the Operator is in good standing

8.8 Operator Covenants with Respect to Student Transportation

Operators shall provide a bus route data sheet including the make, model number, and year of manufacture of the serving any route

9.3 Vehicle Age Vehicles used in the provision of transportation shall confirm to maximum and average age specifications as of August 1st in each year

10.2 Copy of the Driver Training Manual

A current driver training manual is to be provided to the Consortium

11.4 Service and Safety Surveys and Programs

The Operator‟s current safety program will be made available to the Consortium

Review of Operator Records

5.1 Inadmissibility of Charter Runs

Charter runs or other fares are not permitted to prejudice the operation of any regular route

5.6 No Assignment of Subcontracting Without Consent

Prior consent is required in writing

Page | 34


Yes No

8.6 One Driver – One Route and Route Doubling

Unless specifically authorized in writing by the Consortium, each route will be served by the same driver each day; For permanent changes or changes with a duration likely to extend greater than two weeks, written notice must be provided

the Operator shall not, without prior notification to the Consortium and without specific prior acceptance of such notice, use Route Doubling

8.9 Obligation to Perform Trial Runs Prior to the Start of the School Year

Operator shall ensure that all drivers perform trial runs of their route(s)

(Consortium shall determine the requirement for trial runs)

9.1 Licensing and Equipment

All vehicles must comply with current federal, provincial, and municipal laws and regulations

9.4 Licensing in Accordance with Acts, Regulations, and Standards

The Operator shall use a motor vehicle properly licensed as required

9.7 Ministry of Transport Annual Inspection Certificate

Ministry of Transport Annual Inspection Certificate must be provided

9.8 Maintaining Log Books and Consortium Right to Inspect

Daily driver log books are to be available for inspection by the Consortium on request

9.12 Additional School Bus Equipment in Case of Breakdowns or Delays

The Operator will provide for a specified number of spare vehicles

Page | 35


Yes No

10.1 Driver Licensing Requirements

Each driver shall have and maintain in good standing at all times a Class „B‟ or „E‟ license and any additional license required

15.5 Operator‟s Right to Provide Services to Other Boards

Operator to provide copies of other authorized agreements

17. Surety, Bonds and/or Letters of Credit

A performance surety will be provided by the Operator

Onsite Fleet Review

9.5 Vehicle Characteristics

For mini vans and cars, passenger doors must be equipped with child-proof locks

Children 12 years of age or younger will not ride in the front seat in any vehicle equipped with front seat airbag safety devices




9.11 Compliance with the Definition of a School Bus

The Operator shall ensure that all school buses comply with the definition of a “school bus”

Page | 36


Yes No

Route Audit Records

5.2 Drivers to be Aware of and Follow the Conditions of the Agreement

It is the responsibility of the Operator to ensure that its drivers are aware of and follow the relevant and appropriate conditions of the Agreement




9.13 Display of Route Numbers

All vehicles shall display vehicle signs indicating route numbers

9.14 Vehicle Cleanliness The interior and exterior of the vehicles must be maintained to provide a clean appearance

10.4 Driver Characteristics

List of required driver characteristics

(see route audit form)

10.5 Driver Actions List of driver actions required while providing services:

(see route audit form)

Page | 37


Yes No

Consortium Records

2.6 Cancellation of Services by the Consortium as a Result of an Operator Labour Dispute


12.1 Basis for Payment Monthly invoices are to be submitted by each Operator, with required details and itemization

12.2 Fuel Computation and Fuel Adjustments

Monthly invoices are to be submitted by each Operator, with required details and itemization

13.1 Operator and Driver Actions in Event of Accidents

Operator to inform the appropriate school principal or vice-principal and transportation office at the Consortium forthwith and submit detailed written accident report within forty-eight (48) hours of the incident

Page | 38

B.3 Annual Contract Compliance Audit Report5

Bus Operator Name:

Contract Year:

Overall Determination of Compliance: Preliminary Final

Fully Compliant

Substantially Compliant

Not in Compliance

Ref Contract Title Description of Requirement

Description of Non-Compliance

Operator Response and Corrective Action

5 The Annual Contract Compliance Audit Report is also provided separately in electronic format.

Page | 39

Consortium Comments:

Signatures Preliminary Final

For the Consortium: Name:

Date:

For the Operator: Name:

Date:

Page | 40

B.4 Sample Route Audit Form6

6 The Route Audit forms and tracking templates are available separately in electronic format.

Date: Auditor:

Bus Operator: Division:

Type of Route(AM,Mid,PM): Type of Route(Reg, Special): Route/Bus Number:

Drivers Name: Hire Date:

Route Sheet Information:

Ride Time: Riders: Stops: Kilometers:

Scheduled Leave lot time: Date of last update:

Ride Along Information:

Beginning Odometer: Ending Odometer: Total Kilometers:

Pre-Trip Inspection Completed: Y/N Post-Trip Inspection Completed: Y/N

Stop Identity Stop TimeStudent

CountStop Time

Student

Count

Leave Lot

School/Lot Arrival

Total

Scoring Comments:

No. of Deviations:

Route Audit

Actual Route Scheduled Route

Comments ( Note Turnarounds, Safety Issues, etc.)

Page | 41

Date TerminalRoute

Number

Route

Type

(A,M,P)

Route

Type

(Reg, SP)

Deviation

CountComments

4/1/2012 Location 1 A1 P Reg 2 Sample 1

5/12/2012 Location 2 B2 A Sp 1 Sample 2

2 Route Audits Performed 3 Errors Counted

Error Ratio Per Audit: 1.50

Bus Operator:

Page | 42

Appendix C – Performance Measurement Reference Tools, Samples & Templates

This appendix includes a series of sample documents and forms as templates to be used in developing the consortium‟s performance measurement plan. They

include:

C.1 – Sample Performance Measure Matrix

C.2 – Sample Performance Measure Glossary

C.3 – Sample Performance Measure Tracking Summary

C.4 – Sample Operator Performance Report

Page | 43

C.1 Performance Measure Matrix

The following tables provide a comprehensive list of suggested starting performance measures. The Performance Measure Matrix identifies which of the four

categories the measure is related to, whether the measure is compliance or performance related, and which “tier” the measure should be considered for phased

implementation purposes. Tier 1 measures should be considered for initial implementation followed by Tier 2 and Tier 3 measures.

Category Measure Description Purpose Tier

1

Tier

2

Tier

3

Compliance Performance

Safety Preventable Accidents Rate of preventable accidents Measures effectiveness of operator safety and training programs

X X

Safety CVOR MTO CVOR rating Measures operator safety and operational compliance as a passenger transportation provider

X X X

Safety Safety Violations Rate of safety policy and procedure violations

Measures operator compliance with Consortium policies and procedures

X X X

Reliability Route Accuracy* Rate of route errors Determines whether operator works cooperatively with staff to ensure the accuracy of route design

X X X

Reliability Driver Turnover Rate at which permanent driver route assignments are changed

Determines whether driver compensation and workplace environment facilitate consistency and stability in service delivery

X X

Reliability Driver Absenteeism Rate at which assigned route drivers fail to report for work

Same as for Driver Turnover X X

Reliability Reported Complaints* Rate of operator complaints reported to staff

Measures effectiveness of operator management processes

X X

Reliability Vehicle Reliability Rate of vehicle breakdowns Measures effectiveness of operator fleet management policies and processes

X X

Cost Fuel Consumption* Rate of fuel consumption Tracks relative operating efficiency in a contract environment where fuel cost risk is shared between the operator and the consortium

X X

Page | 44

Category Measure Description Purpose Tier

1

Tier

2

Tier

3

Compliance Performance

Cost Route Efficiency* Rate of operator-initiated route changes resulting in efficiency improvements

Measures operator commitment to improving overall system operating efficiency

X X

Responsiveness Self-reported Complaints*

Ratio of self-reported complaints to reported complaints (see Reliability)

Determines whether operator actively works to remedy operational problems not reported to staff

X X

Responsiveness Contract Compliance Rate of contract compliance Measures effectiveness of operator management processes

X X X

Responsiveness Customer Satisfaction Relative measure of service quality

Measures system users‟ satisfaction with operators‟ services

X X

Page | 45

C.2 Performance Measure Glossary

The glossary table expands each performance measure provided in the Performance measure Matrix and describes the five steps required for implementation.

Measure Standard of Performance

Data Collection Approach

Measure Calculation Data Sources Frequency Reporting

Preventable

Accidents

Rate of preventable accidents incurred over a specified period

Operator reported data Total number of preventable accidents divided by total number of kilometers operated time 100,000 for the period specified

Vendor reported data with concurrence by cons0rtium on preventability

Accumulate data monthly with three variants possible: Monthly; school YTD; and rolling 12 months

Enter results in performance tracking spreadsheet for vendor comparison and trend analysis

CVOR As determined by the MTO Access the MTO website Rating as determined by MTO

MTO Annually in December Enter result in performance tracking spreadsheet for vendor comparison

Safety Violations Ratio of safety related policy and procedure violations to number of routes in operation

Consortium data Accumulate violations throughout the school year and divide by the number of buses operated

Consortium Annually in May Enter result in performance tracking spreadsheet for vendor comparison

Route Accuracy Operator-to-operator comparison for documented number of design errors per audited bus route.

Sampling By operator, divide the total number of errors identified by the number of audited routes

Route audit form Audit a percentage of routes for each operator between November and March

Enter results in performance tracking spreadsheet; Utilize results to correct identified errors and for annual operator performance assessment

Driver Turnover Operator-to-operator comparison for the total number of drivers employed throughout the year

Operators report total number of drivers paid in the calendar year regardless of length of time employed

By operator, divide the total number of drivers paid by the number of buses contracted

Operator reported data Annually in January Enter results in performance tracking spreadsheet; Utilize results to identify trends and collaborate with vendors to sustain and/or improve reliability

Driver Absenteeism Operator to operator comparison of absentee rates

Operators report daily absences of drivers

By operator for the desired period divide total absences by number of buses contracted times operating days in the period

Operator reported data Minimum monthly. More frequent periods for chronic problems. Develop twelve month running total.

Enter monthly result in performance spreadsheet for vendor comparison. Use more frequent periods to monitor success of corrective action.

Page | 46

Measure Standard of Performance

Data Collection Approach

Measure Calculation Data Sources Frequency Reporting

Reported Complaints Ratio of self-reported complaints to number of routes in operation

Operator reported details for each complaint received

By operator and reporting period, divide the total number of reported complaints by the number of assigned routes

Operator reported data Collect data monthly to coincide with billing cycle

Enter summary results in performance tracking spreadsheet; Utilize results to investigate unusual trends (i.e., rates too high or too low; results inconsistent with staff observation)

Vehicle Reliability Rate at which self-reported breakdowns occur

Operator reports breakdowns that disrupt or delay any service being reported

For a month the number of breakdowns divided by the total number of kilometers operated times 100,000 to achieve a rate per 100,000 kilometers

Operator reported data Monthly collection of data with three variants possible: Monthly; school YTD; rolling 12 months

Enter results in performance spreadsheet. Utilize results for performance comparison among vendors and to identify trends.

Fuel Consumption Trend analysis for each operator for average fleet fuel consumption

Operator reported total fuel used and total route kilometers

By operator and reporting period, divide the total route kilometers by total fuel used

Operator reported data Collect data monthly to coincide with billing cycle

Enter summary results in performance tracking spreadsheet; Utilize results to investigate adverse trends

Route Efficiency Rate that vendor collaborates with consortium to improve efficiency

Operator reported route changes that result in shorter or fewer routes

Collect and identify vendor initiated route changes divided by the total number of routes operated

Operator reported data Accumulate changes from beginning of school year through March. Calculate rate in April.

Enter result in performance spreadsheet. Utilize results for vendor comparison and evaluation of consortium routing personnel performance.

Self-Reported

Complaints

Ratio of self-reported complaints to total complaints received from all sources on a vendor

Operator reported data plus reports from all other sources

Vendor reported complaints (received prior to other sources) divided vendor reported complaints + complaints from all other sources; maximum rate goal = 100%

Operator reported data; principal, parental, and patron reported data

Accumulate data monthly with three variants possible: Monthly; school YTD; and rolling 12 months

Enter results in performance tracking spreadsheet for vendor comparison and trend analysis

Contract Compliance Rate of achieving administrative contract compliance on time

Operator supplied requirements

Comparison of operator submittals to contract deadlines pass/fail; i.e., Items on time divided by total number of items

Operator submittals and contract

Rate calculated annually in July

Enter rate in performance tracking spreadsheet for vendor comparison

Customer

Satisfaction

Rate at which satisfied or above is achieved in annual principal survey

Survey Total number of satisfactory or higher ratings divided by the number of items rated

Principal survey responses Survey conducted in February and rate calculated in March

Enter rate in performance tracking spreadsheet. Measure year-over-year improvement and compare vendors.

Page | 47

C.3 Sample Performance Measure Tracking Summary7

C.4 Sample Operator Performance Report

7 The performance measure tracking template and operator performance report are available separately in electronic format.

10 Monday January 10 100 300 3 107 0 0 2 2 3519 10349 1 1 1 1 4 8 11 1 1 929 6431 100,000

11 Tuesday January 11 100 300 0 106 1 0 3 2 3983 11715 1 3 1 3 6 2 4 1 0 1052 7279

12 Wednesday January 12 100 300 2 110 0 0 1 2 3923 11539 1 3 1 5 6 8 11 1 1 1036 7170

13 Thursday January 13 100 300 0 114 1 1 3 1 3975 11691 1 1 0 5 7 12 15 0 0 1050 7264

14 Friday January 14 100 300 2 100 0 0 0 1 4001 11769 0 3 0 2 5 8 13 1 0 1057 7313

17 Monday January 17 100 300 1 104 1 1 0 1 3500 10294 1 1 0 4 6 5 10 0 0 924 6396

18 Tuesday January 18 100 300 1 115 0 0 1 1 3582 10535 1 2 0 0 0 2 5 0 1 946 6546



21 Friday January 21 100 300 0 106 1 1 2 2 3806 11194 0 0 2 5 5 3 5 1 1 1005 6956

24 Monday January 24 100 300 2 107 0 0 2 1 4060 11942 1 1 0 1 4 0 2 1 1 1072 7420

25 Tuesday January 25 100 300 1 114 0 0 2 1 3454 10158 1 1 2 1 1 10 13 0 0 912 6312



28 Friday January 28 100 300 1 108 0 0 3 2 3688 10846 1 3 1 1 4 13 16 0 1 974 6739

31 Monday January 31 100 300 3 110 0 1 0 2 3820 11235 0 0 1 0 0 12 17 0 0 1009 6981

32 Tuesday February 1 100 300 3 112 1 0 1 2 3434 10100 0 1 0 0 0 12 16 1 1 907 6276

33 Wednesday February 2 100 300 3 100 0 0 2 0 3719 10939 0 2 0 4 7 2 7 1 0 982 6797

34 Thursday February 3 100 300 0 114 0 0 2 2 4013 11802 1 0 1 3 3 14 19 0 1 1060 7333

Buses2011Julian

Day

Day of

the WeekMonth Day

V.I . Route

ChangesRoutes

Route

Errors

Drivers

Paid

Drivers

Added

Drivers

LostAbsences Breakdowns

Fuel

Consumed

Kilometers

Traveled

Preventable

Accidents

Safety

ViolationsCVOR Fuel in

Litres

KM

Traveled

Measures/

Figures

V.R.

Complaints

Total

Complaints

Satisfactory

or Higher

Items

Rated

Successful

Submittals

Unsuccessful

Submittals

OperatorRoute

Accuracy

Driver

Turnover

Driver

Absenteeism

Reported

Complaints

Vehicle

Reliability

Avg.

km/l

Fuel Con.

Total (liters)

Route

Efficeincy

Vendor

Reported

Complaints

Contract

Compliance

Customer

Satisfaction

Preventable

Accidents

Safety

ViolationsCVOR

Alpha 54.01% 201.00% 1.58% 1.73 0.0815 2.94 777420 70.33% 61.64% 53.05% 69.28% 4.33 2.94

Bravo 46.63% 202.00% 1.55% 1.63 0.0992 2.93 780312 67.67% 60.27% 48.31% 68.22% 4.16 2.98

Charlie 51.12% 203.00% 1.46% 1.76 0.0819 2.94 774363 73.00% 63.33% 52.53% 67.21% 4.70 3.02

Date post:	11-Apr-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

School Bus Operator Contract Performance …...School Bus Operator Contract Performance Management...

Documents