Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Page 2 of 91
Version 4.2, November 2017
Document Control
VERSION NO.
AMENDMENT NO.
DESCRIPTION DATE UPDATED BY
EFFECTIVE DATE
1 0 NOC version 1 Value Management Proposition
Jack Hansby
2 0 • Road User Safety and Health and
Safety KRAs separated • Revised KRA measures
08/09/14 David Kelsey
01/07/14
3 0
• Annual performance assessment business rules
• New methodology to calculate score
• RNPG moderation of results • Reporting timetable
• Reporting templates • Minor wording revision for
assessment criteria
10/05/15 David Kelsey
01/07/15
4 0
• Reorganisation and updates across whole document for consistent wording and plain language, for easier interpretation and application
• Clarifications to all sections, to improve consistency of assessments
28/07/16 Emma Harris
01/07/16
4.1 0
• Minor clarifications and corrections to improve consistency of assessments
• Reformatted diagrams that previously did not print correctly
• Embedding of content of framework clarifications issued after Version 4
• Merged content from Section 2
(Stages of using the framework) and Section 4.1 (Report Structure) into Section 4.4 (Report Types) and Section 4.5 (Moderation of Reported Performance)
• Minor changes to align ‘Data required’
Sections throughout Section 8 Individual KPI Details to Reporting Template
28/11/17 Penny Marriott
01/11/17
4.2 0 • Correction of typo in Measure Score
table of KPI Measure 2.1.1 DSI Trend (p43)
30/11/17 Penny Marriott
01/11/17
Page 3 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Contents
Document Control
Contents
About this Guideline
Overview of the Framework
2.1 Purpose of Framework
2.2 Performance Areas Assessed
2.3 Applying the Annual Performance Assessment Result
Implementing the Framework for a Contract
3.1 Inputs
3.2 Approach
3.3 Outputs
Reporting
4.1 Grace Period before First Reports
4.2 Report Format Decided Jointly
4.3 Reporting Templates
4.4 Report Types
4.5 Moderation of Reported Performance
4.6 Timing of KRA Periods
KRA Structure
5.1 Weighting of Key Result Areas
5.2 KRA Framework Hierarchy
Scoring Approach
6.1 Sources of Measurement Data
6.2 Ranking Performance
6.3 Business Rules for Performance Achieved
6.4 Example of Scoring Calculations
Measuring and Scoring KPIs
7.1 Scores for Performance Rankings
7.2 How the KPIs are presented
7.3 KPI Measurement Types
7.4 Completing Audits to Measure Performance
Page 4 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
7.5 KPIs that Measure Performance of Plans
7.6 ‘Planned’ and ‘Unplanned’ Events
7.7 Trend Line Calculations
Individual KPI Details
KRA 1: Health and Safety
KRA 2: Road User Safety
KRA 3: Customer
KRA 4: Sustainability
KRA 5: Assurance and Value
KRA 6: Network Performance
KRA 7: Health of the Relationship
Appendices
Appendix 1: Performance Assessment Timetable
Appendix 2: Reporting Template
Appendix 3: Scoring Template
Page 5 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
About this Guideline
This section of the guideline describes this document and how to use it.
Purpose
New Zealand Transport Agency (NZTA) has developed this guideline document.
It supports the framework for assessing Contractors’ performance against the key
performance indicators (KPIs) in Network Outcome Contracts (NOC) for maintaining the
New Zealand network. Using the framework described in this document, all the parties to
these Contracts can measure Contractors’ performance consistently and accurately.
Audience
This guideline is mainly intended for:
Contractors who hold a NOC, when assessing and reporting their performance levels
Network managers and NZTA Maintenance Contract Managers (MCMs), when
assessing performance against key result areas (KRAs) to find overall performance
levels for four-monthly KRA periods and annually
Members of all other groups involved in assessing and endorsing KRAs.
How to use this guideline
The early sections of this document give an overview of the framework for assessing
performance. Users should refresh their understanding of the information in these
sections, as needed, at each KRA period and annual assessment. In particular,
Contractors should study this information carefully at the start of a new Contract, and
whenever it is updated.
The later sections provide the practical tools and specific details needed when completing
each assessment.
Note: This guideline is designed to stand alone, i.e. it should be the only document that
users need, to complete assessments. However, the NOC document always has legal
precedence, and may prove useful to refer to.
Terms used
In general, this guideline uses the same terms as the Network Outcomes Contract
(NOC), with the same meaning, such as ‘Contractor’, ‘Principal’ and ‘Network’.
Page 6 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
In addition, the following table explains some further terms and the most common
abbreviations in this guideline.
Important: These descriptions are provided only for helpful context when using the
information in this document. See the NOC for definitions that hold contractual force.
THE TERM… MEANS…
Assessment year
The period from 1 July to 30 June of the following year, over which a Contractor’s performance under the NOC is measured and ranked.
CB Contract Board, made up of representatives from both the Principal and the Contractor.
CMT Contract Management Team, made up of representatives from both the Principal and the Contractor.
KPI Key performance indicator, each of which is an element of a specific KRA.
KPI score The performance ranking measured for a KPI over a KRA period. See Section 7
‘Measuring and Scoring KPIs’ and Section 8 ‘Individual KPI Details’ for full details.
KRA Key result area.
KRA level
The performance ranking measured for a KRA:
• The score to achieve a KRA level for a KRA period is calculated from KPI scores. • The Annual KRA level is calculated from the KRA scores for each KRA period in the
reporting year.
See Section 6 ‘Scoring Approach’ for full details.
KRA period The four-monthly (tri-annual) period at the end of which the Contractor’s performance is measured against KRAs.
MCM Maintenance Contract Manager, a representative of the Principal.
MCOS Minimum condition of satisfaction – performance ranking of Level 2. See Section 6.2 ‘Ranking Performance’ for more details.
Mitigating circumstances
Mitigating circumstances that could be accepted are:
• Client supplied resources (example includes supply of Melter Aggregate) • Client agreed scope change • Unforeseen ground conditions • Extraordinary events (must be accepted unanimously by the Contract Board), an
example includes a major CIMS weather event with a response level of local or above.
NZTA The New Zealand Transport Agency.
OPM Operational performance measure, a separate (but relevant) measurement framework.
Overall annual
result
The performance ranking measured for an assessment year, calculated from the scores
for the KRA levels achieved. See Section 6 ‘Scoring Approach’ for full details.
PPT Planning & Performance Team, made up of NZTA representatives.
Road user A key ‘customer’, a user of the network (e.g. someone who travels on a state highway).
Page 7 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Overview of the Framework
This section of the guideline describes the process and general approach for how the
KRA framework is used to manage performance under each NOC.
2.1 PURPOSE OF FRAMEWORK
The key result area (KRA) performance framework is the core of the system that the
Principal uses to:
Measure each Contractor’s performance against their Network Outcomes Contract
(NOC)
Align Contractors’ expected contract outcomes with the Principal’s strategic
objectives
Keep Contractors’ quality, performance and relationship strategies firmly in line with
the Principal’s responsibilities to align with government direction.
2.1.1 Managing Performance Quality
Using this KRA framework provides the parties with a contractual way to work together,
to assess the Contractor’s actual performance in their activities under their tendered
submission for the NOC. The framework measures this against the desired outcomes,
features, or behaviours within the range of specific KPIs. When the Principal has
validated the results:
IF PERFORMANCE MEASURES… THEN THE KRA SYSTEM PROVIDES…
Too low in any area A structured environment in which the parties can, together, address the relevant issues, raising the Contractor’s level of achievement.
Acceptably Encouragement and support for the Contractor to improve further, achieving even better results.
Well
Encouragement and support (as above), plus specific rewards in the form of:
• Financial benefits • Additional contract tenure.
To achieve this objective:
Each KRA defines the:
Data to record and report, to enable the assessment process
Clear, objective scoring criteria for achieving each performance level, using a
mix of qualitative and quantitative measures, and lead and lag indicators where
applicable
The Principal moderates each self-assessment to ensure the outcomes are applied
fairly and consistently to all NOCs.
Page 8 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
2.1.2 Managing Strategic Alignment
Using this framework facilitates a positive culture between the contract parties, which
will offer value for money in the Principal(s)’ use of public funds. These features of the
framework support achievement of the Principal’s strategic objectives for NOCs:
FEATURE HOW THE FRAMEWORK SUPPORTS STRATEGY
Cooperation Each aspect of the KRA framework involves the Contractor and Principal actively and proactively collaborating about performance:
• The parties are expected to be open and honest in all their interactions, and the processes support this.
• The processes make the information transparent and visible to all parties.
• All parties benefit equally from this mutual engagement.
Efficiency • Achievement within each KRA provides the NZ public with significant
value for money. • The framework applies consistently to all Contractors nationwide, and
provides a single source of information. • The processes and results are trusted and reliable. • The guidance is specific, simple and clear. • Contractors are rewarded for consistently achieving well. • The KRAs are structured into a meaningful hierarchy with:
o Clear definitions of the inputs and outputs o Quantitative and qualitative measurement criteria.
Improvement • Successes are made repeatable - the framework rewards Contractors who share their positive ideas.
• The framework is a flexible, ‘living’ system that the Principal can adjust for developments in practices, technology, policy, etc.
• The approach lets the parties see which activities are proving
effective, and whether they have the right people in the right areas at the right times for the right reasons, to achieve the best results.
• The Principal can use the assessment results to benchmark performance, supporting Contractors to adopt learnings and improved practices, which raises the overall standard in all regions.
• Adopting these principles fosters a culture of stewardship, in which all parties take pride, and in which the Contractor takes ownership and
accountability for the quality of the services delivered.
To support the benefits of these features, the Principal publishes an annual national
report that compares how well each Contractor has achieved the various performance
measures for each Contract.
Page 9 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
2.2 PERFORMANCE AREAS ASSESSED
Using this framework, the Contractor measures performance in the following KRAs:
1 Health and Safety
2 Road User Safety
3 Customer
4 Sustainability
5 Value and Assurance
6 Network Performance
7 Health of the Relationship.
Note: For KRA 7, Health of the Relationship, the Principal assesses the effectiveness of
the culture between the contract partners, in working together to deliver the contract
outcomes. However, this is not currently scored.
See Section 5 ‘KRA Structure’ for more details about the measures.
Performance Areas Not Assessed
This framework does not cover the NOC’s at-risk payment mechanism for compliance
with operational performance measures (OPMs). (See KPI 6.1.1 Overall OPM Score for
details on how OPM scores relate to KRA assessments.)
Page 10 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
2.3 APPLYING THE ANNUAL PERFORMANCE
ASSESSMENT RESULT
For each assessment year, the Contractor’s annual performance result for the contract
year is calculated, validated and endorsed. See:
Section 6 ‘Scoring Approach’, for how it is calculated
The first parts of this Section (2), for how it is validated and endorsed.
This final annual performance result determines what contract entitlement the Contractor
is due1, as the following table shows.
Table 1: Performance Result and Rewards
RESULT DESCRIPTION OF PERFORMANCE OUTCOME(S)
1 Poor Performance shows significant failures, and there are serious gaps in service delivery.
Loss of Tenure (when applicable)
2 Minimum Condition of Satisfaction
The Contractor is performing to an adequate standard, improvement is required.
Additional tenure (when applicable)
3 Best Practice
The results reflect consistent performance achievement, underpinned by constructive behaviours that enhance the relationship and
deliver innovative solutions for both parties.
• 50% of KRA Financial Reward
• Additional tenure
(when applicable)
4 Outstanding
The Contractor is exceeding the Principal’s expectations for performance, and has consistently delivered outstanding results.
• 100% of KRA Financial Reward
• Additional tenure (when applicable)
2.3.1 Adjustments for Part Years
Each Contract has a different start and end date. These may not match the four-monthly
reporting cycles or the annual assessment cycle. When assessing the outcomes granted
for the annual result in the first or last year of a Contract. Financial rewards are based
on the number of calendar months the NOC was active during the KRA assessment year.
Tenure outcomes do not apply in the first year (or part-year) of reporting.
Example: A Contract starts on 1 October 2017, and a grace period of five months is
agreed, i.e. to the end of February 2018. The Contractor submits their first annual result
in by the applicable deadline in August 2018. The financial impact is proportional to the
time the Contract has been running. In this case, the Contract has run for nine months
of the KRA assessment year, so the rewards are adjusted by 9/12. If this Contractor’s
annual result made them eligible for a financial reward of $100,000, this would be
adjusted to $75,000. Additional tenure would not be awarded.
See Section 4.6 ‘Timing of KRA Periods’ for more details on the assessment year cycle.
within the limitations specified in the Conditions of Contract (Part B clause 10.8 Contract Period
and Adjustments)
Page 11 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Implementing the Framework for a
Contract
To achieve the objectives of this framework (see Section 2 ‘Overview of the
Framework’), at the start of each NOC the Contractor must establish systems and
processes that they will implement, to manage their performance in each area covered
by the NOC.
This section of the guideline describes how the Contractor and Principal work together to
implement the KRA performance framework for the Contract.
3.1 INPUTS
The Contractor designates one or more KPI champions, whose role is to coordinate
with the Principal to ensure the KRA performance framework is implemented.
The Principal maintains the KRA Performance Framework Guideline (this document),
processes and templates, and provides these to the Contractor to support them with
their KRA assessments and score calculations.
3.2 APPROACH
The Contractor establishes systems for measuring and monitoring their performance
for each KPI’s measurement, and documents this in their quality management plan
(QMP).
The Principal evaluates the Contractor’s implementation of the framework, to
confirm it is fit for purpose, and will lead to continual improvement. They will look
for the following features:
The implementation is flexible, so the Contractor can learn from each
performance assessment cycle
The implementation takes a structured approach to change management,
education and training, creating opportunities for collective learning about the
application of the framework and providing measurable learning outcomes for
training
The implementation is completed before the Contract commences.
Exception: Certain activities can be implemented after commencement, within
the grace periods specified in the NOC Conditions of Contract. Specifically, this
relates to establishing contract management plans. (See Section 7.5 ‘KPIs that
Measure Performance of Plans’ for more details.)
The Principal provides the Contractor with support to ensure that, using their
implementation of the framework, their ongoing measurement of KPIs will be
consistent with the national standard.
Page 12 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
3.3 OUTPUTS
The Contractor has set up systems and processes that:
Provide the evidence needed to support their KRA assessments
Comply with their quality plan
Are robust enough that the Principal or an independent assessor can evaluate them.
Page 13 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Reporting
This section of the guideline describes the types of reports submitted under this
framework, and how each is used.
4.1 GRACE PERIOD BEFORE FIRST REPORTS
Within each report type, the Contractor assesses their KRA performance at the required
intervals.
When a new Contract starts, however, the Contractor has a grace period of between
three and six months (depending on the Contract’s commencement date) before they
need to report their first assessment. This gives the Contractor time to implement a
system to score their performance.
This grace period lets the Principal and the Contractor work together to:
Ensure the Contractor’s systems and processes support how they collect and verify
the data and information they need, to evaluate their performance effectively
Implement the systems and processes within the time allowed.
4.2 REPORT FORMAT DECIDED JOINTLY
The Principal and Contractor jointly develop a format for their monthly reports that:
Is consistent with the national framework
Makes the information visible to all parties
Provides a clear assessment of the Contractor’s performance over the KRA period(s).
This report provides many of the key details the Contractor will refer to when they are
self-assessing their KPI performance scores.
4.3 REPORTING TEMPLATES
The Principal provides Contractors with templates to use for each four-monthly report.
These templates makes it easier to:
Create the consolidated report
Make the results transparent and visible
Apply consistency to the performance assessments.
The Contractor must complete the following templates in their reporting for each KRA
period:
Page 14 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Use the Reporting Template to provide KPI and KRA scores in the report for each
KRA period assessment, and offer summary information to support each score. Email
[email protected] to request the current admissable2 template. See
Appendix 2 ‘Reporting Template’ (page 90) for an example of this template
Use the Score Calculation Template to list the detailed KPI scores and KRA
performance levels for each KRA period assessment, and – calculated from those –
the indicative annual KRA and overall performance levels. See Appendix 3 ‘Scoring
Template’ (page 91) for the layout of this template, which is available upon request
from [email protected]
4.4 REPORT TYPES
The Contractor needs to compile a number of performance reports during the KRA
assessment year.
See Appendix 1 ‘Performance Assessment Timetable’ (page 89) for the timetable of the
reports required in the current annual performance cycle.
4.4.1 Monthly Contract Report
The Contractor compiles a monthly report3 that includes a range of data used to measure
KPI scores. The CMT tracks this information ongoing so that, if needed, they can work
with the Contractor to improve the rankings before it affects the assessment of their
performance levels at the end of the KRA period.
4.4.2 Four-Monthly Contract Board Report
The Contractor, with input from the CMT, compiles a report for their Contract Board (CB)
at the end of each four-monthly KRA period. This report details all KPI scores and KRA
performance levels, and the rationale for how the Contractor arrived at each result. The
Contractor will draw on the CMT for feedback about their report to ensure information
provided supports self-assessed KPI scores and KRA levels. Note: Irrespective of the
scheduled frequency of Contract Board meetings, the four-monthly KRA report must still
be endorsed by the Contract Board.
The Contractor can include additional information in their CB report, on any mitigating
circumstances4 that has constrained the Contractor’s performance level. However, the
performance level must still be based on the KPI scores as measured, even if the
Contractor provides such additional information.
This report should include a summary of the KPI scores on the NZTA scoring template
(see 4.3 ‘Reporting Templates’).
2 Outdated versions cannot be uploaded. 3 The requirements for this report are detailed in section 3.8.4 of the NOC.
4 A description of mitigating circumstances can be seen in the table of terms used (page 6).
Page 15 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
The CB will consider the report and provide feedback about the scoring, including
moderating the results if needed, and agreeing mitigating circumstances where
appropriate, before endorsing the report.
Any Board endorsed KPI score, that is not reflective of the KPI measure score table and
metrics provided, can then be entered into the template; however none of the
underlying measurement values are to be altered.
Once any changes required by the CB have been made, and the CMT and CB have
endorsed this report, the Performance Team collates the KPI scores, ready for the
moderation process.
Note: See Appendix 2: Reporting Template (page 90) and Appendix 3 ‘Scoring Template’
(page 91) for the templates used. These can be requested by emailing
4.4.3 Annual Performance Report
At the end of each annual performance cycle (i.e. in July each year), the Contractor
compiles their Annual Contract Performance Report. This report includes the annual
summary of the KPI scores by period, and the annual result, on the NZTA scoring
template (see 4.3 ‘Reporting Templates’).
This key report summarises the full year’s performance. The report includes:
The scores assessed for each KPI, and the KRA levels and annual result, calculated
from those
The rationale for how the Contractor arrived at this result, including:
An explanation of the result
Supporting evidence that the Contractor has collected for each measure
Details of any financial reward or tenure adjustment due.
The Annual Performance Report requires consultation with the CMT and endorsement
from the CB in exactly the same way as the Four-monthly Contract Board Report.
Note: See Appendix 3 ‘Scoring Template’ (page 91) for the template used to record and
report this information.
4.5 MODERATION OF REPORTED PERFORMANCE
In preparation for moderation workshops, the Planning & Performance Team (PPT)
consider collective reports from all Contractors. If needed, the PPT provide feedback to
the Contractor and MCM about issues with scores or supporting information, e.g. request
more details.
After each KRA period ends, an NZTA moderation team meets, consisting of
representatives from the Planning & Performance and Procurement teams, and
Maintenance Contract Managers (MCMs). In these meetings, the group considers and
validates each Contractor’s self-assessed results within the context of the framework’s
Page 16 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
intent, and compared with all Contractors; moderates them if required, and recommends
the final scores for approval.
In line with Agency delegations, the relevant approver(s) then considers and approves
the collective moderated responses, tenure and financial rewards.
Trends developing across the overall network, and any potential impact on the Principal’s
strategic objectives are identified, as well as confirming consistency of the KRA
framework.
Contractors are advised about any changes to KPI scores and (as a consequence) KRA
levels, e.g.:
Upgrade e.g. from 3 to 4 because performance is in the top percentage group for
the relevant KPI.
Downgrade e.g. from 4 to 3 because scoring measures were applied incorrectly.
The final KPI scores and KRA levels for each KRA period are confirmed to the MCM.
The KRA framework is reviewed and updated as needed, to clarify guidance and ensure
measures and scores accurately reflect levels of Contractor behaviour, e.g.:
Scores of 1 genuinely reflect poor standards of performance.
Scores of 4 genuinely reflect outstanding performance.
See Appendix 1 ‘Performance Assessment Timetable’ on page 89, for the dates of
moderation meetings.
4.5.1 How Moderation is Conducted
The moderation activity focuses solely on checking for consistency between:
How each Contractor is applying the framework to self-assess their KRAs
performance levels
The supporting evidence each Contractor is providing in their self-assessment
reports
The proposed financial rewards or tenure adjustment to be awarded for each
Contract, for the annual performance assessment.
If this process identifies any areas of inconsistency, the Principal will then:
Work with Contractors to correct the inconsistencies during the next KRA period, i.e.
before the Contractor completes their next report
Consider updating this guideline to state the requirements more explicitly for all
Contractors.
The Principal also retains the right to moderate any score, level or result, if it is aware of
issues that are not reflected in the Contractor’s supporting information – for example, if
a Contractor was involved in a major network event reported over national media, but
that event was not included in the audit sample the Contractor used to calculate the
relevant KPI score.
Page 17 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
4.5.2 Moderation Audits
In addition to the formal moderation meetings, the Principal may audit the results of a
Contractor’s self-audits, to validate their self-assessed KPI scores.
4.6 TIMING OF KRA PERIODS
Contractors assess their performance after the end of each four-month KRA period. In
general, a KRA period is either:
Called ‘P1’ (KRA period 1), 'P2' (KRA period 2), ‘P3’ (KRA period 3), or
Referred to by the final calendar month of the period.
The diagram below shows how periods are spread across calendar months.
Note: These KRA periods are consistent and unchanging, across all Contractors and
assessment years, even if they do not match with (for example) the Contractor’s
financial year dates or the date the Contract began.
Page 18 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KRA Structure
This section of the guideline summarises the KRAs within this performance framework.
5.1 WEIGHTING OF KEY RESULT AREAS
Each of the first six key result areas (KRAs) has equal weight for determining the overall
annual performance level and, from that, determining tenure and financial reward. This
may change if the Principal determines that particular business priorities need a higher
level of focus, e.g. because strategic goals have moved.
Table 2: KRA Weighting
KRA WEIGHTING DESIRED BEHAVIOUR
1 Health and Safety
1 Safety is of paramount importance on the journey to zero harm.
Activities under NOCs show commitment to health and safety
outcomes.
2 Road User Safety
1 Activities under NOCs:
• Provide opportunities to improve journeys for road users. • Contribute to safer journeys through safe roads, speeds
and vehicles, and safe road use.
3 Customer 1 Contractors always:
• Consider road users’ needs for access. • Respond to customers’ requests and their expectations.
4 Sustainability 1 • Contractors contribute to maintaining a sustainable and
engaged contracting market. • Contractors adopt good practice and act in a responsible
manner, which contributes to a transport system that adds positively to New Zealand’s economic, social and environmental welfare.
5 Assurance and Value
1 • Accurate network information and knowledge underpin quality outcomes.
• Contractors make sound investment recommendations and decisions, based on reliable, robust and proven evidence.
6 Network Performance
1 • Activities completed under a NOC provide physical indicators of service quality.
• Contractors deliver on promises made during tendering, and these add value to the network.
• Customers have timely and accurate information that lets them make informed choices.
• Contractors schedule their work to cause minimal disruption for road users.
7 Health of the Relationship
0 The working relationship between the Principal and Contractors:
• Fosters open and honest dialogue and feedback • Involves Sub-Contractors and recognises their value.
TOTAL 6
Page 19 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Special Approach for KRA 7: Health of the Relationship
The Principal surveys the health of the relationship with the Contractor every six months,
and annually with key Sub-Contractors. The surveys involve staff from the:
Principal
Contractor
Any Sub-Contractors.
This survey measures the Contract’s culture, i.e. how well the parties are working
together. This guideline describes the KPIs used to measure performance in the relevant
areas; however:
The assessments are not expressed as scores
This KRA is excluded from the calculations to determine the overall performance
level.
The Contract Management Team discuss the results with the Contract Board. The survey
results are used to support continuous improvement and behavioural change that
delivers successful Contract outcomes.
Page 20 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
5.2 KRA FRAMEWORK HIERARCHY
The diagram below shows the titles and structure of the KPIs and KRAs within the
framework.
KRA Performance Management
KRA 1 Health and Safety
KRA 2 Road User Safety
KRA 3 Customer
KRA 4 Sustainability
KPI 1.1.1 Total Reportable Injury Frequency Rate
KPI 1.1.2 Health and Safety Management Plan
KPI 1.2.1 Traffic Control Plan
KPI 2.1.1 DSI Trend
KPI 2.1.2 Loss of Control in Darkness
KPI 2.1.3 Network Safety Trend Report
KPI 3.1.1 Customer and Stakeholder Communications Management Plan
KPI 3.1.2 Communication of Impact of Events and Incidents on Customers
KPI 3.1.3 Customer Satisfaction Survey
KPI 3.2.1 Ability to Respond to Customers
KPI 4.1.1 Enforcement Action
KPI 4.1.2 Regulatory Non Compliance
KPI 4.1.3 CS-VUE Non Compliance
KPI 4.1.4 Environmental Initiatives Adopted or Actioned
KPI 4.1.5 Environmental Training, Workshops and Presentations
KPI 4.1.6 Environmental Hazards or Opportunities
KPI 4.1.7 Environmental and Social Management Plan
KPI 4.2.1 Healthy Market Pledges
KRA 5 Assurance and Value
KPI 5.1.1 Quality Management Plan
KPI 5.2.1 Maintenance Management Plan
KPI 5.3.1 Innovation
KRA 6 Network Performance
KPI 6.1.1 Overall OPM Score
KPI 6.1.2 Compliance with Contractor’s Monthly Programme of Work
KPI 6.1.3 Network Performance Tender Pledges are Delivered
KPI 6.2.1 Network Availability
KPI 6.2.2 Emergency Procedures Preparedness Plan
KPI 6.2.3 Traffic Flow
Environmental
Triangle
KRA 7 Health of the Relationship
KPI 7.1.1 Relationship Survey
KPI 7.1.2 Integrity and Compliance of KRA System
(assessed differently and not scored)
Page 21 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Scoring Approach
This section of the guideline describes how to determine the performance levels achieved
at each assessment.
6.1 SOURCES OF MEASUREMENT DATA
Each KRA is made up of a group of KPI measures. The assessed KRA level demonstrates
the standard of performance that the Contractor has achieved in that specific area. The
measures are evaluated against agreed criteria such as:
The Contractor’s own records
Network data
Asset condition measures
Customer responses
Stakeholder survey results.
6.2 RANKING PERFORMANCE
The table below shows the four performance rankings, used for the KRA performance
levels (for each KRA period) and the annual performance result. The ranges refer to the
average of the relevant KPI scores – see below in this section for more details.
Table 3: Measure Scoring Table
PERFORMANCE LEVEL SCORE RANGE
1 Poor 1 to less than 2 (<2)
2 Minimum Condition of Satisfaction (MCOS) 2 to less than 2.66 (≥2 and <2.66)
3 Best Practice 2.66 to less than 3.33 (≥2.66 and <3.33)
4 Outstanding 3.33 or more (≥3.33)
Important: See Section 6.3 ‘Business Rules for Performance Achieved' for
important adjustments that may apply, before finalising the above rankings.
Also see Section 2.3 ‘Applying the Annual Performance Assessment Result’ for more
details about how the rankings affect the ongoing Contract.
See the descriptions and examples following in this section, for more details about
how to calculate the scores at each assessment.
Page 22 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
6.2.1 Overview of Calculating Rankings
The overall performance level, and the KRA levels for each four-monthly period and
annually, are based on the averaged KPI scores assessed for each KRA period. This
diagram illustrates the series of calculations – see below for more details.
See Section 6.4 ‘Example of Scoring Calculations’ for a more detailed diagram.
Important: See Section 6.3 ‘Business Rules for Performance Achieved’ for some
important exceptions that are applied before finalising the overall rankings.
The scores to achieve the individual four-monthly KRA levels are also used to
calculate the overall KRA level for the KRA period, but the latter score is not involved
in calculating any other rankings.
6.2.2 Calculating KPI Scores
At each KRA period assessment, use the KPI measures in Section 7 ‘Measuring and
Scoring KPIs’ to find the applicable score for each KPI.
Note: KPI scoring uses the same four-level rankings as the scores to achieve KRAs, e.g.
a KPI score of ‘3’ indicates performance is consistent with ‘Best Practice’.
6.2.3 Calculating Four-Monthly Individual KRA Levels
Once the four-monthly KPI measurements are completed, for each KRA:
Calculate the unrounded average of the KPI scores within the KRA
Look up the resulting score to find the performance level on Table 3: Measure
Scoring Table (see page 21).
The result is the four-monthly performance level for the KRA.
Page 23 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Note: The Scoring template calculates these levels automatically. See Appendix 3
‘Scoring Template’ (page 91).
6.2.4 Calculating Four-Monthly Overall KRA Level
Once the four-monthly KPI measurements are completed:
Calculate the unrounded average of the scores for the KRAs levels achieved
Look up the resulting score to find the performance level on Table 3: Measure
Scoring Table (see page 21)
Apply the business rules in Section 6.3 ‘Business Rules for Performance Achieved’.
The result is the four-monthly performance level for the KRA.
Note: While the business rules are applied here, this level is only used to give an
indication of the current outlook for the annual result.
6.2.5 Calculating Annual KRA Levels
At each annual assessment, for each KRA:
Calculate the unrounded average of the KRA levels, for the three KRA periods in
the annual assessment cycle
Look up the resulting score to find the performance level on Table 3: Measure
Scoring Table (see page 21), for that annual KRA result.
The result is the annual performance level for the KRA.
Note: The Scoring template calculates these levels automatically. See Appendix 3
‘Scoring Template’ (page 91).
6.2.6 Calculating Overall Annual Performance Result
Once the annual KRA performance levels are assessed:
Calculate the unrounded average of the annual KRA levels for KRAs 1 to 6 (only)
Look up the resulting score to find the performance level on Table 3: Measure
Scoring Table (see page 21), for the annual level of all KRAs
Apply the business rules in Section 6.3 ‘Business Rules for Performance Achieved’.
The result is the overall annual performance result.
Note: The Scoring template calculates this result automatically. See Appendix 3 ‘Scoring
Template’ (page 91).
Page 24 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
6.3 BUSINESS RULES FOR PERFORMANCE
ACHIEVED
Before reporting, apply the following business rules when calculating the overall KRA
level for a period, and the annual KRA result (as described in Sections 6.2.4 and 6.2.6):
1 If any two or more KRAs levels in the calculation are Level 1 (Poor) then the
performance ranking is 1 (Poor).
2 If any one KRA level in the calculation is Level 1 (Poor) then the performance
ranking can be no higher than Level 2 (Minimum Condition of Satisfaction).
3 If neither rule 1 nor rule 2 applies, then the performance level calculated using
Section 6.2 ‘Ranking Performance’ applies.
The annual performance level determines the relevant Contract rewards granted to the
Contractor, as shown on Table 1: Performance Result and Rewards (see page 10). For
the specific effects on Contract tenure, see the Conditions of Contract (Part B clause 10.8
Contract Period and Adjustments).
Page 25 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Average Average Average
Average
Final
Stage Use measures in
Section 8 to score each KPI
Stage Average scores for
KPIs within each KRA to find
individual KRA level
Stage Average individual KRA
levels across all KRA periods to
find annual KRA level
Stage Average annual
individual KRA levels to find
overall annual performance result
Each KRA period
Each KRA period
Annual assessment Annual assessment
Average
Stage Average individual
KRA levels to find overall
KRA level for the KRA period
Each KRA period
Best Practice
Average
6.4 EXAMPLE OF SCORING CALCULATIONS
The following diagram illustrates the stages in calculating the overall annual performance
level from the individual KPI scores in each KRA period.
In this example, we complete each stage. However, averaging all KPI scores from
stage finds the same result at stage , i.e. skipping stages and .
At stages 3 and 5, the rules in section 6.3 ‘Business Rules for Performance Achieved’
are also applied.
Page 26 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Measuring and Scoring KPIs
This section of the guideline provides specific instructions for how to go about measuring
the KPI scores detailed in Section 8 ‘Individual KPI Details’.
7.1 SCORES FOR PERFORMANCE RANKINGS
By measuring each KPI, parties evaluate how well the Contractor’s activities are
achieving or progressing toward the relevant intent.
The measurement tables for all KPIs use the same approach to four performance levels
as KRAs.
SCORE PERFORMANCE RANKING DESCRIPTION
1 Poor Contractor has performed below the Principal’s
standard.
2 Minimum Condition of Satisfaction (MCOS)
Contractor’s performance has met the Principal’s standard.
3 Best Practice Contractor’s performance has exceeded the
Principal’s standard.
4 Outstanding Contractor’s performance is among the leaders of those that exceed the Principal’s standard.
Note: See Section 6 ‘Scoring Approach’ for details on using these KPI scores to calculate
KRA levels.
Page 27 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
7.2 HOW THE KPIS ARE PRESENTED
The subsection for each KPI in Section 8 ‘Individual KPI Details’ follows the format
below:
HEADING CONTENTS
Topic heading KPI reference number and title
Intent The desired behaviour that the KPI encourages Contractors to adopt, or the eventual outcome that the KPI drives them to achieve
Definition The area or aspect of Contractor activities that the KPI measures
Business rules Any specific rules to apply when assessing the performance score for the KPI
Process How and when Contractors need to collect and report the data that is used to
assess the performance score
Calculation (Included when relevant) How to calculate the relevant measure
Data required The specific type(s) of data or information needed in reports to support the performance measure, so the Principal can validate the score
Measure score
Depending on the measurement approach for the KPI:
• Quantitative: The ranges of numeric measures to achieve each performance score
• Qualitative: A description of the actions or behaviours demonstrated to earn each performance score
Note: Some measure scores also include a more detailed description of the
behaviour or achievement that each performance score ranking demonstrates.
7.3 KPI MEASUREMENT TYPES
Each KPI measures performance using one of the following approaches:
APPROACH DESCRIPTION
Quantitative Each performance level is measured using a quantitative (numeric) range.
Qualitative Each performance level is measured using a clear description of specific behaviours shown and/or actions completed.
Blended Each performance level is measured using a mix of quantitative and qualitative measures, as above.
Aggregated Each performance level is measured using a quantitative range for a score that is calculated from the results of several other measures.
Page 28 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
7.4 COMPLETING AUDITS TO MEASURE
PERFORMANCE
The Principal expects the Contractor to follow the below guidelines, when measurement
of a KPI requires the Contractor to audit their compliance against specified guidelines or
business rules:
The Contractor details their approach for auditing in their quality management plan
(QMP)
The Contractor audits ideally at least 10% of the relevant activities. Unless
otherwise specified in the framework, for any KPI requiring a data sample, a score of
‘Outstanding’ can only be achieved if the sample size audited is at least 10%
When Sub-Contractors have performed work on the relevant activities, the
Contractor includes Sub-Contractor work in the audit sample.
Note: Requirements for auditing KPIs linked to contract management plans are detailed
under Section 7.5.
7.5 KPIS THAT MEASURE PERFORMANCE OF
PLANS
Seven of the KPIs specifically measure how well the Contractor is using their various
contract management plans and keeping them up to date, linking their delivery of
outcomes to their use of the plan. These KPIs are:
1.1.2 Health and Safety Management Plan
1.2.1 Traffic Control Plan
3.1.1 Customer and Stakeholder Communications Management Plan
4.1.7 Environmental and Social Management Plan
5.1.1 Quality Management Plan
5.2.1 Maintenance Management Plan
6.2.2 Emergency Procedures Preparedness Plan.
7.5.1 Acceptance of plans
Each of the above KPIs includes measuring whether the Principal has accepted the
Contractor’s plan. This is a requirement to achieve performance Level 2 (Minimum
Condition of Satisfaction). This should ideally happen before the Contract commences,
but the Contractor and Principal agree on a reasonable grace period, during which they
work together to establish each plan.
While the plan has not yet been accepted during the grace period, the Contractor should
record a score of NA (not applicable) for the relevant KPI. Entering a score of NA in the
reporting template excludes the KPI from the averaging calculations to find the KRA
level. See Section 4.3 ‘Reporting Templates’ and Appendix 2 (page 90) for more details.
If the Principal does not accept a contract management plan by the time the grace
period has ended, then the KPI linked to that contract management plan will score at
performance Level 1 (Poor) until the Principal can accept it. However, if the Principal has
Page 29 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
not advised a decision or feedback within ten days of the Contractor submitting their
plan for acceptance, the grace period is extended until the Principal can respond.
Once a plan is accepted, each KPI measures how well the Contractor’s activities in each
of these management areas follow the applicable plan, i.e. the Contractor needs to show
they are:
Auditing the related activities, as agreed in their quality management plan, to
confirm these follow the plan (also see the next heading below)
Reviewing the plan regularly, and updating it as needed
Looking for and applying improvements to the plan, to manage the relevant
activities better.
7.5.2 Audits against plans
When auditing activities against a plan, the number of non-compliances are the number
of ways that an activity did not comply with the plan. This means that an audit of one
activity against a plan could result in multiple non-compliances.
The Contractor is expected to begin conducting audits against their plan, after they have
submitted it to the Principal, and are waiting for it to be accepted.
Each KRA period an audit carried out (preferably in conjunction with the MCM) must
cover:
At least 1/3 of all processes within the Maintenance Management Plan must be
audited each period with some of the more important processes being audited
every period. In any KRA year every process contained within the Maintenance
Management Plan must be audited at least once
At least 1/3 of all processes within the Quality Management Plan must be audited
each period with some of the more important processes being audited every
period. In any KRA year every process contained within the Quality Management
Plan must be audited at least once
100% of the processes contained within all remaining plans will be audited every
period, accepting that some processes will not be applicable in every period (eg.
Winter maintenance)
Each plan will have a separate random selection taken from the list of those
activities deemed relevant to that plan
For each KPI linked to a contract management plan at least 5 examples of these
randomly selected sites/activities must be audited against the plan, and the
results documented
If any selected sites/activities are not suitable (ie the result is too many N/As) an
alternative site/activity should be randomly selected from the same list of
relevant activities.
Within each “Measure Score” table for the KPIs linked to contract management plans,
where reference is made to being ‘up to date’, this means:
The plan has been updated at least within the last 12 months AND
Any improvements identified have been actioned and updated within the plan.
Page 30 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
The following business rules apply to all Contractors’ audits of contract management
plans:
The Contractor must have a system in place to track and record the outcome of
audits against their contract management plans
The Contractor must include both results from the audits of the specific plan, and
any other relevant audits carried out during the KRA period
The Contractor must have all audit reports accepted by the Principal
The Contractor must include information on any Sub-Contractors’ activities
The Contractor must record and report any repeat non-compliances identified, which
include:
Any non-compliances identified in any audit become repeat if not closed by the
next audit or a date agreed with the MCM
Closed during either of the previous two KRA periods, and have recurred in this
KRA period.
7.6 ‘PLANNED’ AND ‘UNPLANNED’ EVENTS
EVENT TYPE DEFINITION
Planned event
An activity that is programmed to occur, for example:
• Contractor’s renewal work or planned maintenance • A third party’s activities.
Unplanned event Any incident on the network for which no programme has been agreed. (See section 5.3.5 and 6.6.1 of the NOC Maintenance Specification for examples.)
Page 31 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
7.7 TREND LINE CALCULATIONS
Several KPIs require measuring the linear regression line of a trend, based on a rolling
average for a specified number of months.
For each of these KPIs, the following steps show an example of how this is expected to
be calculated. It is expected that many Contractors will use Microsoft Excel or a similar
tool to automate this task, particularly the regression analysis.
Note: This example is based on reporting a rolling average for the last 12 months. In
this situation, the earliest month in the reported period needs data back to 23 months
ago, to find the annual average. (See the steps for more clarification.)
STEP CALCULATION EXAMPLE
1 For each month in the reported period
and the 11 months prior, find the relevant monthly value, e.g. the total number of relevant instances or occurrences during that month.
2 For each month in the reported period, find the rolling annual total, i.e. the sum of the values for each month in
the year ending at the month in question.
Page 32 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
STEP CALCULATION EXAMPLE
3 For each month in the reported period,
find the rolling annual average, i.e. divide the rolling annual total by 12.
4 Graph the rolling annual average by month.
5 Plot a linear regression line for the rolling annual average data series, using the formula y=mx+c
Hint: If plotting this online, it is easiest
to have the application display the formula values.
6 Find the m value of regression line.
Multiply this by 100 to express this as a
percentage. (This is the value to report
for the KPI.)
• A positive number shows an increasing trend.
• A negative number shows a decreasing trend.
m(-0.0099) = -0.99%
7 Assess the final value against the relevant KPI measure to find the KPI score.
-0.99% is >-1%
Page 33 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Applying the Trend Line Calculation
The following diagram shows how the m value of the regression line applies to the
measures shown in the relevant KPIs. Currently, the intent of all such KPIS is to reduce
the occurrences of a particular event, so a negative trend demonstrates improvement.
See each applicable KPI in Section 8 ‘Individual KPI Details’, for the specific percentage
ranges required for each measure score.
Reminder: A number that is less than negative 1 (<-1) is lower than it, e.g.:
-0.25 is less than (<) -0.1
-0.05 is greater than (>) -0.1
Page 34 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Individual KPI Details
This section of the guideline provides the measures and scoring calculations for the
specific KPIs within each KRA.
KRA 1: HEALTH AND SAFETY
Objectives
Safety is of paramount importance on the journey to zero harm.
Activities under NOCs show commitment to health and safety outcomes.
KPIs in this KRA
1.1.1 Total Recordable Injury Frequency Rate
1.1.2 Health and Safety Management Plan
1.2.1 Traffic Control Plan
Page 35 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 1.1.1 Total Recordable Injury Frequency Rate
Intent
Injury rates are below expected levels or reducing, showing that Contractors are
committed to operating an injury-free ‘zero harm’ work environment for their workforce.
Definition
This KPI measures the lag indicators for Total Recordable Injury Frequency Rate (TRIFR),
i.e. the rate of recordable injuries for each 1,000,000 work hours.
Recordable injuries include the following categories:
Fatal injuries (FI)
Notifiable injuries (SH)
Lost time injury (LTI)
Medical treatment injury (MTI)
Employees on restricted work duties (RWI).
Business rules
The Contractor must:
Include information on injuries and work hours for any Sub-Contractors
Exclude injuries in the ‘First Aid’ category.
The TRIFR must be based on the last 12 months of data, or part thereof if the
Contract commenced within that 12 months.
The Principal identifies and advises the top performers once all Contractors have
provided their performance assessments for the KRA period. This means the
Contractor can only self-assess a score of 4 if the TRIFR is zero. It can otherwise
only be awarded during moderation. (See Section 4.5 ‘Moderation of Reported
Performance’.)
The Contractor must record all health and safety data in the NZTA Zero Harm online
reporting tool.
Process
The Contractor captures the data monthly on their internal health and safety (H&S)
reporting system and the Principal’s online reporting tool. This information forms part of
the Contractor’s monthly report to the Principal.
Data required
Total work hours completed during the last 12 months
Total recordable injuries for each category during the last 12 months
TRIFR for the last 12 months.
Calculation
The following formula is used to calculate the TRIFR for the KRA period. (The
abbreviations refer to the injury categories listed above.)
Page 36 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
TRIFR = (FI+SH+LTI+MTI+RWI) × 1,000,000
Total Hours Worked in last 12 months
Measure score
SCORE MEASUREMENTS
1 TRIFR is more than 30 (>30).
2 TRIFR is more than 25, but not more than 30 (>25 and ≤30).
3 TRIFR is 25 or lower (≤25).
4 Both of the following:
• TRIFIR is 25 or lower (≤25) • TRIFR is either zero, or within the top 10% for all NOCs for the KRA period.
Page 37 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 1.1.2 Health and Safety Management Plan
See also Section 7.5 ‘KPIs that Measure Performance of Plans’ for more details.
Intent
Contractors make effective, consistent use of their plan to manage their workplace
health and safety environment by using, auditing and reviewing it regularly.
Definition
This KPI measures non-compliances discovered against the Health and Safety
Management Plan through:
Audit
Accident/near miss investigation
Any other method that the Contractor or Principal uses.
This KPI also measures whether the Contractor has taken action to address any non-
compliances identified.
Business rules
The Contractor must have a Health and Safety Management Plan accepted by the
Principal within the grace period allowed in the NOC.
The Contractor must observe the business rules for auditing their plan, and keeping
it up to date, as listed in Section 7.5.2 ‘Audits against plans’.
Process
The Contractor defines their approach for auditing activities against their plan in their
Quality Management Plan (QMP), and captures the audit data on their internal quality
management system.
Page 38 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Data required
Most recent date updated (or new) plan has been approved by the Principal
(excluding any updates after the end of the KRA period), and by whom
Date(s) audit(s) completed and by whom
Date(s) audit report(s) accepted by the Principal and by whom
Number of non-compliances (including any repeats) discovered during the KRA
period
Number of repeat non-compliances recognised during the KRA period
Percentage of processes within the plan audited during the KRA period
Total number of sites/activities audited against the plan during the KRA period.
Measure score
SCORE DESCRIPTION MEASUREMENTS
1 Contractor has not taken action to either:
• Agree a plan in the required
timeframe • Audit activities against an
accepted plan5 • Record all non-compliances • Keep the plan current and
accurate.
Any of the following:
• No plan is accepted. • Plan is not up to date.
• Activities are not audited against plan. • Plan is accepted and audit is completed, but any
non-compliances have not been recorded.
2 Contractor is:
• Auditing activities against a current, accepted plan
• Managing actions and learnings.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report. • All non-compliances in this period are recorded. • All repeat non-compliances are recorded.
3 Contractor is both:
• Auditing activities against a current, accepted plan
• Achieving a good level of compliance in the audit results.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report. • All non-compliances in this period are recorded,
and there are five or fewer (≤5) of these. • No repeat non-compliances.
4 Contractor is both:
• Auditing activities against a current, accepted plan
• Achieving an excellent level of
compliance in the audit results.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report.
• All non-compliances in this period are recorded, and there are two or fewer (≤2) of these.
• No repeat non-compliances.
5 See Section 7.5.2 ‘Audits against plans’ for more details on when audits must begin.
Page 39 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 1.2.1 Traffic Control Plan
See also Section 7.5 ‘KPIs that Measure Performance of Plans’ for more details.
Intent
Contractors make effective, consistent use of their traffic control plan to provide a safe
environment for staff and road users by using, auditing and reviewing it regularly.
Definition
This KPI measures non-compliance of activities discovered against the traffic control plan
through:
Audit
Incident investigation
Any other method the Contractor or Principal uses.
This KPI also measures whether the Contractor has taken action to address any non-
compliances identified.
Business rules
The Contractor must have a Traffic Control Plan accepted by the Principal before the
Contract commences.
The Contractor must observe the business rules for auditing their plan, and keeping
it up to date, as listed in Section 7.5.2 ‘Audits against plans’.
Process
The Contractor defines their approach for auditing activities against their plan in their
Quality Management Plan (QMP), and captures the audit data on their internal quality
management system.
Data required
Most recent date updated (or new) plan has been approved by the Principal
(excluding any updates after the end of the KRA period), and by whom
Date(s) audit(s) completed and by whom
Date(s) audit report(s) accepted by the Principal and by whom
Number of non-compliances (including any repeats) discovered during the KRA
period
Number of repeat non-compliances recognised during the KRA period
Percentage of processes within the plan audited during the KRA period
Total number of sites/activities audited against the plan during the KRA period.
Page 40 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Measure score
SCORE DESCRIPTION MEASUREMENTS
1 Contractor has not taken action to either:
• Agree a plan in the required
timeframe • Audit activities against an
accepted plan • Record all non-compliances • Keep the plan current and
accurate.
Any of the following:
• No plan is accepted. • Plan is not up to date.
• Activities are not audited against plan. • Plan is accepted and audit is completed, but any
non-compliances have not been recorded.
2 Contractor is:
• Auditing activities against a current, accepted plan
• Managing actions and
learnings.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report.
• All non-compliances in this period are recorded. • All repeat non-compliances are recorded.
3 Contractor is both:
• Auditing activities against a current, accepted plan
• Achieving a good level of compliance in the audit results.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report. • All non-compliances in this period are recorded,
and there are five or fewer (≤5) of these. • No repeat non-compliances.
4 Contractor is both:
• Auditing activities against a current, accepted plan
• Achieving an excellent level of
compliance in the audit results.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report.
• All non-compliances in this period are recorded, and there are two or fewer (≤2) of these.
• No repeat non-compliances.
Page 41 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KRA 2: ROAD USER SAFETY
Objective
Activities under NOCs:
Contribute to safer journeys through safe roads, speeds and vehicles, and safe road
use.
Contract Requirements
The reporting requirements in this section are covered in Section 5.5.3 of the NOC.
Safety Data and Crash Types
Road user safety is generally measured by the number of crashes on the network in
which people were seriously injured or killed. When assessing KPIs for this KRA, the
casualty data comes from the Crash Analysis System (CAS).
CAS records each casualty with a two-letter movement code for the crash type, such as
BD or DA. The details for some KPIs in this section list the relevant movement type
codes to measure. Understanding what these codes mean may be helpful (but is not
essential) when measuring the relevant KPI score.
In summary, the codes relate to how the vehicle was moving and what type of road the
vehicle was on when the crash occurred. For example, code DB refers to crashes where
the vehicle “loses control or leaves the roadway on curves turning left, or when the
vehicle is turning left at an intersection”.
See the Guide for the Interpretation of Coded Crash Reports for the Crash Analysis
System (CAS) for the full descriptions of each movement type, if needed.
Updating Data to Current State
For scoring each KPI in this KRA, Contractors are only expected to include crashes that
have been logged in CAS as at the time they prepare the KRA report. It is acceptable
that crashes occurring late in the KRA period might not yet have been recorded in CAS.
However, the Contractor is expected to account for this in subsequent reports, i.e. to
extract fresh sets of historical data each time, rather than leave the data for the
preceding period(s) as it was reported earlier.
KPIs in this KRA
2.1.1 DSI Trend
2.1.2 Loss of Control in Darkness
2.1.3 Network Safety Trend Report
Page 42 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 2.1.1 DSI Trend
Intent
Statistics for deaths and serious injuries in crashes on the network are decreasing,
showing that Contractors understand the network’s safety issues and that the activities
they undertake, including improvements they identify and implement, result in a safer
environment for road users.
Definition
This KPI measures the lag indicators of deaths and serious injuries as an indicator of
network safety. The scoring also recognises the Contractor for identifying and
implementing improvements to reduce these statistics.
Business rules
The calculation must include all crashes involving deaths and serious injuries as
reported in CAS. This data must not be filtered.
When claiming safety improvements, the Contractor:
Can include any improvements implemented in this period that they reported as
identified in a previous period. The Contractor may report on these in the period
that they are delivered
Must have had endorsement from the Principal’s Traffic and Safety Engineer for
the safety improvements, and they must have been included in NZTA’s Safety
Improvements Database.
The DSI trend calculation must be based on the:
Current CAS data of deaths and serious injuries for the 35 calendar months
leading up to three months before the end of the KRA period (e.g. for the KRA
period ending June 2016, data would be extracted for March 2016 back to April
2013), regardless of when the Contract commenced
Note: By excluding the most recent three months’ data, CAS is more likely to
include all the relevant statistics.
Process
The Contractor records and reports on trends in CAS crash data of deaths and serious
injuries in relation to vehicle kilometres of travel in their network.
Around July each year, the Principal calculates the annual VKT value for each NOC, and
advises this to each Contractor. This value is based on the annual average daily traffic
(AADT) data in RAMM for the previous calendar year.
Page 43 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Data required
24 month trend line percentage of rolling average monthly DSIs/annual VKT
Number of safety improvements identified and endorsed by the Principal’s Traffic
and Safety Engineer
Number of safety improvements accepted by the Principal and fully implemented
The initiative names relating to those safety improvements.
Calculation
DSI trend calculation = 12 month rolling average monthly DSI x 10^10
Annual VKT
Note:
For annual VKT, use the most recent value provided by the Principal.
The 24 month rolling average trend and linear regression line are calculated using an
approach consistent with the steps in Section 7.7 ‘Trend Line Calculations’.
Measure score
SCORE DESCRIPTION MEASUREMENTS
1 DSI trend is deteriorating DSI trend shows a linear regression line of more than 1% (>1%).
2 DSI trend is steady DSI trend shows a linear regression line between 0% and 1% (>0% and ≤1%).
3 DSI trend is improving, and improvements are
identified
Both of the following:
• DSI trend shows a linear regression line that is 0% or less
(≤0%). • Contractor has identified one or more improvements that
have been endorsed by the Principal’s Traffic and Safety Engineer as having potential to further improve the network safety trend.
4 DSI trend is improving,
and improvements are implemented
All of the following:
• Requirements for Level 3 are met • DSI trend shows a linear regression line that is 0% or less
(≤0%). • In this period, two or more of the safety improvements
identified have been implemented.
Page 44 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 2.1.2 Loss of Control in Darkness
Intent
Statistics for crashes that occur on bends during darkness on the network are
decreasing, showing that Contractors understand those assets that have a direct
relationship to driving in darkness, and implement effective measures that improve road
users’ ability to drive safely.
Definition
This KPI measures the instances of loss of control in darkness, as an indication of the
quality of the driving environment at night on the network, such as delineation and
lighting.
Business rules
The calculation must include:
All crashes (injury and non-injury) recorded in CAS with movement codes BB, BC,
BD, BF, DA and DB that occurred during hours of darkness (i.e. the inverse of the
daylight hours)
All crashes (injury and non-injury) recorded in CAS with movement codes BB, BC,
BD, BF, DA and DB.
CAS crash casualty data for each of the most recent 12 months of complete data
available, irrespective of the Contract’s commencement date.
Process
Crash data in CAS for the movement codes above that involve loss of control in
darkness, is assessed and the trend is reported in proportion to all crashes on bends
over the most recent 12 months of complete data available.
Data required
Number of crashes on bends in darkness each month during the 12 month period
Total number of crashes on bends each month during the 12 month period
Percentage of crashes on bends in darkness during the 12 month period
Trend regression line percentage for the monthly totals.
Calculation
Percentage of crashes in darkness
total no. crashes on bends [BB+BC+BD+BF+DA+DB] in darkness6 during the last 12 months
total no. crashes on bends [BB+BC+BD+BF+DA+DB] during the last 12 months
6 Based on the ‘dark / light’ environment attribute within CAS variables for tabulation.
Page 45 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Trend line
Calculate the trend and linear regression line using an approach consistent with the
steps in Section 7.7 ‘Trend Line Calculations’, using the same 12 months’ worth of data,
plotted on a monthly basis.
Measure score
SCORE DESCRIPTION MEASUREMENTS
1 Proportion is high, and trend is deteriorating
Both of the following:
• More than 35% of reported crashes occurred in darkness. • Trend shows an increasing linear regression line (>0%).
2 Either proportion is high,
or trend is deteriorating
One of the following:
• More than 35% of reported crashes occurred in darkness. • Trend shows an increasing linear regression line (>0%).
3 Proportion is low, and trend is improving
Both of the following:
• 35% or less of reported crashes occurred in darkness. • Trend shows a level or negative linear regression line of
between 0% and -1% (≤0% and >-1%)
4 Proportion is low, and trend is improving significantly
Both of the following:
• 35% or less of reported crashes occurred in darkness. • Trend shows a negative linear regression line of -1% or
lower (≤-1%).
Page 46 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 2.1.3 Network Safety Trend Report
Intent
Contractors demonstrate their understanding of the safety issues on the network. They
are proactively seeking improvement opportunities, implementing improvements, and
monitoring their effectiveness.
Definition
This KPI measures the quality and effectiveness of the Network Safety Trend Report that
Contractors submit to the Principal.
Business Rules
To align the four monthly KRA period with the quarterly safety reporting period, the
Contractor must assess the reports as follows:
IN KRA PERIOD… ASSESS THE SAFETY REPORT FOR THE…
1 Ending October First quarter.
2 Ending February Second quarter.
3 Ending June
Third and fourth quarters.
Note: If these two report results are different, the lower result applies for scoring this KPI.
When claiming safety improvements, the Contractor:
Can include any improvements implemented in this period that they reported as
identified in a previous period. The Contractor may report on these in the period
that they are delivered
Must have had endorsement from the Principal’s Traffic and Safety Engineer for
the safety improvements, and they must have been included in NZTA’s Safety
Improvements Database.
Process
The Contractor captures relevant data, reports their findings, self-assesses performance,
and provides a quarterly Network Safety Trend Report to the Principal.
Data Required
Confirmation that the report was delivered on time
Confirmation that the report was accepted as fit for purpose
Evidence (where applicable) that the report has exceeded the minimum
requirements
Number of safety improvements identified and endorsed by the Principal’s Traffic
and Safety Engineer
Number of safety improvements accepted by the Principal and fully implemented
Page 47 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
The initiative names relating to those safety improvements
Confirmation that the Minor Improvements (Safety) Programme projects agreed for
physical work delivery through the NOC are being delivered on time, or that any
delays have been approved by the CB due to valid mitigating circumstances7.
Measure score
SCORE MEASUREMENTS
1 Either of the following:
• The report(s) are not delivered on time. • The Principal does not accept the standard of the report(s) as ‘fit for purpose’.
2 Both of the following:
• The report(s) are delivered on time. • The Principal accepts the standard of the report(s) as ‘fit for purpose’.
3 All of the following:
• Requirements for Level 2 are met. • The Principal accepts that the standard of the report(s) exceeds the minimum
requirements.
• In this period, Contractor has identified two or more new safety improvements that have been endorsed by the Principal’s Traffic and Safety Engineer (from crash investigations or other means).
4 All of the following:
• Requirements for Level 3 are met.
• In this period, one or more of the safety improvements identified has been implemented.
• Agreed minor improvement physical work delivery through the NOC is being delivered in line with the agreed Minor Improvements (Safety) Programme for this KRA period, or valid mitigating circumstances have been approved by the CB.
A description of mitigating circumstances can be seen in the table of terms used (page 6).
Page 48 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KRA 3: CUSTOMER
Objective
Contractors always:
Consider road users’ needs for access
Respond to customers’ requests and their expectations.
KPIs in this KRA
3.1.1 Customer and Stakeholder Communications Management Plan
3.1.2 Communications of Events and Incidents on Customers
3.1.3 Customer Satisfaction Survey
3.2.1 Ability to Respond to Customers
Page 49 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 3.1.1 Customer and Stakeholder Communications
Management Plan
See also Section 7.5 ‘KPIs that Measure Performance of Plans’ for more details.
Intent
Contractors make effective, consistent use of their plan to manage communications with
customers and stakeholders, by using, auditing and reviewing it regularly.
Definition
This KPI measures how well the Contractor is using their customer and stakeholder
communications plan to measure, review and improve the value and effectiveness of
their processes. Non-compliances discovered against the Customer and Stakeholder
Communications Management Plan are captured through:
Audit
Communicating and interacting with Customers and Stakeholders
Any other method that the Contractor or Principal uses.
This KPI also measures whether the Contractor has taken action to address any non-
compliances identified.
Business Rules
The Contractor must have a Customer and Stakeholder Communications
Management Plan accepted by the Principal within the grace period allowed in the
NOC.
The Contractor must observe the business rules for auditing their plan, and keeping
it up to date, as shown in Section 7.5.2 ‘Audits against plans’.
Process
The Contractor defines their approach for auditing activities against their plan in their
Quality Management Plan (QMP), and captures the audit data on their internal quality
management system.
Page 50 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Data Required
• Most recent date updated (or new) plan has been approved by the Principal
(excluding any updates after the end of the KRA period), and by whom
• Date(s) audit(s) completed and by whom
• Date(s) audit report(s) accepted by the Principal and by whom
• Number of non-compliances (including any repeats) discovered during the
KRA period
• Number of repeat non-compliances recognised during the KRA period
• Percentage of processes within the plan audited during the KRA period
• Total number of sites/activities audited against the plan during the KRA
period.
Measure score
SCORE DESCRIPTION MEASUREMENTS
1 Contractor has not taken action to either:
• Agree a plan in the required timeframe
• Audit activities against an accepted plan
• Record all non-compliances • Keep the plan current and
accurate.
Any of the following:
• No plan is accepted. • Plan is not up to date. • Activities are not audited against plan.
• Plan is accepted and audit is completed, but any non-compliances have not been recorded.
2 Contractor is:
• Auditing activities against a current, accepted plan
• Managing actions and
learnings.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report.
• All non-compliances in this period are recorded. • All repeat non-compliances are recorded.
3 Contractor is both:
• Auditing activities against a current, accepted plan
• Achieving a good level of compliance in the audit results.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report. • All non-compliances in this period are recorded,
and there are five or fewer (≤5) of these. • No repeat non-compliances.
4 Contractor is both:
• Auditing activities against a current, accepted plan
• Achieving an excellent level of compliance in the audit results.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report. • All non-compliances in this period are recorded,
and there are two or fewer (≤2) of these.
• No repeat non-compliances.
Page 51 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 3.1.2 Communications of Impact of Events and
Incidents on Customers
Intent
Contractors communicate proactively with customers, giving them timely, reliable
information about planned and unplanned events on the network.
Definition
This KPI measures the quality of the Contractor’s communication to their Customers.
Business Rules
The Contractor and the Principal must agree on:
The monthly audit sample size (ideally at least 10%. See Section 7.4
‘Completing Audits to Measure Performance’ (page 28))
Communication protocols for managing both planned and unplanned events
Events and incidents that are relevant to customers and need to be
communicated. Most routine cyclic operations (eg litter collection) should be
excluded, unless they require communication with customers for any reason.
The Contractor must record all actual communications on the relevant system(s),
such as:
RAMM CAR, TREIS, CRMS
The Contractor’s own system
Information supplied through TOC.
In addition to the randomly selected audit sample, the Contractor must include in
this measure any planned or unplanned events where there was negative feedback
about the communications (rather than about the event).
To increase the consistency of the numbers reported, events and incidents that
occur at the same site over a number of days should be treated as a single event or
incident in the context of defining the list of events from which to take a sample.
Therefore an event or incident that occurs over multiple days at a site should not be
counted more than once in the potential pool of events for sampling, even if the
event or incident is reported daily or weekly within other systems.
If an event or incident occurring over a number of days is selected for inclusion in
the sample, evidence of good communication must be assessed for each event or
incident as a whole. Duration of the event does not form part of the calculation.
Process
Each month, the Principal and Contractor review a randomly selected set of relevant
planned events and unplanned events (e.g. floods, snow, etc.). For each event, they
review and evaluate:
Information loaded into relevant system(s)
Information on incident response and updates provided through appropriate
system(s)
Page 52 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Notices about road works and other planned events (e.g. sporting events),
evaluating whether these have been provided in a timely and customer-friendly
manner, in accordance with the various plans
Feedback from key stakeholders and community groups, evaluating whether good
levels of collaboration have been demonstrated in responding to relevant incidents
and events.
Data Required
Total number of planned events relevant to customer communication
Total number of unplanned events relevant to customer communication
Number of relevant planned events audited
Number of relevant unplanned events audited
Percentage of audited events that show evidence of good communications
Total number of events where there was negative feedback about the
communications (rather than about the event).
Measure score
SCORE MEASUREMENTS
1 85% or less of randomly selected relevant events show evidence of good
communication.
2 More than 85% (but less than 90%) of randomly selected relevant events show evidence of good communication.
3 90% or more (but less than 95%) of randomly selected relevant events show evidence
of good communication.
4 95% or more of randomly selected relevant events show evidence of good communication.
Page 53 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 3.1.3 Customer Satisfaction Survey
Intent
Survey results show that road users are satisfied with the services, provided by the
Contractor, that they have used when travelling.
Definition
This KPI measures the levels of road users’ satisfaction with the aspects of the network
covered by the Contractor’s NOC. The Principal conducts a quarterly survey of
customers’ experiences with the New Zealand state highway network, including:
Their opinion of the travel experience
The details of their journey
Any disruptions they encountered
Their travel time.
Business Rules
The NOC customer satisfaction score is based on the NOC related questions of the
NZTA customer survey.
The Principal identifies and advises the top performers once the data is available.
Process
The NZTA survey is conducted quarterly, and the results are mapped to the relevant
Contractor, according to the respondents’ post codes. Each KRA period, the Principal’s
Planning & Performance Team extracts the data from the Customer Survey Portal and
informs the Contractor.
Data Required
NZTA customer survey result(s) for the relevant quarter(s)
Calculation
The survey result is the rolling average of the monthly customer results for the last
12 months.
Measure Score
SCORE MEASUREMENTS
1 Survey results show customer satisfaction of less than 45%.
2 Survey results show customer satisfaction of 45% or more, but less than 50%.
3 Survey results show customer satisfaction of 50% or more.
4 Both of the following:
• Survey results show customer satisfaction of 50% or more. • Survey results are within the top 10% for all NOCs for the KRA period.
Page 54 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 3.2.1 Ability to Respond to Customers
Intent
Contractors give customers a timely, professional and useful response to all queries,
complaints and correspondence.
Definition
This KPI measures the responsiveness of the Contractor to the customer queries,
complaints and correspondence received during the KRA period.
Business Rules
Contractors must record each customer query, complaint or correspondence as soon
as possible after they receive it, and in sufficient detail to demonstrate compliance
with this requirement.
Contractors must record when they respond to each customer query, complaint or
correspondence.
Contractors can only record a query, complaint or correspondence as closed when
they have responded directly about its substantive content. It is not enough to
simply acknowledge its receipt.
If the response does not resolve the customer’s issue, the Contractor must inform
the customer of the reason for this.
To provide meaningful results for this KPI, this data can only be collated within five
working days of the end of the KRA period.
Contractors must complete callback surveys for a minimum of four customer
contacts received during the KRA period, or 100% of all customer contacts,
whichever is less.
Exception: If the Contractor has pledged in their tendered submission to a specific
number or proportion of customer contacts, the pledged number of surveys apply.
Process
The Contractor maintains information about customer interactions as they are
received.
Each month, the Contractor reports on:
Number of customer interactions in CRMS for response within the reporting
period, and the response times to them
Number of callback surveys completed, and whether these show the customer
interactions were satisfactorily dealt with.
Around the end of the KRA period, the Contractor collates customer interaction data
from CRMS for each month, to calculate their KPI score.
Page 55 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Data Required
CRMS data on customer complaints and interactions assigned to the Contractor,
including:
Number of complaints that were responded to:
Within two days
Outside two days
Number of other interactions that were responded to:
Within ten days
Outside ten days
Total number of customer interactions
Number of customer interactions resolved
Number of callback surveys completed
KPI score for previous KRA period.
Calculation
Sum the following points to find the Contractor’s total response score for all customer
contacts received during the KRA period:
DETERMINE IF… BY CALCULATING… AND IF SO, ADD…
complaint response ratio is more than 0
complaint response ratio = no. complaint responses within two days – (no.
complaint responses outside two days × 5)
1 point.
interaction response ratio is more than 0
interaction response ratio = no. interaction responses within ten days – (no. interaction responses outside ten days × 5)
1 point.
resolution ratio is more than 0
resolution ratio = no. customer interactions resolved – (no. customer interactions not resolved ×10)
1 point.
Contractor completed callback surveys for a sample of closed customer interactions in this KRA period
1 point.
Contractor achieved a KPI score of 3 or more for this KPI in the previous KRA period
1 point.
Measure score
SCORE DEFINITION MEASUREMENTS
1 Contractor’s responses to customers have
been untimely or unprofessional.
Total response score is 0 or 1 points.
2 Contractor’s responses to customers have been adequately timely and professional.
Total response score is 2 or 3 points.
3 Contractor’s responses in this period have been consistently timely and professional.
Total response score is 4 points.
4 Contractor shows a consistent history of timely, professional responses.
Total response score is 5 points (or more).
Page 56 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KRA 4: SUSTAINABILITY
Objectives
Contractors contribute to maintaining a sustainable and engaged contracting market.
Contractors adopt good practice and act in a responsible manner, which contributes
to a transport system that adds positively to New Zealand’s economic, social and
environmental welfare.
KPIs in this KRA
Environmental Triangle:
4.1.1 Enforcement Action
4.1.2 Regulatory Non Compliance
4.1.3 CS-VUE Non Compliance
4.1.4 Environmental Initiatives
4.1.5 Environmental Training, Workshops and Presentations
4.1.6 Environmental Hazards or Opportunities.
4.1.7 Environmental and Social Management Plan
4.2.1 Healthy Market Pledges
Page 57 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measures 4.1.1 to 4.1.6 Environmental Triangle
Intent
The Contractor demonstrates commitment to environmental and social responsibility
throughout their activities under the NOC.
Definition
This group of KPIs measures environmental sustainability through the parties’:
Impact of works
Ability to meet commitments
Actions taken to improve how they work.
The environmental triangle formula combines measures of several indicators to give an
aggregated KPI score.
Business rules
Contractors must maintain a system for capturing information to support the
measurement of each indicator within this KPI
For KPI 4.1.5, the Contractor may only count an individual attending the identical
course more than once as one attendance. An ‘individual’ is a person who is an
employee or Sub-Contracted person, who undertakes work related to the NOC.
Indicator weightings
The following diagram and table show the relative weightings of each type of event
whose environmental impact these KPIs measure.
SCORE INSTANCES
OF…
AS… I.E. …
KPI 4.1.1 -20 -20 × no.
Negative im
pact
KPIs 4.1.2 and 4.1.3 -10 -10 × (no. +
no.)
KPI 4.1.4 +20 20 × no.
Positiv
e im
pact
KPI 4.1.5 +1 1 x no.
KPI 4.1.6 +0.5 0.5 × no.
See the KPI descriptions below for details on measuring and scoring each element, and
the calculation after those, for details on finding the aggregated total score.
Enforce-
ment Action
Non-Compliance
Environmental Initiatives Adopted or Actioned
Environmental Training, Workshops & Presentations
Environmental Hazards or Opportunities Actioned
Page 58 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Measure 4.1.1 Enforcement Action
This KPI element counts the number of enforcement orders (under section 314) and
abatement notices (under section 322) of the Resource Management Act 1991 that are
relevant to the NOC area.
Process
For each KRA period assessment, the Contractor captures and totals the number of
enforcement orders and abatement notices issued under the above legislation.
Impact
-20 points per enforcement order and/or abatement notice issued
Data Required
Number of orders and notices issued during the KRA period
Measure 4.1.2 Regulatory Non-Compliance
This KPI element counts the number of regulatory monitoring non-compliances issued
against the Contractor. (It excludes regulators’ minor non-compliances.)
Process
For each KRA period assessment, the Contractor captures and totals the number of non-
compliances issued against them.
Impact
-10 points per non-compliance issued
Data Required
Number of non-compliances issued during the KRA period
Measure 4.1.3 CS-VUE Non-Compliance
This KPI element counts the number of non-compliances in NZTA’s CS-VUE system.
Process
Within five (5) days of the end of each KRA period, the Contractor extracts a snapshot of
the ‘compliance status bar graph’ from the dashboard tab of CS-VUE for the contract
area.
Impact
-10 points per CS-VUE non-compliance recorded
Data Required
Number of non-compliances recorded during the KRA period
Page 59 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Measure 4.1.4 Environmental Initiatives
This KPI measures how many environmental initiatives the Contractor has adopted or
actioned within the KRA period that:
Meet the Principal’s environmental plan objectives
Are over and above meeting legal compliance.
Process
Whenever an initiative is put forward, the Contractor prepares a summary detailing it
using the prescribed template. The Environment and Urban Design Team assesses the
initiative and advises the Contractor whether the initiative has been agreed and can be
included in the KPI scoring.
Impact
+20 points per initiative assessed and agreed by the Principal
Data Required
Number of initiatives agreed
Measure 4.1.5 Environmental Training, Workshops and Presentations
This KPI element counts the number of attendances at formal environmental related
training workshops and presentations the Contractor has held. (This does not include
toolbox or tailgate activities.)
Process
For each KRA assessment period, the Contractor captures from their own relevant
systems the number of individual trainees attending relevant courses workshops and
presentations.
Impact
+1 point per individual attendance
Data Required
Number of courses in the KRA period and number of attending individuals
Page 60 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Measure 4.1.6 Environmental Hazards or Opportunities
This KPI element counts the number of environmental near misses, hazards and
opportunities the Contractor has reported.
Process
The Contractor maintains an ongoing register of all environmental hazards and
opportunities identified, recording for each one the:
• Number of staff who reported it
• Names of each staff member who reported it
• Type of near miss, opportunity or hazard reported.
Impact
+ 0.5 points per near miss, hazard or opportunity reported during the period, multiplied
by the number of discrete individuals that reported it
Data Required
Number of reports of near misses, hazards or opportunities identified
Calculation of Aggregated Score
The table below shows how the value for each individual KPI is calculated to find the
aggregated environmental triangle score for the KRA period.
MEASURE CALCULATION
4.1.1 Enforcement Action No. of instances × -20
4.1.2 Regulatory Non-Compliance No. of instances × -10
4.1.3 CS VUE Non-Compliance No. of instances × -10
4.1.4 Environmental Initiatives No. of instances × 20
4.1.5 Training Workshops and Presentations No. of attendances × 1
4.1.6 Hazards or Opportunities No. of reports × 0.5
Aggregated environmental triangle score Sum of the above scores
Measure Score
SCORE MEASUREMENTS
1 Aggregated environmental triangle score is less than zero (<0)
2 Aggregated environmental triangle score is between zero and ten (≥0 and ≤10)
3 Aggregated environmental triangle score is more than ten, but not more than 20 (>10 and <20)
4 Aggregated environmental triangle score is 20 or more (≥20)
Page 61 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 4.1.7 Environmental and Social Management Plan
See also Section 7.5 ‘KPIs that Measure Performance of Plans’ for more details.
Intent
Contractors make effective, consistent use of their plan to manage the impact of their
activities on the physical and social environment by using, auditing and reviewing it
regularly.
Definition
This KPI confirms how well the Contractor’s activities comply with their Environmental
and Social Management Plan8.
This KPI also measures whether the Contractor has taken action to address any non-
compliances identified.
Business Rules
The Contractor must have an Environmental and Social Management Plan accepted
by the Principal within the grace period allowed in the NOC.
The Contractor must observe the business rules for auditing their plan, and keeping
it up to date, as shown in Section 7.5.2 ‘Audits against plans’.
Process
The Contractor defines their approach for auditing activities against their plan in their
Quality Management Plan (QMP), and captures the audit data on their internal quality
management system.
8 The requirements for this plan are covered in Section 4.4 of the NOC Maintenance Specification.
Page 62 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Data Required
Most recent date updated (or new) plan has been approved by the Principal
(excluding any updates after the end of the KRA period), and by whom
Date(s) audit(s) completed and by whom
Date(s) audit report(s) accepted by the Principal and by whom
Number of non-compliances (including any repeats) discovered during the KRA
period
Number of repeat non-compliances recognised during the KRA period
Percentage of processes within the plan audited during the KRA period
Total number of sites/activities audited against the plan during the KRA period.
Measure score
SCORE DESCRIPTION MEASUREMENTS
1 Contractor has not taken action to either:
• Agree a plan in the required timeframe
• Audit activities against an accepted plan
• Record all non-compliances
• Keep the plan current and accurate.
Any of the following:
• No plan is accepted. • Plan is not up to date. • Activities are not audited against plan. • Plan is accepted and audit is completed, but any
non-compliances have not been recorded.
2 Contractor is:
• Auditing activities against a current, accepted plan
• Managing actions and learnings.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report. • All non-compliances in this period are recorded. • All repeat non-compliances are recorded.
3 Contractor is both:
• Auditing activities against a current, accepted plan
• Achieving a good level of compliance in the audit results.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report. • All non-compliances in this period are recorded,
and there are five or fewer (≤5) of these. • No repeat non-compliances.
4 Contractor is both:
• Auditing activities against a current, accepted plan
• Achieving an excellent level of compliance in the audit results.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report. • All non-compliances in this period are recorded,
and there are two or fewer (≤2) of these. • No repeat non-compliances.
Page 63 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 4.2.1 Healthy Market Pledges
Intent
Contractors are consistent and timely in meeting the pledges in their tendered
submission, and implement ongoing value-for-money opportunities to ensure that Sub-
Contractors have fair access to the market for providing services on the network.
Definition
This KPI assesses how well the Contractor is meeting their responsibilities to ensure a
sustainable and healthy Sub-Contracting market. They are expected to deliver at least:
The minimum percentage of Sub-Contractor work agreed in their NOC
Any other healthy market related pledges agreed to in their NOC
Any other healthy market pledges agreed to that they identify during the term of the
NOC.
The Contractor may choose to increase the capability of their Sub-Contractors through
systems and training. The Principal will recognise these activities as value-for-money
opportunities.
Business Rules
The Contractor must deliver on all healthy market pledges they have made in their
tendered submission. If they cannot deliver or must delay delivery, they must
provide evidence of mitigating circumstances9 agreed unanimously by the Board.
The Contractor must identify all healthy market pledges from their tendered
submission, agree a schedule with the Principal for when they will deliver these, and
maintain this schedule.
The Contractor must have a system to record all initiatives that improve the
capability of their Sub-Contractors.
The Contractor can only claim identification and implementation of value-for-money
opportunities under this KPI for value delivered over and above their scheduled
healthy market pledges.
The Contractor should calculate the individual and overall Sub-Contractor spend
percentages for this KPI in all periods, however they are only to be included in
evaluation of scoring in Period 3 of each KRA assessment year. In other KRA
periods, the Contractor must report the equivalent year-to-date percentages,
however they are to be excluded from evaluation of scoring. Scoring in Periods 1
and 2 is to be based on all other healthy market pledges due for delivery, and any
value-for-money opportunities identified.
To meet the pledged percentage of Sub-Contracted work, the Contractor must both
achieve the overall percentage pledged, and the pledged percentages for any
individual Sub-Contractors.
A description of mitigating circumstances can be seen in the table of terms used (page 6).
Page 64 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
If the Contractor claims under this KPI for a Sub-Contractor initiative that has added
value for money, they can potentially also claim it under KPI 5.3.1, if it is a genuine
innovation.
The Principal must have accepted a value-for-money opportunity, for the Contractor
to claim it under this KPI.
Process
At the start of the Contract, the Principal and Contractor agree a schedule of the healthy
market pledges made by the Contractor in their tendered submission, including the
specific planned actions and when they will be delivered. In reporting for each KRA
period, the Contractor demonstrates that they have delivered as scheduled and added
value on the promises made during tendering.
The Contractor also identifies and implements additional value-for-money opportunities
ongoing, and maintains records of these.
Sub-Contractor spend must be detailed by individual Sub-Contractor, showing
percentage pledged, percentage achieved for the current year, overall annual contract
expenditure and any comments to explain the variance. Where mitigating
circumstances10 are put forward these must be agreed by both the MCM and the
Contract Board and explained within the KRA submission.
Data Required
When the schedule of healthy market pledges was accepted, and by whom
Number of healthy market pledges due during the KRA period
How many of the healthy market pledges due were delivered
How many of the healthy market pledges due were not delivered but mitigating
circumstances were unanimously agreed by the Board
Details of any mitigating circumstances recognised by the Board
Year-to-date pledged and actual percentage of overall Sub-Contractor spend (note
this will only affect scores in Period 3. In Periods 1 & 2 it is for information only)
Year-to-date pledged and actual percentage of individual Sub-Contractor spend
(note this will only affect scores in Period 3. In Periods 1 & 2 it is for information
only)
Number of value-for-money opportunities identified
Number of value-for-money opportunities fully implemented
The details relating to those value-for-money opportunities, including how they
increase Sub-Contractor capability and provide ongoing benefit to the Sub-
Contracting market.
A description of mitigating circumstances can be seen in the table of terms used (page 6).
Page 65 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Measure Score
SCORE DEFINITION MEASUREMENTS
1 No schedule agreed, or scheduled healthy market pledges not
delivered
Either of the following:
• No schedule has been accepted • Contractor has not delivered on healthy market pledges due
within this KRA period, and has not had mitigating circumstances11 agreed.
2 Scheduled healthy market pledges delivered
For each healthy market pledge due in this KRA period, Contractor has either:
• Delivered as scheduled
• Had mitigating circumstances agreed.
3 Scheduled healthy market pledges delivered and
opportunities to add value identified
Both of the following:
• Requirements for Level 2 are met • Contractor has identified four or more additional
opportunities to add value for money.
4 Scheduled healthy market pledges delivered and opportunities to add value implemented
Both of the following:
• Requirements for Level 3 are met • Contractor has fully implemented three or more additional
opportunities to add value for money.
A description of mitigating circumstances can be seen in the table of terms used (page 6).
Page 66 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KRA 5: ASSURANCE AND VALUE
Objectives
Accurate network information and knowledge underpin quality outcomes.
Contractors make sound investment recommendations and decisions, based on
reliable, robust and proven evidence.
KPIs in this KRA
5.1.1 Quality Management Plan
5.2.1 Maintenance Management Plan
5.3.1 Innovation
Page 67 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 5.1.1 Quality Management Plan
See also Section 7.5 ‘KPIs that Measure Performance of Plans’ for more details.
Intent
Contractors make effective, consistent use of their plan to manage the quality of their
activities by using, auditing and reviewing it regularly.
Definition
This KPI measures how well the Contractor is using their quality management plan12 to
measure, review and improve value and effectiveness of their quality management
process. As part of their plan activities, the Contractor is expected to:
Identify and acknowledge any quality issues, log them in an issues register and put
a response in place
Make any changes required to the process in the quality management plan.
This KPI also measures whether the Contractor has taken action to address any non-
compliances identified.
Business Rules
The Contractor must have a quality management plan accepted by the Principal
within the grace period allowed in the NOC.
The Contractor must observe the business rules for auditing their plan, and keeping
it up to date, as shown in Section 7.5.2 ‘Audits against plans’.
The quality management plan must include the Contractor’s data quality plan.
Process
The Contractor defines their approach for auditing activities against their plan in their
Quality Management Plan (QMP), and captures the audit data on their internal quality
management system.
Data Required
Most recent date updated (or new) plan has been approved by the Principal
(excluding any updates after the end of the KRA period), and by whom
Date(s) audit(s) completed and by whom
Date(s) audit report(s) accepted by the Principal and by whom
Number of non-compliances (including any repeats) discovered during the KRA
period
Number of repeat non-compliances recognised during the KRA period
Percentage of processes within the plan audited during the KRA period
Total number of sites/activities audited against the plan during the KRA period.
12 Requirements for the Quality Management Plan are covered in Section 4.2 of the NOC Maintenance Specification.
Page 68 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Measure Score
SCORE DESCRIPTION MEASUREMENTS
1 Contractor has not taken action to either:
• Agree a plan in the required timeframe
• Audit activities against an accepted plan
• Record all non-compliances
• Keep the plan current and accurate.
Any of the following:
• No plan is accepted. • Plan is not up to date. • Activities are not audited against plan. • Plan is accepted and audit is completed, but any
non-compliances have not been recorded.
2 Contractor is:
• Auditing activities against a
current, accepted plan
• Managing actions and learnings.
All of the following:
• Plan is up to date.
• Activities in this period are audited against plan
and the Principal has accepted audit report. • All non-compliances in this period are recorded. • All repeat non-compliances are recorded.
3 Contractor is both:
• Auditing activities against a current, accepted plan
• Achieving a good level of compliance in the audit results.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report. • All non-compliances in this period are recorded,
and there are five or fewer (≤5) of these. • No repeat non-compliances.
4 Contractor is both:
• Auditing activities against a current, accepted plan
• Achieving an excellent level of
compliance in the audit results.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report.
• All non-compliances in this period are recorded, and there are two or fewer (≤2) of these.
• No repeat non-compliances.
Page 69 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 5.2.1 Maintenance Management Plan
See also Section 7.5 ‘KPIs that Measure Performance of Plans’ for more details.
Intent
Contractors make effective, consistent use of their plan to manage their maintenance
activities by using, auditing and reviewing it regularly.
Definition
This KPI measures how well the Contractor is using their maintenance management
plan13 to measure, review and improve the value and effectiveness, of their maintenance
management process.
This KPI also measures whether the Contractor has taken action to address any non-
compliances identified.
Business Rules
The Contractor must have a maintenance management plan accepted by the
Principal within the grace period allowed in the NOC.
The Contractor must observe the business rules for auditing their plan, and keeping
it up to date, as shown in Section 7.5.2 ‘Audits against plans’.
Process
The Contractor defines their approach for auditing activities against their plan in their
Quality Management Plan (QMP), and captures the audit data on their internal quality
management system.
Data Required
Most recent date updated (or new) plan has been approved by the Principal
(excluding any updates after the end of the KRA period), and by whom
Date(s) audit(s) completed and by whom
Date(s) audit report(s) accepted by the Principal and by whom
Number of non-compliances (including any repeats) discovered during the KRA
period
Number of repeat non-compliances recognised during the KRA period
Percentage of processes within the plan audited during the KRA period
Total number of sites/activities audited against the plan during the KRA period.
13 Requirements for the Maintenance Management Plan are covered in Section 4.8 of the NOC Maintenance Specification.
Page 70 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Measure Score
SCORE DESCRIPTION MEASUREMENTS
1 Contractor has not taken action to either:
• Agree a plan in the required
timeframe • Audit activities against an
accepted plan • Record all non-compliances • Keep the plan current and
accurate.
Any of the following:
• No plan is accepted. • Plan is not up to date.
• Activities are not audited against plan. • Plan is accepted and audit is completed, but any
non-compliances have not been recorded.
2 Contractor is:
• Auditing activities against a current, accepted plan
• Managing actions and
learnings.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report.
• All non-compliances in this period are recorded. • All repeat non-compliances are recorded.
3 Contractor is both:
• Auditing activities against a current, accepted plan
• Achieving a good level of compliance in the audit results.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report. • All non-compliances in this period are recorded,
and there are five or fewer (≤5) of these. • No repeat non-compliances.
4 Contractor is both:
• Auditing activities against a current, accepted plan
• Achieving an excellent level of
compliance in the audit results.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report.
• All non-compliances in this period are recorded, and there are two or fewer (≤2) of these.
• No repeat non-compliances.
Page 71 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 5.3.1 Innovation
Intent
Contractors’ organisational culture actively values innovation, efficiency and continuous
improvement, which they share across any of the Principal’s Contracts.
Innovation encourages continuous improvement and results in demonstrable benefits.
This KPI is intended to encourage an ongoing attitude that activities can be done better.
Definition
This KPI counts the number of innovations that are either common to the industry or
created within the Contracts, and which result in demonstrable benefits to delivery of the
NOC.
A proprietary innovation is a new practice that the Contractor has created within the
NOC, which has not yet been used by other suppliers or generally within the
industry.
A common innovation is a practice that the Contractor has adopted within the NOC,
which is used by other suppliers or generally within the industry.
Note:
While innovation generally offers value for money, not everything that offers value
for money necessarily qualifies as innovation
The feature must be new to the NOC.
Business Rules
The Contractor must have a system to record and capture the data required to
assess this KPI.
If the Contract has dual Principals (e.g. NZTA and a local authority), the Contractor
can claim an innovation that provides value to either Principal – or both.
The Contractor cannot claim an innovation:
For any activity that is listed as a pledge in their tendered submission
As proprietary that has been previously agreed as a tender pledge by any
Contractor (under KPIs 4.2.1 or 6.1.3)
As common that they have previously claimed as proprietary.
The Principal is responsible for keeping a register of all proprietary innovations that
are claimed in KRA assessments.
As part of the moderation process (see 4.5 ‘Moderation of Reported Performance’),
the collective group will determine the validity of the initiative or practice as an
innovation.
Page 72 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Process
The Contractor maintains their system ongoing, documenting all innovations from all
sources.
When considering the KRA assessment, the moderation groups specifically consider
and validate each innovation claimed, and advises the Contractor of the outcome.
Data Required
Evidence of implementing improvements to service delivery, including:
Date implemented
Description of nature of innovation
Whether proprietary or common
If shared, contract name(s) and number(s) where innovation is being used.
Measure Score
SCORE MEASUREMENTS
1 Contractor has introduced no innovations.
2 Contractor has introduced one or more new innovations.
3 Both of the following:
• Contractor has introduced one or more new common innovations. • Contractor has, in addition, introduced two or more proprietary innovations.
4 Both of the following:
• Requirements for Level 3 are met. • A proprietary innovation introduced by the Contractor is being used by at least two
of the Principal’s other Contracts.
Page 73 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KRA 6: NETWORK PERFORMANCE
Objectives
Activities completed under a NOC provide physical indicators of service quality.
Contractors deliver on promises made during tendering, and these add value to the
network.
Customers have timely and accurate information that lets them make informed
choices.
Contractors schedule their work to cause minimal disruption for road users.
KPIs in this KRA
6.1.1 Overall OPM Score
6.1.2 Compliance with Contractor’s Monthly Programme of Work
6.1.3 Network Performance Tender Pledges are Delivered
6.2.1 Network Availability
6.2.2 Emergency Procedures Preparedness Plan
6.2.3 Traffic Flow
Page 74 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 6.1.1 Overall OPM Score
Intent
Contractors understand all the relevant aspects of their network, so they consistently
manage the operational levels of service within acceptable parameters, avoiding any
penalties.
Definition
This KPI measures the Contractor’s achievement in delivering the Operational
Performance Measures (OPM)14 for the NOC during the KRA reporting period.
Business Rules
The Contractor is responsible for maintaining a system to capture and report their
OPM results.
To determine their score for this KPI, the Contractor must refer to the lower and
upper non-compliance limits defined in the NOC.
The measurement for this KPI must include the most recent four months of OPM
data reported.
A score of 4 can only be achieved once the Contractor has completed 12 months of
OPM reporting.
Process
The Contractor collates and analyses the monthly OPM performance evaluations for the
KRA period.
Data Required
Number of occasions where the monthly OPM score exceeded the lower non-
compliance limit in:
This KRA period
The last 12 months
Number of occasions where the monthly OPM score exceeded the upper non-
compliance limit in this KRA period
Number of non-compliances with Key and Safety OPMs in this KRA period
Total value of financial penalties incurred during the KRA period
Outputs from OPM reporting for last 12 months, where applicable.
Calculation
Calculate the trend and linear regression line using an approach consistent with the
steps in Section 7.7 ‘Trend Line Calculations’.
14 The requirements for evaluating and analysing OPM results are covered in Section 2.3.4, and Table 2.3.2 OPM Monthly Evaluation Weightings, of the NOC Maintenance Specification.
Page 75 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Measure Score
SCORE MEASUREMENTS
1 In this KRA period, either:
• Two or more monthly OPM scores have exceeded the lower non-compliance limit • One or more monthly OPM scores have exceeded the upper non-compliance limit.
2 One monthly OPM score exceeded the lower but not the upper non-compliance limit within this KRA period
3 No monthly OPM score has exceeded the lower non-compliance limit in this KRA period.
4 All of the following:
• Contractor has had no non-compliances within Key and Safety OPMs within this KRA period.
• No monthly OPM score has exceeded the lower non-compliance limit in the last 12 months.
• Contractor’s OPM score shows a steady or improving trend over the past 12 months.
Page 76 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 6.1.2 Compliance with Contractor’s Monthly
Programme of Work
Intent
Contractors plan realistic, achievable monthly programmes of work, and consistently
deliver the activities as scheduled.
Definition
This KPI measures how the timing of the Contractor’s actual work complies with the
monthly planned schedule, in alignment with their annual programme of work15.
Business Rules
KPI 6.1.2 will continue to be scored at the end of each reporting year (instead of each
reporting period)
Process
A score of ‘NA’ is to be entered into the KRA reporting framework in Periods 1 and 2
of each KRA reporting year.
The Performance team will continue work with industry representatives to confirm
the end of year KPI scoring criteria. Once this has been agreed, Contractors will be
notified by NTC.
Data Required
To be confirmed
Calculation
To be confirmed
Measure Score
SCORE MEASUREMENTS
1 To be confirmed
2 To be confirmed
3 To be confirmed
4 To be confirmed
15 Requirements for programmes of work within the Monthly Report are covered under Section 3.8.2 of the NOC Maintenance Specification.
Page 77 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 6.1.3 Network Performance Tender Pledges are
Delivered
Intent
Contractors are consistent and timely in meeting the network performance pledges in
their tendered submission and implement ongoing value-for-money opportunities to
deliver services on the network, adding value to the Contract’s performance.
Definition
This KPI measures how well the Contractor is meeting their responsibilities to deliver on
the schedule agreed with the Principal of the network performance pledges in their
tendered submission.
The Contractor is expected to deliver on all network performance pledges they have
made in their tendered submission, and demonstrate how these have added value.
Contractors are expected to deliver at least:
All the network performance related pledges agreed to in their NOC
Any other network performance pledges agreed to that they identify during the term
of the NOC.
The Contractor may choose to improve their service delivery or network performance
through further initiatives, and assessment of this KPI recognises these as value-for-
money opportunities.
Business Rules
The Contractor must deliver on all network performance pledges they have made in
their tendered submission. If they cannot deliver or must delay delivery, they must
provide evidence of mitigating circumstances16 agreed unanimously by the Board.
The Contractor must identify all network performance pledges from their tendered
submission, agree a schedule with the Principal for when they will deliver these, and
maintain this schedule.
The Contractor must have a system to record all initiatives that improve network
performance.
The Contractor can only claim identification and implementation of value-for-money
opportunities under this KPI for value delivered that are over and above their
scheduled pledges.
If the Contractor claims under this KPI for an initiative that has added value to
network performance, they can potentially also claim it under KPI 5.3.1, if it is a
genuine innovation.
A description of mitigating circumstances can be seen in the table of terms used (page 6).
Page 78 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
The Principal must have accepted a value for money opportunity, for the Contractor
to claim it under this KPI.
Process
At the start of the Contract, the Principal and Contractor agree a schedule of the network
performance pledges made by the Contractor in their tendered submission, including the
specific planned actions and when they will be delivered. In the reporting for each KRA
period, the Contractor demonstrates that they have delivered as scheduled and added
value on the promises made during tendering.
The Contractor also identifies and implements additional value-for-money opportunities
ongoing, and maintains records of these.
Data Required
When the schedule of network performance pledges was accepted, and by whom
Number of network performance pledges due during the KRA period
How many of the network performance pledges due were delivered
How many of the network performance pledges due were not delivered but
mitigating circumstances17 were unanimously agreed by the Board
Details of any mitigating circumstances recognised by the Board
Number of value for money opportunities identified
Number of value for money opportunities fully implemented
The details relating to those value for money opportunities, including how they
provide ongoing benefit to network performance.
Measure Score
SCORE DEFINITION MEASUREMENTS
1 No schedule agreed, or scheduled network performance pledges not delivered
Either of the following:
• No schedule has been accepted • Contractor has not delivered on network performance
pledges due within this KRA period, and has not had mitigating circumstances agreed.
2 Scheduled network performance pledges delivered
For each network performance pledge due in this KRA period, Contractor has either:
• Delivered as scheduled • Had mitigating circumstances agreed.
3 Scheduled network
performance pledges delivered and opportunities to add value identified
Both of the following:
• Requirements for Level 2 are met • Contractor has identified four or more additional
opportunities to add value for money.
4 Scheduled network performance pledges
delivered and opportunities to add value implemented
Both of the following:
• Requirements for Level 3 are met
• Contractor has fully implemented three or more additional opportunities to add value for money.
A description of mitigating circumstances can be seen in the table of terms used (page 6).
Page 79 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 6.2.1 Network Availability
Intent
Contractors understand how planned events will disrupt the network’s availability, and
take appropriate action to limit the impact of that disruption.
Definition
This KPI measures the proportion of occurrences where the actual disruption for planned
events is greater than predicted.
For measuring this KPI, a “planned event” includes:
Any event that requires a traffic management plan (TMP), such as renewals
Contractor’s works
Third party works.
Business Rules
The Contractor must have a system in place to capture:
The predicted and actual disruption of each event, including when the prediction
is for no disruption
Negative media coverage and customer feedback about network availability
around planned events.
To increase the consistency of the numbers reported, events that occur at the same
site over a number of days should be treated as a single event in the context of
defining the list of events from which to take a sample. Therefore an event that
occurs over multiple days at a site should not be counted more than once in the
potential pool of events for sampling, even if the event is reported daily or weekly
within other systems.
If an event occurring over a number of days is selected for inclusion in the sample,
evidence of measurement criteria relating to disruption18, is expected to to be
assessed on a daily basis, across the full duration of the event. The daily information
for all events in the sample will need to be amalgamated so that relevant
measurement criteria can be assessed as a whole for the sample, using the following
calculation:
= Total days criteria failed in the sample x100
Total sum of days of events in the sample
The Contractor is expected to audit a sample of events (ideally at least 10%. See
Section 7.4 ‘Completing Audits to Measure Performance’ (page 28)), selected
randomly from the list of those expected to affect network availability.
18 As opposed to measurement criteria relating to media coverage or customer feedback.
Page 80 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
The Contractor and the Principal must agree on what constitutes actual disruption
and how the Contractor will forecast this. Disruption includes situations such as
when:
Time taken to get through the work site is 15 minutes instead of the predicted
ten minutes
The work area needs speed restrictions for five days instead of the predicted
three days, etc.
It should be noted that assessment of planned vs actual disruption at major capital
work sites that are beyond the control of the NOC supplier can, with agreement of
the Contract Board, be excluded when assessing this KPI. Any negative media
coverage relating to those same sites should be noted in the KRA submission, but
should not be included when assessing the Contractor’s score.
Technology that becomes available during the Contract may influence what
information is used in the measure of disruption.
If any event was predicted to have no adverse effect on network availability, but did
have an effect, this event must also be included in the selection of events audited, in
addition to the random sampling.
Process
The Contractor records relevant planned events and their predicted disruption to network
availability when these are identified.
In each KRA period assessment, the Contractor:
Extracts data on the actual disruption for each planned event
Self-audits to compare this data to their predicted disruption.
Data Required
Number of all planned events during the period, regardless of whether any
disruption was predicted for them
Number of all planned events expected to affect network availability during the
period
Number of events audited (including any events added to the random sample as
a result of unpredicted adverse effects on network availability)
Percentage of total days within audited events where actual disruption was
greater than predicted (including any events added to the random sample as a
result of unpredicted adverse effects on network availability)
Brief description of any negative media coverage and customer feedback about
network availability during the KRA period
Page 81 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Measure Score
SCORE MEASUREMENTS
1 Either of the following:
• Planned events were not audited. • Audits show that more than 20% of planned events caused greater actual disruption
than predicted.
2 Both of the following:
• Audits show that more than 10% but not more than 20%, of planned events caused greater actual disruption than predicted (>10% and ≤20%)
• No negative media coverage reported about network availability during the KRA
period, as a result of the Contractor’s related action or inaction.
3 Both of the following:
• Audits show that more than 5% but not more than 10% of planned events caused greater actual disruption than predicted (>5% and ≤10%)
• No negative media coverage reported about network availability during the KRA period, as a result of the Contractor’s related action or inaction.
4 All of the following:
• Audits show that 5% or less of planned events caused greater actual disruption than predicted.
• No negative media coverage reported • No justifiable negative customer feedback received about network availability during
the KRA period, as a result of the Contractor’s related action or inaction.
Page 82 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 6.2.2 Emergency Procedures Preparedness Plan
See also Section 7.5 ‘KPIs that Measure Performance of Plans’ for more details.
Intent
Contractors make effective, consistent use of their plan to ensure they are ready to react
to incidents on the network, and continuously improve their management of incidents by
using, auditing and reviewing the plan regularly.
Definition
This KPI measures how well the Contractor is using their Emergency Procedures
Preparedness Plan19 to define the roles, practices and procedures in preparation for, and
during, an incident response event.
This KPI also measures whether the Contractor has taken action to address any non-
compliances identified.
Business Rules
The Contractor must have an Emergency Procedures Preparedness Plan accepted by
the Principal before the NOC commences.
The Contractor must have a system in place to track and record the outcome of
audits against their contract management plans.
The Contractor must observe the business rules for auditing their plan, and keeping
it up to date, as shown in Section 7.5.2 ‘Audits against plans’.
Process
The Contractor defines their approach for auditing activities against their plan in their
Quality Management Plan (QMP), and captures the audit data on their internal quality
management system.
Data Required
Most recent date updated (or new) plan has been approved by the Principal
(excluding any updates after the end of the KRA period), and by whom
Date(s) audit(s) completed and by whom
Date(s) audit report(s) accepted by the Principal and by whom
Number of non-compliances (including any repeats) discovered during the KRA
period
Number of repeat non-compliances recognised during the KRA period
Percentage of processes within the plan audited during the KRA period
Total number of sites/activities audited against the plan during the KRA period.
19 Requirements for the Emergency Procedures and Preparedness Plan are covered in Section 4.2 of the NOC Maintenance Specification.
Page 83 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Measure Score
SCORE DESCRIPTION MEASUREMENTS
1 Contractor has not taken action to either:
• Agree a plan in the required
timeframe • Audit activities against an
accepted plan • Record all non-compliances • Keep the plan current and
accurate.
Any of the following:
• No plan is accepted. • Plan is not up to date.
• Activities are not audited against plan. • Plan is accepted and audit is completed, but any
non-compliances have not been recorded.
2 Contractor is:
• Auditing activities against a current, accepted plan
• Managing actions and
learnings.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report.
• All non-compliances in this period are recorded. • All repeat non-compliances are recorded.
3 Contractor is both:
• Auditing activities against a current, accepted plan
• Achieving a good level of compliance in the audit results.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report. • All non-compliances in this period are recorded,
and there are five or fewer (≤5) of these. • No repeat non-compliances.
4 Contractor is both:
• Auditing activities against a current, accepted plan
• Achieving an excellent level of
compliance in the audit results.
All of the following:
• Plan is up to date. • Activities in this period are audited against plan
and the Principal has accepted audit report.
• All non-compliances in this period are recorded, and there are two or fewer (≤2) of these.
• No repeat non-compliances.
Page 84 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 6.2.3 Traffic Flow
Intent
Contractors take effective action to engage with the community and stakeholders on any
planned events that may have adverse effects on traffic flow. They identify and
implement mitigating measures to minimise these effects.
Definition
This KPI measures:
How the Contractor has engaged with the community and stakeholders about
planned events
What mitigations they have identified to reduce the impact on traffic flow
What mitigations they have implemented to reduce the impact on traffic flow
How effective the implemented mitigations were.
For measuring this KPI, a “planned event” includes:
Any event that requires a traffic management plan (TMP), such as renewals
Contractor’s works
Third party works
Other public events that might impact network availability, e.g. a sports event within
the network area that does not require a TMP.
Business Rules
The Contractor must have a system in place to capture the predicted and actual
disruption of each event, including when the prediction is for no disruption.
To increase the consistency of the numbers reported, events that occur at the same
site over a number of days should be treated as a single event in the context of
defining the list of events from which to take a sample. Therefore an event that
occurs over multiple days at a site should not be counted more than once in the
potential pool of events for sampling, even if the event is reported daily or weekly
within other systems.
The Contractor is expected to audit a sample of events (ideally at least 10%. See
Section 7.4 ‘Completing Audits to Measure Performance’ (page 28)), selected
randomly from the list of those expected to have an adverse effect on traffic flow.
If any event was predicted to have no adverse effect on traffic flow, but did have an
effect, this event must also be included in the selection of events audited, in addition
to the random sampling.
Process
Ongoing during the Contract, the Contractor reviews all planned events, and
records:
Whether the events are expected to impact traffic flow
Who they engaged with to inform, and when
What mitigations they identified to minimise the expected impact on traffic flow
Page 85 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
The Contractor selects an audit sample (of ideally at least 10%) of the identified
events and, as these events take place:
Monitors the events to observe and record the actual adverse effects on traffic
flow
Considers how effective the mitigations were, in minimising impact.
If an event occurring over a number of days is selected for inclusion in the sample,
engagement with stakeholders and community, mitigations identified or
implemented and adverse effects on traffic flow must be assessed for each event as
a whole. Duration of the event does not form part of the calculation.
Data Required
Number of relevant parties engaged with
Number of planned events predicted to have an adverse effect on traffic flow
Number of mitigations identified
Number of mitigations implemented
Number of events audited
Number of planned events predicted to have an adverse effect on traffic flow and did
have an adverse effect
Number of planned events predicted to have no adverse effect on traffic flow, but
did have an adverse effect
Details of those events that had an adverse effect on traffic flow.
Measure Score
SCORE MEASUREMENTS
1 Any of the following:
• Contractor has not identified any events with adverse impact on traffic flow during the KRA period
• Contractor has not engaged with stakeholders or the community during the KRA period
• Contractor has not audited any events.
2 Audit shows the Contractor has both:
• Identified events during the KRA period with adverse impact on traffic flow • Engaged with relevant stakeholders and communities about these events.
3 Audit shows both of the following:
• Requirements for Level 2 are met.
• Contractor has identified opportunities to mitigate abnormal traffic flow for planned events.
4 Audit shows all of the following:
• Requirements for Level 3 are met. • Contractor has implemented opportunities to mitigate abnormal traffic flow for
planned events.
• The planned events had no adverse effect on traffic flow.
Page 86 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KRA 7: HEALTH OF THE RELATIONSHIP
Objective
The working relationship between the Principal and Contractors:
Fosters open and honest dialogue and feedback
Involves Sub-Contractors and recognises their value.
KPIs in this KRA
7.1.1 Relationship Survey
7.1.2 Integrity and Compliance of KRA System
Page 87 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 7.1.1 Relationship Survey
Intent
Survey results show that the relationship between the parties is healthy, and the parties
are collaborating effectively.
Definition
This KPI monitors the general health of the relationship between participating parties
(including Sub-Contractors) in the context of this Contract, and the opportunities they
are taking to improve the level of collaboration.
Business Rule
The results of evaluating this KPI are not shown as a score. It is excluded when
calculating the Contractor’s performance level for KRA period and annual
assessments.
The Principal uses the results of this survey in ranking Contractors with equal
results, to produce a leaderboard.
Process
Every six months, NZTA’s Contract Management Team carries out a Network Outcomes
Contract Relationship Survey of all parties to each Contract, then presents the results to
the Contract Board.
The Contract Board then:
Assesses the results to identify opportunities to improve relationships between the
Parties
Engages with the Contractor (and Sub-Contractors, as relevant) to work jointly
toward achieving improvements.
Data Required
Outputs from survey
Page 88 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
KPI Measure 7.1.2 Integrity and Compliance of KRA System
Intent
Audit results show that the KRA Framework’s system has integrity, and is fit for its
purpose, i.e. Contractors are achieving good compliance.
Definition
This KPI evaluates whether the Principal’s audits of KRA assessments results show that
the KRA performance management system demonstrates integrity and compliance.
Business Rule
The results of evaluating this KPI are not shown as a score. It is excluded when
calculating the Contractor’s performance level for KRA period and annual assessments.
Process
The Principal undertakes an annual audit of the KRA system as required, to confirm that
KRA assessment results continue to be effectively measured under this framework. The
Principal uses the audit results to:
Assess whether the KRA system continues to be fit for its purpose
Identify and implement a timely resolution for any problems with the system
Identify and implement any opportunities for improvement in the system.
Data Required
All KRA assessment results at and since the last evaluation
Page 89 of 91
Version 4.2, November 2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Appendices
Appendix 1: Performance Assessment Timetable
The timetable outlined in the table below applies to each KRA year.
ACTION RESPONSIBILITY DATE – PERIOD 1
DATE – PERIOD 2
DATE – PERIOD 3
Assessment period ends (Not applicable) 31 October 28 February 30 June
Prepare NOC performance report
Contract
Management Team
Before board meeting
Before
board meeting
Before board meeting
Meet to consider NOC performance report
Maintenance Contract Manager
1 – 30 November
1 – 31 March
1 – 31 July
Contract Management Board consider and endorse NOC performance report, and send to Performance & Planning
Team
Maintenance Contract Manager
By 1 December
By 1 April By 1 August
Meet for moderation session to consider consolidated KRA results and recommend final scores.
Planning & Performance Team
1 – 31 December
1 – 30 April 1 –31 August
Present consolidated
performance report and CB recommendations to the relevant approver, for consideration
Planning & Performance Team
(Not
applicable)
(Not
applicable)
By mid –
September
Inform MCMs of outcomes Planning & Performance Team
By 31 January
By 30 May 30
September
Publish performance assessment results externally
Planning & Performance
Team
(Not applicable)
(Not applicable)
By 30 November
Page 90 of 91
Version 4.2, November
2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Appendix 2: Reporting Template
NZTA provides a spreadsheet template for Contractors to complete in Microsoft Excel, in the format below. (See Section 4.3 ‘Reporting
Templates’ for more details, and Section 7.5 ‘KPIs that Measure Performance of Plans’ for an exception.)
NB: Tempate versions are validated, and out of date templates cannot be uploaded. To prevent rework, please contact
[email protected] to obtain the current admissible template version.
KRA KPIMeasurement
#Description Measurement Value Score Comment
1Health and Safety
1 1.1.1 Total Recordable Injury Frequency Rate
1 1.1.1 1 Total work hours completed during the last 12 months
1 1.1.1 2 Fatal injuries in last 12 months (FI)
1 1.1.1 3 Notifiable injuries in last 12 months (SH)
1 1.1.1 4 Lost time injuries in last 12 months (LTI)
1 1.1.1 5 Medical treatment injuries in last 12 months (MTI)
1 1.1.1 6 Employees on restricted work duties in last 12 months (RWI)
1 1.1.1 7 Total recordable injury Frequency Rate (TRIFR) #DIV/0!
1 1.1.1 8 Has the score above been adjusted with agreement of the board. (If yes please
provide details in the comment field)
No
Page 91 of 91
Version 4.2, November
2017
Guide to the KRA Performance NZ Transport Agency
Framework Network Outcomes Contract
Appendix 3: Scoring Template
NZTA provides a spreadsheet template in Microsoft Excel, in the below format, for the Contractor to calculate their self-assessed KRA
levels for each assessment period and annually, and the annual result. (See Section 4.3 ‘Reporting Templates’ for more details.)
2017/18 KRA Assessment KRA Score Rating Score Rating Score Rating
KRA KRA Description KPI Measure Description Oct Feb Jun 1
1 Health and Safety 1.1.1 Total Recordable Injury Frequency Rate 2
1.1.2 Health and Safety Management Plan 3
1.2.1 Traffic Control Plan 4
2 Road User Safety 2.1.1 Crash Trend 5
2.1.2 Loss of Control in Darkness 6
2.1.3 Network Safety Trend Report Overall
3 Customer 3.1.1 Customer and Stakeholder Communications Management Plan
3.1.2 Communication of Impact of Events
3.1.3 Customer Satisfaction KRA Score Rating
3.2.1 Ability to Respond to Customers 1
4 Sustainability 4.1.1 Environmental Triangle 2
4.1.7 Environmental and Sustainability Management Plan 3
4.2.1 Healthy Market Pledges 4
5 Assurance and Value 5.1.1 Quality Management Plan 5
5.2.1 Maintenance Management Plan 6
5.3.1 Innovation
6 Network Performance 6.1.1 Overall OPM Score
6.1.2 Compliance with Monthly Programme of Work
6.1.3 Network Performance Tender Pledges
6.2.1 Network Availability
6.2.2 Emergency Planning and Preparedness Plan
6.2.3 Traffic Flow
Oct Feb Jun
Annual KRAs
Annual Performance Assessment
Score Rating
NOC Input