OB15 – Paper 06 – KPI Report

KPI and SLA regime: May 2014 performance summary Reference Outcome Result Target Description

KPI A Fail 99.29% 100% Service Restoration within 10 working days where

household is a primary DTT user

KPI A1 Pass 99.35% 95% Installer visits completed as scheduled with viewer

KPI A2 Pass 93.76% 50% Installer visits completed within 3 working days

KPI A3 Pass 99.65% 85% Installer visits completed within 7 working days

KPI A4 Pass 99.53% 50% Vulnerable visits completed within 3 working days

KPI A5 Pass 100% 99% Vulnerable visits completed within 7 working days

KPI A6 Fail 95.00% 95% Communal installer visits completed as scheduled

with landlord where access and liability is confirmed

KPI A7 Pass 100% 86% Reactive filters issued in 2 working days

KPI A8 Pass 100% 94% Reactive filters issued in 4 working days

KPI A9 Pass 100% 99% Reactive filters issued in 5 working days

[KPI A10] N/A N/A 100% [SUSPENDED FOR REACTIVE-ONLY TRIAL]

Addresses in very high pixels mailed filters

SLA B1 Pass 100% 100% All identifiable addresses within forecast medium,

high and very high risk pixels (high risk only for

London) to be mailed at least once between 3 and 12

weeks ahead of scheduled mast activation

SLA B2

Qualitative and quantitative

reports provided. No specific

targets.

Report on households mailed and not mailed within

forecast pixels (i.e. number of low risk households)

SLA B3 Report on households reporting interference within

un-mailed forecast pixels.

SLA B4 Awareness to be maintained (reminder mailings) with

those viewers at continued risk of experiencing

interference from masts that were not activated as

scheduled.

SLA B5 Awareness to be maintained (reminder mailings) with

those viewers at renewed risk of experiencing

interference from infill masts where the addition of

the mast increases the risk of interference in that

pixel to very high

SLA B6 Report on households mailed and not mailed

reminders within forecast pixels (i.e. number of low

risk households)

SLA B7 Report on above the line comms and research

outcomes, including awareness in rollout areas

SLA C1 0.02% Pass 0.5% Complaint about quality of interaction

SLA C2 0.25% Pass 0.5% Complaint about quality of problem resolution

SLA C3 0.11% Pass 0.5% Complaint about timeliness of problem resolution

SLA C4

Qualitative and quantitative

reports provided. No specific

targets.

Report on total number and categories of complaints

SLA D1 Measure of the numbers of mailings per report of /

case of interference

SLA D2 Measure of the number of reported cases that occur

outside of mailed area

SLA D3 Measure of number of cases of interference per mast

remaining between 0.17 and 1.66 (5k to 50k cases of

interference for rollout)

May 2014: KPI and SLA performance summary, 10 June 2014

Page 2 of 10

KPI A – Service Restoration

KPI A: Service Restoration: 100% within 10 working days where household is a primary DTT user

(communal households and households where cable or satellite services are received are excluded)

Result: 99.29%

Of the 282 confirmed 4G cases in May 2014, 280 (99.29%) were resolved within 10 working

days

There were two breach cases, outlined below:

o Case 7797872

A mailing and proactive filter were sent in summer 2013

A mast was activated 282 metres from the property on 6 November 2013

A reminder mailing was sent in January 2014

The viewer called on 28 April 2014 citing reception problems suffered since before

Christmas

An installer visit on 3 May claimed to resolve the 4G interference being suffered by

installing a filter before the masthead amplifier, but the viewer refused to sign the

form

We made multiple attempts to get in touch with the viewer, finally getting back in

touch on 21 May. The viewer reported ongoing reception problems, and we

booked a further installer visit on 23 May

During this visit, it was discovered that the ongoing reception issues were not a

result of 4G, but that they could be resolved by removing a second signal booster

in the loft space and fitting a four-way splitter

On 28 May, we made a decision to arrange a further installer visit to perform the

above actions for the viewer, as these measures should have been actioned during

the original installer visit. The viewer signed the form after the visit on 29 May

Case closed in 21 working days. Map is below:

May 2014: KPI and SLA performance summary, 10 June 2014

Page 3 of 10

o Case 8488283

The viewer originally contacted us via the webform on 2 October 2013 reporting

interference but was ruled out in triage

A review of properties missed from interference predictions due to low in-pixel

population over-ruled the original triage and led us to re-contact the viewer and

book an installer visit for 5 November. Our installer indicated cables needed

replacing, and that there was no 4G interference. The viewer was reportedly happy

with the situation

An audit visit on 10 December indicated that, despite low DTT readings,

interference was related to 4G. However, an administrative error meant that the

case was not re-opened

The viewer used the webform again on 8 March to indicate that she was still

waiting to hear back from us. After an attempted phone call by us on 11 March, we

followed up by email on 12 March to confirm that the issue was not 4G. This was

because of a mix-up of audit forms

The viewer responded by email on 18 March to tell us that the auditor had

confirmed that the problem was 4G-related and that they were promised that it

would be addressed

We called back on 19 March to attempt to book an appointment. A delayed

response by the viewer meant that the visit was booked for 11 April. During this

visit, against our clear instruction to ‘fix it no matter what’, the installer undertook

a standard visit and confirmed that they had done the job right the first time and it

was not a 4G issue

The viewer called us on 14 April adamant that it was a 4G issue

We picked up the case on 1 May, and an installer visit was booked for 9 May,

delayed at the request of the viewer, during which a new aerial was installed that

resolved the problem

Case closed in 105 working days (based on an opening date of 10 December 2013).

Map is below.

Failed.

May 2014: KPI and SLA performance summary, 10 June 2014

Page 4 of 10

KPI A1: Installer visits completed as scheduled with viewer: 95%

Result: 99.35%

Of the 925 installer visits originally scheduled in May 2014:

o 860 were undertaken and closed on schedule

o 6 visits were cancelled or missed by the installer

o 59 visits were cancelled by the viewer

Total scheduled visits not missed by installer / total visits scheduled

(860 + 59) / (860 + 59 + 6) = 919 / 925

Result: 99.35%

Passed.

KPI A2: Installer visits completed within 3 working days: 50%

Result: 93.76%

Of the 925 scheduled installer visits detailed above, there were 866 visits scheduled where

the viewer did not cancel

Of these 866, 812 (93.76%) were completed within three working days

Passed.

KPI A3: Installer visits completed within 7 working days: 85%

Result: 99.65%

Of the 866 visits, 863 (99.65%) were completed within seven working days

Passed.

KPI A4: Vulnerable visits completed within 3 working days: 50%

Result: 99.53%

In May 2014, there were 211 scheduled installer visits to people marked as vulnerable,

where the viewer did not cancel

Of the 211 visits to be measured, 210 (99.53%) were completed within three working days

Passed.

KPI A5: Vulnerable visits completed within 7 working days: 99%

Result: 100%

All of the 211 visits to be measured were completed within seven working days, the longest

taking four working days

Passed.

KPI A6: Communal installer visits completed as scheduled with landlord where access and liability is

confirmed: 95%

Result: 94.74%

May 2014: KPI and SLA performance summary, 10 June 2014

Page 5 of 10

Of the 40 communal jobs in the month of May 2014:

o 2 were cancelled by the installer

o 2 were cancelled by the viewer

o 36 were completed as agreed with the customer

Therefore, of the 40 jobs scheduled, 38 were either completed as scheduled or cancelled by

the viewer giving a success rate of 95%

Of the two jobs cancelled by the installer:

o One was a clerical error that resulted in the appointment not appearing in our

systems

o The other was booked for the 8am–1pm Tuesday slot, but morning appointments

took longer than expected, the installer called the viewer to say that he might be a

little late and the viewer had to go to work at 1pm. The appointment was

rescheduled and completed the following Saturday morning.

Passed.

KPI A7: Reactive filters issued in 2 working days: 86%

Result: 100%

There were 1,066 filter requests in May 2014. All of these were issued within one working

day, and hence passed the two working day measure

Passed.

KPI A8: Reactive filters issued in 4 working days: 94%

Result: 100%

See KPI A7

Passed.

KPI A9: Reactive filters issued in 5 working days: 99%

Result: 100%

See KPI A7

Passed.

KPI A10 [SUSPENDED]: 100% of identifiable addresses in very high pixels mailed proactive filter.

Not applicable.

May 2014: KPI and SLA performance summary, 10 June 2014

Page 6 of 10

SLA B – Awareness

SLA B1: 100% of identifiable addresses within forecast medium and high risk pixels (high risk only for

London) to be mailed at least once between 3 and 12 weeks ahead of scheduled mast activation

Result: 100%

A total of 90,393 households were identified within “very high”, “high” or “medium” risk

pixels in the June 2014 rollout data and all were mailed on 2 May 2014

Passed.

SLA B2: Report on households mailed and not mailed within forecast pixels (i.e. number of low risk

households)

The following table shows the breakdown of households by risk category and those confirmed as

mailed.

Mailed Date V_HIGH HIGH MED LON MED LOW Grand Total

2 May 2014 33,199 24,294 32,900

90,393

Not mailed 14,653 38,545 53,198

SLA B3: Report on households reporting interference within un-mailed forecast pixels. More than 40%

of total reports of interference – DMSL to address through operational adjustments (e.g. mailing low

risk pixels in a certain area or adjusting the risk score for pixels), however, a de minimis threshold to

apply.

In May 2014, we were contacted by a total of 1,826 households reporting interference that

we believed may be caused by 4G

Of these, 252 (14%) contacted us in May who were suppressed by the risk-based mailing

A further 105 (6%) contacted us who would have remained un-mailed under the previous

model.

SLA B4: Awareness to be maintained (reminder mailings) with those viewers at continued risk of

experiencing interference from masts that were not activated as scheduled. Report on reminder

mailing numbers to be provided.

A reminder mailing was undertaken on 20 May 2014 to 68,024 households forecast to be

affected by masts originally expected to activate in December 2013.

SLA B5 Awareness to be maintained (reminder mailings) with those viewers at renewed risk of

experiencing interference from infill masts where the addition of the mast increases the risk of

interference in that pixel – low to medium, new mail; medium to high, reminder mail

This activity is not distinguished from that detailed in the summary of SLA B4.

May 2014: KPI and SLA performance summary, 10 June 2014

Page 7 of 10

SLA B6: Report on households mailed and not mailed reminders within forecast pixels (i.e. number of

low risk households)

The following table shows the breakdown of households by risk category and those confirmed as

mailed.

Mailed Date V_HIGH HIGH MED LON_MED LOW Grand Total

20 May 2014 496 31,552 35,976

69,023

Not mailed

52,598 85,858 138,456

SLA B7: Report on above the line communications and research outcomes, including awareness in

rollout areas

There were no ‘new’ areas to support in May, and so we did not run any PR, or above-the-line or

digital advertising. Similarly, we did not run any research.

PR, above-the-line and digital advertising has now begun in Belfast on 9 June. This will be followed

by online research.

We engaged with local MPs, councils and social housing providers in South Wales in line with

mailings.

Background: DMSL runs above-the-line, PR or digital advertising communication when masts are

activated/the operators begin offering 4G services (at 800 MHz), and not necessarily in line with

DMSL’s mailing activity. Prior to mast activations/4G at 800 MHz launch, our primary means of

communication with viewers is direct mail postcards.

May 2014: KPI and SLA performance summary, 10 June 2014

Page 8 of 10

SLA C – Quality of Service

SLA C1: No more than 0.5% of households interacting with DMSL make a formal complaint about

quality of interaction

Result: 0.02%

There were a total of 4,365 household interactions in May 2014

There was 1 interaction that could constitute this type of complaint in May 2014. This

amounts to a complaint rate of 0.02%

Passed.

SLA C2: No more than 0.5% of households interacting with DMSL make a formal complaint about

quality of problem resolution

Result: 0.25%

There were a total of 11 interactions that could constitute this type of complaint in May

2014, out of a total of 4,365 interactions. This amounts to a complaint rate of 0.25%

Passed.

SLA C3: No more than 0.5% of households interacting with DMSL make a formal complaint about

timeliness of problem resolution

Result: 0.11%

There were a total of 5 interactions that could constitute this type of complaint in May 2014,

out of a total of 4,365 interactions. This amounts to a total complaint rate of 0.11%

Passed.

SLA C4: Report on total number and categories of complaints

Below is a breakdown of the 37 complaints received in May 2014

Compensation: 15

Installer-related: 6

Rollout in general: 6

Diagnostics over the phone: 4

Non-4G issue that we cannot help with: 3

Communal process: 2

Contact centre: 1.

[SLA C5:] Vulnerable viewer complaints

Out of 416 interactions with viewers flagged as being vulnerable in May 2014, four (1.0%)

complained. Three of these related to the installer; the other one related to compensation.

May 2014: KPI and SLA performance summary, 10 June 2014

Page 9 of 10

SLA D – Scale

SLA D1: Measure of the numbers of mailings per report of / case of interference

The following table shows the number of mailings per reported case and confirmed case of

interference to date for May 2014 rollout households that relate to masts planned to be

activated in May 2014.

Number of mailings per report of interference

∞

Number of mailings per confirmed case of interference

∞

SLA D2: Measure of the number of reported cases that occur outside of mailed area

Of the 282 households identified as experiencing 4G interference in May 2014:

May cases not mailed due to risk profile or low pop in pixel

4 (1.4%)

May cases never identified in modelling 2 (0.7%)

Total May cases outside mailed area 6 (2.1%)

Total May cases caught by mailing 276 (97.9%)

SLA D3: Measure of number of cases of interference per mast remaining between 0.17 and 1.66 (5k

to 50k cases of interference for rollout) exceeding or trending towards exceeding the range will be

raised to the OB for review / action

May 2014: KPI and SLA performance summary, 10 June 2014

Page 10 of 10

Appendix: Tracking against previous KPI A breaches

There were two breaches against KPI A in April 2014 against which we monitored progress.

Case 1 (4919715) - since this case, three further cases have been raised within the same postcode

district (G73). Of these, two are in the same postcode sector (G73 2). All closed in a timely fashion,

and none constituted a breach against KPI A.

Case 2 (7626420) - since this case, 15 further cases of 4G interference have been raised within the

same postcode district (EH9). Eight are in the same postcode sector (EH9 1). All were closed in a

timely fashion, and none constituted a breach against KPI A. A further case is not yet completed.