32
Review of the Television Captioning Standard Final report MARCH 2016
Review of the Television Captioning Standard Final reportMARCH 2016
CanberraRed Building Benjamin OfficesChan Street Belconnen ACT
PO Box 78Belconnen ACT 2616
T +61 2 6219 5555F +61 2 6219 5353
MelbourneLevel 32 Melbourne Central Tower360 Elizabeth Street Melbourne VIC
PO Box 13112Law Courts Melbourne VIC 8010
T +61 3 9963 6800F +61 3 9963 6899
SydneyLevel 5 The Bay Centre65 Pirrama Road Pyrmont NSW
PO Box Q500Queen Victoria Building NSW 1230
T +61 2 9334 7700 or 1800 226 667F +61 2 9334 7799
Copyright notice
http://creativecommons.org/licenses/by/3.0/au/
With the exception of coats of arms, logos, emblems, images, other third-party material or devices protected by a trademark, this content is licensed under the Creative Commons Australia Attribution 3.0 Licence.
We request attribution as © Commonwealth of Australia (Australian Communications and Media Authority) 2016.
All other rights are reserved.
The Australian Communications and Media Authority has undertaken reasonable enquiries to identify material owned by third parties and secure permission for its reproduction. Permission may need to be obtained from third parties to re-use their material.
Written enquiries may be sent to:
Manager, Editorial and DesignPO Box 13112Law CourtsMelbourne VIC 8010Email: [email protected]
Introduction
Decision
Submissions to the Review of the Television Captioning Standard discussion paperSubmissions to the Review 4
Discussion of Option 1—Non-metric standardSubmissions 5Reasons put forward for a non-metric standard 5Reasons put forward against a non-metric standard 5The ACMA’s response 6
Discussion of Option 2—Metric standardSubmissions 7Reasons put forward for a metric standard 7Reasons put forward against a metric standard 7The ACMA’s response 8
Discussion of alternative option/sSubmissions 10Reasons put forward for an alternative option 10The ACMA's response 10
Discussion of proposed amendments to the Standard
Recent international developmentsCanada 15United Kingdom 16
Appendix A—Summary of the public consultation process
Appendix B—Summary of the submissions process
acma | iii
Introduction
The Australian Communications and Media Authority (the ACMA) is Australia’s regulator for broadcasting, the internet, radiocommunications and telecommunications.
The ACMA is responsible for the implementation and enforcement of captioning rules in the Broadcasting Services Act 1992 (the BSA) and in subordinate instruments, such as the Broadcasting Services (Television Captioning) Standard 2013 (the Standard).
The ACMA was required under the BSA to review, and vary as appropriate, the Standard by 19 March 2016 (the Review).
In reviewing the Standard, the ACMA was required to consider the differences between providing captioning services for live, part-live and pre-recorded television programs. ‘Part-live’ refers to television programs that include both live and pre-recorded program material.
Under subsection 130ZZA(2B) of the BSA, the ACMA is not authorised to determine that a lower quality of captioning service is acceptable for a kind of program or program material (where quality includes readability, comprehensibility and accuracy see subsection 130ZZA(2)). This includes not being able to determine that a lower quality of captioning service is acceptable for:> live television programs as opposed to pre-recorded television programs, or
television programs that include both live and pre-recorded program materials> specific genres of television programs.
The focus of the Review has been on the quality of captioning services as they relate to the differences between live and pre-recorded television programs, and television programs that include both live and pre-recorded program materials.
As part of the Review, the ACMA:> considered the application of the Standard to captioning services for live and pre-
recorded television programs, and television programs that include both live and pre-recorded program material
> reviewed international approaches and current research > considered the approaches adopted for captioning services for live and pre-
recorded television programs, and television programs that include both live and pre-recorded content
> consulted with key stakeholders by:> holding a Citizen conversations forum on live captioning titled Live captioning:
let’s talk1 on 15 September 2015, which was attended by free-to-air and subscription television broadcasters and narrowcasters (hereafter referred to as broadcasters), captioning service providers, members of the public, advocacy groups and government agencies
> seeking input in September and October 2015 from key stakeholders about the Standard and its application for captioning services for live and pre-recorded
1 www.acma.gov.au/theACMA/live-captioning-lets-talk-register
acma | 1
television programs and television programs that include both live and pre-recorded content
> releasing the Review of the Television Captioning Standard discussion paper (the discussion paper) in November 2015, which outlined possible options for addressing a range of issues regarding the quality of captioning services for live and pre-recorded television programs, and television programs that include both live and pre-recorded program material.2
This report outlines the results of the Review and the decision to vary the Standard.
Under section 130ZZE of the BSA, the ACMA is also required to conduct a separate, broader statutory review of the operation of Part 9D of the BSA by 31 December 2016 (the Statutory Review). Part 9D of the BSA aims to assist deaf and hearing-impaired viewers by requiring Australian free-to-air broadcasters and subscription television licensees to meet specified levels of captioning for television programs. Under Part 9D of the BSA, free-to-air broadcasters and subscription television licensees are also required to meet captioning targets and record-keeping and reporting requirements, as well as comply with standards relating to quality.
Issues that fall outside of the Review may be considered as part of the broader Statutory Review.
2 www.acma.gov.au/theACMA/Consultations/Consultations/Current/review-of-tv-captioning-standard
2 | acma
Decision After reviewing the submissions received in response to the discussion paper, as well as reviewing and analysing international approaches and relevant research current at the time of the Review, the ACMA considers that a non-metric approach (as outlined in Option 1 of this paper) is the most suitable way of assessing the quality of captioning services under the current legislative framework.
The ACMA considers that the objective of the Standard is to ensure that the quality of a captioning service is meaningful to deaf and hearing-impaired viewers. In order for a captioning service to be meaningful, the viewer must, in accordance with the definition of a captioning service in the Standard, be able to follow the speakers, dialogue, action, sound effects and music of a program.
This ‘meaningful access’ test is best achieved by reference to a range of non-metric factors relating to the accuracy, readability and comprehensibility of a captioning service.
Further, when assessing the quality of a captioning service for a program, the quality of a captioning service is best determined by considering the context of the program as a whole, the circumstances of the broadcast and the nature of the program being broadcast.
As a result of the Review, the ACMA has not made any substantive changes to the current Standard. However, the Standard will be varied to include the following Note under paragraph 6 of the Standard:
Note: Whilst noting that it is not authorised to determine that a lower quality of captioning service is acceptable for a kind of program or program material (see subsection 130ZZA(2B) of the Act), in determining this Standard, the ACMA has considered the differences (including time constraints for live content) between providing captioning services for live and pre-recorded television programs; and wholly live or wholly pre-recorded television programs and television programs that include both live and pre-recorded program material (see subsection 130ZZA(2A) of the Act).
The non-metric approach to determining the quality of captioning provides a broader, less prescriptive framework for assessing the quality of a captioning service. It is outcome-based and places emphasis on the viewer experience. It also allows for greater flexibility and consistency with the Standard’s objective of ensuring captioning services are meaningful to deaf and hearing-impaired viewers.
The ACMA will continue to assess the quality of captioning services on a case-by-case basis, taking into account relevant factors that apply in the context of the program as a whole.
acma | 3
Submissions to the Review of the Television Captioning Standard discussion paper On 17 November 2015, as part of its broader consultation process for the Review, the ACMA released the discussion paper for public comment. The submission period ended on 18 December 2015. Further details of the public consultation process are contained in Appendix A.
The discussion paper addressed issues raised by stakeholders during the earlier phases of the consultation process for the Review, including the ACMA Citizen conversations forum, Live captioning: let’s talk.
These issues included how the Standard is applied to captioning services for live and pre-recorded television programs, and to television programs that include both live and pre-recorded program materials. The discussion paper also examined how the Standard is applied when the ACMA investigates complaints about the quality of captioning services. The discussion paper included a summary of international approaches for assessing the quality of captions and possible future developments, and identified ways in which the Standard could respond to the differences between providing captioning services for live and pre-recorded programs.
The discussion paper nominated two options for assessing the quality of captioning services:1. a non-metric approach, which focuses on whether a captioning service is
meaningful to a deaf and/or hearing-impaired viewer2. a metric approach, which requires the establishment of metrics for, among other
things, the accuracy, comprehensibility and readability of captions provided as part of a captioning service.
The discussion paper also provided the opportunity for stakeholders to submit an alternative option for how the ACMA may consider the differences in quality between providing captioning services for live, part-live and pre-recorded television programs, without determining that a lower quality of captioning service for different kinds of television programs or program material is acceptable.
Submissions to the ReviewThe ACMA received 11 submissions by 18 December 2015. Further details of these submissions are contained in Appendix B. The submissions are also published on the ACMA website.3
The submissions received also raised issues outside the scope of the Review. A summary of these issues is included in Appendix A.
3 www.acma.gov.au/theACMA/Consultations/Consultations/Current/review-of-tv-captioning-standard
4 | acma
Discussion of Option 1—Non-metric standardOption 1: A non-metric standard for determining the quality of captioning services
Submissions Nine of the 11 submissions received indicated a preference for Option 1, being a non-metric standard for determining the quality of captioning services.
Option 1 is in effect a continuation of the Standard, with its emphasis on ensuring captioning services are meaningful for deaf and hearing-impaired viewers.
Reasons put forward for a non-metric standardMost submitters were in favour of the non-metric approach to determining the quality of captioning for the following reasons:> Greater flexibility relating to captioning services—a non-metric approach
allows for useful flexibility, given the variable nature and circumstances of captioning services, and reduces compliance and production costs associated with captioning services.
> Greater consistency in assessing meaning and quality—the broad range of non-metric factors currently used in the Standard allows for meaningfulness, and therefore quality, to be assessed more consistently.
> Nuance and subtlety in meaning—a non-metric approach allows for the variable nature and circumstances of programs to be considered when determining the quality of captioning. This also allows for nuances in meaning (or the lack thereof) to be determined in the context of the length and content of the program as a whole.
> Accommodates differences in programming—a non-metric approach takes into account significant differences between styles of programming (for example, sport, drama, entertainment, news, or music programs).
> Emphasis on viewer experience—a non-metric approach is more outcome and viewer focused and involves a less prescriptive framework in assessing the quality of captioning.
> Consistency with international approaches to the quality of captioning—a non-metric approach is consistent with the prevailing international approaches to the quality of captioning.
Reasons put forward against a non-metric standardOne submitter was not in favour of the non-metric approach to determining the quality of captioning for the following reasons:> Lack of defined criteria—a non-metric approach does not have defined criteria or
a set of parameters for determining the quality of captioning and may not hold live, part-live or pre-recorded programs to the highest standard possible.
> Difficulty to measure quality over time—a non-metric approach is less measurable and cannot measure improvements or decline in the quality of captioning over time.
> Less precise and more subjective—a non-metric approach is less precise and more subjective.
acma | 5
The ACMA’s responseBased on the arguments made in the majority of the submissions, the ACMA prefers a non-metric approach for assessing the quality of captioning services and for accommodating the differences between providing captioning services for live, part-live and pre-recorded television programs.
Under the current Standard, the quality of a captioning service is determined by how meaningful it is for deaf and hearing-impaired viewers. This ‘meaningful access’ test is conducted with reference to a range of non-metric factors relating to the accuracy, readability and comprehensibility of the captioning service.
According to the definition of a captioning service in the Standard, in order for a captioning service to be meaningful, the viewer must be able to follow the speakers, dialogue, action, sound effects and music of a program.
When determining the quality of a captioning service for a program, the captioning service must be considered in the context of the program as a whole, the circumstances of the broadcast and the nature of the program being broadcast.
6 | acma
Discussion of Option 2—Metric standard Option 2: Metric standard/s for determining the quality of captioning services
Submissions One of the 11 submissions received indicated a preference for Option 2, being a metric standard for determining the quality of captioning services.
Option 2 sought information from submitters about the suitability of a one-size-fits-all metric standard that established metrics for readability, accuracy and comprehensibility. It also sought information for suggested metrics, the impact on quality and the impact on relevant stakeholders, including the ACMA.
Reasons put forward for a metric standardThe submission in favour of the metric approach to determine the quality of captioning recommended the following metrics:> at least 95 per cent accuracy and no more than a three to five second delay for live
programs> at least 99 per cent accuracy and no more than a one to two second delay for pre-
recorded programs.
The submission in favour of the metric approach did not provide reasons, but did suggest that further work must be done to determine what type of captions can be delivered and in what context. The submission also called for further consultation so that ‘best practice’ guidelines may be developed and agreed by consumers and broadcasters.
Another submission noted that metrics can be a valuable tool in assessing the quality of live or near-live captioning, but did not recommend Option 2. This submission argued that a metric approach would require the acceptability threshold for various metrics to be set low, as captions for live programs are likely to have more errors and time delays than captions for pre-recorded programs. Without setting the metric low, live programs would not be likely to meet a metric standard. Consequently, this could potentially drive down the overall quality of live captioning in Australia.
Reasons put forward against a metric standardThe majority of submitters were not in favour of the metric approach to determine the quality of captioning for the following reasons:> Overly complex—a metric approach would need to address the range of factors
relating to accuracy, readability and comprehensibility and would involve a large number of variables. The development of a single metric or set of metrics would be highly complex and may be impractical to measure and assess the meaningfulness of the captioning services to the viewer.
> Lack of flexibility regarding mix of programming—a metric approach would be too rigid and not be able to account for the differences in programming. For example, it would not be able to take into account differences in the quantity or quality of information in a news program versus a sports program when providing a
acma | 7
captioning service. Further, the current NER metric method can only be applied to a relatively small sample of programs.4
> Less focused on meaning for the viewer—a metric approach may be able to measure accuracy of captions but would not be able to measure the subjective nature of factors relating to readability and comprehensibility. For example, one submission noted that when using the current NER metric method, only the accuracy of a caption file could be measured against the caption text. This would significantly limit any assessment of whether the captioning service is meaningful to the viewer. A metric approach would also lack flexibility in considering the meaning and quality within the context of a program as a whole.
> Lower quality of captioning services for pre-recorded programs—as a metric approach would need to be a one-size-fits-all model, this may result in the lowering of the quality of captioning services for pre-recorded programs, given that captioning services for live and part-live programs involves the inherent risk of more inaccuracies.
> Significant increase in resourcing and compliance costs—submitters noted that a metric approach would require significantly more resources to measure and assess.
> Lack of flexibility relating to captioning services—a metric approach would not be able to take into account the use of paraphrasing and other measures designed to enhance meaning to the viewer by minimising delay and increasing comprehensibility and readability of captioning services.
> Proposed metric models still being determined—as noted in one submission, the NER model cited in the discussion paper and used by the UK is still relatively new and still requires a subjective assessment about the seriousness of each error. Further, the NER model is yet to address other factors such as delay, colour and position, block scrolling or other presentation factors.
> Future advances in technology—a metric approach would not capture advances in technology or techniques of providing captioning services that may improve the quality of captioning services.
The ACMA’s response Based on the arguments made in the majority of the submissions, the ACMA does not support a metric approach for assessing the quality of captioning.
The ACMA considers that the introduction of a metric approach would risk being too onerous on providers of captioning services for live and part-live programs, or too lenient on providers of captioning services for pre-recorded programs.
Further, a metric approach would risk lowering the quality of captioning across all programs (for example, pre-recorded programs) due to the need to assess live, part-live and pre-recorded programs and program material at the same standard.
A metric approach could also be less flexible with respect to improvements in the quality of captioning services over time. Any changes to the metrics would be likely to require corresponding amendments to the Standard. In contrast, a non-metric approach provides greater flexibility and consistency in assessing the quality of captioning services over time.
4 In the NER model, N stands for the number of words in the text, E for ‘Edition’ errors introduced, and R for ‘Recognition’ errors. The model makes allowances for the seriousness of errors, and produces an accuracy rate expressed as a percentage. Further details about the NER model can be found in the Review of the Television Captioning Standard discussion paper.
8 | acma
The ACMA considers that a metric approach would require extensive consultation, research and trials to determine the appropriate metric/s for assessing the quality of captioning services. However, given that the ACMA is not authorised to determine that a lower quality of captioning services is acceptable for a kind of program or program material, it would be difficult to achieve and implement a single metric to evaluate the quality of live, part-live and pre-recorded programs. Further, the ACMA anticipates that it would be difficult to calculate the various metrics in a way that achieves consensus between the needs of industry and the community.
The ACMA does not consider that the proposed metrics would accord with the legislative requirements, as the proposed metrics recommend a lower standard of captioning services for live television programs. Further, the proposed metrics only relate to the accuracy and time delay for live programs and do not take into account how captioning services for part-live and pre-recorded programs may be assessed. The proposed metrics also do not indicate how the readability or comprehensibility of captioning services would be taken into account in assessing quality.
acma | 9
Discussion of alternative option/sAlternative option: The ACMA undertake research to establish the current live captioning metrics for accuracy and time lag
SubmissionsThe discussion paper sought views from stakeholders about an alternative to Option 1 and Option 2, taking into account the differences between captioning services for all programming, without lowering the quality of captioning services for different types of television programs or program material.
One of the 11 submissions received indicated a preference for an alternative option, being that the ACMA undertake research to establish the current live captioning metrics for accuracy and time lag.
Reasons put forward for an alternative optionThe submission in favour of the alternative option suggested that the ACMA should conduct research to establish the current live captioning metrics for accuracy and time lag, similar to the approach undertaken by the Office of Communications (Ofcom) in the UK.
The submission considered that clear and transparent data on levels of accuracy and synchronicity of captioning services would establish whether a metric or a non-metric approach is better in determining the quality of captioning services.
The submission considered that further research might also provide additional options for broadcasters to improve the quality of live captions. This may include the use of a short delay on live broadcasts to allow for the editing of captions, and/or the captioning of pre-prepared segments as part of part-live broadcasts.
The submission suggested that as part of this research, the ACMA should engage with broadcasters to identify and benchmark the current levels of accuracy and synchronicity for live captions.
The ACMA’s response The ACMA has considered the proposed alternative option, but does not intend to undertake research into the live captioning metrics for accuracy and time lag at this stage.
The ACMA acknowledges that there is currently no publicly available data measuring the current levels of accuracy and time lag for captions provided in Australia. In conducting the Review, the ACMA has relied on international research and information supplied by Australian broadcasters, captioning service providers and community and advocacy groups.
Research from international regulators on the quality of captioning has largely been focused on metrics relating to accuracy, latency and the reading speed of captions.
The information presented by Ofcom so far indicates that its own method for calculating accuracy (the NER model) is still under development and is yet to be tested
10 | acma
for all genres of television programs. Ofcom’s study has only been used to assess the quality of live captioning for news, entertainment and chat shows.
Furthermore, Ofcom’s research has not yet been concluded and still requires additional research and engagement with broadcasters, users and access service providers. The ACMA will closely monitor these developments to determine what implications they have for captioning services in Australia.
The ACMA acknowledges that research on metrics relating to reading speed and latency may be useful considerations when assessing readability or comprehensibility. However, the ACMA considers that such metrics alone would be insufficient in determining the readability or comprehensibility of a captioning service. The current Standard contains a broader range of factors to assess readability and comprehensibility. These additional factors recognise that readability and comprehensibility are inherently more subjective elements of quality and less suited to a metric approach.
acma | 11
Discussion of proposed amendments to the Standard A number of submitters, as part of their submissions, proposed various amendments to the Standard. The following proposals were suggested:
The ACMA to consider broadcasters’ efforts to improve captioning servicesOne submission proposed that the Standard be amended to include a reference to the ACMA taking account of efforts by broadcasters to systemically improve captioning services and, in particular, live captioning. This submission specified that the proposed Note under paragraph 6 of the Standard should state:
In determining the quality of captioning services, the ACMA may have regard to evidence presented by a broadcaster or narrowcaster of its own measures to ensure a quality captioning service, and measures to improve quality over time.
The ACMA’s responseThe ACMA does not consider it appropriate to vary the Standard to refer to the general efforts undertaken by broadcasters to improve the quality of captioning services.
The ACMA acknowledges the work undertaken by broadcasters and captioning service providers to improve the quality of captioning; however, the ACMA’s principal means of reviewing captioning quality is to conduct investigations into the quality of captioning services by reviewing specific captioning complaints for specific programs.
As part of making an assessment of a specific complaint, a broadcaster or captioning service provider may wish to supply information to the ACMA relating to what steps (if any) it has taken to improve its captioning services, and how this relates to the captioning service for the program in question. Such steps are ordinarily part of the ACMA’s consideration of what enforcement action, if any, it takes in relation to a breach of the Standard.
The ACMA to distinguish between the method of a captioning service One submission suggested that the proposed note under paragraph 6 of the Standard be amended to require the ACMA to make a distinction between the methods of captioning used (that is, between live captioning and pre-prepared captioning).
The ACMA’s responseThe ACMA does not support the variation of the proposed Note in paragraph 6 so that it refers to the method of captioning used. The BSA requires the ACMA to consider the differences between providing captioning services for live, part-live and pre-recorded television programs. There is no requirement that the ACMA consider the differences between the various methods used in providing or producing captioning services.
12 | acma
Amendments to the factors relating to readability and comprehensibility One submission proposed the following amendments to the factors in the Standard:> amending 7(b)(ii) of the Standard from ‘… caption lines end at natural linguistic
breaks …’ to ‘… captions end at natural linguistic breaks’, in order to facilitate speed reading
> removing 8(b)(iv) of the Standard as it requires captioning service providers to take into account whether or not the intended target audience of a program is children if the captions are not verbatim. This submission noted that the purpose of captions is to render the audio of a program into a text form and not to rewrite the script of the program.
The ACMA’s responseThe ACMA does not support the proposal to change the factors in subparagraphs 7(b)(ii) and 8(b)(iv) of the Standard. The ACMA considers that the factors in the Standard are flexible enough for the ACMA to apply them to the circumstances of the program as a whole, to the extent that they are relevant.
The ACMA to consider whether a program has been live captioned when assessing qualityOne submission proposed an amendment to the Standard requiring the ACMA to consider whether a program has been live captioned as part of a quality assessment. The proposed amendment, to be incorporated into paragraph 6 of the Standard, reads as follows:
When determining the quality of a captioning service for a program or program segment, the ACMA must take into account whether the program has been live captioned.
The ACMA’s responseThe ACMA believes that a broader, less prescriptive framework for assessing the quality of a captioning service provides greater flexibility when assessing the variety of programming types. The ACMA considers that, as part of the Review, a program that is live captioned already falls for consideration within the factors relating to accuracy, readability and comprehensibility when assessing its quality. For this reason, the ACMA does not support the proposal to amend paragraph 6 of the Standard.
Removal of note to paragraph 5 of the StandardOne submission suggested that the ACMA reconsider the note to paragraph 5 of the Standard, which states that:
… in exercising its enforcement powers under the Act, the ACMA takes the position that a program that does not meet the requirements of section 5 of this Standard will not be eligible to be used by a broadcaster or narrowcaster to comply with its captioning obligations.
This submission argued that a failure to provide a captioning service of a certain quality in accordance with the Standard should not be treated as a failure to comply with a captioning obligation under section 130ZR of the BSA.
The ACMA’s responseThe ACMA does not consider that the note to paragraph 5 of the Standard should be amended such that a broadcaster or narrowcaster may meet its captioning obligations under section 130ZR of the BSA, irrespective of the quality of the captioning service to which its captioning obligation relates.
acma | 13
Adherence to the Standard is designed to operate as a disincentive for broadcasters or narrowcasters to lessen the consistency of the quality of the captioning service they provide, including those captioning services that form part of their captioning obligations.
Nonetheless, the ACMA is conducting a Statutory Review of the captioning provisions under Part 9D of the BSA, which may address issues such as the operation of captioning obligations.
14 | acma
Recent international developments As part of the Review, the ACMA has been closely following developments by overseas regulators as they approach the issue of assessing the quality of captioning services.
In particular, the United Kingdom’s regulator, Ofcom, recently completed a two-year study regarding the quality of live captions. Canada’s regulator, Canadian Radio-television and Telecommunications Commission (CRTC), is also in the middle of a consultation phase with stakeholders about the accuracy rate for English language live programming.
The following outlines further developments by Ofcom and the CRTC since the discussion paper was released in November 2015.
Canada BackgroundIn 2012, the CRTC introduced the ‘Broadcasting Regulatory Policy 2012-362 (Quality Standards for English-language closed captioning)’. The mandatory quality standards relate to a number of factors such as lag time, accuracy rate, speed of captions and captioning format.
The CRTC is currently seeking comments on the effectiveness, achievability and measurability of the English language closed-captioning quality standard relating to the accuracy rate for English language live programming (95 per cent with a lag time not exceeding six seconds) from three perspectives: > the experience of Canadians> the experience of broadcasters> the experience of captioning providers.
Current statusA number of submissions from stakeholders on the effectiveness, achievability and measurability of the Quality Standards for English-language closed captioning have been published by the CRTC. However, the CRTC extended the submission date for comments until 22 March 2016 (the original date for submission was 22 September 2015).
The extension was granted to allow stakeholders to engage with one another before filing comments with the CRTC.
On 15 January 2016, the English Language Broadcasters Group (EBG), which comprises the Canadian Association of Broadcasters’ English membership and the Canadian Broadcasting Corporation, submitted a status report on their discussions with relevant stakeholders.
A task group has since been formed and the EBG, the Captioning Consumer Advocacy Alliance (CCAA) and caption providers have agreed to assess the NER model as a replacement to the verbatim assessment methodology for the quality of live captioning in English in Canadian broadcasting.
acma | 15
By 22 March 2016, the parties expect to have informed opinions to offer on whether the NER model, or any other system, can be used in the Canadian regulatory context.
The deadline for replies to comments from stakeholders has also been extended to 21 April 2016.
ACMA considerationThe extended time allocated by the CRTC and the comments received so far, have highlighted the many issues that must still be addressed when trying to come to a position regarding the accuracy rate for live captioning. In particular, the EBG has recognised that broadcasters need to hear more from users about their experience with captioning and that all parties should come to a mutual understanding of the challenges faced by live captioners.
A metric approach for the accuracy of captions, such as the one in place in Canada where live television programs have a lower accuracy metric and a longer time delay than pre-recorded television programs, would not be permitted in Australia under the current legislation.
United Kingdom BackgroundOfcom’s Code on Television Access Services seeks to ensure that television broadcasters meet ‘best practice’ standards for captioning (referred to as ‘subtitles’) and provides recommendations for broadcasters on how to achieve this.
In 2013, Ofcom commenced a two-year study asking broadcasters to report on the quality of live captioning for news, entertainment and chat shows. Sports broadcasts were excluded from the study. Broadcasters were asked to collect data every six months, with the University of Roehampton validating the measurements from an expert, third-party standpoint.
The four key areas of live captioning qualities were identified as latency, accuracy, intermittent subtitles and presentation. The issues considered by the study included whether live broadcasts should be delayed to allow captions to be synchronised for programs that are not time sensitive, the number of late-delivered programs being captioned live, and issues associated with block and scrolling captions.
Current statusAt the time of the release of the ACMA discussion paper, three reports had been published as part of Ofcom’s two-year study. Ofcom has recently released the last of its reports on the quality of live captioning of British television programs (sports broadcasts were excluded from the study). The results outlined in the fourth round of sampling of live subtitled programs broadcast in April and May 2015 are:> The average accuracy rate as measured through the NER model was 98.55 per
cent. The adjusted average accuracy rate (where captions shown at speeds above 200 words per minute were considered as standard errors) was 97.93 percent.
> The average latency was 5.6 seconds, which was well above Ofcom’s recommended three seconds.
> The average speed of captions was below the recommended range of 160–180 words per minute, although 94 per cent of the samples featured short bursts of rapid captioning.
16 | acma
As a result of the two-year study, Ofcom considers that it requires additional consultation and research and will: > consider a review of Ofcom’s Guidelines on the provision of television access
services> consider whether changes should be made to the rules in the Code of Television
Access Services> discuss the main findings from the study, as well as any additional research with
groups representing users of captions, broadcasters and access service providers> commission independent research on captioning users’ attitudes towards ‘trade-
offs’ that occur when producing live captions, including preferences of users with different levels of hearing loss, ages and educational background.
ACMA considerationOfcom’s study has explored some key concepts about the quality of live captioning, such as the accuracy and timing of captions. These issues have been identified as being of most concern in complaints received about captioning services in Australia.
It is also worth noting that the reassessment of accuracy scores to treat rapid subtitling as standard errors introduced a new element of readability and comprehensibility to the accuracy score.
The ACMA considers that at this stage, it would be difficult to establish a metric approach that addresses all the elements affecting the quality of captioning, including readability and comprehensibility. In particular, the information presented by Ofcom so far indicates that its own method for calculating accuracy (the NER model) is still under development and has not been tested for all genres of television programs.
acma | 17
Appendix A—Summary of the public consultation processLive captioning: let’s talkOn 15 September 2015, the ACMA hosted the Citizen conversations forum, Live captioning: let’s talk. Attendees included members of the community, deaf and hearing-impaired advocacy representatives, captioning service providers and broadcasters. Over 80 representatives from a number of different stakeholder groups attended.
The event provided an opportunity for attendees to discuss the challenges of live captioning. International and local speakers presented on topics such as lessons learnt from an international perspective, how decisions are made live (with a particular emphasis on sports), the challenges of captioning news, and the future of captioning.
In addition, citizens joined broadcasters and captioning service providers on a panel to exchange ideas and experiences about the issues associated with live captioning affecting them.
Copies of the presentations and a transcript of the day’s proceedings are available on the ACMA website.5
Stakeholder consultation Between September and October 2015, the ACMA contacted key stakeholders to seek their input about their experiences of captioning services for live and pre-recorded television programs, as well as television programs with both live and pre-recorded program materials.
The stakeholders consisted of representatives from community and advocacy groups, broadcasters and captioning service providers.
This consultation allowed the ACMA to seek specific information from stakeholders, such as what they saw as being the main differences between captioning services for live and pre-recorded programs, and their opinions or experiences on how effective the existing Standard was in dealing with these differences.
The responses from the consultation highlighted that the main differences between captioning services for these programs related to differences in error rates and delays between live and part-live television programs.
Responses also acknowledged that there is more work to be done to improve the quality of captions for live television programs and television programs that have a short turnaround time between production and broadcasting. Citizens, in particular, expressed the desire for pre-prepared block captions.
5 www.acma.gov.au/theACMA/live-captioning-lets-talk-register
18 | acma
Review of Television Captioning Standard discussion paper On 17 November 2015, the ACMA released the Review of the Television Captioning discussion paper for public comment. The due date for submissions was 18 December 2015. A summary of the submissions received is contained in Appendix B.
Summary of additional issues raised in submissions The submissions raised some additional issues that did not directly fall for consideration within the Review. A summary of these issues is contained below. Issues that fall outside of the Review may be considered as part of the broader Statutory Review.
Practical matters related to the types of captioning usedOne submission suggested that the ACMA should engage with broadcasters and captioning service providers to ensure that:> programs are only captioned live when there is no alternative> captions for pre-recorded or scripted segments of otherwise live programs are, as
far as possible, prepared in advance.
Another submission noted that all captioning services should be pre-prepared unless it is genuinely not possible to do so.
Another submission also suggested further consultation with broadcasters and consumers to discuss the type of captions (for example, verbatim, block or scrolling) provided to deaf and hearing-impaired viewers.
The ACMA’s responseWhen assessing compliance with the Standard, the ACMA already considers whether a program is live, part-live or pre-recorded, and whether a program or program material has been prepared using live or pre-prepared captions. The ACMA also already considers the type of captions (for example, verbatim) as part of this assessment.
The ACMA acknowledges that there are many aspects involved in the decision-making process for choosing the method of captioning for various programs, and there is no one-size-fits-all solution.
Complaints processOne submission suggested that the ACMA should undertake an ad hoc survey of captioning of live programming across all broadcasters, in order to develop a better understanding of the complaints process regarding captioning on Australian television. The submission noted that the method for lodging complaints can be onerous for complainants.
Another submission discussed the current complaint mechanism and made the following points:> end users (viewers) may not be aware of the complaints processes, as information
on lodging complaints may not be readily available> some viewers may find the complaints process drawn out and of limited benefit, in
circumstances where broadcasters are not required to re-broadcast programs that are the subject of captioning complaints and do not incur a penalty for failing to meet the provisions of the Standard.
acma | 19
The ACMA’s response The ACMA notes the concerns about the complaints process relating to captioning services. Currently, the complaint mechanism allows complaints about captioning services to be made directly to the ACMA, and in some circumstances, referred on from complaints made to broadcasters. The ACMA has developed videos, which include Auslan signing and captioning, to assist deaf and hearing-impaired viewers in making complaints. Further information about the complaints process and relevant videos can be accessed on the ACMA’s website.6
The ACMA will consider how to improve awareness of the complaints processes relating to captioning services. The ACMA intends to engage with stakeholder channels to improve awareness amongst the deaf and hearing-impaired community about the complaints process for captioning services on Australian television.
New quality assessmentOne submission proposed the establishment of a five-star rating system to improve the experience and quality of captioning.
The ACMA’s responseThe ACMA acknowledges that there may be different ways to determine the quality of captioning services, but emphasises the importance of the ‘meaningful access’ test.
Rebroadcasting live captionsOne submission suggested that programs that were originally broadcast live with live captioning (in whole or in part) should be rebroadcast as pre-prepared captions. Another submission noted that due to the short shelf life of certain programs, it would be inefficient to invest additional resources in re-captioning programs being rebroadcast.
The ACMA’s responseThe ACMA acknowledges that there are many aspects involved in the decision-making process for choosing the method of captioning for various programs, and there is no one-size-fits-all solution. Consequently, the ACMA does not propose to specify that a certain captioning method be used for programs that are being rebroadcast.
6 www.acma.gov.au/Citizen/Consumer-info/Rights-and-safeguards/Captioning/complaints-about-captioning-i-acma-1
20 | acma
Appendix B—Summary of the submissions processThe ACMA received the following submissions in response to the discussion paper:
Table 1: Submissions received
Submission number
Name/Organisation
1 Special Broadcasting Service
2 Ericsson Australia Pty Ltd
3 Australian Communications Consumer Action Network
4 Media Access Australia
5 Access Innovation Media Pty Ltd
6 Australian Subscription Television and Radio Association
7 Australian Broadcasting Corporation
8 Free TV Australia
9 Deaf Australia Inc.
10 Henge Design Pty Ltd
11 Caption It Pty Ltd
Submissions were received from a number of stakeholder groups:
Table 2: Breakdown of submissions received
Group Number of submissions received
Broadcasters 47
Captioning service providers 3
Community and advocacy groups 3
Other 1
Support of options presentedThe discussion paper presented two possible options for determining the quality of captioning services, based on a metric or non-metric approach, and allowed stakeholders to present an alternative option for consideration.
Table 3: Support of options presented
7 Two submissions were made on behalf of Australia’s commercial free-to-air television broadcasting licensees and subscription television licensees.
acma | 21
Option Number of submissions received in support of each option
Option 1—Non-metric 98
Option 2—Metric 1
Alternative option 1
8 Four submissions indicated a preference for Option 1 but with slight variations.
22 | acma
