AI and live captioning. Comparing the quality of automatic and human live captions in English

Closed captions play a vital role in making live broadcasts/events accessible to many viewers. Traditionally, stenographers and respeakers have been in charge of their production, but this scenario is changing due to the steady improvements that automatic speech recognition (ASR) has experienced in recent years. Broadcasters and service providers are beginning to roll out this technology to produce intralingual live captions in different contexts. Human and automatic captions co-exist now in different settings and, while some research has focused on the accuracy of human live captions, comprehensive assessments of the accuracy and quality of automatic captions are still needed. This presentation will tackle this issue by introducing the main findings of the largest study comparing the accuracy of automatic and human live captions conducted to date. Through five case studies including approximately 17.000 live captions analysed with the NER model from 2018 to 2023 in the UK, the U.S. and Canada (Romero-Fresco and Fresno-Cañada, forthcoming), this presentation will track the recent developments of automatic captions, including the very latest generation of AI tools, to compare their accuracy to that achieved by humans.

Beyond this, and within the framework of the Spanish-government-funded Qualisub project, the presentation will end by addressing the potential full automation of the NER model (given the issues caused by the use of the WER model) and by reflecting on what the future of live captioning looks like for both human and automatic captions.