Title
Automatic bilingual subtitles in Spanish TV broadcast. The case of "L’informatiu. Comunitat Valenciana"
Conference name
Media for All 10 Conference
City
Country
Belgium
Modalities
Date
06/07/2023-07/07/2023
Abstract
The improvements in automatic speech recognition (ASR) software have led to its integration in live subtitling environments. As part of QuaLiSub (The Quality of Live Subtitling: A regional, national and international study, funded by the Spanish Ministry of Science and led by Universidade de Vigo), this paper presents results on the quality of subtitles broadcasted by Spanish public television (CRTVE) in the regional news for Comunitat Valenciana. These subtitles are live, intralinguistic, not human-edited, and bilingual.
The analysis undertaken has measured accuracy by two methods. Under the framework of a research contract between Universitat Jaume I and CRTVE, the percentage of errors in transcription (substitution, deletion, and insertion) was calculated following the Word Error Rate (WER, US National Institute of Standards and Technology). In the context of QuaLiSub, accuracy was estimated using the NER model (Romero Fresco & Martínez, 2015). We considered, on the one hand, to which extent errors affect the coherence of the subtitled text and, on the other hand, how many correct editions (recognitions, in the case of automatic subtitles) were included.
Preliminary results show that, from the perspective of user comprehension, these automatic bilingual subtitles are not yet acceptable. Although some subtitles have excellent linguistic accuracy (better in Spanish than in Catalan), the coherence of the subtitled text is generally unsatisfactory. Code switching is so frequent in the corpus that the software needs about 5 seconds to transcribe in the correct language - this increases subtitle delay. Besides, subtitle speed exceeds Spanish UNE 153010 (AENOR, 2012) recommendation of 15 CPS, dual speakers in a subtitle are not indicated, and text segmentation is poor. In consequence, our conclusion is that the understanding of users may be compromised.
The analysis undertaken has measured accuracy by two methods. Under the framework of a research contract between Universitat Jaume I and CRTVE, the percentage of errors in transcription (substitution, deletion, and insertion) was calculated following the Word Error Rate (WER, US National Institute of Standards and Technology). In the context of QuaLiSub, accuracy was estimated using the NER model (Romero Fresco & Martínez, 2015). We considered, on the one hand, to which extent errors affect the coherence of the subtitled text and, on the other hand, how many correct editions (recognitions, in the case of automatic subtitles) were included.
Preliminary results show that, from the perspective of user comprehension, these automatic bilingual subtitles are not yet acceptable. Although some subtitles have excellent linguistic accuracy (better in Spanish than in Catalan), the coherence of the subtitled text is generally unsatisfactory. Code switching is so frequent in the corpus that the software needs about 5 seconds to transcribe in the correct language - this increases subtitle delay. Besides, subtitle speed exceeds Spanish UNE 153010 (AENOR, 2012) recommendation of 15 CPS, dual speakers in a subtitle are not indicated, and text segmentation is poor. In consequence, our conclusion is that the understanding of users may be compromised.