APVV-15-0517
Automatic Subtitling of Audiovisual Content for Hearing Impaired
Only a few of us could imagine that obtaining information from TV broadcast is one of the basic problems of the hearing impaired. In the present time there is no equivalent access for the given group of people to the television broadcast content as it is in the case of the hearing population. Within the meaning of the legislation (Law no. 373/2013 of the Code from October 20. 2013), broadcaster is obliged to ensure multimodal approach to the digital broadcast service in a way that at least 50% is accompanied with open or closed captions corresponding with the content of the program. In a similar way, at least 10% is obligatory in the case of the licensed broadcasters. Recently, the European Federation of Hard of Hearing People (EFHOH) is pushing ahead idea to enhance ratio of the programs accompanied by open or closed captions to 100% in each EU member state. Reaching the desired goal in Slovakia using the current approach of subtitling the audiovisual content would mean spending huge amount of financial resources by the television broadcaster, because manufacturing of the closed captions is subject of laborious manual transcription of the spoken words to text by certified workers and consecutive adjustment specified by the requirements of the edict of the Ministry of the Culture of the Slovak Republic. The only economically viable option is to head towards utilization of the automatic spontaneous speech recognition and to apply modern principles and methods of the speech technologies in automatic transcription of spoken words to text. The main goal of this project proposal is applied research in the area of the natural speech processing and development of a customized pilot system for automatic subtitling of audiovisual content based on large vocabulary continuous speech recognition. Results of the applied research are going to be a base of development of system solutions (software application or service) for automatic subtitling in Slovak.
The aim of this project is an applied research in the area of natural speech and language processing in order to design and develop a system specifically to automatic subtitling of audiovisual content for deaf and hard of hearing people based on automatic large vocabulary continuous speech recognition. For purpose of successful project completion following tasks are required to be fulfilled and realized:
systematic collection, processing and annotation of a corpus of new speech and text data including speaker identification, topic classification, speech style, and acoustic environment influence for purpose of existing speech and text corpora extension and subsequent actualization and adaptation of a large vocabulary continuous speech recognition system to the area of newscast and TV news broadcast;
design and develop system core for large vocabulary continuous speech recognition placed on computing server with subsequent adaptation all of its components (acoustic and language models, and vocabularies) for the automatic subtitling of an audiovisual content for live broadcast in the Slovak language;
applied research and development of advanced methods for acoustic model to speaker's gender and voice adaptation, language model to topic and speaker's speaking style adaptation, and dynamic vocabulary update on regular basis;
the comprehensive evaluation using different deployment scenarios including specific conditions simulating real environment in order to determine influence level of developed methods and approaches to resulting accuracy of the automatic transcription in automatic closed captioning of audiovisual content in the Slovak language.
Results of the applied research are going to be a base of development of system solution (in the form of a software application or service) for automatic subtitling of audiovisual content in Slovak provided to people with hearing impairments.
Principal Investigator, FEEI TUKE
Full Professor, FEEI TUKE
Assistant Proffesor, FEEI TUKE
Assistant Proffesor, FEEI TUKE
Assistant Proffesor, FEEI TUKE
Research Assistant, FEEI TUKE
Researcher, FEEI TUKE
Researcher, FEEI TUKE
PhD. Student, FEEI TUKE
PhD. Student, FEEI TUKE
PhD. Student, FEEI TUKE
Co-Principal Investigator, II SAS BA
Research Assistant, II SAS BA
Research Assistant, II SAS BA
Research Assistant, II SAS BA
Research Assistant, II SAS BA
Research Assistant, II SAS BA
Associate Proffesor, FF UKF Nitra