Speech Recognition for Subtitling Japanese Live Broadcasts

There is a great need for more TV programs to be subtitled to help hearing impaired and elderly people to watch TV. NHK has researched automatic speech recognition for subtitling live TV programs in real time efficiently. Our speech recognition system learns frequent words and expressions expected in the program beforehand and also learns characteristics of announcers’ voices in order to reduce recognition errors. It periodically outputs recognition results while the announcer is speaking, which shortens the delay for texts to be displayed on the screen. NHK has been using speech recognition to subtitle some of its news, sports and music shows. In news programs, speech read by an announcer in a quiet studio is automatically recognized and errors are immediately corrected by hand. Live TV programs, e.g., the Winter Olympic Games, the World Cup Football Games, and the Grand Sumo Tournaments, have been subtitled by using a re-speak method in which an announcer listens to the program contents and rephrases them. The method improves the recognition accuracy and makes the subtitles easier to be read because it can generate subtitling for programs with high background noise and also allows summarization and paraphrasing of the original speakers’ words.