Fast Calculation of Translation Model Score for Simultaneous Automatic Speech Recognition of Multilingual Audio Contents

This paper addresses automatic speech recognition (ASR) for multilingual audio contents, such as international conference recordings and broadcast news. For handling such contents efficiently, a simultaneous ASR is promising. Conventionally, ASR has been performed independently, namely, language by language, although multilingual speech, which consists of utterances in several languages representing identical meaning, is available. We previously proposed a bilingual speech recognition framework based on statistical ASR and machine translation in which bilingual ASR is performed simultaneously and complementarily. In this simultaneous recognition framework, ASR systems use not only acoustic and language model scores but also a translation model (TM) score. In this study, we investigate an efficient calculation method of TM scores. A TM score represents how a sentence corresponds to another sentence of different languages. In general, between different languages a word can be translated into various words. Moreover, word orders are different. Considering these characteristics, TM scores should be modeled statistically. In a statistical translation model, each word in source language is modeled to have a possibility to be translated into every word in target language. For instance, for the matching (alignment) of n-word sentences and m-word sentences, there are n to the m-th power word-alignments. For a strict calculation of statistical TM scores, first, we calculate the probability of each alignment and then calculate their sum. However, this calculation costs too much and is inadequate for a real-time system. In this study, we reduce the computational cost. Specifically, since for almost all alignments, their probabilities are much smaller compared with the highest alignment probability, we regard the highest alignment probability as a TM score. We compared TM score calculation methods for time and accuracy in a Japanese ASR using English information based on a bilingual recognition framework. We significantly reduced processing time for TM score calculation without any degradation of ASR accuracy.