Missing data techniques: Feature reconstruction

Automatic speech recognition (ASR) performance degrades rapidly when speech is corrupted with increasing levels of noise. Missing data techniques (MDT) constitute a family of methods that tackle noise robust speech recognition based on the so called missing data assumption proposed in [1]. MDTs assume that (i) the noisy speech signal can be divided in speech-dominated (reliable) and noise-dominated (unreliable) spectro-temporal components prior to decoding and (ii) the unreliable elements do not retain any information about the corresponding clean speech values. This means that the clean speech values corresponding to noise-dominated components are effectively missing, and speech recognition must proceed with partially observed data. Techniques for speech recognition with missing features divide in roughly two categories, marginalization and feature reconstruction. The marginalization approach, discussed in Chapter ??, is based on disregarding the missing components when calculating acoustic model likelihoods: likelihoods that correspond to the missing components are calculated by integrating over the full range of possible missing feature values [2, 3]. In this chapter, we focus on the reconstruction approach, where the missing values are substituted (imputed) with clean speech estimates prior to calculating the acoustic model likelihoods [4, 5, 6]. Since the reconstructed features do not contain any missing data, likelihood calculation does not need to be modified. In general, all missing feature imputation methods employ a model of the clean speech to estimate the missing values. The models range from simple smoothness assumptions [6] to advanced statistical models and exemplar-based approaches, although the acoustic models employed by the recognizer may also be used. Given the clean speech model and a noisy observation, the missing features are estimated as the values that best match the assumptions of clean speech components at the missing locations.