Multi-Channel Multi-Frame ADL-MVDR for Target Speech Separation

Many purely neural network based speech separation approaches have been proposed to improve objective assessment scores, but they often introduce nonlinear distortions that are harmful modern automatic recognition (ASR) systems. Minimum variance distortionless response (MVDR) filters adopted remove distortions, however, conventional mask-based MVDR systems still result in relatively high levels of residual noise. Moreover, the matrix inverse involved solution is sometimes numerically unstable during joint training with networks. In this study, we propose a multi-channel multi-frame (MCMF) all deep learning (ADL)-MVDR approach for target separation, which extends our preliminary ADL-MVDR approach. The MCMF system addresses linear and distortions. Spatio-temporal cross correlations also fully utilized evaluated using Mandarin audio-visual corpus compared several state-of-the-art approaches. Experimental results demonstrate superiority under different scenarios across evaluation metrics, including ASR performance.