LAS-Transformer: An Enhanced Transformer Based on the Local Attention Mechanism for Speech Recognition

Recently, Transformer-based models have shown promising results in automatic speech recognition (ASR), outperforming based on recurrent neural networks (RNNs) and convolutional (CNNs). However, directly applying a Transformer to the ASR task does not exploit correlation among frames effectively, leaving model trapped sub-optimal solution. To this end, we propose local attention for that combines high frames. Specifically, use relative positional embedding, rather than absolute improve generalization of sequences different lengths. Secondly, add parametric relations self-attentive module explicitly incorporate prior knowledge into make training process insensitive hyperparameters, thus improving performance. Experiments carried out LibriSpeech dataset show our proposed approach achieves word error rate 2.3/5.5% by language fusion without any external data reduces 17.8/9.8% compared baseline. The are also close to, or better than, other state-of-the-art end-to-end models.