Local Information Assisted Attention-Free Decoder for Audio Captioning

Automated audio captioning aims to describe data with captions using natural language. Existing methods often employ an encoder-decoder structure, where the attention-based decoder (e.g., Transformer decoder) is widely used and achieves state-of-the-art performance. Although this method effectively captures global information within via self-attention mechanism, it may ignore event short time duration, due its limitation in capturing local signal, leading inaccurate prediction of captions. To address issue, we propose a pretrained neural networks (PANNs) as encoder assisted attention-free (LocalAFT) decoder. The novelty our proposal LocalAFT decoder, which allows signal be captured while retaining information. This enables events different including for more precise caption generation. Experiments show that outperforms Task 6 DCASE 2021 Challenge standard