Onset Detection Exploiting Wavelet Transform with Bidirectional Long Short-term Memory Neural Networks

A plethora of different onset detection methods have been proposed in the recent years. However few attempts have been made with regard to widely-applicable approaches in order to achieve superior performances over different types of music and with considerable temporal precision. This paper concerns the usage of Wavelet Packet Transform in order to exploits multi-resolution time-frequency features. We apply early fusion in the feature space by combining Wavelet Packet Energy Coefficients and auditory spectral features. The features are then processed by a bidirectional Long Short-Term Memory recurrent neural network, acting as reduction function. The network is trained with a large database of onset data covering various genres and onset types. Due to the data driven nature, our approach does not require the onset detection method and its parameters to be tuned to a particular type of music. 1. ALGORITHM DESCRIPTION The algorithm can be seen divided in three parts. Firstly, the audio data is transformed into the frequency domain via a Discrete Wavelet Packet Transform (DWPT) with 22 bands (Table 1 shows frequency division) and via two parallel STFTs with two different window sizes. Energy-base information and their evolution in time are obtained from the transformations leading to the final feature set. Secondly, the features are used as inputs to the BLSTM network, which produces an onset activation function at its output. Finally, network output is post-processed by a thresholding and peak picking methods in order to obtain the correct onsets’ position. Figure 1 shows this basic signal flow. The individual blocks are described in more detail in the following sections. This document is licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 License. http://creativecommons.org/licenses/by-nc-sa/3.0/ c © 2013 The Authors. Auditory Spectral Feat. WPEC BLSTM Network Peak detection Signal Onsets Figure 1. General block scheme. 1.1 Feature Extraction Discrete input audio files, sampled at Fs = 44.1kHz, have been used for our experiments. A new features set is obtained exploiting wavelet transformation (cf. Figure 2) by obtaining Wavelet Packet Energy Coefficients (WPEC). The discrete input audio signal is segmented into overlapping frames ofW46 = 2048 samples, which are sampled at a rate of 100 fps, log-energy of each frame is calculated before applying the Hamming window following: