Descriptor-based spatialization

The translation of monophonic soundtracks to new audio formats is the object of a growing demand particularly from the DVD producers. However operations like “upmixing” a monophonic track to a multi-channel format are time-consuming tasks for the sound engineer, who has to choose, adapt and tune different spatialization tools. In order to simplify the upmix, we introduce a new spatialization approach based on the use of perceptive descriptors. It consists in automatically detecting audio source characteristics, based on perceptive aspects and exploiting these characteristics to control spatialization tools. This paper presents an experimentation driven by Arkamys and Sony CSL Paris to evaluate the interest of this technique, to address the needs of sound engineers in the field of spatialization. EDS [1,2], a generic audio information extractor, was used to control the Arkamys spatializer [3]. We describe the aim and interest of this descriptor-based approach and discuss its performances and limitations. 1. THE ARKAMIX SPATIALIZATION TOOL 1.1. Presentation Arkamix is a multipurpose spatialization toolbox developed by Arkamys. Its application for old movie spatialization must be realized in respect of the original soundtrack meanings. This transformation must be downmix compatible in mono and stereo. The Arkamix toolbox was made to simplify digital audio effects design and tuning. There are three different layers in Arkamix: • C++, audio effects and tools, • Graphic User Interface (GUI), • XML Configuration file. The GUI and the audio effects are completely driven by the XML configuration files. The GUI is able to run without the audio process and reciprocally. XML files are split in two categories: block based description of the process, and controlled values. At the startup, both GUI and process read XML files to execute a self configuration. Thanks to this architecture, Arkamix is Monceaux, Pachet et al. Descriptor-based spatialization AES 118th Convention, Barcelona, Spain, 2005 May 28–31 Page 2 of 8 flexible software for audio effects designers and sound engineers. In our case, extractors from EDS (Extractor Discovery System) interact with the audio process, in complement of the original Arkamix GUI. We have developed a specific Arkamix spatialization toolbox taking into account sound engineers techniques. We have designed generic tools such as: equalization, multi-channel compressor, limiter, panoramic mixer, etc. In complement, we added more specific tools like: mono extractor, surround extractor, short stereo and multi-channel reverberation. Usually, we control these algorithms by a graphic user interface, but in this case, we substitute it with a control module, which is designed to translate sources identification generated by the extractor from EDS to a series of control commands. The driven controls are exactly the same as the standard controls driven by sound engineer. Control values are principally Boolean values that answer a simple question "Is Source A detected" for instance. This information can be translated to Boolean value such as "bypass effect #1", a linear value "Control Value B9 go to value 1.0 smoothly", or it can be a mix of them. It can also be a complex command such as: “bypass effect #1, Control Value B9 go to value 1.0 smoothly AND surround level go to 0 quickly”.