Sonic Interaction Design: New Applications and Challenges for Interactive Sonification

Sonic Interaction Design (SID) is the exploitation of sound as a principal channel to convey information, meaning as well as aesthetic and emotional qualities in interactive contexts [1]. SID is a new young research field that offers novel perspectives for interactive artefacts and multimodal user interfaces that use sound at the core of their designs as means to interact with the user or to communicate and express specific facets. The COST Action IC0601 SID investigates the various aspects of sonic interaction design with the focus on (a) perception, cognition and emotion, (b) product design, (c) interactive art and (d) sonification and information display. This talk will provide an overview of SID, present examples and design procedures that take sound, its synthesis and generation, as well as our modes of communication about sound serious. Sonification is the data-dependent, reproducible generation of sound using a systematic transformation, and it is a central component to shape the functional aspect of interactive artefacts [2]. The discussion of a definition and taxonomy will provide a solid basis for reviewing the most important sonification techniques: Auditory Icons, Earcons, Audification, Parameter-Mapping Sonification and Model-based Sonification. A particular focus will then be set on the importance of Interactive Sonification and its potential to enhance the functional aspect of auditory information during interaction [3, 4]. This will be demonstrated with various new applications, interfaces and sonification methods. In particular, the approach of Model-Based Sonification (MBS) will be presented, which provides an interaction-based mediation between abstract information spaces and the sound space [5]. Model-Based Sonification turns data into dynamic models capable of producing sound when interacted with. Opposite to Parameter Mapping Sonification where data are used to control or drive a synthesizer, in MBS the data becomes the synthesizer, or better: is used to configure the setup of a sound-capable object, and excitation or interactive exploration is left to the user. This not only offers a very generic and reusable way to explore data auditorily, it also integrates the user tightly into a closed-loop and thereby faciliates the process of querying a sonic system. Inspired from the observation that for understanding a 3D visual world of objects we need multiple different perspectives, in analogy we need multiple auditory ’views’ to understand an object from listening to its sound, and it is by interaction that we query objects for information effortless in everyday interaction. MBS enables an analogue query mechanism for information spaces. An important application of MBS is auditory datamining, and different sonification models will be demonstrated such as data sonograms, particle trajectories in data potentials and the growing neural gas sonification model. The benefits of MBS will be discussed as well as its main problem: MBS can be computationally very demanding. MBS will also be contrasted to parametermapping sonification and physical modelling in sound synthesis. Particularly the latter discussion will highlight some interesting areas where MBS and more generally SID can profit from research and methods addressed in DAFx. The talk closes with some examples and ideas how Sonic Interaction Design can influence our daily lives in the information age, how we can be better immersed into information spaces using Ambient Intelligence environments, and how closed-loop auditory systems can help to induce a healthy behaviour.