End-user Development of Sonifications Using Soundscapes

Designing sonifications requires knowledge in many domains including sound design, sonification design, and programming. Thus end users typically do not create sonifications on their own, but instead work with sonification experts to iteratively co-design their systems. However, once a sonification system is deployed there is little a user can do to make adjustments. In this work, we present an approach for sonification system design that puts end users in the control of the design process by allowing them to interactively generate, explore, and refine sonification designs. Our approach allows a user to start creating sonifications simply by providing an example soundscape (i.e., an example of what they might want their sonification to sound like), and an example dataset illustrating properties of the data they would like to sonify. The user is then provided with the ability to employ automated or semi-automated design of mappings from features of the data to soundscape controls. To make this possible, we describe formal models for soundscape, data, and sonification, and an optimization-based method for creating sonifications that is informed by design principles outlined in past auditory display research.