This paper introduces DeepSynth, an end-to-end neural network model for generating the sound of a musical instrument based on a silent video of it being played. We specifically focus on building a synthesizer for the piano, but the ideas proposed in this paper are applicable to a wide range of musical instruments. At a high level, the model consists of a convolutional neural network (CNN) to ex...

This paper discusses some of the issues pertaining to the design of digital musical instruments that are to effectively fill the role of traditional instruments (i.e. those based on physical sound production mechanisms). The design and implementation of a musical instrument that addresses some of these issues, using scanned synthesis coupled to a “smart” physical system, is described.

In this demonstration we present Mimesis, a virtual environment with interactive stereoscopic real time graphics and interactive sound that works as a visual and musical instrument. We have developed two display systems, a spatialized sound system and a passive horizontal stereoscopic visual system, that help users create visuals and music. Both systems work together increasing the sense of imm...

A handheld electronic musical instrument, named the BentoBox, was developed. The motivation was to develop an instrument which one can easily carry around and play in moments of free time, for example when riding public transportation or during short breaks at work. The device was designed to enable quick learning by having various scales programmed for different styles of music, and also be ex...

Spectrum matching of musical instrument tones is a fundamental problem in computer music. Two methods are presented for determining near-optimal parameters for the synthesis of harmonic musical instrument or voice sounds using the addition of several fixed wavetables with time-varying weights. The overall objective is to find wavetable spectra and associated amplitude envelopes which together p...

In pattern recognition applications, finding compact and efficient feature set is important in overall problem solving. In this paper, feature analysis using wavelet coefficient histogram for the musical instrument recognition has been presented and compared with traditional features. The new proposed wavelet coefficient histograms features found compact and efficient with existing traditional ...

In order to map the spectral characteristics of the large variety of sounds a musical instrument may produce, different notes were performed and sampled in several intensity levels across the whole extension of a clarinet. Amplitude and frequency time-varying curves of partials were measured by Discrete Fourier Transform. A limited set of orthogonal spectral bases was derived by Principal Compo...

In the continuing goal of codifying the classification of musical sounds and extracting rules for data mining, we present the following methodology of categorization, based on numerical parameters. The motivation for this paper is based upon the fallibility of Hornbostel and Sachs generic classification scheme, used in Music Information Retrieval for instruments. In eliminating the redundancy a...

A set of features is evaluated for recognition of musical instruments out of monophonic musical signals. Aiming to achieve a compact representation, the adopted features regard only spectral characteristics of sound and are limited in number. On top of these descriptors, various classification methods are implemented and tested. Over a dataset of 1007 tones from 27 musical instruments, support ...

This paper considers how to achieve new creative advances in the design of programming languages. It is based on the analysis of a single application domain, the practice of Live Coding in a new area of musical performance known as “Laptop” music. Analysis of live coding as a context for programming allows us to escape the implicit assumptions of the commercial office environment in which so mu...