In this study, a complete system that generates visual speech by synthesizing 3D face points has been implemented. The estimated face points drive MPEG-4 facial animation. This system is speaker independent and can be driven by audio or both audio and text. The synthesis of visual speech was realized by a codebook-based technique, which is trained with audio-visual data from a speaker. An audio...

The goal of the present study was to establish the nature of visual input (featural vs holistic) and the mode of its presentation that facilitates best audio-visual speech perception. Sixteen participants were asked to repeat acoustically strongly and mildly degraded syllables, presented in auditory and three audio-visual conditions, within which one contained holistic and two contained featura...

The proposed digital system consists of several different modules that can be divided into three main groups: input modules, audio output modules, and video output modules. All of these modules will be clocked off a 27MHz clock signal provided by the 6.111 Labkit. The system’s input comes from an analog signal produced by a pedometer, and it is then processed by the input modules to produce rel...

This paper proposes a method to re-estimate output visual parameters for speech-to-lip movement synthesis using audio-visual Hidden Markov Models(HMMs) under the Expectation-Maximization(EM) algorithm. In the conventional methods for speech-to-lip movement synthesis, there is a synthesis method estimating a visual parameter sequence through the Viterbi alignment of an input acoustic speech sign...

A system that builds desktop virtual reality models based on topographic maps and its application to tidal dynamics analysis is shown. The virtual reality model allows the user to explore the scene from any possible point of view, also permitting to alter the level of the sea simulating tides and flooding. Flooded and dry areas are visible at each water level, and the flooding pattern can be as...

For the flooding seasons of 2011-2012 multiple space assets were used in a “sensorweb” to track major flooding in Thailand. Worldview-2 multispectral data was used in this effort and provided extremely high spatial resolution (2m / pixel) multispectral (8 bands at 0.45-1.05 μ m spectra) data from which mostly automated workflows derived surface water extent and volumetric water information for ...

We propose a novel paradigm of consuming rich web content in a mobile setting (which are often eyes-free), through a predominantly 3D audio interface. Web content, formatted in audio, is streamed via the mobile device’s network connection, and placed virtually in a 3D audio space. The user moves in the virtual space, using a variety of human computer interaction (HCI) means, such as voice input...