Speech is a means of communication that is intrinsically bimodal: the audio signal originates from the dynamics of the articulators. This paper reviews recent works in the field of audiovisual speech and more specifically on techniques developed to measure the level of correspondence between audio and visual speech. It overviews the most common audio and visual speech front-end processing, tran...

Oscillatory synchrony could be used to establish dynamic links between the various cortical areas participating in the same cognitive process. Is it possible to detect oscillatory synchrony in humans, and is it relevant to behavior? There is now converging evidence for the existence of a transient oscillatory activity in the gamma range (30-60 Hz), obtained in response to static visual objects,...

The synchronous occurrence of the unisensory components of a multisensory stimulus contributes to their successful merging into a coherent perceptual representation. Oscillatory gamma-band responses (GBRs, 30-80 Hz) have been linked to feature integration mechanisms and to multisensory processing, suggesting they may also be sensitive to the temporal alignment of multisensory stimulus component...

Three experiments investigated perception of audio-visual (A-V) speech synchrony in 4- to 10-month-old infants. Experiments 1 and 2 used a convergent-operations approach by habituating infants to an audiovisually synchronous syllable (Experiment 1) and then testing for detection of increasing degrees of A-V asynchrony (366, 500, and 666 ms) or by habituating infants to a detectably asynchronous...

We measured spiking activity from 175 pairs of visually selective neurons while a monkey performed a visual classification task in which the same image pairs appeared in informative and uninformative contexts. Spike counts and synchrony counts were higher during fixations of informative configurations. Spike field coherence showed low frequency (approximately 5-15 Hz) desynchronization when vie...

We propose a novel method to automatically extract the audio-visual objects that are present in a scene. First, the synchrony between related events in audio and video channels is exploited to identify the possible locations of the sound sources. Video regions presenting a high coherence with the soundtrack are automatically labelled as being part of the audio-visual object. Next, a graph cut s...

The role of audio–visual speech synchrony for speaker diarisation is investigated on the multiparty meeting domain. We measured both mutual information and canonical correlation on different sets of audio and video features. As acoustic features we considered energy and MFCCs. As visual features we experimented both with motion intensity features, computed on the whole image, and Kanade Lucas T...

This study assessed an intersensory redundancy hypothesis, which holds that in early infancy information presented redundantly and in temporal synchrony across two sense modalities selectively recruits attention and facilitates perceptual differentiation more effectively than does the same information presented unimodally. Five-month-old infants' sensitivity to the amodal property of rhythm was...