This thesis presents a novel lip-reading approach to classifying utterances from video data, without evaluating voice signals. This work addresses two important issues which are • the efficient representation of mouth movement for visual speech recognition • the temporal segmentation of utterances from video. The first part of the thesis describes a robust movement-based technique used to ident...

INTRODUCTION A key mechanistic principle of the nervous system is one of initial segmentation, whereby each environmental object or event is subdivided into its elemental parts. For example, in the visual system, features such as colour, location, motion and texture are analyzed by largely separate cortical regions. Nowhere is the segregation of inputs more obvious than across the sensory syste...

This paper presents the application of fuzzy set theory to automatic computer lip-reading from video images. Simple rules based on fuzzy sets were generated using the mass assignment theory and were used for automatic feature extraction from video sequences. Probabilistic grid models were used to derive a knowledge base representing the visual data for phonemes or sounds. Phonemes from a medium...

The problem of interpolating between specified images in an image sequence is a simple, but important task in model-based vision. We describe an approach based on the abstract task of "manifold learning" and present results on both synthetic and real image se­ quences. This problem arose in the development of a combined lip-reading and speech recognition system.

Speech audiometry is used to measure threshold, to assess suprathreshold intelligibility, to measure progress in lip-reading and auditory training, to detect the presence of malingering, to evaluate the effectiveness of different aids, to predict the success of otologic surgery and to aid in the diagnosis of both peripheral and cortical disorder. They are useful and often essential in modern au...

This paper addresses the problem of speaker-dependent isolate digits recognition using sole visual information. We employ intensity transformation and spatial filter to estimate the minimum enclosing rectangle of mouth in each frame. Thus, for each utterance, we can obtain two vectors composed of width and height of mouth, respectively. Then, we propose an approach to recognize the speech based...