In this work, a novel gesture recognition system is presented that allows the control of the SmartWalker by performing simple gestures. The SmartWalker is a normal walker equipped with sensors and actuators that can move autonomously. It is constructed as a walking aid for elderly people. Therefore our gesture recognition system is developed for the same target group which means that gestures w...

As may be seen at concerts and in various everyday listening situations, people often make spontaneous gestures when listening to music. We believe these gestures are interesting to study because they may reveal important features of musical experience. In particular, hand movements may give us information on what features are perceived as salient by listeners. Based on various current ideas on...

A dialogue system has to deal with the problem of interruptions by the user, e.g. changes of requests (called »barge−in«). This contribution is concerned with this problem in the special case of the multimodal dialogue system SmartKom. How are gestures used during such interruptions if they are utilized at all? To answer this question we analyzed a number of human− machine dialogues qualitative...

Gesture segmentation is an extremely difficult task due to both the multitude of possible gesture variations in spatio-temporal space and the co-articulation of successive gestures. In this paper, a robust framework for this problem is proposed which has been used to segment out component gestures from a continuous stream of gestures using finite state machine and motion features in a vision ba...

We describe preliminary work towards an objective method for identifying visemes. Active appearance model (AAM) features are used to parameterise a speaker’s lips and jaw during speech. The temporal behaviour of AAM features between automatically identified salient points is used to represent visual speech gestures, and visemes are created by clustering these gestures using dynamic time warping...

This paper reports on a method developed for extracting and analyzing head gestures taken from motion capture data of spontaneous dialogue in Swedish. The head gestures were extracted automatically and then manually classified using a 3D player which displays time-synced audio and 3D point data of the motion capture markers together with animated characters. Prosodic features were extracted fro...

Todays multimodal systems, which allow full-body (3D) gestures and speech as input modalities, are quite restricted to easily interpretable coverbal gestures with a prede ned shape and meaning. In this paper, we propose methods to abstract the concrete shape of gestures by using high-level features and to integrate them with coexpressive words using their phonological attributes. The applicatio...

Alphabetic orthographies represent speech at the level ofthe phoneme. Yet the definition and functional status of the phonemic segment are widely viewed as problematic. Studies of child phonology have shown that the initial domain of a child's articulatory organization is the word rather than the segment, and have attributed a child's errors to failure to organize the features of a target word....

This paper presents gesture analysis under the scope of motor control theory. Following the motor program view, some studies have revealed a number of invariant features that characterize movement trajectories in human handarm gestures. These features express general spatio-temporal laws underlying coordination and motor control processes. Some typical invariants are described and illustrated f...