Even in inter−human dialogues there are situations where it is not clear whether the speaker will continue with his/her speech or whether the dialogue partner wants to take the next turn. Of course, regarding multimodal dialogues, we not only have speech for input and output but also gestural input or graphical output. In this paper we concentrate on the linguistic analysis of the user’s lingui...

Multimodal interfaces, combining the use of speech, graphics, gestures, and facial expressions in input and output, promise to provide new possibilities to deal with information in more effective and efficient ways, supporting for instance: the understanding of possibly imprecise, partial or ambiguous multimodal input; the generation of coordinated, cohesive, and coherent multimodal presentatio...

Speech and gesture are two types of multimodal inputs that can be used to facilitate more natural humanmachine interaction in applications for which the traditional keyboard and mouse input mechanisms are inappropriate, however the possibility of their concurrent use raises the issue of how best to fuse the two inputs. This paper analyses data collected from a speech and manual gesture-based di...

In this article, after briefly reviewing some previous work on the design of multimodal HCI, we present two projects, the first results obtained from experimental studies, and draw conclusions on design principles for multimodal interfaces. ON THE DESIGN OF MULTIMODAL HCI Classical design principles for HCI (e.g. Mayhew, 1999) recommend conducting iterative evaluations at different stages in th...

Multimodal interaction is a way to make user interfaces natural and efficient with parallel and synergistic use of two or more input or output modalities. Two-handed interaction is a special case of multimodal interaction that makes use of both hands in a combined and coordinated manner. This dissertation gives a comprehensive survey on issues that are related to multimodal and twohanded intera...

Multimodal interfaces, combining the use of speech, graphics, gestures, and facial expressions in input and output, promise to provide new possibilities to deal with information in more effective and efficient ways, supporting for instance: the understanding of possibly imprecise, partial or ambiguous multimodal input; the generation of coordinated, cohesive, and coherent multimodal presentatio...

Multimodal interaction is a way to make user interfaces natural and efficient with parallel and synergistic use of two or more input or output modalities. Two-handed interaction is a special case of multimodal interaction that makes use of both hands in a combined and coordinated manner. This dissertation gives a comprehensive survey on issues that are related to multimodal and twohanded intera...

Multimodal interaction is a way to make user interfaces natural and efficient with parallel and synergistic use of two or more input or output modalities. Two-handed interaction is a special case of multimodal interaction that makes use of both hands in a combined and coordinated manner. This dissertation gives a comprehensive survey on issues that are related to multimodal and twohanded intera...

Multimodal interfaces enable more natural and effective humancomputer interaction by providing multiple channels through which input or output may pass. In order to realize their full potential, they need to support not just input from multiple modes, but synchronized integration of semantic content from different modes. This paper describes a multimodal language processing architecture which a...

Designing and building automated systems with which people can interact naturally is one of the emerging objective of Mechatronics. In this perspective multimodality and adaptivity represent focal issues, enabling users to communicate more freely and naturally with automated systems. One of the basic problem of multimodal interaction is the fusion process. Current approaches to fusion are mainl...