Designing and Executing Multimodal Interfaces for the Web based on State Chart XML

The design and implementation of multimodal interfaces that support a variety of modes to enable natural interaction is still limited. We propose a multimodal interaction web framework that considers on the one hand the current W3C standardization activities, such as Start Chart XML and the Model-Based UI Working Group. But on the other hand it implements recent research results enabling the direct execution of multimodal interfaces based on the information of the design models. This paper presents the basic concepts of modeling and executing multimodal interfaces for the web and illustrates our approach by presenting two case studies that have been implemented so far.