This paper aims to show where ubiquitous sensor environments in combination with multimodal user interfaces can be used to further enhance the user (and usage) experience. The challenge in multimodal user interface design is to combine the available user input modalities with the available sensor data in a meaningful way. A conceptual framework for sensorised modality input composition and sema...

Multimodal biometrics technology that uses more than two sorts of biometrics data has been universally applied for person certification and proof. Researchers have advised that the ear may have benefits over the face for biometric recognition. In this study, a technique for face and ear recognition has suggested. The face image and ear images are prearranged as input. From the pre-processed inp...

In this paper we present the MUMS Multimodal Route Navigation System which combines speech, pen, and graphics into a PDA-based multimodal system. We focus especially on the three-level modality fusion component which we believe provides an accurate and more flexible input fusion than the usual twolevel approaches. The modular architecture of the system supports flexible component management, an...

background many multimodal analgesia techniques have been tried to provide adequate analgesia for midline incisions extending above and below the umbilicus aiming at limiting the perioperative use of morphine thus limiting side effects. ultrasound (us) guidance made the anesthesiologist reconsider old techniques for wider clinical use. the rectus sheath block (rsb) is a useful technique under-u...

Activity recognition from computer vision plays an important role in research towards applications like human computer interfaces, intelligent environments, surveillance or medical systems. In this paper, we propose a gesture recognition system based on a deep learning architecture and show how it performs when trained with changing multimodal input data on an Italian sign language dataset. The...

The use of speech-accompanying hand gestures to depict objects, to structure the discourse or to give directions is ubiquitous in face-to-face interaction. In this paper, we analyse multimodal signals consisting of speech and gesture from the perspective of semantic composition in constraint-based grammars: we elevate standard methods from linguistics to description of multimodal input so as to...

In this article, we present the MMIR (Multimodal Mobile Interaction and Rendering) framework that is targeted at building lightweight dialog systems. The framework is geared towards multimodal interaction development for applications where only limited resources are available, e.g. so that they can run self-contained even on mobile devices. The framework utilizes SCXML for modeling application-...

User interfaces have emerged from command lines to direct manipulation and now faces a new generation. From the user’s point of view, intelligent interfaces should provide adaptivity, context sensitivity and more assistance with the task at hand. To achieve these goals, the system needs to understand multimodal and possibly imprecise input, generate coordinated multimodal presentations and mana...

This paper presents a multimodal biometric fusion algorithm that supports biometric image quality and case-based context switching approach for selecting appropriate constituent unimodal traits and fusion algorithms. Depending on the quality of input samples, the proposed algorithm intelligently selects appropriate fusion algorithm for optimal performance. Experiments and correlation analysis o...

Process parameter window selection in semiconductor manufacturing field is usually the problem to find out the ranges of input parameters that meet production requirements, which requires allocating optima of a multimodal function efficiently. To achieve good results under the conditions of multimodal model and process control requirement, a NichePSO algorithm based method for parameter window ...