This paper introduce one innovations into the world of multimodal function optimization: a new Cultural Algorithm (CA) based on Genetic Algorithm (GA) with a new adaptive mutation (AM-CGA). The idea is to use knowledge about local optima found during the search with other techniques utilized for escape from local minimum in multimodal functions. The search knowledge was maintained using a Cultu...

During face-to-face conversation, people naturally integrate speech, gestures and higher level language interpretations to predict the right time to start talking or to give backchannel feedback. In this paper we introduce a new model called Latent Mixture of Discriminative Experts which addresses some of the key issues with multimodal language processing: (1) temporal synchrony/asynchrony betw...

An advanced content-based sports video browsing and retrieval system, SportsVBR, is proposed in this work. Its main features include event-based sports video browsing and keyword-based sports video retrieval. The paper first defines the basic structure of our SportsVBR system, and then introduces a novel approach that integrates multimodal analysis, such as visual streams analysis, speech recog...

This paper presents a novel method for denoising MR images that relies on an optimal estimation, combining a likelihood model with an adaptive image prior. The method models images as random fields and exploits the properties of independent Rician noise to learn the higher-order statistics of image neighborhoods from corrupted input data. It uses these statistics as priors within a Bayesian den...

Facial expressions and speech are elements that provide emotional information about the user through multiple communication channels. In this paper, a novel multimodal emotion recognition system based on visual and auditory information processing is proposed. The proposed approach is used in real affective human robot communication in order to estimate five different emotional states (i.e., hap...

This paper focuses on the architecture of affective multimodal user interfaces that exist in e-learning applications. The OO (Object Oriented) approach has been adopted in order to combine evidence from multiple modalities of interaction and data from emotion stereotypes and classify them into well structured objects with their own properties and methods. The proposed architecture can be adopte...

Multimodal biometrics system becomes an inexorable trend in future. Multimodal biometrics overcomes many drawbacks over unimodal biometrics. Some of the drawbacks are non-universality, noisy data and spoof attack. Multimodal biometric is more efficient and accurate than unimodal system. In this paper we present a multimodal biometric recognition system using Iris and finger Inner-knuckle print....

Multimodal question answering in the cultural heritage domain allows visitors to museums, landmarks or other sites to ask questions in a more natural way. This in turn provides better user experiences. In this paper, we propose the construction of a golden standard dataset dedicated to aiding research into multimodal question answering in the cultural heritage domain. The dataset, soon to be re...

A system is classified as multimodal if its language has more than one modal operator as primitive, and such operators are not interdefinable. We extend the anodic and cathodic modal systems, introduced in [BS09a] and [BS10], to a class of the so-called basilar multimodal systems generating, in this way, the classes of anodic and cathodic multimodal logics. The cathodic multimodal systems are d...

|This paper presents a statistical approach to developing multimodal recognition systems and, in particular, to integrating the posterior probabilities of parallel input signals involved in the multimodal system. We rst identify the primary factors that innuence multimodal recognition performance by evaluating the multimodal recognition probabilities. We then develop two techniques, an estimate...