Although there are various ways to represent data patterns and models, visualization has been primarily taught in many science courses for its efficiency. Such vision-dependent output may cause critical barriers against those who blind visually impaired people with learning disabilities. We argue that instructors need teach multiple representation methods so all students can produce products mo...

With the fast development of data collection techniques, a huge amount of complex multi-modal data are generated, shared and stored on the Internet. The burden of extracting multi-modal features for test instances in data analysis becomes the main fact that hurts the efficiency of prediction. In this paper, in order to reduce the modal extraction cost in serialized classification system, we pro...

We present a comparative analysis of multi-modal user inputs with speech and pen gestures, together with their semantically equivalent uni-modal (speech only) counterparts. The multimodal interactions are derived from a corpus collected with a Pocket PC emulator in the context of navigation around Beijing. We devise a cross-modality integration methodology that interprets a multi-modal input an...

In recent years, camera networks have been widely employed in several application domains such as surveillance, ambient intelligence or video conferencing. The integration of heterogeneous sensors can provide complementary and redundant information that fused to visual cues allows the system to obtain an enriched and more robust scene interpretation. A discussion about possible architectures an...

For the design of component-based software, the behavioral specification of component interfaces is crucial. We propose an extension of the theory of modal I/O-transition systems by Larsen et al. to cope with both control flow and data states of reactive components at the same time. In our framework, transitions model incoming or outgoing operation calls which are constrained by preand postcond...

The MISTRAL system, a service oriented architecture for semantic extraction of multimedia data from meeting recordings is described shortly. It improves on other similar systems by extracting a variety of semantic metadata from one media type and integrating it with concepts derived from other media types, as well as by adding inference capabilities to resolve ambiguities and further enrich ext...

We study the problem of automatically building hypernym taxonomies from textual and visual data. Previous works in taxonomy induction generally ignore the increasingly prominent visual data, which encode important perceptual semantics. Instead, we propose a probabilistic model for taxonomy induction by jointly leveraging text and images. To avoid hand-crafted feature engineering, we design end-...

This paper investigates audio-visual interaction, i.e. inter-modal influences, in linear-regressive model adaptation for multi-modal speech recognition. In the multi-modal adaptation, inter-modal information may contribute the performance of speech recognition. Thus the influence and advantage of intermodal elements should be examined. Experiments were conducted to evaluate several transformati...

Clustering groups records that are similar to each other into the same group, and those that are less similar into different groups. Clustering data of mixed types is difficult due to different data characteristics. Extending Gower’s metric for nominal and ordinal data is incorporated into an agglomerative hierarchical clustering algorithm to cluster mixed type data. This paper describes the ex...