Comparing to graphical user interfaces, speech-only interfaces face several problems: robustness, making clear what functionality is available, and making clear how the functionality may be accessed. We explore a potential solution for these problems by presenting a visual representation of the domain of discourse and of the state of the dialogue. We describe an experiment in which uni-modal an...

We consider multi-modal logics interpreted over edge-labelled graphs with a modality 〈#〉, where 〈#〉φ means ‘φ is accessible by an edge with some label’. In a logic with finitely many edge labels, 〈#〉 is definable, but if the set of labels is infinite, it is an independent modality. We axiomatise multi-modal K, deterministic multi-modal K, and PDL with converse and a single nominal, extended wit...

This paper establishes undecidability of satisfiability for multi-modal logic equipped with the hybrid binder ↓ , with respect to frame classes over which the same language with only one modality is decidable. This is in contrast to the usual behaviour of many modal and hybrid logics, whose uni-modal and multi-modal versions do not differ in terms of decidability and, quite often, complexity. T...

We present a comparative analysis of multi-modal user inputs with speech and pen gestures, together with their semantically equivalent uni-modal (speech only) counterparts. The multimodal interactions are derived from a corpus collected with a Pocket PC emulator in the context of navigation around Beijing. We devise a cross-modality integration methodology that interprets a multi-modal input an...

While the incipient internet was largely text-based, the modern digital world is becoming increasingly multi-modal. Here, we examine multi-modal classification where one modality is discrete, e.g. text, and the other is continuous, e.g. visual representations transferred from a convolutional neural network. In particular, we focus on scenarios where we have to be able to classify large quantiti...

One main group of a transportation network is a discrete hub covering problem that seeks to minimize the total transportation cost. This paper presents a multi-product and multi-mode hub covering model, in which the transportation time depends on travelling mode between each pair of hubs. Indeed, the nature of products is considered different and hub capacity constraint is also applied. Due to ...

In the field of virtual reality, there is much attention to the notion of multi-modal environments. These environments use various senses of the human body to achieve an overall immersion in the environment leading to an increased sense of presence. In addition, there is much work in collaborative architectures allowing users to interact with each other and the world itself. This tutorial intro...

Intelligent Transportation System (ITS) is one of the most important urban systems that its functionality affects other urban systems directly and indirectly. In developing societies, increasing the transportation system efficiency is an important concern, because variety of problems such as heavy traffic condition, rise of the accident rate and the reduced performance happen with the rise of p...

This paper introduces a multi-modal polymorphic type theory to model epistemic processes characterized by trust, defined as a second-order relation affecting the communication process between sources and a receiver. In this language, a set of senders is expressed by a modal prioritized context, whereas the receiver is formulated in terms of a contextually derived modal judgement. Introduction a...

In this paper, we report on the design and implementation of a multi-modal user interface of the virtual world database system (VWDB). The VWDB is currently under development at Ochanomizii University with the aim of realizing a new generation database system for cyberspace applications. The system will be constructed by integrating a virtual reality system and an object-oriented database syste...