Multimodal interface aims at overcoming the shortcomings of conventional human-computer communication -serialized input, excessive accuracy, and low input efficiency -by exploiting the complementary nature of multiple modalities in interpreting user intent, thus improving the efficiency and naturalness of humancomputer interaction. In this paper, we present motivations driving multimodal interf...

A common tool for improving the performance quality of natural language processing systems is the use of contextual information for disambiguation. Here I describe the use of a finite state machine (FSM) to disambiguate speech acts in a machine translation system. The FSM has two layers that model, respectively, the global and local structures found in naturally-occurring conversations. The FSM...

Semantic communications focus on the transmission of semantic features. In this letter, we consider a task-oriented multi-user communication system for multimodal data transmission. Particularly, partial users transmit images while others texts to inquiry information about images. To exploit correlation among from multiple users, propose deep neural network enabled system, named MU-DeepSC, exec...