Models that learn semantic representations from both linguistic and perceptual input outperform text-only models in many contexts and better reflect human concept acquisition. However, experiments suggest that while the inclusion of perceptual input improves representations of certain concepts, it degrades the representations of others. We propose an unsupervised method to determine whether to ...

We participated in the high-level feature extraction task in TRECVID 2007. This paper describes the details of our system for the task. For feature extraction, we propose an EMD-based bag-of-feature method to exploit visual/spatial information, and utilize WordNet to expand semantic meanings of text to boost up the generalization of detectors. We also explore audio features and extract the moti...

This chapter presents a physical-computational model of sensory-motor grounded language interpretation for simple speech acts. The model is based on an implemented conversational robot. It combines a cybernetic closed-loop control architecture with structured conceptual schemas. The interpretation of directive and descriptive speech acts consists of translating utterances into updates of memory...

This paper investigates intonational strategies for backchanneling in Bari Italian task-oriented dialogues. Results show that whilst giving feedback, listeners indicate their intention to take the floor or not by using a falling or rising intonation contour respectively. However, this general tendency can be reversed if added paralinguistic meanings are conveyed.

Pattern matching, an important feature of functional languages, is in conflict with data abstraction and extensibility, which are central to object-oriented languages. Modal abstraction offers an integration of deep pattern matching and convenient iteration abstractions into an object-oriented setting; however, because of data abstraction, it is challenging for a compiler to statically verify p...

We present a logic language with encapsulation based on an object-oriented data model. We give a formal account of the semantics of this language by defining a proof theory, a model theory and a fixed point theory. We also show that the different characterizations of the semantics are equivalent. We then study the logical foundations of encapsulation by introducing a mapping from our language t...

We propose multi-modal story-oriented video summarization (MMSS) which, unlike previous works that use fine-tuned, domain-specific heuristics, provides a domain-independent, graph-based framework. MMSS uncovers correlations between information of different modalities and gives meaningful story-oriented news video summaries. MMSS can also be applied for video retrieval, achieving performance tha...

We present a logic language with encapsulation based on an object-oriented data model. We give a formal account of the semantics of this language by deening a proof theory, a model theory and a xed point theory. We also show that the diierent characterizations of the semantics are equivalent. We then study the logical foundations of encapsulation by introducing a mapping from our language to a ...