Quantifying the speech-gesture relation with massive multimodal datasets: Informativity in time expressions
نویسندگان
چکیده
منابع مشابه
Temporal Expressions in Speech and Gesture
People use spatial metaphors to talk about temporal concepts. They also gesture frequently during speech. The characteristics of these gestures give information regarding the mental timelines people form to experience time. The present study investigates the expression of temporal concepts on a natural setting with Turkish speakers. We found that Turkish speakers used more metaphoric temporal p...
متن کاملGesture and Speech Multimodal Conversational Interaction
Gesture and speech combine to form a rich basis for human conversational interaction. To exploit these modalities in HCI, we need to understand the interplay between them and the way in which they support communication. We propose a framework for the gesture research done to date, and present our work on the cross-modal cues for discourse segmentation in free-form gesticulation accompanying spe...
متن کاملMultimodal Complex Emotions: Gesture Expressivity and Blended Facial Expressions
One of the challenges of designing virtual humans is the definition of appropriate models of the relation between realistic emotions and the coordination of behaviors in several modalities. In this paper we present the annotation, representation and modeling of multimodal visual behaviors occurring during complex emotions. We illustrate our work using a corpus of TV interviews. This corpus has ...
متن کاملImprovement of multimodal gesture and speech recognition performance using time intervals between gestures and accompanying speech
We propose an integrative method of recognizing gestures such as pointing, accompanying speech. Speech generated simultaneously with gestures can assist in the recognition of gestures, and since this occurs in a complementary manner, gestures can also assist in the recognition of speech. Our integrative recognition method uses a probability distribution which expresses the distribution of the t...
متن کاملThe influence of target size and distance on the production of speech and gesture in multimodal referring expressions
In this paper we report on a production experiment for multimodal referring expressions. Subjects performed an object identification task in an interactive setting. 20 subjects participated and were asked if they could identify 30 countries on a world map on the wall. Subjects performed their tasks on two distances: close (10 subjects) and at a distance of 2.5 meters (10 subjects). The assumpti...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: PLOS ONE
سال: 2020
ISSN: 1932-6203
DOI: 10.1371/journal.pone.0233892