Interpreters as laminated speakers: Gaze and gesture as interpersonal deixis in consecutive dialogue interpreting

Understanding multimodal deixis with gaze and gesture in conversational interfaces

Interpreting Situated Dialogue Utterances: an Update Model that Uses Speech, Gaze, and Gesture Information

In situated dialogue, speakers share time and space. We present a statistical model for understanding natural language that works incrementally (i.e., in real, shared time) and is grounded (i.e., links to entities in the shared space). We describe our model with an example, then establish that our model works well on nonsituated, telephony application-type utterances, show that it is effective ...

Gaze direction as a backchannel inviting cue in dialogue

In this study, we experimentally explore the relationship between gaze direction and backchannels in face-to-face interaction. The overall motivation is to use gaze direction in a virtual agent as a mean to elicit user feedback. The relationship between gaze and backchannels was tested in an experiment in which participants were asked to provide feedback when listening to a storytelling virtual...

Pointing as an Instrumental Gesture: Gaze Representation Through Indication

We call those gestures “instrumental” that can enhance certain thinking processes of an agent by offering him representational models of his actions in a virtual space of imaginary performative possibilities. We argue that pointing is an instrumental gesture in that it represents geometrical information on one’s own gaze direction (i.e., a spatial model for attentional/ocular fixation/orientati...

party observer gaze as a continuous measure of dialogue flow

We present an attempt at using 3 party observer gaze to get a measure of how appropriate each segment in a dialogue is for a speaker change. The method is a step away from the current dependency of speaker turns or talkspurts towards a more general view of speaker changes. We show that 3 party observers do indeed largely look at the same thing (the speaker), and how this can be captured and uti...