This article proposes a methodology to create a multimodal corpus that can be shared with a group of researchers in order to analyze synchronous online pedagogical interactions. Epistemological aspects involved in studying online interactions from a multimodal and semiotic perspective are addressed. Then, issues and challenges raised by corpus creation and sharing are examined with a particular...

We present in this paper a formal approach for the representation of multimodal information. This approach, thanks to the to use of typed feature structures and hypergraphs, generalizes existing ones (typically annotation graphs) in several ways. It first proposes an homogenous representation of different types of information (nodes and relations) coming from different domains (speech, gestures...

Despite a multiliteracies turn to understanding L2 writing activities, most scholars in China have continued to adopt a cognitive perspective to study Chinese students’ English composing. In this article, we argue that Chinese students have always composed English writing in a multilingual and multimodal fashion. Then, drawing on recent discussions in literacy and composition studies, we propos...

The use of speech-accompanying hand gestures to depict objects, to structure the discourse or to give directions is ubiquitous in face-to-face interaction. In this paper, we analyse multimodal signals consisting of speech and gesture from the perspective of semantic composition in constraint-based grammars: we elevate standard methods from linguistics to description of multimodal input so as to...

Film and television works are multimodal discourse composed of a variety symbol systems such as text, sound image, so the audience's various senses can be mobilized at same time when watching movies. Starting from perspective analysis, this paper applies Delu Zhang’s theoretical framework analysis to analyze subtitle translation Harry Potter Philosopher's Stone four aspects: culture, context, c...

Abstract. Multimodal feedback needs to be studied and researched enhance student’s ability. Thus, this research aims examine the which using technology conventional without in speaking, wanted explore how multimodal could influence students’ speaking ability, know perception on reason improve The used a sequential mixed method design with test interview as instruments. sample was 54 students fo...

Split-hom together structure is a new heavy load rack structure, and its reliability needs to be verified. Through analyzing multimodal hom-connection principle from the perspective of bionics and contact mechanics, the article puts forward three kinds of hom-connection methods and establishes the finite element model and mathematical model of the rack. Afterwards, the finite element model and ...