This paper describes the University of Sheffield’s submission for the WMT16 Multimodal Machine Translation shared task, where we participated in Task 1 to develop German-to-English and Englishto-German statistical machine translation (SMT) systems in the domain of image descriptions. Our proposed systems are standard phrase-based SMT systems based on the Moses decoder, trained only on the provi...

This paper describes Oregon State University’s submissions to the shared WMT’17 task “multimodal translation task I”. In this task, all the sentence pairs are image captions in different languages. The key difference between this task and conventional machine translation is that we have corresponding images as additional information for each sentence pair. In this paper, we introduce a simple b...

We present a doubly-attentive multimodal machine translation model. Our model learns to attend to source language and spatial-preserving CONV5,4 visual features as separate attention mechanisms in a neural translation model. In image description translation experiments (Task 1), we find an improvement of 2.3 Meteor points compared to initialising the hidden state of the decoder with only the FC...

The aim of the article is to discuss specifics a multimodal pun functioning in English source text popular science discourse and its translation into Ukrainian. Multimodal considered as combination two inhomogeneous elements – verbal visual, that variety intersemiotic game/play intended for producing humorous effect. While leading role belongs component, visual one performs either creative or a...

Under the guidance of Systemic-functional synthetic framework for multimodal discourse analysis, this paper collects corpus from classical American TV show Friends and undertakes a thorough qualitative analysis how cross-modal relations influence Chinese translation English subtitles. As is shown, in order to give viewers better understanding subtitles contexts, different translational a...

Automatic generation of video descriptions in natural language, also called captioning, aims to understand the visual content and produce a language sentence depicting objects actions scene. This challenging integrated vision problem, however, has been predominantly addressed for English. The lack data linguistic properties other languages limit success existing approaches such languages. In th...

Many image-to-image translation problems are ambiguous, as a single input image may correspond to multiple possible outputs. In this work, we aim to model a distribution of possible outputs in a conditional generative modeling setting. The ambiguity of the mapping is distilled in a low-dimensional latent vector, which can be randomly sampled at test time. A generator learns to map the given inp...

The multimodal teaching model originates from the discourse analysis. theory of analysis mainly emphasized important role social symbols such as images and actions in communication. In past twenty years, has gradually entered field education. practice language teaching, it is helpful to improve effectiveness by learning give full play synergy participation. Business English translation course c...