In this paper we present a novel user interface that integrates two popular approaches to language translation for travelers allowing multimodal communication between the parties involved: the picture-book, in which the user simply points to multiple picture icons representing what they want to say, and the statistical machine translation (SMT) system that can translate arbitrary word sequences...

We tackle the problem of machine translation manga, Japanese comics. Manga involves two important problems in translation: context-aware and multimodal translation. Since text images are mixed up an unstructured fashion Manga, obtaining context from image is essential for manga However, it still open how to extract integrate into MT models. In addition, corpus benchmarks train evaluate such mod...

Given a (static) scene, a human can effortlessly describe what is going on (who is doing what to whom, how, and why). The process requires knowledge about the world, how it is perceived, and described. In this paper we study the problem of interpreting and verbalizing visual information using abstract scenes created from collections of clip art images. We propose a model inspired by machine tra...

backround: we aimed to evaluate analgesic efficacy, opioid-sparing, and opioid-related adverse effects of intravenous paracetamol and intravenous dexketoprofen trometamol in combination with iv morphine after total abdominal hysterectomy. materials and methods: sixty american society of anesthesiologist physical status classification i-ii patients scheduled for total abdominal hysterectomy were...

This paper presents the multimodal Interlingual Map Task Corpus (ILMT-s2s corpus) collected at Trinity College Dublin, and discuss some of the issues related to the collection and analysis of the data. The corpus design is inspired by the HCRC Map Task Corpus which was initially designed to support the investigation of linguistic phenomena, and has been the focus of a variety of studies of comm...

We present a dataset and method for improving the translation of noisy image captions that were created by users of Wikimedia Commons. The dataset is multilingual but non-parallel, and is several orders of magnitude larger than existing parallel data for multimodal machine translation. Our retrieval-based method pivots on similar images and uses the associated captions in the target language to...

This paper introduces and summarises the findings of a new shared task at the intersection of Natural Language Processing and Computer Vision: the generation of image descriptions in a target language, given an image and/or one or more descriptions in a different (source) language. This challenge was organised along with the Conference on Machine Translation (WMT16), and called for system submi...

Visual context provides grounding information for multimodal machine translation (MMT). However, previous MMT models and probing studies on visual features suggest that is less explored in as it often redundant to textual information. In this paper, we propose an Object-level Context modeling framework (OVC) efficiently capture explore translation. With detected objects, the proposed OVC encour...

A new participatory ecology of translation facilitated by digital technologies has significant implications for understanding and translators. This article examines YouTube comment on Bilibili in China to reconceptualize translators taking the Will Smith-Chris Rock confrontation at Oscars 2022 assassination Shinzo Abe as two illustrative case studies. It demonstrates that Chinese netizens parti...