Multi-source Information Enhanced End-to-end Neural Machine Translation

In recent years,the rapid development of machine translation technology has brought great convenience to the communication between people of different countries.Machine translation refers to the transformation of a modal language representation into another language representation.Conventional machine translation translates text or speech in the source language into text in the target language.However,due to the diversity of languages and the ambiguity of semantic expression,neural machine translation models that only rely on single source information as input often have mistranslations.With the development of multi-modal learning and related cross-fields,not only text can be combined with knowledge such as image information or part-of-speech information,but also information complementarity between different modalities becomes possible.Therefore,this paper is centered on the multi-source information enhanced end-to-end neural machine translation,so that multi-source information can be better integrated into the machine translation model.Firstly,this paper points out the difficulty of semantic understanding in the conventional machine translation model.Neural machine translation systems based on a standard encoderdecoder framework encode a source sentence to generate a target sentence,in which the encoder treats all words equally.It's natural to understand that content words containing descriptive information of a sentence should be considered more important than function words.In order to focus on content words and help machine translation models to understand the key semantic information,this paper utilizes part-of-speech tags to extract the content words of the source sentence to form its condensed version and design a condensed sentences augmented neural machine translation model.Experimental results on mainstream datasets show that the proposed method significantly improves the translation quality.Secondly,this paper points out the difficulty of text generation in the machine translation model based on sub-word preprocessing method,and defines a wrong translation type called "half right and half wrong",which is intuitively reflected in the phenomenon of wrong translation of a few characters in the translated words.In order to solve this problem,this paper introduces part of speech tag information to the decoder to strengthen the association of different sub-words in the same word.Experimental results on mainstream datasets show that the proposed method improves the translation quality based on the model proposed above.Finally,this paper also introduces multi-source information into speech translation model,and add corresponding video information for end-to-end speech translation system.This method is suitable for video caption generation.Experimental results on multimodal datasets show that the proposed method effectively improves the quality of speech translation.In summary,the multi-source information enhanced end-to-end neural machine translation which is proposed in this paper can effectively improve the translation quality in both text translation and speech translation.