Research On Modeling Word Sense In Neural Machine Translation

Neural Machine Translation(NMT)provides a new approach to machine translation and has emerged as the most promising approach,due to its remarkable success over the state-of-the-art statistical machine translation(SMT)on various language pairs.However,NMT also has its own shortcomings.One of the main challenges lies in the unfaithful translation which leads to wrong translations.This paper explores the application of the word sense in NMT.The word sense provides useful information for the model to translate words correctly.By modeling word senses,the issue of wrong translation in traditional NMT models can be alleviated.The main content of this paper includes:(1)This paper analyses the reasons of wrong translation in NMT and evaluates word-level translation performance of different NMT models.Then,by simply viewing a word's translations as its word senses,this paper proposes a simple word dictionary approach,a random translation candidate approach,and a word sense disambiguation(WSD)approach to effectively model the word context for better word sense.(2)Once the word sense is obtained,this paper proposes three cross-lingual encoders to explicitly incorporate word senses into NMT:1)Factored encoder that encodes a word's sense as its feature and directly concatenates word embedding and sense embedding as input;2)Gated encoder that uses a gated mechanism to selectively control the amount of word sense moving forward;and 3)Mixed encoder that learns a word and its sense over sequences where words and their senses are alternatively mixed.Experimentation on Chinese-to-English translation demonstrates that the proposed encoders are able to improve the translation accuracy.(3)Splitting words into subwords is a widely used strategy in NMT to address the translation of rare and unknown words.In order to evaluate the effect of our approach in subword-based NMT,this paper first proposes to obtain subword senses and then applies the subword senses to the subword-based NMT.Experimental results show that our approach also benefits subword-based NMT.