Advanced Research On Constrained Neural Machine Translation

Under the rapid growing of economic and trade globalization,the magnitude of information exchange among countries is huge.Machine translation(MT)plays an important role in this process.Various MT systems need to have the ability of using external dictionaries,so that to quickly improve the translation quality.How to impose MT’s translation satisfy the pre-specified translation and obtain better translation quality is an indispensable ability of a MT system.Neural Machine Translation(NMT)has become the mainstream MT method in recent years,NMT’s end-to-end model architecture makes it difficult to leverage prespecified or user-provided translations to affect the translation results.There have been a few researches on task of constraining NMT,which are mainly based on two kinds of ideas:a data augmentation method and a constrained decoding method.The main problem of the existing methods is that the translation quality of the entire sentence is lost while imposing the pre-specified translations coming into effect.Different from the existing methods,the method proposed in this paper can not only impose the effectiveness of the pre-specified translations,but also avoid the negative affect to the overall translation quality.This article is composed of the following three parts:(1)Constrained NMT based on replacement and copyThis chapter investigate a novel method based on data augmentation.The idea is to construct synthetic parallel sentences from the original parallel training data.The synthetic sentence pairs resemble code-switched source sentenees and their translations,where certain source words are replaced with their eorresponding target translations.The motivation is to make the model learn to"translate" embedded pre-specified translations by copying them from the modified source.The translation quality of Constrained NMT is better than the original NMT,and it is also better than the existing Constrained NMT methods.(2)Constrained NMT based on word alignment optimization and constrained decodingThis chapter investigate a method of employing pre-specified translations to guide NMT decoding.A dedicated attention head is used to learn the word alignment based on supervision from external alignment signals.During decoding,if source pre-specified word is aligned to current decoding word,the pre-specified is forced to be current translation.The proposed method can obtain significantly better word alignment than vanilla NMT system,leading to improvement in dictionary-guided decoding.(3)Improve word alignment based on pre-trained language model and guide the constrained decodingThe proposed method make use of monolingual data,which is more abundantly available than bilingual data,to better capture the similarity of words within and across languages.A cross-lingual language model which is pre-trained on unpaired monolingual text of source and target languages,is used to initialize the both the Transformer part and a dedicate alignment layer.Experiments on benchmark data sets show that the proposed method are effective.The experiments further proves that better Transformer’s word alignment lead to better constraint translation quality.This paper includes two effective methods on constraining NMT with pre-specified translations:data augmentation based method and dictionary-guided decoding based method.In addition,this paper proposes two methods to improve the accuracy of Transformer’s word alignment,which can lead to better effect of constraining NMT with provided translations.