Research On Example-Guided Neural Machine Translation

The example-guided neural machine translation studies a special translation scenario.In this scenario,the model will retrieve a similar example pair in addition to the given source sentence.At this time,the example pair can play an external knowledge role to help improve the translation quality of source sentences.Typical applications of this translation scenario include the fusion of translation memories,instance-driven domain adaptation,etc.,and therefore have important research and application value.As a result,this paper conducts an example-guided research on neural machine translation.The main contents are as follows:(1)Example-guided neural machine translation based on gating mechanism.In order to introduce example pairs into the translation model,this paper proposes to use an additional example encoder to encode example target.However,it is difficult for the retrieved example pairs to be completely consistent with the source sentence.Directly introducing the example pairs will undoubtedly bring irrelevant noise into the model.In order to introduce useful information in example pairs while avoiding the effects of noise,we control the flow of information through a gating mechanism.The experimental results in Chinese-English cor-pus demonstrate the effectiveness of proposed method.(2)Example-guided neural machine translation based on noise masked encoder and auxiliary decoder.The gating mechanism is a more implicit method to avoid the noise of example pairs.After obtaining the example pair and the source sentence,we can directly compare their source to obtain the matching parts and mismatching parts explicitly.Later,with the word alignment tool,we can use a special symbol to mask out the noise content of the target example.This method can directly remove the noise in the example pairs,but the word alignment information may also be wrong.We further propose an auxiliary decoder model and introduce an auxiliary decoder to help the model further learn the information of the example pairs.Through joint training and parameter sharing,the standard decoder can better reuse the matching parts in the example pairs with the help of the auxiliary decoder.We have conducted experiments on Chinese-English,English-German and English-Spanish corpus,and the experimental results and analysis show that proposed methods can be im-proved in the example pairs with any degree of similarity.(3)Example-guided neural machine translation based on data augmentation.The above two methods are mainly from the perspective of the model structure,and the example infor-mation is encoded by the example encoder,which will increase the parameters.In addition to merging example pairs into the translation process in this way,we also propose a method based on data augmentation to fuse example pairs.We spliced the retrieved target example after the source sentence,and distinguished them by the label vector to expand the scale of the data.The experimental results show that stitching example pairs after the source sentence can bring more benefits than directly adding them to the model as training data.