Research On Key Issues In Low-resource Neural Machine Translation

Neural Machine Translation(NMT)has achieved unprecedented achievement with the rapid development of machine learning and deep learning,as well as the rapid improvement of computer hardware capabilities.Although benefiting from the superior expression ability of many kinds of deep neural network,the model can automatically learn the characteristics to a certain extent,but the scarcity of corpus,sparse data,single semantic expression makes the learning representation ability of the model unable to give full play,which is also the key factor to hinder the further development of low-resource NMT.These inherent features can lead to a range of translation problems,such as UNK problem,poor generalization ability of model,reference error and overfitting problem.This thesis focuses on some key issues in low-resource NMT based on deep learning.We focused on in-depth analysis of the cause and nature of the problem,and proposed corresponding solutions on this basis,and conducted experimental verification and analysis on multiple low-resource translation tasks.The research content of this thesis mainly focuses on the following aspects:1.Aiming at the problem of exposure bias and consistency of evaluation indicators,this thesis applies reinforcement learning to low-resource NMT,and uses target reward mechanism and dynamic sampling algorithm to make NMT guide training through evaluation indicators.We also try to apply value iteration to the reinforcement training process to explore the impact of different granularities on the sequence decoding process.Therefore,this thesis proposes a training strategy based on free-granular input,which uses the advantages of each granularity in the decoding stage to solve the problem of single semantic expression in low-resource tasks.2.Aiming at the problem of semantic loss in reinforcement training,this thesis proposes a strategy that takes the semantic loss based on the cosine angle between sequences as one of the optimization objectives,and explores how to improve the readability of the translation while effectively improving the BLEU(Bilingual Evaluation Understudy)score.3.Aiming at the problems of UNK and referential errors in low-resource NMT,this thesis proposes a noise generalization training strategy,which uses the game training mechanism of Generative Adversarial Networks to make the model generalize the added noise during the training process.The noise strategy converts the cause of the problem into sequence noise,which is then generalized by the model in the adversarial training,so as to solve the problem caused by a fixed cause in a type of NMT.Since the strategy itself is not constrained by the model or the form of noise,it may have a reference value for identified reasons in natural language processing.4.Aiming at the prominent over-fitting problem in low-resource NMT,this thesis proposes a fusion drop method to obtain better weights and node distribution.We have verified the rationality and effectiveness of the proposed method on multiple low-resource language tasks for the above key issues.The main experimental results and contributions include:(1)The reinforcement training method based on dynamic sampling can effectively solve the problem of exposure bias and inconsistent evaluation indicators.The method improves the baseline by 2-4 BLEU scores on three low-resource tasks,(2)The proposed reinforcement training method based on semantic constraints can effectively alleviate the phenomenon of "high BLEU score--low readability" in the translation,and significantly improve the fluency of the translation on the basis of ‘(1)',(3)The proposed free granularity training strategy can provide more rich semantic information for input.In addition,it can accurately find the appropriate decoding granularity through value iteration.Compared with the baseline system,the model improves 1-5 BLEU scores on three multi granularity low resource language translation tasks,and the value iteration module can shorten the training time by nearly 1/3 compared with the baseline systems,(4)The proposed noise training strategy significantly reduces the number of symbols of <9)6)>.For the reference error problem,the BLEU score can be increased by 2 points on average in(20×10000)iteration steps,(5)With reasonable probability distribution,the number of training subnets provided by the proposed method is exponentially higher than that of Dropout and Dropconnect.It is verified that the model training efficiency of the low resource task in the experiment is the highest when the probability between [0.2-0.3].