| In recent years,neural machine translation based on autoregressive decoding has achieved great success,its translation quality has closed to human-beings.However,the autoregressive decoding method can’t efficiently use GPU for parallel acceleration.When faces to long input,high delay is very obvious.In order to speed up translation,nonautoregressive translation models have been widely studied.The NAT models assume that the target words are independent from each other so that all words can be output in parallel,usually generating the final result in one step or a certain number of steps.Compared with the autoregressive model,it has a great acceleration effect.However,the translation quality of NAT model differs greatly from that of autoregressive model due to the lack of dependency between words.In order to improve the translation quality of NAT model and narrow the gap between NAT model and AT model,based on extensive research on NAT model,this paper optimizes two different types of NAT machine translation models in view of the shortcomings of NAT model.(1)Iterative NAT model based on N-gram mask optimization.By studying the iterative NAT model CMLM and analyzing its shortcomings,two training strategies are proposed to optimize the model: N-gram mask and consistency learning regularization.The N-gram mask strategy helps the model capture coarse-grained information of the target language,alleviates the shortcomings of the "strong independence" hypothesis to a certain extent,and thus improves the translation quality of the model.Consistency learning regularization is used to improve the robustness of the model training.(2)Optimized fully NAT model based on gating mechanism and curriculum learning.By studying the principles of a large number of complete NAT models,this paper points out the main reasons that limit their translation quality,and puts forward two optimization strategies accordingly: an enhanced input mechanism based on gating and a training strategy based on curriculum learning.The gating mechanism is used to dynamically integrate the original text word vector and high-level semantic feature information to enhance the decoder input information.The training strategy based on curriculum learning can dynamically adjust the difficulty of the learning task according to the current performance of the model,so that the model can realize the learning "from easy to difficult" and slowly approach from local optimal to global optimal.Extensive experiments conducted on public commonly used machine translation benchmark show that the proposed method is effective.Compared with the autoregressive baseline model Transformer,the iterative mask NAT model achieves about 1.3 times the acceleration effect and the average translation quality difference is only 0.2 BLEU.On the other hand,the Fully NAT model based on curriculum learning behind 1.5 BLEU on average in translation quality compared to Transformer while speeding up decoding 8.6times.The two NAT models compromise translation quality and acceleration effect to varying degrees,and both have their own applicable scenarios and application values. |
