Research On The Improvement Of Neural Machine Translation With Recurrent Attention Modeling

With the deep research in the field of deep learning and machine translation,neural machine translation(NMT)based on the attention mechanism gradually replaced statistical machine translation(SMT).The improvement of the attention mechanism has become a research hotspot in machine translation.In recent years,attention has been paid to improving the attention mechanism of these neural machine translations.The attention information at each time step is still independent of each other;and in the 2017 EACL,Yang et al.proposed a circular attention model that uses the attention information in each time step to generate dynamic memory information through the recurrent network structure,thereby improving translation performance;this topic is based on the problems of Yang et al.Two aspects of improvement research:1.It proposes a Recurrent Attention Mechanism that introduces automatic borders.Since the model proposed by Yang et al.empirically designs the window size of historical attention through a hyperparameter,this does not conform to the intuitive feeling of contextual information with semantic boundaries.Therefore,the proposed model structure can improve the window fixing problem of historical attention information combination,and proposes two designs of symmetric and non-symmetrical borders to realize the automatic regulation of borders.2.It proposes a Recurrent Attention Mechanism that introduces weight control.The model proposed by Yang et al.does not discriminate historical information at different times in the process of transmitting contextual history information.We clearly know that different contextual information has different degrees of influence on translation.Therefore,we use historical attention to control the model.To achieve the weight control of historical attention information,and then to achieve different context information has different degrees of impact on translation,in addition we also integrate the asymmetric boundary proposed asymmetric border-historical attention regulation model.We have constructed four neural machine translation models using two improved methods.Compared with the studies of Yang et al.,our models all achieved better performance in terms of Bleu scores.At the same time,they also showed better results in the test results and translation performance.