Research On Neural Machine Translation Based On Attention Convolution

With the development of society and technology,machine translation has become an indispensable part of people's daily life.In recent years,machine translation with neural networks has gradually become the mainstream method in industry and academia due to the development of deep learning.The existing neural machine translation generally adopts a sequence-to-sequence translation model of an encoder-decoder framework based on at-tention mechanism.The attention mechanism is to link the predicted words at the target end with the source sentences through attention weight distribution.At each moment,the attention mechanism will update its alignment probability with all words at the source to obtain the attention weight distribution for the current moment,which is used to help pre-dict the output of the target word.Attention meehanism is an indispensable part of eurrent neural machine translation system.Therefore,a more effective attention weight distribution information is helpful to improve the effect of machine translation.In order to solve the problem of attention weight distribution information optimization in neural machine translation model,this paper proposes a method of establishing a multi-layer convolution neural network on attention mechanism and conducts in-depth research on neural machine translation model based on attention convolution.From the perspective of attention information at current moment and attention information at historical moment,this paper fully obtains the help of attention information to target-side translation,aiming at improving the performance of machine translation.In the process of building a multi-layer convolution neural network:Firstly,the attention information at the current moment is convolved followed by building new convolution layer and activation function.Through multiple sets of comparative experiments,the effects of different convolution kernel shapes and convolution layer parameters on machine translation results are tested.Secondly,the attention information of historical moments is convolved,and the convolution network is upgraded from one dimension to two dimensions.Not only the attention information of the source side sentence corresponding to the target side word is considered,but also the atten-tion information of the target side sentence corresponding to the source side word is included in the overall operation process,and the phrase-level attention information is promoted to the phrase-block-level attention information by utilizing the information interaction between the source side sentence and the target side sentence.Finally,the ensemble learning is adopted to combine various decoding models and output the results.The selected model is combined by means of averaging and fix weighted parameters to enhance the accuracy of the overall prediction probability.The experiment results show that neural machine translation based on attention convolution can effectively improve the quality of machine translation,and the accuracy of machine translation can be further improved by model ensemble.