Multi-subspace Attention Neural Machine Translation

With the fierce development of Internet technology and the widespread use of computers,machine translation has gradually been applied to multiple fields from the field of natural language processing,such as industrial fields,education fields,and so on.Driven by artificial intelligence,machine translation and neural networks are combined with each other,making traditional machine translation methods gradually expand to neural machine translation methods.Although most of the existing neural machine translation models are used in combination with the attention mechanism,the current attention mechanism only uses an attention calculation function.A single attention calculation function causes the model to always ignore important information.Therefore,this paper first proposes a new attention mechanism.The research focus is on how to combine multiple attention calculation functions and maximize the advantages of each attention calculation function.Therefore,a neural machine translation model with multiple attention calculation functions is proposed.Bidirectional long-short term memory network is a kind of recurrent neural network widely used in the field of natural language processing.This paper proposes a fast-converging long-short term memory network,which proves that the fusion of future information and historical information can extract more sufficient contextual semantic information.Therefore,this paper proposes a multi-subspace attention mechanism,and integrates multiple attention calculation functions in the multi-subspace attention mechanism.The multi-attention mechanism first maps the hidden layer states of the bidirectional long-term and short-term memory network to multiple subspaces,and then uses multiple attention calculation functions in the multi-attention mechanism to calculate the attention score,which is finally applied to the neural machine translation model.This paper compares the proposed multi-attention neural machine translation model with several neural machine translation models on the WMT 14 dataset.The experimental results prove that the multi-subspace attention neural machine translation model proposed in this paper can effectively improve the translation quality of text.