Research On Improving Document-Level Neural Machine Translation

In recent years,deep learning has developed rapidly,and related research in the field of machine translation has continued to deepen.Among them,the attention-based encoder-decoder neural machine translation framework that appeared in previous years completely surpassed the traditional statistical machine translation framework in performance.Further,the nearest Transformer framework has raised the performance of neural machine translation to a new level.Due to the limitations of training methods,these advanced frameworks consider sentence as a whole in the process of translation.In the actual translation process,the text we face is often composed of multiple sentences.As document has independent characteristics,these sentence-level models are often lack of coherence and cohesiveness in the task of document translation.Therefore,the goal of our paper is to propose a document-level framework to improve the performance of neural machine translation models on document translation tasks.In our work,we learned a lot from the cross-sentence research of the frontier neural machine translation model.Combining the characteristics of two ideas in these researches,this paper proposes an improved document-level neural machine translation model based on the cache model.Our model is based on the encoder-decoder framework,that considering the document as a whole and using a cache model between each translation step in order to remember the historical encoder state of the source text.Except for the first sentence of the document,we set a multi-headed attention network and gating structure,which introduce the historical encoder state in the cache as context information into the current step decoder to improve the translation performance.In addition,the structure and mechanism of the cache model is also very important.Therefore,we have conducted in-depth research on this problem and proposed an improved cache model guided by the theme-rheme information.The structure of the cache model uses a key-value model in which information storage and updates are controlled by a theme-rheme labeling network and a logistic regression model.We have done a lot of experiments on the improved model,tried different cache model internal mechanisms and context information fusing strategies.Then we compared our document-level model with the state-of-art sentence-level model.The final experimental results show that the performance of our model has a significant improvement on the document translation tasks compared with the sentence-level model.