Exploitation And Application On The Implicit Learning Ability Of Neural Machine Translation Model

Since mankind entered into modern society,international interlingual commu-nication has become increasingly frequent.With Internet,people break the physical spatial limitation but still find it difficult to overcome the obstacles of languages.In the information age where the demand for translation is increasing exponentially,relying on human translators only is greatly deficient.And machine translation,as an efficient and convenient automation tool,emerges as the times require,and has achieved significant developmentAcademics have proposed numerous ideas for improving neural machine trans-lation,including model structure,training targets,decoding speed,etc.These efforts focus on the changes to the basic framework but pay little attention to the learning ability of its ownHowever,neural machine translation model itself has a strong implicit learning ability and there is a great deal of room for exploitation and interpretation.On the one hand,its local submodules can implicitly learn the decomposed features in translation processing such as word embedding,attention(word alignment),etc.On the other hand,its end-to-end training has strong adaptability and can be expanded to many tasks.These two learning abilities need to be further developed.This paper will focus on the implicit learning ability of neural machine translation,with research and applications on three subtasks of machine translation1.On the diverse translation task,we mine the data pattern which is learned implicitly by multi-head attention modules in neural machine translation,and take advantage of this intrinsic phenomenon to enhance the translation diversity of the model and dynamically achieve a balance between translation quality and diversity.Further usage combined with back-translation technique also enhances the performance of the model2.On the low-resource translation task,the problem of unbalanced training of attention head in neural machine translation is analyzed and validated experimentally.And we propose a local masking strategy to alleviate the problem,improving the translation quality of low-resource language pairs3.On the document translation task,we expand the end-to-end training style of neural machine translation,excavate its potential of modeling long-range context,and build a new paradigm for document translation.Also,we propose a new large-scale dataset as well as targeted metrics,breaking the past limitations of training data and scenarios.