Research On The Application Of Constrained Optimization In Neural Machine Translation

Different languages exist among different countries,regions,and peoples.Nowa-days,the need for people to communicate across languages is increasing.The automatic translation between two languages by machines is getting more and more attention.Ma-chine translation is such a powerful weapon to solve this problem.It is also the research hotspot in natural language processing.Neural machine translation is currently the dominant method in the field of ma-chine translation.The neural machine translation uses parallel corpus as the training data,uses the deep neural network to model,and then optimizes the loss function to train the model by the method of stochastic gradient descent.On one hand,compared to statistical machine translation,the training process of neural machine translation is a process of training deep neural networks.The neural machine translation model has a complex structure and a large number of parameters to be trained.This makes it diffi-cult to learn the parameters of the model in an end-to-end manner.On the other hand,it helps to improve the performance of the model if prior information related to translation task is introducted for neural machine translation.Constrained optimization is an important method to improve the training perfor-mance and introduce a priori information for neural networks.Based on the method of constrained optimization,this paper provides solutions to the above two problems.The main work is as follows:· Aiming at the problems in the learning of neural machine translation parameters,we propose a neural machine translation model training method based on checkpoint constraints.Compared with the previous method solving parameter learning prob-lems,this method only relies on the checkpoints generated during the model training process,and constructs the constraint function by means of knowledge distillation.In addition,the method continually constructs a new teacher model from newly ac-quired checkpoints,thereby enabling dynamic adjustment of constraints based on the training process.The method proposed in this paper significantly improves the training effect on the four translation datasets,and alleviates the over-fitting phe-nomenon in training on low resource data.· The semantic representation space of cross-language sharing helps to improve the performance of neural machine translation models.The role of this a priori infor-mation is widely confirmed in multilingual machine translation.By sharing vo-cabularies and model parameters across languages,the model is anble to learn the semantic representation space for cross-language sharing.However,we have found that a similar approach cannot be utilized in a neural machine translation model for a single language pair.In order to introduce cross-lingual sharing priority for rep-resentation in machine translation models,we propose a constraint optimization method based on component sharing.The experimental results in the Chinese-English,Japanese-English and Korean-English prove that this kind of constraint does introduce a shared semantic representation for the model,and the translation performance in both directions is improved.