Research On Chinese-Mongolian Neural Machine Translation Based On Multi-task Learning

Machine translation uses computers to convert a source language into a target language automatically,which is one of the import research fields of natural language processing and artificial intelligence.At present,deep learning has made great progress in the field of machine translation,and the performance in many languages has surpassed the traditional machine translation method.However,deep learning is a"data driven"approach,and the scale and domain of data has a significant impact on the performance of the method.In view of the sufficiency of Chinese-Mongolian parallel,the great difference of word order between Chinese and Mongolian,and the complex rules of Mongolian word formation,this paper proposes a multi-task learning method to improve the performance of Chinese-Mongolian neural machine translation.The main research work of this paper is summarized as follows:(1)Aiming at the complex word formation rules of Mongolian language,we propose a morphological segmentation of Mongolian language by using bi-directional Jong short term memory network and conditional random fields.In this method,we consider the Mongolian morphology segmentation as a sequence labeling problem.First,we input the Mongolian characters into the model,and then utilize bi-directional long short term memory network to extract features of the sentence.Finally,we employ the conditional random fields to predict the attributes of the characters.The bi-directional long short term memory network can effectively utilize the context information of the sentence,thereby improving the classification performance of the model.(2)Transfer learning is used to improve the performance of Chinese-Mongolian neural machine translation.In this method,the high resource neural machine translation model is first trained by the encoder-decoder framework in neural machine translation,and then the low resource neural machine translation model is initialized by using part of the network parameters of the model,so the low resource neural machine translation model can fully utilize the prior knowledge acquired from the high resource model.Finally,we train the low resource neural machine translation model until it converges.The parameters of the word embedding layer and the fully connected layer of low-resource neural machine translation model are randomly initialized and continuously updated during the iterative process of the model.(3)A weight sharing method is utilized to improve the performance of Chinese-Mongolian neural machine translation.In this method,the cross-lingual word embedding is first used to align the word vector space of the high resource parallel corpus and the low resource parallel corpus source language,that is,the cosine similarity between the word vectors with similar semantics is very large.Then the low resource neural machine translation model and the high resource neural machine translation model share part of weight of encoder and ownership value of the decoder,so that the low resource neural machine translation model can generate high-quality context vectors and the language models of the target language.Finally,the high resource parallel corpus and the low resource parallel corpus train the model in turn,respectively updating the corresponding neural network weights.