Research On Low-Resource Machine Translation Based On Parallel Transfer And Contrastive Learning

Low-resource machine translation is one of the important research hotspots in the field of machine translation at present.The training process of machine translation models requires large-scale parallel or comparable linguistic resources,however,such linguistic resources are often relatively scarce in practical situation.In particular,parallel or comparable linguistic resources are not available for low-resource languages.Due to the lack of such linguistic resources,machine translation models fail to fully grasp the semantic,pragmatic and syntactic translation features of a specific language,which makes the machine translation models unable to provide high-quality translations.In this thesis,we investigate two problems of the lack of mutual translation knowledge at word-sense level and sentence level in low resource scenarios.In this thesis,three studies are conducted to address the above issues.The first two studies address the problem of lack of knowledge of word-sense level inter-translation and the third study addresses the problem of lack of knowledge of sentence-level inter-translation.The details of the studies are as follows.1)Word alignment is one of the fundamental tasks of machine translation.In lowresource scenarios,the lack of parallel corpus will lead to the model’s inability to capture semantically similar words between two languages,i.e.,low performance of word alignment at the semantic level.The poor performance of word alignment will directly affect the performance of the subsequent translation process.Therefore,this thesis first investigates word alignment in low-resource scenarios,motivated by the fact that the accuracy improvement of word alignment can help the sentence-level machine translation task.Specifically,this thesis proposes a word alignment method based on contextual representation and bilingual lexicon,which uses the contextual semantics and bilingual lexicon in the pre-trained model to help the word alignment model introduce contextual information based on the understanding of word meanings,so as to improve the word alignment accuracy and translation quality.2)In low-resource scenarios,migration learning helps to improve the performance of translation models.Such approaches train parent translation models on high-resource languages(e.g.,English and German)and,based on this,migrate them to the translation learning process for low-resource languages so that they start from a higher learning starting point and further train child translation models.One of the commonly used methods is Embedding Transfer,which establishes a migration mechanism between the parent model and the child model for the same Sub-Word,i.e.,the distributed semantic representation of the same SubWord is cloned directly.Although this type of approach migrates the translation knowledge of the parent model,the corresponding migration effect is not satisfactory in the case of large differences in word lists.In this thesis,we conduct a study to address this problem,i.e.,we propose a low-resource machine translation optimization method based on cross-language subword semantic migration.The method clones distributed representations of subwords with similar word meanings at the same time on the basis of cloning distributed representations of the same subwords,and the method helps to further expand the initial translation knowledge of the child model in the low-resource scenario.3)The scarcity of parallel data affects the translation model’s understanding of semantics.To address this problem,this thesis proposes a low-resource neural machine translation method based on contrastive learning,which improves the translation model through data augmentation and contrastive learning.Specifically,the method ensure the consistency of the vector representation of inter-translated sentence pairs in the process of applying contrastive learning.Meanwhile,the method differentially correcting the vector representation of non-inter-translated sentence pairs,so as to enhance the semantic understanding of the translation model for the whole sentence.In this thesis,we investigate two problems of lack of mutual translation knowledge at word-sense level and sentence level in low-resource scenarios.We propose a word alignment method based on contextual representation and bilingual lexicon,a low-resource machine translation optimization method based on cross-language subword sense migration,and a low-resource neural machine translation method based on contrastive learning,respectively.In this thesis,the above methods are tested on international authoritative datasets(WPT 2005,ALT,PAN Localization BPPT and WMT17 news translation task).The experimental results show that the above-mentioned word alignment model based on contextual representation and bilingual dictionaries obtains significant performance advantages on the WPT 2005 word alignment dataset.Specifically,the word alignment error rate(AER)is reduced by 3.5%and 2.0%for English-French and English-Romanian,respectively;the low-resource machine translation optimization method based on cross-language subword semantic migration achieves a significant performance advantage on the Burmese-English translation scenario of ALT,the Indonesian-English translation scenario of PAN Localization BPPT,and the Turkish-English translation scenario of WMT17 News translation task on TurkishEnglish translation scenario by 2.0%,2.0%,and 1.1%,respectively;the low-resource neural machine translation model based on contrastive learning improves the BLEU performance on Burmese-English translation scenario for ALT,Indonesian-English translation scenario for PAN Localization BPPT,and Turkish-English translation scenario for WMT17 news translation task by English translation scenario on BLEU performance improved by 1.12%,2.48%and 0.59%,respectively.