On Machine Translation Approaches Based On Multi-Level Knowledge

Machine translation(MT)is a core technology to realize the dream of communication without borders,which has received considerable attention from both academia and industry.However,there are unresolved problems in conventional statistical machine translation(SMT)and rapidly growing neural machine translation(NMT),specifically speaking,how to improve system performance in the situation of low-resource machine translation,how to adapt the methods that have been proven to be effective for SMT to the field of NMT,how to use translation skeletons to guide the translation process,and how to utilize linguistic knowledge to improve translation quality.To address the aforementioned problems,we propose to incorporate multi-level knowledge into MT models.In this dissertation,multi-level knowledge includes pivot-language knowledge,generalization and phrase knowledge,skeleton knowledge,and linguistic knowledge.The contents of this dissertation consists of the following four parts:First,we propose an approach to machine translation based on pivot-language knowledge.Our preliminary experiments in low-resource machine translation show that NMT severely lags behind SMT in such a setting.So in this dissertation we focus on SMT and propose some methods to improve the translation quality of low-resource SMT.Specifically,we implement foreign-Chinese translation models through two different methods:one method on the base of corpus-level pivot languages that aims to optimize word alignments and one method on the base of phrase-level pivot languages that builds on decoding-generation.Moreover,we propose a combination approach to merge corpus-level and phrase-level translation results,which is based on minimum error rate training(MERT)and further improves the translation performance.We translate Bengali,Tamil,Uzbek,and Hungarian into Chinese by using our pivot language based methods.Second,we propose an approach to machine translation based on generalization and phrase knowledge.Previous work on phrase-based SMT shows that models trained on generalized corpora perform better than models trained on words(without generalization).Motivated by the observation,we propose to examine the effect of data generalization and phrase generation for NMT.Specifically,we combine data generalization techniques with subword methods to correct mistaken translations that are caused by subword methods when translating out-of-vocabulary(OOV)words and low-frequency words.To incorporate generalization knowledge into NMT,we propose a parallel consistency checking method and a decoding optimization method.To incorporate phrase knowledge into our NMT model,we propose a phrase generation method that is based on the idea of data compression.By incorporating such generalization and phrase knowledge,we manage to improve NMT performance.Third,we propose an approach to machine translation based on skeleton knowledge,and examine the effectiveness of such skeleton knowledge in both SMT and NMT.Regarding SMT,so-called skeleton phrase pairs provide local translation templates for the generation of target languages.Skeleton phrase pairs can be generated by applying operations of decomposition,substitution,and generation to phrase pairs in the translation table.This way,our method can generate many high-quality skeleton phrase pairs that help to alleviate the problem of insufficient phrase generation and then improve translation performance.Regarding NMT,skeletons provide global translation templates for the generation of target languages,and we can formalize the translation problem into a slot-filling problem.Our model uses an independent encoder to represent translation skeletons,and during the decoding phase our model uses knowledge gate and attention gate to dynamically control the influence of information from surface words and skeletons.From the translation results we can see that our proposed model can guide the process of translation and finally achieves improvements on translation performance.Experiments also show that translation performance rises as more content words are considered in skeletons.Last,we propose an approach to machine translation based on linguistic knowledge,and examine the effectiveness of such linguistic knowledge in both SMT and NMT.Regarding SMT,we analyze the phenomenon of word deletion by human and classify the cases of word deletion into two categories:desired word deletion and undesired word deletion,respectively.For these two categories of word deletion,we propose a maximum-entropy-based word deletion classification model.For training the maximum entropy model,we incorporate three types of linguistic knowledge,including part-of-speech tags,named entity tags,and chunk tags,into knowledge blocks.Experiments show that the proposed model for word deletion significantly improves translation quality.Regarding NMT,our proposed method uses an encoder to model additional linguistic features in parallel to the encoder for word features.We incorporate four kinds of linguistic knowledge,including part-of-speech tags,named entity tags,chunk tags,and dependency labels,into knowledge blocks.The core idea is that we utilize knowledge gate and attention gate to dynamically control the influence of information from word and linguistic knowledge.Finally,extensive experiments show that our approach effectively improves translation quality,and can address the word deletion problem and incorrect sentence structure problem in translation results.The above techniques have been implemented in the open-sourced SMT system NiuTrans and the open-sourced NMT system LiNMT.NiuTrans won the first place and the second place in NTCIR,LoReHLT,and CWMT translation competitions for many times,and LiNMT won the second place in the CWMT2017 English-to-Chinese translation competition.