Diverse Machine Translation Based On Uncertainty Modeling And Beam Search

Through constructing neural network-based generation models,machine translation has gained massive development and boosted numerous applications in recent years.However,as a result of inherent linguistic diversities,different languages are not completely equivalent in terms of words,syntactic structures or affective styles.Therefore,machine translation is essentially a one-to-many supervised task,and uncertainty is widespread in understanding source sentence and generating target sentence.The current mainstream research on machine translation concentrates on improving the accuracy of translation.Generally,maximum likelihood estimation and beam search are exploited to do deterministic modelling and searching in the translation process respectively,while the uncertainty in text generation are only captured to some degree and the diversity in machine translation is ignored.The diverse machine translation task aims at generating multiple valid target hypotheses based on a source sentence,which not only provides software users with a variety of choices,but also circumvents the ambiguity problems of text generation.This paper comprehensively takes the syntactic and word-level diversity into account and develops three novel approaches to modelling and searching for uncertainty in translation process.Firstly,this paper proposes a creative global diverse search algorithm.On the basis of the mixture models of experts,global diverse search comprehensively considers the whole sequence generated at previous time steps and achieves asynchronous interaction between different beams.Secondly,this paper conducts a retrieval-translation model for diverse machine translation.We take the lead in introducing translation memory as a role of expert to guide the translation model to generate diverse hypotheses without sacrificing translation quality.In the end,this paper further explores the syntactic structures of translation memory and integrates them into translation model as the initialization state of decoder.This work enhance encoding capability of syntactic representation and effectively avoid the information loss caused by encoding and discretizing the syntax tree in the previous research.Experiments on WMT and JRC-Acquis datasets show that global diverse search algorithm can improve the diverity effectively while retrieval-translation model not only increases the diversity of generated hypotheses,but also achieves better translation quality.