Research On Domain Adaptation Methods For Neural Machine Translation

The performance of data-driven machine translation techniques depends heavily on the degree of domain match between training and test data.Due to the significant varieties of training data across domains,the performance of cross-domain translation is degraded.Researchers often improve the translation performance by adapting the MT system to the test domain.As for NMT,the dominant strategies for domain adaptation generally fall into two categories:l)transfer out-of-domain knowledge to in-domain translation,2)directly use a mixed-domain training corpus to construct NMT model for the translated sentences derived from different domains.In this paper,the domain adaptation methods for neural machine translation are studied for these two kinds of scenarios.The main work of this article includes:(1)Sentence weighting for Neural Machine Translation Domain Adaptation.In machine translation,out-of-domain instances related to the target domain are often useful for training the model,while those that are not relevant to the target domain may degrade the quality of the translation.In this paper,we propose a sentence weighting method for Neural Machine Translation Domain Adaptation,evaluating the weight of the sentences according to the degree of relevance ot the sentences to the target domain,and integrating the weight into the NMT which will impact parameters updating.We apply the methods to the standard domain adaptation and the NMT trained with only synthetic parallel data for low-resource domain.In the Chinese-English IWSLT domain adaptation tasks and low-resource e-commerce domain translation tasks,the methods gam signifcant improvement in specific domain translation.(2)Domain-aware self-attention tor multi-dornain Neural Machine Translation Domain Adaptation.Models trained on the generic parallel corpus are often affected by the diversity of the training data.We expect to use a single model to simultaneously improve the translation quality of multiple domains.In this paper,we propose a domain-aware self-attention mechanism for multi-domain translation,forcing the NMT model to encode and decode both semantic and domain information,and joint learning domain with the NMT training.In the multi-domain Chinese-Enghsh and English-French translation tasks,the experimental results show that the translation quality of each domain has been improved,indicating the effectiveness of the domain-aware self-attention for multi-domain Neural Machine Translation method.(3)Unsupervised word-level adaptation method for multi-domain Neural Machine Translation Domain Adaptation.In real-world application scenarios,the model 1s required to be flexible and can be applied to the training and translation of sentence inputs without domain information.In this paper,we propose an unsupervised word-level adaptation method for multi-domain Neural Machine Translation.With the domain-aware self-attention mechanism,the approach uses domain attention network based word-level unsupervised learning and guided learning with an auxiliary loss method to learn the domain representation for each word.Experiments and analysis on Chinese-Enghsh multi-domain translation task show that our model can achieve significant improvements on the baseline system and indeed learn the domain structure of the training data.Further analysis shows that even without prior knowledge of the domain structure,our model can learn domain information and clustering sentences by domain.