On Learning And Decoding Approaches To Tree-to-tree Statistical Machine Translation

Machine translation is one of man’s oldest dreams and has received growing interests over a long period of time. Recently statistical approaches have been successfully applied to machine translation. More and more studies have focused on learning translation systems from the large collection of bilingual sentence pairs and automatically translating new sentences using the resulting system. In statistical machine translation, traditional approaches are modeled in either word or n-gram (phrase) level. While these approaches are robust and easy to implement, they ignore the underlying (syntactic) structure of sentence and thus have limited capabilities in dealing with long distance dependencies and generating grammatically-correct outputs. To address these problems, the syntax-based approach has been recognized as one of the most desirable solutions. Among various syntax-based models, tree-to-tree models (i.e., translating from a given source-language parse tree into a target-language parse tree) are no-doubt the most promising directions due to their obvious advantages over other phrase-based and syntax-based counterparts, such as:better use of bilingual syntax in modeling the reordering problem, better analysis of source tree and syntactic generation of target-language syntactic structure. In this article, we investigate approaches to tree-to-tree translation. In particular, we focus on developing better model learning and decoding methods for tree-to-tree systems. Our contributions are summarized as follows:We present an unsupervised sub-tree alignment model. In this work, we first model the sub-tree alignment problem as derivations of tree-to-tree transfer rules, and decompose the model into a product of several factors under reasonable assumptions. The model parameters are then learned on the bilingual tree-pairs using the EM algorithm. Moreover, as a by-product, the proposed model can produce a sub-tree alignment matrix, rather than1-best/k-best alignments. As sub-tree alignment matrix encodes an exponentially large number of possible alignments, we can extract additional translation rules from the alignment matrix. As a result, we can increase the coverage rate of the extracted rule set and thus improve the translation quality.We present a beam-width limited approach to training tree-to-tree models. Unlike traditional approaches, we do not ignore the search problem in the training stage, but instead directly parameterize the beam search problem by incorporating various lose functions into modeling. In particular, we consider both the beam-width limited search and the measure of translation quality (e.g., BLEU) in training, and design two loss functions to model these two factors. Furthermore, we propose a simple and effective method to learn our model from the bilingual corpus in an iterative manner. Our experimental studies show that our proposed approach is very helpful in improving a state-of-the-art tree-to-tree system due to the reduction of mismatch between training and decoding.We present two improved approaches to tree-to-tree decoding. The first of these is a course-to-fine approach. Unlike previous approaches, we do not resort to a single grammar, but instead decode with various grammars that have different use of syntax (ranging from course-grained grammar to fine-grained grammar). As course-grained grammars can make a "large" search for decoding, the decoder suffers less from search errors. On the other hand, fine-grained grammars can assign a more accurate model score to each translation hypothesis and thus reduce model errors. The second decoding approach is based on ensemble learning techniques. In this approach, we first learn a number of different systems using a single translation model (or decoder), and then "select" a better translation from the pool of the translation outputs of these systems. Experimental results show that the proposed approach significantly outperforms the baseline approach that relies on a single MT output.We proposed a tree-substitution grammar-based evaluation model of target-tree structure (syntax-based language model) for tree-to-tree translation. First, we model the target tree structure using tree-substitution grammars (TSGs), and then measure the goodness of the tree structures generated during decoding using various parsing models. Our proposed model can be learned on the auto-parsed data. Experimental results show that it is able to benefit a state-of-the-art tree-to-tree translation system, even achieves promising BLEU improvements. In addition, we present three methods for the integration of the proposed evaluation model into decoding. All these methods lead to a further improvement in translation accuracy of the tree-to-tree system.The above techniques have been employed to an open-source machine translation NiuTrans (http://www.nlplab.com/NiuPlan/NiuTrans.html4) which has been released to the community for the research purpose. Also, the achievements herein help us to achieve top-performance in recent translation evaluations tasks, such as NTCIR5and CWMT6.