Research On Machine Translation Automatic Evaluation Based On Extended Reference

The development of machine translation has gone through several stages such as dictionary matching method,method of combining with linguistic knowledge,statistical machine translation method combined with corpus,and neural machine translation method.The automatic evaluation of machine translation is an important research content in machine translation technology.Machine translation evaluation can be used to find defects in the translation system and further promote the development of the translation system.The quality of the translation can be evaluated by two methods,manual and computer.Manual evaluation occurs earlier,and the evaluation results of this method are more accurate,but it greatly consumes human resources and generally takes a long time.Another problem with manual evaluation is that it is subjective.For example,when evaluating the same sentence,the results given by different evaluators may be different and lack consistency.The emergence of automatic evaluation methods just makes up for the shortcomings of manual evaluation.It can quickly evaluate the quality of a machine translation result.Compared with manual evaluation methods,this method has low cost and can evaluate machine translation more efficiently.It is the strategy used by most automatic evaluation methods to judge the quality of the translation by comparing the similarity between the machine translation and the reference.Therefore,the number of reference and the extent of their information utilization are the two main factors that affect the performance of the evaluation method.This paper improves the existing machine translation evaluation method DPF(Metric based on Dependency Parsing and F-score of Unigram)by extending the reference and the consistency of the two-end dependency analysis algorithm.The main research content includes the following three aspects.(1)The reference is given manually,and the number is relatively small.Generally,the number of references given in the evaluation of machine translation evaluation methods is one or four.For example,there is only one reference in the evaluation method DPF.Generally speaking,the smaller the number of references,the less information covered.The degree ofinformation covered by the reference will affect the performance of the evaluation method.Aiming at the problem of the small number of references,this paper proposes an evaluation method Ex DPF(Extend DPF)based on extended reference and dependency parsing models.This method can expand the reference translation to increase the information covered by the original reference,and add the expanded reference to the evaluation method DPF based on the dependency parsing model to obtain a better performance evaluation method.(2)The information in reference is certain.The more fully used it is,the better the performance of the evaluation method will be.On the contrary,the worse the performance will be.The existing evaluation method DPF needs to use the dependency tree at both ends of the reference and the machine translation in the evaluation process.When obtaining the dependency tree,DPF uses different dependency analysis algorithms at these two ends.The inconsistency of the algorithms at the two ends makes the evaluation method cannot make full use of the reference information,which has a certain impact on the quality of the machine translation dependency tree,and then affects the performance of the evaluation method.Aiming at the problem of insufficient use of reference information by DPF method,this paper proposes an evaluation method MDPF(Maxmum entropy DPF)based on the maximum entropy dependence parsing model.This method uses the maximum entropy dependency parsing algorithm when obtaining the dependency tree at both ends of the reference and the machine translation.The algorithms at both ends are consistent,which makes the evaluation method more fully utilize the reference information.Experimental results prove that the MDPF method achieves better performance than DPF on the language pair whose target is English.(3)In order to simultaneously solve the problem of the low number of reference and insufficient use of reference information in the DPF method,this paper proposes an evaluation method MEx DPF(Maxmum entropy Extend DPF),combining extended reference and maximum entropy dependence parsing model.MEx DPF first expands the the reference translation,and then uses the maximum entropy-based dependency parsing model whenobtaining the dependency tree for both the machine translation and the reference.The extension of the reference increases the extent of the information it covers,and the consistency of the dependency parsing algorithms at both ends can make the information of multiple references fully utilized.Experiments show that this method achieves better performance than Ex DPF and MDPF.