Research On Automatic Evaluation Of Machine Translation Based On Linguistic Knowledge

Since automatic evaluation of machine translation (MT) can effectively promote the development of machine translation, more and more researchers are concern about it. With a good automatic evaluation metric, MT developers can optimize the model continuously without bothering the costly and time-consuming human assessments. Although the main trend of machine translation evaluation is the string similarity oriented metrics, linguistic features have been successfully applied to establish high performance automatic machine translation evaluation metrics, and seeking more linguistic features seems promising for further improvements. The related works at home and abroad demonstrate that linguistic features can greatly improve the performance of machine translation evaluation, but no doubt it also resulted in that people would like to employ more and more linguistic features into MT evaluation. The current performance of machine translation evaluation is still a wide gap between the evaluations of human and can't meet the need of people.This may be caused by the limit of the natural language processing technology; the tool which can automatic analyze linguistic features still can't work well. Besides, the evaluation of translation is a highly subjective process of human and many factors may influence the result of the translation evaluation.To verify this hypothesis, we explore the following aspects:1. Adopting a corpus annotated for cognitive aspect of translation quality.2. With the foundation of translation evaluation corpus, investigates the relationship between the linguistic features and human perception of translation quality in various linguistic levels.3. We investigate that if linguistic features are enough for modeling of machine translation evaluation by constructing a number of linguistic feature set.4. Classify the automatic metric and the linguistic features, and do a deep analysis on the contribution of each feature type to the modeling of automatic translation evaluation.5. Study on how to combine linguistic features and the traditional metrics in order to build a best machine translation evaluation system.It is revealed that many linguistic features have a high correlation coefficient to human scores on translation quality, with others hardly correlating with human judgments. We argued that linguistic features could help the performance of automatic MT evaluation, while an exhaustive employment of linguistic features available is not the final solution to the high performance automatic evaluation metrics. It is also confirmed that the best evaluation result is achieved by integrating linguistic features into the current string similarity oriented metrics, but this seems not the whole space of automatic translation evaluation modeling under the machine learning framework yet.