Research On Incorporating The Source Information To Automatic Evaluation Of Machine Translation

Automatic metrics is to evaluate the quality of the translation quantitatively by calculating the similarity between the translations and the references.Automatic evaluation of machine translation can not only measure the overall performance of the translation system to some extent,but also guide the optimization of its feature weights during the development of the translation system.Therefore,it is of great practical and theoretical significance to study automatic translation evaluation of machine translation.In recent years,many automatic evaluation metrics were put forward in succession.The main idea of most metrics,however,is to follow BLEU's basic point,in other words,the similarity between the machine translations and the references is calculated,so the source sentences are completely ignored in automatic evaluation of machine translation,while the source sentences can reflect the fidelity information of the machine translation.Therefore,in view of the deficiency that the automatic evaluation of machine translation completely ignores the source sentence information and only uses the references to measure translation quality,the paper proposes an automatic evaluation metric with incorporating source sentence information.We incorporate the source sentence information indirectly by using the static quality embeddings and dynamic quality embeddings.Firstly,extract the quality embeddings that describes the translation quality from a tuple consist of the machine translations and their corresponding source sentences,and incorporate it into the automatic evaluation method based on contextual embeddings by using a deep neural network,so as to further enhance the performance of the automatic evaluation method.In order to verify the performance of our methods,we have carried out sufficient experimental verification on the WMT'17-19 metric tasks,and the experimental results show that the incorporating of the source sentence information can not only improve the sentence-level correlation with the human judgments,and to a certain extent,improve the system-level correlation.In addition,it is found that the UNQE method and the TUNQE method,which do not use the reference information at all,have a good correlation with the human judgments,which indicates that the source sentence information is very helpful to automatic evaluation metric,which proves from one side that the correctly incorporating of the quality embeddings into automatic evaluation of machine translation will improve the performance of automatic evaluation of translation machine.Deep analysis further reveals that the information of the source sentences plays an important role in automatic evaluation of machine translation.