Automatic Evaluation of Machine Translation Integrated with Fine-grained Error Analysis

Automatic evaluation of machine translation can access the quality of machine translation by quantifying the similarity or difference between machine translation and human reference translation. To address the defects in lacking detailed error description and score bias in current pre-trained language model based methods, this paper proposes to integrate fine-grained error analysis for the automatic evaluation method of machine translation. The large language model is guided by the chain-of-thought to capture various errors in machine translation, punish errors of different severity, and quantify the number of errors per unit length of machine translation. Experimental results on the WMT '23 benchmark dataset for automatic evaluation of machine translation show that the automatic evaluation methods integrated with fine-grained error analysis can effectively improve the evaluation results compared with the classical automatic methods and the best results in the evaluation campaign.