Quality Estimation of Machine Translation By Introducing Chain-of-thought

Quality estimation of machine translation aims to evaluate the translation quality of machine translation without relying on the human reference translation. In contrast to the previous methods based on pretrained language models, this paper proposes a method of quality estimation of machine translation by Chain-of-Thought: On the one hand, the task of quality estimation is decomposed into two sub-tasks: accuracy evaluation and multidimensional evaluation. A modularized reasoning chain is designed to drive the large language model to predict translation quality, by capturing the bilingual semantic consistency. On the other hand, the pretrained language model is used to extract the deep semantic representation of machine translation and source language sentence, and the neural network model is constructed to predict the quality of machine translation. Experimental results on the WMT’23 sentence-level quality estimation of machine translation shared task dataset show that the proposed method significantly improves its relevance to human evaluation and outperforms the optimal system in most language pairs.