高级检索

引入思维链的机器译文质量估计方法

Quality Estimation of Machine Translation By Introducing Chain-of-thought

  • 摘要: 机器译文质量估计指在无需人工参考译文的情况下仅根据源语言句子评估机器译文的翻译质量。过去基于预训练语言模型或利用直接提示的机器译文质量估计方法都存在一定程度的翻译忠实度偏见,这类方法会给予流畅且无语法错误的译文较高评分,而忽略了机器译文与源语言句子间的语义一致性。该文提出了一种引入思维链的机器译文质量估计方法:一方面,将质量估计任务分解为准确性评估和多维度评估两个子任务,利用思维链技术设计模块化的推理链驱动大语言模型预测译文质量,关注双语语义一致性的同时挖掘机器译文与源语言句子间的深层语义关系;另一方面,利用预训练语言模型提取机器译文与源语言句子的深层语义表征,构建神经网络模型预测机器译文质量。在WMT’23句子级别机器译文质量估计测评任务数据集上的实验结果表明,所提方法显著提高了与人类评价的相关性,且在大多数语言对上的表现超过了参与评测的最优系统。

     

    Abstract: Quality estimation of machine translation aims to evaluate the translation quality of machine translation without relying on the human reference translation. In contrast to the previous methods based on pretrained language models, this paper proposes a method of quality estimation of machine translation by Chain-of-Thought: On the one hand, the task of quality estimation is decomposed into two sub-tasks: accuracy evaluation and multidimensional evaluation. A modularized reasoning chain is designed to drive the large language model to predict translation quality, by capturing the bilingual semantic consistency. On the other hand, the pretrained language model is used to extract the deep semantic representation of machine translation and source language sentence, and the neural network model is constructed to predict the quality of machine translation. Experimental results on the WMT’23 sentence-level quality estimation of machine translation shared task dataset show that the proposed method significantly improves its relevance to human evaluation and outperforms the optimal system in most language pairs.

     

/

返回文章
返回