利用语义关联增强的跨语言预训练模型的译文质量评估

叶恒,贡正仙

PDF(2439 KB)
PDF(2439 KB)
中文信息学报 ›› 2023, Vol. 37 ›› Issue (3) : 79-88.
机器翻译

利用语义关联增强的跨语言预训练模型的译文质量评估

  • 叶恒,贡正仙
作者信息 +

A Semantic Connection Enhanced Cross-language Pre-trained Model for MT Quality Estimation

  • YE Heng, GONG Zhengxian
Author information +
History +

摘要

机器翻译质量评估(QE)是在不依赖参考译文的条件下,自动对机器翻译译文进行评估。当前人工标注数据稀缺,使得神经QE模型在自动检测译文错误方面还存在较大问题。为了更好地利用规模庞大但却缺少人工标注信息的平行语料,该文提出一种基于平行语料的翻译知识迁移方案。首先采用跨语言预训练模型XLM-R构建神经质量评估基线系统,在此基础上提出三种预训练策略增强XLM-R的双语语义关联能力。该文方法在WMT 2017和WMT 2019的英德翻译质量评估数据集上都达到了最高性能。

Abstract

Quality Estimation(QE) of Machine Translation(MT) can automatically estimate the quality of MT outputs without references. Due to the lack of manual data, the current QE Systems with neural network architecture still have problems in automatically detecting translation errors. For the sake of utilizing the vast but unlabeled parallel data, this paper proposes a translation knowledge transfer method. First, the cross-lingual pre-trained model XLM-R is used to construct the neural quality estimation baseline system, then we propose three pre-training strategies to enhance the bilingual semantic connection ability of XLM-R. The proposed method in this paper has reached the new SOTA performance on both the WMT2017 and WMT2019 quality estimation data sets.

关键词

机器翻译质量评估 / 跨语言预训练模型 / 语义关联 / 预训练策略

Key words

quality estimation of machine translation / cross-lingual pretrained model / semantic connection / pre-training strategy

引用本文

导出引用
叶恒,贡正仙. 利用语义关联增强的跨语言预训练模型的译文质量评估. 中文信息学报. 2023, 37(3): 79-88
YE Heng, GONG Zhengxian. A Semantic Connection Enhanced Cross-language Pre-trained Model for MT Quality Estimation. Journal of Chinese Information Processing. 2023, 37(3): 79-88

参考文献

[1] SUTSKEVER I, VINYALS O, LE Q V. Sequence to sequence learning with neural networks[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems, 2014: 3104-3112.
[2] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[C]//Proceedings of ICLR. 2015:1-12.
[3] PAPINENI K, ROUKOS S, WARD T, et al. Bleu: A method for automatic evaluation of machine translation[C]//Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, 2002: 311-318.
[4] BANERJEE S, LAVIE A. Meteor: An automatic metric for MT evaluation with improved correlation with human judgments[C]//Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, 2005: 65-72.
[5] LIN C Y. ROUGE: A package for automatic evaluation of summaries[G].Text Summarization Branches Out. 2004: 74-81.
[6] BLATZ J, FITZGERALD E, FOSTER G, et al. Confidence estimation for machine translation[C]//Proceedings of the 20th International Conference on Computational Linguistics, 2004: 315-321.
[7] SPECIA L, SHAH K, DE SOUZA J G C, et al. QuEst: A translation quality estimation framework[C]//Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 2013: 79-84.
[8] SNOVER M, DORR B, SCHWARTZ R, et al. A study of translation edit rate with targeted human annotation[C]//Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers, 2006: 223-231.
[9] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of NAACL,2019:4171-4186.
[10] LAMPLE G, CONNEAU A. Cross-lingual language model pretraining[C]//Proceedings of the 33rh International Conference on Neural Information Processing Systems,2019:7059-7069.
[11] CONNEAU A, KHANDELWAL K, GOYAL N, et al. Unsupervised cross-lingual representation learning at scale[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Lingaistics,2020:8440-8451.
[12] FOMICHEVA M, SUN S, YANKOVSKAYA L, et al. Unsupervised quality estimation for neural machine translation[J]. Transactions of the Association for Computational Linguistics, 2020, 8: 539-555.
[13] MARTINS A F T, JUNCZYS-DOWMUNT M, KEPLER F N, et al. Pushing the limits of translation quality estimation[J]. Transactions of the Association for Computational Linguistics, 2017, 5: 205-218.
[14] KIM H, LEE J H, NA S H. Predictor-estimator using multilevel task learning with stack propagation for neural quality estimation[C]//Proceedings of the Second Conference on Machine Translation, 2017: 562-568.
[15] IVE J, BLAIN F, SPECIA L. DeepQuest: A framework for neural-based quality estimation[C]//Proceedings of the 27th International Conference on Computational Linguistics. Association for Computational Linguistics, 2018: 3146-3157.
[16] BLAIN F, ALETRAS N, SPECIA L. Quality in, quality out: Learning from actual mistakes[C]//Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, 2020: 145-153.
[17] MIAO G, DI H, XU J, et al. Improved quality estimation of machine translation with pre-trained language representation[C]//Proceedings of the CCF International Conference on Natural Language Processing and Chinese Computing. Springer, Cham, 2019: 406-417.
[18] LI M, XIANG Q, CHEN Z, et al. A unified neural network for quality estimation of machine translation[J]. IEICE Transactions on Information and Systems, 2018, 101(9): 2417-2421.
[19] 陆金梁, 张家俊. 基于多语言预训练语言模型的译文质量估计方法[J]. 厦门大学学报 (自然科学版), 202059(2):151-158.
[20] LEE D. Two-phase cross-lingual language model fine-tuning for machine translation quality estimation[C]//Proceedings of the 5th Conference on Machine Translation, 2020: 1024-1028.
[21] ZHAO M, WU H, NIU D, et al. Verdi: Quality estimation and error detection for bilingual corpora[C]//Proceedings of the Web Conference, 2021: 3023-3031.
[22] FAN K, WANG J, LI B, et al. “Bilingual expert” can find translation errors[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(01): 6367-6374.
[23] CUI Q, HUANG S, LI J, et al. DirectQE: Direct pretraining for machine translation quality estimation[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(14): 12719-12727.
[24] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the Advances in Neural Information Processing Systems, 2017: 5998-6008.
[25] JAWAHAR G, SAGOT B, SEDDAH D. What does BERT learn about the structure of language?[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.
[26] HUANG H, LIANG Y, DUAN N, et al. Unicoder: A universal language encoder by pre-training with multiple cross-lingual tasks[C]//Proceedings of EMNLP,2019:2485-2492.
[27] SENNRICH R, HADDOW B, BIRCH A. Neural machine translation of rare words with subword units[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics,2016:1715-1725.
[28] LOSHCHILOV I, HUTTER F. Decoupled weight decoy regularization[C]//Proceedings of the 7th International Conference on Learning Represeatations,2019.
[29] KIM H, LIM J H, KIM H K, et al. QE BERT: Bilingual BERT using multi-task learning for neural quality estimation[C]//Proceedings of the 4th Conference on Machine Translation, 2019: 85-89.
[30] RUBINO R, SUMITA E. Intermediate self-supervised learning for machine translation quality estimation[C]//Proceedings of the 28th International Conference on Computational Linguistics, 2020: 4355-4360.
[31] HOU Q, HUANG S, NING T, et al. NJU submissions for the WMT19 quality estimation shared task[J]. WMT, 2019: 95.
[32] ZHOU J, ZHANG Z, HU Z. Source: Source-conditional elmo-style model for machine translation quality estimation[C]//Proceedings of the 4th Conference on Machine Translation, 2019: 106-111.

基金

国家自然科学基金(61976148);江苏高校优势学科建设工程资助项目
PDF(2439 KB)

Accesses

Citation

Detail

段落导航
相关文章

/