当域外和域内分别表示不同的语言时,语言之间的差异会导致域外知识难以适应至域内。因此提出域对抗迁移学习方法来改进机器翻译模型。采用对抗学习方法,加入一个域判别器对域外和域内的语义特征进行预测,通过最小化域外和域内语义特征预测值优化编码器。当两个领域的语义特征预测值相近时,说明模型学习到一个可以把域内数据映射到域外的映射函数。通过实验,该方法在蒙古语-汉语和维吾尔语-汉语等翻译任务上展现出一定的泛化能力。
Abstract
When the out-of-domain and in-domain represent different languages, the differences between languages will make it difficult adapt the out-of-domain knowledge to the in-domain. This paper proposes a domain-adversarial transfer learning method to improve the neural machine translation model. Under the adversarial learning framework, a domain discriminator is employed to predict the semantic features that from out-of-domain or in-domain, and the encoder is optimized by minimizing the prediction values of the semantic features. When the predicted values of semantic features in the two domains are similar, it means that the model has learned the mapping function that can transfer in-domain data into out-of-domain. Experiments show a certain generalization ability of this method on Mongolian-Chinese and Uyghur-Chinese translation tasks.
关键词
对抗 /
机器翻译 /
多语言 /
对抗学习
{{custom_keyword}} /
Key words
domain adaption /
machine translation /
multi-language /
adversarial learning
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] DZMITRY B, KYUNGHYUN C, YOSHUA B.Neural machine translation by jointly learning to align and translate[C]//Proceedings of ACL-IJCNLP, 2015.
[2] SUTSKEVER I, VINYALS O, LE Q V.Sequence to sequence learning with neural networks[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems, 2014: 3104-3112.
[3] FREITAG M, AL-ONAIZAN Y.Fast domain adaptation for neural machine translation.[J].Computing Research Repository, 2016, 1(12): 1-12.
[4] YANG Z, CHEN W, WANG F.Unsupervised domain adaptation for neural machine translation[C]//Proceedings of the 24th International Conference on Pattern Recognition, 2018: 338-343.
[5] SATO S, SAKUMA J, YOSHINAGA N, et al.Vocabulary adaptation for distant domain adaptation in neural machine translation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020: 4269-4279.
[6] JOHNSON M, SCHUSTER M, LE Q V, et al.Googles multilingual neural machine translation system: enabling zero-shot translation[J].Computing Research Repository, 2016, 1(8): 1-17.
[7] VASWANI A, SHAZEER N, PARMAR N, et al.Attention is all you need[C]//Proceedings of International Annual Conference on Neural Information Processing Systems.Long Beach: NIPS, 2017: 5998-6008.
[8] RUI W, UTIYAMA M, LIU L, et al.Instance weighting for neural machine translation domain adaptation[C]//Proceedings of EMNLP, 2017: 1482-1488.
[9] 任众, 侯宏旭, 吉亚图, 等. 子字粒度切分在蒙汉神经机器翻译中的应用[J].中文信息学报, 2019, 33(1): 85-92.
[10] 孙硕, 侯宏旭, 乌尼尔,等. 基于迭代知识精炼的对偶学习蒙汉机器翻译[J].厦门大学学报(自然科学版),2021, 60(4): 687-692.
[11] 武子玉, 侯宏旭, 白天罡,等. 基于CSGAN的多模型融合蒙汉神经机器翻译研究[J].江西师范大学学报(自然科学版), 2020, 044(002): 153-159.
[12] LAMPLE G, CONNEAU A.Cross-lingual language model pretraining[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems, 2019: 7059-7069.
[13] KEMENTCHEDJHIEVA Y, RUDER S, COTTERELL R, et al.Generalizing procrustes analysis for better bi-lingual dictionary induction[C]//Proceedings of the SIGNLL Conference on Computational Natural Language Learning, 2018: 211-220.
[14] JOULIN A, GRAVE E, BOJANOWSKI P, et al.Fasttext.zip: Compressing text classification models[J].Computing Research Repository, 2016, 1(12): 1-13.
[15] PAPINENI S.Bleu: A method for automatic evaluation of machine translation[C]//Proceedings of Meeting of the Association for Computational Linguistics, 2002: 311-318.
[16] ZOPH B, YURET D, MAY J, et al.Transfer learning for low-resource neural machine translation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2016: 1568-1575.
[17] WU L J, XIA Y, ZHAO L, et al.Adversarial neural machine translation[J].Computing Research Repository, 2017, 1(9): 1-11.
[18] CONNEAU A, LAMPLE G, RANZATO M, et al.Word translation without parallel data[J].Computing Research Repository, 2017, 1(1): 1-14.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
内蒙古自治区科技成果转化专项(2019CG028);内蒙古自然科学基金(2018MS06005,14020202-0114)
{{custom_fund}}
PDF(3858 KB)
