在神经机器翻译中,因词表受限导致的集外词问题很大程度上影响了翻译系统的准确性。对于训练语料较少的资源稀缺型语言的神经机器翻译,这种问题表现得更为严重。近几年,受到外部知识融入的启发,该文在RNNSearch模型基础上,提出了一种融入分类词典的汉越混合网络神经机器翻译集外词处理方法。对于给定的源语言句子,扫描分类词典以确定候选短语句对并标签标记,解码端利用词级组件和短语组件的混合解码网络,很好地生成单词集外词和短语集外词的翻译,从而改善汉越神经机器翻译的性能。在汉越、英越和蒙汉翻译实验上表明,该方法显著提高了准确率,对于资源稀缺型语言的神经机器翻译性能有一定的提升。
Abstract
In neural machine translation, the problem of unknown words caused by limited vocabulary significantly affects the translation quality. Inspired by the integration of external knowledge, this paper investigates to improve the RNNSearch NMT by incorporating the classification dictionary, and proposes a new hybrid network to deal with the unknown words problem in the Chinese-Vietnamese neural machine translation. For source language sentence, the model scans classification dictionary to determine candidate phrase pairs and tags, the decoder uses hybrid network with both word and phrase level components to generate the translations. Experiments on Chinese-Vietnamese, English-Vietnamese and Mongolian-Chinese NMT show that this method significantly improves the translation performance.
关键词
神经机器翻译 /
分类词典 /
资源稀缺 /
集外词
{{custom_keyword}} /
Key words
neural machine translation /
classification dictionaries /
resource-scarce /
unknown words
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Kalchbrenner N, Blunsom P. Recurrent continuous translation models[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Seattle, 2013: 1700-1709.
[2] Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio. Neural machine translation by jointly learning to align and translate[C]//Proceedings of the 2015 International Conference on Learning Representations. USA, Diego, 2015: 865-881.
[3] Ilya Sutskever, Oriol Vinyals, Quoc V Le. Sequence to sequence learning with neural networks[C]//Proceedings of the Advances in Neural Information Processing Systems. Montreal, Quebec, 2014: 3104-3112.
[4] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Doha, Qatar, 2014: 1724-1734.
[5] Tang Y, Meng F, Lu Z, et al. Neural machine translation with external phrase memory[J]. arXiv preprint arXiv: 1606.01792, 2016.
[6] Zhang J, Zong C. Bridging neural machine translation and bilingual lexicon[J].arXiv: 1610.01272,2016.
[7] Caglar Gulcehre, Sungjin Ahn, Ramesh Nallapati, et al. Pointing the unknown words[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Berlin, Germany, 2016: 140-149.
[8] Rico Sennrich, Barry Haddow, Alexandra Birch. Neural machine translation of rare words with subword units[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Berlin, Germany, 2016: 1715-1725.
[9] Marta R Costa-jussa, Jose A R Fonollosa. Character-based neural machine translation[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Berlin, Germany,2016: 357-361.
[10] Wang Ling, Isabel Trancoso, Chris Dyer, et al. Character-based neural machine translation[C]//Proceedings of the 2016 International Conference on Learning Representations. San Juan, Puerto Rico,2016: 785-796.
[11] Yonghui Wu, Mike Schuster, Zhifeng Chen, et al. Google’s neural machine translation system: Bridging the gap between human and machine translation[J]. arXiv preprint arXiv: 1609.08144,2016.
[12] Junyoung Chung, Kyunghyun Cho, Yoshua Bengio. A character-level decoder without explicit segmentation for neural machine translation[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Berlin, Germany,2016: 1693-1703.
[13] Xiaoqing Li, Jiajun Zhang, Chengqing Zong. Towards zero unknown word in neural machine translation[C]//Proceedings of the International Joint Conference on Artificial Intelligence. New York,USA,2016.
[14] Minh-Thang Luong, Ilya Sutskever, Quoc V Le. Addressing the rare word problem in neural machine translation[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Beijing,China, 2015: 11-19.
[15] Sébastien Jean, Kyunghyun Cho, Roland Memisevic. On using very large target vocabulary for neural machine translation[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics.Beijing,China, 2015: 1-10.
[16] Arthur P, Neubig G, Nakamura S. Incorporating discrete translation lexicons into neural machine translation[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Texas, USA, 2016: 1557-1567.
[17] Zhang J, Zong C. Bridging neural machine translation and bilingual dictionaries[J]. arXiv preprint arXiv: 1610.07272, 2016.
[18] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Washington, USA,2014: 1724-1734.
[19] Chung J, Gulcehre C, Cho K, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[C]//Processing of the NIPS 2014 Workshop on Deep Learning. Montreal,Canada, 2014.
[20] Tong Xiao, Jingbo Zhu, Hao Zhang, et al. NiuTrans: An open source toolkit for phrase-based and syntax-based machine translation[C]//Proceedings of the Association for Computational Linguistics, Jeju Island, Korea, 2012: 19-24.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家重点研发计划(2018YFC0830105,2018YFC0830100);国家自然科学基金(61732005,61672271,61761026,61762056,61866020);云南省高新技术产业专项(201606);云南省自然科学基金(2018FB104);云南省科技人才培养项目(KKSY201703015)
{{custom_fund}}
PDF(4187 KB)
