Unknown Words Processing Method for Chinese-Vietnamese Neural Machine Translation Based on Hybrid Network Integrating Classification Dictionaries
CHE Wanjin1,2, YU Zhengtao1,2, GUO Junjun1,2, WEN Yonghua1,2, YU Zhiqiang1,2
1.Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, Yunnan 650500, China; 2.Yunnan Key Laboratory of Artificial Intelligence, Kunming University of Science and Technology, Kunming, Yunnan 650500, China
Abstract：In neural machine translation, the problem of unknown words caused by limited vocabulary significantly affects the translation quality. Inspired by the integration of external knowledge, this paper investigates to improve the RNNSearch NMT by incorporating the classification dictionary, and proposes a new hybrid network to deal with the unknown words problem in the Chinese-Vietnamese neural machine translation. For source language sentence, the model scans classification dictionary to determine candidate phrase pairs and tags, the decoder uses hybrid network with both word and phrase level components to generate the translations. Experiments on Chinese-Vietnamese, English-Vietnamese and Mongolian-Chinese NMT show that this method significantly improves the translation performance.
[1] Kalchbrenner N, Blunsom P. Recurrent continuous translation models[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Seattle, 2013: 1700-1709. [2] Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio. Neural machine translation by jointly learning to align and translate[C]//Proceedings of the 2015 International Conference on Learning Representations. USA, Diego, 2015: 865-881. [3] Ilya Sutskever, Oriol Vinyals, Quoc V Le. Sequence to sequence learning with neural networks[C]//Proceedings of the Advances in Neural Information Processing Systems. Montreal, Quebec, 2014: 3104-3112. [4] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Doha, Qatar, 2014: 1724-1734. [5] Tang Y, Meng F, Lu Z, et al. Neural machine translation with external phrase memory[J]. arXiv preprint arXiv: 1606.01792, 2016. [6] Zhang J, Zong C. Bridging neural machine translation and bilingual lexicon[J].arXiv: 1610.01272,2016. [7] Caglar Gulcehre, Sungjin Ahn, Ramesh Nallapati, et al. Pointing the unknown words[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Berlin, Germany, 2016: 140-149. [8] Rico Sennrich, Barry Haddow, Alexandra Birch. Neural machine translation of rare words with subword units[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Berlin, Germany, 2016: 1715-1725. [9] Marta R Costa-jussa, Jose A R Fonollosa. Character-based neural machine translation[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Berlin, Germany,2016: 357-361. [10] Wang Ling, Isabel Trancoso, Chris Dyer, et al. Character-based neural machine translation[C]//Proceedings of the 2016 International Conference on Learning Representations. San Juan, Puerto Rico,2016: 785-796. [11] Yonghui Wu, Mike Schuster, Zhifeng Chen, et al. Google’s neural machine translation system: Bridging the gap between human and machine translation[J]. arXiv preprint arXiv: 1609.08144,2016. [12] Junyoung Chung, Kyunghyun Cho, Yoshua Bengio. A character-level decoder without explicit segmentation for neural machine translation[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Berlin, Germany,2016: 1693-1703. [13] Xiaoqing Li, Jiajun Zhang, Chengqing Zong. Towards zero unknown word in neural machine translation[C]//Proceedings of the International Joint Conference on Artificial Intelligence. New York,USA,2016. [14] Minh-Thang Luong, Ilya Sutskever, Quoc V Le. Addressing the rare word problem in neural machine translation[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Beijing,China, 2015: 11-19. [15] Sébastien Jean, Kyunghyun Cho, Roland Memisevic. On using very large target vocabulary for neural machine translation[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics.Beijing,China, 2015: 1-10. [16] Arthur P, Neubig G, Nakamura S. Incorporating discrete translation lexicons into neural machine translation[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Texas, USA, 2016: 1557-1567. [17] Zhang J, Zong C. Bridging neural machine translation and bilingual dictionaries[J]. arXiv preprint arXiv: 1610.07272, 2016. [18] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Washington, USA,2014: 1724-1734. [19] Chung J, Gulcehre C, Cho K, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[C]//Processing of the NIPS 2014 Workshop on Deep Learning. Montreal,Canada, 2014. [20] Tong Xiao, Jingbo Zhu, Hao Zhang, et al. NiuTrans: An open source toolkit for phrase-based and syntax-based machine translation[C]//Proceedings of the Association for Computational Linguistics, Jeju Island, Korea, 2012: 19-24.
2019, Vol. 33 
