神经网络机器翻译研究热点与前沿趋势分析

林倩,刘庆,苏劲松,林欢,杨静,罗斌

PDF(2970 KB)
PDF(2970 KB)
中文信息学报 ›› 2019, Vol. 33 ›› Issue (11) : 1-14.
综述

神经网络机器翻译研究热点与前沿趋势分析

  • 林倩,刘庆,苏劲松,林欢,杨静,罗斌
作者信息 +

Focuses and Frontiers Tendency in Neural Machine Translation Research

  • LIN Qian, LIU Qing, SU Jinsong, LIN Huan, YANG Jing, LUO Bin
Author information +
History +

摘要

机器翻译是指利用计算机将一种语言文本转换成具有相同语义的另一种语言文本的过程。它是人工智能领域的一项重要研究课题。近年来,随着深度学习研究和应用的快速发展,神经网络机器翻译成为机器翻译领域的重要发展方向。该文首先简要介绍近一年神经网络机器翻译在学术界和产业界的影响,然后对当前的神经网络机器翻译的研究进展进行分类综述,最后对后续的发展趋势进行展望。

Abstract

Machine translation is the process of to attempting convert text from one language to another using computers, which has already become the research issues of great importance in artificial intelligence. With the fast growth of deep learning research and application, it has been revealed that neural machine translation become a mainstream of research for machine translation. This paper firstly introduces the influence of neural machine translation in academia and industry in the past year, and then reviews the research progress on neural machine translation, finally we outline the outlook for its future development.

关键词

人工智能 / 深度学习 / 神经网络机器翻译

Key words

artificial intelligence / deep learning / neural machine translation

引用本文

导出引用
林倩,刘庆,苏劲松,林欢,杨静,罗斌. 神经网络机器翻译研究热点与前沿趋势分析. 中文信息学报. 2019, 33(11): 1-14
LIN Qian, LIU Qing, SU Jinsong, LIN Huan, YANG Jing, LUO Bin. Focuses and Frontiers Tendency in Neural Machine Translation Research. Journal of Chinese Information Processing. 2019, 33(11): 1-14

参考文献

[1] Passban P,Liu Q,Way A. Improving character-based decoding using target-side morphological information for neural machine translation[C]//Proceedings of the North American Chapter of the Association for Computational Linguistics,2018: 58-68.
[2] Chen H,Huang S,Chiang D,et al. Combining character and word information in neural machine translation using a multi-level attention[C]//Proceedings of the North American Chapter of the Association for Computational Linguistics,2018: 1284-1293.
[3] Zhao S,Zhang Z.Attention-via-attention neural machine translation[C]//Proceedings of the National Conference on Artificial Intelligence,2018: 563-570.
[4] Sennrich R,Haddow B,Birch A. Neural machine translation of rare words with subword units[C]//Proceedings of Meeting of the Association for Computational Linguistics,2016: 1715-1725.
[5] Kudo T. Subword regularization: Improving neural network translation models with multiple subword candidates[C]//Proceedings of Meeting of the Association for Computational Linguistics,2018: 66-75.
[6] Morishita M,Suzuki J,Nagata M. Improving neural machine translation by incorporating hierarchical subword yeatures[C]//Proceedings of the International Conference on Computational Linguistics,2018: 618-629.
[7] Song K,Zhang Y,Zhang M,et al.Improved English to Russian translation by neural suffix prediction[C]//Proceedings of the National Conference on Artificial Intelligence,2018: 410-417.
[8] Passban P,Way A,Liu Q. Tailoring neural architectures for translating from morphologically rich languages[C]//Proceedings of the International Conference on Computational Linguistics,2018: 3134-3145.
[9] Lample G,Ott M,Conneau A,et al.Phrase-based & neural unsupervised machine translation[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 5039-5049.
[10] Lample G,Conneau A,Denoyer L,et al.Unsupervised machine translation using monolingual corpora only[C]//Proceedings of the International Conference on Learning Representations,2018: 1-14.
[11] Artetxe M,Labaka G,Agirre E,et al.Unsupervised neural machine translation[C]//Proceedings of the International Conference on Learning Representations,2018: 1-12.
[12] Yang Z,Chen W,Wang F,et al.Unsupervised neural machine translation with weight sharing[C]//Proceedings of Meeting of the Association for Computational Linguistics,2018: 46-55.
[13] Fadaee M,Monz C. Back-translation sampling by targeting difficult words in neural machine translation[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 436-446.
[14] Zhang Z,Liu S,Li M,et al.Joint training for neural machine translation models with monolingual data[C]//Proceedings of the National Conference on Artificial Intelligence,2018: 555-562.
[15] Wang Y,Xia Y,Zhao L,et al.Dual transfer learning for neural machine translation with marginal distribution regularization[C]//Proceedings of the National Conference on Artificial Intelligence,2018: 5553-5560.
[16] Chen Y,Liu Y,Li V O. Zero-Resource neural machine translation with multi-agent communication game[C]//Proceedings of the National Conference on Artificial Intelligence,2018: 5086-5093.
[17] Ren S,Chen W,Liu S,et al. Triangular architecture for rare language Ttranslation[C]//Proceedings of the Meeting of the Association for Computational Linguistics,2018: 56-65.
[18] Zhang S,Xiong D. Sentence weighting for neural machine translation domain adaptation[C]//Proceedings of the International Conference on Computational Linguistics,2018: 3181-3190.
[19] Wang R,Utiyama M,Finch A M,et al. Sentence selection and weighting for neural machine translation domain adaptation[J]. IEEE Transactions on Audio,Speech,and Language Processing,2018,26(10): 1727-1741.
[20] Zeng J,Su J,Wen H,et al. Multi-domain neural machine translation with word-level domain context discrimination[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 447-457.
[21] Ha D,Dai A M,Le Q V,et al.Hyper Networks[C]//Proceedings of the International Conference on Learning Representations,2017: 1-18.
[22] Platanios E A,Sachan M,Neubig G,et al.Contextual parameter generation for universal neural machine translation[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 425-435.
[23] Chu C,Wang R. A survey of domain adaptation for neural machine translation[C]//Proceedings of the International Conference on Computational Linguistics,2018: 1304-1319.
[24] Delbrouck J B,Dupont S. Modulating and attending the source image during encoding improves Multimodal Translation[C]//Proceedings of the NIPS 2017 Workshop on Visually-Grounded Interaction and Language (ViGIL),2017: 1-9.
[25] Zhou M,Cheng R,Lee Y,et al.A visual attention grounding neural model for multimodal machine translation[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 3643-3653.
[26] Gu J,Hassan H,Devlin J,et al.Universal neural machine translation for extremely Low resource languages[C]//Proceedings of the North American Chapter of the Association for Computational Linguistics,2018: 344-354.
[27] Blackwood G W,Ballesteros M,Ward T. Multilingual neural machine translation with task-specific attention[C]//Proceedings of the International Conference on Computational Linguistics,2018: 3112-3122.
[28] Gu J,Wang Y,Chen Y,et al.Meta-learning for low-resource neural machine translation[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 3622-3631.
[29] Wang Y,Zhang J,Zhai F,et al.Three strategies to improve one-to-many multilingual translation[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 2955-2960.
[30] Kiperwasser E,Ballesteros M. Scheduled multi-task learning: From syntax to translation[J].Transactions of the Association for Computational Linguistics,2018: 225-240.
[31] Zaremoodi P,Haffari G. Neural machine translation for bilingually scarce scenarios: A deep multi-task learning approach[C]//Proceedings of the North American Chapter of the Association for Computational Linguistics,2018: 1356-1365.
[32] Niu X,Rao S,Carpuat M. Multi-task neural models for translating between styles within and across languages[C]//Proceedings of the International Conference on Computational Linguistics,2018: 1008-1021.
[33] Su J,Zeng J,Xiong D,et al.A hierarchy-to-sequence attentional neural machine translation model[J]. IEEE Transactions on Audio,Speech,and Language Processing,2018,26(3): 623-632.
[34] Zhang W,Hu J,Feng Y,et al. Refining source representations with relation networks for neural machine translation[C]//Proceedings of the International Conference on Computational Linguistics,2018: 1292-1303.
[35] Xiong H,He Z,Hu X,et al. Multi-channel encoder for neural machine translation[C]//Proceedings of the National Conference on Artificial Intelligence,2018: 4962-4969.
[36] Shen Y,Tan X,He D,et al. Dense information flow for neural machine translation[C]//Proceedings of the North American Chapter of The Association for Computational Linguistics,2018: 1294-1303.
[37] Werlen L M,Pappas N,Ram D,et al.Self-attentive residual decoder for neural machine translation[C]//Proceedings of the North American Chapter of The Association for Computational Linguistics,2018: 1366-1379.
[38] Wang M,Xie J,Tan Z,et al.Neural machine translation with decoding history enhanced attention[C]//Proceedings of the International Conference on Computational Linguistics,2018: 1464-1473.
[39] Zhang B,Xiong D,Su J. Neural machine translation with deep attention[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2018: 11.
[40] Luong T,Pham H,Manning C D. Effective approaches to attention-based neural machine translation[C]//Proceedings of the Empirical Methods in Natural Language Processing,2015: 1412-1421.
[41] Chen K,Wang R,Utiyama M,et al.Syntax-directed attention for neural machine translation[C]//Proceedings of the National Conference on Artificial Intelligence,2018: 4792-4799.
[42] Meng F,Tu Z,Cheng Y,et al.Neural machine translation with key-value memory-augmented attention[C]//Proceedings of the International Joint Conference on Artificial Intelligence,2018: 2574-2580.
[43] Li X,Liu L,Tu Z,et al.Target foresight based attention for neural machine translation[C]//Proceedings of the North American Chapter of The Association for Computational Linguistics,2018: 1380-1390.
[44] Huang P,Wang C,Huang S,et al. Towards neural phrase-based machine translation[C]//Proceedings of the International Conference on Learning Representations,2018: 1-14.
[45] Wang C,Wang Y,Huang P,et al.Sequence modeling via segmentations[C]//Proceedings of the International Conference on Machine Learning,2017: 3674-3683.
[46] Shi X,Knight K. Speeding up neural machine translation decoding by shrinking run-time vocabulary.[C]//Proceedings of Meeting of the Association for Computational Linguistics,2017: 574-579.
[47] Zhang W,Huang L,Feng Y,et al.Speeding up neural machine translation decoding by cube pruning[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 4284-4294.
[48] Post M,Vilar D. Fast lexically constrained decoding with dynamic beam allocation for neural machine translation[C]//Proceedings of the North American Chapter of The Association for Computational Linguistics,2018: 1314-1324.
[49] Kuang S,Li J,Branco A,et al.Attention focusing for neural machine translation by bridging source and target embeddings[C]//Proceedings of Meeting of the Association for Computational Linguistics,2018: 1767-1776.
[50] Nguyen T Q,Chiang D. Improving lexical choice in neural machine translation[C]//Proceedings of the North American Chapter of The Association for Computational Linguistics,2018: 334-343.
[51] Liu F,Lu H,Neubig G. Handling homographs in neural machine translation[C]//Proceedings of the North American Chapter of The Association for Computational Linguistics,2018: 1336-1345.
[52] Wang L,Tu Z,Shi S,et al.Translating pro-drop languages with reconstruction models[C]//Proceedings of the National Conference on Artificial Intelligence,2018: 4937-4945.
[53] Li Y,Li J,Zhang M. Adaptive weighting for neural machine translation[C]//Proceedings of the International Conference on Computational Linguistics,2018: 3038-3048.
[54] Zhang B,Xiong D,Su J,et al.Simplifying neural machine translation with addition-subtraction twin-gated recurrent networks[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 4273-4283.
[55] See A,Luong M,Manning C D,et al. Compression of neural machine translation models via pruning[C]//Proceedings of the Conference on Computational Natural Language Learning,2016: 291-301.
[56] Shu R,Nakayama H. Compressing word embeddings via deep compositional code learning[C]//Proceedings of the International Conference on Learning Representations,2018: 1-13.
[57] Xia Y,Tian F,Wu L,et al.Deliberation networks: Sequence generation beyond one-pass decoding[C]//Proceedings of the Neural Information Processing Systems,2017: 1784-1794.
[58] Zhang X,Su J,Qin Y,et al.Asynchronous bidirectional decoding for neural machine translation[C]//Proceedings of the National Conference on Artificial Intelligence,2018: 5698-5705.
[59] Geng X,Feng X,Qin B,et al. Adaptive multi-pass decoder for neural machine translation[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 523-532.
[60] Su J,Wu S,Xiong D,et al. Variational recurrent neural machine translation[C]//Proceedings of the National Conference on Artificial Intelligence,2018: 5488-5495.
[61] Schulz P,Aziz W,Cohn T. A stochastic decoder for neural tachine translation[C]//Proceedings of Meeting of The Association for Computational Linguistics,2018: 1243-1252.
[62] Lin J,Sun X,Ren X,et al.Deconvolution-based global decoding for neural machine translation[C]//Proceedings of the International Conference on Computational Linguistics,2018: 3260-3271.
[63] Zheng Z,Zhou H,Huang S,et al. Modeling past and future for neural machine translation[J].Transactions of the Association for Computational Linguistics,2018: 145-157.
[64] Kuang S,Xiong D. Fusing recency into neural machine translation with an inter-sentence gate model[C]//Proceedings of the International Conference on Computational Linguistics,2018: 607-617.
[65] Tu Z,Liu Y,Shi S,et al. Learning to remember translation history with a continuous cache[J].Transactions of the Association for Computational Linguistics,2018: 407-420.
[66] Maruf S,Haffari G. Document context neural machine translation with memory networks[C]//Proceedings of Meeting of the Association for Computational Linguistics,2018: 1275-1284.
[67] Kuang S,Xiong D,Luo W,et al. Modeling coherence for neural machine translation with dynamic and topic Caches[C]//Proceedings of the International Conference on Computational Linguistics,2018: 596-606.
[68] Wang X,Tu Z,Zhang M,et al.Incorporating statistical machine translation word knowledge into neural machine translation[J]. IEEE Transactions on Audio,Speech,and Language Processing,2018,26(12): 2255-2266.
[69] Zhao Y,Wang Y,Zhang J,et al. Phrase table as recommendation memory for neural machine translation[C]//Proceedings of the International Joint Conference on Artificial Intelligence,2018: 4609-4615.
[70] Zhao Y,Zhang J,He Z,et al.Addressing troublesome words in neural machine translation[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 391-400.
[71] Chen H,Huang S,Chiang D,et al. Improved neural machine translation with a syntax-aware encoder and decoder[C]//Proceedings of Meeting of the Association for Computational Linguistics,2017: 1936-1945.
[72] Li J,Xiong D,Tu Z,et al.Modeling source syntax for neural machine translation.[C]//Proceedings of Meeting of the Association for Computational Linguistics,2017: 688-697.
[73] Wu S,Zhang D,Yang N,et al.Sequence-to-dependency neural machine translation[C]//Proceedings of Meeting of the Association for Computational Linguistics,2017: 698-707.
[74] Wu S,Zhang D,Zhang Z,et al.Dependency-to-dependency neural machine translation[J].IEEE Transactions on Audio,Speech,and Language Processing,2018,26(11): 2132-2141.
[75] Ma C,Tamura A,Utiyama M,et al. Forest-based neural machine translation[C]//Proceedings of Meeting of the Association for Computational Linguistics,2018: 1253-1263.
[76] Zaremoodi P,Haffari G. Incorporating syntactic uncertainty in neural machine translation with a forest-to-sequence model[C]//Proceedings of the International Conference on Computational Linguistics,2018: 1421-1429.
[77] Gu J,Shavarani H,Sarkar A. Top-down tree structured decoding with syntactic connections for neural machine translation and parsing[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 401-413.
[78] Bastings J,Titov I,Aziz W,et al. Graph convolutional encoders for syntax-aware neural machine translation[C]//Proceedings of the Empirical Methods in Natural Language Processing,2017: 1957-1967.
[79] Li Q,Wong D,Chao L,et al.Linguistic knowledge-aware neural machine translation[J]. IEEE Transactions on Audio,Speech,and Language Processing,2018,26(12): 2341-2354.
[80] Gu J,Wang Y,Cho K,et al. Search engine guided neural machine translation[C]//Proceedings of the National Conference on Artificial Intelligence,2018: 5133-5140.
[81] Zhang J,Utiyama M,Sumita E,et al.Guiding neural machine translation with retrieved translation pieces[C]//Proceedings of the North American Chapter of the Association for Computational Linguistics,2018: 1325-1335.
[82] Ugawa A,Tamura A,Ninomiya T,et al.Neural machine translation incorporating named entity[C]//Proceedings of the International Conference on Computational Linguistics,2018: 3240-3250.
[83] Vaswani A,Shazzer N,Parmar N,et al.Attention is all you need[C]//Proceedings of the Neural Information Processing Systems,2017: 5998-6008.
[84] Gu J,Bradbury J,Xiong C,et al.Non-autoregressive neural machine translation[C]//Proceedings of the International Conference on Learning Representations,2018: 1-13.
[85] Wang C,Zhang J,Chen H,Semi-autoregressive neural machine translation[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 479-488.
[86] Shaw P,Uszkoreit J,Vaswani A,Self-attention with relative position representations[C]//Proceedings of the North American Chapter of the Association for Computational Linguistics,2018: 464-468.
[87] Yang B,Tu Z,Wong D,et al.Modeling localness for self-attention networks[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 4449-4458.
[88] Shen T,Zhou T,Long G,et al.DiSAN: Directional self-attention network for RNN/CNN-Free language understanding[C]//Proceedings of the National Conference on Artificial Intelligence,2018: 5446-5455.
[89] Zhang B,Xiong D,Su J,Accelerating neural transformer via an average attention network[C]//Proceedings of Meeting of the Association for Computational Linguistics,2018: 1789-1798.
[90] Li J,Tu Z,Yang B,et al.Multi-head attention with disagreement regularization[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 2897-2903.
[91] Shen T,Zhou T,Long G,et al.Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling[C]//Proceedings of the International Conference on Learning Representations,2018: 1-18.
[92] Dou Z,Tu Z,Wang X,et al.Exploiting deep representations for neural machine translation[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 4253-4262.
[93] Wang Q,Li F,Xiao T,et al.Multi-layer representation fusion for neural machine translation[C]//Proceedings of the International Conference on Computational Linguistics. 2018: 3015-3026.
[94] Gehring J,Auli M,Grangier D,et al. Convolutional sequence to sequence learning[C]//Proceedings of the International Conference on Machine Learning,2017: 1243-1252.
[95] Gehring J,Auli M,Grangier D,et al.A Convolutional encoder model for neural machine translation[C]//Proceedings of Meeting of the Association for Computational Linguistics,2017: 123-135.
[96] Kaiser L,Gomez A N,Chollet F. Depthwise separable convolutions for neural machine translation[C]//Proceedings of the International Conference on Learning Representations,2018: 1-10.
[97] Chen M X,Firat O,Bapna A,et al.The best of both worlds: Combining recent advances in neural machine translation[C]//Proceedings of Meeting of the Association for Computational Linguistics,2018: 76-86.
[98] Szegedy C,Vanhoucke V,Ioffe S,et al. Rethinking the Inception Architecture for Computer Vision[J]. Computer Vision and Pattern Recognition,2016: 2818-2826.
[99] Domhan T. How much attention do you need A granular analysis of neural machine translation architectures[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics,2018,1: 1799-1808.
[100] Wang M X. Neural machine translation with decoding history enhanced attention[C]//Proceedings of the International Conference on Computational Linguistics,2018: 1464-1473.
[101] Lakew S M,Cettolo M,Federico M. A comparison of transformer and recurrent neural networks on multilingual neural machine translation[C]//Proceedings of the International Conference on Computational Linguistics,2018: 641-652.
[102] Tang G,Muller M,Gonzales A R,et al.Why self-attention? A targeted evaluation of neural machine translation architectures[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 4263-4272.
[103] Ott M A,Auli M,Grangier D,et al.Analyzing uncertainty in neural machine translation[C]//Proceedings of the International Conference on Machine Learning,2018: 3953-3962.
[104] Belinkov Y,Bisk Y. Synthetic and natural noise both break neural machine translation[C]//Proceedings of the International Conference on Learning Representations,2018: 1-13.
[105] Tan X,Wu L,He D,et al. Beyond error propagation in neural machine translation: Characteristics of language also matter[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 3602-3611.
[106] Bahdanau D,Brakel P,Xu K,et al.An actor-critic algorithm for sequence prediction[C]//Proceedings of the International Conference on Learning Representations,2017: 1-17.
[107] He D,Lu H,Xia Y,et al.Decoding with value networks for neural machine translation[C]//Proceedings of the Neural Information Processing Systems,2017: 178-187.
[108] Wu L,Tian F,Qin T,et al.A study of reinforcement learning for neural machine translation[C]//Proceedings of the Empirical Methods in Natural Language Processing,2018: 3612-3621.
[109] Ebrabimi J,Lowd D,Dou D. On adversarial examples for character-level neural machine translation[C]//Proceedings of the International Conference on Computational Linguistics,2018: 653-663.
[110] Cheng Y,Tu Z,Meng F,et al.Towards robust neural machine translation[C]//Proceedings of Meeting of the Association for Computational Linguistics,2018: 1756-1766.
[111] Gu J,Im D J,Li V O. Neural machine translation with gumbel-greedy decoding[C]//Proceedings of the National Conference on Artificial Intelligence,2018: 5125-5132.
[112] Yang Z,Chen W,Wang F,et al. Improving neural machine translation with conditional sequence generative adversarial nets[C]//Proceedings of the North American Chapter of Association for Computational Linguistics,2018: 1346-1355.

基金

国家自然科学基金(61672440);国家语委一般项目课题(YB135-49);厦门大学校长基金(ZK1024);国家重点研发计划(2019QY1803)
PDF(2970 KB)

787

Accesses

0

Citation

Detail

段落导航
相关文章

/