Tibetan Poem Generation with Attention Based Encoder-Decoder Model
SE Chajia1,2, HUA Guocairang1,2, CAI Rangjia1,2, CI Zhenjiacuo1,2, ROU Te1,2
1.MOE Key Laboratory of Tibetan Information Processing, Qinghai Normal University, Xining, Qinghai 810008, China; 2.Provincial Key Laboratory of Tibetan Information Processing and Machine Translation, Qinghai Normal University, Xining, Qinghai 810008, China
Abstract：In this paper, an end-to-end model based on attention is proposed to generate Tibetan poems. The method is built on an end-to-end style without involving manual feature engineering. Under the framework BiLSTM, Tibetan word embedding, attention mechanism and multi-task learning are introduced. The experimental results show that the proposed method reaches 59.27% BLEU score and 62.34% ROUGE value, respectively.
[1] Wu Y, Schuster M, Chen Z, et al. Google’s neural machine translation system: Bridging the gap between human and machine translation[J]. arXiv preprint arXiv:1609.08144, 2016. [2] Gehring J, Auli M, Grangier D, et al. Convolutional sequence to sequence learning [J]. arXiv preprint arXiv:1705.03122v2,2017. [3] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of NIPS 30,2017:1-11. [4] Luan Y, Ji Y, Ostendorf. LSTM based conversation models[J]. arXiv preprint , arXiv: 1603.09457,2016. [5] Yao K, Peng B, Zweig G,et al. An attentional neural conversation model with improved specificity[J]. arXiv preprint,arXiv: 1606.01292,2016. [6] Zhang X, Lapata M. Chinese poetry generation with recurrent neural networks[C]// Proceedings of Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics,2014:670-680. [7] Yi X, Li R, Sun M. Generating Chinese classical poems with RNN encoder-decoder[J]. arXiv preprint arXiv:1604.01537,2017. [8] Manurung H M. An Evolutionary algorithm approach to poetry generation[D]. Edinburgh, UK:University of Edinburgh,2004. [9] Tosa N, Obara H, Minoh M. Hitch Haiku: An interactive supporting system for composing Haiku Poem[C]//Proceedings of Entertainment Computing. Ber-Lin:Springer,2008:209-216. [10] 周昌乐, 游维, 丁晓君. 一种宋词自动生成的遗传算法及其机器实现[J]. 软件学报, 2010, 21(3):427-437. [11] Jiang L, Zhou M. Generating Chinese couplets using a statistical MT approach.[C]//Proceedings of the 22nd International Conference on Computational Linguistics. Stroudsburg, PA:Association for Coputational Linguistics, 2008:377-384. [12] Schuster M, Paliwal K K. Bidirectional recurrent neural networks[J]. IEEE Transactions on Signal Processing, 1997, 45(11):2673-2681. [13] Graves A, Jaitly N, Mohamed A. Hybrid speech recognition with deep bidirectional LSTM [C]//Proceedings of the 2013 IEEE Workshop on Automatic Speech Recognition and Understanding(ASRU).IEEE,2013:273-278. [14] Huang Z,Xu W,Yu K. Bidirectional LSTM-CRF models for sequence tagging[J].arXiv preprint arXiv:1508.01991,2015. [15] Kiperwasser E, Goldberg Y. Simple and accurate dependency parsing using bidirectional LSTM feature representations[J].arXiv preprint arXiv:1603.04351,2016. [16] Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate[J]. arXiv preprint arXiv:1409. 0473, 2014. [17] Mikolov T, Chen K, Corrado G, et al. Efficient estimation of word representations in Vector Space[J]. arXiv preprint arXiv: 1301.3781, 2013. [18] Luong M T, Pham H, Manning C D. Effective approaches to attention-based neural machine translation[J]. arXiv preprint arXiv: 1508.04025, 2015. [19] 张瑾,王小磊,许洪波.自动文摘评价方法综述[J].中文信息学报,2008,22(3):81-88.
2019, Vol. 33 
