文本摘要旨在对冗长的文本进行简短精确的总结,同时保留文本的原始语义。该文提出一种融合义原的中文摘要生成方法(Add Sememe-Pointer Model, ASPM),以词为单位在LCSTS数据集上进行实验。算法利用基于Seq2Seq的指针网络模型以解决由于词汇表规模导致的未登录词问题。考虑到中文一词多义现象较多,只通过指针网络模型难以很好地理解文本语义,导致生成的摘要可读性不高。方法引入了义原知识库,训练多义词的词向量表示,准确地捕捉一个词在上下文的具体含义,并对LCSTS中的一些多义词进行义原标注,以使算法能更好地获取数据集中词语的语义信息。实验结果表明,该文提出的融合义原的中文摘要生成方法可以得到更高的ROUGE分数,使生成的摘要更加具有可读性。
Abstract
Text summarization aims at generating a brief and accurate summary from lengthy text without changing the original semantics of the text. A novel summarization method called Add Sememe-Pointer Model (ASPM) is proposed in this paper. The ASPM applies the pointer network in the Seq2Seq framework to solve the out-of-vocabulary problem. Considering the polysemous phenomenon in Chinese, the pointer network model does not fully understand the text semantics, leading to the poor performance of the model. Our method also uses the sememe knowledge bases to train the word vector representation of polysemous words, which can accurately capture the specific meaning of a word in the context, and we annotate some polysemous words in the LCSTS dataset so that the method can better understand the semantic information of the words in the dataset. The experimental results show that the ASPM can achieve higher ROUGE scores and make the Chinese summary more readable.
关键词
文本摘要 /
义原 /
指针网络 /
文本语义 /
词向量
{{custom_keyword}} /
Key words
text summarization /
sememe /
pointer network /
text semantics /
word embedding
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Lei L, Xiaojun W.Overview of the NLPCC shared task: single document summarization[C]//Proceedings of the CCF International Conference on Natural Language Processing and Chinese Computing,2018: 457-463.
[2] 侯圣峦,张书涵,费超群.文本摘要常用数据集和方法研究综述[J].中文信息学报,2019,33(5): 1-16.
[3] 王红玲,周国栋,朱巧明.面向冗余度控制的中文多文档自动文摘[J].中文信息学报,2012,26(2): 92-97.
[4] Liu F,Flanigan J,Thomson S,et al.Toward abstractive summarization using semantic represe-ntations[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,2015: 1077-1086.
[5] 吴仁守,张宜飞,王红玲,等.基于层次结构的生成式自动文摘[J].中文信息学报,2019,33(10): 90-98.
[6] Dong Z, Dong Q.HowNet: A hybrid language and knowledge resource[C]//Proceedings of the International Conference on Natural Language Processing and Knowledge Engineering,2003: 820-824.
[7] Yilin N, Ruobing X, Zhiyuan L, et al. Improved word representation learning with sememes[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics,2017: 2049-2058.
[8] Sutskever I,Vinyals O,Le Q V.Sequence to sequence learning with neural networks[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems,2014: 3104-3112.
[9] Rush A M,Chopra S,Weston J.A neural attention model for abstractive sentence summarization[C]//Proceedings of the Conference on Empirical Methods for Natural Language Processing,2015: 379-389.
[10] Hu B,Chen Q,Zhu F.LCSTS: A large scale chinese short text summarization dataset[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing,2015: 1967-1972.
[11] Yunheng Z, Leihan Z, Ke X,et al. A hierarchical hybrid neural network architecture for chinese text summarization[C]//Proceedings of the Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data,2018: 277-288.
[12] 林鸿飞,杨志豪.面向中文新闻领域的移动摘要系统[J],中文信息学报,2008,22(1): 87-92.
[13] Chopra S, Michael A, Alexander M R. Abstractive sentence summarization with attentive recurrent neural networks[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,2016: 93-98.
[14] Konstantin Lopyrev. Generating news headlines with recurrent neural networks[J].arXiv preprint arXiv:1512.01712,2015.
[15] Vinyals O, Meire F, Navdeep J. Pointer networks[C]//Proceedings of the Neural Information Processing Systems,2015: 2692-2700.
[16] Jiatao G, Zhengdong L, Hang L, et al.Incorporating copying mechanism in sequence-to-seq-uence learning[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics,2016: 1631-1640.
[17] Caglar G, Sungjin A, Ramesh N, et al. Pointing the unknown words[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics,2016: 140-149.
[18] Ramesh N, Bowen Z, Cicero dos S, et al. Abstractive text summarization using sequence-to-sequence RNNs and beyond[C]//Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning,2016: 280-290.
[19] Yishu M, Phil B. Language as a latent variable: discrete generative models for sentence compression[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing,2016: 319-328.
[20] Abigail S, Peter L, Christopher M. Get to the point: summarization with pointer-generator networks[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics,2017: 1073-1083.
[21] 唐共波,于东,荀恩东.基于知网义原词向量表示的无监督词义消歧方法[J].中文信息学报,2015,29(6): 23-29.
[22] 孙茂松,陈新雄.借重于人工知识库的词和义项的向量表示: 以HowNet为例[J].中文信息学报,2016,30(6): 1-6.
[23] Tomas M, Kai C, Greg C,et al.Efficient estimation of word representations in vector space[C]//Proceedings of the International Conference on Learning Representations,2013.
[24] Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate[C]//Proceedings of the International Conference on Learning Representations,2015.
[25] Chin-Yew L. ROUGE: A package for automatic evaluation of summaries[C]//Proceedings of the Workshop on Text Summarization Branches Out,2004: 74-81.
[26] John D, Elad H, Yoram S. Adaptive subgradient methods for online learning and stochastic optimization[J].Machine Learning Research, 2011: 2122-2159.
[27] Markus F, Yaser A O. Beam search strategies for neural machine translation[C]//Proceedings of the 1st Workshop on Neural Machine Translation, 2017: 56-60.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(61671070);国家语委重点项目(ZDI135-53);国家社会科学基金(14@ZH036)
{{custom_fund}}
PDF(2537 KB)
