文本摘要的目标是将长文本进行压缩、归纳和总结,从而形成具有概括性含义的短文本,其能帮助人们快速获取文档的主要信息。当前大多数的抽取式文本摘要的研究都是以整句作为抽取单元,而整句作为抽取单元会引入冗余信息,因此该文考虑使用粒度更细的抽取单元。已有研究表明,细粒度的子句单元比整句单元在抽取式摘要上更具有优势。结合当下热门的图神经网络,该文提出了一种基于子句单元异构图网络的抽取式摘要模型,有效融合了词、实体和子句单元等不同层次的语言信息,能够实现更细粒度的抽取式摘要。在大规模基准语料库(CNN/DM和NYT)上的实验结果表明,该模型产生了突破性的性能并优于以前的抽取式摘要模型。
Abstract
The goal of text summarization is to summarize long text into a short text with main information. To avoid the redundant information brought by the sentence extraction, we propose an extractive summarization model based on a heterogeneous graph network of sub-sentence units, which effectively integrates different levels of language information such as words, entities, and sub-sentential units. Experiments on two large scale benchmark corpora (CNN/DM and NYT) demonstrate that our model yields ground-breaking performance and outperforms previous extractive summarizers.
关键词
子句 /
异构图 /
抽取式摘要
{{custom_keyword}} /
Key words
sub-sentential /
heterogeneous graph /
extractive summarization
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] CHENG J P,LAPATA M.Neural summarization by extracting sentences and words[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 484-494.
[2] MARTINS A F T, SMITH N A. Summarization with a joint model for sentence extraction and compression[C]//Proceedings of the Workshop on Integer Linear Programming for Natural Language Processing, 2009: 1-9.
[3] CARBONELL J, GOLDSTEIN J. The use of MMR, diversity-based reranking for reordering documents and producing summaries[C]//Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1998: 335-336.
[4] LARSON M. Automatic summarization[J]. Foundations and Trends in Information Retrieval, 2012, 5(3): 235-422.
[5] LEBANOFF L, SONG K, DERNONCOURT F, et al. Scoring sentence singletons and pairs for abstractive summarization[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.
[6] LIU Y, LAPATA M. Text summarization with pretrained encoders[C]//Proceedings of the EMNLP-IJCNLP, 2019: 3730-3740.
[7] ZHOU Q, WEI F, ZHOU M. At which level should we extract?: An empirical analysis on extractive document summarization[C]//Proceedings of the 28th International Conference on Computational Linguistics, 2020: 5617-5628.
[8] YUAN R, WANG Z, LI W. Fact-level extractive summarization with hierarchical graph mask on BERT[C]//Proceedings of the 28th International Conference on Computational Linguistics, 2020: 5629-5639.
[9] ERKAN G, RADEV D R. Lexrank: Graph-based lexical centrality as salience in text summarization[J]. Journal of Artificial Intelligence Research, 2004, 22: 457-479.
[10] MIHALCEA R, TARAU P. Textrank: Bringing order into text[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2004: 404-411.
[11] YASUNAGA M, ZHANG R, MEELU K, et al. Graph-based neural multi-document summarization[C]//Proceedings of the 21st Conference on Computational Natural Language Learning, 2017: 452-462.
[12] XU J, GAN Z, CHENG Y, et al. Discourse-aware neural extractive text summarization[C]//Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, 2020: 5021-5031.
[13] WANG D, LIU P, ZHENG Y, et al.Heterogeneous graph neural networks for extractive document summarization[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 6209-6219.
[14] JIA R, CAO Y, TANG H, et al. Neural extractive summarization with hierarchical attentive heterogeneous graph network[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020: 3622-3631.
[15] RUSH A M, CHOPRA S, WESTON J. A neural attention model for abstractive sentence summarization[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2015: 379-389.
[16] NARAYAN S, COHEN S B, LAPATA M. Ranking sentences for extractive summarization with reinforcement learning[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018: 1747-1759.
[17] JADHAV A, RAJAN V. Extractive Summarization with SWAP-NET: Sentences and words from alternating pointer networks[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 142-151.
[18] ARUMAE K, LIU F. Reinforced extractive summarization with question-focused rewards[C]//Proceedings of ACL, Student Research Workshop, 2018: 105-111.
[19] LIU F, FLANIGAN J, THOMSON S, et al. Toward abstractive summarization using semantic representations[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2015: 1077-1086.
[20] KONCEL KEDZIORSKI R, BEKAL D, LUAN Y, et al. Text generation from knowledge graphs with graph transformers[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 2284-2293.
[21] WILLIAMS A, DROZDOV A, BOWMAN S R. Do latent tree learning models identify meaningful structure in sentences?[J]. Transactions of the Association for Computational Linguistics, 2018, 6(1): 253-267.
[22] LIU Y, TITOV I, LAPATA M. Single document summarization as tree induction[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 1745-1755.
[23] FERNANDES P, ALLAMANIS M, BROCKSCHMIDT M. Structured neural summarization[C]//Proceedings of ICLR,2019: 1-18.
[24] YAO L, MAO C, LUO Y. Graph convolutional networks for text classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(01): 7370-7377.
[25] DING R, XIE P, ZHANG X, et al. A neural multi-digraph model for Chinese NER with gazetteers[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 1462-1467.
[26] LIU P, CHANG S, HUANG X, et al. Contextualized non-local neural networks for sequence learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(01): 6762-6769.
[27] VELIC^KOVIC′ P, CUCURULL G, CASANOVA A, et al. Graph Attention Networks[J]. arXiv E-prints, 2017: arXiv: 1710.10903.
[28] HERMANN K M, KOCISKY T, GREFENSTETTE E, et al. Teaching machines to read and comprehend[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems, 2015: 1693-1701.
[29] SEE A, LIU P J, MANNING C D. Get to the point: Summarization with pointer-generator networks[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017: 1073-1083.
[30] XU J, DURRETT G. Neural extractive text summarization with syntactic compression[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 3292-3303.
[31] LIN C Y, HOVY E. Automatic evaluation of summaries using n-gram co-occurrence statistics[C]//Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, 2003: 150-157.
[32] ZHANG X, WEI F, ZHOU M. HIBERT: Document level pre-training of hierarchical bidirectional transformers for document summarization[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 5059-5069.
[33] KINGMA D P, BA J. Adam: A method for stochastic optimization[C]//Proceedings of ICLR, 2015: 1-15.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(61976016,61976015,61876198)
{{custom_fund}}
PDF(1701 KB)
