Abstract：Keyphrase Generation (KG) is the task of capturing themes from a document, revealing the key information necessary to understand the content. Existing neural keyphrase generation approaches focus only on the token-level information while ignore sentence-level information such as document structure. In this paper, we incorporate the sentence-level inductive bias into KG and propose a new method named Sentence Selective Network (SenSeNet), which can automatically learn the sentence-level information and determine whether the sentence more likely to generate the keyphrase. We use straight-through estimator to train the model in an end-to-end manner and incorporate a weakly-supervised setting which is helpful for the training of the sentence selection module. Experiments show that our model successfully captures the document structure and reasonably distinguishes the significance of sentences, and consistent improvements achieved on two metrics in five datasets.
[1] Hulth A, Megyesi B. A study on automatically extracted keywords in text categorization[C]//Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, 2006: 537-544. [2] Wang L, Cardie C. Domain-independent abstract generation for focused meeting summarization[C]//Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, 2013: 1395-1405. [3] 马亮,何婷婷,李芳,等. 以关键词抽取为核心的文摘句选择策略[J]. 中文信息学报, 2008, 22(6): 50-54. [4] Berend G. Opinion expression mining by exploiting keyphrase extraction[C]//Proceedings of the IJCNLP, 2011. [5] Liu Z, Huang W, Zheng Y, et al. Automatic keyphrase extraction via topic decomposition[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2010: 366-376. [6] Meng R, Zhao S, Han S, et al. Deep keyphrase generation[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017: 582-592. [7] Luong M T, Pham H, Manning C D. Effective approaches to attention-based neural machine translation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2015: 1412-1421. [8] Gu J, Lu Z, Li H, et al. Incorporating copying mechanism in sequence-to-sequence learning[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 1631-1640. [9] Chen J, Zhang X, Wu Y, et al. Keyphrase generation with correlation constraints[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 4057-4066. [10] Tu Z, Lu Z, Liu Y, et al. Modeling coverage for neural machine translation[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 76-85. [11] Chen W, Gao Y, Zhang J, et al. Guided encoding for keyphrase generation[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33: 6268-6275. [12] Chan H P, Chen W, Wang L, et al. Neural keyphrase generation via reinforcement learning with adaptive rewards[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2163-2174. [13] Bengio Y, Léonard N, Courville A. Estimating or propagating gradients through stochastic neurons for conditional computation[J]. arXiv preprint arXiv: 1308.3432, 2013. [14] 周宁,石雯茜,朱昭昭. 基于粗糙数据推理的TextRank关键词提取算法[J]. 中文信息学报, 2020, 34(9): 44-52. [15] 索红光,刘玉树,曹淑英. 一种基于词汇链的关键词抽取方法[J]. 中文信息学报, 2006, 20(6): 27-32. [16] Wang M, Zhao B, Huang Y. PTR: Phrase-based topical ranking for automatic keyphrase extraction in scientific publications[C]//Proceedings of the International Conference on Neural Information Processing. Springer, Cham, 2016: 120-128. [17] Hulth A. Improved automatic keyword extraction given more linguistic knowledge[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2003: 216-223. [18] Nguyen T D, Kan M Y. Keyphrase extraction in scientific publications[C]//Proceedings of the International Conference on Asian Digital Libraries. Springer, Berlin, Heidelberg, 2007: 317-326. [19] Mihalcea R, Tarau P. Textrank: Bringing order into text[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2004: 404-411. [20] Sutskever I, Vinyals O, Le Q V. Sequence to sequence learning with neural networks[C]//Proceedings ofAdvances in Neural Information Processing Systems, 2014: 3104-3112. [21] Yuan X, Wang T, Meng R, et al. One size does not fit all: Generating and evaluating variable number of keyphrases[J]. arXiv preprint arXiv: 1810.05241, 2018. [22] Cho K, Merrinboer B, Gulcehre C, et al. Learning Phrase Representations using RNN Encoder-Decoderfor Statistical Machine Translation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 1724-1734. [23] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of Advances in Neural Information Processing Systems, 2017: 5998-6008. [24] Kim Y. Convolutional neural networks for sentence classification[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014: 1746-1751. [25] Courbariaux M, Hubara I, Soudry D, et al. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1[J]. arXiv preprint arXiv: 1602.02830, 2016. [26] Zhou Z H. A brief introduction to weakly supervised learning[J]. National Science Review, 2018, 5(1): 44-53. [27] Mikalai Krapivin, Aliaksandr Autayeu, Maurizio Marchese. Large dataset for keyphrases extraction,DISI-09-055[R/OL]. http://eprints.biblio.unitn.it/1671/1/disi09055-krapivin-autayeu-marchese.pdf.2008. [28] Kim S N, Medelyan O, Kan M Y, et al. Semeval-2010 task 5: Automatic keyphrase extraction from scientific articles[C]//Proceedings of the 5th International Workshop on Semantic Evaluation, 2010: 21-26.[29] Guo Q, Qiu X, Liu P, et al. Star-Transformer[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 1315-1325. [30] Yan H, Deng B, Li X, et al. Tener: Adapting transformer encoder for name entity recognition[J]. arXiv preprint arXiv: 1911.04474, 2019.
2021, Vol. 35 
