段落级问题生成是指从给定的段落中生成一个或多个与之相关的问题。目前的研究主要使用序列到序列的神经网络最大程度地利用段落信息,但这种方法存在冗余信息干扰、无法聚焦重点句子的问题。针对上述问题,该文提出了一种基于双注意力的段落级问题生成模型。该模型首先对段落和答案所在句子分别使用注意力机制,然后利用门控机制动态地分配权重并融合上下文信息,最后利用改进的指针生成网络结合上下文向量和注意力分布来生成问题。实验结果表明,该模型在SQuAD数据集上比现有主流模型具有更高的性能。
Abstract
Paragraph-level question generation is to generate one or more related questions from a given paragraph. Current studies on sequence-to-sequence based neural networks fail to filter redundant information or focus on key sentences. To solve this issue, this paper proposes a dual attention based model for paragraph-level question generation. The model first uses the attention mechanism for the paragraph and the sentence where the answer is located. Then, the model uses the gating mechanism to dynamically assign weights and merges them into context information. Finally, it improves pointer-generator network to combine the context vector and attention distribution to generate questions. Experimental results show that this model has a better performance than exiting models on the SQuAD dataset.
关键词
问题生成 /
双注意力 /
指针生成网络
{{custom_keyword}} /
Key words
question generation /
dual attention /
pointer-generator network
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Rus V, Wyse B, Piwek P, et al. The first question generation shared task evaluation challenge[C]//Proceedings of the 6th International Natural Language Generation Conference, 2010: 251-257.
[2] Heilman M, Smith N A. Good question! Statistical ranking for question generation[C]//Proceedings of Human Language Technologies: The Annual Conference of the North American Chapter of the Association for Computational Linguistics, 2010: 609-617.
[3] Pan L, Lei W, Chua T S, et al. Recent advances in neural question generation[J]. arXiv preprint arXiv:1905.08949, 2019.
[4] Liu M, Calvo R A, Rus V. Automatic question generation for literature review writing support[C]//Proceedings of the International Conference on Intelligent Tutoring Systems, 2010: 45-54.
[5] Sutskever I, Vinyals O, Le Q V. Sequence to sequence learning with neural networks[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems, 2014: 3104-3112.
[6] Du X, Shao J, Cardie C. Learning to ask: Neural question generation for reading comprehension[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017: 1342-1352.
[7] Rajpurkar P, Zhang J, Lopyrev K, et al. SQuAD: 100,000+ questions for machine comprehension of text[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2016: 2383-2392.
[8] Zhou Q, Yang N, Wei F, et al. Neural question generation from text: A preliminary study[C]//Proceedings of National CCF Conference on Natural Language Processing and Chinese Computing, 2017: 662-671.
[9] Song L, Wang Z, Hamza W, et al. Leveraging context information for natural question generation[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, 2018: 569-574.
[10] Du X, Cardie C. Harvesting paragraph-level question-answer pairs from Wikipedia[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 1907-1917.
[11] Zhao Y, Ni X, Ding Y, et al. Paragraph-level neural question generation with maxout pointer and gated self-attention networks[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 3901-3910.
[12] Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[13] See A, Liu P J, Manning C D. Get to the point: Summarization with pointer-generator networks[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017: 1073-1083.
[14] Labutov I, Basu S, Vanderwende L. Deep questions without deep understanding[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, 2015: 889-898.
[15] 董孝政,洪宇,朱芬红,等. 基于密令位置信息特征的问题生成[J]. 中文信息学报, 2019, 33(8): 93-100.
[16] 谭红叶,孙秀琴,闫真. 基于答案及其上下文信息的问题生成模型[J]. 中文信息学报, 2020, 34(5): 74-81.
[17] Sachan M, Xing E. Self-training for jointly learning to ask and answer questions[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistic, 2018: 629-640.
[18] Zhang S, Bansal M. Addressing semantic drift in question generation for semi-supervised question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 2495-2509.
[19] Tuan L A, Shah D J, Barzilay R. Capturing greater context for question generation[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020: 9065-9072.
[20] Gu J, Lu Z, Li H, et al. Incorporating copying mechanism in sequence-to-sequence learning[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 1631-1640.
[21] Pennington J, Socher R, Manning C. GloVe: Global vectors for word representation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014: 1532-1543.
[22] Luong T, Pham H, Manning C D. Effective approaches to attention-based neural machine translation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2015: 1412-1421.
[23] Papineni K, Roukos S, Ward T, et al. BLEU: A method for automatic evaluation of machine translation[C]//Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, 2002: 311-318.
[24] Denkowski M, Lavie A. Meteo runiversal: Language specific translation evaluation for any target language[C]//Proceedings of the 9th Workshop on Statistical Machine Translation, 2014: 376-380.
[25] Lin C Y. ROUGE: A package for automatic evaluation of summaries[C]//Proceedings of ACL Workshop Text Summarization Branches Out, 2004: 74-81.
[26] Kim Y, Shin J, Jung K. Improving neural question generation using answer separation[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2019: 6602-6609.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(62076103);广东省普通高校人工智能重点领域专项(2019KZDZX1033);广东省信息物理融合系统重点实验室(2020B1212060069);广东省基础与应用基础研究基金项目(2021A1515011171)
{{custom_fund}}
PDF(2979 KB)
