问题生成是指在理解特定陈述句语义的前提下,自动地生成一条或多条关于该陈述句的问题。该文主要针对其中一项子任务开展研究,即一对一的问题生成(Point-wise Question Generation,PQG)。现有PQG研究,主要以端到端的序列化生成模型为框架,相应方法生成的问句,在流畅度方面已达到有限的可接受度(BlEU-4约13%)。尽管如此,现有方法缺乏语块一级的注意力建模,从而无法将“潜在提问对象”的语义独立且整体地纳入表示学习过程。这一不足往往负面影响解码端的问题类型预测和提问词估计。针对这一问题,该文提出了一种融合密令注意力机制的端对端PQG模型。其中,密令是对短语和语块一级的潜在答案的总体概括,其往往表现为陈述句中的一组连续的词项。在方法实现方面,该文在端对端架构的编码过程中,将密令的位置信息与全句语义信息进行融合,而在解码过程中,则加强了针对密令的注意力。实验采用SQuAD语料予以实施,测试结果显示,该文所提方法的性能优于现有主流模型,其获得的BLEU-4指标高于基准系统1.98%。
Abstract
The question generation task aims to automatically generate one or more questions on the condition of understanding the semantics of a declarative sentence. This paper focuses on one of the sub-tasks, Point-wise Question Generation (PQG), and proposes a seq2seq PGQ model that combines attention mechanism about tokens. Among them, the token is a general summary of the potential answers for the sentences level, which is often shown as a series of consecutive terms in a declarative sentence. In terms of method implementation, the position information of the token and the semantic information of the whole sentence are integrated in the process of encoding. While in the process of decoding, the attention of token is strengthened. The experiment is carried out on the SQuAD corpus, revealing a better performance of 1.98% improvement in BLEU-4.
关键词
问题生成 /
密令 /
端到端
{{custom_keyword}} /
Key words
question generation /
token /
seq2seq
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Rus V,Wyse B,Piwek P,et al.The first question generation shared task evaluation challenge[C]//Proceedings of the 6th International Natural Language Generation Conference.Association for Computational Linguistics,2010: 251-257.
[2] Zhou Q,Yang N,Wei F,et al.Neural question generation from text: A preliminary study[C]//Proceedings of National CCF Conference on Natural Language Processing and Chinese Computing.Springer,Cham,2017: 662-671.
[3] 郑实福,刘挺,秦兵,等.自动问答综述[J].中文信息学报,2002,16(6):46-52.
[4] 李舟军,李水华.基于Web的问答系统综述[J].计算机科学,2017,44(6):1-7.
[5] 付文博,孙涛,梁藉,等.深度学习原理及应用综述[J].计算机科学,2018,45(6A):12-40.
[6] Hochreiter S,Schmidhuber J.Long short-term memory[J].Neural Computation,1997,9(8): 1735-1780.
[7] Kim Y.Convolutional neural networks forsentence classification[J].arXiv preprint arXiv:1408.5882,2014.
[8] Rajpurkar Pranav,Zhang Jian,Lopyrev Konstant et al.SQuAD:100 000+questions for machine comprehension of text[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:2383-2392.
[9] Heilman M,Smith N A.Good question! statistical ranking for question generation[C]//Proceedings of Human Language Technologies:The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics.Association for Computational Linguistics,2010:609-617.
[10] Kutner M H,Nachtsheim C,Neter J.Applied linear regression models[M].McGraw-Hill/Irwin,2004:4-318.
[11] Yao X.Generating more specific questions[M/OL].Question Generation,Papers from the 2011 {AAAI} Fall Symposium,Arlington,Virginia,USA,November 4-6,2011.[2011-11-04],http://www.aaai.org/Library/Symposia/Fall/fs11-04.php.
[12] Liu M,Rus V,Liu L.Automatic Chinese factual question generation[J].IEEE Transactions on Learning Technologies,2017,10(2): 194-204.
[13] Du X,Shao J,Cardie C.Learning to ask: Neural question generation for reading comprehension[J].arXiv preprint arXiv:1705.00106,2017.
[14] Duan N,Tang D,Chen P,et al.Question generation for question answering[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing,2017: 866-874.
[15] Greff K,Srivastava R K,Koutnkí J,et al.LSTM: A search space odyssey[J].IEEE Transactions on Neural Networks and Learning Systems,2017,28(10): 2222-2232.
[16] Pennington J,Socher R,Manning C.Glove: Global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP),2014: 1532-1543.
[17] Dong X,Hong Y,Chen X,et al.Neural question generation with semantics of question type[C]//Proceedings of CCF International Conference on Natural Language Processing and Chinese Computing.Springer,Cham,2018: 213-223.
[18] Papineni K,Roukos S,Ward T,et al.BLEU: A method for automatic evaluation of machine translation[C]//Proceedings of the 40th Annual Meeting on Association for Computational Linguistics.Association for Computational Linguistics,2002: 311-318.
[19] Denkowski M,Lavie A.Meteor universal: Language specific translation evaluation for any target language[C]//Proceedings of the 9th Workshop on Statistical Machine Translation.2014: 376-380.
[20] Lin C Y.Rouge:A package for automatic evaluation of summaries[M].Text Summarization Branches Out,2004:74-81.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(61672367,61672368,61773276)
{{custom_fund}}
PDF(1624 KB)
