问题生成是无监督常识问答模型的一个核心子任务,目前的方法主要是根据给定知识生成问题和答案,并为每个问题随机生成多个干扰项,然而这些方法存在干扰项与问题相关性不强且随机性较大的问题。该文提出一种面向干扰项增强的无监督常识问答模型,首先根据知识三元组生成问题和正确答案,再为问题建立对应的问题子图,得到与问题相关的三元组集合,使用注意力机制增强特征并根据问题和正确答案确定干扰项,最后使用生成的数据对模型进行训练。该模型在四个不同类型的测试任务上的结果表明,该模型优于目前的最新方法,证明了该模型的有效性。
Abstract
In contrast to the popular issue of question generation in unsupervised commonsense question-answering, this paper proposes an unsupervised commonsense question-answering model via distractor options enhancement. In our method, questions and correct answers are first generated according to the knowledge triples. Then the corresponding question subgraph is established for each question to obtain the knowledge triples related to the question. The attention mechanism is used to decide the distractors according to the questions and correct answers. Finally, the generated data with enhanced distractors are used to train the question-answering model. The experiment results show the model is superior to the latest methods on four different types of tasks.
关键词
干扰项增强 /
问题子图 /
注意力机制
{{custom_keyword}} /
Key words
negative samples enhancement /
problem subgraph /
attention mechanism
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] LEWIS P, DENOYER L, RIEDEL S. Unsupervised question answering by cloze translation[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics, 2019: 4896-4910.
[2] SHWARTZ V, WEST P, BRAS R L, et al. Unsupervised commonsense question answering with self-talk[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020: 4615-4629.
[3] FABBRI A, NG P, WANG Z, et al. Template-based question generation from retrieved sentences for improved unsupervised question answering[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 4508-4513.
[4] 谭红叶,孙秀琴,闫真.基于答案及其上下文信息的问题生成模型[J].中文信息学报,2020,34(05): 74-81.
[5] 武恺莉,朱朦朦,朱鸿雨,等.结合问题类型及惩罚机制的问题生成[J].中文信息学报, 2021, 35(04): 110-119.
[6] MA K, ILIEVSKI F, FRANCIS J,et al. Knowledge-driven data construction for zero-shot evaluation in commonsense question answering[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2021: 13507-13515.
[7] TALMOR A, HERZIG J, LOURIE N,et al. CommonsenseQA: A question answering challenge targeting commonsense knowledge[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis, Minnesota: Association for Computational Linguistics, 2019: 4149-4158.
[8] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics. Minneapolis, Minnesota: Association for Computational Linguistics, 2019: 4171-4186.
[9] LIU Y, OTT M, GOYAL N, et al. RoBERTa: A robustly optimized bert pretraining approach[EB/OL]https: //arxiv.org/pdf/1907.11692.pdf[2019-12-15].
[10] MA K, FRANCIS J, LU Q, et al.Towards generalizable neuro-symbolic systems for commonsense question answering[C]//Proceedings of the 1st Workshop on Commonsense Inference in Natural Language Processing. Hong Kong, China, 2019: 22-32.
[11] SPEER R, CHIN J, HAVASI C. ConceptNet 5.5: An open multilingual graph of general knowledge[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence. San Francisco, California, 2017: 4444-4451.
[12] SAP M, BRAS R L, ALLAWAY E,et al. ATOMIC: An atlas of machine commonsense for if-then reasoning[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2019: 3027-3035.
[13] PETERS M E, NEUMANN M L, et al. Knowledge enhanced contextual word representations[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Hong Kong, China, 2019: 43-54.
[14] MILLER G A. WordNet: A lexical database for English[J]. Communications of the ACM, 1995, 38(11): 39-41.
[15] VRANDECIC D, KRTZSCH M. Wikidata: A free collaborative knowledgebase[J]. Communications of the ACM, 2014, 57(10): 78-85.
[16] ILIEVSKI F, SZEKELY P, CHENG J,et al. Consolidating commonsense knowledge[J/OL]. arXiv preprint arXiv: 2006.06114, 2020.
[17] CHEN W, AIST G, MOSTOW J. Generating questions automatically from informational text[C]//Proceedings of the 2nd Workshop on Question Generation, 2009: 17-24.
[18] DU X, SHAO J, CARDIE C. Learning to ask: Neural question generation for reading comprehension[C]//Proceedings of the 55 Annual Meeting of the Association for Computational Linguistics, 2017: 1342-1352.
[19] BANERJEE P, BARAL C. Self-supervised knowledge triplet learning for zero-shot question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Online: Association for Computational Linguistics, 2020: 151-162.
[20] YE Z X, CHEN Q, WANG W, et al. Align, mask and select: A simple method for incorporating commonsense knowledge into language representation models[J/OL]. arXiv preprint arXiv: 1908.06725, 2019.
[21] YANG Y, MALAVIYA C, FERNANDEZ J, et al.Generative data augmentation for commonsense reasoning[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Online: Association for Computational Linguistics, 2020: 1008-1025.
[22] KRISHNA R, ZHU Y, GROTH O, et al. Visual genome: Connecting language and version using crowd sourced dense image annotations[J]. International Journal of Computer Vision, 2017, 123(1): 32-73.
[23] MURPHY M L. Semantic relations and the lexicon: Antonymy, synonymy and other paradigms[M]. Cambridge University Press, 2003.
[24] BHAGAVATULA C, BRAS R L, MALAVIYA C,et al. Abductive commonsense reasoning[J/OL] arXiv preprint arXiv: 1908.05739, 2019.
[25] BISK Y, ZELLERS R, BRAS R L,et al. PIQA: Reasoning about physical commonsense in natural language[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020: 7432-7439.
[26] SAKAGUCHI K, BRAS R L, BHAGAVATULA C,et al. WinoGrande: An adversarial winograd schema challenge at scale[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020: 8732-8740.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家社会科学基金(17XXW005);重庆理工大学研究生创新项目(clgycx203115)
{{custom_fund}}
PDF(3736 KB)
