基于代表性答案选择与注意力机制的短答案自动评分

PDF(2896 KB)

中文信息学报 ›› 2019, Vol. 33 ›› Issue (11) : 134-142.

信息检索与问答系统

基于代表性答案选择与注意力机制的短答案自动评分

谭红叶¹,午泽鹏¹,卢宇^2,3,段庆龙²,李茹¹,张虎¹

作者信息 +

Using Representative Answers and Attentions for Short Answer Grading

TAN Hongye¹, WU Zepeng¹, LU Yu^2,3, DUAN Qinglong², LI Ru¹, ZHANG Hu¹

Author information +

History +

摘要

短答案自动评分是智慧教学中的一个关键问题。目前自动评分不准确的主要原因是: (1)预先给定的参考答案不能覆盖多样化的学生答题情况; (2)不能准确刻画学生答案与参考答案匹配情况。针对上述问题,该文采用基于聚类与最大相似度方法选择代表性学生答案构建更完备的参考答案,尽可能覆盖学生不同的答题情况;在此基础上,利用基于注意力机制的深度神经网络模型来提升系统对学生答案与参考答案匹配情况的刻画。相关数据集上的实验结果表明: 该文模型有效提升了自动评分的准确率。

Abstract

Automatic short answer grading (ASAG) is a key issue in intelligent tutoring systems. The main challenges in ASAG lie in 1) the reference answer for a given question cannot cover the diverse student answers; and 2) the similarity between student answer and the reference is hard to estimate. This paper applies clustering and maximum similarity to select representative answers, constructing the reference answer set to cover various student answers. Then, this paper employs a deep neural network model based on the attention mechanism to approximate the similarity between the student answer and the reference answer set. Experimental results show that the proposed model effectively improves the accuracy of automatic scoring.

导出引用

谭红叶,午泽鹏,卢宇,段庆龙,李茹,张虎. 基于代表性答案选择与注意力机制的短答案自动评分. 中文信息学报. 2019, 33(11): 134-142

TAN Hongye, WU Zepeng, LU Yu, DUAN Qinglong, LI Ru, ZHANG Hu. Using Representative Answers and Attentions for Short Answer Grading. Journal of Chinese Information Processing. 2019, 33(11): 134-142

参考文献

[1] 郑庆华,董博,钱步月等,智慧教育研究现状与发展趋势[J].计算机研究与发展,2019,56(1): 209-224.
[2] Madnani N,Loukina A.Cahill A. A large scale quantitative exploration of modeling strategies for content scoring[C]//Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications,2017: 457-467.
[3] Page E B. The imminence of grading essays by computer[J]. Phi Delta Kappan,1966,47(5): 238-243.
[4] Bachman L F,Carr N,Kamei G,et al. A reliable approach to automatic assessment of short answer free responses[C]//Proceedings of the 19th International Conference on Computational Linguistics,2002: 1-4.
[5] Tom M,Nicola A,Peter B. Computerized marking of short-answer free-text responses[C]//Proceedings of the 29th Annual Conference of the International Association for Educational Assessment,2003.
[6] Pulman S,Sukkarieh J. Automatic short answer marking[C]//Proceedings of the 2nd Workshop on Building Educational Applications Using Natural Language Processing,2008: 29.
[7] Sultan M A,Salazar C,Sumner T. Fast and easy short answer grading with high accuracy[C]//Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,2016: 1070-1075.
[8] Saha S,Dhamecha T,Marvaniya S,et al. Sentence level or token level features for automatic short answer grading?: use both[C]//Proceedings of Artificial Intelligence in Education,2018: 503-517.
[9] Bailey S,Meurers D. Diagnosing meaning errors in short answers to reading comprehension questions[C]//Proceedings of the 3rd Workshop on Innovative Use of NLP for Building Educational Applications at ACL,2008: 107-115.
[10] Zhang Y,Shah R,ChiM. Deep learning + student modeling + clustering: a recipe for effective automatic short answer grading[C]//Proceedings of the 9th International Conference on Educational Data Mining,2016: 562-567.
[11] Riordan B,Horbach A,Cahill A,Zesch T,et al. Investigating neural architectures for short answer scoring[C]//Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications,2017: 159-168.
[12] Marvaniya S,Saha S,Dhamecha T,et al. Creating scoring rubric from representative student answers for improved short answer grading[C]//Proceedings of Conference on Information and Knowledge Management,2018: 993-1002.
[13] Lan A,Vats D,Waters A,et al. Mathematical language processing: automatic grading and feedback for open response mathematical questions[C]//Proceedings of the 2nd ACM Corfernce on Learning at Scale,2015: 167-176.
[14] Bahdanau D,Cho K,Bengio Y. Neural machine translation by jointly learning to align and translate[J]. arXiv preprint arXiv: 1409.0473,2014.
[15] Chaudhari S,Polatkan G,Ramanath R,et al. An attentive survey of attention models[J]. arXiv preprint arXiv: 1904.02874,2019.
[16] Kim Y.Convolutional neural networks for sentence classification[J]. arXiv preprint arXiv: 1408.5802,2014.
[17] Kingma D,Ba J. Adam. A Method for Stochastic Optimization[C]//Proceedings of the 3rd International Conference for Learning Representations,San Diego,2015.
[18] Boer P,Kroese D,Mannor S,et al. A tutorial on the cross-entropy method[J]. Annals of Operations Research,2005,134(1): 19-67.
[19] Huang Y,Yang X,Zhuang F,et al. Automatic chinese reading comprehension grading by lstm with knowledge adaptation[C]//Proceedings of Pacific-asia Conference on Knowledge Discovery & Data Mining,2018: 118-129.
[20] 彭敏,姚亚兰,谢倩倩,等. 基于带注意力机制CNN的联合知识表示模型[J]. 中文信息学报,2019,33(2): 51-58.
[21] 谭咏梅,刘姝雯,吕学强. 基于CNN与双向LSTM的中文文本蕴含识别方法[J]. 中文信息学报,2018,32(7): 11-1.
[22] 包乌格德勒,赵小兵. 基于RNN和CNN的蒙汉神经机器翻译研究[J]. 中文信息学报,2018,32(8): 60-67.
[23] 沈龙骧,邹博伟,叶静,等. 基于双向LSTM与CRF融合模型的否定聚焦点识别[J]. 中文信息学报,2019,33(1): 25-34.
[24] 彭敏,杨绍雄,朱佳晖. 基于双向LSTM语义强化的主题建模[J]. 中文信息学报,2018,32(4): 44-49.
[25] Yin W,Kann K,Yu M,et al. Comparative study of CNN and RNN for natural language processing[J]. arXiv preprint arXiv: 1702.01923,2017.

基金

国家自然科学基金(61673248,61772324);国家社会科学基金(18BYY074)

PDF(2896 KB)

671

Accesses

Citation

Detail

段落导航

摘要
Abstract
关键词
Key words
引用本文
参考文献
基金

Received	Published
2019-05-14	2019-11-11
Issue Date
2019-11-11

选择文件类型/文献管理软件名称

选择包含的内容

摘要

Abstract

关键词

Key words

引用本文

{{custom_sec.title}}

{{custom_sec.title}}

参考文献

{{custom_fnGroup.title_cn}}

脚注

基金