由于中文语法的复杂性,中文语法错误检测(CGED)的难度较大,而训练语料和相关研究的缺乏,使得CGED的效果还远未达到实用的程度。该文提出一种CGED模型,APM-CGED,采用数据增强、预训练语言模型和基于语言学特征多任务学习的方式,弥补训练语料的不足。数据增强能够有效地扩充训练集,而预训练语言模型蕴含丰富的语义信息又有助于语法分析,基于语言学特征多任务学习对语言模型进行优化则可以使语言模型学习到跟语法错误检测相关的语言学特征。该文提出的方法在NLPTEA的CGED数据集进行测试,取得了优于其他对比模型的结果。
Abstract
Due to the complexity of Chinese grammars and insufficient training data, Chinese grammar error diagnosis (CGED) is a challenging task without applicable approaches in practice. In this paper, we propose a CGED model, APM-CGED, with data augmentation, pre-trained language model and linguistic feature based multi-task learning. Data augmentation can effectively expand the training set, and pre-trained language models are rich in semantic information helpful to grammatical analysis. Meanwhile, the linguistic feature based multi-task learning enables the language model to learn linguistic features useful for grammatical error diagnosis. The method proposed in this paper get better result on the CGED dataset than other compared models.
关键词
中文语法错误检测 /
数据增强 /
多任务学习
{{custom_keyword}} /
Key words
Chinese grammar error detection /
data enhancement /
multi-task learning
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Fur, Peiz, Gong J, et al. Chinese grammatical error diagnosis using statistical and prior knowledge driven features with probabilistic ensemble enhancement[C]//Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications, NLP-TEA@ACL, 2018: 52-59.
[2] Chris B, Dolan W B, Gamon M.Correcting ESL errors using phrasal SMT techniques[C]//Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, ACL, 2006: 249-256.
[3] Zheng Y, Briscoe T. Grammatical error correction using neural machine translation[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016: 380-386.
[4] Zhou J, Li C, Liu H, et al.Chinese grammatical error correction using statistical and neural models[C]//Proceedings of Natural Language Processing and Chinese Computing-7th CCF International Conference, NLPCC, 2018: 117-128.
[5] Chang R Y, Wu C H, Prasetyop K. Error diagnosis of Chinese sentences using inductive learning algorithm and decomposition-based testing mechanism[J]. ACM Transactions on Asian Language Information Processing, 2012(3): 24.
[6] Lee L H, Chang L P, Lee K C, et al.Linguistic rules based Chinese error detection for second language learning[C]//Proceedings of the 21st International Conference on Computers in Education, 2013: 27-29.
[7] Yang Y, Xie P, Tao J, et al.Embedding grammatical features into LSTMs for Chinese grammatical error diagnosis task[C]//Proceedings of the IJCNLP, Shared Tasks, 2017: 41-46.
[8] Li C, Zhou J, Bao Z, et al.A hybrid system for Chinese grammatical error diagnosis and correction[C]//Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications, NLP-TEA@ACL, 2018: 60-69.
[9] Zhao J, Li S, Lin Z.Contextualized character representation for Chinese grammatical error diagnosis[C]//Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications, NLP-TEA@ACL, 2018: 172-179.
[10] Li C, Qi J.Chinese grammatical error diagnosis based on policy gradient LSTM model[C]//Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications, NLP-TEA@ACL, 2018: 77-82.
[11] Wang C, Yang L, Wang Y, et al. Chinese grammatical error correction method based on transformer enhanced architecture[J]. Journal of Chinese Information Processing, 2020, 34(6): 106-114.
[12] Zhang Y, Hu Q, Liu F, et al.CMMC-BDRC solution to the NLP-TEA-2018 Chinese grammatical error diagnosis task[C]//Proceedings of the 5th Workshop on Natural Language Proceeding Techniques for Educational Applications, NLP-TEA@ACL, 2018: 180-187.
[13] Bell S, Yannakoudakis H, Rei M. Context is key: Grammatical error detection with contextual word representations[C]//Proceedings of the 14th Workshop on Innovative Use of NLP for Building Educational Applications, BEA@ACL,2019: 103-115.
[14] Kaneko M, Komach I M. Multi-head multi-layer attention to deep language representations for grammatical error detection[J]. Computacion Sistemas 2019,23(3): 883-891.
[15] Devlin J, Chang M W, Lee K, et al.BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,2019: 4171-4186.
[16] Anon.pyltp: the python extension for LTP. [CP/OL].https://github.com/HIT-SCIR/pyltp[2020-5-21].
[17] Suttonc A, Mccallum A.An introduction to conditional random fields[J]. Foundations and Trends in Machine Learning 2012,4(4): 267-373.
[18] Rao G, Gong Q, Zhang B, et al.Overview of NLPTEA-2018 share task Chinese grammatical error diagnosis[C]//Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications, NLP-TEA@ACL, 2018: 42-51.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家重点研发计划(2019YFB1406302);国家自然科学基金(61573028,61432020);北京市自然科学基金(4142023);北京新星计划项目(XX2015B010)
{{custom_fund}}
PDF(4967 KB)
