1.State Key Laboratory of Digital Publishing Technology, Peking University Founder Group Co. LTD., Beijing 100871, China; 2.Wangxuan Institute of Computer Technology, Peking University, Beijing 100871, China; 3.Yanqi Lake Beijing Institute of Mathematical Science and Applications, Beijing 101408, China
Abstract：Due to the complexity of Chinese grammars and insufficient training data, Chinese grammar error diagnosis (CGED) is a challenging task without applicable approaches in practice. In this paper, we propose a CGED model, APM-CGED, with data augmentation, pre-trained language model and linguistic feature based multi-task learning. Data augmentation can effectively expand the training set, and pre-trained language models are rich in semantic information helpful to grammatical analysis. Meanwhile, the linguistic feature based multi-task learning enables the language model to learn linguistic features useful for grammatical error diagnosis. The method proposed in this paper get better result on the CGED dataset than other compared models.
[1] Fur, Peiz, Gong J, et al. Chinese grammatical error diagnosis using statistical and prior knowledge driven features with probabilistic ensemble enhancement[C]//Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications, NLP-TEA@ACL, 2018: 52-59. [2] Chris B, Dolan W B, Gamon M.Correcting ESL errors using phrasal SMT techniques[C]//Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, ACL, 2006: 249-256. [3] Zheng Y, Briscoe T. Grammatical error correction using neural machine translation[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016: 380-386. [4] Zhou J, Li C, Liu H, et al.Chinese grammatical error correction using statistical and neural models[C]//Proceedings of Natural Language Processing and Chinese Computing-7th CCF International Conference, NLPCC, 2018: 117-128. [5] Chang R Y, Wu C H, Prasetyop K. Error diagnosis of Chinese sentences using inductive learning algorithm and decomposition-based testing mechanism[J]. ACM Transactions on Asian Language Information Processing, 2012(3): 24. [6] Lee L H, Chang L P, Lee K C, et al.Linguistic rules based Chinese error detection for second language learning[C]//Proceedings of the 21st International Conference on Computers in Education, 2013: 27-29. [7] Yang Y, Xie P, Tao J, et al.Embedding grammatical features into LSTMs for Chinese grammatical error diagnosis task[C]//Proceedings of the IJCNLP, Shared Tasks, 2017: 41-46. [8] Li C, Zhou J, Bao Z, et al.A hybrid system for Chinese grammatical error diagnosis and correction[C]//Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications, NLP-TEA@ACL, 2018: 60-69. [9] Zhao J, Li S, Lin Z.Contextualized character representation for Chinese grammatical error diagnosis[C]//Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications, NLP-TEA@ACL, 2018: 172-179. [10] Li C, Qi J.Chinese grammatical error diagnosis based on policy gradient LSTM model[C]//Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications, NLP-TEA@ACL, 2018: 77-82. [11] Wang C, Yang L, Wang Y, et al. Chinese grammatical error correction method based on transformer enhanced architecture[J]. Journal of Chinese Information Processing, 2020, 34(6): 106-114. [12] Zhang Y, Hu Q, Liu F, et al.CMMC-BDRC solution to the NLP-TEA-2018 Chinese grammatical error diagnosis task[C]//Proceedings of the 5th Workshop on Natural Language Proceeding Techniques for Educational Applications, NLP-TEA@ACL, 2018: 180-187. [13] Bell S, Yannakoudakis H, Rei M. Context is key: Grammatical error detection with contextual word representations[C]//Proceedings of the 14th Workshop on Innovative Use of NLP for Building Educational Applications, BEA@ACL,2019: 103-115. [14] Kaneko M, Komach I M. Multi-head multi-layer attention to deep language representations for grammatical error detection[J]. Computacion Sistemas 2019,23(3): 883-891. [15] Devlin J, Chang M W, Lee K, et al.BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,2019: 4171-4186. [16] Anon.pyltp: the python extension for LTP. [CP/OL].https://github.com/HIT-SCIR/pyltp[2020-5-21]. [17] Suttonc A, Mccallum A.An introduction to conditional random fields[J]. Foundations and Trends in Machine Learning 2012,4(4): 267-373. [18] Rao G, Gong Q, Zhang B, et al.Overview of NLPTEA-2018 share task Chinese grammatical error diagnosis[C]//Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications, NLP-TEA@ACL, 2018: 42-51.
2022, Vol. 36 
