A Chinese Grammatical Error Correction Model Based on Encoder-decoder Architecture

Grammatical error correction ensures language standardization and improves text data quality. Using the open-source T5 of encoder-decoder architecture, we build the Chinese Grammar Error Correction (CGEC) model CGEC-T5 via layer expansion and parameter reuse. By injecting noise, we automatically generate large-scale training samples of different difficulties for the model, enhancing its generalization ability. We adopt a multi-stage training strategy, using 128 A100 GPUs for full-parameter learning on 200 billion tokens. Experimental results show that CGEC-T5 achieves the state-of-the-art performance on public datasets NLPCC18 and FCGEC.