中学数学术语抽取方法未考虑句子的依存结构信息,导致对句子的语义理解能力有限。此外,由于依赖依存结构信息的术语抽取方法存在分词或依存结构错误,导致术语抽取准确性和完整性不佳。为解决上述问题,该文提出一种基于依存结构学习的中学数学术语鲁棒抽取模型。模型利用预训练模型得到文本向量语义表示,并借助带有去噪注意力机制层的图神经网络和双向循环神经网络分别捕获文本的依存结构信息和上下文信息,进一步采用注意力机制融合文本结构信息和上下文信息以实现在融入依存结构信息的同时缓解错误分词或依存结构的影响。模型在自建的中学数学术语数据集上抽取精度P和F1值分别达到了83.82%、82.51%,相较于基准模型分别提升了2.21%、1.22%,表明该文所提方法能够鲁棒融合依存结构信息,从而提升中学数学术语抽取的精确性和完整性。
Abstract
Term extraction methods in middle school mathematics do not consider the dependency structure information of the sentence, which leads to a limited semantic understanding of the sentence. In addition, the term extraction method of dependency structure information exists some errors in dependency structures, resulting in poor accuracy and completeness. This paper proposes a robust extraction model for middle school mathematical terms based on dependency structure learning. The dependency structure information and context information of text is captured by the graph neural network model with a denoising attention mechanism layer and bidirectional recurrent neural network, respectively. The attention mechanism is further used to integrate text structure information and context information to integrate the dependency structure information while alleviate the influence of incorrect word segmentation or dependency structure. The proposed model achieves 83.82% precision and 82.51% and F1 value in the experiment, indicating 2.21% and 1.22% improvements compared with the baseline model, respectively.
关键词
术语抽取 /
依存结构 /
图神经网络
{{custom_keyword}} /
Key words
term extraction /
dependency structure /
graph neural network
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 周长青.术语工作原则与方法(续四)(ISO DIS704)[J].中国科技术语, 1999(4): 24-32.
[2] YUAN Y, GAO Y, ZHANG Y, et al. Cross-lingual terminology extraction for translation quality estimation[C]//Proceedings of the 11th International Conference on Language Resources and Evaluation, 2018: 3774-3780.
[3] LIS, LI J, SONG T, et al. A novel topic model for automatic term extraction[C]//Proceedings of the 36th international ACM SIGIR Conference on Research and Development in Information Retrieval, 2013: 885-888.
[4] RANI M, DHAR A, VYAS O. Semi-automatic terminology ontology learning based on topic modeling[J].Engineering Applications of Artificial Intelligence, 2017,63: 108-125.
[5] PAULHEIM H. Knowledge graph refinement: A survey of approaches and evaluation methods[J]. Semantic Web,2017,8(3): 489-508.
[6] 张雪,孙宏宇,辛东兴,等. 自动术语抽取研究综述[J]. 软件学报,2020,31(7): 2062-2094.
[7] 周璐. 基础教育数学教材语言研究[D]. 厦门: 厦门大学博士学位论文,2019.
[8] NPTOOL A. A detector of english noun phrases[C]//Proceedings of the Workshop on Very Large Corpora,1995:48-57.
[9] BOLSHAKOVA E, LOUKACHEVITCH N, NOKEL M. Topic models can improve domain term extraction[C]//Proceedings of the European Conference on Information Retrieval, 2013: 684-687.
[10] 李思良,许斌,杨玉基. DRTE:面向基础教育的术语抽取方法[J]. 中文信息学报,2018,32(3): 101-109.
[11] 刘宇瀚,刘常健,徐睿峰,等. 结合字形特征与迭代学习的金融领域命名实体识别[J]. 中文信息学报,2020,34(11): 74-83.
[12] ZHANG Y, YANG J. Chinese ner using lattice lstm[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 1554-1564.
[13] VASWANI A,SHAZEER N, PARMAR N,et al. Attention is all you need[C]//Proceedings of the 31st Conference on Neural Information Processing Systems, Long Beach,CA.USA, 2017:1-11.
[14] ZHENG D, ZHAO T, YANG J. Research on domain term extraction based on conditional random fields[C]//Proceedings of International Conference on Computer Processing of Oriental Languages. Springer, 2009: 290-296.
[15] 吴俊,程垚,郝瀚,等. 基于BERT嵌入BILSTM-CRF模型的中文专业术语抽取研究[J]. 情报学报,2020,39(4): 10.
[16] 翟羽佳,田静文,赵玥. 基于BERT-BiLSTM-CRF模型的算法术语抽取与创新演化路径构建研究[J]. 情报科学,2022,40(4): 8-16.
[17] 李韧,李童,杨建喜,等. 基于Transformer-BiLSTM-CRF的桥梁检测领域命名实体识别[J]. 中文信息学报,2021,35(4): 83-91.
[18] 华鑫. 基于序列标注模型的中学数学术语抽取方法研究[D]. 南昌: 江西师范大学硕士学位论文,2021.
[19] WU S, FEI H, REN Y, et al. Learn from syntax: Improving pair-wise aspect and opinion terms extraction with rich syntactic knowledge[C]//Proceedings of International Joint Conference on Artificial Intelligence,2021: 3957-3963.
[20] TIAN Y, CHEN G, SONG Y, et al. Dependency-driven relation extraction with attentive graph convolutional networks[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the Eleventh International Joint Conference on Natural Language Processing, 2021: 4458-4471.
[21] WU S, FEI H, REN Y, et al. High-order pair-wise aspect and opinion terms extraction with edge-enhanced syntactic graph convolution[J].IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 2396-2406.
[22] DEVLIN, JACOB, CHANG, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of Annual Conference of the North American Chapter of the Association for Computational Linguistics, 2019: 4171-4186.
[23] 王元. 数学大辞典[M]. 北京: 科学出版社,2010: 100-316.
[24] LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural architectures for named entity recognition[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016: 260-270.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(62266023);江西省教育厅研究生创新基金项目(YC2022-s348);江西省教育厅科学技术研究项目(GJJ210325,GJJ2200354)
{{custom_fund}}
PDF(6740 KB)
