基于可控解码策略的生成式生物医学事件抽取

苏方方,李霏,姬东鸿

PDF(4390 KB)
PDF(4390 KB)
中文信息学报 ›› 2023, Vol. 37 ›› Issue (11) : 68-80.
信息抽取与文本挖掘

基于可控解码策略的生成式生物医学事件抽取

  • 苏方方,李霏,姬东鸿
作者信息 +

Generative Biomedical Event Extraction Based on Controllable Decoding

  • SU Fangfang, LI Fei, JI Donghong
Author information +
History +

摘要

该文在预训练语言模型T5的框架基础上构建了一个生成式生物医学事件抽取模型,该方法可以自由定义输出序列,由此可以联合建模触发词识别、关系抽取和论元组合三个子任务。模型采用了生成序列字典树和事件类型-论元角色字典树,用于规范序列生成和减少论元角色的搜索空间。另外还采用可控解码策略便于限制每一步生成时所使用的候选词汇集,最后在训练时使用了课程学习,便于T5模型熟悉生物医学语料和有层次结构的完整事件的学习。该文模型在Genia 2011年和Genia 2013年的语料上分别获得了62.40% 和 54.85%的F1值,说明了使用生成式的方式进行生物医学事件抽取是可行的。

Abstract

This paper presents a generative biomedical event extraction model based on the framework of the pre-trained language model T5, which allows the joint modeling of the three subtasks of trigger recognition, relation extraction and argument combination. The model employs a trie-based constrained decoding algorithm, which regulates sequence generation and reduces the search space for argument roles. Finally, curriculum learning algorithm is used in training, which familiarizes T5 with biomedical corpora and events with hierarchical structure. The model obtains 62.40% F1-score on the Genia 2011 and 54.85% F1-score on the Genia 2013, respectively, demonstrating the feasibility of using a generative approach to biomedical event extraction.

关键词

生物医学事件抽取 / 生成式模型 / 可控解码策略

Key words

biomedical event extraction / generative models / controllable decoding algorithm

引用本文

导出引用
苏方方,李霏,姬东鸿. 基于可控解码策略的生成式生物医学事件抽取. 中文信息学报. 2023, 37(11): 68-80
SU Fangfang, LI Fei, JI Donghong. Generative Biomedical Event Extraction Based on Controllable Decoding. Journal of Chinese Information Processing. 2023, 37(11): 68-80

参考文献

[1] JIN DONG K, TOMOKO O, SAMPO P, et al.Overview of bionlp’09 shared task on event extraction[C]//Proceedings of the BioNLP Workshop Companion Volume for Shared Task, 2009: 1-9.
[2] SENDONG Z, CHANG S, ZHIYONG L, et al. Recent advances in biomedical literature mining[J]. Briefings in Bioinformatics, 2021, 22(3):bbaa057.
[3] SEBASTIAN R, ANDREW M. Fast and robust joint models for biomedical event extraction[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2011: 1-12.
[4] ANDREAS V, MARK C. Biomedical event extraction from abstracts and full papers using search-based structured prediction[C]//Proceedings of the BioNLP Shared Task Workshop, BioNLP Shared Task’11, 2011: 36-40.
[5] JARI B, JUHO H, FILIP G, et al. Extracting complex biological events with rich graph-based feature sets[C]//Proceedings of the BioNLP Workshop Companion Volume for Shared Task, 2009: 10-18.
[6] MAKOTO M, RUNE S, JIN-DONG K, et al. Event extraction with complex event classification using rich features[J]. Journal of Bioinformatics and Computational Biology, 2010, 8(01): 131-146.
[7] SEBASTIAN R, ANDREW M. Robust biomedical event extraction with dual decomposition and minimal domain adaptation[C]//Proceedings of BioNLP Shared Task Workshop, 2011: 46-50.
[8] DAVID M, SEBASTIAN R, MIHAI S, et al. Combining joint models for biomedical event extraction[J].BMC Bioinformatics, 2012, 13(1): 1-12.
[9] COLIN R, NOAM S, ADAM R, et al. Exploring the limits of transfer learning with a unified text-to-text transformer[J]. J. Mach. Learn. Res., 2020, 21(140): 1-67.
[10] MIKE L, YINHAN L, NAMAN G, et al. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 7871-7880.
[11] NICOLA D C, GAUTIER I, SEBASTIAN R, et al. Autoregressive entity retrieval[C]//Proceedings of the International Conference on Learning Representations, 2021.
[12] PINZHEN C, NIKOLAY B, KENNETH H, et al. Parallel sentence mining by constrained decoding[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 1672-1678.
[13] LIFU H, TAYLOR C, XIAOCHENG F, et al. Liberal event extraction and event schema induction[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 258-268.
[14] YOSHUA B, JéRME L, RONAN C, et al. Curriculum learning[C]//Proceedings of the 26th Annual International Conference on Machine Learning, 2009: 41-48.
[15] BENFENG X, LICHENG Z, ZHENDONG M, et al. Curriculum learning for natural language understanding[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 6095-6104.
[16] JIN DONG K, YUE W, TOSHIHISA T, et al. Overview of genia event task in bionlp shared task 2011[C]//Proceedings of BioNLP Shared Task Workshop, 2011: 7-15.
[17] JIN DONG K, YUE W, YAMAMOTO Y. The genia event extraction shared task, 2013 edition-overview[C]//Proceedings of the BioNLP Shared Task 2013 Workshop, 2013: 8-15.
[18] GIOVANNI P, BEN A, JASON K, et al. Structured prediction as translation between augmented natural languages[C]//Proceedings of the International Conference on Learning Representations, 2021.
[19] BEN A, CICERO NOGUEIRA DOS S, JASON K, et al. Augmented natural language for generative sequence labeling[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020: 375-385.
[20] DZMITRY B, KYUNGHYUN C, YOSHUA B. Neural machine translation by jointly learning to align and translate[C]//Proceedings of the International Conference on Learning Representations, 2015.
[21] ASHISH V, NOAM S, NIKI P, et al. Attention is all you need[C]//Proceedings of the 31st International Corference on Neural Information Processing Systems, 2017: 6000-6010.
[22] ILYA L, FRANK H. Decoupled weight decay regularization[C]//Proceedings of the International Conference on Learning Representations, 2019: 1-18.
[23] GUILLAUME L, MIGUEL B, SANDEEP S, et al. Neural architectures for named entity recognition[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016: 260-270.
[24] RRUBAA P, ARAVINDH A. Bidirectional lstm-crf for named entity recognition[C]//Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation, 2018: 531-540.
[25] MARKUS E, ADRIAN U. Span-based joint entity and relation extraction with transformer pre-training[J]. arXiv preprine arXiv 1909.07755, 2019.
[26] XING D W, LEON W, ULF L. Biomedical event extraction as multi-turn question answering[C]//Proceedings of the 11th International Workshop on Health Text Mining and Information Analysis, 2020: 88-96.
[27] KUNG-HSIANG H, MU Y, NANYUN P. Biomedical event extraction with hierarchical knowledge graphs[G]//Proceedings of the Association for Computational Linguistics: EMNLP, 2020: 1277-1285.
[28] WEIZHONG Z, JINYONG Z, JINCAI Y, et al. A novel joint biomedical event extraction framework via two-level modeling of documents[J]. Information Sciences, 2021: 550: 27-40.
[29] HAI LONG T, THY THY T, KHOA NA D, et al. Deep eventmine: End-to-end neural nested event extraction from biomedical texts[J]. Bioinformatics, 2020: 36(19): 4910-4917.
[30] DEEPAK V, CHEN C, VIBHAV G, et al. Relieving the computational bottleneck: Joint inference for event extraction with high-dimensional features[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014: 831-843.
[31] XIAOLEI M, YANG L, YINAN L, et al. Biomedical event extraction using a new error detection learning approach based on neural network[J]. Computers, Materials & Continua, 2020, 63(2): 923-941.
[32] JARI B, TAPIO S. Biomedical event extraction using convolutional neural networks and dependency parsing[C]//Proceedings of the BioNLP Workshop, 2018: 98-108.
[33] MAKOTO M, PAUL T, SOPHIA A. Boosting automatic event extraction from the literature using domain adaptation and coreference resolution[J]. Bioin-Formatics, 2012, 28(13): 1759-1765.
[34] AMIT M, ASIF E, SUDIP K N. Biomolecular event extraction using a stacked generalization based classifier[C]//Proceedings of the 13th International Conference on Natural Language Processing, 2016: 55-64.
[35] DIYA L, LIFU H, HENG J, et al. Biomedical event extraction based on knowledge-driven tree-LSTM[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 1421-1430.
[36] ALAN R, ROB VAN DER G, ROSARIO L, et al. Biomedical event extraction as sequence labeling[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020: 5357-5367.
[37] QUOC CHINH B, DAVID C, ERIK-VAN M, et al. A fast rule-based approach for biomedical event extraction[C]//Proceedings of the BioNLP Shared Task Workshop, 2013: 104-108.
[38] KAI H, SOFIE-VAN L, TAPIO S, et al. Evex inST’13: Application of a large-scale text mining resource to event extraction and network construction[C]//Proceedings of the BioNLP Shared Task Workshop, 2013: 26-34.
[39] JARI B, TAPIO S. Tees 2.2: Biomedical event extraction for diverse corpora[J]. BMC Bioinformatics, 2015, 16.
[40] SUDHA R, DANIEL M, KEVIN K, et al. Biomedical event extraction using abstract meaning representation[C]//Proceedings of the BioNLP, 2017: 126-135.
[41] JINGBO X, ALEX C F, XING Z. A novel feature selection strategy for enhanced biomedical event extraction using the turku system[J]. BioMed Research International, 2014:205-239.
[42] SABENABANU A, BALU B, JEYAKUMAR N. Multiscale laplacian graph kernel combined with lexico-syntactic patterns for biomedical event extraction from literature[J]. Knowledge and Information Systems, 2021, 63(1): 143-173.
[43] LISHUANG L, YIWEN W, DEGEN H. Improving feature-based biomedical event extraction system by integrating argument information[C]//Proceedings of the BioNLP Shared Task Workshop, 2013: 109-115.
[44] XIAO L, ANTOINE B, YVES G. Biomedical event extraction by multi-class classification of pairs of text entities[C]//Proceedings of BioNLP Shared Task Workshop, 2013: 45-49.
[45] JARI B, TAPIO S. Generalizing biomedical event extraction[C]//Proceedings of BioNLP Shared Task Workshop, 2011: 183-191.
[46] DEYU Z, DAYOU Z. A semi-supervised learning framework for biomedical event extraction based on hidden topics[J]. Artificial Intelligence in Medicine, 2015: 64(1): 51-58.
[47] JARI B, FILIP G, TAPIO S. University of turku in the bionlp’11 shared task[J]. BMC Bioinformatics, 2012, 13: 1-13.
[48] ANDREAS V, MARK C. Search-based structured prediction applied to biomedical event extraction[C]//Proceedings of the 15th Conference on Computational Natural Language Learning, 2011: 49-57.
[49] ANRAN W, JIAN W, HONGFEI L, et al. A multiple distributed representation method based on neural network for biomedical event extraction[J]. BMC Medical Informatics and Decision Making, 2017, 17(3): 59-66.
[50] XINYU H, LISHUANG L, XINGCHEN S, et al. Multi-level attention based BiLSTM neural network for biomedical event extraction[J]. Ieice Transactions on Information and Systems, 2019, 102(9): 1842-1850.
[51] LISHUANG L, JIA W, JIEQIONG Z, et al. Biomedical event extraction based on GRU integrating attention mechanism[J]. Bmc Bioinformatics, 2018, 19(9): 93-100.
[52] XINYI Y, WENGE R, JINGSHUANG L, et al. Lstm-based end-to-end framework for biomedical event extraction[J]. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2019: 17(6): 2029-2039.

基金

国家自然科学基金(62176187)
PDF(4390 KB)

Accesses

Citation

Detail

段落导航
相关文章

/