李青青,杨志豪,罗凌,林鸿飞,王健. 基于多任务学习的生物医学实体关系抽取[J]. 中文信息学报, 2019, 33(8): 84-92.
LI Qingqing, YANG Zhihao, LUO Ling, LIN Hongfei, WANG Jian. A Multi-task Learning Approach to Biomedical Entity Relation Extraction. , 2019, 33(8): 84-92.
基于多任务学习的生物医学实体关系抽取
李青青,杨志豪,罗凌,林鸿飞,王健
大连理工大学 计算机科学与技术学院,辽宁 大连 116024
A Multi-task Learning Approach to Biomedical Entity Relation Extraction
LI Qingqing, YANG Zhihao, LUO Ling, LIN Hongfei, WANG Jian
School of Computer Science and Technology, Dalian University of Technology, Dalian, Liaoning 116024, China
Abstract：Biomedical relation extraction plays an important role in biomedical text mining, which can automatically extract high-quality biomedical relationships from biomedical texts. In this paper, we apply neural network-based multi-task learning method to explore the correlation among multiple biomedical relation extraction tasks. In our study, we construct a fully-shared model (FSM) and a shared-private model (SPM) and propose an attention-based main-auxiliary model (Att-MAM). Experimental results on five public biomedical relation extraction datasets show that the multi-task learning can obtain better performance than the single task method.
[1] Lu Z.PubMed and beyond: A survey of web tools for searching biomedical literature[J].Database(Oxford):2011,2011(1):baq036. [2] Zeng D,Liu K,Lai S,et al.Relation classification via convolutional deep neural network[C]//Proceedings of ICCL 2014.Dublin,2014:2335-2344. [3] Zhang S,Zheng D,Hu X,et al.Bidirectional long short-term memory networks for relation classification[C]//Proceedings of Pacific Asia Conference on Language Information and Computation 2015.Shanghai,China,2015. [4] Vu N T,Adel H,Gupta P,et al.Combining recurrent and convolutional neural networks for relation classification[J].arXiv preprint arXiv:1605.07333,2016. [5] 李丽双,蒋振超,万佳,等.利用词表示和深层神经网络抽取蛋白质关系[J].中文信息学报,2017,31(01): 31-40. [6] Zhang Y,Lin H,Yang Z,et al.A hybrid model based on neural networks for biomedical relation extraction[J].Journal of Biomedical Informatics,2018,81: 83-92. [7] Caruana R.Multitask learning[J].Machine Learning,1997,28(1):41-75. [8] Ruder S.An overview of multi-task learning in deep neural networks[J].arXiv preprint arXiv:1706.05098,2017. [9] Wu Z,Valentini-Botinhao C,Watts O,et al.Deep neural networks employing multi-task learning and stacked bottleneck features for speech synthesis[C]//Proceedings of IEEE ICASSP 2015.South Brisbane,Queen-sland,Australia,2015:4460-4464. [10] Ando R K,Zhang T.A framework for learning predictive structures from multiple tasks and unlabeled data[J].Journal of Machine Learning Research,2005(6): 1817-1853. [11] Maurer A,Pontil M,Romera Paredes B.The benefit of multitask representation learning[J].Journal of Machine Learning Research,2016,17(1): 2853-2884. [12] MarasoviAc＇,Frank A.SRL4ORL: Improving opinion role labelling using multi-task learning with semantic role labeling[J].arXiv preprint arXiv:1711.00768,2017. [13] Liu P,Qiu X,Huang X.Adversarial multi-task learning for text classification[J].arXiv preprint arXiv:1704.05742,2017. [14] Mikolov T,Sutskever I,Chen K,et al.Distributed representations of words and phrases and their compositionality[C]//Proceedings of Advances in Neural Information Processing Systems 2013.Stateline,Nevada,USA,2013: 3111-3119. [15] Schmidhuber J,Hochreiter S.Long short-term memory[J].Neural Computation,1997,9(8): 1735-1780. [16] Mnih V,Heess N,Graves A.Recurrent models of visual attention[J]//Proceedings of Advances in Neural Information Processing Systems,2014,3:2204-2212. [17] Bahdanau D,Cho K,Bengio Y.Neural machine translation by jointly learning to align and translate[J].xrXiv.preprint arXiv:1409.047303,2014. [18] Wang L,Cao Z,Melo G D,et al.Relation classification via multi-level attention CNNs[C]//Proceedings of Meeting of the Association for Computational Linguistics 2016.Berlin,Germany,2016:1298-1307. [19] 杨东,王移芝.基于Attention-based C-GRU神经网络的文本分类[J].计算机与现代化,2018(02): 96-100. [20] Bunescu R,Ge R,Kate R J,et al.Comparative experiments on learning information extractors for proteins and their interactions[J].Artificial Intelligence in Medicine,2005,33(2):139-155. [21] Pyysalo S,Ginter F,Heimonen J,et al.BioInfer: A corpus for information extraction in the biomedical domain[J].BMC Bioinformatics,2007,8(1):50. [22] Martin K,Rabal O,Saber A.Overview of the BioCreative VI chemical-protein interaction Track[C]//Proceedings of the BioCreative VI Workshop 2017.Bethesda,Maryland,USA,2017:142-147. [23] Segura Bedmar I,Martinez P,Sánchez Cisneros D.The 1st DDIExtraction-2011 challenge task: Extraction of Drug-Drug Interactions from biomedical texts[C]//Proceedings of DDIExtracton-2011 challenge task,HUelva,Spain,2011:1-9. [24] Uzuner ,South B R,Shen S,et al.2010 I2b2/VA challenge on concepts,assertions,and relations in clinical text[J].Journal of the American Medical Informatics Association,2011,18(5): 552-556. [25] Hinton G,Srivastava N,Swersky K.RMSprop: Divide the gradient by a running average of its recent magnitude[C]//Proceedings of Neural Networks for Machine Learning,Coursera Lecture 6e,2012. [26] Sahu S K,Anand A,Oruganty K,et al.Relation extraction from clinical texts using domain invariant convolutional neural network[C]//Proceedings of the 15th Workshop on Biomedical Natural Language Processing.Berlin,Germany,2016: 71. [27] Raj D,Sahu S K,Anand A.Learning local and global contexts using a convolutional recurrent network model for relation classification in biomedical text[C]//Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL2017).Vancouver,Canada,2017: 311-321.
2019, Vol. 33 
