在实体关系抽取任务中,通常采用远程监督(distant supervision,DS)数据集,远程监督方法能通过大规模语料库自动标注数据来扩张数据集,但这无疑会使数据集充满大量的噪声。为此,该文将深度残差网络(deep residual network,ResNet)应用到关系提取的远程监督数据集上,通过加深网络层数来提高模型降噪能力。同时,提出了Gate模块,有效提高了深度残差网络的性能。该模块可以学习到每个特征通道的重要性,通过权重增强或抑制各个特征通道的比重,从而防止过拟合。另外,为了进一步解决数据集降噪问题,还提出了一种双池化层的池化层新方案。实验结果表明所提方法相比于目前效果较好的PCNN+ATT模型,在准确率和召回率上都有3%左右的提升。
Abstract
In the entity relationship extraction task, the distant supervision data set with substantial noise is often used. This paper applies ResNet to the distant supervision data set of relation extraction, to exploit its denoising ability by deepening the network. This paper also proposes a Gate module that can effectively improve the performance of deep residual networks, which can learn the importance between each feature channel. In addition, in order to further reduce the noise, this paper also proposes a new pooling layer called double pooling layer. The experimental results show that the proposed method achieves an improvement of 3% in precision and recall rate compared with the PCNN+ATT model.
关键词
实体关系提取 /
远程监督 /
深度残差网络
{{custom_keyword}} /
Key words
relationship extraction /
distant supervision /
deep residual network
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 王元卓,靳小龙,程学旗.网络大数据: 现状与展望[J].计算机学报,2013,36(6): 1125-1138.
[2] 刘峤,李杨,段宏,等.知识谱图构建技术综述[J].计算机研究与发展,2016,53(3): 582-600.
[3] Daojian Zeng,Kang Liu, Siwei Lai,et al.Relation classification via convolutional deep neural network[C]//Proceedings of the 25th International Conference on Computational Linguistics: Technical Papers,2007:2335-2344.
[4] Peng Zhou,Wei Shi, Jun Tian, et al. Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistic, 2016:207-212.
[5] Mike Mintz,Steven Bills, Rion Snow,et al. Distant supervision for relation extraction without labeled data[C]//Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 2009:1003-1011.
[6] Daojian Zeng,Kang Liu, Yubo Chen,et al.Distant supervision for relation extraction via piecewise convolutional neural networks[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing,2015:1753-1762.
[7] Yankai Lin,Shiqi Shen, Zhiyuan Liu,et al. Neural relation extraction with selective attention over instances[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016:2124-2133.
[8] Guoliang Ji,Kang Liu, Shizhu He,et al. Distant supervision for relation extraction with sentence-level attention and entity descriptions[C]// Proceedings of the AAAI, 2017:3060-3066.
[9] Xiaotian Jiang,Quan Wang, Peng Li,et al. Relation extraction with multi-instance multi-label convolutional neural networks[C]// Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers. 2016:1471-1480.
[10] Yankai Lin,Zhiyuan Liu, Maosong Sun,et al. Neural relation extraction with multi-lingual attention[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017:34-43.
[11] Bingfeng Luo,Yansong Feng, Zheng Wang,et al. Learning with noise: Enhance distantly supervised relation extraction with dynamic transition matrix[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017:430-439.
[12] Yi Yao Huang,Wang William Yang. Deep residual learning for weakly supervised relation extraction[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017:1803-1807.
[13] Limin Yao,Sebastian Riedel, Andrew McCallum,et al. Collective crossdocument relation extraction without labelled data[C]//Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2010:1013-1023.
[14] Prajit Ramachandran,Zoph Barret,Le Quoc V,et al.Searching for activation functions[J].arXiv preprint arXiv: 1710.05941, 2017.
[15] Ioffe Sergey,Szegedy Christian.Batch normalization: Accelerating deepnetwork training by reducing internal covariate shift[J].arXiv preprint arXiv: 1502.03167,2015.
[16] Xiangrong Zeng,Shizhu He, Kang Liu,et al. Large scaled relation extraction with reinforcement learning[C]//Proceedings of the AAAI, 2018:33-41.
[17] Barret Zoph,Ramachandran Prajit, Le Quoc V,et al. Swish: A self-gated activation function[J].arXiv preprint arXiv: 1710.05941,2017.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
江苏省青年科学基金(BK20150159);国家自然科学基金(61673193);中国博士后科学基金(2017M621625);江苏省自然科学基金(BK20181341)
{{custom_fund}}
PDF(3497 KB)
