汉语框架语义角色识别是汉语框架语义分析的重要任务之一。该文基于汉语词语、词性等特征的分布式表示,使用一种多特征融合的神经网络结构来构建汉语框架语义角色识别模型。鉴于可用的训练语料规模有限,该文采用了Dropout正则化技术来改进神经网络的训练过程。实验结果表明,Dropout正则化的加入有效地缓解了模型的过拟合现象,使得模型的F值有了近7%的提高。该文进一步优化了学习率以及分布式表示的初始值,最终的汉语框架语义角色识别的F值达到70.54%,较原有的最优结果提升2%左右。
Abstract
Semantic role identification is an important task for semantic parsing according to Chinese FrameNet. Based on distributed representations of Chinese words, the part-of-speech and other symbolic features, we build our semantic role identification model by employing a kind of multi-feature-integrated neural network architecture. Due to the relative small training corpus, we adopt the dropout regularization to improve quality of the training process. Experimental results indicate that, 1) dropout regularization can effectively alleviate over-fitting of our model, and 2) the F-measure increases upto 7%. With further optimization of the learning rate and the pre-trained word embeddings, the final F-measure of our semantic role identification model reaches 70.54%, which is about 2% higher than the state-of-the-art result.
关键词
汉语框架网络 /
语义角色识别 /
Dropout正则化
{{custom_keyword}} /
Key words
Chinese FrameNet /
semantic role identification /
dropout regularization;
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Fillmore C J, Baker C F. Frame semantics for text understanding[C]//Proceedings of WordNet and Other Lexical Resources Workshop, NAACL. 2001.
[2] 李济洪, 王瑞波, 王蔚林, 等. 汉语框架语义角色的自动标注[J]. 软件学报, 2010, 21(4): 597-611.
[3] 宋毅君, 王瑞波, 李济洪, 等. 基于条件随机场的汉语框架语义角色自动标注[J]. 中文信息学报, 2014, 28(3): 36-47.
[4] Collobert R, Weston J, Bottou L, et al. Natural language processing (almost) from scratch[J]. The Journal of Machine Learning Research, 2011, 12: 2493-2537.
[5] Turian J, Ratinov L, Bengio Y. Word representations: a simple and general method for semi-supervised learning[C]//Proceedings of the 48th annual meeting of the association for computational linguistics. Association for Computational Linguistics, 2010: 384-394.
[6] 刘挺, 车万翔, 李生. 基于最大熵分类器的语义角色标注[J]. 软件学报, 2007, 18(3): 565-573.
[7] Pradhan S, Ward W, Hacioglu K, et al. Shallow Semantic Parsing using Support Vector Machines[C]//Proceedings of HLT-NAACL. 2004: 233-240.
[8] Zhou J, Xu W. End-to-end learning of semantic role labeling using recurrent neural networks[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. 2015.
[9] Hong S, Noh H, Han B. Decoupled deep neural network for semi-supervised semantic segmentation[C]//Proceedings of Advances in Neural Information Processing Systems. 2015: 1495-1503.
[10] Shi L, Mihalcea R. Putting pieces together: Combining FrameNet, VerbNet and WordNet for robust semantic parsing[M]//Computational linguistics and intelligent text processing. Springer Berlin Heidelberg, 2005: 100-111.
[11] 邵艳秋, 穗志方, 吴云芳. 基于词汇语义特征的中文语义角色标注研究[J]. 中文信息学报, 2009, 23(6): 3-11.
[12] Mikolov T, Chen K, Corrado G, et al. Efficient estimation of word representations in vector space[J]. arXiv preprint arXiv: 1301.3781, 2013.
[13] Pennington J, Socher R, Manning C D. Glove: Global Vectors for Word Representation[C]//Proceedings of EMNLP. 2014, 14: 1532-1543.
[14] 李国臣, 党帅兵, 王瑞波, 等. 基于字的分布表征的汉语基本块识别[J]. 中文信息学报, 2014, 28(6): 18-25.
[15] 李国臣, 王瑞波, 李济洪. 基于条件随机场模型的汉语功能块自动标注[J]. 计算机研究与发展, 2010, 47(2): 336-343.
[16] Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: A simple way to prevent neural networks from overfitting[J]. The Journal of Machine Learning Research, 2014, 15(1): 1929-1958.
[17] Yu W, Ruibo W, Huichen J, et al. Blocked 3× 2 cross-validated t-test for comparing supervised classification learning algorithms[J]. Neural computation, 2014, 26(1): 208-235.
[18] Mikolov T, Karafiát M, Burget L, et al. Recurrent neural network based language model[C]//Proceedings of INTERSPEECH. 2010, 2: 3.
[19] Che W, Zhang M, Aw A, et al. Using a Hybrid Convolution Tree Kernel for Semantic Role Labeling[J]. ACM Transactions on Asian Language Information Processing, 2008, 7(4).
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(NNSFC-61503228);NSFC- 广东联合基金(第二期)
{{custom_fund}}
PDF(1888 KB)
王瑞波(1985—),博士研究生,主要研究领域为统计自然语言处理。E-mail: wangruibo@sxu.edu.cn
李济洪(1964—),教授,博士生导师,主要研究领域为统计自然语言处理,统计机器学习。E-mail: lijih@sxu.edu.cn
李国臣(1963—),教授,主要研究领域为中文信息处理。E-mail: ligc@sxu.edu.cn
