汉语框架语义角色标注对汉语框架语义分析具有重要作用。目前汉语框架语义角色标注任务主要针对动词框架,但是汉语没有丰富的形态变化,很多语法意义都是通过虚词来表现的,其中副词研究是现代汉语虚词研究的重要部分,因此该文从副词角度出发构建了汉语副词框架及数据集,且对框架下的词元按照语义强弱进行了等级划分。目前的语义角色标注模型大多基于BiLSTM网络模型,该模型虽然可以很好地获取全局信息,但容易忽略句子局部特征,且无法并行训练。针对上述问题,该文提出了基于BERT特征融合与膨胀卷积的语义角色标注模型,该模型包括四层: BERT层用于表达句子的丰富语义信息,Attention层对BERT获取的每一层信息进行动态权重融合,膨胀卷积(IDCNN)层进行特征提取,CRF层修正预测标签。该模型在三个副词框架数据集上表现良好,F1值均达到了82%以上。此外,将该模型应用于CFN数据集上,F1值达到88.29%,较基线模型提升了4%以上。
Abstract
Chinese frame semantic role labeling plays an important role in Chinese frame semantic analysis. At present, the task of semantic role labeling in Chinese frame is mainly aimed at verb frame. This paper constructs a Chinese adverb framework and dataset, and classifies the word in the framework according to its semantic strength. Then, this paper proposes a semantic role labeling model based on Bert feature fusion and expansion convolution. The model includes four layers, with the bert layer to reperesent the rich semantic information of sentences, the attention layer to dynamical weighs the information from each BERT layer, the expansion convolution (IDCNN) layer to extract features, and the CRF layer to predict tags. The model performs well in three adverb frame datasets, achieveing 82% or more F1 value. In addition, the model achieves 88.29% F1 value in CFN dataset, which is 4% above the baseline model.
关键词
汉语框架语义角色标注 /
副词 /
BERT /
膨胀卷积 /
CRF
{{custom_keyword}} /
Key words
Chinese frame semantic role labeling /
adverb /
BERT /
IDCNN /
CRF
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] ZHAN F F, ZHANG J J, ZHOU Y, et al. Machine transla tion by modeling predicate-argument structure transformation[C]//Proceedings of the 24th International Conference on Computational Linguistics. New York, USA: ACM Press,2012: 1-7.
[2] CHRISTENSEN J, SODERLANG S, ETZIONI O. Semantic role label ing for open information extraction [C]//Proceedings of the lst International Workshop on Fomalisms and Methodology for Learning by Reading.New York, USA: ACM Press,2010: 52-60.
[3] NARAYANAN S, HARABAGIU S. Question answering based on semantic structures [C]//Proceedings of the 20th International Conference on Computational Linguistics.New York, USA: ACM Press,2004: 693-670.
[4] FILLMORE C J. Frame semantics and the nature of language[J]. Annals of the New York Academy of Sciences, 2006, 280(1): 20-32.
[5] 吕叔湘.汉语语法分析问题[M].北京: 商务印书馆,1997: 464-551.
[6] 郭锐.现代汉语词类研究[M].北京: 商务印书馆,2002.
[7] 张谊生.现代汉语副词研究[M].上海: 学林出版社,2000.
[8] 李晓琪.现代汉语虚词讲义[M].北京: 北京大学出版社,2005.
[9] 王力.中国现代语法[M].北京: 商务印书馆, 1982.
[10] 张苗苗,张玉洁,刘明童,等.基于Gate机制与Bi-LSTM-CRF的汉语语义角色标注[J].计算机与现代化,2018,(4): 1-6.
[11] GILDEA D, JURAFSKY D. Automatic labeling of semantic roles[J]. Computational Linguistics, 2002,28(3): 245-288.
[12] CHEN J, RAMBOW O. Use of deep linguistic features for the recognition and labeling of semantic arguments [C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing.Sapporo: ACL,2003.41-48.
[13] PRADHAN S, HACIOGLU K, KRUGLER V, et al. Support vector learning for semantic argument classification[J].Machine Learning Journal,2005,60(3): 11-39.
[14] 刘挺,车万翔,李生.基于最大熵分类器的语义角色标注[J].软件学报, 2007(03): 565-573.
[15] BOXWELL S A, MEHAY D, BREW C. Brutus: A semantic role labeling system incorporating CCG, CFG, and dependency features [C]//Proceedings of the ACL-JCNLP, Stroudsburg: ACL,2009: 37-45.
[16] 李济洪,王瑞波,王蔚林,等.汉语框架语义角色自动标注[J].软件学报, 2010, 30(4): 597-611.
[17] 王智强, 李茹, 阴志洲,等.基于依存特征的汉语框架语义角色自动标注[J].中文信息学报,2013,27(2): 34-40.
[18] 杨凤玲,周俏丽,蔡东风,等.结合短语结构句法的语义角色标注[J].中文信息学报, 2018, 32(6): 1-11.
[19] COLLOBERT R, WESTON J, BOTTOU L, et al. Natural language processing (almost) from scratch[J]. Journal of Machine Learning Research, 2011,12(1): 2493-2537.
[20] 王臻,常宝宝,穗志方.基于分层输出神经网络的汉语语义角色标注[J].中文信息学报, 2014,28(60): 51-61.
[21] KSHIRSAGAR M, THOMSON S, SCHNEIDER N, et al. Frame-semantic role labeling with heterogeneous annotations[C]//Proceedings of the International Joint Conference on Natural Language Processing.Beijing,China: IJCNLP,2015: 218-224.
[22] ZHOU J, XU W. End-to-end learning of semantic role labeling using recurrent neural networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing.Beijing, China: ACL,2015: 1127-1137.
[23] 王瑞波,李济洪,李国臣,等.基于Dropout正则化的汉语框架语义角色识别[J].中文信息学报,2017,31(1): 147-154.
[24] YANG B, MITCHELL T M.A joint sequential and relational model for frame-semantic parsing[C]//Proceedings of the Empirical Methods in Natural Language Processing. Copenhagen, Denmark: ACL,2017: 1247-1256.
[25] 高李政,周刚,黄永忠,等.基于Bert模型的框架语义角色标注方法[J].信息工程大学学报,2020,103(03): 45-51
[26] MARCHEGGIANI D, TITOV I. Graph convolutions over constituent trees for syntax-aware semantic role labeling[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Online: ACL,2020: 3915-3928.
[27] TAN Z, WANG M, XIE J,et al. Deep semantic role labeling with self-attention[C]//Proceedings of the National Conference on Artificial Intelligence.AAAI: New Orlearns,USA,2018: 4929-4936.
[28] STRUBELL E,VERGA P, ANDOR D, et al. Linguistically-informed self-attention for semantic role labeling[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: ACL,2018: 5027-5038.
[29] HE L H, LEE K, LEVY O, et al. Syntac-enhanced self-attention-based semantic role labeling[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Hong Kong, China: EMNLP-IJCNLP,2019: 616-626.
[30] 王晓晖, 李茹, 王智强,等. 基于self-attention的句法感知汉语框架语义角色标注[C]//Proceedings of the 19th Chinese National Conference on Computational Linguistics.Haikou,China: Chinese Information Processing Society of China,2020: 616-623.
[31] GANESH J, BENOT S, DJAM S. What does BERT learn about the structure of language? [C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Florence, Italy: ACL, 2019: 3651-3657.
[32] FISHER Y, VLADLEN K. Multi-scale context aggregation by dilated convolutions[C]//Proceedings of the International Conference on Learning Representations. San Juan, Puerto Rico: ICLR,2016: 674-685.
[33] 李妮,关焕梅,杨飘,等.基于BERT-IDCNN-CRF的中文命名实体识别方法[J].山东大学学报(理学版),2020, 55(01): 106-113.
[34] 王臻,常宝宝,穗志方.基于分层输出神经网络的汉语语义角色标注[J].中文信息学报,2014: 28(6): 51-61.
[35] 吕雷. 汉语框架语义角色自动标注研究[D]. 山西大学硕士学位论文, 2014.
[36] 杨耀文. 基于神经网络模型的汉语框架语义角色识别[D]. 山西大学硕士学位论文, 2016.
[37] 党帅兵. 基于词分布表征的汉语框架语义角色识别研究[D]. 山西大学硕士学位论文, 2015.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家社会科学基金(18BYY009);山西省“四个一批”科技兴医创新计划项目(2022XM01)
{{custom_fund}}
PDF(3223 KB)
