深度学习中汉语字向量和词向量结合方式探究

摘要
参考文献
相关文章 (15)

全文: PDF (2046 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要该文旨在探究深度学习中汉语字向量和词向量的有效结合方式。我们在以词作为基础语义单元和以字作为基础语义单元这两个方向进行探究,实验了字、词信息多种浅层结合方式和深层结合方式。为了验证该文提出的结合方式的有效性,我们改进了一种compare-aggregate模型,并在基于文档的问答系统上进行了实验。实验结果表明,有效的汉语字向量和词向量的结合方式超越了单独的字向量和词向量,提升了基于文档的问答系统的性能,使其结果与目前最好的结果可媲美。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	李伟康
	李炜
	吴云芳

关键词 ：字向量, 词向量, 深度学习, 问答系统

Abstract：This paper investigates the combination of Chinese character and word embeddings in deep learning. We propose to do experiments considering shallow and deep combinations based on word and character. In order to demonstrate the effectiveness of combination, we present a compare-aggregate model solving the problem of question answering. Extensive experiments conducted on the open DBQA data demonstrate that the effective combination of characters and words significantly improves the system achieving comparable results with state-of-art systems.

Key words： character embedding word embedding deep learning question answering system

收稿日期: 2017-09-29

中图分类号:

TP391

基金资助:教育部人文社会科学研究规划基金(13YJA740060);国家自然科学基金(61371129,61773026)

作者简介: 李伟康(1993—),硕士,主要研究领域为智能问答系统,计算语言学。E-mail:wavejkd@pku.edu.cn
吴云芳(1973—),通信作者,博士,副教授,主要研究领域为智能问答系统,计算语言学。E-mail:wuyf@pku.edu.cn
李炜(1993—),硕士,主要研究领域为智能问答系统,计算语言学。E-mail:liweitj47@pku.edu.cn

链接本文:

http://jcip.cipsc.org.cn/CN/Y2017/V31/I6/140

[1]Feng M,Xiang B,Glass M R,et al.Applying deep learning to answer selection:A study and an open task[C]//Proceeding of Automatic Speech Recognition and Understanding (ASRU),2015 IEEE Workshop on.IEEE,2015:813-820.
[2]Tan M,Santos C,Xiang B,et al.LSTM-based deep learning models for non-factoid answer selection[J].arXiv preprint arXiv:1511.04108,2015.
[3]Tan M,Dos Santos C,Xiang B,et al.Improved representation learning for question answer matching[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics,2016.
[4]Tai K S,Socher R,Manning C D.Improved semantic representations from tree-structured long short-term memory networks[J].arXiv preprint arXiv:1503.00075,2015.
[5]Bowman S R,Angeli G,Potts C,et al.A large annotated corpus for learning natural language inference[J].arXiv preprint arXiv:1508.05326,2015.
[6]Seo M,Kembhavi A,Farhadi A,et al.Bidirectional attention flow for machine comprehension[J].arXiv preprint arXiv:1611.01603,2016.
[7]Sukhbaatar S,Weston J,Fergus R.End-to-end memory networks[C]//Advances in Neural Information processing systems,2015:2440-2448.
[8]Wang S,Jiang J.Learning natural language inference with LSTM[J].arXiv preprint arXiv:1512.08849,2015.
[9]Goldberg Y,Levy O.word2vec explained:Deriving mikolov et al.s negative-sampling word-embedding method[J].arXiv preprint arXiv:1402.3722,2014.
[10]Pennington J,Socher R,Manning C D.Glove:Global vectors for word representation[C]//EMNLP.2014,14:1532-1543.
[11]Alexandrescu A,Kirchhoff K.Factored neural language models[C]//Proceedings of the Human Language Technology Conference of the NAACL,Companion Volume:Short Papers.Association for Computational Linguistics,2006:1-4.
[12]Luong T,Socher R,Manning C D.Better word representations with recursive neural networks for morphology[C]//Proceedings of CoNLL 2013,2013:104-113.
[13]Huang E H,Socher R,Manning C D,et al.Improving word representations via global context and multiple word prototypes[C]//Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics:Long Papers-Volume 1.Association for Computational Linguistics,2012:873-882.
[14]Chen X,Xu L,Liu Z,et al.Joint learning of character and word embeddings[C]//Proceedings of the 24th International Joint Conference on Artificial Intelligence.2015.
[15]Yang Y,Yih W,Meek C.WikiQA:A challenge dataset for open-domain question answering[C]//Proceedings of the EMNLP 2015.2015:2013-2018.
[16]Wang S,Jiang J.A compare-aggregate model for matching text sequences[J].arXiv preprint arXiv:1611.01747,2016.
[17]Duan N.Overview of the NLPCC-ICCPOL 2016 Shared task:open domain Chinese question answering[C]//Proceedings of International Conference on Computer Processing of Oriental Languages.Springer InternationalPublishing,2016:942-948.
[18]Sun J.‘Jieba’Chinese word segmentation tool[CP/OL].2012.https://github.com/whtsky/jieba/
[19]Fu J,Qiu X,Huang X.Convolutional deep neural networks for document-based question answering[C]//Proceedings of International Conference on Computer Processing of Oriental Languages.Springer International Publishing,2016:790-797.
[20]Wu F,Yang M,Zhao T,et al.A hybrid approach to DBQA[C]//Proceedings of International Conference on Computer Processing of Oriental Languages.Springer International Publishing,2016:926-933.