基于笔画中文字向量模型设计与研究

摘要
参考文献
相关文章 (8)

全文: PDF (1815 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要中文汉字在横向、纵向展开具有二维的复杂结构。现有的中文词向量研究大都止步于汉字字符,没有利用中文笔画序列生成字向量,且受限于统计模型本质,无法为低频、未登录字词生成高质量向量表示。为此,该文提出了一种依靠中文笔画序列生成字向量的模型Stroke2Vec,扩展Word2Vec模型CBOW结构,使用卷积神经网络替换上下文信息矩阵、词向量矩阵,引入注意力机制,旨在模拟笔画构造汉字的规律,通过笔画直接生成字向量。将Stroke2Vec模型与Word2Vec、GloVe模型在命名实体识别任务上进行评测对比。实验结果显示,Stroke2Vec模型F₁值达到81.49%,优于Word2Vec 1.21%,略优于GloVe模型0.21%,而Stroke2Vec产生的字向量结合Word2Vec模型结果,在NER上F₁值为81.55%。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	赵浩新
	俞敬松
	林杰

关键词 ：字向量, 笔画, 连续词袋模型

Abstract：Chinese characters have a two-dimensional complex structure that spreads horizontally and vertically. Most of the studies about Chinese word embedding explore Chinese character level without considering the strokes sequence. This paper proposes a novel Stroke2Vec model that generates word embedding based on its stroke sequence. The model expands CBOW of Word2Vec model by using CNN and attention model instead of matrix. The Stroke2Vec aims to simulate the rule of strokes structure of Chinese characters and produce better character embedding with only strokes sequence. Compared with the Word2Vec and GloVe in NER task, the results show that our model achieves 81.49% F₁-score, out-performing Word2Vec by 1.21%, and GloVe by nearly 0.21%. And combining Stroke2Vecs and Word2Vecs leads to an F₁-score of 81.55%.

Key words： character embedding stroke CBOW

收稿日期: 2018-05-16

中图分类号:

TP391

作者简介: 赵浩新(1992—),硕士研究生,主要研究领域为自然语言处理。E-mai:xindada@pku.edu.cn
俞敬松(1971—),通信作者,副教授,主要研究领域为自然语言处理、信息检索、计算机辅助翻译。E-mail:yjs@ss.pku.edu.cn
林杰(1991—),硕士研究生,主要研究领域为信息检索、自然语言处理。E-mail:jie.l@outlook.com

链接本文:

http://jcip.cipsc.org.cn/CN/Y2019/V33/I5/17

[1] Bengio Y,Ducharme R,Vincent P,et al. A neural probabilistic language model[J]. Journal of Machine Learning Research,2006,3(6):1137-1155.
[2] Mnih A,Hinton G. A scalable hierarchical distributed language model[C]//Proceedings of the 21st International Conference on Neural Information Processing Systems. Curran Associates Inc,2008:1081-1088.
[3] Mikolov T,Karafiát M,Burget L,et al. Recurrent neural network based language model[C]//Proceedings of the INTERSPEECH 2010,Conference of the International Speech Communication Association,Makuhari,Chiba,Japan,September. DBLP,2010:1045-1048.
[4] Mikolov T,Chen K,Corrado G,et al. Efficient estimation of word representations in vector space[J]. arXiv preprint arXiv: 1301.3781,2013.
[5] Mikolov T. Statistical language models based on neural networks[D]. Czech,Brno University of Technology,2012.
[6] Mikolov T,Sutskever I,Chen K,et al. Distributed representations of words and phrases and their compositionality[J].arXiv preprint arXiv: 1310.4546,2013.
[7] Levy O,Goldberg Y. Neural word embedding as implicit matrix factorization[J]. Advances in Neural Information Processing Systems,2014(3):2177-2185.
[8] Ji S H,Yun H,Yanardag P,et al. WordRank: Learning word embeddings via robust ranking[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing,2016:658-668.
[9] Pennington J,Socher R,Manning C. Glove: Global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing,2014:1532-1543.
[10] Bojanowski P,Grave E,Joulin A,et al. Enriching word vectors with subword information[J]. Transactions of the Association of Computational Linguistics,2017,5(1):135-146.
[11] Pinter Y,Guthrie R,Eisenstein J. Mimicking word embeddings using subword RNNs[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing,2017:102-112.
[12] Chen X,Xu L,Liu Z,et al. Joint learning of character and word embeddings[C]//Proceedings of the 24th International Conference on Artificial Intelligence. AAAI Press,2015:1236-1242.
[13] Sun Y,Lin L,Yang N,et al. Radical-enhanced Chinese character embedding[J].arXiv preprint arXiv: 1404.4714,2014.
[14] Cao S,Lu W,Zhou J,et al. Cw2Vec: Learning Chinese word embeddings with stroke n-gram information[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence,2018.
[15] CAO S,LU W. Improving word embeddings with convolutional feature learning and subword information[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence,North America,feb.2017.
[16] Lecun Y,Bottou L,Bengio Y,et al. Gradientbased learning applied to document recognition[J]. Proceedings of the IEEE,1998,86(11):2278-2324.
[17] Gehring J,Auli M,Grangier D,et al. Convolutional sequence to sequence learning[J].arXiv preprint arXiv: 1705.03122.2017.
[18] Huang Z,Xu W,Yu K. Bidirectional LSTM-CRF models for sequence tagging[J].arXiv preprint arXiv: 1508.01991.2015.