维吾尔语自动标注是维吾尔语信息处理后续句法分析、语义分析及篇章分析必不可少的基础工作。词性是词的重要的语法信息,假如一个词的词性无法确定或一个词给予错误的词性,对后续句法分析造成直接的影响。本文使用感知器训练算法和viterbi算法对维吾尔语进行词性标注,并在词性标注时利用词的上下文信息作为特征。实验结果表明,该方法对维吾尔语词性标注有良好的效果。
Abstract
Uyghur POS tagging is essential for subsequent tasks such as Uyghur sentence analysis, semantic analysis and discourse analysis. In this paper, perceptron training algorithm and viterbi algorithm are used for Uyghur POS tagging, and the context information of the words are employed. Experiment results show that this method has good results for the Uyghur POS tagging.
关键词
词性标注 /
感知器算法 /
维吾尔语词性标注
{{custom_keyword}} /
Key words
The speech tagging /
Perceptron algorithm /
Uyghur speech tagging
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 吐尔根·依不拉音,阿里甫·库尔班.基于词典的现代维吾尔语词性自动标注系统的研究[A].中文输入技术发展历程及输入方案汇编(论文集)[C],2006.11.
[2] Màrquez, Lluís, LluisPadro et al. A Machine Learning Approach to POS Tagging. Machine Learning 2000,39(1): 59-91.
[3] Brill Eric. Transformation-based Error-driven Learning and Natural Language Processing: A Case Study in Part-of-speech Tagging. Computational linguistics.1995,21(4): 543-565.
[4] 周明, 吴进, 黄昌宁. 用于词性标注的一种快速学习算法对Brill 的基于变换算法的一项改进[J]. 计算机学报,1998 (4) : 357-366
[5] 买合木提·买买提,吐尔根·依布拉音.基于n‐gram 的维吾尔语词性标注研究[C]//第二届中国少数民族青年自然语言处理学术研讨会.2008 年10 月,中国安徽合肥.2008: 185-189.
[6] 艾斯卡尔·亚克甫,肖克来提,玉素甫·艾白都拉.维吾尔语词频统计子系统的体系结构[J].新疆师范大学学报 (自然科学版)2006,25(2): 16-20
[7] 艾山·吾买尔·维吾尔语词法句法分析关键技术的研究[D].博士论文,新疆大学,2010年.
[8] Ratnaparkhi A. A Maximum Entropy Model for Part-of-speech Tagging[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1996, 1: 133-142.
[9] Dobrushin R L. Central Limit Theorem for Nonstationary Markov Chains[J]. Theory of Probability & Its Applications, 1956, 1(1): 65-80.
[10] Lafferty John, Andrew McCallum, Fernando CN Pereira. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. ICML 18(2001): 45-54.
[11] Manshadi V H, Gharan S O, Saberi A. Online Stochastic Matching: Online Actions Based on Offline Statistics[J]. Mathematics of Operations Research, 2012, 37(4): 559-573.
[12] Freund Y, Schapire R E. Large Margin Classification Using the Perceptron Algorithm [J]. Machine Learning, 1999, 37(3): 277-296.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
PDF(730 KB)
