该文探索了古代汉语,特别是先秦文献的词切分及词性标注。首先对《左传》文本进行了词汇处理(分词和词性标注)和考察分析,然后采用条件随机场模型(CRF),进行自动分词、词性标注、分词标注一体化的对比实验。结果表明,一体化分词比单独分词的准确率和召回率均有明显提高,开放测试的F值达到了94.60%;一体化词性标注的F值达到了89.65%,比传统的先分词后标注的“两步走”方法有明显提高。该项研究可以服务于古代汉语词汇研究和语料库建设,以弥补人工标注的不足。

This paper explores the cross field between NLP and ancient Chinese, particularly the pre-Qin documents. The text of "Zuo Zhuan" is firstly analyzed after manual segmentationand POS tagging. Then the Conditional Random Fields model (CRF) is adopted for the word segmentation (WS), POS tagging (PT) and a unified process of WS and PT, respectively. The precision and recall of the unified approach are much higher than the independent WS and PT in the open test, with a F-score of 94.60% in WS and 89.65% in PT. This method is suitable for the study of ancient Chinese vocabulary and corpus construction, and can be applied to compensatethe manual tagging.
Key wordscomputer application; Chinese information processing; Pre-Qin Chinese; word segmentation; POS tagging; Zuo Zhuan; conditional random fields model