摘要
随着短信息应用的普及,用户、运营商及政府管理部门均迫切需要智能短信处理工具。语料库是研究算法,开发系统,测试性能等必不可少的基础资源。但受到技术、版权保护、隐私权利等种种原因,目前还没有公开的标准短信息语料库。SMS-2008标注短信息库是本项目组在国内外率先建立的多用途中文短信息语料库,它包括原始语料库、预处理语料库、隐私标注语料库、内容标注语料库、错误标注语料库等。该语料库可用于短信语言现象研究、短信分类过滤算法研究、隐私保护算法研究、自动纠错算法研究等。
Abstract
With the popularity of short messages, smart SMS tools are urgently demanded by users, operators and government departments. However, there is no open standard SMS corpus, which is an indispensable resource for the algorithm research, system development and performance test etc, due to the technology, the copyright protection, the privacy right and other various reasons. SMS-2008, as an annotated Chinese SMS Corpus, takes the lead in establishing a multi-purpose Chinese text message corpus, which includes the original corpus, privacy tagged corpus, content tagged corpus, errors tagged corpus. This Corpus can be applied in the research of SMS language, SMS classification, privacy protection algorithm or automatically correcting system.
Key words computer application; Chinese information processing; Chinese short message; tagged corpus
关键词
计算机应用 /
中文信息处理 /
中文短信息 /
标注语料库
{{custom_keyword}} /
Key words
computer application /
Chinese information processing /
Chinese short message /
tagged corpus
{{custom_keyword}} /
马旭,徐蔚然,郭军,胡日勒.
SMS-2008标注中文短信息库. 中文信息学报. 2009, 23(4): 22-27
MA Xu, XU Weiran, GUO Jun, HU Rile
(. Peking University Health Science Center, Beijing 0008, China;
. School of Information and Communication Engineering, Beijing University of Posts and Telecommunications,
Beijing 00876, China; . Nokia Research Center(China), Beijing 000, China)
.
SMS-2008: An Annotated Chinese Short Messages Corpus. Journal of Chinese Information Processing. 2009, 23(4): 22-27
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Graeme Kennedy. An Introduction to Corpus Linguistics[M]. Beijing: Foreign Language Teaching and Research Press, 2000.
[2] GUO Jun. The classification on short message [J]. The Journal of China Universities of Posts and Telecommunications, 2007,14(21): 111-114.
[3] Yang Zhen,Nie Xiang-fei,Xu Wei-ran,et al.Application of the character-based statistical method in text categorization [C]//Proceeding of 2006 International Conference on Computational Intelligence and Security,2006: 1412-1417.
[4] 冯志伟. 中国语料库研究的历史与现状[J]. Journal of Chinese Language and Computing, 2002,12(1): 43-62.
[5] 俞士汶,朱学锋,王惠,等. 现代汉语语法信息词典详解[M]. 北京:清华大学出版社,1998.
[6] 俞士汶,段慧明,朱学峰, 等. 北京大学现代汉语语料库基本加工规范[J]. 中文信息学报,2002,16(5): 49-64.
[7] 龚才春.短文本语言计算的关键技术研究[D]. 中国科学院研究生院(计算技术研究所),2008.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金资助项目(60675001);国家863计划资助项目(2007AA01Z417)
{{custom_fund}}
PDF(393 KB)
