微博作为一种新型的社会媒体,以其信息的高实时性、话题动态关注、传播速度快的特点,逐渐被人们所接受和使用。筛选出相关话题的微博信息,帮助用户关注话题的动态发展,成为迫切需要解决的问题。由于微博信息篇幅极短、包含的信息和特征少等特点,为相关话题微博信息的筛选带来了新的挑战,而传统的文本分类技术已不再适用。该文提出了基于信息熵的筛选规则学习算法,利用学习得到的规则对微博信息进行有效的筛选。算法利用信息熵来评价规则的好坏,同时基于模拟退火的随机策略使算法中的规则选择避免了过于贪心。分别通过来自新浪微博的约九万条标注数据和TREC2011中约三千条特定话题的标注数据进行实验,该文算法相比于CPAR和SVM算法,学习得到的规则在筛选时取得了较高的F值。
Abstract
Microblog as a new social media plays more and more important role in current life due to its real time, trends and spreading of information. The issue that filtering tweets according to a concerning topic for tracking its trends is of substantial significance to the users. Since a tweet is extremely short, containing less information and textual features, how to filter the short tweets becomes a challenge in that the traditional text classification is no longer applicable. In this paper, we proposed a entropy-based classification rule learning algorithm to filter tweets by topics. The experimental results on nearly 90 000 tweets and 3 000 officially labeled tweets from Sina Weibo and TREC 2011 show that our algorithm achieves higher F-score in filtering tweets by topics than CPAR and SVM algorithms.
Key wordstweets filtering; rule mining; information entropy
关键词
微博信息筛选 /
规则学习 /
信息熵
{{custom_keyword}} /
Key words
tweets filtering /
rule mining /
information entropy
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 中国互联网络信息中心. 第28次中国互联网络发展状况统计报告[OL]. 2011.download.chinagate.cn/ch/pdf/hulianwang.pdf
[2] A. Mathes. Folksonomies—cooperative classification and communication through shared metadata[J]. Tech. Rep., UIUC, 2004.
[3] S. Ahmed, F. Mithun. Word stemming to enhance spam filtering[C]//Proceedings of CEAS, July 2004.
[4] S. Banerjee, K. Ramanathan, A. Gupta. Clustering Short Texts using Wikipedia[C]//Proceedings of the SIGIR, ACM,2007:787-788.
[5] B. Sriram, D. Fuhry, E. Demir, et al. Short text classification in twitter to improve information filtering[C]//Proceedings of the ACM SIGIR 2010 Posters and Demos. ACM, 2010.
[6] A. Tumasjan, T. O. Sprenger, P. G. Sandner, et al. Predicting elections with Twitter: What 140 characters reveal about political sentiment[C]//Proceedings of International AAAI Conference on Weblogs and Social Media, Washington, D.C., 2010.
[7] Go A., R. Bhayani, L. Huang. Twitter sentiment classification using distant supervision[R]. Technical Report, Stanford Digital Library Technologies Project. 2009.
[8] J. Hurlock, M. L. Wilson. Searching Twitter: Separating the Tweet from the Chaff[C]//Proceedings of AAAI, 2011.
[9] K. Lee, B. D. Eoff, J. Caverlee. Seven Months with the Devils: A Long-Term Study of Content Polluters on Twitter[C]//Proceedings of AAAI, 2011.
[10] S. Asur, B. A. Huberman. Predicting the Future with Social Media[J]. Arxiv preprint arXiv,2010.
[11] T. Sakaki, M. Okazaki, Y. Matsuo. Earthquake Shakes Twitter Users: Real-time Event Detection by Social Sensors[C]//Proceedings of WWW 2010: the International World Wide Web Conference, 2010: 851-860.
[12] B. Boser, I. Guyon, V. N. Vapnik. A training algorithm for optimal margin classifiers[C]//Proceedings of Fifth AnnualWorkshop on Computational Learning Theory, ACM Press: San Mateo, CA, 1992: 144-152.
[13] A. Genkin, D. Lewis, D. Madigan. Large-scale bayesian logistic regression for text categorization[J]. Technometrics.2007,49(3):291-304.
[14] J. R. Quinlan, R. M. Cameron-Jones. FOIL: A midterm report[C]//Proceedings of 1993 European Conf. Machine Learning, Vienna, Austria, 1993: 3-20.
[15] X. Yin, J. Han. CPAR: Classification based on predictive association rules[C]//Proceedings of 2003 SIAM Int. Conf. Data Mining (SDM03), San Francisco, CA. 2003: 331-335.
[16] C. Grier, K. Thomas, V. Paxson, et al. The underground in 140 characters or less[C]//Proceedings of ACM CCS10, Chicago, IL, Oct. 2010.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金资助项目(60903139,60873243,60933005);国家863计划重点资助项目(2010AA012502,2010AA012503)
{{custom_fund}}
PDF(1389 KB)
