林祥辉,张瑾,黄康平,许磊,许洪波,程学旗,程工. 一种基于内存的高效在线数据处理服务框架[J]. 中文信息学报, 2014, 28(1): 80-86.
LIN Xianghui, ZHANG Jin, HUANG Kangping, XU Lei, XU Hongbo, CHENG Xueqi, CHENG Gong. An Effective On-line Data Process and Service Framework Based on Memory. , 2014, 28(1): 80-86.
1. Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China; 2. University of Chinese Academy of Sciences, Beijing 100190, China; 3. National Computer Network Emergency Response Technical Team/Coordination Center, Beijing 100190, China
Abstract：Under the environment of big data, traditional database-centered data processing architecture cannot meet the requirement of high concurrency of read/write requests. At the same time, serial usages of data limit the effectiveness of data processing. This paper describes an effective on-line data process and service framework based on memory. This framework uses multi-index data access method and pub/sub data control mechanism to improve the effectiveness of data processing while reducing the interaction with the database. Experimental results show that the memory based on-line data process and service framework can significantly improve the response speed of database and shorten the latency of data processing.
[1] 李盛韬,余智华,程学旗,白硕. Web信息采集研究进展[J]. 计算机科学,2003(02): 151-171. [2] 郭岩,刘春阳,余智华,等. 网络舆情信息源影响力的评估研究[J]. 中文信息学报, 2011,25(3):64-71. [3] 曹鹏,李静远,满彤,等.Twitter中近似重复消息的判定方法研究[J].中文信息学报,2011,25(1):20-27. [4] 郭浩,陆余良.基于信息传播的微博用户影响力度量[C]//CCIR2011. [5] Fitzpatrick B. Distributed caching with memcached[J]. Journal Linux, 2004,124:7559. [6] Krakowiak, Sacha. “Whats middleware?”. ObjectWeb.org. Retrieved 2005-05-06. [7] 徐晶,许炜. 消息中间件综述[J]. 计算机工程, 2005, 33(16):73-76. [8] 李文逍,杨小虎. 基于分布式缓存的消息中间件存储模型[J]. 计算机工程,2010,36(13):93-95. [9] Birman K, Joseph T. Exploiting virtual synchrony in distributed systems[C]//Proceeding of SOSP 87 the eleventh ACM Symposium on Operating systems principles, 1987: 123-138. [10] Hasan, Souleiman. Approximate Semantic Matching of Heterogeneous Events [C]//Proceeding of the 6th ACM International Conference on Distributed Event-Based Systems 2012, 252-263. [11] Eugster P T, Felber P A, Guerraoui R. The Many Faces of Publish/Subscribe Proceeding of ACM Computing Surveys, 2003,35(2): 114. [12] The Apache Software Foundation ActiveMQ[DB/OL] http://activemq.apache.org/, 2012. [13] Snyder, Bruce, Bosanac, Dejan, et al. ActiveMQ in Action (1st ed.)[M], Manning Publications, 2010: 375. [14] Iyengar A, Jessani V, Chilanti M. WebSphere Business Integration Primer: Process Server, BPEL, SCA, and SOA 1st[M], IBM Press , 2007. [15] Kloppmann M. IBM Deutschland Entwicklung GmbH Business process choreography in WebSphere: Combining the power of BPEL and J2EE[J], IBM Systems Journal, 2004, 43(2): 270-296. [16] B.Mann. Worldwide Product Manager Providing a Backbone for Connectivity with SOA Messaging[M], 2009. [17] Kreps J, Narkhede N, Rao J. Kafka: a Distributed Messaging System for Log Processing[M], 2011. [18] Videla A, Williams J J W. RabbitMQ in Action:Distributed messaging for everyone[M]. 2012. [19] home page http://www.rabbitmq.com/[DB/OL], 2012. [20] Seeger M. Key-Value Stores: a practical overview Computer Science and Media[A], Ultra-Large-Sites September 2009. [21] Automatic Workload Repository (AWR) in Oracle Database[DB/OL], http://www.oracle-base.com/articles/10g/automatic-workload-repository-10g.php. [22] Oracle Database Performance Method page[M/CD]. [23] http://docs.oracle.com/cd/B19306_01/server.102/b28051/tdppt_method.htm[DB/OL].
2014, Vol. 28 
