该文提出了一种基于统计与正文特征的网页正文抽取方法。该方法继承了统计方法的优点,同时利用正文特征克服了原有基于统计的方法无法抽取多正文体网页的缺陷。源于多正文体在网页的DOM树中对应着正文区域下的多棵具有相似特征的正文子树,该文首先基于统计的方法获取一条正文路径,然后学习该路径的正文特征识别正文区域和子树主干,最后根据区域及该主干具有的正文特征进而得到完整的正文。实验表明该方法抽取单正文和多正文的精确率分别为94%和91%。

This paper presents a new method for content extraction from Web pages based on statistic and content-features. This method not only inherits the merits of the traditional statistic method, but also can extract the multi-body documents which can not be obtained by the pure statistic method. According to the fact that the multi-body documents are corresponding to multi-subtrees with the similar characteristics in the DOM tree of the web page, we first get a content path using the statistic method. Then, the content region and a trunk of subtree are modeled by the important features of the path, which are applied to get the whole information of the body content. Our experiment results show that the extraction precision of the single-body documents is 94%, and the multi-body documents is 91%.
Key words computer application; Chinese information processing; content extraction;single-body documents;multi-body documents