The mining and extraction of primary informative blocks and data objects from systematic Web pages

With the fast development of Internet, the Web has already been an enormous database so far, which contains extremely abundant information. Most of Web pages are represented their content by using a list of objects, such as search engine results, product information of shopping Web sites and so on, and these objects form the primary information of each page. In this paper, we focus on the issues of mining primary information and the constituted object groups. The system is divided into three major phases: (1) By transforming each Web page into corresponding tree structures, our system can visit all regions of the Web page in an efficient way, and detects the informative parts. (2) We design and quantize several novel features according to the characters of regions of a Web page. (3) A weighting model is proposed that calculates the important degree of each region, we then extract the primary information of the Web pages. The experimental result proves our system can be applied to a large number of Web pages with different themes and styles to find the correct primary information and the list of corresponding objects.