Topic generation for web document summarization

Over the past decade, more and more users of the Internet rely on the search engines to help them find the information they need. However, the information they find depends, to a large extent, on the ranking mechanism of the search engines they use. Not surprisingly, it, in general, consists of a large amount of information that is completely irrelevant. To help users of the Internet find the information they are looking for quickly, an efficient algorithm for building the summaries of a collection of documents found by a search engine in response to a user query, called DISCO (Distribution Scoring) is proposed. To demonstrate the performance of the proposed algorithm, Reuters-21578 text categorization collection and some search results from Google are used in our simulation. Moreover, several metrics such as coverage, overlap, and the computation time are employed in evaluating the quality and quantity of the proposed algorithm. All our simulation results indicate that the proposed algorithm outperforms all the existing algorithms in terms of not only the usefulness of the summaries but also the running time.