An entropy-based hierarchical search result clustering method by utilizing augmented information

Abstract

Because of the improvement of the technology of search engines, and the massively increase of the number of web pages, the results returned by the search engines are always mixed and disordered. Especially for the queries with multiple topics, the mixed and disorderly situation of the search results would be more obvious. The search engines can return information of several hundred to thousand of the pages' titles, snippets and URLs. Almost all of the technologies about search result clustering must attain further information from the contents of the returned lists. However, long execution time is not permitted for a real-time clustering system. In this paper we propose some methods with better efficiency to improve the previous technologies. We utilize and augment information that search engines returned and use entropy calculation to attain the term distribution in snippets. We also propose several new methods to attain better clustered search results and reduce execution time. Our experiments indicate that these proposed methods obtain the better clustered results. © 2008 Springer-Verlag Berlin Heidelberg.