IDBP: A distributed min-cut density-balanced algorithm for incremental web-pages ranking

Abstract

A link analysis on a distribute system is a viable choice to evaluate relationships between web-pages in a large web-graph. Each computational processor in the system contains a partial local web-graph and it locally performs web ranking. Since a distributed web ranking is generally incur penalties on execution times and accuracy from data synchronization, a web-graph can preliminary partitioned with a desired structure before a link analysis algorithm is started to improve execution time and accuracy. However, in the real-word situation, the numbers of web-pages in the web-graph can be continuously increased. Therefore, a link analysis algorithm has to re-partition a web-graph and re-perform web-pages ranking every time when the new web-pages are collected. In this paper, an efficient distributed web-pages ranking algorithm with min-cut density-balanced partitioning is proposed to improve the execution time of this scenario. The algorithm will re-partition the web-graph and re-perform the web-pages ranking only when necessary. The experimental results show that the proposed algorithm outperform in terms of the ranking’s execution times and the ranking’s accuracy.