Local Graph Edge Partitioning with a Two-Stage Heuristic Method

Abstract

Graph edge partitioning divides the edges of an input graph into multiple balanced partitions of a given size to minimize the sum of vertices that are cut, which is critical to the performance of distributed graph computation platforms. Existing graph partitioning methods can be classified into two categories: offline graph partitioning and streaming graph partitioning. The first category requires global information for a graph during the partitioning, which is expensive in terms of time and memory for large-scale graphs. The second category, however, creates partitions solely based on the received edge information, which may result in lower performance than the offline methods. Therefore, in this study, the concept of local graph partitioning is introduced from local community detection to consider only local information, i.e., a part of the graph, instead of the graph as a whole, during the partitioning. The characteristic of storing only local information is important because real-world graphs are often large in scale, or they increase incrementally. Based on this idea, we propose a two-stage local partitioning algorithm, where the partitioning process is divided into two stages according to the structural changes of the current partition, and two different strategies are introduced to deal with the respective stages. Experimental results with real-world graphs demonstrate that the proposed algorithm outperforms the rival algorithms in most cases, including the state-of-the-art algorithm METIS.