NOLGP: A Novel Optimized Large-Scale Graph Partitioning Algorithm

Abstract

In big data era, large-scale graphs with billions of vertices and trillions of edges have emerged in wide range of applications, such as social networks and knowledge graphs. For efficient analysis to a large-scale graph on big data/distributed platforms, a key pre-step is to divide vertices/edges of the graph into balanced partitions, i.e., kway balanced partitions. Although several graph partitioning algorithms have been proposed trying to solve this NP-hard problem, the existing algorithms suffer from different defects, such as poor partitioning efficiency and quality, and falling into local optima. To address these issues, this paper first models the problem of k-way partitioning large-scale graph as a multi-objective optimization problem, and proposes a novel optimized algorithm based on the introduced improved label propagation algorithm and a set of optimal strategies. We conducted experiments on real-world large-scale graphs, and the theoretical analysis and experimental results show that our algorithm outperforms the state-of-the-art algorithms, in both partitioning quality and efficiency.