GraSP: distributed streaming graph partitioning

Abstract

This paper presents a distributed, streaming graph parti- tioner, Graph Streaming Partitioner (GraSP), which makes partition decisions as each vertex is read from memory, sim- ulating an online algorithm that must process nodes as they arrive. GraSP is a lightweight high-performance comput- ing (HPC) library implemented in MPI, designed to be easily substituted for existing HPC partitioners such as ParMETIS. It is the rst MPI implementation for streaming partition- ing of which we are aware, and is empirically orders-of- magnitude faster than existing partitioners while providing comparable partitioning quality. We demonstrate the scala- bility of GraSP on up to 1024 compute nodes of NERSC's Edison supercomputer. Given a minute of run-time, GraSP can partition a graph three orders of magnitude larger than ParMETIS can.