Stability of K-Means Clustering

Abstract

We phrase K-means clustering as an empirical risk minimization procedure over a class ℋK and explicitly calculate the covering number for this class. Next, we show that stability of K-means clustering is characterized by the geometry of ℋK with respect to the underlying distribution. We prove that in the case of a unique global minimizer, the clustering solution is stable with respect to complete changes of the data, while for the case of multiple minimizers, the change of Ω(n1/2) samples defines the transition between stability and instability. While for a finite number of minimizers this result follows from multinomial distribution estimates, the case of infinite minimizers requires more refined tools. We conclude by proving that stability of the functions in ℋK implies stability of the actual centers of the clusters. Since stability is often used for selecting the number of clusters in practice, we hope that our analysis serves as a starting point for finding theoretically grounded recipes for the choice of K.