We answer the question: "Does local progress (on batches) imply global progress (on the entire dataset) for mini-batch $k$-means?". Specifically, we consider mini-batch $k$-means which terminates only when the improvement in the quality of the clustering on the sampled batch is below some threshold. Although at first glance it appears that this algorithm might execute forever, we answer the above question in the affirmative and show that if the batch is of size $\tilde{\Omega}((d/\epsilon)^2)$, it must terminate within $O(d/\epsilon)$ iterations with high probability, where $d$ is the dimension of the input, and $\epsilon$ is a threshold parameter for termination. This is true regardless of how the centers are initialized. When the algorithm is initialized with the $k$-means++ initialization scheme, it achieves an approximation ratio of $O(\log k)$ (the same as the full-batch version). Finally, we show the applicability of our results to the mini-batch $k$-means algorithm implemented in the scikit-learn (sklearn) python library.
翻译:我们回答了以下问题:“对于小批量 $k$-均值算法,局部进展(基于批次)是否意味着全局进展(基于整个数据集)?”具体而言,我们考虑当采样批次聚类质量的改善低于某个阈值时才终止的小批量 $k$-均值算法。尽管初看该算法可能无限执行,但我们给出了肯定回答,并证明如果批次大小满足 $\tilde{\Omega}((d/\epsilon)^2)$,则该算法以高概率在 $O(d/\epsilon)$ 次迭代内终止,其中 $d$ 是输入维度,$\epsilon$ 是终止的阈值参数。这一结论与初始中心点的选取方式无关。当使用 $k$-means++ 初始化方案初始化算法时,其近似比达到 $O(\log k)$(与全批量版本相同)。最后,我们展示了研究结果在 scikit-learn (sklearn) Python 库实现的小批量 $k$-均值算法中的适用性。