Many learning problems hinge on the fundamental problem of subset selection, i.e., identifying a subset of important and representative points. For example, selecting the most significant samples in ML training cannot only reduce training costs but also enhance model quality. Submodularity, a discrete analogue of convexity, is commonly used for solving subset selection problems. However, existing algorithms for optimizing submodular functions are sequential, and the prior distributed methods require at least one central machine to fit the target subset. In this paper, we relax the requirement of having a central machine for the target subset by proposing a novel distributed bounding algorithm with provable approximation guarantees. The algorithm iteratively bounds the minimum and maximum utility values to select high quality points and discard the unimportant ones. When bounding does not find the complete subset, we use a multi-round, partition-based distributed greedy algorithm to identify the remaining subset. We show that these algorithms find high quality subsets on CIFAR-100 and ImageNet with marginal or no loss in quality compared to centralized methods, and scale to a dataset with 13 billion points.

翻译：摘要：许多学习问题都依赖于子集选择这一基础问题，即识别出重要且具代表性的点集。例如，在机器学习训练中选择最具代表性的样本，不仅能降低训练成本，还能提升模型质量。子模性作为凸性的离散对应概念，常被用于解决子集选择问题。然而，现有子模函数优化算法均为串行处理，其分布式方法至少需要一台中心机器来容纳目标子集。本文提出了一种新型分布式边界算法，通过放宽对目标子集需中心机器的要求，并提供了可证明的近似保证。该算法通过迭代计算最小与最大效用值的边界，筛选高质量点并剔除不重要点。当边界算法无法确定完整子集时，我们采用基于多轮分区的分布式贪心算法来识别剩余子集。实验表明，在CIFAR-100和ImageNet数据集上，这些算法相比集中式方法能以极小甚至为零的质量损失获得高质量子集，并可扩展至包含130亿个数据点的数据集。