The quality of machine learning models depends heavily on their training data. Selecting high-quality, diverse training sets for large language models (LLMs) is a difficult task, due to the lack of cheap and reliable quality metrics. While querying existing LLMs for document quality is common, this is not scalable to the large number (billions) of documents used in training. Instead, practitioners often use classifiers trained on sparse quality signals. In this paper, we propose a text-embedding-based hierarchical clustering approach that adaptively selects the documents to be evaluated by the LLM to estimate cluster quality. We prove that our method is query efficient: under the assumption that the hierarchical clustering contains a subtree such that each leaf cluster in the tree is pure enough (i.e., it mostly contains either only good or only bad documents), with high probability, the method can correctly predict the quality of each document after querying a small number of documents. The number of such documents is proportional to the size of the smallest subtree with (almost) pure leaves, without the algorithm knowing this subtree in advance. Furthermore, in a comprehensive experimental study, we demonstrate the benefits of our algorithm compared to other classifier-based filtering methods.

翻译：机器学习模型的质量在很大程度上取决于其训练数据。由于缺乏廉价且可靠的质量评估指标，为大型语言模型（LLM）筛选高质量、多样化的训练集是一项具有挑战性的任务。虽然通常通过查询现有LLM来评估文档质量，但这种方法难以扩展到训练所使用的海量文档（数十亿级别）。实践中，研究者往往采用基于稀疏质量信号训练的分类器进行筛选。本文提出一种基于文本嵌入的层次聚类方法，该方法自适应地选择需要由LLM评估的文档以估计聚类质量。我们证明了该方法具有查询高效性：在层次聚类包含某个子树的假设下（该子树的每个叶簇具有足够纯度，即主要包含优质或劣质文档），本方法能以高概率在查询少量文档后正确预测每个文档的质量。所需查询的文档数量与具有（近似）纯净叶节点的最小子树规模成正比，且算法无需预先知晓该子树结构。此外，通过全面的实验研究，我们验证了该算法相较于其他基于分类器的过滤方法的优势。