Machine Learning (ML) in low-data settings remains an underappreciated yet crucial problem. This challenge is pronounced in low-to-middle income countries where access to large datasets is often limited or even absent. Hence, data augmentation methods to increase the sample size of datasets needed for ML are key to unlocking the transformative potential of ML in data-deprived regions and domains. Unfortunately, the limited training set constrains traditional tabular synthetic data generators in their ability to generate a large and diverse augmented dataset needed for ML tasks. To address this technical challenge, we introduce CLLM, which leverages the prior knowledge of Large Language Models (LLMs) for data augmentation in the low-data regime. While diverse, not all the data generated by LLMs will help increase utility for a downstream task, as for any generative model. Consequently, we introduce a principled curation process, leveraging learning dynamics, coupled with confidence and uncertainty metrics, to obtain a high-quality dataset. Empirically, on multiple real-world datasets, we demonstrate the superior performance of LLMs in the low-data regime compared to conventional generators. We further show our curation mechanism improves the downstream performance for all generators, including LLMs. Additionally, we provide insights and understanding into the LLM generation and curation mechanism, shedding light on the features that enable them to output high-quality augmented datasets. CLLM paves the way for wider usage of ML in data scarce domains and regions, by allying the strengths of LLMs with a robust data-centric approach.

翻译：机器学习在低数据场景下仍是一个未受足够重视但至关重要的问题。这一挑战在低收入和中等收入国家尤为突出，这些国家获取大型数据集的机会通常有限甚至完全缺失。因此，为增加机器学习所需数据集样本量的数据增强方法，是释放机器学习在数据匮乏地区和领域变革潜力的关键。然而，有限的训练集限制了传统表格合成数据生成器生成机器学习任务所需的大规模、多样化增强数据集的能力。为应对这一技术挑战，我们提出了CLLM，该方法利用大型语言模型的先验知识在低数据场景下进行数据增强。尽管生成的数据多样性丰富，但与任何生成模型一样，并非所有由LLM生成的数据都能提升下游任务的效用。因此，我们引入了一种基于学习动态、结合置信度与不确定性度量的原则性策展流程，以获得高质量数据集。在多个真实世界数据集上的实验表明，与常规生成器相比，LLM在低数据场景下展现出更优性能。我们进一步证明，所提出的策展机制对所有生成器（包括LLM）的下游性能均有提升。此外，我们提供了对LLM生成与策展机制的见解与理解，揭示了使其能输出高质量增强数据集的特征。通过结合LLM的优势与稳健的数据中心方法，CLLM为在数据匮乏领域和区域更广泛地应用机器学习铺平了道路。