Oversampling is one of the most widely used approaches for addressing imbalanced classification. The core idea is to generate additional minority samples to rebalance the dataset. Most existing methods, such as SMOTE, require converting categorical variables into numerical vectors, which often leads to information loss. Recently, large language model (LLM)-based methods have been introduced to overcome this limitation. However, current LLM-based approaches typically generate minority samples with limited diversity, reducing robustness and generalizability in downstream classification tasks. To address this gap, we propose a novel LLM-based oversampling method designed to enhance diversity. First, we introduce a sampling strategy that conditions synthetic sample generation on both minority labels and features. Second, we develop a new permutation strategy for fine-tuning pre-trained LLMs. Third, we fine-tune the LLM not only on minority samples but also on interpolated samples to further enrich variability. Extensive experiments on 10 tabular datasets demonstrate that our method significantly outperforms eight SOTA baselines. The generated synthetic samples are both realistic and diverse. Moreover, we provide theoretical analysis through an entropy-based perspective, proving that our method encourages diversity in the generated samples.

翻译：过采样是解决不平衡分类问题最广泛使用的方法之一，其核心思想是生成额外的少数类样本以平衡数据集。现有方法（如SMOTE）大多需要将分类变量转换为数值向量，这常导致信息损失。近期，基于大语言模型（LLM）的方法被引入以克服这一局限。然而，当前基于大语言模型的方法生成的少数类样本多样性有限，降低了下游分类任务的鲁棒性和泛化能力。为解决这一问题，我们提出一种新颖的基于大语言模型的过采样方法，旨在增强多样性。首先，我们引入一种采样策略，使合成样本的生成同时以少数类标签和特征为条件。其次，我们开发了一种新的排列策略用于微调预训练的大语言模型。第三，我们不仅对少数类样本进行微调，还对插值样本进行微调，以进一步丰富变异性。在10个表格数据集上的大量实验表明，我们的方法显著优于八种最先进基线方法，生成的合成样本既真实又多样。此外，我们通过基于熵的视角提供理论分析，证明我们的方法能促进生成样本的多样性。