We present Label Space Reduction (LSR), a novel method for improving zero-shot classification performance of Large Language Models (LLMs). LSR iteratively refines the classification label space by systematically ranking and reducing candidate classes, enabling the model to concentrate on the most relevant options. By leveraging unlabeled data with the statistical learning capabilities of data-driven models, LSR dynamically optimizes the label space representation at test time. Our experiments across seven benchmarks demonstrate that LSR improves macro-F1 scores by an average of 7.0% (up to 14.2%) with Llama-3.1-70B and 3.3% (up to 11.1%) with Claude-3.5-Sonnet compared to standard zero-shot classification baselines. To reduce the computational overhead of LSR, which requires an additional LLM call at each iteration, we propose distilling the model into a probabilistic classifier, allowing for efficient inference.

翻译：我们提出了一种新颖的标签空间约简方法，旨在提升大语言模型在零样本分类任务中的性能。该方法通过系统性地排序与约简候选类别，迭代式地精炼分类标签空间，使模型能够聚焦于最相关的选项。LSR利用未标注数据，结合数据驱动模型的统计学习能力，在测试时动态优化标签空间的表示。我们在七个基准数据集上的实验表明，相较于标准的零样本分类基线，LSR使用Llama-3.1-70B模型将宏平均F1分数平均提升了7.0%（最高达14.2%），使用Claude-3.5-Sonnet模型平均提升了3.3%（最高达11.1%）。为降低LSR因每次迭代均需额外调用LLM而产生的计算开销，我们提出将模型蒸馏为一个概率分类器，从而实现高效的推理。