Despite the remarkable success of large-scale Language Models (LLMs) such as GPT-3, their performances still significantly underperform fine-tuned models in the task of text classification. This is due to (1) the lack of reasoning ability in addressing complex linguistic phenomena (e.g., intensification, contrast, irony etc); (2) limited number of tokens allowed in in-context learning. In this paper, we introduce Clue And Reasoning Prompting (CARP). CARP adopts a progressive reasoning strategy tailored to addressing the complex linguistic phenomena involved in text classification: CARP first prompts LLMs to find superficial clues (e.g., keywords, tones, semantic relations, references, etc), based on which a diagnostic reasoning process is induced for final decisions. To further address the limited-token issue, CARP uses a fine-tuned model on the supervised dataset for $k$NN demonstration search in the in-context learning, allowing the model to take the advantage of both LLM's generalization ability and the task-specific evidence provided by the full labeled dataset. Remarkably, CARP yields new SOTA performances on 4 out of 5 widely-used text-classification benchmarks, 97.39 (+1.24) on SST-2, 96.40 (+0.72) on AGNews, 98.78 (+0.25) on R8 and 96.95 (+0.6) on R52, and a performance comparable to SOTA on MR (92.39 v.s. 93.3). More importantly, we find that CARP delivers impressive abilities on low-resource and domain-adaptation setups. Specifically, using 16 examples per class, CARP achieves comparable performances to supervised models with 1,024 examples per class.

翻译：尽管大规模语言模型（如GPT-3）取得了显著成功，但在文本分类任务中其性能仍明显低于微调模型。原因在于：（1）处理复杂语言现象（如强化、对比、反讽等）时缺乏推理能力；（2）上下文学习允许的标记数量有限。本文提出线索与推理提示（CARP）方法。CARP采用渐进式推理策略，专门针对文本分类涉及的复杂语言现象：首先引导大语言模型寻找表层线索（如关键词、语气、语义关系、指代等），基于此诱导出诊断性推理过程以做出最终决策。为解决标记数量限制问题，CARP在有监督数据集上使用微调模型进行k近邻演示搜索，使模型既能利用大语言模型的泛化能力，又能获取完整标注数据集提供的任务特定证据。值得注意的是，CARP在5个广泛使用的文本分类基准测试中4个取得最新最优性能：SST-2上97.39（+1.24）、AGNews上96.40（+0.72）、R8上98.78（+0.25）、R52上96.95（+0.6），并在MR上达到与最优性能相当的水平（92.39对比93.3）。更重要的是，我们发现CARP在低资源和领域自适应场景中展现出令人印象深刻的能力：具体而言，每类仅使用16个样本，CARP即可达到每类1,024个样本的有监督模型的同等性能。