We introduce LAMPO, a novel paradigm that leverages Large Language Models (LLMs) for solving few-shot multi-class ordinal classification tasks. Unlike conventional methods, which concatenate all demonstration examples with the test instance and prompt LLMs to produce the pointwise prediction, our framework uses the LLM as a preference machine that makes a relative comparative decision between the test instance and each demonstration. A self-supervised method is then introduced to aggregate these binary comparisons into the final ordinal decision. LAMPO addresses several limitations inherent in previous methods, including context length constraints, ordering biases, and challenges associated with absolute point-wise estimation. Extensive experiments on seven public datasets demonstrate LAMPO's remarkably competitive performance across a diverse spectrum of applications (e.g., movie review analysis and hate speech detection). Notably, in certain applications, the improvement can be substantial, exceeding 20% in an absolute term. Moreover, we believe LAMPO represents an interesting addition to the non-parametric application layered on top of LLMs, as it supports black-box LLMs without necessitating the outputting of LLM's internal states (e.g., embeddings), as seen in previous approaches.

翻译：本文提出LAMPO，一种利用大语言模型解决少样本多类序数分类任务的新范式。与传统方法将所有演示样本与测试实例拼接后提示LLM生成逐点预测不同，我们的框架将LLM作为偏好机，在测试实例与每个演示样本之间进行相对比较决策。随后引入自监督方法将这些二元比较聚合为最终的序数决策。LAMPO解决了先前方法固有的若干局限，包括上下文长度限制、排序偏差以及与绝对逐点估计相关的挑战。在七个公开数据集上的大量实验表明，LAMPO在多样化应用场景（如影评分析与仇恨言论检测）中均展现出显著竞争力。值得注意的是，在某些应用中改进幅度尤为显著，绝对性能提升超过20%。此外，我们认为LAMPO为基于LLM的非参数应用层提供了有价值的补充，因其支持黑盒LLM而无需像先前方法那样要求输出LLM内部状态（如嵌入向量）。