The development of largely human-annotated benchmarks has driven the success of deep neural networks in various NLP tasks. To enhance the effectiveness of existing benchmarks, collecting new additional input-output pairs is often too costly and challenging, particularly considering their marginal impact on improving the current model accuracy. Instead, additional or complementary annotations on the existing input texts in the benchmarks can be preferable as an efficient way to pay the additional human cost. In this paper, we investigate task-specific preferences between pairs of input texts as a new alternative way for such auxiliary data annotation. From 'pair-wise' comparisons with respect to the task, the auxiliary preference learning enables the model to learn an additional informative training signal that cannot be captured with 'instance-wise' task labels. To this end, we propose a novel multi-task learning framework, called prefer-to-classify (P2C), which can enjoy the cooperative effect of learning both the given classification task and the auxiliary preferences. Here, we provide three different ways to collect preference signals in practice: (a) implicitly extracting from annotation records (for free, but often unavailable), (b) collecting explicitly from crowd workers (high paid), or (c) pre-trained large language models such as GPT-3 (low paid). Given existing classification NLP benchmarks, we demonstrate that the proposed auxiliary preference learning via P2C on them is effective in improving text classifiers. Our codes are publicly available.

翻译：大规模人工标注基准数据集的发展推动了深度神经网络在各类自然语言处理任务中的成功。为增强现有基准数据集的效能，收集新的输入-输出对往往成本高昂且具有挑战性，尤其是考虑到其对提升当前模型准确率的边际影响有限。相比之下，对基准数据集中现有输入文本进行额外或补充性标注，可作为有效利用额外人力成本的更优方案。本文首次提出以输入文本对的任务特定偏好作为辅助数据标注的新范式。通过任务相关的“成对”比较，辅助偏好学习使模型能够捕获“实例级”任务标签无法提供的额外信息训练信号。为此，我们提出名为优先分类（P2C）的新型多任务学习框架，该框架可协同利用给定分类任务与辅助偏好学习的互补效应。实践中，我们提供了三种偏好信号采集方式：(a) 从标注记录中隐式提取（零成本但通常不可用），(b) 通过众包工作者显式采集（高成本），(c) 基于GPT-3等预训练大语言模型生成（低成本）。在现有分类NLP基准数据集上的实验表明，通过P2C框架实施的辅助偏好学习能有效提升文本分类器性能。我们的代码已开源。