We consider the task of few-shot intent detection, which involves training a deep learning model to classify utterances based on their underlying intents using only a small amount of labeled data. The current approach to address this problem is through continual pre-training, i.e., fine-tuning pre-trained language models (PLMs) on external resources (e.g., conversational corpora, public intent detection datasets, or natural language understanding datasets) before using them as utterance encoders for training an intent classifier. In this paper, we show that continual pre-training may not be essential, since the overfitting problem of PLMs on this task may not be as serious as expected. Specifically, we find that directly fine-tuning PLMs on only a handful of labeled examples already yields decent results compared to methods that employ continual pre-training, and the performance gap diminishes rapidly as the number of labeled data increases. To maximize the utilization of the limited available data, we propose a context augmentation method and leverage sequential self-distillation to boost performance. Comprehensive experiments on real-world benchmarks show that given only two or more labeled samples per class, direct fine-tuning outperforms many strong baselines that utilize external data sources for continual pre-training. The code can be found at https://github.com/hdzhang-code/DFTPlus.

翻译：我们研究少样本意图检测任务，该任务旨在利用少量标注数据训练深度学习模型，根据话语的潜在意图对其进行分类。当前解决该问题的主流方法是采用持续预训练策略，即在将预训练语言模型（PLMs）用作话语编码器训练意图分类器之前，先在外部资源（如对话语料库、公开意图检测数据集或自然语言理解数据集）上对其进行微调。本文证明持续预训练或许并非必要，因为PLMs在该任务上的过拟合问题可能并未如预期般严重。具体而言，我们发现与采用持续预训练的方法相比，直接对仅有少量标注示例的PLMs进行微调已能取得相当的结果，且随着标注数据量的增加，性能差距迅速缩小。为最大化利用有限的可用数据，我们提出了一种上下文增强方法，并利用序列自蒸馏技术提升性能。在真实基准数据集上的综合实验表明，当每类仅有2个或更多标注样本时，直接微调的性能优于许多利用外部数据源进行持续预训练的强基线方法。代码可在https://github.com/hdzhang-code/DFTPlus获取。