Prompt Tuning is emerging as a scalable and cost-effective method to fine-tune Pretrained Language Models (PLMs). This study benchmarks the performance and computational efficiency of Prompt Tuning and baseline methods on a multi-label text classification task. This is applied to the use case of classifying companies into an investment firm's proprietary industry taxonomy, supporting their thematic investment strategy. Text-to-text classification with PLMs is frequently reported to outperform classification with a classification head, but has several limitations when applied to a multi-label classification problem where each label consists of multiple tokens: (a) Generated labels may not match any label in the industry taxonomy; (b) During fine-tuning, multiple labels must be provided in an arbitrary order; (c) The model provides a binary decision for each label, rather than an appropriate confidence score. Limitation (a) is addressed by applying constrained decoding using Trie Search, which slightly improves classification performance. All limitations (a), (b), and (c) are addressed by replacing the PLM's language head with a classification head. This improves performance significantly, while also reducing computational costs during inference. The results indicate the continuing need to adapt state-of-the-art methods to domain-specific tasks, even in the era of PLMs with strong generalization abilities.

翻译：提示微调正成为一种可扩展且成本效益高的预训练语言模型微调方法。本研究在多标签文本分类任务上，对提示微调及基线方法的性能与计算效率进行了基准测试。该方法应用于将企业分类至投资公司专有行业分类体系的实际场景，以支持其主题投资策略。基于预训练语言模型的文本到文本分类常被报道优于使用分类头的分类方法，但应用于每个标签由多个令牌构成的多标签分类问题时存在若干局限：（a）生成标签可能无法匹配行业分类体系中的任何标签；（b）微调过程中，多个标签必须以任意顺序提供；（c）模型对每个标签仅提供二元决策，而非适当的置信度分数。针对局限（a），通过应用基于Trie搜索的约束解码加以解决，该方法可略微提升分类性能。将所有局限（a）、（b）、（c）通过将预训练语言模型的语言头替换为分类头来解决，此举在显著提升性能的同时，也降低了推理阶段的计算成本。研究结果表明，即便在具备强大泛化能力的预训练语言模型时代，仍需将前沿方法适配至领域特定任务。