In the field of large models (LMs) for natural language processing (NLP) and computer vision (CV), Parameter-Efficient Fine-Tuning (PEFT) has emerged as a resource-efficient method that modifies a limited number of parameters while keeping the pretrained weights fixed. This paper investigates the traditional PEFT approach, which applies modifications to all position indices, and questions its necessity. We introduce a new paradigm called Token-Selective PEFT (TS-PEFT), in which a function S selectively applies PEFT modifications to a subset of position indices, potentially enhancing performance on downstream tasks. Our experimental results reveal that the indiscriminate application of PEFT to all indices is not only superfluous, but may also be counterproductive. This study offers a fresh perspective on PEFT, advocating for a more targeted approach to modifications and providing a framework for future research to optimize the fine-tuning process for large models.

翻译：在自然语言处理（NLP）和计算机视觉（CV）领域的大模型（LMs）中，参数高效微调（PEFT）已成为一种资源高效的方法，它仅修改有限数量的参数，同时保持预训练权重固定。本文研究了传统的PEFT方法，该方法对所有位置索引均施加修改，并质疑其必要性。我们提出了一种称为令牌选择性PEFT（TS-PEFT）的新范式，其中函数S选择性地将PEFT修改应用于位置索引的子集，可能提升下游任务的性能。我们的实验结果表明，不加区分地对所有索引应用PEFT不仅多余，甚至可能适得其反。本研究为PEFT提供了新的视角，倡导采用更具针对性的修改方法，并为未来优化大模型微调过程的研究提供了一个框架。