Recent studies applied Parameter Efficient Fine-Tuning techniques (PEFTs) to efficiently narrow the performance gap between pre-training and downstream. There are two important factors for various PEFTs, namely, the accessible data size and fine-tunable parameter size. A natural expectation for PEFTs is that the performance of various PEFTs is positively related to the data size and fine-tunable parameter size. However, according to the evaluation of five PEFTs on two downstream vision-language (VL) tasks, we find that such an intuition holds only if the downstream data and task are not consistent with pre-training. For downstream fine-tuning consistent with pre-training, data size no longer affects the performance, while the influence of fine-tunable parameter size is not monotonous. We believe such an observation could guide the choice of training strategy for various PEFTs.

翻译：近期研究应用参数高效微调技术（PEFTs）有效缩小了预训练与下游任务之间的性能差距。影响各类PEFT方法的有两个重要因素，即可访问的数据规模与可微调参数规模。对于PEFTs而言，一个自然的预期是：各类PEFTs的性能与数据规模和可微调参数规模呈正相关。然而，通过对两种下游视觉语言（VL）任务上五种PEFTs的评估，我们发现这种直觉仅在当下游数据和任务与预训练不一致时成立。对于与预训练一致的下游微调，数据规模不再影响性能，而可微调参数规模的影响也并非单调递增。我们相信这一观察结果可为不同PEFTs的训练策略选择提供指导。