Large-scale Visual Instruction Tuning (VIT) has become a key paradigm for advancing the performance of vision-language models (VLMs) across various multimodal tasks. However, training on the large-scale datasets is computationally expensive and inefficient due to redundancy in the data, which motivates the need for multimodal data selection to improve training efficiency. Existing data selection methods for VIT either require costly training or gradient computation. Training-free alternatives often depend on proxy models or datasets, instruction-agnostic representations, and pairwise similarity with quadratic complexity, limiting scalability and representation fidelity. In this work, we propose ScalSelect, a scalable training-free multimodal data selection method with linear-time complexity with respect to the number of samples, eliminating the need for external models or auxiliary datasets. ScalSelect first constructs sample representations by extracting visual features most attended by instruction tokens in the target VLM, capturing instruction-relevant information. It then identifies samples whose representations best approximate the dominant subspace of the full dataset representations, enabling scalable importance scoring without pairwise comparisons. Extensive experiments across multiple VLMs, datasets, and selection budgets demonstrate that ScalSelect achieves over 97.5% of the performance of training on the full dataset using only 16% of the data, and even outperforms full-data training in some settings. The code is available at \href{https://github.com/ChangtiWu/ScalSelect}{ScalSelect}.

翻译：大规模视觉指令微调已成为提升视觉语言模型在多模态任务中性能的关键范式。然而，由于数据冗余，在大规模数据集上进行训练计算成本高昂且效率低下，这推动了对多模态数据选择以提升训练效率的需求。现有的视觉指令微调数据选择方法需要昂贵的训练或梯度计算过程。免训练替代方案通常依赖于代理模型或数据集、与指令无关的表征以及具有二次复杂度的成对相似性计算，限制了方法的可扩展性与表征保真度。本研究提出ScalSelect，一种可扩展的免训练多模态数据选择方法，其时间复杂度相对于样本数量呈线性增长，且无需外部模型或辅助数据集。ScalSelect首先通过提取目标视觉语言模型中指令标记最关注的视觉特征来构建样本表征，从而捕获与指令相关的信息。随后，该方法识别出那些表征最能逼近完整数据集表征主导子空间的样本，无需成对比较即可实现可扩展的重要性评分。在多种视觉语言模型、数据集及选择预算下的大规模实验表明，ScalSelect仅使用16%的数据即可达到全数据集训练性能的97.5%以上，在某些设定下甚至超越全数据训练效果。代码发布于\href{https://github.com/ChangtiWu/ScalSelect}{ScalSelect}。