Vision Transformers have emerged as powerful, scalable and versatile representation learners. To capture both global and local features, a learnable [CLS] class token is typically prepended to the input sequence of patch tokens. Despite their distinct nature, both token types are processed identically throughout the model. In this work, we investigate the friction between global and local feature learning under different pre-training strategies by analyzing the interactions between class and patch tokens. Our analysis reveals that standard normalization layers introduce an implicit differentiation between these token types. Building on this insight, we propose specialized processing paths that selectively disentangle the computational flow of class and patch tokens, particularly within normalization layers and early query-key-value projections. This targeted specialization leads to significantly improved patch representation quality for dense prediction tasks. Our experiments demonstrate segmentation performance gains of over 2 mIoU points on standard benchmarks, while maintaining strong classification accuracy. The proposed modifications introduce only an 8% increase in parameters, with no additional computational overhead. Through comprehensive ablations, we provide insights into which architectural components benefit most from specialization and how our approach generalizes across model scales and learning frameworks.

翻译：视觉Transformer已成为强大、可扩展且通用的表示学习器。为同时捕获全局与局部特征，通常会在图像块标记输入序列前添加一个可学习的[CLS]类别标记。尽管二者本质不同，这两类标记在模型全程处理中却采用相同方式。本研究通过分析类别标记与图像块标记间的交互作用，探究了不同预训练策略下全局与局部特征学习间的冲突。我们的分析表明，标准归一化层会在这两类标记间引入隐式区分。基于此发现，我们提出专用处理路径，选择性地解耦类别标记与图像块标记的计算流，特别是在归一化层和早期查询-键-值投影中。这种针对性专门化处理显著提升了密集预测任务中的图像块表示质量。实验证明，在保持强大分类精度的同时，该方法在标准基准测试上实现了超过2 mIoU点的分割性能提升。所提改进仅增加8%参数量，且未引入额外计算开销。通过系统消融实验，我们揭示了哪些架构组件从专门化中获益最大，以及该方法如何在不同模型规模和学习框架中保持泛化能力。