Modern computer vision is converging on a closed loop in which perception, reasoning and generation mutually reinforce each other. However, this loop remains incomplete: the top-down influence of high-level reasoning on the foundational learning of low-level perceptual features is not yet underexplored. This paper addresses this gap by proposing a new paradigm for pretraining foundation models in downstream domains. We introduce Visual insTruction Pretraining (ViTP), a novel approach that directly leverages reasoning to enhance perception. ViTP embeds a Vision Transformer (ViT) backbone within a Vision-Language Model and pretrains it end-to-end using a rich corpus of visual instruction data curated from target downstream domains. ViTP is powered by our proposed Visual Robustness Learning (VRL), which compels the ViT to learn robust and domain-relevant features from a sparse set of visual tokens. Extensive experiments on 16 challenging remote sensing and medical imaging benchmarks demonstrate that ViTP establishes new state-of-the-art performance across a diverse range of downstream tasks. The code is available at https://github.com/zcablii/ViTP.

翻译：现代计算机视觉正朝着感知、推理与生成相互增强的闭环范式发展。然而，该闭环目前仍不完整：高层推理对底层感知特征基础学习的自上而下影响尚未得到充分探索。本文针对这一空白，提出了一种面向下游领域的基础模型预训练新范式。我们引入了视觉指令预训练（ViTP），这是一种直接利用推理来增强感知的新方法。ViTP将Vision Transformer（ViT）骨干网络嵌入到视觉-语言模型中，并利用从目标下游领域精心构建的丰富视觉指令数据对其进行端到端预训练。ViTP的核心是我们提出的视觉鲁棒性学习（VRL）机制，该机制迫使ViT从稀疏的视觉标记中学习鲁棒且与领域相关的特征。在16个具有挑战性的遥感与医学影像基准测试上进行的大量实验表明，ViTP在多种下游任务中均取得了最先进的性能。代码已发布于 https://github.com/zcablii/ViTP。