Current visual representation learning remains bifurcated: vision-language models (e.g., CLIP) excel at global semantic alignment but lack spatial precision, while self-supervised methods (e.g., MAE, DINO) capture intricate local structures yet struggle with high-level semantic context. We argue that these paradigms are fundamentally complementary and can be integrated into a principled multi-task framework, further enhanced by dense spatial supervision. We introduce MTV, a multi-task visual pretraining framework that jointly optimizes a shared backbone across vision-language contrastive, self-supervised, and dense spatial objectives. To mitigate the need for manual annotations, we leverage high-capacity "expert" models -- such as Depth Anything V2 and OWLv2 -- to synthesize dense, structured pseudo-labels at scale. Beyond the framework, we provide a systematic investigation into the mechanics of multi-task visual learning, analyzing: (i) the marginal gain of each objective, (ii) task synergies versus interference, and (iii) scaling behavior across varying data and model scales. Our results demonstrate that MTV achieves "best-of-both-worlds" performance, significantly enhancing fine-grained spatial reasoning without compromising global semantic understanding. Our findings suggest that multi-task learning, fueled by high-quality pseudo-supervision, is a scalable path toward more general visual encoders.

翻译：当前视觉表征学习仍存在分野：视觉-语言模型（如CLIP）擅长全局语义对齐但缺乏空间精度，而自监督方法（如MAE、DINO）能捕捉精细局部结构却难以理解高层语义语境。我们认为这两种范式本质互补，可通过密集空间监督增强，整合为原则性的多任务框架。本文提出MTV——一种多任务视觉预训练框架，通过联合优化视觉-语言对比、自监督与密集空间目标来训练共享主干网络。为减少人工标注依赖，我们利用高性能"专家"模型（如Depth Anything V2和OWLv2）大规模合成结构化密集伪标签。除框架设计外，我们系统探究了多任务视觉学习机制，分析：（1）各目标的边际增益，（2）任务协同与干扰效应，（3）不同数据与模型规模下的扩展规律。实验表明MTV实现了"双赢"性能，在保持全局语义理解的同时显著提升细粒度空间推理能力。我们的研究证实：借助高质量伪监督的多任务学习，是构建通用视觉编码器的可扩展路径。