We propose a label-free approach to adapt powerful but generic vision foundation models to specialized scientific domains. Standard supervised fine-tuning is often ill-suited to these settings: labels are scarce, and task-specific training can collapse the model's generality and hurt robustness. We instead leverage metadata to adapt representations to new domains in a self-supervised manner. Our method, FINO, combines a standard self-supervised objective with flexible metadata guidance that handles both highly granular discrete metadata and continuous metadata. It encourages the representation to preserve informative factors while suppressing spurious ones. Across subcellular fluorescence microscopy, Earth observation, wildlife monitoring, and medical imaging, FINO consistently outperforms standard unsupervised domain adaptation and fully supervised adaptation. It also exceeds highly-specialized domain-specific state of the art, while using no task labels for backbone adaptation and only lightweight probes for supervision.

翻译：我们提出一种无标签方法，用于将功能强大但通用的视觉基础模型适配至专业科学领域。标准有监督微调通常不适用于这些场景：标签稀缺，且任务特定训练可能破坏模型的泛化性并损害鲁棒性。相反，我们利用元数据以自监督方式将表征适配至新领域。我们的方法FINO将标准自监督目标与灵活的元数据指导相结合，既能处理高度细粒度的离散元数据，也能处理连续元数据。该方法鼓励表征保留信息性因素，同时抑制虚假因素。在亚细胞荧光显微镜、地球观测、野生动物监测和医学影像领域，FINO持续优于标准的无监督域适应和全监督适应方法。它在不使用任务标签进行骨干网络适配、仅使用轻量级探头进行监督的情况下，甚至超越了高度专业化领域的最新技术水平。