Reliable zero-shot detection of out-of-distribution (OOD) inputs is critical for deploying vision-language models in open-world settings. However, the lack of labeled negatives in zero-shot OOD detection necessitates proxy signals that remain effective under distribution shift. Existing negative-label methods rely on a fixed set of textual proxies, which (i) sparsely sample the semantic space beyond in-distribution (ID) classes and (ii) remain static while only visual features drift, leading to cross-modal misalignment and unstable predictions. In this paper, we propose CoEvo, a training- and annotation-free test-time framework that performs bidirectional, sample-conditioned adaptation of both textual and visual proxies. Specifically, CoEvo introduces a proxy-aligned co-evolution mechanism to maintain two evolving proxy caches, which dynamically mines contextual textual negatives guided by test images and iteratively refines visual proxies, progressively realigning cross-modal similarities and enlarging local OOD margins. Finally, we dynamically re-weight the contributions of dual-modal proxies to obtain a calibrated OOD score that is robust to distribution shift. Extensive experiments on standard benchmarks demonstrate that CoEvo achieves state-of-the-art performance, improving AUROC by 1.33% and reducing FPR95 by 45.98% on ImageNet-1K compared to strong negative-label baselines.

翻译：在开放世界场景中部署视觉语言模型时，可靠的零样本分布外输入检测至关重要。然而，零样本OOD检测中缺乏带标注的负样本，因此需要能在分布偏移下保持有效的代理信号。现有的负标签方法依赖于固定的文本代理集合，这存在两个问题：(i) 对分布内类别之外的语义空间采样稀疏；(ii) 在视觉特征漂移时保持静态，导致跨模态失准和预测不稳定。本文提出CoEvo，一种无需训练和标注的测试时框架，通过双向、样本条件化的方式同时适配文本与视觉代理。具体而言，CoEvo引入代理对齐的协同演化机制，维护两个动态演化的代理缓存：在测试图像引导下动态挖掘上下文文本负样本，并迭代优化视觉代理，逐步实现跨模态相似度的重新对齐并扩大局部OOD边界。最后，我们动态重加权双模态代理的贡献度，从而获得对分布偏移具有鲁棒性的校准OOD分数。在标准基准上的大量实验表明，CoEvo取得了最先进的性能，在ImageNet-1K数据集上相较于强负标签基线，AUROC提升1.33%，FPR95降低45.98%。