Reliable zero-shot detection of out-of-distribution (OOD) inputs is critical for deploying vision-language models in open-world settings. However, the lack of labeled negatives in zero-shot OOD detection necessitates proxy signals that remain effective under distribution shift. Existing negative-label methods rely on a fixed set of textual proxies, which (i) sparsely sample the semantic space beyond in-distribution (ID) classes and (ii) remain static while only visual features drift, leading to cross-modal misalignment and unstable predictions. In this paper, we propose CoEvo, a training- and annotation-free test-time framework that performs bidirectional, sample-conditioned adaptation of both textual and visual proxies. Specifically, CoEvo introduces a proxy-aligned co-evolution mechanism to maintain two evolving proxy caches, which dynamically mines contextual textual negatives guided by test images and iteratively refines visual proxies, progressively realigning cross-modal similarities and enlarging local OOD margins. Finally, we dynamically re-weight the contributions of dual-modal proxies to obtain a calibrated OOD score that is robust to distribution shift. Extensive experiments on standard benchmarks demonstrate that CoEvo achieves state-of-the-art performance, improving AUROC by 1.33% and reducing FPR95 by 45.98% on ImageNet-1K compared to strong negative-label baselines.

翻译：可靠地零样本检测分布外输入对于在开放世界环境中部署视觉语言模型至关重要。然而，零样本分布外检测中缺乏标记负样本，需要依赖在分布偏移下仍保持有效的代理信号。现有负标签方法依赖于固定的文本代理集，这些代理：(i) 稀疏采样了分布内类别之外的语义空间；(ii) 仅在视觉特征漂移时保持静态，导致跨模态错位和不稳定预测。本文提出CoEvo——一种无需训练和标注的测试时框架，对文本和视觉代理进行双向、样本条件自适应调整。具体而言，CoEvo引入代理对齐的共演化机制，维护两个动态演化的代理缓存，该机制在测试图像引导下动态挖掘上下文文本负样本，并迭代优化视觉代理，逐步重新校准跨模态相似性并扩大局部分布外边界。最后，我们动态重加权双模态代理的贡献，以获得对分布偏移鲁棒的校准分布外评分。在标准基准上的大量实验表明，CoEvo达到了最先进性能，相比强负标签基线，在ImageNet-1K上将AUROC提升1.33%，并将FPR95降低45.98%。