Multi-label recognition with frozen Vision-Language Models (VLMs) is brittle under distribution shift: standard zero-shot inference scores labels independently, ignoring co-occurrence structure and producing incoherent label sets where dominant concepts suppress weaker but compatible labels. We introduce Bayesian Conditional Priors (BCP) Estimation, a gradient-free test-time adaptation method that injects label dependency without tuning the backbone. BCP views zero-shot logits as a proxy for marginal posteriors under a fixed image-text likelihood and attributes shift-induced errors mainly to a mismatched label prior. For each test image, it selects a high-confidence anchor label and applies an anchor-conditioned Bayesian refinement. This update is closed-form in logit space and admits a pointwise mutual information (PMI) interpretation, explicitly promoting compatible labels and suppressing incompatible ones. BCP operates without target annotations by estimating anchor-conditioned priors online from the unlabeled test stream via lightweight second-order co-occurrence statistics, adding negligible overhead beyond a single forward pass. Across standard multi-label benchmarks and multiple CLIP backbones, BCP consistently outperforms strong TTA baselines, e.g., improving RN50 average mAP from 57.31 to 69.22 and ViT-B/16 from 62.61 to 71.79.

翻译：基于冻结视觉-语言模型（VLM）的多标签识别在面对分布偏移时存在脆弱性：标准零样本推理独立判定标签，忽视了共现结构，导致生成不连贯的标签集合——其中占主导地位的概念会抑制兼容但较弱的标签。本文提出贝叶斯条件先验（BCP）估计，一种无需梯度的测试时自适应方法，可在不调整主干网络的情况下注入标签依赖关系。BCP将零样本logits视为固定图文似然下边缘后验的代理，并将主要由先验失配引起的偏移误差归因于标签先验的偏差。对于每张测试图像，该方法选取高置信度锚点标签，并应用锚点条件贝叶斯精炼。该更新在logit空间中具有闭式解，并可用逐点互信息（PMI）进行解释，显式增强兼容标签并抑制不兼容标签。BCP无需目标标注：通过轻量级二阶共现统计量从未标注测试流中在线估计锚点条件先验，除单次前向传播外仅增加极小计算开销。在标准多标签基准测试和多种CLIP骨干网络上，BCP均持续优于强基线TTA方法，例如将RN50的平均mAP从57.31提升至69.22，ViT-B/16从62.61提升至71.79。