Mainstream Test-Time Adaptation (TTA) methods for adapting vision-language models, e.g., CLIP, typically rely on Shannon Entropy (SE) at test time to measure prediction uncertainty and inconsistency. However, since CLIP has a built-in bias from pretraining on highly imbalanced web-crawled data, SE inevitably results in producing biased estimates of uncertainty entropy. To address this issue, we notably find and demonstrate that Tsallis Entropy (TE), a generalized form of SE, is naturally suited for characterizing biased distributions by introducing a non-extensive parameter q, with the performance of SE serving as a lower bound for TE. Building upon this, we generalize TE into Adaptive Debiasing Tsallis Entropy (ADTE) for TTA, customizing a class-specific parameter q^l derived by normalizing the estimated label bias from continuously incoming test instances, for each category. This adaptive approach allows ADTE to accurately select high-confidence views and seamlessly integrate with a label adjustment strategy to enhance adaptation, without introducing distribution-specific hyperparameter tuning. Besides, our investigation reveals that both TE and ADTE can serve as direct, advanced alternatives to SE in TTA, without any other modifications. Experimental results show that ADTE outperforms state-of-the-art methods on ImageNet and its five variants, and achieves the highest average performance on 10 cross-domain benchmarks, regardless of the model architecture or text prompts used. Our code is available at https://github.com/Jinx630/ADTE.

翻译：主流的测试时适应（TTA）方法在适配视觉-语言模型（例如CLIP）时，通常依赖于测试时的香农熵（SE）来衡量预测的不确定性和不一致性。然而，由于CLIP在高度不平衡的网络爬取数据上进行预训练而存在固有偏差，SE不可避免地会导致产生有偏的不确定性熵估计。为解决此问题，我们显著地发现并证明了Tsallis熵（TE）——SE的一种广义形式——通过引入一个非广延参数q，天然适合刻画有偏分布，且SE的性能可作为TE的下界。基于此，我们将TE推广为用于TTA的自适应去偏Tsallis熵（ADTE），通过将由持续流入的测试实例估计的标签偏差进行归一化，为每个类别定制一个类别特定的参数q^l。这种自适应方法使ADTE能够准确选择高置信度的视图，并无缝集成标签调整策略以增强适应能力，而无需引入针对特定分布的超参数调优。此外，我们的研究表明，TE和ADTE均可在TTA中作为SE的直接、高级替代方案，无需任何其他修改。实验结果表明，无论使用何种模型架构或文本提示，ADTE在ImageNet及其五个变体上均优于最先进的方法，并在10个跨域基准测试中取得了最高的平均性能。我们的代码可在 https://github.com/Jinx630/ADTE 获取。