Despite progress, deep neural networks still suffer performance declines under distribution shifts between training and test domains, leading to a substantial decrease in Quality of Experience (QoE) for multimedia applications. Existing test-time adaptation (TTA) methods are challenged by dynamic, multiple test distributions within batches. This work provides a new perspective on analyzing batch normalization techniques through class-related and class-irrelevant features, our observations reveal combining source and test batch normalization statistics robustly characterizes target distributions. However, test statistics must have high similarity. We thus propose Discover Your Neighbours (DYN), the first backward-free approach specialized for dynamic TTA. The core innovation is identifying similar samples via instance normalization statistics and clustering into groups which provides consistent class-irrelevant representations. Specifically, Our DYN consists of layer-wise instance statistics clustering (LISC) and cluster-aware batch normalization (CABN). In LISC, we perform layer-wise clustering of approximate feature samples at each BN layer by calculating the cosine similarity of instance normalization statistics across the batch. CABN then aggregates SBN and TCN statistics to collaboratively characterize the target distribution, enabling more robust representations. Experimental results validate DYN's robustness and effectiveness, demonstrating maintained performance under dynamic data stream patterns.
翻译:尽管取得了进展,深度神经网络在训练域与测试域之间的分布偏移下仍会出现性能下降,导致多媒体应用的体验质量(QoE)大幅降低。现有的测试时自适应(TTA)方法难以应对批处理中动态、多变的测试分布。本文通过类相关特征与类无关特征的分析,为批归一化技术提供了新视角。我们的观察表明,结合源域与测试域的批归一化统计量能够稳健地表征目标分布,但测试统计量需具备高度相似性。为此,我们提出"发现你的邻居"(DYN)——首个专门针对动态TTA的无反向传播方法。其核心创新在于通过实例归一化统计量识别相似样本,并将其聚类为组,从而提供一致的类无关表征。具体而言,我们的DYN包含逐层实例统计聚类(LISC)和聚类感知批归一化(CABN)。在LISC中,我们通过计算批处理中实例归一化统计量的余弦相似度,在每个批归一化层对近似特征样本进行逐层聚类。随后,CABN聚合源域批归一化(SBN)与测试域批归一化(TCN)统计量,协同表征目标分布,实现更鲁棒的表征。实验结果验证了DYN的鲁棒性与有效性,证明其在动态数据流模式下仍能保持性能。