Deploying deep models in real-world scenarios remains challenging due to significant performance drops under distribution shifts between training and deployment environments. Test-Time Adaptation (TTA) has recently emerged as a promising solution, enabling on-the-fly model adaptation. However, its effectiveness deteriorates in the presence of mixed distribution shifts -- common in practical settings -- where multiple target domains coexist. In this paper, we study TTA under mixed distribution shifts and move beyond conventional whole-batch adaptation paradigms. By revisiting distribution shifts from a spectral perspective, we find that the heterogeneity across latent domains is often pronounced in Fourier space. In particular, high-frequency components encode domain-specific variations, which facilitates clearer separation of samples from different distributions. Motivated by this observation, we propose to un-mix heterogeneous data streams using high-frequency domain cues, making diverse shift patterns more tractable. To this end, we propose Frequency-based Decentralized Adaptation (FreDA), a novel framework that decomposes globally heterogeneous data stream into locally homogeneous clusters in the Fourier space. It leverages decentralized learning and augmentation strategies to robustly adapt under mixed domain shifts. Extensive experiments across various environments (corrupted, natural, and medical) show the superiority of our method over the state-of-the-arts.

翻译：在现实场景中部署深度模型仍然面临挑战，主要原因在于训练环境与部署环境之间的分布偏移会导致模型性能显著下降。测试时自适应作为一种新兴的解决方案，能够实现模型的实时自适应调整。然而，当面临混合分布偏移时——即多个目标域共存的常见实际场景——该方法的有效性会明显降低。本文针对混合分布偏移下的测试时自适应问题展开研究，突破了传统整批次自适应范式的局限。通过从频谱视角重新审视分布偏移现象，我们发现潜在域间的异质性在傅里叶空间中往往更为显著。具体而言，高频分量编码了域特定的变异信息，这有助于更清晰地区分来自不同分布的样本。基于此发现，我们提出利用高频域线索对异构数据流进行解混处理，使多样化的偏移模式更易于处理。为此，我们提出了基于频率的去中心化自适应框架，该创新框架通过傅里叶空间将全局异构数据流分解为局部同质聚类。该框架结合去中心化学习与数据增强策略，能够在混合域偏移环境下实现鲁棒的自适应调整。在多种环境下的实验表明，本方法在性能上显著优于现有最先进技术。