Continual Test-Time Adaptation (CTTA) aims to enable models to adapt online to unlabeled data streams under distribution shift without accessing source data. Existing CTTA methods face an efficiency-generalization trade-off: updating more parameters improves adaptation but severely reduces online inference efficiency. An ideal solution is to achieve comparable adaptation with minimal feature updates; we call this minimal subspace the golden subspace. We prove its existence in a single-step adaptation setting and show that it coincides with the row space of the pretrained classifier. To enable online maintenance of this subspace, we introduce the sample-wise Average Gradient Outer Product (AGOP) as an efficient proxy for estimating the classifier weights without retraining. Building on these insights, we propose Guided Online Low-rank Directional adaptation (GOLD), which uses a lightweight adapter to project features onto the golden subspace and learns a compact scaling vector while the subspace is dynamically updated via AGOP. Extensive experiments on classification and segmentation benchmarks, including autonomous-driving scenarios, demonstrate that GOLD attains superior efficiency, stability, and overall performance. Our code is available at https://github.com/AIGNLAI/GOLD.
翻译:连续测试时自适应(CTTA)旨在使模型能够在分布偏移下在线适应无标签数据流,同时无需访问源数据。现有CTTA方法面临效率与泛化之间的权衡:更新更多参数能提升自适应效果,但会严重降低在线推理效率。理想方案是在最小特征更新的条件下实现同等自适应效果,我们将这一最小子空间称为“金域子空间”。我们证明了该子空间在单步自适应设置中的存在性,并揭示其与预训练分类器行空间的一致性。为实现该子空间的在线维护,我们引入样本级平均梯度外积(AGOP)作为无需重训练的高效代理来估计分类器权重。基于这些发现,我们提出引导式在线低秩方向自适应方法(GOLD),该方法通过轻量级适配器将特征投影至金域子空间,并学习紧凑缩放向量,同时利用AGOP动态更新子空间。在包括自动驾驶场景在内的分类与分割基准上的大量实验表明,GOLD在效率、稳定性及整体性能方面均达到最优。我们的代码开源在 https://github.com/AIGNLAI/GOLD。