Vision-and-Language Navigation (VLN) has witnessed significant advancements in recent years, largely attributed to meticulously curated datasets and proficiently trained models. Nevertheless, when tested in diverse environments, the trained models inevitably encounter significant shifts in data distribution, highlighting that relying solely on pre-trained and fixed navigation models is insufficient. To enhance models' generalization ability, test-time adaptation (TTA) demonstrates significant potential in the computer vision field by leveraging unlabeled test samples for model updates. However, simply applying existing TTA methods to the VLN task cannot well handle the adaptability-stability dilemma of VLN models, i.e., frequent updates can result in drastic changes in model parameters, while occasional updates can make the models ill-equipped to handle dynamically changing environments. Therefore, we propose a Fast-Slow Test-Time Adaptation (FSTTA) approach for VLN by performing decomposition-accumulation analysis for both gradients and parameters in a unified framework. Specifically, in the fast update phase, gradients generated during the recent multi-step navigation process are decomposed into components with varying levels of consistency. Then, these components are adaptively accumulated to pinpoint a concordant direction for fast model adaptation. In the slow update phase, historically recorded parameters are gathered, and a similar decomposition-accumulation analysis is conducted to revert the model to a stable state. Extensive experiments show that our method obtains impressive performance gains on four popular benchmarks.

翻译：视觉与语言导航（Vision-and-Language Navigation, VLN）近年来取得了显著进展，这主要归功于精心策划的数据集和高效训练的模型。然而，在多样化环境中测试时，训练好的模型不可避免地会遭遇数据分布的显著变化，这凸显了仅依赖预训练且固定的导航模型是不够的。为了提升模型的泛化能力，测试时自适应（Test-Time Adaptation, TTA）通过利用未标记的测试样本进行模型更新，在计算机视觉领域展现出巨大潜力。然而，将现有TTA方法直接应用于VLN任务难以有效解决VLN模型的自适应-稳定性困境，即频繁更新会导致模型参数剧烈变化，而偶发更新又使模型难以应对动态变化的环境。为此，我们提出了一种面向VLN的快速-慢速测试时自适应（Fast-Slow Test-Time Adaptation, FSTTA）方法，通过在统一框架中对梯度和参数进行分解-累积分析。具体而言，在快速更新阶段，将近期多步导航过程中产生的梯度分解为具有不同一致性程度的成分，再对这些成分进行自适应累积，以确定模型快速自适应的协同方向。在慢速更新阶段，收集历史记录参数，并对其执行类似的分解-累积分析，从而使模型恢复到稳定状态。大量实验表明，我们的方法在四个主流基准测试上均取得了显著的性能提升。