3D point cloud models suffer significant performance degradation under distribution shifts caused by sensor noise, occlusions, and environmental changes. Test-time adaptation (TTA) has emerged as a practical paradigm for mitigating this issue during inference. Recently, leveraging multi-view augmentation has shown promise in improving 3D TTA performance. However, existing multi-view approaches are often constrained by sequential optimization that treats each view independently. This sequential optimization leads to substantial inference latency due to repetitive optimization steps, making real-time adaptation impractical. To address this, we propose Masked Multi-View Test-Time Adaptation (MAMVI), which replaces sequential optimization with a unified single-step adaptation. Specifically, MAMVI utilizes a hybrid masking strategy that combines fixed ratios for stability with Beta-distributed sampling for diversity. By aggregating losses across multiple views, MAMVI performs adaptation through a single backward pass based on multi-view consensus. Additionally, a confidence-based adaptive learning rate is used to dynamically adjust the adaptation intensity for each sample. Extensive experiments on ModelNet-40C, ShapeNet-C, and ScanObjectNN-C demonstrate that MAMVI achieves state-of-the-art accuracy on ShapeNet-C and ScanObjectNN-C. Moreover, it remains competitive on ModelNet-40C while delivering 4.9-8.9 times faster inference, making it highly suitable for real-time applications. Our code is available at https://github.com/Inseok-kong/MAMVI
翻译:3D点云模型在传感器噪声、遮挡及环境变化引起的分布偏移下性能显著下降。测试时自适应(TTA)已成为推理阶段缓解该问题的实用范式。近年来,利用多视角增强技术在提升3D TTA性能方面展现出潜力。然而,现有方法通常受限于将每个视角独立处理的顺序优化策略。这种顺序优化因重复的优化步骤导致推理延迟显著增加,使其难以满足实时自适应需求。为此,我们提出掩蔽多视角测试时自适应(MAMVI),用统一的单步自适应替代顺序优化。具体而言,MAMVI采用混合掩蔽策略,结合固定比例(以维持稳定性)与Beta分布采样(以增强多样性)。通过聚合多视角损失,MAMVI基于多视角一致性通过单次反向传播完成自适应。此外,基于置信度的自适应学习率被用于动态调整每个样本的自适应强度。在ModelNet-40C、ShapeNet-C和ScanObjectNN-C上的大量实验表明,MAMVI在ShapeNet-C和ScanObjectNN-C上达到了最先进精度,同时在ModelNet-40C上保持竞争力,且推理速度提升4.9-8.9倍,使其高度适用于实时应用。我们的代码开源在https://github.com/Inseok-kong/MAMVI。