Test-time adaptation (TTA) adapts the pre-trained models during inference using unlabeled test data and has received a lot of research attention due to its potential practical value. Unfortunately, without any label supervision, existing TTA methods rely heavily on heuristic or empirical studies. Where to update the model always falls into suboptimal or brings more computational resource consumption. Meanwhile, there is still a significant performance gap between the TTA approaches and their supervised counterparts. Motivated by active learning, in this work, we propose the active test-time adaptation for semantic segmentation setup. Specifically, we introduce the human-in-the-loop pattern during the testing phase, which queries very few labels to facilitate predictions and model updates in an online manner. To do so, we propose a simple but effective ATASeg framework, which consists of two parts, i.e., model adapter and label annotator. Extensive experiments demonstrate that ATASeg bridges the performance gap between TTA methods and their supervised counterparts with only extremely few annotations, even one click for labeling surpasses known SOTA TTA methods by 2.6% average mIoU on ACDC benchmark. Empirical results imply that progress in either the model adapter or the label annotator will bring improvements to the ATASeg framework, giving it large research and reality potential.

翻译：测试时自适应（TTA）在推理过程中利用无标签测试数据调整预训练模型，因其潜在的实际价值而受到广泛研究关注。然而，由于缺乏任何标签监督，现有TTA方法严重依赖启发式或经验性研究。模型更新位置的确定始终陷入次优状态，或带来更多计算资源消耗。同时，TTA方法与有监督方法之间仍存在显著性能差距。受主动学习启发，本研究针对语义分割任务提出主动测试时自适应框架。具体而言，我们在测试阶段引入人机协同模式，通过在线方式查询极少量标签以促进预测与模型更新。为此，我们提出一个简洁高效的ATASeg框架，该框架包含两个核心组件：模型适配器与标签标注器。大量实验表明，ATASeg仅需极少量标注即可弥合TTA方法与有监督方法之间的性能差距，在ACDC基准测试中，单次点击标注即可使平均mIoU超越已知最优TTA方法2.6%。实证结果表明，模型适配器或标签标注器任一方面的改进都能提升ATASeg框架性能，使其具有巨大的研究与现实应用潜力。