Existing test-time adaptation (TTA) approaches often adapt models with the unlabeled testing data stream. A recent attempt relaxed the assumption by introducing limited human annotation, referred to as Human-In-the-Loop Test-Time Adaptation (HILTTA) in this study. The focus of existing HILTTA lies on selecting the most informative samples to label, a.k.a. active learning. In this work, we are motivated by a pitfall of TTA, i.e. sensitive to hyper-parameters, and propose to approach HILTTA by synergizing active learning and model selection. Specifically, we first select samples for human annotation (active learning) and then use the labeled data to select optimal hyper-parameters (model selection). A sample selection strategy is tailored for choosing samples by considering the balance between active learning and model selection purposes. We demonstrate on 4 TTA datasets that the proposed HILTTA approach is compatible with off-the-shelf TTA methods which outperform the state-of-the-art HILTTA methods and stream-based active learning methods. Importantly, our proposed method can always prevent choosing the worst hyper-parameters on all off-the-shelf TTA methods. The source code will be released upon publication.

翻译：现有的测试时适应方法通常利用未标记的测试数据流来调整模型。近期研究通过引入有限的人工标注放宽了这一假设，本研究将其称为人机协同测试时适应。现有HILTTA方法的核心在于选择信息量最大的样本进行标注，即主动学习。本研究受测试时适应方法对超参数敏感这一缺陷的启发，提出通过协同主动学习与模型选择来实现HILTTA。具体而言，我们首先选择需要人工标注的样本，然后利用标注数据选择最优超参数。我们设计了一种样本选择策略，通过权衡主动学习与模型选择的目标来选取样本。在4个TTA数据集上的实验表明，所提出的HILTTA方法可与现成的TTA方法兼容，其性能优于最先进的HILTTA方法及流式主动学习方法。值得注意的是，本方法在所有现成TTA方法上均能有效避免选择最差超参数。源代码将在论文发表后公开。