It is common to observe performance degradation when transferring models trained on some (source) datasets to target testing data due to a domain gap between them. Existing methods for bridging this gap, such as domain adaptation (DA), may require the source data on which the model was trained (often not available), while others, i.e., source-free DA, require many passes through the testing data. We propose an online test-time adaptation method for depth completion, the task of inferring a dense depth map from a single image and associated sparse depth map, that closes the performance gap in a single pass. We first present a study on how the domain shift in each data modality affects model performance. Based on our observations that the sparse depth modality exhibits a much smaller covariate shift than the image, we design an embedding module trained in the source domain that preserves a mapping from features encoding only sparse depth to those encoding image and sparse depth. During test time, sparse depth features are projected using this map as a proxy for source domain features and are used as guidance to train a set of auxiliary parameters (i.e., adaptation layer) to align image and sparse depth features from the target test domain to that of the source domain. We evaluate our method on indoor and outdoor scenarios and show that it improves over baselines by an average of 21.1%.

翻译：在将源数据集上训练的模型迁移到目标测试数据时，由于领域差异，模型性能下降的现象非常普遍。现有缩小领域差异的方法，如领域自适应（DA），可能需要模型训练时所依赖的源数据（通常无法获取），而其他方法，即无源DA，则需要多次遍历测试数据。我们提出一种用于深度补全任务的在线测试时自适应方法，该任务旨在从单张图像及其关联的稀疏深度图中推断出稠密深度图，我们的方法能在单次遍历中弥补性能差距。首先，我们研究了每种数据模态中的领域偏移对模型性能的影响。基于稀疏深度模态的协变量偏移远小于图像模态这一观测结果，我们设计了一个在源域训练的嵌入模块，该模块保留了一个从仅编码稀疏深度的特征到同时编码图像和稀疏深度特征的映射。在测试时，稀疏深度特征利用该映射作为源域特征的代理进行投影，并作为指导来训练一组辅助参数（即自适应层），以使目标测试域的图像和稀疏深度特征与源域的特征对齐。我们在室内和室外场景下评估了该方法，结果表明其相比基线平均提升了21.1%。