It is common to observe performance degradation when transferring models trained on some (source) datasets to target testing data due to a domain gap between them. Existing methods for bridging this gap, such as domain adaptation (DA), may require the source data on which the model was trained (often not available), while others, i.e., source-free DA, require many passes through the testing data. We propose an online test-time adaptation method for depth completion, the task of inferring a dense depth map from a single image and associated sparse depth map, that closes the performance gap in a single pass. We first present a study on how the domain shift in each data modality affects model performance. Based on our observations that the sparse depth modality exhibits a much smaller covariate shift than the image, we design an embedding module trained in the source domain that preserves a mapping from features encoding only sparse depth to those encoding image and sparse depth. During test time, sparse depth features are projected using this map as a proxy for source domain features and are used as guidance to train a set of auxiliary parameters (i.e., adaptation layer) to align image and sparse depth features from the target test domain to that of the source domain. We evaluate our method on indoor and outdoor scenarios and show that it improves over baselines by an average of 21.1%.

翻译：在将基于某些（源）数据集训练的模型迁移到目标测试数据时，由于领域差异，常出现性能下降现象。现有弥合这一差距的方法，如领域自适应（DA），可能需要模型训练所用的源数据（通常不可获取），而其他方法（如无源DA）则需多次遍历测试数据。我们提出一种面向深度补全的在线测试时自适应方法——该任务旨在从单张图像及其关联的稀疏深度图中推断稠密深度图——可在单次遍历中消除性能差距。我们首先研究各数据模态的领域偏移如何影响模型性能。基于稀疏深度模态的协变量偏移远小于图像模态的观察，我们设计了一个在源域训练的嵌入模块，该模块保留从仅编码稀疏深度的特征到编码图像与稀疏深度特征的映射。在测试阶段，利用该映射将稀疏深度特征投影为源域特征的代理，并以此为引导训练一组辅助参数（即自适应层），使目标测试域的图像与稀疏深度特征对齐至源域。我们在室内外场景中评估了该方法，结果表明其相较基线方法平均提升21.1%。