We introduce LaGTran, a novel framework that utilizes readily available or easily acquired text descriptions to guide robust transfer of discriminative knowledge from labeled source to unlabeled target data with domain shifts. While unsupervised adaptation methods have been established to address this problem, they show limitations in handling challenging domain shifts due to their exclusive operation within the pixel-space. Motivated by our observation that semantically richer text modality has more favorable transfer properties, we devise a transfer mechanism to use a source-trained text-classifier to generate predictions on the target text descriptions, and utilize these predictions as supervision for the corresponding images. Our approach driven by language guidance is surprisingly easy and simple, yet significantly outperforms all prior approaches on challenging datasets like GeoNet and DomainNet, validating its extreme effectiveness. To further extend the scope of our study beyond images, we introduce a new benchmark to study ego-exo transfer in videos and find that our language-aided LaGTran yields significant gains in this highly challenging and non-trivial transfer setting. Code, models, and proposed datasets are publicly available at https://tarun005.github.io/lagtran/.

翻译：摘要：我们提出LaGTran，一种新颖的框架，利用现成或易于获取的文本描述，指导判别性知识从有标签源域向无标签目标域的鲁棒迁移，以应对域偏移问题。尽管无监督域适应方法已被用于解决该问题，但由于其仅在像素空间内操作，在处理具有挑战性的域偏移时表现有限。受语义更丰富的文本模态具有更优迁移特性这一观察的启发，我们设计了一种迁移机制：使用源域训练的文本分类器对目标文本描述生成预测，并将这些预测作为对应图像的监督信号。这种由语言引导驱动的方案出乎意料地简洁高效，但在GeoNet和DomainNet等具有挑战性的数据集上显著优于所有先前方法，验证了其极强有效性。为进一步将研究范围扩展至图像之外，我们引入了一个视频中自我-他人迁移研究的新基准，发现语言辅助的LaGTran在这一极具挑战性的非平凡迁移设置中取得了显著提升。代码、模型及所提数据集已开源发布于https://tarun005.github.io/lagtran/。