Unsupervised video domain adaptation is a practical yet challenging task. In this work, for the first time, we tackle it from a disentanglement view. Our key idea is to handle the spatial and temporal domain divergence separately through disentanglement. Specifically, we consider the generation of cross-domain videos from two sets of latent factors, one encoding the static information and another encoding the dynamic information. A Transfer Sequential VAE (TranSVAE) framework is then developed to model such generation. To better serve for adaptation, we propose several objectives to constrain the latent factors. With these constraints, the spatial divergence can be readily removed by disentangling the static domain-specific information out, and the temporal divergence is further reduced from both frame- and video-levels through adversarial learning. Extensive experiments on the UCF-HMDB, Jester, and Epic-Kitchens datasets verify the effectiveness and superiority of TranSVAE compared with several state-of-the-art approaches. Code is publicly available.

翻译：无监督视频域适应是一项实用但具有挑战性的任务。在本工作中，我们首次从解耦的视角来处理这一问题。我们的核心思想是通过解耦分别处理空间和时间上的域差异。具体而言，我们考虑从两组潜在因子生成跨域视频，一组编码静态信息，另一组编码动态信息。为此，我们开发了迁移序列变分自编码器（TranSVAE）框架来建模这一生成过程。为了更好地服务于域适应，我们提出了多个目标来约束潜在因子。在这些约束下，通过解耦静态域特定信息可以轻松消除空间差异，而时间差异则通过对抗学习在帧级和视频级进一步缩小。在UCF-HMDB、Jester和Epic-Kitchens数据集上的大量实验验证了TranSVAE相比几种最先进方法的有效性和优越性。代码已公开。