Unsupervised video domain adaptation is a practical yet challenging task. In this work, for the first time, we tackle it from a disentanglement view. Our key idea is to handle the spatial and temporal domain divergence separately through disentanglement. Specifically, we consider the generation of cross-domain videos from two sets of latent factors, one encoding the static information and another encoding the dynamic information. A Transfer Sequential VAE (TranSVAE) framework is then developed to model such generation. To better serve for adaptation, we propose several objectives to constrain the latent factors. With these constraints, the spatial divergence can be readily removed by disentangling the static domain-specific information out, and the temporal divergence is further reduced from both frame- and video-levels through adversarial learning. Extensive experiments on the UCF-HMDB, Jester, and Epic-Kitchens datasets verify the effectiveness and superiority of TranSVAE compared with several state-of-the-art methods. The code with reproducible results is publicly accessible.

翻译：无监督视频域适应是一项实际但具有挑战性的任务。在本工作中，我们首次从解耦的角度来解决这一问题。我们的核心思想是通过解耦分别处理空间域和时间域上的差异。具体而言，我们将跨域视频的生成视为由两组潜在因子作用的结果：一组编码静态信息，另一组编码动态信息。为此，我们开发了一个迁移序列化变分自编码器（TranSVAE）框架来建模这一生成过程。为了更好地适应任务，我们提出了多个目标函数来约束潜在因子。在这些约束下，空间差异可以通过剔除静态域特定信息得到有效消除，而时间差异则进一步通过对抗学习在帧级和视频级两个层面被削弱。在UCF-HMDB、Jester和Epic-Kitchens数据集上的大量实验表明，TranSVAE相较于多种现有最优方法具有有效性和优越性。可复现结果的代码已公开发布。