To successfully apply trained neural network models to new domains, powerful transfer learning solutions are essential. We propose to introduce a novel cross-domain latent modulation mechanism to a variational autoencoder framework so as to achieve effective transfer learning. Our key idea is to procure deep representations from one data domain and use it to influence the reparameterization of the latent variable of another domain. Specifically, deep representations of the source and target domains are first extracted by a unified inference model and aligned by employing gradient reversal. The learned deep representations are then cross-modulated to the latent encoding of the alternative domain, where consistency constraints are also applied. In the empirical validation that includes a number of transfer learning benchmark tasks for unsupervised domain adaptation and image-to-image translation, our model demonstrates competitive performance, which is also supported by evidence obtained from visualization.

翻译：为成功将训练好的神经网络模型应用于新域，强大的迁移学习解决方案至关重要。我们提出在变分自编码器框架中引入一种新颖的跨域潜在调制机制，以实现有效的迁移学习。核心思想是从一个数据域获取深度表示，并将其用于影响另一个域潜在变量的重参数化。具体而言，首先通过统一推理模型提取源域和目标域的深度表示，并通过梯度反转进行对齐。随后，将学习到的深度表示交叉调制到另一域的潜在编码中，并同时施加一致性约束。在包含多项迁移学习基准任务（用于无监督域适应和图像到图像翻译）的实验验证中，我们的模型展现了具有竞争力的性能，可视化证据也进一步支持了这一结论。