Variational autoencoders (VAEs) are popular likelihood-based generative models which can be efficiently trained by maximizing an Evidence Lower Bound (ELBO). There has been much progress in improving the expressiveness of the variational distribution to obtain tighter variational bounds and increased generative performance. Whilst previous work has leveraged Markov chain Monte Carlo (MCMC) methods for the construction of variational densities, gradient-based methods for adapting the proposal distributions for deep latent variable models have received less attention. This work suggests an entropy-based adaptation for a short-run Metropolis-adjusted Langevin (MALA) or Hamiltonian Monte Carlo (HMC) chain while optimising a tighter variational bound to the log-evidence. Experiments show that this approach yields higher held-out log-likelihoods as well as improved generative metrics. Our implicit variational density can adapt to complicated posterior geometries of latent hierarchical representations arising in hierarchical VAEs.

翻译：变分自编码器（VAEs）是流行的基于似然的生成模型，可通过最大化证据下界（ELBO）进行高效训练。在提升变分分布的表示能力以获得更紧的变分界和更优生成性能方面已有诸多进展。虽然先前的工作利用马尔可夫链蒙特卡洛（MCMC）方法构建变分密度，但基于梯度的深层潜变量模型提议分布自适应方法受到的关注较少。本文提出一种基于熵的自适应方法，用于短程Metropolis调整Langevin（MALA）或哈密顿蒙特卡洛（HMC）链，同时优化对数证据的更紧变分界。实验表明，该方法能获得更高的留出对数似然以及更优的生成指标。我们的隐式变分密度能够适应分层VAE中由潜变量层次表示产生的复杂后验几何结构。