Variational Autoencoders (VAE) are popular generative models used to sample from complex data distributions. Despite their empirical success in various machine learning tasks, significant gaps remain in understanding their theoretical properties, particularly regarding convergence guarantees. This paper aims to bridge that gap by providing non-asymptotic convergence guarantees for VAE trained using both Stochastic Gradient Descent and Adam algorithms.We derive a convergence rate of $\mathcal{O}(\log n / \sqrt{n})$, where $n$ is the number of iterations of the optimization algorithm, with explicit dependencies on the batch size, the number of variational samples, and other key hyperparameters. Our theoretical analysis applies to both Linear VAE and Deep Gaussian VAE, as well as several VAE variants, including $\beta$-VAE and IWAE. Additionally, we empirically illustrate the impact of hyperparameters on convergence, offering new insights into the theoretical understanding of VAE training.

翻译：变分自编码器（VAE）是一种广泛使用的生成模型，用于从复杂数据分布中采样。尽管其在各种机器学习任务中取得了经验上的成功，但对其理论性质，特别是收敛性保证的理解仍存在显著差距。本文旨在通过为使用随机梯度下降和Adam算法训练的VAE提供非渐近收敛性保证来弥合这一差距。我们推导出$\mathcal{O}(\log n / \sqrt{n})$的收敛速率，其中$n$为优化算法的迭代次数，并明确依赖于批量大小、变分样本数量及其他关键超参数。我们的理论分析适用于线性VAE和深度高斯VAE，以及包括$\beta$-VAE和IWAE在内的多种VAE变体。此外，我们通过实验说明了超参数对收敛的影响，为VAE训练的理论理解提供了新的见解。