The variational autoencoder (VAE) typically employs a standard normal prior as a regularizer for the probabilistic latent encoder. However, the Gaussian tail often decays too quickly to effectively accommodate the encoded points, failing to preserve crucial structures hidden in the data. In this paper, we explore the use of heavy-tailed models to combat over-regularization. Drawing upon insights from information geometry, we propose $t^3$VAE, a modified VAE framework that incorporates Student's t-distributions for the prior, encoder, and decoder. This results in a joint model distribution of a power form which we argue can better fit real-world datasets. We derive a new objective by reformulating the evidence lower bound as joint optimization of KL divergence between two statistical manifolds and replacing with $\gamma$-power divergence, a natural alternative for power families. $t^3$VAE demonstrates superior generation of low-density regions when trained on heavy-tailed synthetic data. Furthermore, we show that $t^3$VAE significantly outperforms other models on CelebA and imbalanced CIFAR-100 datasets.

翻译：变分自编码器（VAE）通常采用标准正态先验作为概率潜在编码器的正则化项。然而，高斯分布的尾部衰减过快，难以有效容纳编码后的数据点，导致无法保留数据中隐藏的关键结构。本文探索利用重尾模型来缓解过度正则化问题。借鉴信息几何的见解，我们提出$t^3$VAE——一种改进的VAE框架，将学生t分布应用于先验、编码器和解码器。由此产生的联合模型分布具有幂形式，我们认为该分布能更好地拟合真实世界数据集。通过将证据下界重新表述为两个统计流形之间KL散度的联合优化，并用$\gamma$-功率散度（幂分布族的自然替代方案）替代KL散度，我们推导出一个新的目标函数。$t^3$VAE在重尾合成数据上训练时展现出对低密度区域的卓越生成能力。此外，我们证明$t^3$VAE在CelebA和不平衡CIFAR-100数据集上显著优于其他模型。