In this paper, we explore adaptive inference based on variational Bayes. Although several studies have been conducted to analyze the contraction properties of variational posteriors, there is still a lack of a general and computationally tractable variational Bayes method that performs adaptive inference. To fill this gap, we propose a novel adaptive variational Bayes framework, which can operate on a collection of models. The proposed framework first computes a variational posterior over each individual model separately and then combines them with certain weights to produce a variational posterior over the entire model. It turns out that this combined variational posterior is the closest member to the posterior over the entire model in a predefined family of approximating distributions. We show that the adaptive variational Bayes attains optimal contraction rates adaptively under very general conditions. We also provide a methodology to maintain the tractability and adaptive optimality of the adaptive variational Bayes even in the presence of an enormous number of individual models, such as sparse models. We apply the general results to several examples, including deep learning and sparse factor models, and derive new and adaptive inference results. In addition, we characterize an implicit regularization effect of variational Bayes and show that the adaptive variational posterior can utilize this.

翻译：本文探索了基于变分贝叶斯的自适应推断方法。尽管已有若干研究分析了变分后验的收缩性质，但目前仍缺乏一种既能实现自适应推断又具有计算可行性的通用变分贝叶斯方法。为填补这一空白，我们提出了一种新颖的自适应变分贝叶斯框架，该框架可作用于一组模型。所提出的框架首先分别计算每个独立模型的变分后验，然后通过特定权重将其组合，从而产生整个模型的变分后验。结果表明，该组合变分后验是预定义逼近分布族中与整体模型后验距离最近的成员。我们证明，在非常一般的条件下，自适应变分贝叶斯能够自适应地达到最优收缩速率。我们还提出了一种方法，使得即便在包含大量独立模型（如稀疏模型）的情况下，自适应变分贝叶斯仍能保持可计算性和自适应最优性。我们将一般结果应用于多个实例，包括深度学习和稀疏因子模型，并推导出新的自适应推断结果。此外，我们刻画了变分贝叶斯的一种隐式正则化效应，并表明自适应变分后验能够利用这一效应。