Bayesian online learning provides a coherent framework for sequential inference. However, its theoretical understanding remains limited, particularly in the one-pass setting. Existing theoretical guarantees typically require the mini-batch sample size to diverge, a condition that fails in the one-pass regime. In this paper, we propose a new Bayesian online learning algorithm tailored to the one-pass setting, which incorporates a warm-start phase to ensure stable sequential updates. For this algorithm, we show that the sequentially updated posterior attains the optimal convergence rate. Building on this, we establish an online analogue of the Bernstein-von Mises theorem, which guarantees valid uncertainty quantification without diverging mini-batch sample sizes. Our analysis is based on a novel theoretical framework that differs fundamentally from existing approaches in the online learning literature. Numerical experiments on generalized linear models show that the proposed method matches the performance of the batch estimator while outperforming existing online procedures.

翻译：贝叶斯在线学习为序贯推断提供了自洽的理论框架，然而其理论理解仍存在局限性，特别是在单遍处理场景中。现有理论保证通常要求小批量样本量趋于无穷，这一条件在单遍处理模式下无法满足。本文提出一种面向单遍设置的新型贝叶斯在线学习算法，通过引入预热阶段确保序贯更新的稳定性。我们证明该算法下序贯更新后验分布能够达到最优收敛速率。基于此，我们建立了伯恩斯坦-冯·米塞斯定理的在线对应形式，确保无需发散的小批量样本量即可实现有效的不确定性量化。我们的分析基于与现有在线学习文献截然不同的新型理论框架。广义线性模型上的数值实验表明，所提方法在匹配批量估计器性能的同时，优于现有在线算法。