We develop sampling algorithms to fit Bayesian hierarchical models, the computational complexity of which scales linearly with the number of observations and the number of parameters in the model. We focus on crossed random effect and nested multilevel models, which are used ubiquitously in applied sciences. The posterior dependence in both classes is sparse: in crossed random effects models it resembles a random graph, whereas in nested multilevel models it is tree-structured. For each class we identify a framework for scalable computation, building on previous work. Methods for crossed models are based on extensions of appropriately designed collapsed Gibbs samplers, where we introduce the idea of local centering; while methods for nested models are based on sparse linear algebra and data augmentation. We provide a theoretical analysis of the proposed algorithms in some simplified settings, including a comparison with previously proposed methodologies and an average-case analysis based on random graph theory. Numerical experiments, including two challenging real data analyses on predicting electoral results and real estate prices, compare with off-the-shelf Hamiltonian Monte Carlo, displaying drastic improvement in performance.

翻译：我们开发了适用于贝叶斯分层模型的采样算法，其计算复杂度随观测数和模型参数数量线性增长。研究聚焦于交叉随机效应与嵌套多层模型——这两类模型在应用科学领域被广泛使用。两类模型的后验依赖性均具有稀疏性：交叉随机效应模型呈现随机图结构，而嵌套多层模型则表现为树状结构。针对每类模型，我们在已有工作基础上构建了可扩展计算框架。交叉模型方法基于经过适当设计的压缩吉布斯采样器的扩展，引入了局部中心化概念；嵌套模型方法则基于稀疏线性代数和数据增强技术。我们在简化设定下对提出算法进行理论分析，包括与现有方法的比较及基于随机图论的平均情况分析。数值实验涵盖选举结果预测和房地产价格预测两项具有挑战性的真实数据分析，与现成的哈密顿蒙特卡洛方法相比，展现出显著的性能提升。