We develop sampling algorithms to fit Bayesian hierarchical models, the computational complexity of which scales linearly with the number of observations and the number of parameters in the model. We focus on crossed random effect and nested multilevel models, which are used ubiquitously in applied sciences. The posterior dependence in both classes is sparse: in crossed random effects models it resembles a random graph, whereas in nested multilevel models it is tree-structured. For each class we identify a framework for scalable computation, building on previous work. Methods for crossed models are based on extensions of appropriately designed collapsed Gibbs samplers, where we introduce the idea of local centering; while methods for nested models are based on sparse linear algebra and data augmentation. We provide a theoretical analysis of the proposed algorithms in some simplified settings, including a comparison with previously proposed methodologies and an average-case analysis based on random graph theory. Numerical experiments, including two challenging real data analyses on predicting electoral results and real estate prices, compare with off-the-shelf Hamiltonian Monte Carlo, displaying drastic improvement in performance.

翻译：我们开发了用于拟合贝叶斯分层模型的采样算法，其计算复杂度随观测数量与模型参数数量呈线性增长。研究重点聚焦于在应用科学领域广泛使用的交叉随机效应模型与嵌套多水平模型。两类模型的后验依赖结构均呈现稀疏性：交叉随机效应模型类似于随机图结构，而嵌套多水平模型则具有树状结构。针对每类模型，我们在前人工作基础上构建了可扩展计算的框架。交叉模型的方法基于适当设计的压缩吉布斯采样器的扩展，我们引入了局部中心化思想；嵌套模型的方法则基于稀疏线性代数与数据增强技术。我们在简化设定下对提出算法进行了理论分析，包括与既有方法的比较以及基于随机图理论的平均情况分析。数值实验（包含两项具有挑战性的真实数据分析：选举结果预测与房地产价格预测）与现成的哈密顿蒙特卡洛方法相比，显示出计算性能的显著提升。