Deep Boltzmann machines (DBMs), one of the first ``deep'' learning methods ever studied, are multi-layered probabilistic models governed by a pairwise energy function that describes the likelihood of all variables/nodes in the network. In practice, DBMs are often constrained, i.e., via the \emph{restricted} Boltzmann machine (RBM) architecture (which does not permit intra-layer connections), in order to allow for more efficient inference. In this work, we revisit the generic DBM approach, and ask the question: are there other possible restrictions to their design that would enable efficient (approximate) inference? In particular, we develop a new class of restricted model, the monotone DBM, which allows for arbitrary self-connection in each layer, but restricts the \emph{weights} in a manner that guarantees the existence and global uniqueness of a mean-field fixed point. To do this, we leverage tools from the recently-proposed monotone Deep Equilibrium model and show that a particular choice of activation results in a fixed-point iteration that gives a variational mean-field solution. While this approach is still largely conceptual, it is the first architecture that allows for efficient approximate inference in fully-general weight structures for DBMs. We apply this approach to simple deep convolutional Boltzmann architectures and demonstrate that it allows for tasks such as the joint completion and classification of images, within a single deep probabilistic setting, while avoiding the pitfalls of mean-field inference in traditional RBMs.

翻译：深度玻尔兹曼机（DBM）作为最早被研究的"深度"学习方法之一，是一种由描述网络中所有变量/节点可能性的成对能量函数所支配的多层概率模型。在实践中，DBM通常通过约束架构（如限制玻尔兹曼机（RBM）架构禁止层内连接）以实现更高效推理。本研究重新审视通用DBM方法，并提出问题：是否存在其他可能的设计约束以实现高效（近似）推理？具体而言，我们开发了一类新型约束模型——单调DBM，该模型允许每层任意自连接，但通过约束权重的形式保证均值场不动点的存在性与全局唯一性。为实现这一目标，我们利用近期提出的单调深度均衡模型中的工具，证明特定激活函数选择产生的固定点迭代可提供变分均值场解。尽管该方法仍主要停留在概念层面，但它是首个允许DBM在全通用权重结构下实现高效近似推理的架构。我们将该方法应用于简单深度卷积玻尔兹曼架构，并证明其能在单个深度概率框架内实现图像联合补全与分类等任务，同时避免传统RBM中均值场推理的缺陷。