We consider the problem of learning a sparse graph underlying an undirected Gaussian graphical model, a key problem in statistical machine learning. Given $n$ samples from a multivariate Gaussian distribution with $p$ variables, the goal is to estimate the $p \times p$ inverse covariance matrix (aka precision matrix), assuming it is sparse (i.e., has a few nonzero entries). We propose GraphL0BnB, a new estimator based on an $\ell_0$-penalized version of the pseudo-likelihood function, while most earlier approaches are based on the $\ell_1$-relaxation. Our estimator can be formulated as a convex mixed integer program (MIP) which can be difficult to compute beyond $p\approx 100$ using off-the-shelf commercial solvers. To solve the MIP, we propose a custom nonlinear branch-and-bound (BnB) framework that solves node relaxations with tailored first-order methods. As a key component of our BnB framework, we propose large-scale solvers for obtaining good primal solutions that are of independent interest. We derive novel statistical guarantees (estimation and variable selection) for our estimator and discuss how our approach improves upon existing estimators. Our numerical experiments on real and synthetic datasets suggest that our BnB framework offers significant advantages over off-the-shelf commercial solvers, and our approach has favorable performance (both in terms of runtime and statistical performance) compared to the state-of-the-art approaches for learning sparse graphical models.

翻译：本文研究无向高斯图模型下稀疏图的估计问题，这是统计机器学习中的一个关键问题。给定多元高斯分布的 $n$ 个样本（含 $p$ 个变量），目标是估计 $p \times p$ 的逆协方差矩阵（即精度矩阵），并假设其是稀疏的（即仅有少量非零元素）。我们提出一种名为 GraphL0BnB 的新估计量，它基于伪似然函数的 $\ell_0$ 惩罚项，而大多数早期方法则基于 $\ell_1$ 松弛。该估计量可表述为一个凸混合整数规划（MIP）问题，但使用现成商业求解器计算时，在 $p\approx 100$ 以上便难以处理。为解决这一 MIP 问题，我们提出了一种定制的非线性分支定界（BnB）框架，该框架通过定制的一阶方法求解节点松弛。作为 BnB 框架的关键组成部分，我们提出了用于获取优质原始解的大规模求解器，该成果本身也具有独立研究价值。我们为该估计量推导了新的统计保证（包括估计与变量选择），并讨论了该方法相比现有估计量的改进之处。在真实与合成数据集上的数值实验表明，我们的 BnB 框架相比现成商业求解器具有显著优势，且与当前最先进的稀疏图模型学习方法相比，该框架在运行时间和统计性能两方面均表现更优。