We consider the problem of learning a sparse graph underlying an undirected Gaussian graphical model, a key problem in statistical machine learning. Given $n$ samples from a multivariate Gaussian distribution with $p$ variables, the goal is to estimate the $p \times p$ inverse covariance matrix (aka precision matrix), assuming it is sparse (i.e., has a few nonzero entries). We propose GraphL0BnB, a new estimator based on an $\ell_0$-penalized version of the pseudolikelihood function, while most earlier approaches are based on the $\ell_1$-relaxation. Our estimator can be formulated as a convex mixed integer program (MIP) which can be difficult to compute at scale using off-the-shelf commercial solvers. To solve the MIP, we propose a custom nonlinear branch-and-bound (BnB) framework that solves node relaxations with tailored first-order methods. As a by-product of our BnB framework, we propose large-scale solvers for obtaining good primal solutions that are of independent interest. We derive novel statistical guarantees (estimation and variable selection) for our estimator and discuss how our approach improves upon existing estimators. Our numerical experiments on real/synthetic datasets suggest that our method can solve, to near-optimality, problem instances with $p = 10^4$ -- corresponding to a symmetric matrix of size $p \times p$ with $p^2/2$ binary variables. We demonstrate the usefulness of GraphL0BnB versus various state-of-the-art approaches on a range of datasets.

翻译：我们考虑学习无向高斯图模型底层稀疏图的问题，这是统计机器学习中的一个关键问题。给定来自具有 p 个变量的多元高斯分布的 n 个样本，目标是估计 p × p 的逆协方差矩阵（即精度矩阵），假设其是稀疏的（即仅有少量非零元素）。我们提出 GraphL0BnB，一种基于伪似然函数的 ℓ₀ 惩罚版本的新估计量，而大多数早期方法基于 ℓ₁ 松弛。我们的估计量可以表述为凸混合整数规划（MIP），使用现成的商业求解器在大规模下可能难以计算。为解决该 MIP，我们提出一个自定义的非线性分支定界（BnB）框架，该框架使用定制的基于一阶方法求解节点松弛。作为我们 BnB 框架的副产品，我们提出了用于获得良好原始解的大规模求解器，这些求解器具有独立的研究意义。我们为我们的估计量推导了新颖的统计保证（估计和变量选择），并讨论了我们的方法如何改进现有估计量。我们在真实/合成数据集上的数值实验表明，我们的方法可以将 p = 10⁴ 的问题实例求解到接近最优——这对应于大小为 p × p 且包含 p²/2 个二进制变量的对称矩阵。我们在多个数据集上展示了 GraphL0BnB 相对于各种最先进方法的实用性。