Studying conditional independence among many variables with few observations is a challenging task. Gaussian Graphical Models (GGMs) tackle this problem by encouraging sparsity in the precision matrix through $l_q$ regularization with $q\leq1$. However, most GMMs rely on the $l_1$ norm because the objective is highly non-convex for sub-$l_1$ pseudo-norms. In the frequentist formulation, the $l_1$ norm relaxation provides the solution path as a function of the shrinkage parameter $\lambda$. In the Bayesian formulation, sparsity is instead encouraged through a Laplace prior, but posterior inference for different $\lambda$ requires repeated runs of expensive Gibbs samplers. Here we propose a general framework for variational inference with matrix-variate Normalizing Flow in GGMs, which unifies the benefits of frequentist and Bayesian frameworks. As a key improvement on previous work, we train with one flow a continuum of sparse regression models jointly for all regularization parameters $\lambda$ and all $l_q$ norms, including non-convex sub-$l_1$ pseudo-norms. Within one model we thus have access to (i) the evolution of the posterior for any $\lambda$ and any $l_q$ (pseudo-) norm, (ii) the marginal log-likelihood for model selection, and (iii) the frequentist solution paths through simulated annealing in the MAP limit.

翻译：在少量观测下研究众多变量间的条件独立性是一项具有挑战性的任务。高斯图模型通过采用$q\leq1$的$l_q$正则化惩罚精度矩阵的稀疏性来解决这一问题。然而，由于目标函数在次$l_1$伪范数下高度非凸，大多数GGM依赖于$l_1$范数。在频率学派模型中，$l_1$范数松弛提供了作为收缩参数$\lambda$函数的解路径。在贝叶斯模型中，稀疏性通过拉普拉斯先验来促进，但对不同$\lambda$的后验推断需要重复运行昂贵的吉布斯采样器。本文提出一个基于矩阵变量归一化流的高斯图模型变分推断通用框架，融合了频率学派与贝叶斯框架的优势。作为对前人工作的关键改进，我们通过单个流联合训练所有正则化参数$\lambda$及所有$l_q$范数（包含非凸次$l_1$伪范数）的稀疏回归模型连续统。通过单一模型，我们即可获得：(i) 任意$\lambda$和任意$l_q$（伪）范数下后验的演化过程，(ii) 用于模型选择的边际对数似然，以及(iii) 通过最大后验极限下的模拟退火获得的频率学派解路径。