We study the problem of regression in a generalized linear model (GLM) with multiple signals and latent variables. This model, which we call a matrix GLM, covers many widely studied problems in statistical learning, including mixed linear regression, max-affine regression, and mixture-of-experts. In mixed linear regression, each observation comes from one of $L$ signal vectors (regressors), but we do not know which one; in max-affine regression, each observation comes from the maximum of $L$ affine functions, each defined via a different signal vector. The goal in all these problems is to estimate the signals, and possibly some of the latent variables, from the observations. We propose a novel approximate message passing (AMP) algorithm for estimation in a matrix GLM and rigorously characterize its performance in the high-dimensional limit. This characterization is in terms of a state evolution recursion, which allows us to precisely compute performance measures such as the asymptotic mean-squared error. The state evolution characterization can be used to tailor the AMP algorithm to take advantage of any structural information known about the signals. Using state evolution, we derive an optimal choice of AMP `denoising' functions that minimizes the estimation error in each iteration. The theoretical results are validated by numerical simulations for mixed linear regression, max-affine regression, and mixture-of-experts. For max-affine regression, we propose an algorithm that combines AMP with expectation-maximization to estimate intercepts of the model along with the signals. The numerical results show that AMP significantly outperforms other estimators for mixed linear regression and max-affine regression in most parameter regimes.

翻译：我们研究广义线性模型（GLM）中具有多重信号与潜变量的回归问题。该模型（称为矩阵GLM）涵盖了统计学习领域中诸多广泛研究的问题，包括混合线性回归、最大仿射回归以及专家混合模型。在混合线性回归中，每个观测值来自$L$个信号向量（回归变量）之一，但具体来源未知；在最大仿射回归中，每个观测值由$L$个仿射函数的最大值生成，每个函数通过不同信号向量定义。所有这些问题旨在从观测数据中估计信号，并可能估计部分潜变量。我们提出一种用于矩阵GLM估计的新型近似消息传递（AMP）算法，并严格刻画其在高维极限下的性能。该刻画通过状态演化递推实现，使我们能精确计算渐近均方误差等性能指标。状态演化特性可用于定制AMP算法，使其充分利用信号已知的结构信息。通过状态演化，我们推导出AMP“去噪”函数的最优选择，以最小化每次迭代的估计误差。理论结果通过混合线性回归、最大仿射回归和专家混合模型的数值模拟得到验证。针对最大仿射回归，我们提出一种将AMP与期望最大化相结合的算法，用于同时估计模型的截距与信号。数值结果表明，在大多数参数范围内，AMP在混合线性回归和最大仿射回归中的表现显著优于其他估计方法。