We consider the landscape of empirical risk minimization for high-dimensional Gaussian single-index models (generalized linear models). The objective is to recover an unknown signal $\boldsymbolθ^\star \in \mathbb{R}^d$ (where $d \gg 1$) from a loss function $\hat{R}(\boldsymbolθ)$ that depends on pairs of labels $(\mathbf{x}_i \cdot \boldsymbolθ, \mathbf{x}_i \cdot \boldsymbolθ^\star)_{i=1}^n$, with $\mathbf{x}_i \sim \mathcal{N}(0, I_d)$, in the proportional asymptotic regime $n \asymp d$. Using the Kac-Rice formula, we analyze different complexities of the landscape -- defined as the expected number of critical points -- corresponding to various types of critical points, including local minima. We first show that some variational formulas previously established in the literature for these complexities can be drastically simplified, reducing to explicit variational problems over a finite number of scalar parameters that we can efficiently solve numerically. Our framework also provides detailed predictions for properties of the critical points, including the spectral properties of the Hessian and the joint distribution of labels. We apply our analysis to the real phase retrieval problem for which we derive complete topological phase diagrams of the loss landscape, characterizing notably BBP-type transitions where the Hessian at local minima (as predicted by the Kac-Rice formula) becomes unstable in the direction of the signal. We test the predictive power of our analysis to characterize gradient flow dynamics, finding excellent agreement with finite-size simulations of local optimization algorithms, and capturing fine-grained details such as the empirical distribution of labels. Overall, our results open new avenues for the asymptotic study of loss landscapes and topological trivialization phenomena in high-dimensional statistical models.

翻译：我们考虑高维高斯单指标模型（广义线性模型）的经验风险最小化景观。目标是从损失函数 $\hat{R}(\boldsymbolθ)$ 中恢复未知信号 $\boldsymbolθ^\star \in \mathbb{R}^d$（其中 $d \gg 1$），该函数依赖于标签对 $(\mathbf{x}_i \cdot \boldsymbolθ, \mathbf{x}_i \cdot \boldsymbolθ^\star)_{i=1}^n$，其中 $\mathbf{x}_i \sim \mathcal{N}(0, I_d)$，且处于比例渐近区域 $n \asymp d$。利用 Kac-Rice 公式，我们分析了景观的不同复杂度——定义为临界点的期望数量——对应于各类临界点，包括局部极小值。我们首先证明，文献中先前建立的关于这些复杂度的某些变分公式可以大幅简化，简化为有限个标量参数上的显式变分问题，从而能够进行高效的数值求解。我们的框架还为临界点的性质提供了详细预测，包括 Hessian 矩阵的谱特性以及标签的联合分布。我们将分析应用于实相位恢复问题，并推导出损失景观的完整拓扑相图，特别刻画了 BBP 型相变，其中局部极小值处的 Hessian 矩阵（由 Kac-Rice 公式预测）在信号方向上变得不稳定。我们测试了分析框架在刻画梯度流动力学方面的预测能力，发现其与局部优化算法的有限尺寸模拟结果高度吻合，并能捕捉诸如标签经验分布等精细细节。总体而言，我们的研究结果为高维统计模型中损失景观和拓扑平凡化现象的渐近分析开辟了新途径。