High-dimensional non-convex loss landscapes play a central role in the theory of Machine Learning. Gaining insight into how these landscapes interact with gradient-based optimization methods, even in relatively simple models, can shed light on this enigmatic feature of neural networks. In this work, we will focus on a prototypical simple learning problem, which generalizes the Phase Retrieval inference problem by allowing the exploration of overparametrized settings. Using techniques from field theory, we analyze the spectrum of the Hessian at initialization and identify a Baik-Ben Arous-P\'ech\'e (BBP) transition in the amount of data that separates regimes where the initialization is informative or uninformative about a planted signal of a teacher-student setup. Crucially, we demonstrate how overparameterization can bend the loss landscape, shifting the transition point, even reaching the information-theoretic weak-recovery threshold in the large overparameterization limit, while also altering its qualitative nature. We distinguish between continuous and discontinuous BBP transitions and support our analytical predictions with simulations, examining how they compare to the finite-N behavior. In the case of discontinuous BBP transitions strong finite-N corrections allow the retrieval of information at a signal-to-noise ratio (SNR) smaller than the predicted BBP transition. In these cases we provide estimates for a new lower SNR threshold that marks the point at which initialization becomes entirely uninformative.

翻译：高维非凸损失景观在机器学习理论中扮演着核心角色。深入理解这些景观如何与基于梯度的优化方法相互作用，即使在相对简单的模型中，也能为神经网络这一神秘特性提供启示。本研究将聚焦于一个典型简单学习问题，该问题通过允许探索过参数化设置推广了相位恢复推断问题。运用场论技术，我们分析了初始化时海森矩阵的谱分布，并识别出数据量中的Baik-Ben Arous-P\'ech\'e（BBP）相变现象——该相变区分了初始化对师生设置中植入信号是否具有信息性的两种机制。关键的是，我们论证了过参数化如何能够弯曲损失景观，移动相变临界点（在大过参数化极限下甚至可达信息论弱恢复阈值），同时改变其定性特征。我们区分了连续与不连续的BBP相变，并通过仿真验证理论预测，考察其与有限N行为的对比。对于不连续的BBP相变，强烈的有限N修正使得在低于预测BBP相变的信噪比（SNR）条件下仍能恢复信息。针对这些情形，我们给出了新的更低SNR阈值的估计值，该阈值标志着初始化完全失去信息性的临界点。