Low-rank matrix sensing is a fundamental yet challenging nonconvex problem whose optimization landscape typically contains numerous spurious local minima, making it difficult for gradient-based optimizers to converge to the global optimum. Recent work has shown that over-parameterization via tensor lifting can convert such local minima into strict saddle points, an insight that also partially explains why massive scaling can improve generalization and performance in modern machine learning. Motivated by this observation, we propose a Simulated Oracle Direction (SOD) escape mechanism that simulates the landscape and escape direction of the over-parametrized space, without resorting to actually lifting the problem, since that would be computationally intractable. In essence, we designed a mathematical framework to project over-parametrized escape directions onto the original parameter space to guarantee a strict decrease of objective value from existing local minima. To the best of the our knowledge, this represents the first deterministic framework that could escape spurious local minima with guarantee, especially without using random perturbations or heuristic estimates. Numerical experiments demonstrate that our framework reliably escapes local minima and facilitates convergence to global optima, while incurring minimal computational cost when compared to explicit tensor over-parameterization. We believe this framework has non-trivial implications for nonconvex optimization beyond matrix sensing, by showcasing how simulated over-parameterization can be leveraged to tame challenging optimization landscapes.

翻译：低秩矩阵感知是一个基础但具有挑战性的非凸问题，其优化景观通常包含大量虚假局部极小值，使得基于梯度的优化器难以收敛到全局最优解。近期研究表明，通过张量升维进行过参数化可以将此类局部极小值转化为严格鞍点，这一见解也部分解释了为何大规模参数扩展能够提升现代机器学习中的泛化能力与性能。受此观察启发，我们提出一种模拟预言机方向逃离机制，该机制模拟过参数化空间的景观与逃离方向，而无需实际对问题进行升维处理——因为后者在计算上是不可行的。本质上，我们设计了一个数学框架，将过参数化逃离方向投影到原始参数空间，以保证从现有局部极小值出发能严格降低目标函数值。据我们所知，这是首个能够保证逃离虚假局部极小值的确定性框架，尤其在不依赖随机扰动或启发式估计的情况下实现。数值实验表明，我们的框架能可靠地逃离局部极小值并促进向全局最优解的收敛，同时与显式张量过参数化方法相比仅产生极小的计算开销。我们相信这一框架对超越矩阵感知的非凸优化问题具有重要启示，它展示了如何利用模拟过参数化技术来驯服复杂的优化景观。