In this paper we study how the choice of loss functions of non-convex optimization problems affects their robustness and optimization landscape, through the study of noisy matrix sensing. In traditional regression tasks, mean squared error (MSE) loss is a common choice, but it can be unreliable for non-Gaussian or heavy-tailed noise. To address this issue, we adopt a robust loss based on nonparametric regression, which uses a kernel-based estimate of the residual density and maximizes the estimated log-likelihood. This robust formulation coincides with the MSE loss under Gaussian errors but remains stable under more general settings. We further examine how this robust loss reshapes the optimization landscape by analyzing the upper-bound of restricted isometry property (RIP) constants for spurious local minima to disappear. Through theoretical and empirical analysis, we show that this new loss excels at handling large noise and remains robust across diverse noise distributions. This work offers initial insights into enhancing the robustness of machine learning tasks through simply changing the loss, guided by an intuitive and broadly applicable analytical framework.

翻译：本文通过研究含噪矩阵感知问题，探讨非凸优化问题中损失函数的选择如何影响其鲁棒性与优化景观。在传统回归任务中，均方误差（MSE）损失是常用选择，但对于非高斯或重尾噪声分布，该损失函数可能不可靠。为解决此问题，我们采用基于非参数回归的鲁棒损失函数，该方法利用残差密度的核估计并最大化估计的对数似然。该鲁棒公式在高斯误差下与MSE损失一致，但在更一般的噪声设定下仍保持稳定性。我们进一步通过分析虚假局部极小点消失所需的限制等距性（RIP）常数上界，探究该鲁棒损失如何重塑优化景观。理论与实证分析表明，该新型损失函数在处理强噪声时表现优异，且在不同噪声分布下均保持鲁棒性。本研究通过直观且广泛适用的分析框架，为通过简单改变损失函数来增强机器学习任务鲁棒性提供了初步见解。