Solving non-convex, NP-hard optimization problems is crucial for training machine learning models, including neural networks. However, non-convexity often leads to black-box machine learning models with unclear inner workings. While convex formulations have been used for verifying neural network robustness, their application to training neural networks remains less explored. In response to this challenge, we reformulate the problem of training infinite-width two-layer ReLU networks as a convex completely positive program in a finite-dimensional (lifted) space. Despite the convexity, solving this problem remains NP-hard due to the complete positivity constraint. To overcome this challenge, we introduce a semidefinite relaxation that can be solved in polynomial time. We then experimentally evaluate the tightness of this relaxation, demonstrating its competitive performance in test accuracy across a range of classification tasks.

翻译：求解非凸、NP难优化问题是训练机器学习模型（包括神经网络）的关键。然而，非凸性往往导致机器学习模型成为内部机制不明确的"黑箱"。尽管凸优化公式已用于验证神经网络的鲁棒性，但其在神经网络训练中的应用仍较少被探索。针对这一挑战，我们将无限宽度双层ReLU网络的训练问题重新表述为有限维（提升）空间中的凸完全正定规划问题。尽管具有凸性，但由于完全正定性约束，求解该问题仍是NP难的。为克服此挑战，我们引入了一种可在多项式时间内求解的半定松弛方法。随后通过实验评估该松弛的紧致性，证明其在一系列分类任务的测试准确率上具有竞争力。