We consider the problem of finding weights and biases for a two-layer fully connected neural network to fit a given set of data points as well as possible, also known as EmpiricalRiskMinimization. Our main result is that the associated decision problem is $\exists\mathbb{R}$-complete, that is, polynomial-time equivalent to determining whether a multivariate polynomial with integer coefficients has any real roots. Furthermore, we prove that algebraic numbers of arbitrarily large degree are required as weights to be able to train some instances to optimality, even if all data points are rational. Our result already applies to fully connected instances with two inputs, two outputs, and one hidden layer of ReLU neurons. Thereby, we strengthen a result by Abrahamsen, Kleist and Miltzow [NeurIPS 2021]. A consequence of this is that a combinatorial search algorithm like the one by Arora, Basu, Mianjy and Mukherjee [ICLR 2018] is impossible for networks with more than one output dimension, unless $\mathsf{NP}=\exists\mathbb{R}$.

翻译：我们考虑为一个两层全连接神经网络寻找权重和偏置以尽可能拟合给定数据集的问题，即经验风险最小化问题。我们的主要结果是，相关的判定问题是$\exists\mathbb{R}$-完全的，即多项式时间等价于确定一个具有整数系数的多元多项式是否存在实根。此外，我们证明，即使所有数据点均为有理数，训练某些实例以达到最优性所需的权重必须包含代数数，且其次数可任意大。该结果已适用于具有两个输入、两个输出以及一个ReLU神经元隐藏层的全连接实例。由此，我们强化了Abrahamsen、Kleist和Miltzow [NeurIPS 2021]的结果。这一结论的推论是，对于具有多个输出维度的网络，像Arora、Basu、Mianjy和Mukherjee [ICLR 2018]提出的组合搜索算法是不可能的，除非$\mathsf{NP}=\exists\mathbb{R}$。