Determining whether deep neural network (DNN) models can reliably recover target functions at overparameterization is a critical yet complex issue in the theory of deep learning. To advance understanding in this area, we introduce a concept we term "local linear recovery" (LLR), a weaker form of target function recovery that renders the problem more amenable to theoretical analysis. In the sense of LLR, we prove that functions expressible by narrower DNNs are guaranteed to be recoverable from fewer samples than model parameters. Specifically, we establish upper limits on the optimistic sample sizes, defined as the smallest sample size necessary to guarantee LLR, for functions in the space of a given DNN. Furthermore, we prove that these upper bounds are achieved in the case of two-layer tanh neural networks. Our research lays a solid groundwork for future investigations into the recovery capabilities of DNNs in overparameterized scenarios.

翻译：确定深度神经网络（DNN）模型在过参数化情况下能否可靠地恢复目标函数，是深度学习理论中一个关键而复杂的问题。为了推动该领域的理解，我们引入了一个称为“局部线性恢复”（LLR）的概念，这是一种较弱形式的目标函数恢复，使得该问题更易于进行理论分析。在LLR的意义上，我们证明了可由较窄DNN表达的函数，保证能够从比模型参数更少的样本中恢复。具体而言，我们为给定DNN空间中的函数，建立了乐观样本量的上限，该样本量定义为保证LLR所需的最小样本量。此外，我们证明了这些上限在两层tanh神经网络的情况下是可以达到的。我们的研究为未来探究DNN在过参数化场景下的恢复能力奠定了坚实的基础。