We propose SDORE, a semi-supervised deep Sobolev regressor, for the nonparametric estimation of the underlying regression function and its gradient. SDORE employs deep neural networks to minimize empirical risk with gradient norm regularization, allowing computation of the gradient norm on unlabeled data. We conduct a comprehensive analysis of the convergence rates of SDORE and establish a minimax optimal rate for the regression function. Crucially, we also derive a convergence rate for the associated plug-in gradient estimator, even in the presence of significant domain shift. These theoretical findings offer valuable prior guidance for selecting regularization parameters and determining the size of the neural network, while showcasing the provable advantage of leveraging unlabeled data in semi-supervised learning. To the best of our knowledge, SDORE is the first provable neural network-based approach that simultaneously estimates the regression function and its gradient, with diverse applications including nonparametric variable selection and inverse problems. The effectiveness of SDORE is validated through an extensive range of numerical simulations and real data analysis.

翻译：我们提出SDORE——一种半监督深度Sobolev回归器，用于非参数估计潜在回归函数及其梯度。SDORE采用深度神经网络，通过梯度范数正则化最小化经验风险，从而能够在无标签数据上计算梯度范数。我们全面分析了SDORE的收敛速率，并建立了回归函数的极小化最优速率。关键在于，即使在存在显著域偏移的情况下，我们也推导了相应插件梯度估计量的收敛速率。这些理论发现为选择正则化参数和确定神经网络规模提供了有价值的先验指导，同时展示了在半监督学习中利用无标签数据的可证明优势。据我们所知，SDORE是首个可证明的基于神经网络的回归函数及其梯度联合估计方法，其应用涵盖非参数变量选择和反问题等广泛领域。通过大量数值模拟和真实数据分析，我们验证了SDORE的有效性。