Deep neural networks are powerful tools for solving nonlinear problems in science and engineering, but training highly accurate models becomes challenging as problem complexity increases. Non-convex optimization and sensitivity to hyperparameters make consistent performance improvement difficult, and traditional approaches prioritize minimizing mean squared error while overlooking the $L^{\infty}$ norm error that is critical in safety-sensitive applications. To address these challenges, we present HiPreNets, a progressive framework for training high-precision neural networks through sequential residual refinements. Starting from an initial network, each stage trains a refinement network on the normalized residuals of the ensemble so far, systematically reducing both average and worst-case error. A key theme throughout the framework is concentrating training effort on high-error regions of the input domain, which we pursue through complementary techniques including loss function design, adaptive data sampling, localized patching, and boundary-aware training. We validate the framework on benchmark regression problems from the Feynman dataset, where it consistently outperforms standard fully connected networks and reported Kolmogorov-Arnold Networks results, with accuracy approaching machine precision depending on select problems. We further apply the framework to learning the flow map of a 20-dimensional power system ODE, which appears to be the highest dimensional problem studied using this class of multistage methods, achieving substantial reductions in both RMSE and $L^{\infty}$ norm error while enabling a surrogate that predicts system state $238\times$ faster than direct numerical simulation.

翻译：深度神经网络是解决科学与工程中非线性问题的有力工具，但随着问题复杂度的增加，训练高精度模型变得极具挑战性。非凸优化及对超参数的敏感性使得性能难以持续提升，而传统方法优先最小化均方误差，忽略了在安全敏感应用中至关重要的$L^{\infty}$范数误差。为应对这些挑战，我们提出HiPreNets——一种通过序列残差修正来训练高精度神经网络的渐进式框架。从初始网络开始，每个阶段都会基于当前集成模型的归一化残差训练一个修正网络，从而系统性地降低平均误差和最差情形误差。该框架的核心主题是将训练资源集中作用于输入域中高误差区域，我们通过互补技术实现这一目标，包括损失函数设计、自适应数据采样、局部补丁优化以及边界感知训练。我们在费曼数据集上的基准回归问题中验证了该框架，其表现持续优于标准全连接网络及已报道的科洛莫戈罗夫-阿诺德网络结果，对于特定问题其精度可接近机器精度。此外，我们将该框架应用于学习20维电力系统常微分方程的流映射——这似乎是此类多阶段方法研究中的最高维度问题，在显著降低均方根误差和$L^{\infty}$范数误差的同时，构建的代理模型预测系统状态的速度比直接数值模拟快238倍。