We study the properties of differentiable neural networks activated by rectified power unit (RePU) functions. We show that the partial derivatives of RePU neural networks can be represented by RePUs mixed-activated networks and derive upper bounds for the complexity of the function class of derivatives of RePUs networks. We establish error bounds for simultaneously approximating $C^s$ smooth functions and their derivatives using RePU-activated deep neural networks. Furthermore, we derive improved approximation error bounds when data has an approximate low-dimensional support, demonstrating the ability of RePU networks to mitigate the curse of dimensionality. To illustrate the usefulness of our results, we consider a deep score matching estimator (DSME) and propose a penalized deep isotonic regression (PDIR) using RePU networks. We establish non-asymptotic excess risk bounds for DSME and PDIR under the assumption that the target functions belong to a class of $C^s$ smooth functions. We also show that PDIR has a robustness property in the sense it is consistent with vanishing penalty parameters even when the monotonicity assumption is not satisfied. Furthermore, if the data distribution is supported on an approximate low-dimensional manifold, we show that DSME and PDIR can mitigate the curse of dimensionality.

翻译：本文研究了由修正幂单元（RePU）函数激活的可微分神经网络的性质。我们证明了RePU神经网络的偏导数可由RePU混合激活网络表示，并推导了RePU网络导数函数类复杂度的上界。建立了利用RePU激活深度神经网络同时逼近$C^s$光滑函数及其导数的误差界。进一步地，当数据具有近似低维支撑时，我们得到了改进的逼近误差界，展示了RePU网络缓解维数灾难的能力。为说明研究结果的有效性，我们考虑深度分数匹配估计量（DSME），并提出使用RePU网络的惩罚深度保序回归（PDIR）方法。在目标函数属于$C^s$光滑函数类的假设下，建立了DSME和PDIR的非渐近超额风险界。我们还证明，即使单调性假设不满足，PDIR仍具有鲁棒性——在惩罚参数趋零时保持一致性。此外，若数据分布支撑在近似低维流形上，我们证明了DSME和PDIR可缓解维数灾难问题。