We propose a novel framework for the regularised inversion of deep neural networks. The framework is based on the authors' recent work on training feed-forward neural networks without the differentiation of activation functions. The framework lifts the parameter space into a higher dimensional space by introducing auxiliary variables, and penalises these variables with tailored Bregman distances. We propose a family of variational regularisations based on these Bregman distances, present theoretical results and support their practical application with numerical examples. In particular, we present the first convergence result (to the best of our knowledge) for the regularised inversion of a single-layer perceptron that only assumes that the solution of the inverse problem is in the range of the regularisation operator, and that shows that the regularised inverse provably converges to the true inverse if measurement errors converge to zero.

翻译：我们提出了一种用于深度神经网络正则化逆问题的新型框架。该框架基于作者近期在无需激活函数微分的情况下训练前馈神经网络的研究工作。通过引入辅助变量，该框架将参数空间提升至更高维空间，并利用定制的Bregman距离对这些变量施加惩罚。我们基于这些Bregman距离提出了一系列变分正则化方法，给出了理论结果，并通过数值算例验证了其实际应用。特别地，我们在单层感知机的正则化逆问题中首次（据我们所知）给出了收敛性结果——该结果仅需假设逆问题的解位于正则化算子的值域内，并证明了当测量误差趋近于零时，正则化逆解可靠地收敛至真实逆解。