The Moore-Penrose inverse is widely used in physics, statistics, and various fields of engineering. It captures well the notion of inversion of linear operators in the case of overcomplete data. In data science, nonlinear operators are extensively used. In this paper we characterize the fundamental properties of a pseudo-inverse (PI) for nonlinear operators. The concept is defined broadly. First for general sets, and then a refinement for normed spaces. The PI for normed spaces yields the Moore-Penrose inverse when the operator is a matrix. We present conditions for existence and uniqueness of a PI and establish theoretical results investigating its properties, such as continuity, its value for operator compositions and projection operators, and others. Analytic expressions are given for the PI of some well-known, non-invertible, nonlinear operators, such as hard- or soft-thresholding and ReLU. Finally, we analyze a neural layer and discuss relations to wavelet thresholding.

翻译：摩尔-彭若斯逆在物理学、统计学及工程学各领域中被广泛应用，它能很好地刻画超完备数据情形下线性算子的逆概念。在数据科学中，非线性算子被广泛使用。本文刻画了非线性算子伪逆（PI）的基本性质。该概念的定义具有广泛性：首先针对一般集合，随后对赋范空间进行精化。当算符为矩阵时，赋范空间中的伪逆可还原为摩尔-彭若斯逆。我们给出了伪逆存在性与唯一性的条件，并建立了研究其性质的理论成果，如连续性、算子复合与投影算子的伪逆取值等。针对一些已知不可逆的非线性算子（如硬阈值、软阈值及ReLU），给出了其伪逆的解析表达式。最后，我们分析了一个神经网络层，并讨论了其与小波阈值法的关联。