When can the input of a ReLU neural network be inferred from its output? In other words, when is the network injective? We consider a single layer, $x \mapsto \mathrm{ReLU}(Wx)$, with a random Gaussian $m \times n$ matrix $W$, in a high-dimensional setting where $n, m \to \infty$. Recent work connects this problem to spherical integral geometry giving rise to a conjectured sharp injectivity threshold for $\alpha = \frac{m}{n}$ by studying the expected Euler characteristic of a certain random set. We adopt a different perspective and show that injectivity is equivalent to a property of the ground state of the spherical perceptron, an important spin glass model in statistical physics. By leveraging the (non-rigorous) replica symmetry-breaking theory, we derive analytical equations for the threshold whose solution is at odds with that from the Euler characteristic. Furthermore, we use Gordon's min--max theorem to prove that a replica-symmetric upper bound refutes the Euler characteristic prediction. Along the way we aim to give a tutorial-style introduction to key ideas from statistical physics in an effort to make the exposition accessible to a broad audience. Our analysis establishes a connection between spin glasses and integral geometry but leaves open the problem of explaining the discrepancies.

翻译：何时能从ReLU神经网络的输出反推其输入？换言之，网络在何种条件下是单射的？本文考虑单层结构 $x \mapsto \mathrm{ReLU}(Wx)$，其中 $W$ 为随机高斯 $m \times n$ 矩阵，在高维极限 $n, m \to \infty$ 下展开研究。近期工作通过球形积分几何将这一问题与某随机集期望欧拉示性数联系起来，提出了关于 $\alpha = \frac{m}{n}$ 的尖锐单射性阈值猜想。我们采用不同视角，证明单射性等价于球形感知器（统计物理学中重要的自旋玻璃模型）基态的一个性质。借助（非严格的）副本对称破缺理论，我们推导出阈值的解析方程，其解与欧拉示性数预测存在分歧。进一步利用Gordon的极小-极大定理证明，副本对称上界否定了欧拉示性数的预测。在此过程中，我们以教程形式系统介绍统计物理学的核心思想，力求使论述面向广泛受众。本文分析建立了自旋玻璃与积分几何之间的关联，但尚未解决两者差异的成因问题。