We show that the output of a ReLU neural network can be interpreted as the value of a zero-sum, turn-based, stopping game, which we call the ReLU net game. The game runs in the direction opposite to that of the network, and the input of the network serves as the terminal reward of the game. In fact, evaluating the network is the same as running the Shapley-Bellman backward recursion for the value of the game. Using the expression of the value of the game as an expected total payoff with respect to the path measure induced by the transition probabilities and a pair of optimal policies, we derive a discrete Feynman-Kac-type path-integral formula for the network output. This game-theoretic representation can be used to derive bounds on the output from bounds on the input, leveraging the monotonicity of Shapley operators, and to verify robustness properties using policies as certificates. Moreover, training the neural network becomes an inverse game problem: given pairs of terminal rewards and corresponding values, one seeks transition probabilities and rewards of a game that reproduces them. Finally, we show that a similar approach applies to neural networks with Softplus activation functions, where the ReLU net game is replaced by its entropic regularization.

翻译：本文证明，ReLU神经网络的输出可被解释为一类零和、回合制、可停止博弈（称为ReLU网络博弈）的博弈值。该博弈的运行方向与网络前向传播方向相反，且网络的输入作为博弈的终止奖励。事实上，评估神经网络等价于对博弈值执行沙普利-贝尔曼后向递归。通过将博弈值表示为在由转移概率及一对最优策略诱导的路径测度下的期望总收益，我们推导出网络输出的离散型费曼-卡茨路径积分公式。此博弈论表示可用于：利用沙普利算子的单调性，从输入的边界推导输出的边界；以及使用策略作为证书验证鲁棒性性质。此外，训练神经网络可转化为逆博弈问题：给定多组终止奖励与对应博弈值，寻求能复现这些数据的博弈转移概率与奖励函数。最后，我们证明类似方法适用于采用Softplus激活函数的神经网络，此时ReLU网络博弈被其熵正则化形式所替代。