This paper is concerned with the computation of the local Lipschitz constant of feedforward neural networks (FNNs) with activation functions being rectified linear units (ReLUs). The local Lipschitz constant of an FNN for a target input is a reasonable measure for its quantitative evaluation of the reliability. By following a standard procedure using multipliers that capture the behavior of ReLUs,we first reduce the upper bound computation problem of the local Lipschitz constant into a semidefinite programming problem (SDP). Here we newly introduce copositive multipliers to capture the ReLU behavior accurately. Then, by considering the dual of the SDP for the upper bound computation, we second derive a viable test to conclude the exactness of the computed upper bound. However, these SDPs are intractable for practical FNNs with hundreds of ReLUs. To address this issue, we further propose a method to construct a reduced order model whose input-output property is identical to the original FNN over a neighborhood of the target input. We finally illustrate the effectiveness of the model reduction and exactness verification methods with numerical examples of practical FNNs.

翻译：本文研究了激活函数为修正线性单元(ReLU)的前馈神经网络(FNN)局部Lipschitz常数的计算问题。针对目标输入，FNN的局部Lipschitz常数是其可靠性定量评估的合理度量。通过采用捕捉ReLU行为特性的乘子标准流程，我们首先将局部Lipschitz常数的上界计算问题转化为半定规划问题。在此过程中，我们创新性地引入共正乘子以精确刻画ReLU行为。随后，通过分析该上界计算半定规划的对偶问题，我们推导出用于判定所计算上界精确性的可行验证方法。然而，这些半定规划对于具有数百个ReLU的实际FNN而言难以求解。为解决这一难题，我们进一步提出一种降阶模型构建方法，该模型在目标输入的邻域内与原始FNN具有相同的输入-输出特性。最后，通过实际FNN的数值算例验证了模型降阶与精确性验证方法的有效性。