Robustness of neural networks is commonly quantified via local or global Lipschitz constants. However, Lipschitz continuity can be overly coarse or overly restrictive as global robustness measure, failing to capture nuanced, data-dependent behavior. We propose a data-driven, architecture-agnostic framework based on the discrete modulus of continuity (DMOC), a non linear generalization of Lipschitz continuity that provides a finer notion of robustness. Unlike many existing approaches, DMOC does not require access to model internals and instead evaluates regularity relative to the data distribution. This shifts the focus from the model to the data, which provide a data-driven baseline of regularity against which the network's robustness is assessed. We establish convergence results for DMOC-induced seminorms with explicit data-driven rates in terms of the separation distance, and introduce a scalable minibatch algorithm that reduces the quadratic cost of exact computation, enabling application to large-scale data sets such as ImageNet. Empirically, DMOC serves as an architecture independent diagnostic: it distinguishes trained from untrained networks, reveals underfitting and overfitting regimes, and yields, as a special case, tight Lipschitz estimates comparable to state-of-the-art method such as ECLipsE and ECLipsE-fast.
翻译:神经网络鲁棒性通常通过局部或全局利普希茨常数来衡量。然而,利普希茨连续性作为全局鲁棒性度量可能过于粗糙或限制过强,无法捕捉细微的数据依赖行为。我们提出了一种基于离散模量连续性(DMOC)的数据驱动、架构无关的框架,这是利普希茨连续性的非线性推广,提供了更精细的鲁棒性概念。与现有方法不同,DMOC无需访问模型内部结构,而是评估相对于数据分布的规则性。这将关注点从模型转向数据,数据提供了规则性的数据驱动基准,据此评估网络的鲁棒性。我们建立了DMOC诱导半范数的收敛结果,给出了基于分离距离的显式数据驱动收敛速率,并引入了一种可扩展的小批量算法,将精确计算的二次成本降低,从而适用于ImageNet等大规模数据集。实验上,DMOC作为架构无关的诊断工具:它区分已训练和未训练的网络,揭示欠拟合和过拟合区域,并作为特例,产生与ECLipsE和ECLipsE-fast等最先进方法相当的紧密利普希茨估计。