Transformers and their multi-head attention mechanism have completely changed the machine learning landscape in just a few years, by outperforming state-of-art models in a wide range of domains. Still, little is known about their robustness from a theoretical perspective. We tackle this problem by studying the local Lipschitz constant of self-attention, that provides an attack-agnostic way of measuring the robustness of a neural network. We adopt a measure-theoretic framework, by viewing inputs as probability measures equipped with the Wasserstein distance. This allows us to generalize attention to inputs of infinite length, and to derive an upper bound and a lower bound on the Lipschitz constant of self-attention on compact sets. The lower bound significantly improves prior results, and grows more than exponentially with the radius of the compact set, which rules out the possibility of obtaining robustness guarantees without any additional constraint on the input space. Our results also point out that measures with a high local Lipschitz constant are typically made of a few diracs, with a very unbalanced distribution of mass. Finally, we analyze the stability of self-attention under perturbations that change the number of tokens, which appears to be a natural question in the measure-theoretic framework. In particular, we show that for some inputs, attacks that duplicate tokens before perturbing them are more efficient than attacks that simply move tokens. We call this phenomenon mass splitting.

翻译：Transformer及其多头注意力机制在短短几年内通过在一系列广泛领域超越最先进模型，彻底改变了机器学习格局。然而，从理论角度对其鲁棒性的了解仍然有限。我们通过研究自注意力机制的局部Lipschitz常数来解决这一问题，该常数提供了一种与攻击无关的神经网络鲁棒性度量方法。我们采用测度论框架，将输入视为配备Wasserstein距离的概率测度。这使得我们能够将注意力机制推广到无限长度输入，并在紧集上推导出自注意力Lipschitz常数的上界和下界。该下界显著优于先前结果，且随着紧集半径以超指数方式增长，这排除了不对输入空间施加额外约束而获得鲁棒性保证的可能性。我们的结果还指出，局部Lipschitz常数较高的测度通常由少量狄拉克函数构成，且质量分布极不均衡。最后，我们分析了在改变令牌数量的扰动下自注意力机制的稳定性——这似乎是测度论框架下的一个自然问题。特别地，我们证明对于某些输入，先复制令牌再对其进行扰动的攻击比简单移动令牌的攻击更有效。我们将这种现象称为质量分裂。