Transformer based large language models with emergent capabilities are becoming increasingly ubiquitous in society. However, the task of understanding and interpreting their internal workings, in the context of adversarial attacks, remains largely unsolved. Gradient-based universal adversarial attacks have been shown to be highly effective on large language models and potentially dangerous due to their input-agnostic nature. This work presents a novel geometric perspective explaining universal adversarial attacks on large language models. By attacking the 117M parameter GPT-2 model, we find evidence indicating that universal adversarial triggers could be embedding vectors which merely approximate the semantic information in their adversarial training region. This hypothesis is supported by white-box model analysis comprising dimensionality reduction and similarity measurement of hidden representations. We believe this new geometric perspective on the underlying mechanism driving universal attacks could help us gain deeper insight into the internal workings and failure modes of LLMs, thus enabling their mitigation.

翻译：基于Transformer且具备涌现能力的大型语言模型在社会中日益普及。然而，在对抗攻击背景下理解和解释其内部工作机制的任务仍远未解决。基于梯度的通用对抗攻击已被证明对大型语言模型极为有效，且由于其与输入无关的特性而可能带来危险。本研究提出了一种新颖的几何视角，用于解释大型语言模型上的通用对抗攻击。通过攻击1.17亿参数的GPT-2模型，我们发现证据表明通用对抗触发器可能仅仅是近似其对抗训练区域语义信息的嵌入向量。这一假设得到了白盒模型分析的支持，该分析包括隐藏表示的降维和相似性测量。我们相信，这种关于通用攻击底层机制的新几何视角有助于更深入地理解大型语言模型的内部工作机制和失效模式，从而能够缓解这些问题。