This paper tackles the challenge of teaching code semantics to Large Language Models (LLMs) for program analysis by incorporating code symmetries into the model architecture. We introduce a group-theoretic framework that defines code symmetries as semantics-preserving transformations, where forming a code symmetry group enables precise and efficient reasoning of code semantics. Our solution, SymC, develops a novel variant of self-attention that is provably equivariant to code symmetries from the permutation group defined over the program dependence graph. SymC obtains superior performance on five program analysis tasks, outperforming state-of-the-art code models without any pre-training. Our results suggest that code LLMs that encode the code structural prior via the code symmetry group generalize better and faster.

翻译：本文通过将代码对称性融入模型架构，解决了向大型语言模型（LLMs）教授代码语义以进行程序分析的挑战。我们引入了一个群论框架，将代码对称性定义为保持语义不变的变换，其中构成代码对称群能够实现对代码语义的精确高效推理。我们的解决方案SymC开发了一种新颖的自注意力变体，该变体被证明对定义在程序依赖图上的置换群所产生的代码对称性具有等变性。SymC在五项程序分析任务上取得了优越的性能，无需任何预训练即超越了最先进的代码模型。我们的结果表明，通过代码对称群编码代码结构先验的代码LLMs能够实现更好更快的泛化。