Contemporary LLMs pretrained on code are capable of succeeding at a wide variety of programming tasks. However, their performance is very sensitive to syntactic features, such as the names of variables and types, the structure of code, and presence of type hints. We contribute an inference-time technique to make CodeLLMs more robust to syntactic distractors that are semantically irrelevant. Our methodology relies on activation steering, which involves editing internal model activations to steer the model towards the correct prediction. We contribute a novel way to construct steering vectors by taking inspiration from mutation testing, which constructs minimal semantics-breaking code edits. In contrast, we construct steering vectors from semantics-preserving code edits. We apply our approach to the task of type prediction for the gradually typed languages Python and TypeScript. This approach corrects up to 90% of type mispredictions. Finally, we show that steering vectors calculated from Python activations reliably correct type mispredictions in TypeScript, and vice versa. This result suggests that LLMs may be learning to transfer knowledge of types across programming languages.

翻译：当前在代码上预训练的当代大语言模型能够胜任多种编程任务，但其性能对变量名、类型名、代码结构及类型注解等语法特征高度敏感。我们提出一种推理时技术，使代码大语言模型对语义无关的语法干扰因素更具鲁棒性。该方法基于激活引导——通过编辑模型内部激活状态，引导模型产生正确预测。受变异测试（通过最小化语义破坏性代码编辑生成测试用例）启发，我们提出一种新颖的引导向量构建方法。与之相反，我们通过保留语义的代码编辑来构建引导向量。我们将该方法应用于渐进类型语言Python和TypeScript的类型预测任务，可纠正高达90%的类型误判。最后，我们证明基于Python激活状态计算的引导向量能可靠纠正TypeScript的类型误判，反之亦然。这一结果表明大语言模型可能正在学习跨编程语言迁移类型知识。