Large language models (LLMs) are becoming an integrated part of software development. These models are trained on large datasets for code, where it is hard to verify each data point. Therefore, a potential attack surface can be to inject poisonous data into the training data to make models vulnerable, aka trojaned. It can pose a significant threat by hiding manipulative behaviors inside models, leading to compromising the integrity of the models in downstream tasks. In this paper, we propose an occlusion-based human-in-the-loop technique, OSeql, to distinguish trojan-triggering inputs of code. The technique is based on the observation that trojaned neural models of code rely heavily on the triggering part of input; hence, its removal would change the confidence of the models in their prediction substantially. Our results suggest that OSeql can detect the triggering inputs with almost 100% recall. We discuss the problem of false positives and how to address them. These results provide a baseline for future studies in this field.

翻译：大语言模型正逐渐成为软件开发中不可或缺的组成部分。这类模型基于大规模代码数据集进行训练，但每个数据点的验证难度较高。因此，潜在的攻击面可能是在训练数据中注入恶意数据（即木马化），从而使模型产生脆弱性。这种攻击通过在模型内部隐藏操控行为，可能对下游任务中模型的完整性构成严重威胁。本文提出了一种基于遮挡的人机协同技术OSeql，用于识别代码中的木马触发输入。该技术基于以下观察：被木马化的代码神经模型高度依赖输入中的触发部分，因此移除该部分将显著改变模型预测的置信度。实验结果表明，OSeql能够以近乎100%的召回率检测触发输入。我们讨论了误报问题及其应对策略。这些结果可为该领域的后续研究提供基准参考。