In this work, we evolve assembly code for the CodeGuru competition. The competition's goal is to create a survivor -- an assembly program that runs the longest in shared memory, by resisting attacks from adversary survivors and finding their weaknesses. For evolving top-notch solvers, we specify a Backus Normal Form (BNF) for the assembly language and synthesize the code from scratch using Genetic Programming (GP). We evaluate the survivors by running CodeGuru games against human-written winning survivors. Our evolved programs found weaknesses in the programs they were trained against and utilized them. In addition, we compare our approach with a Large-Language Model, demonstrating that the latter cannot generate a survivor that can win at any competition. This work has important applications for cyber-security, as we utilize evolution to detect weaknesses in survivors. The assembly BNF is domain-independent; thus, by modifying the fitness function, it can detect code weaknesses and help fix them. Finally, the CodeGuru competition offers a novel platform for analyzing GP and code evolution in adversarial environments. To support further research in this direction, we provide a thorough qualitative analysis of the evolved survivors and the weaknesses found.
翻译:本研究针对CodeGuru竞赛演化汇编代码。该竞赛旨在创建生存者——一种在共享内存中运行时间最长、通过抵御对手生存者的攻击并发现其弱点的汇编程序。为演化出顶级求解器,我们为汇编语言指定了巴科斯范式(BNF),并利用遗传规划(GP)从头合成代码。通过让演化出的生存者与人工编写的优胜生存者进行CodeGuru对战,我们评估了其性能。结果表明,演化程序能够发现训练对手的弱点并加以利用。此外,我们将本方法与大型语言模型进行对比,证明后者无法生成在任何竞赛中获胜的生存者。该工作对网络安全具有重要应用价值,因为我们利用演化机制检测生存者的弱点。由于汇编语言的BNF与领域无关,通过修改适应度函数,该方法可检测并辅助修复代码漏洞。最后,CodeGuru竞赛为分析对抗环境中的遗传规划与代码演化提供了新颖平台。为支持该方向的后续研究,我们对演化生存者与所发现弱点进行了深入的定性分析。