LLM-based code interpreter agents are increasingly deployed in critical workflows, yet their robustness against risks introduced by their code execution capabilities remains underexplored. Existing benchmarks are limited to static datasets or simulated environments, failing to capture the security risks arising from dynamic code execution, tool interactions, and multi-turn context. To bridge this gap, we introduce CIBER, an automated benchmark that combines dynamic attack generation, isolated secure sandboxing, and state-aware evaluation to systematically assess the vulnerability of code interpreter agents against four major types of adversarial attacks: Direct/Indirect Prompt Injection, Memory Poisoning, and Prompt-based Backdoor. We evaluate six foundation models across two representative code interpreter agents (OpenInterpreter and OpenCodeInterpreter), incorporating a controlled study of identical models. Our results reveal that Interpreter Architecture and Model Alignment Set the Security Baseline. Structural integration enables aligned specialized models to outperform generic SOTA models. Conversely, high intelligence paradoxically increases susceptibility to complex adversarial prompts due to stronger instruction adherence. Furthermore, we identify a "Natural Language Disguise" Phenomenon, where natural language functions as a significantly more effective input modality than explicit code snippets (+14.1% ASR), thereby bypassing syntax-based defenses. Finally, we expose an alarming Security Polarization, where agents exhibit robust defenses against explicit threats yet fail catastrophically against implicit semantic hazards, highlighting a fundamental blind spot in current pattern-matching protection approaches.

翻译：基于大型语言模型的代码解释器智能体正日益部署于关键工作流中，然而其面对由代码执行能力引入风险的鲁棒性仍未得到充分探究。现有基准局限于静态数据集或模拟环境，未能捕捉动态代码执行、工具交互和多轮上下文所产生的安全风险。为弥补这一空白，我们提出了CIBER——一个结合动态攻击生成、隔离安全沙箱和状态感知评估的自动化基准，用于系统评估代码解释器智能体面对四类主要对抗攻击的脆弱性：直接/间接提示注入、记忆污染和基于提示的后门攻击。我们在两个代表性代码解释器智能体（OpenInterpreter和OpenCodeInterpreter）上评估了六个基础模型，并纳入了对相同模型的对照研究。我们的结果表明：解释器架构与模型对齐设定了安全基线。结构化集成使得经过对齐的专用模型能够超越通用SOTA模型。相反，由于更强的指令遵循性，高智能水平反而会通过更复杂的对抗性提示增加脆弱性。此外，我们发现了"自然语言伪装"现象：自然语言作为输入模态比显式代码片段具有显著更高的攻击成功率（+14.1% ASR），从而能够绕过基于语法的防御机制。最后，我们揭示了一个令人警觉的安全极化现象：智能体对显式威胁表现出强健防御，却在面对隐式语义危害时发生灾难性失效，这凸显出现有模式匹配保护方法存在根本性的盲区。