LLM agents are increasingly relevant to research domains such as vulnerability discovery. Yet, the strongest systems remain closed and cloud-only, making them resource-intensive, difficult to reproduce, and unsuitable for work involving proprietary code or sensitive data. Consequently, there is an urgent need for small, local models that can perform security tasks under strict resource budgets, but methods for developing them remain underexplored. In this paper, we address this gap by proposing a two-stage post-training pipeline. We focus on the problem of Linux privilege escalation, where success is automatically verifiable and the task requires multi-step interactive reasoning. Using an experimental setup that prevents data leakage, we post-train a 4B model in two stages: supervised fine-tuning on traces from procedurally generated privilege-escalation environments, followed by reinforcement learning with verifiable rewards. On a held-out benchmark of 12 Linux privilege-escalation scenarios, supervised fine-tuning alone more than doubles the baseline success rate at 20 rounds, and reinforcement learning further lifts our resulting model, PrivEsc-LLM, to 95.8%, nearly matching Claude Opus 4.6 at 97.5%. At the same time, the expected inference cost per successful escalation is reduced by over 100x.

翻译：LLM代理在漏洞发现等研究领域的应用日益广泛，但现有最强系统仍为封闭式云端架构，存在资源消耗高、难以复现且不适用于涉及专有代码或敏感数据的场景。因此，亟需能够在严格资源预算下执行安全任务的小型本地化模型，然而相关开发方法仍处于探索阶段。本文针对这一空白，提出两阶段后训练流水线，聚焦于可自动验证成功性的Linux权限提升问题——该任务需要多步交互式推理。通过构建防止数据泄露的实验环境，我们对4B参数模型进行两阶段后训练：首先在程序化生成的特权提升环境轨迹上进行监督微调，随后采用可验证奖励的强化学习。在包含12个Linux权限提升场景的保留测试中，仅监督微调阶段即在20轮交互内使基线成功率提升两倍以上，而强化学习进一步将所得模型PrivEsc-LLM的成功率提升至95.8%，近乎持平Claude Opus 4.6的97.5%。同时，每次成功提权所需的推理成本降低超过100倍。