Designing rewards for autonomous cyber attack and defense learning agents in a complex, dynamic environment is a challenging task for subject matter experts. We propose a large language model (LLM)-based reward design approach to generate autonomous cyber defense policies in a deep reinforcement learning (DRL)-driven experimental simulation environment. Multiple attack and defense agent personas were crafted, reflecting heterogeneity in agent actions, to generate LLM-guided reward designs where the LLM was first provided with contextual cyber simulation environment information. These reward structures were then utilized within a DRL-driven attack-defense simulation environment to learn an ensemble of cyber defense policies. Our results suggest that LLM-guided reward designs can lead to effective defense strategies against diverse adversarial behaviors.

翻译：在复杂动态环境中为自主网络攻防学习智能体设计奖励函数，对领域专家而言是一项具有挑战性的任务。本文提出一种基于大语言模型（LLM）的奖励设计方法，用于在深度强化学习（DRL）驱动的实验仿真环境中生成自主网络防御策略。研究构建了多种攻防智能体角色，以体现智能体行为的异质性，从而生成LLM引导的奖励设计方案——首先向LLM提供网络仿真环境的上下文信息。随后，在DRL驱动的攻防仿真环境中利用这些奖励结构，学习出一组网络防御策略集合。实验结果表明，LLM引导的奖励设计能够针对多样化的对抗行为产生有效的防御策略。