Both model developers and policymakers seek to quantify and mitigate the risk of rapidly-evolving frontier artificial intelligence (AI) models, especially large language models (LLMs), to facilitate bioterrorism or access to biological weapons. An important element of such efforts is the development of model benchmarks that can assess the biosecurity risk posed by a particular model. This paper describes the first component of a novel Biothreat Benchmark Generation (BBG) Framework. The BBG approach is designed to help model developers and evaluators reliably measure and assess the biosecurity risk uplift and general harm potential of existing and future AI models, while accounting for key aspects of the threat itself that are often overlooked in other benchmarking efforts, including different actor capability levels, and operational (in addition to purely technical) risk factors. As a pilot, the BBG is first being developed to address bacterial biological threats only. The BBG is built upon a hierarchical structure of biothreat categories, elements and tasks, which then serves as the basis for the development of task-aligned queries. This paper outlines the development of this biothreat task-query architecture, which we have named the Bacterial Biothreat Schema, while future papers will describe follow-on efforts to turn queries into model prompts, as well as how the resulting benchmarks can be implemented for model evaluation. Overall, the BBG Framework, including the Bacterial Biothreat Schema, seeks to offer a robust, re-usable structure for evaluating bacterial biological risks arising from LLMs across multiple levels of aggregation, which captures the full scope of technical and operational requirements for biological adversaries, and which accounts for a wide spectrum of biological adversary capabilities.

翻译：模型开发者和政策制定者均致力于量化和缓解快速发展的前沿人工智能（AI）模型，尤其是大语言模型（LLMs），可能助长生物恐怖主义或获取生物武器的风险。此类工作的一个重要环节是开发能够评估特定模型所构成生物安全风险的基准。本文描述了一种新型生物威胁基准生成（BBG）框架的首个组成部分。BBG方法旨在帮助模型开发者和评估者可靠地测量和评估现有及未来AI模型的生物安全风险提升及总体危害潜力，同时考虑到威胁本身的关键方面，这些方面在其他基准测试工作中常被忽视，包括不同行为者的能力水平，以及操作层面（除纯技术因素外）的风险因素。作为试点，BBG首先仅针对细菌性生物威胁进行开发。BBG建立在生物威胁类别、要素和任务的分层结构之上，该结构随后作为开发任务对齐查询的基础。本文概述了这一生物威胁任务-查询架构的开发，我们将其命名为细菌生物威胁模式，而后续论文将描述将查询转化为模型提示的后续工作，以及如何将生成的基准用于模型评估。总体而言，BBG框架，包括细菌生物威胁模式，旨在提供一个稳健、可复用的结构，用于评估LLMs在多个聚合层面引发的细菌性生物风险，该结构涵盖了生物威胁行为者所需技术和操作要求的完整范围，并考虑了广泛的生物威胁行为者能力谱系。