Large language models (LLMs) such as those embedded in 'chatbots' are accelerating and democratizing research by providing comprehensible information and expertise from many different fields. However, these models may also confer easy access to dual-use technologies capable of inflicting great harm. To evaluate this risk, the 'Safeguarding the Future' course at MIT tasked non-scientist students with investigating whether LLM chatbots could be prompted to assist non-experts in causing a pandemic. In one hour, the chatbots suggested four potential pandemic pathogens, explained how they can be generated from synthetic DNA using reverse genetics, supplied the names of DNA synthesis companies unlikely to screen orders, identified detailed protocols and how to troubleshoot them, and recommended that anyone lacking the skills to perform reverse genetics engage a core facility or contract research organization. Collectively, these results suggest that LLMs will make pandemic-class agents widely accessible as soon as they are credibly identified, even to people with little or no laboratory training. Promising nonproliferation measures include pre-release evaluations of LLMs by third parties, curating training datasets to remove harmful concepts, and verifiably screening all DNA generated by synthesis providers or used by contract research organizations and robotic cloud laboratories to engineer organisms or viruses.

翻译：大型语言模型（LLM），例如嵌入在“聊天机器人”中的那些，正通过提供来自多个领域的易懂信息和专业知识，加速并推动研究的民主化。然而，这些模型也可能为获取能造成巨大伤害的双重用途技术提供便利。为评估这一风险，麻省理工学院开设的“守护未来”课程让非科学背景的学生研究LLM聊天机器人是否能被诱导，协助非专业人士引发大流行病。在一小时内，聊天机器人提出了四种潜在的大流行病原体，解释了如何利用反向遗传学从合成DNA生成它们，提供了不太可能筛查订单的DNA合成公司名称，详述了具体方案及其故障排除方法，并建议任何缺乏反向遗传学技能的人寻求核心设施或合同研究机构的帮助。综合来看，这些结果表明，一旦可信地识别出大流行类病原体，LLM将使其广泛可及，甚至对几乎没有实验室培训经历的人也是如此。有前景的不扩散措施包括：由第三方对LLM进行发布前评估，精心整理训练数据集以消除有害概念，以及可验证地筛查所有由合成提供商生成的DNA、或被合同研究机构和机器人化云端实验室用于工程改造生物体或病毒的DNA。