The rapid advancement of open-source foundation models has brought transparency and accessibility to this groundbreaking technology. However, this openness has also enabled the development of highly-capable, unsafe models, as exemplified by recent instances such as WormGPT and FraudGPT, which are specifically designed to facilitate criminal activity. As the capabilities of open foundation models continue to grow, potentially outpacing those of closed-source models, the risk of misuse by bad actors poses an increasingly serious threat to society. This paper addresses the critical question of how open foundation model developers should approach model safety in light of these challenges. Our analysis reveals that open-source foundation model companies often provide less restrictive acceptable use policies (AUPs) compared to their closed-source counterparts, likely due to the inherent difficulties in enforcing such policies once the models are released. To tackle this issue, we introduce PRISM, a design framework for open-source foundation model safety that emphasizes Private, Robust, Independent Safety measures, at Minimal marginal cost of compute. The PRISM framework proposes the use of modular functions that moderate prompts and outputs independently of the core language model, offering a more adaptable and resilient approach to safety compared to the brittle reinforcement learning methods currently used for value alignment. By focusing on identifying AUP violations and engaging the developer community in establishing consensus around safety design decisions, PRISM aims to create a safer open-source ecosystem that maximizes the potential of these powerful technologies while minimizing the risks to individuals and society as a whole.

翻译：开源基础模型的快速发展为这一突破性技术带来了透明度和可访问性。然而，这种开放性也催生了能力强大但具有安全隐患的模型，近期出现的WormGPT和FraudGPT等专门为犯罪活动设计的模型便是例证。随着开源基础模型能力持续增强，甚至可能超越闭源模型，恶意行为者滥用模型对社会构成日益严重的威胁。本文针对开源基础模型开发者应如何应对这些挑战来处理模型安全性的关键问题展开研究。我们的分析表明，与闭源模型相比，开源基础模型公司通常提供限制性较低的可接受使用政策（AUPs），这很可能源于模型发布后执行此类政策存在固有困难。为解决这一问题，我们提出PRISM框架——一种强调以最小边际计算成本实现私有化、鲁棒化、独立化安全措施的开源基础模型安全性设计框架。PRISM框架建议采用与核心语言模型分离的模块化函数来调节提示词和输出，相比当前用于价值观对齐的脆弱强化学习方法，该框架提供了更具适应性和韧性的安全解决方案。通过聚焦于识别AUP违规行为，并推动开发者社区就安全设计决策建立共识，PRISM旨在构建更安全的开源生态系统，在充分发挥这些强大技术潜力的同时，最大限度降低对个人和社会的整体风险。