In this paper, we identify and characterize the emerging area of representation engineering (RepE), an approach to enhancing the transparency of AI systems that draws on insights from cognitive neuroscience. RepE places population-level representations, rather than neurons or circuits, at the center of analysis, equipping us with novel methods for monitoring and manipulating high-level cognitive phenomena in deep neural networks (DNNs). We provide baselines and an initial analysis of RepE techniques, showing that they offer simple yet effective solutions for improving our understanding and control of large language models. We showcase how these methods can provide traction on a wide range of safety-relevant problems, including honesty, harmlessness, power-seeking, and more, demonstrating the promise of top-down transparency research. We hope that this work catalyzes further exploration of RepE and fosters advancements in the transparency and safety of AI systems.

翻译：本文识别并系统阐述了表征工程（RepE）这一新兴研究方向。该方向借鉴认知神经科学的研究视角，旨在提升人工智能系统的透明度。与以神经元或神经回路为核心的传统分析方法不同，表征工程将群体级表征置于分析中心，为我们监控和操控深度神经网络中的高层级认知现象提供了全新方法。我们建立了该技术的基础基准并进行了初步分析，结果表明这些方法为改进我们对大型语言模型的理解与控制提供了简单而有效的解决方案。通过展示这些方法在诚实性、无害性、权力寻求等广泛安全相关问题中的应用潜力，本研究充分证明了自上而下透明度研究的前景。我们期望这项工作能推动表征工程的进一步发展，促进人工智能系统透明度与安全性研究取得突破。