Large Language Models (LLMs) have demonstrated impressive abilities in symbol processing through in-context learning (ICL). This success flies in the face of decades of predictions that artificial neural networks cannot master abstract symbol manipulation. We seek to understand the mechanisms that can enable robust symbol processing in transformer networks, illuminating both the unanticipated success, and the significant limitations, of transformers in symbol processing. Borrowing insights from symbolic AI on the power of Production System architectures, we develop a high-level language, PSL, that allows us to write symbolic programs to do complex, abstract symbol processing, and create compilers that precisely implement PSL programs in transformer networks which are, by construction, 100% mechanistically interpretable. We demonstrate that PSL is Turing Universal, so the work can inform the understanding of transformer ICL in general. The type of transformer architecture that we compile from PSL programs suggests a number of paths for enhancing transformers' capabilities at symbol processing. (Note: The first section of the paper gives an extended synopsis of the entire paper.)

翻译：大型语言模型（LLM）通过情境学习（ICL）在符号处理方面展现出卓越能力。这一成功与数十年来关于人工神经网络无法掌握抽象符号操作的预测背道而驰。本研究旨在探究Transformer网络实现稳健符号处理的内在机制，从而阐明Transformer在符号处理中既取得意外成功又存在显著局限性的深层原因。借鉴符号人工智能中产生式系统架构的理论洞见，我们开发了一种高级编程语言PSL，该语言支持编写执行复杂抽象符号处理的符号程序，并构建了能够将PSL程序精确编译为Transformer网络的编译器。通过构造，这些Transformer网络具备100%的机制可解释性。我们证明了PSL具有图灵完备性，因此本研究成果可为理解Transformer的通用情境学习机制提供理论依据。基于PSL程序编译得到的Transformer架构类型，为增强Transformer的符号处理能力提出了若干技术路径。（注：论文首章包含对全文的扩展性概要说明。）