The rapid development of modern artificial intelligence (AI) systems has created an urgent need for their scientific quantification. While their fluency across a variety of domains is impressive, modern AI systems fall short on tests requiring symbolic processing and abstraction - a glaring limitation given the necessity for interpretable and reliable technology. Despite a surge of reasoning benchmarks emerging from the academic community, no comprehensive and theoretically-motivated framework exists to quantify reasoning (and more generally, symbolic ability) in AI systems. Here, we adopt a framework from computational complexity theory to explicitly quantify symbolic generalization: algebraic circuit complexity. Many symbolic reasoning problems can be recast as algebraic expressions. Thus, algebraic circuit complexity theory - the study of algebraic expressions as circuit models (i.e., directed acyclic graphs) - is a natural framework to study the complexity of symbolic computation. The tools of algebraic circuit complexity enable the study of generalization by defining benchmarks in terms of their complexity-theoretic properties (i.e., the difficulty of a problem). Moreover, algebraic circuits are generic mathematical objects; for a given algebraic circuit, an arbitrarily large number of samples can be generated for a specific circuit, making it an optimal testbed for the data-hungry machine learning algorithms that are used today. Here, we adopt tools from algebraic circuit complexity theory, apply it to formalize a science of symbolic generalization, and address key theoretical and empirical challenges for its successful application to AI science and its impact on the broader community.

翻译：现代人工智能（AI）系统的快速发展催生了对其科学量化的迫切需求。尽管这些系统在多个领域展现出令人印象深刻的流畅性，但在需要符号处理和抽象能力的测试中仍显不足——考虑到对可解释且可靠技术的需求，这一局限尤为突出。尽管学术界涌现了大量推理基准测试，但目前仍缺乏一个全面且具有理论动机的框架来量化AI系统的推理能力（更广义地说，符号能力）。本文采用计算复杂性理论中的框架——代数电路复杂性——来显式量化符号泛化能力。许多符号推理问题可被重构为代数表达式。因此，代数电路复杂性理论（将代数表达式作为电路模型即定向无环图进行研究）成为研究符号计算复杂性的自然框架。代数电路复杂性工具通过基于问题的复杂性理论特性（即问题的难度）定义基准，使得泛化研究成为可能。此外，代数电路是通用的数学对象；对于给定的代数电路，可为特定电路生成任意数量的样本，这使其成为当前数据密集型机器学习算法的理想测试平台。本文采用代数电路复杂性理论工具，将其应用于形式化符号泛化科学，并解决其在AI科学成功应用及对更广泛领域产生影响过程中面临的关键理论与实证挑战。