The concept of intelligent software is flawed. The behaviour of software is determined by the hardware that "interprets" it. This undermines claims regarding the behaviour of theorised, software superintelligence. Here we characterise this problem as "computational dualism", where instead of mental and physical substance, we have software and hardware. We argue that to make objective claims regarding performance we must avoid computational dualism. We propose a pancomputational alternative wherein every aspect of the environment is a relation between irreducible states. We formalise systems as behaviour (inputs and outputs), and cognition as embodied, embedded, extended and enactive. The result is cognition formalised as a part of the environment, rather than as a disembodied policy interacting with the environment through an interpreter. This allows us to make objective claims regarding intelligence, which we argue is the ability to "generalise", identify causes and adapt. We then establish objective upper bounds for intelligent behaviour. This suggests AGI will be safer, but more limited, than theorised.

翻译：智能软件的概念存在缺陷。软件的行为由"解释"它的硬件决定，这削弱了关于理论化的软件超级智能行为的论断。本文将此问题定性为"计算二元论"，即用软件与硬件取代了心智与物质实体。我们认为，要做出关于性能的客观论断，就必须避免计算二元论。我们提出一种泛计算替代方案，其中环境的每个方面都是不可约状态之间的关系。我们将系统形式化为行为（输入与输出），将认知形式化为具身、嵌入、延展和生成的。其结果是，认知被形式化为环境的一部分，而非通过解释器与环境交互的离身策略。这使我们能够对智能做出客观论断，我们认为智能是"泛化"、识别原因和适应的能力。随后我们建立了智能行为的客观上界。这表明通用人工智能将比理论化的更安全，但也更受限。