What is the prospect of developing artificial general intelligence (AGI)? I investigate this question by systematically comparing living and algorithmic systems, with a special focus on the notion of "agency." There are three fundamental differences to consider: (1) Living systems are autopoietic, that is, self-manufacturing, and therefore able to set their own intrinsic goals, while algorithms exist in a computational environment with target functions that are both provided by an external agent. (2) Living systems are embodied in the sense that there is no separation between their symbolic and physical aspects, while algorithms run on computational architectures that maximally isolate software from hardware. (3) Living systems experience a large world, in which most problems are ill-defined (and not all definable), while algorithms exist in a small world, in which all problems are well-defined. These three differences imply that living and algorithmic systems have very different capabilities and limitations. In particular, it is extremely unlikely that true AGI (beyond mere mimicry) can be developed in the current algorithmic framework of AI research. Consequently, discussions about the proper development and deployment of algorithmic tools should be shaped around the dangers and opportunities of current narrow AI, not the extremely unlikely prospect of the emergence of true agency in artificial systems.

翻译：开发通用人工智能（AGI）的前景如何？我通过系统比较生命系统和算法系统来探究这一问题，特别关注“能动性”概念。需考虑三个根本差异：（1）生命系统是自创生的，即自我制造，因此能够设定自身内在目标；而算法存在于计算环境中，其目标函数均由外部智能体提供。（2）生命系统是具身的，其符号层面与物理层面不可分割；而算法运行在最大限度隔离软硬件的计算架构上。（3）生命系统体验的是一个“大世界”，其中多数问题定义不良（且并非所有问题都可定义）；而算法存在于“小世界”中，所有问题均有良好定义。这三个差异表明，生命系统与算法系统具有截然不同的能力与局限性。尤其值得指出的是，在现行AI研究的算法框架下，极不可能发展出真正的AGI（超越单纯的模仿）。因此，关于算法工具合理开发与部署的讨论，应聚焦于当前狭义AI的风险与机遇，而非人工系统中涌现真正能动性这一极不现实的前景。