AI's central challenge is shifting from capability to coexistence. The dominant paradigm in AI research focuses on developing powerful agents that treat the world as an exogenous and stationary source of feedback. We contend that superintelligence, an extremely capable task solver, born out of such a solipsistic approach to AI design, is unlikely to be cooperative. Deploying AI systems induces endogenous non-stationarity, resulting in a train-test-deploy gap where historical distributions diverge from the deployment context. We refer to this as the self-undermining property of unilateral optimization. Closing this gap requires AI that participates in cooperation: the equilibrium-selection process through which multiple actors navigate their interdependence. We call for a non-solipsistic research paradigm that treats this interdependence as a core design principle rather than approaching cooperation as a task to solve. This entails building dynamic evaluation testbeds involving adaptive counterparties, treating institutions as design primitives, and preserving human agency as a structural feature of the systems we build.
翻译:人工智能的核心挑战正从能力转向共存。当前人工智能研究的主流范式聚焦于开发将世界视为外生且固定反馈源的强大智能体。我们认为,源自这种唯我论式人工智能设计方法的超级智能(一种极其强大的任务求解器)将难以合作。部署人工智能系统会引发内生的非平稳性,导致训练-测试-部署的分布鸿沟——历史数据分布与部署环境产生偏离。我们将此称为单边优化的自损特性。弥合这一鸿沟需要人工智能参与合作:即多个行动者在相互依存关系中通过均衡选择进行协调的过程。我们呼吁建立非唯我论的研究范式,将这种相互依存性视为核心设计原则,而非将合作视为待解决的任务。这意味着需要构建包含自适应对手方的动态测试平台,将制度视为设计原语,并将人类能动性作为所构建系统的结构性特征予以保留。