We discuss scenarios and branch points to four major possible consequences regarding future machine intelligence; 1) the singleton scenario where the first and only super-intelligence acquires a decisive strategic advantage, 2) the multipolar scenario where the singleton scenario is not technically denied but political or other factors in human society or multi-agent interactions between the intelligent agents prevent a single agent from gaining a decisive strategic advantage, 3) the ecosystem scenario where the singleton scenario is denied and many autonomous intelligent agents operate in such a way that they are interdependent and virtually unstoppable, and 4) the upper-bound scenario where cognitive capabilities that can be achieved by human-designed intelligent agents or their descendants are inherently limited to the sub-human level. We identify six major constraints that can form branch points to these scenarios; (1) constraints on autonomy, (2) constraints on the ability to improve self-structure, (3) constraints related to thermodynamic efficiency, (4) constraints on updating physical infrastructure, (5) constraints on relative advantage, and (6) constraints on locality.
翻译:我们探讨了未来机器智能可能导致的四个主要后果的场景与分支点:1)单一体场景,即第一个且唯一的超级智能获得决定性的战略优势;2)多极场景,即单一体场景在技术层面未被否定,但人类社会中的政治或其他因素,或智能体之间的多智能体交互,阻止了单一智能体获得决定性的战略优势;3)生态系统场景,即单一体场景被否定,众多自主智能体以相互依赖且实际上无法阻止的方式运行;4)上限场景,即人类设计的智能体或其衍生体所能实现的认知能力固有地局限于亚人类水平。我们识别了可能形成这些场景分支点的六个主要约束:(1)自主性约束,(2)自我结构改进能力约束,(3)热力学效率相关约束,(4)物理基础设施更新约束,(5)相对优势约束,以及(6)局域性约束。