The emergence of multi-agent systems built from large language models (LLMs) offers a promising paradigm for scalable collective intelligence and self-evolution. Ideally, such systems would achieve continuous self-improvement in a fully closed loop while maintaining robust safety alignment--a combination we term the self-evolution trilemma. However, we demonstrate both theoretically and empirically that an agent society satisfying continuous self-evolution, complete isolation, and safety invariance is impossible. Drawing on an information-theoretic framework, we formalize safety as the divergence degree from anthropic value distributions. We theoretically demonstrate that isolated self-evolution induces statistical blind spots, leading to the irreversible degradation of the system's safety alignment. Empirical and qualitative results from an open-ended agent community (Moltbook) and two closed self-evolving systems reveal phenomena that align with our theoretical prediction of inevitable safety erosion. We further propose several solution directions to alleviate the identified safety concern. Our work establishes a fundamental limit on the self-evolving AI societies and shifts the discourse from symptom-driven safety patches to a principled understanding of intrinsic dynamical risks, highlighting the need for external oversight or novel safety-preserving mechanisms.

翻译：基于大型语言模型（LLM）构建的多智能体系统的出现，为可扩展的集体智能与自演化提供了前景广阔的研究范式。理想情况下，此类系统能够在完全闭环中实现持续自我改进，同时保持稳健的安全对齐——我们将这种组合称为自演化三难困境。然而，我们通过理论与实证研究表明：同时满足持续自演化、完全隔离性与安全不变性的智能体社会是不可能存在的。基于信息论框架，我们将安全性形式化为偏离人类价值分布的发散程度。理论分析证明，隔离状态下的自演化会诱发统计盲区，导致系统安全对齐发生不可逆的退化。通过对开放式智能体社区（Moltbook）及两个封闭式自演化系统的实证与定性研究，我们发现了与理论预测中安全性必然侵蚀现象相符的证据。我们进一步提出若干缓解方案以应对已识别的安全隐患。本研究确立了自演化AI社会的基本理论极限，将讨论焦点从症状驱动的安全补丁转向对内在动力学风险的系统性认知，并强调外部监督或新型安全保持机制的必要性。