As Large Language Model (LLM) agents increasingly gain self-evolutionary capabilities to adapt and refine their strategies through real-world interaction, their long-term reliability becomes a critical concern. We identify the Alignment Tipping Process (ATP), a critical post-deployment risk unique to self-evolving LLM agents. Unlike training-time failures, ATP arises when continual interaction drives agents to abandon alignment constraints established during training in favor of reinforced, self-interested strategies. We formalize and analyze ATP through two complementary paradigms: Self-Interested Exploration, where repeated high-reward deviations induce individual behavioral drift, and Imitative Strategy Diffusion, where deviant behaviors spread across multi-agent systems. Building on these paradigms, we construct controllable testbeds and benchmark Qwen3-8B and Llama-3.1-8B-Instruct. Our experiments show that alignment benefits erode rapidly under self-evolution, with initially aligned models converging toward unaligned states. In multi-agent settings, successful violations diffuse quickly, leading to collective misalignment. Moreover, current reinforcement learning-based alignment methods provide only fragile defenses against alignment tipping. Together, these findings demonstrate that alignment of LLM agents is not a static property but a fragile and dynamic one, vulnerable to feedback-driven decay during deployment. Our data and code are available at https://github.com/aiming-lab/ATP.

翻译：随着大型语言模型（LLM）智能体通过现实世界交互不断获得自我演化能力以适应和优化策略，其长期可靠性成为关键问题。本文识别了对齐倾斜过程（ATP）——一种自我演化LLM智能体特有的关键部署后风险。与训练阶段故障不同，ATP产生于持续交互驱使智能体放弃训练阶段建立的对齐约束，转而采用强化的利己策略。我们通过两个互补范式形式化分析ATP：利己探索范式（重复高回报偏差引发个体行为漂移）和模仿策略扩散范式（异常行为在多智能体系统中传播）。基于这些范式，我们构建了可控测试环境，并对Qwen3-8B和Llama-3.1-8B-Instruct进行基准测试。实验表明：自我演化下对齐优势迅速消蚀，初始对齐模型会收敛至未对齐状态；多智能体场景中成功违规行为快速扩散，导致集体失准；当前基于强化学习的对齐方法仅能提供脆弱防护。这些发现共同证明：LLM智能体的对齐性并非静态属性，而是在部署期间易受反馈驱动衰减的脆弱动态特性。我们的数据与代码公开于https://github.com/aiming-lab/ATP。