Applying LLM-based multi-agent software systems in safety-critical domains such as lifespan echocardiography introduces system-level risks that cannot be addressed by improving model accuracy alone. During system operation, beyond individual LLM behavior, uncertainty propagates through agent coordination, data pipelines, human-in-the-loop interaction, and runtime control logic. Yet existing work largely treats uncertainty at the model level rather than as a first-class software engineering concern. This paper approaches uncertainty from both system-level and runtime perspectives. We first differentiate epistemological and ontological uncertainties in the context of LLM-based multi-agent software system operation. Building on this foundation, we propose a lifecycle-based uncertainty management framework comprising four mechanisms: representation, identification, evolution, and adaptation. The uncertainty lifecycle governs how uncertainties emerge, transform, and are mitigated across architectural layers and execution phases, enabling structured runtime governance and controlled adaptation. We demonstrate the feasibility of the framework using a real-world LLM-based multi-agent echocardiographic software system developed in clinical collaboration, showing improved reliability and diagnosability in diagnostic reasoning. The proposed approach generalizes to other safety-critical LLM-based multi-agent software systems, supporting principled operational control and runtime assurance beyond model-centric methods.
翻译:在寿命超声心动图等安全关键领域应用基于LLM的多智能体软件系统,会引入无法仅通过提升模型准确性来解决的系统级风险。在系统运行过程中,不确定性不仅源于单个LLM的行为,还会通过智能体协同、数据流水线、人在回路交互以及运行时控制逻辑进行传播。然而现有研究大多将不确定性视为模型层面的问题,而非作为首要的软件工程关切点。本文从系统级和运行时双重视角探讨不确定性问题。我们首先区分了基于LLM的多智能体软件系统运行背景下的认知不确定性与本体不确定性。在此基础上,我们提出了一个基于生命周期的四机制不确定性管理框架,包含表征、识别、演化和适应机制。该不确定性生命周期描述了不确定性如何在架构层和执行阶段中产生、转化与消减,从而实现结构化的运行时治理与受控适应。我们通过临床合作开发的实际基于LLM的多智能体超声心动图软件系统验证了该框架的可行性,证明了其在诊断推理中可提升系统可靠性与可诊断性。所提出的方法可推广至其他安全关键的基于LLM的多智能体软件系统,为超越以模型为中心的方法、实现原则性运行控制与运行时保障提供支持。