LLM-enabled applications are rapidly reshaping the software ecosystem by using large language models as core reasoning components for complex task execution. This paradigm shift, however, introduces fundamentally new reliability challenges and significantly expands the security attack surface, due to the non-deterministic, learning-driven, and difficult-to-verify nature of LLM behavior. In light of these emerging and unavoidable safety challenges, we argue that such risks should be treated as expected operational conditions rather than exceptional events, necessitating a dedicated incident-response perspective. Consequently, the primary barrier to trustworthy deployment is not further improving model capability but establishing system-level threat monitoring mechanisms that can detect and contextualize security-relevant anomalies after deployment -- an aspect largely underexplored beyond testing or guardrail-based defenses. Accordingly, this position paper advocates systematic and comprehensive monitoring of security threats in LLM-enabled applications as a prerequisite for reliable operation and a foundation for dedicated incident-response frameworks.
翻译:基于大语言模型的应用程序正通过将大语言模型作为复杂任务执行的核心推理组件,迅速重塑软件生态系统。然而,由于大语言模型行为具有非确定性、学习驱动且难以验证的特性,这种范式转变引入了根本性的新型可靠性挑战,并显著扩大了安全攻击面。鉴于这些新兴且不可避免的安全挑战,我们认为此类风险应被视为预期的运行条件而非异常事件,需要采取专门的事件响应视角。因此,可信部署的主要障碍并非进一步提升模型能力,而是建立系统级威胁监控机制,以便在部署后检测并关联安全相关异常——这一方面在测试或基于护栏的防御之外,目前很大程度上尚未得到充分探索。为此,本立场文件主张对基于大语言模型的应用程序进行系统全面的安全威胁监控,将其作为可靠运行的前提条件和专门事件响应框架的基础。