Large language models are increasingly embedded in regulated and safety-critical software, including clinical research platforms and healthcare information systems. While these features enable natural language search, summarization, and configuration assistance, they introduce risks such as hallucinations, harmful or out-of-scope advice, privacy and security issues, bias, instability under change, and adversarial misuse. Prior work on machine learning testing and AI assurance offers useful concepts but limited guidance for interactive, product-embedded assistants. This paper proposes a risk-based testing framework for LLM features in regulated software: a six-category risk taxonomy, a layered test strategy mapping risks to concrete tests across guardrail, orchestration, and system layers, and a case study applying the approach to a Knowledgebase assistant in a clinical research platform.

翻译：大型语言模型正日益嵌入受监管及安全关键型软件中，包括临床研究平台和医疗信息系统。尽管这些功能实现了自然语言搜索、摘要生成和配置辅助，但也带来了幻觉、有害或超范围建议、隐私安全问题、偏见、变更下的不稳定性以及对抗性滥用等风险。现有的机器学习测试与人工智能保障研究提供了有益概念，但对交互式产品嵌入式助手的指导有限。本文提出了一种面向受监管软件中LLM功能的风险测试框架：包含六类风险分类体系、将风险映射到防护栏层、编排层和系统层具体测试的分层测试策略，以及将该方法应用于临床研究平台知识库助手的案例研究。