Background: Patient-facing medical chatbots based on retrieval-augmented generation (RAG) are increasingly promoted to deliver accessible, grounded health information. AI-assisted development lowers the barrier to building them, but they still demand rigorous security, privacy, and governance controls. Objective: To report an anonymized, non-destructive security assessment of a publicly accessible patient-facing medical RAG chatbot and identify governance lessons for safe deployment of generative AI in health. Methods: We used a two-stage strategy. First, Claude Opus 4.6 supported exploratory prompt-based testing and structured vulnerability hypotheses. Second, candidate findings were manually verified using Chrome Developer Tools, inspecting browser-visible network traffic, payloads, API schemas, configuration objects, and stored interaction data. Results: The LLM-assisted phase identified a critical vulnerability: sensitive system and RAG configuration appeared exposed through client-server communication rather than restricted server-side. Manual verification confirmed that ordinary browser inspection allowed collection of the system prompt, model and embedding configuration, retrieval parameters, backend endpoints, API schema, document and chunk metadata, knowledge-base content, and the 1,000 most recent patient-chatbot conversations. The deployment also contradicted its privacy assurances: full conversation records, including health-related queries, were retrievable without authentication. Conclusions: Serious privacy and security failures in patient-facing RAG chatbots can be identified with standard browser tools, without specialist skills or authentication; independent review should be a prerequisite for deployment. Commercial LLMs accelerated this assessment, including under a false developer persona; assistance available to auditors is equally available to adversaries.

翻译：背景：基于检索增强生成（RAG）的面向患者医疗聊天机器人正日益推广，以提供可获取、有据可查的健康信息。AI辅助开发降低了构建此类系统的门槛，但对其安全性、隐私性和治理控制提出了严格要求。目的：报告对公开可访问的面向患者医疗RAG聊天机器人进行的匿名化、非破坏性安全评估，并总结健康领域生成式AI安全部署的治理经验。方法：我们采用两阶段策略。首先，Claude Opus 4.6支持了基于提示的探索性测试和结构化漏洞假设。其次，利用Chrome开发者工具手动验证候选发现，检查浏览器可见的网络流量、负载、API模式、配置对象及存储的交互数据。结果：大语言模型辅助阶段识别出一个关键漏洞：敏感系统与RAG配置通过客户端-服务器通信暴露，而非限制在服务器端。手动验证证实，通过普通浏览器检查即可收集系统提示、模型与嵌入配置、检索参数、后端端点、API模式、文档与块元数据、知识库内容，以及最近的1000条患者-聊天机器人对话记录。该部署还与其隐私承诺相矛盾：包括健康相关查询在内的完整对话记录在无身份验证的情况下即可检索。结论：使用标准浏览器工具无需专业技能或身份验证即可发现面向患者RAG聊天机器人的严重隐私与安全缺陷；独立审查应作为部署的前提条件。商业大语言模型加速了本次评估（包括在虚假开发者身份下进行）；审计人员可获得的辅助能力同样可被攻击者利用。