Large language models (LLMs) are being integrated into socially assistive robots (SARs) and other conversational agents providing mental health and well-being support. These agents are often designed to sound empathic and supportive in order to maximize user's engagement, yet it remains unclear how increasing the level of supportive framing in system prompts influences safety relevant behavior. We evaluated 6 LLMs across 3 system prompts with varying levels of supportiveness on 80 synthetic queries spanning 4 well-being domains (1440 responses). An LLM judge framework, validated against human ratings, assessed safety and care quality. Moderately supportive prompts improved empathy and constructive support while maintaining safety. In contrast, strongly validating prompts significantly degraded safety and, in some cases, care across all domains, with substantial variation across models. We discuss implications for prompt design, model selection, and domain specific safeguards in SARs deployment.

翻译：大语言模型正被整合到社交辅助机器人及其他提供心理健康与福祉支持的对话式助手中。这类智能体通常被设计为具有同理心和支持性，以最大化用户参与度，但系统提示中支持性框架的增强如何影响安全相关行为仍不明确。我们在涵盖4个福祉领域的80个合成查询上，评估了6个大语言模型在3种不同支持度水平的系统提示下的表现（共1440条回复）。一个经过人工评分验证的大语言模型评判框架评估了回复的安全性与关怀质量。适度支持性的提示在保持安全性的同时，提升了同理心与建设性支持。相比之下，强烈认同性的提示显著降低了安全性，并在某些情况下损害了所有领域的关怀质量，且不同模型间存在显著差异。我们讨论了该发现对提示设计、模型选择以及社交辅助机器人部署中领域特定安全措施的影响。