Conversational agents powered by large language models (LLM) have increasingly been utilized in the realm of mental well-being support. However, the implications and outcomes associated with their usage in such a critical field remain somewhat ambiguous and unexplored. We conducted a qualitative analysis of 120 posts, encompassing 2917 user comments, drawn from the most popular subreddit focused on mental health support applications powered by large language models (u/Replika). This exploration aimed to shed light on the advantages and potential pitfalls associated with the integration of these sophisticated models in conversational agents intended for mental health support. We found the app (Replika) beneficial in offering on-demand, non-judgmental support, boosting user confidence, and aiding self-discovery. Yet, it faced challenges in filtering harmful content, sustaining consistent communication, remembering new information, and mitigating users' overdependence. The stigma attached further risked isolating users socially. We strongly assert that future researchers and designers must thoroughly evaluate the appropriateness of employing LLMs for mental well-being support, ensuring their responsible and effective application.

翻译：大型语言模型驱动的对话代理已越来越多地应用于心理健康支持领域。然而，此类关键领域中的使用所带来的影响与结果仍存在一定的不确定性与研究空白。我们对来自聚焦于大型语言模型（u/Replika）驱动心理健康支持应用的热门子版块中的120篇帖子（涵盖2917条用户评论）进行了定性分析。本研究旨在揭示将这些复杂模型整合至面向心理健康支持的对话代理中所带来的优势与潜在风险。研究发现，该应用（Replika）在提供即时、非评判性支持、增强用户自信及辅助自我探索方面具有积极作用。然而，其亦面临有害内容过滤困难、对话连贯性维持不足、新信息记忆能力有限以及用户过度依赖问题缓解不力等挑战。此外，与之相关的污名化现象进一步加剧了用户的社会孤立风险。我们强烈主张，未来研究者和设计者必须全面评估将大型语言模型用于心理健康支持的适当性，确保其负责任且有效地应用。