Eating disorders (ED) are complex mental health conditions that require long-term management and support. Recent advancements in large language model (LLM)-based chatbots offer the potential to assist individuals in receiving immediate support. Yet, concerns remain about their reliability and safety in sensitive contexts such as ED. We explore the opportunities and potential harms of using LLM-based chatbots for ED recovery. We observe the interactions between 26 participants with ED and an LLM-based chatbot, WellnessBot, designed to support ED recovery, over 10 days. We discovered that our participants have felt empowered in recovery by discussing ED-related stories with the chatbot, which served as a personal yet social avenue. However, we also identified harmful chatbot responses, especially concerning individuals with ED, that went unnoticed partly due to participants' unquestioning trust in the chatbot's reliability. Based on these findings, we provide design implications for safe and effective LLM-based interventions in ED management.

翻译：饮食失调（ED）是一种复杂的心理健康状况，需要长期的管理与支持。基于大语言模型（LLM）的聊天机器人的最新进展，为个体获得即时支持提供了可能。然而，在ED等敏感情境中，其可靠性与安全性仍存疑虑。本文探讨了使用基于LLM的聊天机器人辅助ED康复的机遇与潜在危害。我们观察了26名ED参与者与一个旨在支持ED康复的基于LLM的聊天机器人WellnessBot在10天内的互动。我们发现，参与者通过与聊天机器人讨论ED相关经历，在康复过程中获得了赋能感，这提供了一条私密而又具社交属性的途径。然而，我们也识别出聊天机器人产生的有害回应，尤其是在涉及ED个体时，部分由于参与者对聊天机器人可靠性的无条件信任，这些危害未被察觉。基于这些发现，我们为在ED管理中安全有效地实施基于LLM的干预措施提供了设计启示。