The global mental health crisis is looming with a rapid increase in mental disorders, limited resources, and the social stigma of seeking treatment. As the field of artificial intelligence (AI) has witnessed significant advancements in recent years, large language models (LLMs) capable of understanding and generating human-like text may be used in supporting or providing psychological counseling. However, the application of LLMs in the mental health domain raises concerns regarding the accuracy, effectiveness, and reliability of the information provided. This paper investigates the major challenges associated with the development of LLMs for psychological counseling, including model hallucination, interpretability, bias, privacy, and clinical effectiveness. We explore potential solutions to these challenges that are practical and applicable to the current paradigm of AI. From our experience in developing and deploying LLMs for mental health, AI holds a great promise for improving mental health care, if we can carefully navigate and overcome pitfalls of LLMs.

翻译：全球心理健康危机正日益严峻，表现为心理障碍患者数量快速增加、治疗资源有限以及寻求治疗的社会污名化。近年来，随着人工智能（AI）领域取得显著进展，能够理解并生成类人文本的大型语言模型（LLMs）可用于支持或提供心理咨询。然而，LLMs在心理健康领域的应用引发了关于所提供信息准确度、有效性和可靠性的担忧。本文探讨了开发用于心理咨询的LLMs所面临的主要挑战，包括模型幻觉、可解释性、偏见、隐私和临床有效性。我们探索了针对这些挑战的实用且适用于当前AI范式的潜在解决方案。根据我们在开发并部署用于心理健康的LLMs方面的经验，若能谨慎引导并克服LLMs的缺陷，人工智能在改善心理健康护理方面具有巨大潜力。