This work explores the impact of moderation on users' enjoyment of conversational AI systems. While recent advancements in Large Language Models (LLMs) have led to highly capable conversational AIs that are increasingly deployed in real-world settings, there is a growing concern over AI safety and the need to moderate systems to encourage safe language and prevent harm. However, some users argue that current approaches to moderation limit the technology, compromise free expression, and limit the value delivered by the technology. This study takes an unbiased stance and shows that moderation does not necessarily detract from user enjoyment. Heavy handed moderation does seem to have a nefarious effect, but models that are moderated to be safer can lead to a better user experience. By deploying various conversational AIs in the Chai platform, the study finds that user retention can increase with a level of moderation and safe system design. These results demonstrate the importance of appropriately defining safety in models in a way that is both responsible and focused on serving users.

翻译：本研究探讨了内容审核对对话式AI系统用户体验的影响。尽管大型语言模型（LLM）的最新进展催生了高度智能的对话式AI，并越来越多地部署于实际场景，但AI安全问题日益引发关注——系统需要适度审核以鼓励安全用语、防范潜在危害。然而，部分用户认为当前审核方式限制了技术潜力，损害了表达自由，降低了技术价值。本研究以中立立场表明：内容审核未必削弱用户愉悦感。过度强硬的审核措施虽会带来负面效应，但经过安全优化的AI模型反而能提升用户体验。通过在Chai平台部署多种对话式AI，研究发现适度的内容审核与安全系统设计能提升用户留存率。这些结果凸显了以负责任且服务用户为核心的方式，合理定义模型安全标准的重要性。