Growing evidence shows that proactive content moderation supported by AI can help improve online discourse. However, we know little about designing these systems, how design impacts efficacy and user experience, and how people perceive proactive moderation across public and private platforms. We developed a mobile keyboard with built-in proactive content moderation which we tested (N=575) within a semi-functional simulation of a public and private communication platform. Where toxic content was detected, we used different interventions that embedded three design factors: timing, friction, and the presentation of the AI model output. We found moderation to be effective, regardless of the design. However, friction was a source of annoyance while prompts with no friction that occurred during typing were more effective. Follow-up interviews highlight the differences in how these systems are perceived across public and private platforms, and how they can offer more than moderation by acting as educational and communication support tools.

翻译：越来越多的证据表明，由人工智能支持的主动内容审核有助于改善在线讨论。然而，我们对如何设计这些系统、设计如何影响有效性和用户体验，以及人们对公共和私人平台上主动审核的看法知之甚少。我们开发了一款内置主动内容审核功能的移动键盘，并在一个半功能化的公共与私人通信平台模拟环境中进行了测试（N=575）。当检测到有害内容时，我们采用了嵌入三种设计因素的不同干预措施：时机、干扰程度以及人工智能模型输出的呈现方式。我们发现，无论采用何种设计，审核都是有效的。然而，干扰因素会引起用户反感，而在打字过程中出现的无干扰提示效果更佳。后续访谈凸显了这些系统在公共和私人平台上的认知差异，以及它们如何不仅提供审核功能，还能作为教育和沟通支持工具发挥作用。