User reporting is an essential component of content moderation on many online platforms -- in particular, on end-to-end encrypted (E2EE) messaging platforms where platform operators cannot proactively inspect message contents. However, users' privacy concerns when considering reporting may impede the effectiveness of this strategy in regulating online harassment. In this paper, we conduct interviews with 16 users of E2EE platforms to understand users' mental models of how reporting works and their resultant privacy concerns and considerations surrounding reporting. We find that users expect platforms to store rich longitudinal reporting datasets, recognizing both their promise for better abuse mitigation and the privacy risk that platforms may exploit or fail to protect them. We also find that users have preconceptions about the respective capabilities and risks of moderators at the platform versus community level -- for instance, users trust platform moderators more to not abuse their power but think community moderators have more time to attend to reports. These considerations, along with perceived effectiveness of reporting and how to provide sufficient evidence while maintaining privacy, shape how users decide whether, to whom, and how much to report. We conclude with design implications for a more privacy-preserving reporting system on E2EE messaging platforms.
翻译:用户举报是许多在线平台内容审核的重要组成部分——尤其是在端到端加密(E2EE)消息平台中,平台运营商无法主动审查消息内容。然而,用户在考虑举报时的隐私顾虑可能削弱该策略在治理网络骚扰方面的有效性。本文通过对16名E2EE平台用户进行访谈,了解用户对举报机制的心理模型及其由此产生的隐私关切与考量。我们发现在举报过程中,用户预期平台会存储丰富的纵向举报数据集,既认识到这些数据在改善滥用行为治理方面的潜力,也意识到平台可能滥用或未能保护这些数据的隐私风险。同时,用户对平台管理员与社区管理员各自的能力和风险存在先入为主的看法——例如,用户更信任平台管理员不会滥用职权,但认为社区管理员有更多时间处理举报。这些考量,连同用户对举报有效性的感知、如何在维护隐私的同时提供充分证据等因素,共同决定了用户是否举报、向谁举报以及举报多少内容。最后,我们为在E2EE消息平台上构建更具隐私保护性的举报系统提出设计启示。