Volunteer moderators play a crucial role in sustaining online dialogue, but they often disagree about what should or should not be allowed. In this paper, we study the complexity of content moderation with a focus on disagreements between moderators, which we term the ``gray area'' of moderation. Leveraging 5 years and 4.3 million moderation log entries from 24 subreddits of different topics and sizes, we characterize how gray area, or disputed cases, differ from undisputed cases. We show that one-in-seven moderation cases are disputed among moderators, often addressing transgressions where users' intent is not directly legible, such as in trolling and brigading, as well as tensions around community governance. This is concerning, as almost half of all gray area cases involved automated moderation decisions. Through information-theoretic evaluations, we demonstrate that gray area cases are inherently harder to adjudicate than undisputed cases and show that state-of-the-art language models struggle to adjudicate them. We highlight the key role of expert human moderators in overseeing the moderation process and provide insights about the challenges of current moderation processes and tools.

翻译：志愿版主在维持在线对话中发挥着关键作用，但他们对于应允许或禁止哪些内容常存在分歧。本文以版主间的分歧（我们称之为内容审核的“灰色地带”）为重点，研究内容审核的复杂性。通过分析来自24个不同主题和规模的subreddit、历时5年共计430万条审核日志记录，我们刻画了灰色地带（即争议案例）与非争议案例的差异。研究表明，每七个审核案例中就有一个存在版主争议，这些争议通常涉及用户意图难以直接辨识的违规行为（如恶意挑衅和跨社区围攻），以及围绕社区治理的紧张关系。值得注意的是，近半数的灰色地带案例涉及自动化审核决策，这尤其令人担忧。通过信息论评估，我们证明灰色地带案例本质上比非争议案例更难裁决，且当前最先进的语言模型也难以处理此类案例。我们强调了专业人工版主在监督审核流程中的核心作用，并就当前审核流程与工具面临的挑战提出了深刻见解。