Content moderation typically combines the efforts of human moderators and machine learning models.However, these systems often rely on data where significant disagreement occurs during moderation, reflecting the subjective nature of toxicity perception.Rather than dismissing this disagreement as noise, we interpret it as a valuable signal that highlights the inherent ambiguity of the content,an insight missed when only the majority label is considered.In this work, we introduce a novel content moderation framework that emphasizes the importance of capturing annotation disagreement. Our approach uses multitask learning, where toxicity classification serves as the primary task and annotation disagreement is addressed as an auxiliary task.Additionally, we leverage uncertainty estimation techniques, specifically Conformal Prediction, to account for both the ambiguity in comment annotations and the model's inherent uncertainty in predicting toxicity and disagreement.The framework also allows moderators to adjust thresholds for annotation disagreement, offering flexibility in determining when ambiguity should trigger a review.We demonstrate that our joint approach enhances model performance, calibration, and uncertainty estimation, while offering greater parameter efficiency and improving the review process in comparison to single-task methods.

翻译：内容审核通常结合人工审核员与机器学习模型的协作。然而，这些系统往往依赖于审核过程中存在显著分歧的数据，这反映了毒性感知的主观性。我们并非将这种分歧视为噪声而忽略，而是将其解读为一种有价值的信号，用以突显内容固有的模糊性——这一洞见在仅考虑多数标签时会被忽略。本文提出了一种新颖的内容审核框架，强调捕捉标注分歧的重要性。我们的方法采用多任务学习，其中毒性分类作为主任务，而标注分歧的处理作为辅助任务。此外，我们利用不确定性估计技术，特别是合规预测，以同时考虑评论标注的模糊性以及模型在预测毒性和分歧时的内在不确定性。该框架还允许审核员调整标注分歧的阈值，从而在决定何时应触发模糊性审查时提供灵活性。我们证明，与单任务方法相比，我们的联合方法提升了模型性能、校准度和不确定性估计，同时实现了更高的参数效率并优化了审核流程。