Content moderation typically combines the efforts of human moderators and machine learning models. However, these systems often rely on data where significant disagreement occurs during moderation, reflecting the subjective nature of toxicity perception. Rather than dismissing this disagreement as noise, we interpret it as a valuable signal that highlights the inherent ambiguity of the content,an insight missed when only the majority label is considered. In this work, we introduce a novel content moderation framework that emphasizes the importance of capturing annotation disagreement. Our approach uses multitask learning, where toxicity classification serves as the primary task and annotation disagreement is addressed as an auxiliary task. Additionally, we leverage uncertainty estimation techniques, specifically Conformal Prediction, to account for both the ambiguity in comment annotations and the model's inherent uncertainty in predicting toxicity and disagreement.The framework also allows moderators to adjust thresholds for annotation disagreement, offering flexibility in determining when ambiguity should trigger a review. We demonstrate that our joint approach enhances model performance, calibration, and uncertainty estimation, while offering greater parameter efficiency and improving the review process in comparison to single-task methods.

翻译：内容审核通常结合人工审核员与机器学习模型的协同工作。然而，这些系统往往依赖于审核过程中存在显著分歧的数据，这反映了毒性感知的主观性。我们并未将这些分歧视为噪声而予以忽略，而是将其解读为凸显内容内在模糊性的宝贵信号——这一洞见在仅考虑多数标签时会被忽略。本文提出了一种新颖的内容审核框架，强调捕捉标注分歧的重要性。我们的方法采用多任务学习架构，其中毒性分类作为主任务，标注分歧则作为辅助任务进行处理。此外，我们利用不确定性估计技术（特别是合规预测）来同时处理评论标注的模糊性以及模型在预测毒性与分歧时的固有不确定性。该框架还允许审核员调整标注分歧的阈值，从而灵活决定何时应触发模糊性内容的复审流程。实验表明，与单任务方法相比，我们的联合学习方法在提升模型性能、校准度和不确定性估计能力的同时，具有更高的参数效率，并能优化审核流程。