Online news outlets are grappling with the moderation of user-generated content within their comment section. We present a recommender system based on ranking class probabilities to support and empower the moderator in choosing featured posts, a time-consuming task. By combining user and textual content features we obtain an optimal classification F1-score of 0.44 on the test set. Furthermore, we observe an optimum mean NDCG@5 of 0.87 on a large set of validation articles. As an expert evaluation, content moderators assessed the output of a random selection of articles by choosing comments to feature based on the recommendations, which resulted in a NDCG score of 0.83. We conclude that first, adding text features yields the best score and second, while choosing featured content remains somewhat subjective, content moderators found suitable comments in all but one evaluated recommendations. We end the paper by analyzing our best-performing model, a step towards transparency and explainability in hybrid content moderation.

翻译：在线新闻媒体正努力应对其评论区中用户生成内容的审核问题。我们提出了一种基于排序类别概率的推荐系统，以支持并赋能审核员完成选择精选帖子这一耗时任务。通过结合用户特征与文本内容特征，我们在测试集上获得了0.44的最优分类F1分数。此外，我们在大量验证文章上观察到了0.87的平均NDCG@5最优值。作为专家评估，内容审核员根据推荐结果评估了随机选择文章的帖子，以选取建议置顶的评论，最终获得了0.83的NDCG分数。我们得出结论：首先，添加文本特征可获得最佳分数；其次，尽管选择置顶内容仍具有一定主观性，但在所有评估的推荐中，除一例之外，内容审核员均找到了合适的评论。文章结尾，我们分析了表现最优的模型，这标志着在迈向混合内容审核透明性与可解释性的道路上迈出了一步。