The Digital Services Act, recently adopted by the EU, requires social media platforms to report the "accuracy" of their automated content moderation systems. The colloquial term is vague, or open-textured -- the literal accuracy (number of correct predictions divided by the total) is not suitable for problems with large class imbalance, and the ground truth and dataset to measure accuracy against is unspecified. Without further specification, the regulatory requirement allows for deficient reporting. In this interdisciplinary work, we operationalize "accuracy" reporting by refining legal concepts and relating them to technical implementation. We start by elucidating the legislative purpose of the Act to legally justify an interpretation of "accuracy" as precision and recall. These metrics remain informative in class imbalanced settings, and reflect the proportional balancing of Fundamental Rights of the EU Charter. We then focus on the estimation of recall, as its naive estimation can incur extremely high annotation costs and disproportionately interfere with the platform's right to conduct business. Through a simulation study, we show that recall can be efficiently estimated using stratified sampling with trained classifiers, and provide concrete recommendations for its application. Finally, we present a case study of recall reporting for a subset of Reddit under the Act. Based on the language in the Act, we identify a number of ways recall could be reported due to underspecification. We report on one possibility using our improved estimator, and discuss the implications and need for legal clarification.

翻译：欧盟近期通过的《数字服务法》要求社交媒体平台报告其自动化内容审核系统的"准确性"。这一日常用语存在模糊性或开放性——字面意义上的准确性（正确预测数除以总数）并不适用于类别严重不平衡的问题，且衡量准确性所需的地面实况与数据集尚未明确。若无进一步明确规范，该监管要求将导致报告存在缺陷。在这项跨学科研究中，我们通过细化法律概念并将其与技术实施相关联，对"准确性"报告进行操作化定义。首先阐明该法案的立法目的，从法理上论证将"准确性"解释为精确率与召回率的合理性。这两项指标在类别不平衡场景下仍具信息量，并体现了欧盟基本权利宪章的比例平衡原则。随后聚焦召回率估算问题，因其朴素估算可能产生极高的标注成本，并对平台经营自主权造成不当干预。通过模拟研究，我们证明可利用训练分类器的分层抽样高效估算召回率，并提出具体应用建议。最终以Reddit平台子集为例，呈现该法案框架下召回率报告的案例研究。基于法案文本表述，我们识别出因规定不明确可能导致的多种召回率报告方式，采用改进后的估算器报告其中一种可能性，并探讨需法律澄清的要点及其影响。