This paper presents a novel approach of leveraging Inter-Annotator Agreement (IAA), traditionally used for assessing labeling consistency, to optimize Data Management Operations (DMOps). We advocate for the use of IAA in predicting the labeling quality of individual annotators, leading to cost and time efficiency in data production. Additionally, our work highlights the potential of IAA in forecasting document difficulty, thereby boosting the data construction process's overall efficiency. This research underscores IAA's broader application potential in data-driven research optimization and holds significant implications for large-scale data projects prioritizing efficiency, cost reduction, and high-quality data.

翻译：本文提出一种创新方法，将传统用于评估标注一致性的标注者间一致性（IAA）用于优化数据管理操作（DMOps）。我们主张利用IAA预测个体标注者的标注质量，从而提升数据生产的成本与时间效率。此外，本研究揭示了IAA在预估文档难度方面的潜力，进而整体提升数据构建过程的效率。该研究强调了IAA在数据驱动研究优化中的更广泛应用前景，对注重效率、成本降低及高质量数据的大型数据项目具有重要启示。