The performance of current supervised AI systems is tightly connected to the availability of annotated datasets. Annotations are usually collected through annotation tools, which are often designed for specific tasks and are difficult to customize. Moreover, existing annotation tools with an active learning mechanism often only support limited use cases. To address these limitations, we present EASE, an Easily-Customized Annotation System Powered by Efficiency Enhancement Mechanisms. \sysname provides modular annotation units for building customized annotation interfaces and also provides multiple back-end options that suggest annotations using (1) multi-task active learning; (2) demographic feature based active learning; (3) a prompt system that can query the API of large language models. We conduct multiple experiments and user studies to evaluate our system's flexibility and effectiveness. Our results show that our system can meet the diverse needs of NLP researchers and significantly accelerate the annotation process.

翻译：摘要：当前监督式AI系统的性能与标注数据集的可用性密不可分。标注通常通过标注工具收集，这些工具往往针对特定任务设计，难以定制。此外，现有集成主动学习机制的标注工具通常仅支持有限的使用场景。为解决这些局限，我们提出EASE——一种基于效率增强机制的易定制标注系统。该系统提供模块化标注单元以构建定制化标注界面，并支持多种后端选项，通过以下方式生成标注建议：（1）多任务主动学习；（2）基于人口统计学特征的主动学习；（3）可查询大语言模型API的提示系统。我们通过多项实验和用户研究评估系统的灵活性与有效性。结果表明，本系统能够满足NLP研究者的多样化需求，并显著加速标注过程。