Supervised machine learning has become the cornerstone of today's data-driven society, increasing the need for labeled data. However, the process of acquiring labels is often expensive and tedious. One possible remedy is to use active learning (AL) -- a special family of machine learning algorithms designed to reduce labeling costs. Although AL has been successful in practice, a number of practical challenges hinder its effectiveness and are often overlooked in existing AL annotation tools. To address these challenges, we developed ALANNO, an open-source annotation system for NLP tasks equipped with features to make AL effective in real-world annotation projects. ALANNO facilitates annotation management in a multi-annotator setup and supports a variety of AL methods and underlying models, which are easily configurable and extensible.

翻译：监督式机器学习已成为当今数据驱动社会的基石，随之而来的是对标注数据需求的日益增长。然而，数据标注的过程往往既昂贵又繁琐。主动学习（AL）作为一种专门旨在降低标注成本的机器学习算法分支，为此提供了可行的解决方案。尽管主动学习在实践中取得了成功，但许多实际挑战仍阻碍其有效性，且现有AL标注工具往往忽略了这些问题。为应对这些挑战，我们开发了ALANNO——一个面向自然语言处理任务的开源标注系统，该系统配备了使AL在真实标注项目中发挥效用的多项特性。ALANNO支持多标注者协作场景下的标注管理，兼容多种主动学习方法及底层模型，且具有良好的可配置性与可扩展性。