Machine learning approaches often require training and evaluation datasets with a clear separation between positive and negative examples. This risks simplifying and even obscuring the inherent subjectivity present in many tasks. Preserving such variance in content and diversity in datasets is often expensive and laborious. This is especially troubling when building safety datasets for conversational AI systems, as safety is both socially and culturally situated. To demonstrate this crucial aspect of conversational AI safety, and to facilitate in-depth model performance analyses, we introduce the DICES (Diversity In Conversational AI Evaluation for Safety) dataset that contains fine-grained demographic information about raters, high replication of ratings per item to ensure statistical power for analyses, and encodes rater votes as distributions across different demographics to allow for in-depth explorations of different aggregation strategies. In short, the DICES dataset enables the observation and measurement of variance, ambiguity, and diversity in the context of conversational AI safety. We also illustrate how the dataset offers a basis for establishing metrics to show how raters' ratings can intersects with demographic categories such as racial/ethnic groups, age groups, and genders. The goal of DICES is to be used as a shared resource and benchmark that respects diverse perspectives during safety evaluation of conversational AI systems.

翻译：机器学习方法通常需要训练和评估数据集在正例和负例之间具有明确区分。这可能会简化和甚至掩盖许多任务中固有的主观性。在数据集中保留这种内容差异和多样性往往代价高昂且费时费力。对于构建对话式AI系统的安全数据集而言，这一点尤其令人担忧，因为安全本身具有社会和文化情境依属性。为了证明对话式AI安全这一关键方面，并促进深入的模型性能分析，我们引入了DICES（对话式AI安全评估中的多样性）数据集。该数据集包含评估者的细粒度人口统计信息、每个项目的高重复评级以确保统计分析的统计功效，并将评估者的投票编码为不同人口统计群体的分布，从而支持对不同聚合策略的深入探索。简而言之，DICES数据集能够观察和测量对话式AI安全背景下的方差、模糊性和多样性。我们还说明该数据集如何为建立指标提供基础，以展示评估者的评级如何与种族/民族群体、年龄组和性别等人口统计类别产生交叉。DICES的目标是作为共享资源和基准，在对话式AI系统的安全评估中尊重多元视角。