AI alignment relies on annotator judgments, yet annotation pipelines often treat annotators as interchangeable, obscuring how their social position shapes annotation. We introduce reflexive annotating as a probe that invites crowd workers to reflect on how their positionality informs subjective annotation judgments in a language model alignment context. Through a qualitative study with crowd workers (N=30) and follow-up interviews (N=5), we examine how our probe shapes annotators' behaviour, experience, and the situated metadata it elicits. We find that reflexive annotating captures epistemic metadata beyond static demographics by eliciting intersectional reasoning, surfacing positional humility, and nudging viewpoint change. Crucially, we also denote tensions between reflexive engagement and affective demands such as emotional exposure. We discuss the implications of our work for richer value elicitation and alignment practices that treat annotator judgments as situated and selectively integrate positional metadata.

翻译：人工智能对齐依赖于标注者的判断，然而标注流程往往将标注者视为可互换的，从而掩盖了其社会位置如何塑造标注结果。我们提出反思性标注作为一种探测方法，邀请众包工作者反思其位置性如何在语言模型对齐语境中影响主观标注判断。通过对众包工作者开展的定性研究（N=30）及后续访谈（N=5），我们考察了该探测方法如何影响标注者的行为、体验及其所引发的情境化元数据。我们发现，反思性标注能够捕获超越静态人口统计特征的认知元数据，具体表现为：引发交叉性推理、凸显位置性谦逊、并促使观点转变。关键的是，我们还注意到反思性参与与情感暴露等情感需求之间存在张力。我们讨论了这项工作对更丰富的价值观引出及对齐实践的意义——这些实践将标注者判断视为情境化的，并选择性整合位置性元数据。