Research into the detection of human activities from wearable sensors is a highly active field, benefiting numerous applications, from ambulatory monitoring of healthcare patients via fitness coaching to streamlining manual work processes. We present an empirical study that evaluates and contrasts four commonly employed annotation methods in user studies focused on in-the-wild data collection. For both the user-driven, in situ annotations, where participants annotate their activities during the actual recording process, and the recall methods, where participants retrospectively annotate their data at the end of each day, the participants had the flexibility to select their own set of activity classes and corresponding labels. Our study illustrates that different labeling methodologies directly impact the annotations' quality, as well as the capabilities of a deep learning classifier trained with the data. We noticed that in situ methods produce less but more precise labels than recall methods. Furthermore, we combined an activity diary with a visualization tool that enables the participant to inspect and label their activity data. Due to the introduction of such a tool were able to decrease missing annotations and increase the annotation consistency, and therefore the F1-Score of the deep learning model by up to 8% (ranging between 82.1 and 90.4% F1-Score). Furthermore, we discuss the advantages and disadvantages of the methods compared in our study, the biases they could introduce, and the consequences of their usage on human activity recognition studies as well as possible solutions.

翻译：基于可穿戴传感器的人类活动检测研究是一个高度活跃的领域，其应用广泛涵盖从医疗患者的动态监测、健身指导到优化人工工作流程等多个方面。本文通过实证研究，评估并对比了在野外数据收集中常用的四种标注方法。无论是用户驱动的现场标注（参与者在实际记录过程中实时标注自身活动），还是回忆式标注（参与者在每日结束时回溯标注数据），参与者均可自主选择活动类别及相应标签。研究表明，不同的标注方法会直接影响标注质量以及基于该数据训练的深度学习分类器的性能。我们发现，与回忆式方法相比，现场标注方法产生的标签数量较少但精度更高。此外，我们结合活动日记与可视化工具开发了一套系统，使参与者能够查看并标注其活动数据。该工具的应用有效减少了标注缺失，提升了标注一致性，从而使深度学习模型的F1分数最高提升8%（F1分数范围介于82.1%至90.4%之间）。最后，我们深入探讨了本研究所比较方法的优缺点、可能引入的偏差、其对人类活动识别研究的影响以及潜在的解决方案。