Labels are widely used in augmented reality (AR) to display digital information. Ensuring the readability of AR labels requires placing them occlusion-free while keeping visual linkings legible, especially when multiple labels exist in the scene. Although existing optimization-based methods, such as force-based methods, are effective in managing AR labels in static scenarios, they often struggle in dynamic scenarios with constantly moving objects. This is due to their focus on generating layouts optimal for the current moment, neglecting future moments and leading to sub-optimal or unstable layouts over time. In this work, we present RL-LABEL, a deep reinforcement learning-based method for managing the placement of AR labels in scenarios involving moving objects. RL-LABEL considers the current and predicted future states of objects and labels, such as positions and velocities, as well as the user's viewpoint, to make informed decisions about label placement. It balances the trade-offs between immediate and long-term objectives. Our experiments on two real-world datasets show that RL-LABEL effectively learns the decision-making process for long-term optimization, outperforming two baselines (i.e., no view management and a force-based method) by minimizing label occlusions, line intersections, and label movement distance. Additionally, a user study involving 18 participants indicates that RL-LABEL excels over the baselines in aiding users to identify, compare, and summarize data on AR labels within dynamic scenes.
翻译:摘要:标签在增强现实(AR)中被广泛用于显示数字信息。确保AR标签的可读性需要在保持视觉连线清晰的同时避免遮挡,尤其当场景中存在多个标签时。尽管现有的基于优化的方法(如基于力的方法)在静态场景中能有效管理AR标签,但面对物体持续运动的动态场景时往往表现不佳。这是因为这些方法仅聚焦于生成当前时刻的最优布局,忽略了未来时刻,导致布局随时间推移趋于次优或不稳定。本文提出RL-LABEL——一种基于深度强化学习的方法,用于管理包含运动物体的AR标签放置问题。RL-LABEL综合考虑物体和标签的当前与预测未来状态(如位置、速度)以及用户视角,做出明智的标签放置决策,平衡即时收益与长期目标之间的权衡。我们在两个真实数据集上的实验表明,RL-LABEL能有效学习面向长期优化的决策过程,通过最小化标签遮挡、连线交叉和标签移动距离,优于两种基线方法(即无视图管理和基于力的方法)。此外,一项包含18名参与者的用户研究表明,RL-LABEL在帮助用户识别、比较和汇总动态场景中AR标签上的数据方面显著优于基线方法。