Cooking is a central activity of daily living, supporting independence as well as mental and physical health. However, prior work has highlighted key barriers for people with low vision (LV) to cook, particularly around safely interacting with tools, such as sharp knives or hot pans. Drawing on recent advancements in computer vision (CV), we present CookAR, a head-mounted AR system with real-time object affordance augmentations to support safe and efficient interactions with kitchen tools. To design and implement CookAR, we collected and annotated the first egocentric dataset of kitchen tool affordances, fine-tuned an affordance segmentation model, and developed an AR system with a stereo camera to generate visual augmentations. To validate CookAR, we conducted a technical evaluation of our fine-tuned model as well as a qualitative lab study with 10 LV participants for suitable augmentation design. Our technical evaluation demonstrates that our model outperforms the baseline on our tool affordance dataset, while our user study indicates a preference for affordance augmentations over the traditional whole object augmentations.

翻译：烹饪是日常生活的核心活动，有助于维持独立性以及身心健康。然而，先前的研究强调了低视力人群在烹饪时面临的主要障碍，尤其是在安全使用工具（如锋利的刀具或高温锅具）方面。借鉴计算机视觉领域的最新进展，我们提出了CookAR，这是一个头戴式增强现实系统，通过实时物体可供性增强来支持与厨房工具的安全高效交互。为了设计和实现CookAR，我们收集并标注了首个厨房工具可供性的第一人称视角数据集，微调了一个可供性分割模型，并开发了一个配备立体摄像头的增强现实系统以生成视觉增强效果。为了验证CookAR，我们对我们微调后的模型进行了技术评估，并与10名低视力参与者进行了定性实验室研究以确定合适的增强设计。我们的技术评估表明，在我们的工具可供性数据集上，我们的模型性能优于基线模型；而我们的用户研究表明，相较于传统的整体物体增强，参与者更倾向于可供性增强。