Autonomous robots can benefit greatly from human-provided semantic characterizations of uncertain task environments and states. However, the development of integrated strategies which let robots model, communicate, and act on such 'soft data' remains challenging. Here, the Human Assisted Robotic Planning and Sensing (HARPS) framework is presented for active semantic sensing and planning in human-robot teams to address these gaps by formally combining the benefits of online sampling-based POMDP policies, multimodal semantic interaction, and Bayesian data fusion. This approach lets humans opportunistically impose model structure and extend the range of semantic soft data in uncertain environments by sketching and labeling arbitrary landmarks across the environment. Dynamic updating of the environment model while during search allows robotic agents to actively query humans for novel and relevant semantic data, thereby improving beliefs of unknown environments and states for improved online planning. Simulations of a UAV-enabled target search application in a large-scale partially structured environment show significant improvements in time and belief state estimates required for interception versus conventional planning based solely on robotic sensing. Human subject studies in the same environment (n = 36) demonstrate an average doubling in dynamic target capture rate compared to the lone robot case, and highlight the robustness of active probabilistic reasoning and semantic sensing over a range of user characteristics and interaction modalities.

翻译：自主机器人可从人类提供的关于不确定任务环境与状态的语义描述中获益良多。然而，开发能够让机器人对这种"软数据"进行建模、通信并采取行动的集成策略仍极具挑战性。本文提出的人助机器人规划与感知（HARPS）框架，通过正式结合基于在线采样POMDP策略、多模态语义交互与贝叶斯数据融合的优势，旨在弥合人机团队中主动语义感知与规划方面的上述差距。该方法允许人类在不确定环境中通过手绘并标注任意地标来灵活施加模型结构、扩展语义软数据的覆盖范围。搜索过程中对环境模型的动态更新使机器人智能体能够主动向人类查询新颖且相关的语义数据，从而改善对未知环境与状态的信念估计，以实现更优的在线规划。在大型部分结构化环境中开展的无人机目标搜索应用仿真结果表明，与仅依赖机器人感知的传统规划相比，该方法在拦截所需的时间与信念状态估计方面均有显著提升。同一环境的人类受试者研究（n=36）显示，动态目标捕获率较单机器人工况平均提升一倍，并凸显了主动概率推理与语义感知在不同用户特征及交互模式下的鲁棒性。