In an efficient and flexible human-robot collaborative work environment, a robot team member must be able to recognize both explicit requests and implied actions from human users. Identifying "what to do" in such cases requires an agent to have the ability to construct associations between objects, their actions, and the effect of actions on the environment. In this regard, semantic memory is being introduced to understand the explicit cues and their relationships with available objects and required skills to make "tea" and "sandwich". We have extended our previous hierarchical robot control architecture to add the capability to execute the most appropriate task based on both feedback from the user and the environmental context. To validate this system, two types of skills were implemented in the hierarchical task tree: 1) Tea making skills and 2) Sandwich making skills. During the conversation between the robot and the human, the robot was able to determine the hidden context using ontology and began to act accordingly. For instance, if the person says "I am thirsty" or "It is cold outside" the robot will start to perform the tea-making skill. In contrast, if the person says, "I am hungry" or "I need something to eat", the robot will make the sandwich. A humanoid robot Baxter was used for this experiment. We tested three scenarios with objects at different positions on the table for each skill. We observed that in all cases, the robot used only objects that were relevant to the skill.

翻译：在高效灵活的人机协作工作环境中，机器人团队成员必须能够识别来自人类用户的显式请求和隐含动作。在此类情况下确定"该做什么"需要智能体具备构建对象、对象动作及动作对环境效果之间关联的能力。为此，我们引入语义记忆来理解显式线索及其与可用对象和所需技能之间的关系，以完成"泡茶"和"制作三明治"等任务。我们扩展了先前的分层机器人控制架构，新增了基于用户反馈和环境上下文执行最合适任务的能力。为验证该系统，我们在层次化任务树中实现了两类技能：1) 泡茶技能和2) 制作三明治技能。在机器人与人类的对话过程中，机器人能够利用本体论确定隐含上下文并据此采取行动。例如，若用户说"我渴了"或"外面很冷"，机器人将开始执行泡茶技能；相反，若用户说"我饿了"或"我需要吃点东西"，机器人则会制作三明治。本实验使用人形机器人Baxter进行，我们针对每项技能测试了三种场景（桌面物体位于不同位置）。观察发现，在所有情况下机器人仅使用与技能相关的物体。