To support humans in their daily lives, robots are required to autonomously learn, adapt to objects and environments, and perform the appropriate actions. We tackled on the task of cooking scrambled eggs using real ingredients, in which the robot needs to perceive the states of the egg and adjust stirring movement in real time, while the egg is heated and the state changes continuously. In previous works, handling changing objects was found to be challenging because sensory information includes dynamical, both important or noisy information, and the modality which should be focused on changes every time, making it difficult to realize both perception and motion generation in real time. We propose a predictive recurrent neural network with an attention mechanism that can weigh the sensor input, distinguishing how important and reliable each modality is, that realize quick and efficient perception and motion generation. The model is trained with learning from the demonstration, and allows the robot to acquire human-like skills. We validated the proposed technique using the robot, Dry-AIREC, and with our learning model, it could perform cooking eggs with unknown ingredients. The robot could change the method of stirring and direction depending on the status of the egg, as in the beginning it stirs in the whole pot, then subsequently, after the egg started being heated, it starts flipping and splitting motion targeting specific areas, although we did not explicitly indicate them.

翻译：为辅助人类日常生活，机器人需自主学习和适应物体与环境，并执行恰当动作。我们以真实食材制作炒蛋为任务，机器人需在鸡蛋持续受热、状态不断变化时，实时感知蛋液状态并调整搅拌动作。此前研究显示，处理动态变化物体颇具挑战，因为传感器信息包含动态、重要与噪声信息并存的多模态数据，且需重点关注的信息模态随时间变化，难以同时实现实时感知与运动生成。我们提出一种带注意力机制的预测递归神经网络，可对传感器输入进行加权，区分各模态的重要性与可靠性，从而实现快速高效的感知与运动生成。该模型通过示教学习训练，使机器人获得类人技能。我们利用Dry-AIREC机器人验证所提技术，采用本学习模型的机器人能对未知食材完成烹饪操作。机器人可根据鸡蛋状态改变搅拌方法与方向：初始阶段在全锅搅拌，待鸡蛋开始受热后，转而针对特定区域执行翻转与分割动作——尽管这些动作并未被明确编码。