Robotic Assisted Feeding (RAF) addresses the fundamental need for individuals with mobility impairments to regain autonomy in feeding themselves. The goal of RAF is to use a robot arm to acquire and transfer food to individuals from the table. Existing RAF methods primarily focus on solid foods, leaving a gap in manipulation strategies for semi-solid and deformable foods. This study introduces Long-horizon Visual Action (LAVA) based food acquisition of liquid, semisolid, and deformable foods. Long-horizon refers to the goal of "clearing the bowl" by sequentially acquiring the food from the bowl. LAVA employs a hierarchical policy for long-horizon food acquisition tasks. The framework uses high-level policy to determine primitives by leveraging ScoopNet. At the mid-level, LAVA finds parameters for primitives using vision. To carry out sequential plans in the real world, LAVA delegates action execution which is driven by Low-level policy that uses parameters received from mid-level policy and behavior cloning ensuring precise trajectory execution. We validate our approach on complex real-world acquisition trials involving granular, liquid, semisolid, and deformable food types along with fruit chunks and soup acquisition. Across 46 bowls, LAVA acquires much more efficiently than baselines with a success rate of 89 +/- 4% and generalizes across realistic plate variations such as different positions, varieties, and amount of food in the bowl. Code, datasets, videos, and supplementary materials can be found on our website.

翻译：机器人辅助喂食（RAF）旨在解决行动障碍患者恢复自主进食这一基本需求。其目标是通过机械臂从餐盘中获取食物并递送至患者口中。现有RAF方法主要针对固体食物，在半固态及可变形食物的操作策略方面存在研究空白。本研究提出基于长程视觉动作（LAVA）的液态、半固态及可变形食物获取方法。其中"长程"指通过顺序获取碗中食物实现"清空碗具"的目标。LAVA采用分层策略处理长程食物获取任务：高层策略通过ScoopNet决策基本操作原语；中层策略利用视觉信息确定原语参数；为在真实世界执行序列化规划，LAVA将动作执行委托给低层策略，该策略接收中层策略提供的参数并结合行为克隆技术确保精确轨迹执行。我们在包含颗粒状、液态、半固态、可变形食物以及水果块和汤类获取的复杂真实环境场景中验证了该方法。在46个碗具的测试中，LAVA获取效率显著优于基线方法，成功率达89±4%，并能泛化至不同碗具位置、食物种类及食物量等真实场景变化。相关代码、数据集、演示视频及补充材料均可在项目网站获取。