Inspired by the unique neurophysiology of the octopus, we propose a hierarchical framework that simplifies the coordination of multiple soft arms by decomposing control into high-level decision making, low-level motor activation, and local reflexive behaviors via sensory feedback. When evaluated in the illustrative problem of a model octopus foraging for food, this hierarchical decomposition results in significant improvements relative to end-to-end methods. Performance is achieved through a mixed-modes approach, whereby qualitatively different tasks are addressed via complementary control schemes. Here, model-free reinforcement learning is employed for high-level decision-making, while model-based energy shaping takes care of arm-level motor execution. To render the pairing computationally tenable, a novel neural-network energy shaping (NN-ES) controller is developed, achieving accurate motions with time-to-solutions 200 times faster than previous attempts. Our hierarchical framework is then successfully deployed in increasingly challenging foraging scenarios, including an arena littered with obstacles in 3D space, demonstrating the viability of our approach.

翻译：受章鱼独特神经生理学的启发，本文提出了一种分层框架，通过将控制分解为高层决策、低层运动激活以及基于感觉反馈的局部反射行为，简化了多个软体臂的协调问题。在以模型章鱼觅食为示例的评估中，这种分层分解相较于端到端方法带来了显著提升。性能的达成依赖于混合模态方法，即通过互补控制方案处理不同性质的任务：采用无模型强化学习进行高层决策，而基于模型的能量整形则负责臂级运动执行。为使该组合在计算上可行，本文开发了一种新型神经网络能量整形控制器，其运动精度达到要求的同时，求解时间比以往方法快200倍。该分层框架随后被成功应用于渐趋复杂的觅食场景，包括布满三维空间障碍物的竞技场，充分验证了本方法的可行性。