Trajectory planning in robotics aims to generate collision-free pose sequences that can be reliably executed. Recently, vision-to-planning systems have garnered increasing attention for their efficiency and ability to interpret and adapt to surrounding environments. However, traditional modular systems suffer from increased latency and error propagation, while purely data-driven approaches often overlook the robot's kinematic constraints. This oversight leads to discrepancies between planned trajectories and those that are executable. To address these challenges, we propose iKap, a novel vision-to-planning system that integrates the robot's kinematic model directly into the learning pipeline. iKap employs a self-supervised learning approach and incorporates the state transition model within a differentiable bi-level optimization framework. This integration ensures the network learns collision-free waypoints while satisfying kinematic constraints, enabling gradient back-propagation for end-to-end training. Our experimental results demonstrate that iKap achieves higher success rates and reduced latency compared to the state-of-the-art methods. Besides the complete system, iKap offers a visual-to-planning network that seamlessly integrates kinematics into various controllers, providing a robust solution for robots navigating complex and dynamic environments.

翻译：机器人学中的轨迹规划旨在生成可可靠执行的无碰撞位姿序列。近年来，视觉到规划系统因其高效性以及解释和适应周围环境的能力而受到越来越多的关注。然而，传统的模块化系统存在延迟增加和误差传播的问题，而纯数据驱动的方法往往忽视机器人的运动学约束。这种疏忽导致规划轨迹与可执行轨迹之间存在差异。为应对这些挑战，我们提出iKap——一种新颖的视觉到规划系统，它将机器人的运动学模型直接集成到学习流程中。iKap采用自监督学习方法，并在可微分的双层优化框架内整合状态转移模型。这种集成确保网络在学习无碰撞路径点的同时满足运动学约束，从而实现端到端训练的梯度反向传播。我们的实验结果表明，与最先进的方法相比，iKap实现了更高的成功率和更低的延迟。除了完整的系统外，iKap还提供了一个视觉到规划网络，能够将运动学无缝集成到各种控制器中，为机器人在复杂动态环境中的导航提供了鲁棒的解决方案。